Commit Graph

586 Commits

Author SHA1 Message Date
Ruben Fiszel
22aa3d8403 rhel fixes 2026-02-19 00:57:44 +00:00
Ruben Fiszel
bdcbb1ced5 nit: remove temporary release-as override
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 17:27:40 +00:00
Ruben Fiszel
c07efdd980 fix: override release-please to 1.635.0 to prevent unintended 2.0 bump
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 17:12:52 +00:00
centdix
a2b1cafc1b internal: git action cleaning (#7920)
* cleaning

* cleaning
2026-02-11 22:15:19 +00:00
centdix
0df5b1ffc6 docs command (#7912) 2026-02-11 16:30:07 +00:00
Ruben Fiszel
fc8a546ff2 fix: bump rust version from 1.90.0 to 1.93.0 2026-02-10 23:55:36 +00:00
centdix
def319db91 internal: fix git actions (#7892)
* fix git actions

* fix

* rm
2026-02-10 23:11:03 +00:00
Ruben Fiszel
682264cd8e resolve Windows build warnings treated as errors (#7870)
* fix: resolve Windows build warnings treated as errors

- Gate UV_PATH import behind #[cfg(unix)] in python_versions.rs
- Remove unused tokio::time::sleep import in worker.rs (use fully qualified path)
- Fix unused `file` variable warnings in ansible_executor.rs on Windows

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* ci: add Windows cargo check workflow

Runs cargo check with ee_windows features on push to backend/**
using the blacksmith-16vcpu-windows-2025 runner.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* ci: add cargo check step to Windows build, remove separate check workflow

Add a cargo check step with -D warnings before the full build to fail
fast on any warnings. Remove the separate windows-check.yml workflow.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 09:58:56 +00:00
Ruben Fiszel
be7b1f6396 ci: add libcurl4-openssl-dev to backend-test workflow
Required by rdkafka-sys 4.10.0 which unconditionally includes curl/curl.h.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 08:39:45 +00:00
Ruben Fiszel
8fe519fde1 ci: improve Windows builds with faster runner, caching, and 3h timeout
- Switch to blacksmith-16vcpu-windows-2025 runner
- Replace deprecated actions-rs/toolchain with actions-rust-lang/setup-rust-toolchain with cargo caching
- Increase build timeout from 90min to 180min

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 08:31:27 +00:00
Ruben Fiszel
ddade4e59c adapt to libkafka-changes 2026-02-10 08:06:06 +00:00
Ruben Fiszel
8e7fd16925 test-thread 10 2026-02-09 20:50:23 +00:00
Ruben Fiszel
5b791390bc ci cap 2026-02-09 00:03:21 +00:00
Ruben Fiszel
7a338427be ci: add mcp feature to backend test workflow
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 21:07:39 +00:00
Ruben Fiszel
14cd7fc7d3 test: set RUST_LOG=off in CI and gate mcp_tools test behind mcp feature
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 21:06:51 +00:00
Ruben Fiszel
ac8cd098eb ci: limit test threads to prevent pool exhaustion (#7852)
* ci: limit test threads to 16 to prevent postgres pool exhaustion

Running all tests with unlimited parallelism exhausts postgres
max_connections (default 100), causing sqlx::test databases to fail
setup and producing spurious RowNotFound errors.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: CI ruby env vars and mysql test binary coercion

- Ruby executor reads RUBY_BUNDLE_PATH and RUBY_GEM_PATH but CI was
  setting BUNDLE_PATH and GEM_PATH, causing "Executable bundle not
  found on worker" errors.
- MySQL test CAST(CONCAT(...) AS CHAR) returns binary type when param
  is bound as bytes. Use CONVERT(? USING utf8mb4) to ensure character
  result type.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 18:25:10 +00:00
Ruben Fiszel
acb2bd0517 ci nit 2026-02-08 17:54:48 +00:00
Ruben Fiszel
94865d3e69 ci: increase postgres max_connections to 500 for parallel tests
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 17:41:33 +00:00
Ruben Fiszel
2b4c16c16c chore: improve quickjs disabled error message, disable rust cache in CI
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 17:25:06 +00:00
Ruben Fiszel
76f5f836bd ubicloud 16 2026-02-08 17:22:35 +00:00
Ruben Fiszel
2b08f812a4 chore: add quickjs feature to all_sqlx_features and CI test command
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 17:20:07 +00:00
Ruben Fiszel
377d30e73e fix: box push() future to prevent stack overflow in nested async chains
The push() function generates a ~13KB async state machine that gets inlined
into every caller's future. In deeply nested async chains (e.g. flow execution),
this causes stack overflows. Boxing the future at the definition site via a thin
wrapper reduces each caller's stack footprint to a single pointer.

This also reverts the RUST_MIN_STACK workaround from CI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 15:51:44 +00:00
Ruben Fiszel
ec9be6f44a fix: increase test thread stack size to 8MB in CI
Rust's test harness default stack (2MB) is borderline for the deep
async state machines in worker tests. Set RUST_MIN_STACK=8388608
to prevent stack overflows in tests like test_workflow_as_code.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 15:37:48 +00:00
Ruben Fiszel
bc119011ae refactor: extract windmill-api-scripts and windmill-api-users subcrates (#7850)
* refactor: extract windmill-api-scripts and windmill-api-users subcrates

Split the monolithic windmill-api crate by extracting scripts.rs, flows.rs,
users.rs, and users_oss.rs into dedicated subcrates. This reduces incremental
rebuild times when editing these modules.

Changes:
- Create windmill-api-scripts crate (scripts.rs + flows.rs, ~4.3K lines)
- Create windmill-api-users crate (users.rs + users_oss.rs, ~2.4K lines)
- Move clear_schedule to windmill-queue (shared by scripts, flows, workspaces)
- Move username utilities (VALID_USERNAME, INVALID_USERNAME_CHARS,
  generate_instance_wide_unique_username) to windmill-common/src/usernames.rs
- Move COOKIE_DOMAIN, IS_SECURE, WithStarredInfoQuery, BulkDeleteRequest,
  WebhookShared to windmill-common for cross-crate access
- Original files in windmill-api become thin stubs with pub use re-exports
- EE-dependent route handlers remain in windmill-api (create_user, rename_user,
  set_password, reset_password, etc.)
- Feature forwarding for enterprise, private, parquet, no_auth

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: extract windmill-api-workspaces subcrate (Step 3)

Move workspaces.rs, workspaces_extra.rs, workspaces_oss.rs, and
workspaces_ee.rs into a new windmill-api-workspaces crate (~7K lines).

Routes that depend on windmill-api internals (AI copilot, teams,
tarball export, critical alerts, stripe) remain in the windmill-api
stub. The subcrate handles all other workspace management routes.

Also moved send_email_if_possible to windmill-common/email_oss.rs
to make it available across subcrates without circular deps.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

* refactor: extract windmill-api-groups subcrate (groups.rs + folders.rs)

Extract groups.rs (1,093 lines) and folders.rs (833 lines) into a new
windmill-api-groups subcrate. Both modules had clean dependencies on
already-extracted crates (windmill-api-auth, windmill-common,
windmill-api-workspaces). Also removes unused re-exports of
get_instance_username_or_create_pending and INVALID_USERNAME_CHARS
from windmill-api/src/utils.rs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: add granular_acls.rs and folder_history.rs to windmill-api-groups

Extract granular_acls.rs (395 lines) and folder_history.rs (68 lines) into
the windmill-api-groups subcrate. Both modules only depend on already-extracted
crates and belong to the same access-control domain as groups and folders.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove unused imports and dead code from subcrate extraction

- Remove unused BASE_URL import from lib.rs
- Remove workspaces_extra.rs and workspaces_oss.rs re-export stubs (no consumers in windmill-api)
- Remove dead send_email_if_possible OSS stub (callers moved to windmill-api-users)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

* chore: bust CI cargo cache for subcrate split

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: re-export BASE_URL for EE files that use crate::BASE_URL

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: forward no_auth feature to windmill-api-users, remove dead code

- Add "windmill-api-users/no_auth" to windmill-api's no_auth feature
  so the login bypass in users.rs:1600 activates correctly
- Remove dead send_email_if_possible from windmill-api-users/users_oss.rs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: re-enable cargo cache for backend tests

Cache was disabled to bust stale entries from before subcrate split.
Now that a clean build has run, re-enable for faster CI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: install mold+clang in CI workflows

The .cargo/config.toml uses mold linker for x86_64-linux.
Build scripts require linking even during cargo check.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: increase cargo test timeout to 30 min

Exit code 143 (SIGTERM) means the 20-min timeout was hit during
compilation without cache. Bump to 30 min as safety net.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: limit cargo build jobs to 4 to prevent OOM in CI

Exit code 143 (SIGTERM) after 8 min = OOM kill during compilation.
8 parallel LLVM codegen jobs exhaust memory on ubicloud-standard-8.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 12:12:03 +00:00
Ruben Fiszel
bc2ce410dd test: add integration tests for all feasible language runtimes in CI (#7849)
* test: add integration tests for all feasible language runtimes in CI

Add integration tests and CI setup for languages that were previously
untested. Each test runs a simple "hello world" job through the full
worker pipeline to verify end-to-end execution.

New language tests added to worker.rs:
- Nativets (4 tests): basic string, numeric args, object return, datetime
- Bunnative: TypeScript execution via Bun native runtime
- CSharp: .NET compilation and execution (feature-gated)
- PHP: PHP script execution (feature-gated)
- Ruby: Ruby script execution (feature-gated)
- MySQL: SQL query via async MySQL client (feature-gated)
- PowerShell: pwsh script execution
- PostgreSQL: SQL query against test database

CI changes (backend-test.yml):
- Add MySQL 8.0 service container
- Add setup-php (8.3 + composer), setup-ruby (3.3), pwsh install
- Enable feature flags: csharp, php, ruby, mysql
- Pass language binary paths: PHP_PATH, COMPOSER_PATH, RUBY_PATH,
  BUNDLE_PATH, GEM_PATH, POWERSHELL_PATH, DOTNET_PATH
- Uncomment and modernize CSharp test (was commented out)
- Increase test timeout 16m -> 20m for additional runtimes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* sqlx

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 09:11:53 +00:00
Ruben Fiszel
df6d081ec0 refactor: extract windmill-dep-map crate for parallel api/worker compilation (#7846)
* refactor: extract windmill-dep-map crate for parallel api/worker compilation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: resolve WebhookShared type mismatch and missing enterprise propagation

- Make windmill-api webhook_util re-export from windmill-common instead of
  duplicating types, fixing Extension<WebhookShared> mismatch between
  windmill-store and windmill-api
- Add windmill-api-jobs/enterprise to windmill-trigger enterprise feature
  so check_license_key_valid is available when trigger subcrates enable
  enterprise on windmill-trigger

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: stop trigger features from unconditionally enabling enterprise

Move enterprise propagation for all trigger subcrates from individual
trigger feature definitions to the enterprise feature itself, so
enterprise is only enabled when explicitly requested.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: remove unused pub use re-exports and disable CI cargo cache

- Remove unused re-exports from windmill-worker/src/lib.rs:
  trigger_dependents_to_recompute_dependencies, handle_job_error,
  and unused bun/otel items
- Fix callers to use direct module paths instead
- Add windmill-dep-map as dev-dependency for tests
- Disable cargo cache in backend-check CI (faster from-scratch builds)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: restore bun re-exports used by tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

* chore: re-enable cargo cache for check_ee_full CI job

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 00:39:56 +00:00
Ruben Fiszel
3afdd2c1e4 reorganize meta features and introduce an oss meta-feature with no_auth 2026-02-06 09:13:32 +00:00
Alexander Petric
fae5f25673 feat: kafka trigger kerberos/gssapi support (#7815)
* feat: kafka trigger kerberos/gssapi support

* chore: update ee-repo-ref to bb32d562120dc34bbd8d659d92a0d4b8824b8c4c

This commit updates the EE repository reference after PR #405 was merged in windmill-ee-private.

Previous ee-repo-ref: 128c6549d4557895a362fb720f56afa54d6f566b

New ee-repo-ref: bb32d562120dc34bbd8d659d92a0d4b8824b8c4c

Automated by sync-ee-ref workflow.

* adding kafka-gssapi to all_sqlx_features

* ee ref

* ee ref

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-02-06 06:23:24 +00:00
Alexander Petric
e9887e9a38 feat: mssql integrated auth (gssapi) (#7760)
* feat: mssql integrated auth (kerberos/ntlm)

* install krb5 headers

* also make it work for windows

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-02-05 18:35:31 +00:00
Diego Imbert
9aeb96f893 Fix spawnbackend github cmd (#7819) 2026-02-05 17:46:45 +00:00
Diego Imbert
7a8c864e5b Fullstack CI Preview (#7665)
* update cf worker hostname

* set remote_url cookie from param

* ephemeral backends v1

* nit

* Run queue server

* ntis

* timeout

* better db process management

* commit hash and worktree

* nit use map

* nit

* err handling

* Revert "err handling"

This reverts commit 19de00c0c0.

* nits

* auto cleanup

* Ephemeral backend command action

* remove checkout

* checkout ee repo

* nits

* process.env.GIT_EE_DEPLOY_KEY_FILE

* resumeURLs logic

* nit

* use windmill flow for ephemeral backend action

* fixes

* new token

* worktree pools

* Delete GH secret on cleanup

* linux deploy

* nit

* nit

* unhandled promises

* nit

* fix docker bridge IP on linux

* pass cf_frontend_url to wmill flow

* git fetch

* release worktree when binary started

* send error

* logger

* logging

* logging 2

* delete log files periodically

* redirect to raw app with logs

* CORS

* MANAGER_AUTH_TOKEN

* Check organization membership

* nit

* bwrap

* nit

* return timeoutAt in resumeUrl

* nit

* Change password

* nit remove https
2026-02-05 17:26:51 +00:00
Ruben Fiszel
9d51491f38 feat: make nsjail available in all standard images (CE) (#7793)
* feat: make nsjail available in all standard images (CE)

Include nsjail binary and runtime deps in the main Dockerfile and
DockerfileSlim so sandboxing is available out of the box. Flip
DISABLE_NSJAIL default to false so nsjail is enabled by default.

Remove DockerfileNsjail (now redundant) and the build_ee_nsjail CI job,
pointing publish_ecr_s3 at the base EE image instead. Add iptables to
DockerfileFullEe to preserve the functionality from the removed nsjail
image.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* revert: keep DISABLE_NSJAIL default as true

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: pin publish_ecr_s3 to exact commit hash

Add type=sha tag to build_ee so it pushes a commit-pinned image tag.
Restore git hash lookup in publish_ecr_s3 to reference the exact image
for that commit, avoiding race conditions with the mutable dev tag.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: publish_ecr_s3 depends on build_ee_full, uses release tag

Only publish to S3 on tag releases, extracting static frontend from the
ee-full image using the semver tag.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: remove stale windmill-ee-nsjail references, add nsjail to EE slim

The windmill-ee-nsjail image is no longer published since DockerfileNsjail
was deleted. Update all references to use the base EE image (which now
includes nsjail), remove redundant nsjail deps from DockerfileExtra, and
add nsjail build to DockerfileSlimEe for consistency with CE slim.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 18:48:16 +00:00
hugocasa
c3c39cf6a4 feat: upgrade bun to v1.3.8 with regression tests (#7761)
* test: add bun executor tests with minimal production code changes

- Add comprehensive bun job tests (bun_jobs.rs) covering:
  - Basic execution, error handling, annotation modes
  - Relative imports, deeply nested imports
  - Dedicated worker protocol for both Node.js and Bun runtimes
  - Builder tests for lockfile generation (import scanning)

- Minimize changes to bun_executor.rs by exposing:
  - RELATIVE_BUN_LOADER and RELATIVE_BUN_BUILDER constants
  - build_loader() function and LoaderMode enum
  - BUN_DEDICATED_WORKER_ARGS constant
  - generate_dedicated_worker_wrapper() function

- Tests call production code directly (build_loader) instead of
  duplicating script generation logic

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit

* fix: reuse BUN_PATH/NODE_BIN_PATH from windmill-worker, add node to CI

- Tests now use exported BUN_PATH and NODE_BIN_PATH constants instead
  of duplicating env var logic
- Update backend-test.yml:
  - Upgrade bun to v1.3.8
  - Add setup-node action
  - Add NODE_BIN_PATH to cargo test command

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* add private repo test

* fix private repo test

* try fix again

* fix

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 11:38:15 +00:00
Ruben Fiszel
67206f4b75 cli test nits 2026-01-29 08:44:16 +00:00
Ruben Fiszel
b334d42f95 nit cli tests 2026-01-29 08:38:07 +00:00
hugocasa
40fb61da07 fix: 404 triggers listing in CE (#7705) 2026-01-27 17:30:36 +00:00
dieriba
25b3c1b959 feat: nextcloud native triggers (#6797) 2026-01-26 16:49:52 +00:00
centdix
e57325ce6f feat(ai): native bedrock compatibility (#7668)
* check endpoint

* use flag for cred check on worker

* use sdk for auth chat side

* cleaning

* cleaning

* also handle GET

* Add session token support in BedrockClient::from_credentials

Users with temporary STS credentials need session tokens for AWS
authentication. This adds the optional session_token parameter to
support these use cases.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Cache AWS SDK config loading to avoid repeated I/O

aws_config::load_defaults() performs environment variable lookups and
potentially file I/O on each call. Cache the result in a static OnceCell
to improve performance on high-frequency Bedrock requests.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Add debouncing to Bedrock credential check button

Prevent rapid clicks from spawning multiple concurrent flow preview
jobs for the worker credential check.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Remove orphaned SigV4 signing comment

The manual SigV4 signing code was removed in favor of the AWS SDK,
but this documentation comment was left behind.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Remove verbose tracing info and change-related comments

- Remove per-request auth method info logs (too verbose for production)
- Simplify from_env log to single debug-level message
- Remove comments describing future changes rather than current behavior

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* cleaning

* Fix Bedrock feature flag compilation warnings

Gate AWS-specific imports, struct fields, and methods behind the bedrock
feature flag to eliminate dead code warnings when building without the
bedrock feature enabled.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* bedrock flag

* cleaning

* Move BedrockCredentialsCheck from edit drawer to add drawer

The component was incorrectly shown when editing resources instead of
when adding new ones. Moved it from ResourceEditor.svelte to
ApiConnectForm.svelte.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* cleaning

* cleaning

* make aws-config optional

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-01-26 11:04:14 +00:00
Ruben Fiszel
a4d0b965ef refactor: replace v2 live migration with regular SQL migration (#7678)
* refactor: replace v2 live migration with regular SQL migration

Remove the v2_finalize live migration that waited for workers to reach
v1.461+ and replace it with an idempotent SQL migration. Since enough
time has passed, all deployments are assumed to be on v1.461+.

Changes:
- Remove v2_finalize() function and spawn task from live_migrations.rs
- Remove MIN_VERSION_IS_AT_LEAST_1_461 constant from min_version.rs
- Add 20260125000000_v2_finalize.up.sql with all finalization steps
- All SQL operations use IF EXISTS/CASCADE for idempotency

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: remove migration-related delays and tighten CLI tests CI trigger

- Remove 5s backend initialization delay in cargo_backend.ts that was
  needed for the v2 live migration (now replaced with SQL migration)
- Restrict CLI tests workflow to only trigger on cli/** changes
  (removed backend/**, openapi.yaml, openflow.openapi.yaml paths)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: skip v2_finalize migration if live migration already ran

Wrap migration in DO block that checks for 'v2_finalize_job_completed'
in windmill_migrations table. If present (live migration already ran),
skip entirely to avoid unnecessary table locks on upgraded instances.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 22:32:21 +00:00
Ruben Fiszel
35e20fec93 add GitHub Actions CI and raw app sync tests (#7677)
* feat(cli): add GitHub Actions CI and raw app sync tests

- Add CLI tests GitHub Action that runs on Linux and Windows
- Add build check job that runs on CLI and openapi.yaml changes
- Uses Rust cargo backend instead of Docker for better CI compatibility
- Add cargo_backend.ts and test_backend.ts for test infrastructure
- Fix Windows path separator bug in raw_apps.ts (use "/" for relative paths)
- Fix PostgreSQL URL parsing in cargo_backend.ts
- Update tests to use gitBranches format instead of deprecated overrides
- Add raw_app_sync.test.ts for raw app sync workflow testing (ignored for now - needs EE)
- Skip tests that require EE features (git sync settings, raw apps)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): Fix Windows path compatibility issues in tests

- Use fromFileUrl() in cargo_backend.ts for proper Windows path handling
- Normalize path separators to forward slashes in resource_folders.ts
- Fix readDirRecursive to return normalized paths in test helper
- Use forward slashes consistently in buildMetadataPath and detection functions

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): Use SEP in test assertions instead of modifying logic

- Revert resource_folders.ts to use SEP as intended
- Update test assertions to use SEP for platform-specific paths
- Keep readDirRecursive normalization for consistent test comparisons

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): Use SEP for all path separators in test assertions

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): Use resolve() for proper cross-platform path handling in cargo_backend

String concatenation with path separators creates malformed paths on Windows.
Use path.resolve() instead for proper cross-platform path resolution.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(backend): Add cfg attributes for Windows compatibility

- Add #[cfg(unix)] to anyhow::anyhow import (only used in unix cfg block)
- Add #[cfg(not(windows))] to parse_file function (uses cat, only for cgroups)
- Remove unused std::io import, use std::io::Result directly

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: Windows compilation + convert integration tests to withTestBackend

- Fix unused import SYSTEM_ROOT in csharp_executor.rs on Windows by
  requiring both windows and csharp feature
- Fix unused variable id in handle_child.rs on Windows by adding
  #[allow(unused_variables)] since id is only used in cfg(unix) code
- Convert all RUN_INTEGRATION_TESTS dependent tests in sync_pull_push.test.ts
  to use withTestBackend pattern for automatic backend setup

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: configurable test features with CI_MINIMAL_FEATURES env var

- Default: full features (zip, private, enterprise) for local development
- CI mode: minimal features (zip only) when CI_MINIMAL_FEATURES=true
- Add shouldSkipOnCI() helper for tests requiring EE features
- Update EE-dependent tests to use shouldSkipOnCI()
- Add test instructions to cli/README.md

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: enable raw app tests (not EE-dependent)

Raw apps work with minimal features. 2 tests pass, 2 have test logic
bugs to investigate separately:
- "delete file and push" - file deletion not syncing correctly
- "dry-run push shows expected changes" - JSON output parsing issue

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: gate cgroups module to Linux only

cgroups are Linux-specific, the module was causing dead_code warnings
on Windows compilation.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(ci): add CI_MINIMAL_FEATURES env var to CLI tests workflow

Set CI_MINIMAL_FEATURES=true in both Linux and Windows test jobs so
the backend compiles with minimal features (zip only) and EE-dependent
tests self-skip via shouldSkipOnCI().

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): raw app tests and backend startup timing

- Add 5s delay after backend ready for migrations to complete
- Fix dry-run JSON output parsing (handle pretty-printed JSON)
- Temporarily ignore "delete file" test (needs isSuperset fix)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): raw app file deletion sync

- Add deepEqual check for files in raw_apps.ts isSuperset comparison
- Handle raw_app file deletions in sync.ts by re-pushing the entire app
- Fix test to remove CSS import before deleting the file

When deleting a file from a raw app, the sync now properly updates
the backend with the new file list (excluding the deleted file).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): Windows path separators in tests

Normalize paths for cross-platform comparison by converting
backslashes to forward slashes before path assertions.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): normalize featurePaths in multi_instance_workspace test

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test(cli): add mixed case paths sync tests for Windows compatibility

Add comprehensive tests for sync pull/push with capitalized folder paths
to catch Windows case-insensitivity issues:
- Scripts in f/MyFolder/MyScript
- Flows in f/MyFlows/DataProcessor
- Apps in f/MyApps/Dashboard
- Variables in f/MyVars/ApiKey
- Deeply nested paths with mixed case
- Multiple resources in same capitalized folder
- CamelCase folder names with numbers

Each test verifies the full pull -> modify -> push -> verify cycle.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test(cli): add idempotency check to mixed case paths tests

After each push, pull again with --dry-run --json-output and verify
that no changes are detected. This ensures the sync is stable and
catches issues where pull/push cycles cause spurious diffs.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 20:38:33 +00:00
Diego Imbert
06469b4409 (CI requires to be on main to be tested) use windmill flow for ephemeral backend action 2 (#7675)
* use windmill flow for ephemeral backend action

* fix

* nits

* nit
2026-01-24 19:34:51 +00:00
Diego Imbert
6a1d457257 use windmill flow for ephemeral backend action (#7673) 2026-01-24 19:16:58 +00:00
Diego Imbert
34f71179c2 github action for /spawn-backend command (#7669) 2026-01-23 17:09:01 +00:00
Alexander Petric
ff51a885d3 improve RHEL9 build (#7661)
* fix(ci): switch RHEL9 build from Depot to native arm64 buildx

* fix: sync frontend package-lock.json with package.json

* refactor(ci): use Depot multi-arch + crane for RHEL9 build

- Single multi-arch image (no -arm64/-amd64 suffixes)
- Use crane export to extract binaries directly from registry
- Avoids disk space issues by streaming instead of pulling full image
- Extracts both arm64 and amd64 binaries and libraries

* fix(ci): extract release/ directory to resolve hard links
2026-01-23 07:49:49 +00:00
Diego Imbert
85e617cd83 Playwright E2E (#7520)
* clean plate

* npm i

* log in e2e

* global setup login

* set license key

* Revert "set license key"

This reverts commit 86d5db2c48.

* create datatable test

* fix wrong pg_creds

* data table + db manager e2e test

* DbManagerPage class

* small refactor

* create resource test + improvements

* text db manager in resources

* Factor test logic in classes

* refactoring

* refacto

* alter table test

* alter table e2e test

* set schema in test

* nits

* fix wrong schema var

* Correct setup and parallelization

* reducedMotion

* tests passing headless !

* bigger timeout

* start e2e docker compose

* e2e runs on all databases

* nit test uid fix

* refactp

* stash

* Better Workspace Storage settings

* minio setup

* nit

* nit

* super nit

* Permission settings in modal

* badge indicator

* Fetch alter table metadata much faster

* Upgrade duckdb to 1.4.3

* Ducklake tests

* Disable transactional DDL for Ducklake (bug on their side)

* git ignore env

* bigquery tests passes

* getJsonEnv

* load coldef in parallel

* Make Bigquery schema fetching much faster

* makeLoadTableMetaDataQuery for entire db in bigquery

* refactor getDbSchemas to avoid assignment side effect

* fix col def

* Better loading state mgmt

* snowflake

* fix snowflake primary keys

* Test CI

* fix setTimeout type

* remove type node

* test e2e ci

* Revert "test e2e ci"

This reverts commit bf98a755dc.

* remove ci

* fix snowflake pk query in alternate schemas

* nit wait for coldefs

* nit snowflake

* Snowflake fk fix

* UNPROCESSABLE_ENTITY instead of INTERNAL_ERROR

* nits

* fix alter pk in snowflake

* yet other fixes

* snowflake tests pass

* nits
2026-01-22 16:20:01 +00:00
Pyra
293fdf54ff fix: bump uv 0.6.2 -> 0.9.24 (#7559)
* nit: bump uv 0.6.2 -> 0.9.24

Signed-off-by: pyranota <pyra@duck.com>

* update DockerfileBackendTests

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
2026-01-14 14:12:24 +00:00
centdix
5c41d9dd87 remove plugins (#7562) 2026-01-14 11:51:07 +00:00
centdix
4b7aac65c2 internal: typo in plugins 2026-01-14 11:16:52 +00:00
Ruben Fiszel
5c665a6cbb publish extra require sleep 2026-01-13 18:26:23 +00:00
Ruben Fiszel
894c64c510 refactor: change debugger port from 5679 to 3003
Updates the debugger service to use port 3003 instead of 5679 across
all configuration files, documentation, and code references. This aligns
the debugger with the other windmill-extra services which use ports
3001 (LSP) and 3002 (Multiplayer).

Changes:
- docker-compose.yml: Update port exposure and add DEBUGGER_PORT env
- docker/entrypoint-extra.sh: Change default port from 5679 to 3003
- debugger/dap_debug_service.ts: Update default port in code and docs
- debugger/README.md: Update port documentation
- debugger/test_debug_service.ts: Update test URLs
- docker/test_windmill_extra.ts: Update test configuration
- .github/workflows/publish_extra.yml: Update test container ports
- frontend/src/lib/components/debug/*: Update frontend examples and defaults

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-13 17:58:12 +00:00