Compare commits

...

99 Commits

Author SHA1 Message Date
HugoCasa
e033c73b79 chore: update ee-repo-ref to batch-pulling latest
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 18:07:36 +01:00
HugoCasa
8c3ac22d8d feat: add as_worker_tag() helper, benchmark results and model
- Extract bunnative→nativets tag logic into ScriptLang::as_worker_tag()
- Add benchmark results for batch pull vs direct SQL (1W and 3W)
- Add throughput model script comparing batch vs SQL at scale
- Add nativets_sleep benchmark script support

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 18:05:15 +01:00
HugoCasa
21398e5447 fix: use BATCH_PULL_URL env var and add JWT exp claim
- Replace BASE_INTERNAL_URL overloading with dedicated BATCH_PULL_URL
  env var for native workers' HTTP pull endpoint
- Add exp claim to JWT token (required by jsonwebtoken validation)
- Token expires in 30 days, renewed on worker restart

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 19:15:12 +01:00
HugoCasa
980cbcccf0 Merge remote-tracking branch 'origin/main' into batch-pulling 2026-03-05 17:14:53 +01:00
HugoCasa
dd422fcc5d fix: enable batch pull for worker-only mode with BASE_INTERNAL_URL
Native workers in Mode::Worker (no co-located server) can now use HTTP
batch pull when BASE_INTERNAL_URL is explicitly set pointing to the
remote server. The batch buffer itself only runs on the server side.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 16:33:20 +01:00
HugoCasa
876a9cfc8e feat: batch job pulling for native workers
Reduce DB polling overhead for native workers by batch-fetching jobs
server-side and serving them from an in-memory buffer via HTTP.

- Add batch_pull() in windmill-queue: single SELECT...FOR UPDATE SKIP LOCKED LIMIT N
- Add batch pull SQL helpers (make_batch_pull_query, format_batch_pull_query)
- OSS stubs for agent-workers accept batch_buffer parameter (4-tuple return)
- Native workers self-sign JWT and pull jobs via HTTP when co-located with server
- Add uses_batch_http_pull column to worker_ping for server-side tracking
- Worker pull loop: HTTP batch pull when client available, SQL otherwise

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 16:08:28 +01:00
Ruben Fiszel
86065aaac8 chore(main): release 1.651.1 (#8242)
* chore(main): release 1.651.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-03-05 14:51:38 +00:00
Ruben Fiszel
e3f4130c68 nits 2026-03-05 14:36:51 +00:00
Ruben Fiszel
2e582b1bc1 fix: prevent slow loading toast interval from leaking on promise cancellation (#8240)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 14:23:36 +00:00
Ruben Fiszel
2d583826dc fix: suppress unused variable warnings on windows builds (#8241)
* fix: suppress unused variable warnings on windows builds

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref.txt to merged commit

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 15:17:51 +01:00
Ruben Fiszel
972ae7aa29 chore(main): release 1.651.0 (#8235)
* chore(main): release 1.651.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-03-05 13:42:07 +00:00
Ruben Fiszel
d46913b74a fix: write fallback package.json for codebase mode nsjail (#8239)
* fix: write fallback package.json for codebase mode to fix nsjail ERR_INVALID_PACKAGE_CONFIG

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: add e2e tests for codebase mode with and without nsjail

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-05 13:35:53 +00:00
Roderik-WU
90f4c64ee1 fix(python-client): add delete_s3_object (#8216)
* Implement remove_s3_file method

Add method to permanently delete a file from S3 bucket.

* Add test for removing S3 file

Added a test case to verify removal of a file from S3.

* Add remove_s3_file function to delete S3 files

Added a function to permanently delete a file from the S3 bucket.

* Rename remove_s3_file to remove_3_object

* Rename remove_3_object to remove_s3_object

* Rename test method and update S3 object handling

* Rename remove_s3_object to delete_s3_object

* Rename test_remove_s3_object to test_delete_s3_object and remove_s3_object to delete_s3_object
2026-03-05 12:49:59 +00:00
hugocasa
a8cbe9396f fix: update CLI bun template to match UI template (#8238)
* fix: update CLI bun template to match UI template

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: simplify CLI bun template, only add mode comments

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 12:04:38 +00:00
centdix
ce041e8a5e feat: hash-based MCP tool names for long paths (#8133)
* feat: replace _TRUNC with hash-based MCP tool names (50 char limit)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: reduce MCP tool name limit from 50 to 40 chars

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: use path prefix filtering instead of separate DB query for hashed name resolution

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: remove long path warning from MCP token creation (hashing handles long names)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: unify tool prefix parsing and fix extract_path_prefix_from_hashed for Hs- names

- Replace `is_hashed_name` + `parse_hashed_name` with unified `parse_tool_prefix`
  that returns `(type_str, is_hub, is_hashed)` in one call
- Fix `extract_path_prefix_from_hashed` to dynamically determine prefix length
  (3 for `Hs-`, 2 for `S-`/`F-`) instead of hardcoding index 2
- Simplify `reverse_transform` to reuse `parse_tool_prefix`
- Add tests for invalid prefixes and `Hs-` prefix handling

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: escape LIKE wildcards in MCP hashed name path prefix query

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: respect favorites scope in hashed tool name resolution

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: deduplicate MCP tool name resolution and rename get_path_or_id

- Extract `unescape_path` helper in transform.rs to deduplicate the
  3-step placeholder unescape logic
- Extract `find_matching_path` helper in runner.rs to deduplicate
  script/flow candidate matching via ToolableItem trait
- Remove verbose tracing::info! logs from hashed tool resolution hot path
- Fix doc comment referencing nonexistent `is_hashed_name` function
- Rename `get_path_or_id` to `get_transformed_path` for clarity

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: update stale doc comments to reflect MAX_PATH_LENGTH=40

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-05 12:04:20 +00:00
Ruben Fiszel
65082159d8 tighten volume limits (#8236)
* feat: add volume limits info in CE volumes drawer

Show an info alert in the volumes drawer when running in Community
Edition, mentioning the 20 volumes per workspace and 50 MB per file
limits. Update ee-repo-ref for companion EE changes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref to a61366dd4d9e9b1f98a421aaa6d3f63194615275

This commit updates the EE repository reference after PR #438 was merged in windmill-ee-private.

Previous ee-repo-ref: 05385738e36e81f5bc51d15c0ca60bba30457c21

New ee-repo-ref: a61366dd4d9e9b1f98a421aaa6d3f63194615275

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-05 06:44:32 +00:00
Ruben Fiszel
5f0ef936d1 feat: add sandbox annotations, volume mounts, for AI sandbox starting with claude (#8058) 2026-03-05 06:19:51 +00:00
Ruben Fiszel
bee50b83d1 chore(main): release 1.650.0 (#8218)
* chore(main): release 1.650.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-03-05 05:29:05 +00:00
hugocasa
e56ccd200b feat: token expiration notifications (#8190)
* feat: add token expiration notifications via email, critical alerts, and webhooks

- Monitor loop checks for tokens expiring within 7 days and sends
  email notifications to token owners. Tracks notification state via
  new `expiry_notified` column on the token table to avoid duplicates.
- When tokens expire and are deleted, owners are also notified.
- Critical alerts (in-app UI) are gated behind a new instance setting
  `critical_alerts_on_token_expiry` (off by default); emails are
  always sent regardless of the setting.
- Add TokenExpiringSoon and TokenExpired webhook message variants for
  workspace webhook integrations.
- Frontend: show expiration badges and a warning banner on the tokens
  table for tokens expiring within 30 days.
- Exclude session and ephemeral tokens from all notifications.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: use separate token_expiry_notification table for dedup

- Replace `expiry_notified` column on token table with a dedicated
  `token_expiry_notification` table (token, expiration)
- Insert notification row on token creation via shared
  `register_token_expiry_notification()` helper
- Delete notification row atomically when sending the notification
- Clean up orphaned rows in `delete_expired_items()`
- No FK constraint to avoid cascade overhead on token deletions
- Add index on expiration column for efficient range queries

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: calendar-based expiration badge and move notification cleanup

- Fix daysUntilExpiration to compare calendar dates instead of time diff
- Move notification row cleanup from delete_expired_items to
  check_expiring_tokens to keep it off the hot path
- Use simple expiration <= now() index scan instead of NOT EXISTS join

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 05:22:46 +00:00
Ruben Fiszel
eab789beeb chore: upgrade rquickjs from 0.8 to 0.11 (#8233)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 05:13:42 +00:00
Ruben Fiszel
077779ec52 fix: improve windows compatibility
* ci: add Windows backend integration test workflow

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* ci: temporarily add push trigger for testing

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* ci: add --no-fail-fast to run all test binaries

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: Windows path handling for backend integration tests

- WINDMILL_DIR: use std::env::temp_dir() on Windows instead of /tmp/windmill
- HOME_ENV: fall back to USERPROFILE on Windows when HOME is not set
- loader.bun.js: normalize paths to forward slashes for consistent
  comparison with Bun's resolver output on Windows
- bun_executor.rs: convert job_dir to forward slashes in JS template
  strings to avoid backslash escape issues (\t -> tab, etc.)
- go_executor.rs: fix windows_gopath() double backslash bug (r"\\" -> "\\")
- bash_executor.rs: default to "bash" (in PATH) on Windows instead of /bin/bash

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: improve Windows diagnostics and fix onLoad handler

- Include path in create_directory_async/sync panic messages
- Add WINDMILL_DIR initialization debug output
- Fix loader.bun.js onLoad: use properly escaped regex instead of
  returning undefined (Bun requires onLoad to return an object)
- Add env var debug output to CI workflow

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: sanitize Windows-invalid characters in test worker names and fix cargo path

- Replace :: with __ in worker names (colons illegal in Windows dir names)
- Fix HOME_DIR to fall back to USERPROFILE on Windows
- Add PATH fallback for cargo discovery on Windows
- Add debug logging to bun loader for fetch errors

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: handle single colons in worker names, pass MSVC linker env vars, revert bun debug

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use .exe binary name on Windows and normalize bun import URL paths

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use absolute path for rust binary, normalize bun resolve paths

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use .wurl extension instead of .url for bun import resolution on Windows

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use custom namespace for bun plugin to bypass default file resolution

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use virtual namespace for bun import resolution to avoid Windows path issues

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: handle Windows 8.3 paths and namespace-prefixed importers in bun loader

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: strip namespace prefix from args.path and handle absolute imports without leading slash in bun loader

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: simplify bun loader and remove redundant cargo path lookups

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use platform-specific cargo binary path with .exe on Windows

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: replace HOME_DIR with HOME_ENV in rust_executor to remove duplication

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: keep original bun loader on linux, use virtual namespace loader only on windows

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-04 20:20:18 +00:00
hugocasa
63ebae8829 feat: replace hub error toasts with warning alerts and add disable hub setting (#8225)
* feat: replace hub error toasts with warning alerts and add disable hub setting

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: guard hub script cache refresh when hub is disabled

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 15:12:00 +00:00
centdix
87ebeaa51d chore: make rust-analyzer plugin opt-in via USE_RUST_PLUGIN env var (#8227)
* feat: optionally enable rust-analyzer plugin in worktree settings

When USE_RUST_PLUGIN env var is set, the worktree-env script now includes
the rust-analyzer-lsp plugin in .claude/settings.local.json.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: remove rust-analyzer plugin from default settings

The rust-analyzer plugin is now opt-in via USE_RUST_PLUGIN env var
in worktree-env, so it no longer needs to be in the shared settings.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: add WM_CLONE_DB and USE_RUST_PLUGIN to wmdev startup envs

Defaults both to false so they can be toggled per-worktree.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: use explicit truthy checks for WM_CLONE_DB and USE_RUST_PLUGIN

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 15:09:42 +00:00
hugocasa
62382fd286 fix: wrap set_encryption_key in a single database transaction (#8212)
Prevent workspace corruption when re-encryption fails mid-loop by
wrapping the key update and variable re-encryption in a single
transaction. If any step fails, the entire operation rolls back.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 14:53:56 +00:00
Ruben Fiszel
19c065bed5 fix: handle multipart stream errors gracefully instead of panicking (#8226)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 14:44:33 +00:00
hugocasa
164e499c64 feat: add variable and resource types to flow env variables (#8214)
* feat: add variable and resource types to flow env variables

Flow env variables can now reference workspace variables ($var:path)
and resources ($res:path) that are resolved at runtime. Adds Variable
and Resource type options to the flow env editor with ItemPicker and
ResourcePicker components, and resolves references in both the flow
worker (via transform_json) and the API fallback endpoint.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(frontend): use inline DollarSign icon for variable picker

Replace the separate "Pick" button with the standard inline DollarSign
icon overlay that appears on hover, matching the existing ArgInput
pattern. Also add the icon to the string type input for quick variable
linking from any string field.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: simplify flow env var resolution and json_path handling in API

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(frontend): always show flow env variables in property picker

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: update flow_env openapi type to allow any JSON value

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(frontend): remove redundant variable type from env var dropdown

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(frontend): use Label component and fix alert text in flow env vars editor

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(frontend): avoid redundant stringify/parse roundtrip in env type switch

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: address PR review comments for flow env vars

- Deduplicate db_authed in jobs.rs $var/$res resolution
- Add warn logging on variable/resource resolution failures
- Consolidate $effect blocks and remove auto-type-correction effect
- Make linked variable text a clickable link to variable editor
- Add hash-based variable editor opening on variables page

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* perf: avoid cloning entire FlowValue to resolve flow_env references

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 14:20:50 +00:00
Ruben Fiszel
8a859ff7b9 add full-code app import with tabbed YAML/JSON format selection (#8224)
Combine YAML/JSON import into tabs within a single drawer (YAML default)
and add full-code app import option. Uses sessionStorage to persist import
data across the full page reload required by cross-origin isolation headers
when navigating to /apps_raw/add.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 13:29:51 +00:00
Guilhem
c9c3baecb3 add context menu with delete option to preprocessor nodes (#8223)
* fix: add context menu with delete option to preprocessor nodes

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add delete styling and shortcuts to right-click context menu

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-04 12:48:02 +00:00
Pyra
baf2bcf14d feat: make WM_END_USER_EMAIL display users from different workspaces (#8208)
Signed-off-by: pyranota <pyra@duck.com>
2026-03-04 11:50:59 +00:00
claude[bot]
7fe1594d22 add data tables comment to scheduled poll templates (#8221)
Add a comment to each scheduled poll template (Python, Deno, Bun, Go)
mentioning that data tables can be used for more complex states, with
a link to the documentation.

Closes #8220

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-04 11:47:36 +00:00
Guilhem
c0c9388415 feat: add move, delete, and duplicate to flow node context menu (#8050)
* feat: add context menu, multi-select actions, and keyboard shortcuts to flow editor

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address review feedback on context menu PR

- Revert accidental static import of @scalar/openapi-parser (keep lazy-loaded)
- Restore [data-context-menu] in portalDivs for clickOutside compatibility
- Make noteDisabled reactive ($derived) in ModuleNode
- Use platform-aware shortcut hint (⌫ on Mac, Del on Windows/Linux)
- Optimize resolveSelectedModuleIds with single-pass ancestor map

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address additional review feedback on flow context menu PR

- Use $derived.by instead of $derived for computed bounds in SelectionBoundingBox
- Remove redundant structuredClone wrappers around $state.snapshot
- Add null guard for originalModules/targetModules in move handler
- Add upper-bound guard (n < 10000) to copyId loop
- Fix fragile toggle comparison in moveManager with full array equality

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 10:53:01 +00:00
Diego Imbert
4bf827bea4 feat: persistent Db manager state in URI (#8134)
* DB Manager state in URL

* Fix state not saving

* shorted uri params

* infer db_type from prefix

* Revert "infer db_type from prefix"

This reverts commit 7415fbed3d.

* dbm syntax

* infer database type

* Omit main and public

* remove legacy #dbmanager:

* Preserve hash

* nit

* Fix remaining dbManagerDrawer objects
2026-03-04 10:46:34 +00:00
Diego Imbert
53caecf1da feat: Ducklake typechecker (#8118)
* Typedchecked ducklake queries

* Display script preview error as SQL error

* Fix duplication

* fix replacer

* Revert "fix replacer"

This reverts commit c5492033c8.

* Don't recompile regex every call

* nit OOB

* avoid potential panic

* Apply suggestions from code review

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* safety throw

* Update backend/windmill-worker/src/duckdb_executor.rs

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* Try catch individual chunks in prepareDatatableQueries

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* format

* nit comment

* Revert "Try catch individual chunks in prepareDatatableQueries"

This reverts commit ae64a8ad27.

* Correct try catch

* better error messages

* nit unused variable

* comment

* handle non describable queries

* npm i

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
2026-03-04 10:46:08 +00:00
Ruben Fiszel
424ca59dfe feat: make WINDMILL_DIR configurable via environment variable (#8215)
* fix: auto-heal corrupted python runtime cache on remote workers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Revert "fix: auto-heal corrupted python runtime cache on remote workers"

This reverts commit 0ea013a554.

* feat: make WINDMILL_DIR configurable via environment variable

Allow users to configure the base directory for Windmill's tmp/cache files
via the WINDMILL_DIR env var (default: /tmp/windmill). This fixes Python
runtime cache corruption on RHEL systems where systemd-tmpfiles-clean
removes files from /tmp.

Converts TMP_DIR (renamed to WINDMILL_DIR) and all derived cache directory
constants from compile-time const &str (concatcp!) to runtime lazy_static
String values.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee ref

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee ref

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: deref ERROR_DIR lazy_static for AsRef<Path> and Display traits

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee ref to branch name for CI compatibility

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: deref lazy_static constants in all executor files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee ref

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee ref

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee ref

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: panic if WINDMILL_DIR has trailing slash

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: also reject trailing backslash in WINDMILL_DIR for Windows

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: deref GO_BIN_CACHE_DIR in test utils

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: replace remaining hardcoded /tmp/windmill paths and validate empty WINDMILL_DIR

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: nsjail powershell mount dst, Windows path assumptions, pwsh deref consistency

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: restore Windows /tmp path translation in go and bun executors

The Windows path translation replaces /tmp with the Windows temp dir
(e.g. C:\tmp) before normalizing slashes. Without this, the default
WINDMILL_DIR=/tmp/windmill produces paths without a drive letter on
Windows.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref to 6fd5a2ce908235a17975ad4dbdf0051cd89334f3

This commit updates the EE repository reference after PR #436 was merged in windmill-ee-private.

Previous ee-repo-ref: e8c03e16720833230ebd1878b4c63642ecc6c80f

New ee-repo-ref: 6fd5a2ce908235a17975ad4dbdf0051cd89334f3

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-04 08:53:25 +00:00
Ruben Fiszel
fafa809670 chore(main): release 1.649.0 (#8198)
* chore(main): release 1.649.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-03-04 07:14:00 +00:00
hugocasa
c97d8b4715 feat(frontend): add script recorder for offline replay (#8200)
* feat(frontend): add script recorder for offline replay of script test executions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(frontend): use Video icon for recording instead of Circle

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(frontend): use Disc icon for recording

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(frontend): improve script recorder replay and recording privacy

- Record schema at capture time in ScriptRecording (lockfile unavailable for previews)
- Read schema from recording instead of job object in replay view
- Remove lockfile tab (not available via normal job API for preview jobs)
- Use text-xs for code/schema views, remove max-height limits
- Disable log download button in replay (endpoint won't work without real job)
- Truncate UUIDs in downloaded recordings (last 8 chars) for privacy
- Make activeReplay a $state so $derived(isReplay) in FlowStatusViewerInner
  updates reactively, preventing stale reads that caused API calls during replay
- Use JSON round-trip instead of structuredClone to unwrap $state proxies

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 18:41:11 +00:00
wendrul
f6ceb2e366 Remove edit in fork button for app.windmill.dev (#8213)
* Remove edit in fork button for app.windmill.dev

* remove duplicate import
2026-03-03 18:39:24 +00:00
Ruben Fiszel
ef7b2ec81c sqlx 2026-03-03 16:48:40 +00:00
Ruben Fiszel
ee01acd9a6 feat: move index management out of /srch/, add storage size reporting (#8169)
* feat: move index management endpoints out of /srch/, add storage size reporting

- Mount management_service() at /api/indexer (authenticated)
- Add management_service() OSS stub in indexer_oss.rs
- Update OpenAPI: /indexer/delete/{idx_name} and /indexer/storage
- Show disk + S3 storage sizes in IndexerMemorySettings UI

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add index storage section with refresh button

Move storage sizes into a dedicated "Index storage" section with a
refresh button to reload sizes after clearing an index.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add indexer status endpoint with liveness detection and improve settings UI

Add GET /indexer/status endpoint that combines lock-based liveness
detection with storage sizes. Frontend now shows running/stopped
indicators with last-active timestamps for each indexer.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* update ee ref

* fix

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 16:46:10 +00:00
Ruben Fiszel
7b6f1deeb1 update ee ref 2026-03-03 16:25:05 +00:00
Henri Courdent
f331e1f0ad Error frontend links (#8210) 2026-03-03 16:11:54 +00:00
centdix
aafe716823 chore: add env config for wmdev (#8209)
* add wmdev startup envs

* name
2026-03-03 15:25:34 +00:00
Guilhem
e97da86067 fix(frontend): prevent subflow expansion from hiding all insertion points (#8203)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-03 13:26:29 +00:00
Ruben Fiszel
26f4f2b399 fix: clean up slow-load toast interval on component destroy (#8207)
The slowStreamIntervalId (which fires "Loading is taking a long time..."
toasts every 15s) was not cleared in onDestroy, causing it to keep
firing after navigating away from the runs page.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 13:25:17 +00:00
Ruben Fiszel
cac4bdd54f fix: gracefully handle malformed OAuth entries in instance config (#8205)
When an OAuth provider entry in instance settings has unexpected types
(e.g. `"true"` instead of `true` for req_body_auth), the entire
/api/settings/instance_config endpoint would fail with a deserialization
error, preventing access to any instance settings.

Introduce OAuthClientEntry enum that tries typed OAuthClient
deserialization first and falls back to raw JSON, logging the
deserialization error. This allows the settings page to load even when
individual OAuth entries are malformed.

Also show a user-visible error toast in SaveButton on save failure
instead of only logging to console.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 12:37:37 +00:00
Ruben Fiszel
4a14e9436e prevent async lock gen race condition in mixed case path tests (#8202)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 07:19:16 +00:00
Ruben Fiszel
e6f7775d4d fix: skip stop_after_if evaluation for skipped (identity) flow steps (#8201)
* fix: skip stop_after_if evaluation for skipped (identity) flow steps

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: relax is_identity_job guard to only require skip_if

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 07:00:27 +00:00
Ruben Fiszel
c5b440e569 cli tests nit 2026-03-03 06:09:25 +00:00
Ruben Fiszel
2b2be38f12 fix: use exact matching for python requirements directive parsing (#8199)
* fix: use exact matching for python requirements directive parsing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: apply same exact matching fix to CLI parser

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 21:21:28 +00:00
Ruben Fiszel
50defdded1 perf: use two-step query in input history to leverage v2_job index (#8197)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 21:05:35 +00:00
Ruben Fiszel
759eb68a7f use polling loop in schedule integration tests to avoid CI flakes (#8196)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-02 18:49:32 +00:00
Ruben Fiszel
3e6b1bee59 sqlx 2026-03-02 18:07:30 +00:00
lubu0
f412fbc3b7 add top-level get_job wrapper function (#8192) 2026-03-02 18:01:56 +00:00
Diego Imbert
cf3ddce68a Fix data tables not working with non-secret pg variables (#8195) 2026-03-02 18:01:18 +00:00
Ruben Fiszel
e906818982 chore(main): release 1.648.0 (#8182)
* chore(main): release 1.648.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-03-02 16:09:50 +00:00
claude[bot]
18552046c2 feat: add right-click context menu to ObjectViewer (#8181)
* feat: add right-click context menu to ObjectViewer

Add a contextual menu to ObjectViewer.svelte that appears on right-click
with three actions:
- Copy value: copies the field's value to clipboard
- Copy object key: copies the property key name
- Copy entire object: copies the parent object as JSON

Uses setContext/getContext to share the context menu handler across
recursive ObjectViewer instances, rendering a single menu at the root
level via Portal. Reuses existing contextMenuStyles for visual consistency.

Closes #8177

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* Fix popover closing

* Use existing ContextMenuItem patterns

* hover style

* close contextmenu on pointerdown outside

* try catch for circular objects

* Fix copying undefined not working

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Diego Imbert <diego@windmill.dev>
Co-authored-by: Diego Imbert <70353967+diegoimbert@users.noreply.github.com>
2026-03-02 16:03:06 +00:00
hugocasa
a111653c6d fix: don't insert underscore after digit in PascalCase to snake_case conversion (#8184)
* fix: don't insert underscore after digit in PascalCase to snake_case conversion (#7934)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* update parsers

* remove unused wasms + fix build

* update cli lock

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-02 16:02:48 +00:00
centdix
e0d4a4b38e chore(workmux): add name field to config (#8186)
* chore(workmux): add name field to config

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Update .workmux.yaml

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 16:02:06 +00:00
Ruben Fiszel
9e92445fae fix: preserve debouncing settings for post-preprocessing arg accumulation (#8191)
* fix: preserve debouncing settings for post-preprocessing arg accumulation

After preprocessing completes, store the flow's debouncing settings in
runnable_settings_handle on v2_job_queue so that maybe_apply_debouncing
can find them when the surviving job is pulled. Without this, the handle
is NULL and arg accumulation silently does nothing for flows with
preprocessors.

Also adds a debouncing badge in flow settings and 4 focused accumulation
tests covering scripts, flows without preprocessor, flows with
preprocessor (with and without the fix).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update sqlx prepared query for worker_flow.rs change

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 16:01:36 +00:00
Ruben Fiszel
5faeae9486 nit copy license key on workmux creation 2026-03-02 15:21:45 +00:00
Ruben Fiszel
cfd9541ab1 fix(frontend): preserve keycloak realm url between instance settings saves (#8189)
* fix(frontend): preserve keycloak realm url between instance settings saves

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(backend): preserve provider-specific oauth fields through round-trip

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 14:26:34 +00:00
centdix
b121f4388b docs: move autonomous-mode reference to system prompt (#8173)
* docs: move autonomous-mode reference from CLAUDE.md to system prompt

Remove the autonomous-mode.md bullet from CLAUDE.md and instead reference
it via the workmux system prompt, matching the workmux-web pattern. Also
remove the duplicated "Dev Environment (tmux)" section from
autonomous-mode.md since that info is already in the system prompt.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: add autonomous-mode.md reference to wmdev sandbox system prompt

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 10:43:19 +00:00
HugoCasa
5ebaa43aa1 internal(workmux): allow cloning main db using WM_CLONE_DB or --clone-db 2026-03-02 11:18:36 +01:00
Guilhem
7a5e487878 feat(frontend): add drag-and-drop node movement in flow editor (#8076)
* feat: add drag-and-drop node movement in flow editor

Replace the 2-step click-based move with drag-and-drop: grab a node's
Move icon, drag it near an insert point, see a visual drop indicator,
and drop to move. Click-based move is preserved as fallback.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: hide insert buttons on edges during drag-and-drop

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: unify drop zone and legacy move target styles

Use consistent dot indicator for both drag-and-drop and click-based
move targets. Use text-accent theming, hide insert buttons during drag.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: render real SvelteFlow graph in drag ghost for subflows

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: center drag ghost on the dragged node instead of the whole subflow

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: pass isSubflow prop through drag system and improve move UX

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: fade entire subflow during legacy move and drag-and-drop

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* style: use text-secondary for move and drop target indicators

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: improve drag-and-drop visual feedback with proximity cues

Ghost opacity reacts to drop zone proximity (dims when far, brightens
when near). Add move icon badge near cursor that highlights on valid
drop target. Switch hit detection from circular radius to axis-aligned
bounding box matching the node gap dimensions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: unify DragGhost to always use MiniFlowGraph

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: scale drag ghost using flow viewport zoom instead of fixed width

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: register drop zone positions from BaseEdge instead of recomputing from node data

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: hide node UI clutter during drag and polish drag ghost

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: fade all deeply nested nodes when dragging a subflow

Previously only immediate children of a dragged subflow would fade —
deeply nested nodes (e.g. steps inside a forloop inside a branchall)
stayed at full opacity. Store the full set of dragged node IDs on
DragManager and check set membership instead of single-parent comparison.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: rename DragManager to MoveManager and eliminate moving prop drilling

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: unify subflow node computation for both move modes

Extract getSubflowNodeIds() to moveManager.svelte.ts and populate
draggedNodeIds via a single $effect in DragCoordinator for both legacy
click-to-move and drag-and-drop. Consumers (MapItem, NodeWrapper) now
only check draggedNodeIds set membership instead of dual-checking.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: clean up drag-and-drop code review issues

Fix toggle risk in DragCoordinator by using forceSetMoving instead of
the toggle-based setMoving. Remove dead code (DragInfo unused fields,
parentSubflowId, GHOST_ZOOM_FACTOR, debug log), extract duplicated
expressions to $derived variables, and add missing type annotations.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: clear click-to-move when drag starts to prevent dual mode activation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: centralize draggedNodeIds cleanup in $effect

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: adjust insertion index when moving node forward in same array

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: address PR review feedback for node move feature

- Snapshot drag ghost once at drag start using untrack() to avoid
  recomputing on every nodes/edges change during drag
- Rename setMoving/forceSetMoving to toggleMoving/setMoving for clarity
- Add capture: true to DragCoordinator's Escape handler for consistency
- Rename MOVE_BTN_OFFSET to DRAG_HANDLE_OFFSET with descriptive comment
- Move misplaced import to top of moveManager.svelte.ts
- Replace (n.data as any).offset with typed nodeOffset() helper

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: register asset/AI node types in MiniFlowGraph for drag ghost

MiniFlowGraph was missing asset, assetsOverflowed, aiTool, and
newAiTool node types, so these nodes rendered as invisible elements
that inflated the drag ghost bounding box. Register them so the
ghost renders all node types correctly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: resolve relative positions to absolute for xyflow child nodes in drag ghost

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use initialViewport instead of fitView so drag ghost matches flow zoom

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* style: format BaseEdge.svelte

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: fade asset and AI tool nodes when their parent is being moved

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: include child nodes of edge-matched nodes in subflow ID collection

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: hide +Tool button when moving nodes

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: address PR review feedback (listener cleanup, set iteration, dead code)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style: position cancel move button on top of node instead of above it

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: compute draggedNodeIds eagerly via callback instead of reactive effect

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: remove redundant parentModuleId from NodeWrapper

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: address PR review comments for drag ghost and move manager

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 10:06:23 +00:00
Ruben Fiszel
cfc8ab5b2d chore(main): release 1.647.2 (#8180)
* chore(main): release 1.647.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-03-02 09:45:06 +00:00
Ruben Fiszel
758b35f8eb fix: update oracle instant client arm64 download url (#8179)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 09:40:00 +00:00
Ruben Fiszel
b34ba965c1 chore: bump Bun to v1.3.10 (#8178)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 09:39:43 +00:00
Ruben Fiszel
889c98b38b chore(main): release 1.647.1 (#8171)
* chore(main): release 1.647.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-03-02 07:40:38 +00:00
Ruben Fiszel
db44b8be74 fix: add missing display_name and tenant fields to instance config OAuthClient (#8176)
* fix: add missing grant_types field to instance config OAuth structs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add missing display_name and tenant fields to instance config OAuthClient

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 07:35:40 +00:00
Ruben Fiszel
fca94f88dd fix: add missing grant_types field to instance config OAuth structs (#8175)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 07:30:36 +00:00
Ruben Fiszel
c70307d3f2 fix: show sync endpoint timeout setting on all instances (#8170)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-01 23:21:06 +00:00
centdix
89f835727b chore: use Nix profiles in sandbox Docker image (#8140)
* feat: use Nix profiles in sandbox Docker image

Replace manual tool installs (rustup, nodesource, curl installers) in
sandbox-image/Dockerfile.sandbox with a single `nix profile install .#sandbox`.
All tools (Rust, Node, Bun, Deno, Go, gh, sqlx-cli, cargo-watch, Chromium,
Playwright, etc.) are now managed declaratively via flake.nix.

- Add `packages.sandbox` and `packages.sandbox-full` buildEnv outputs to flake.nix
- Add `sandbox-env` helper script for browser tooling env vars
- Update playwrightWrapper to export PLAYWRIGHT_BROWSERS_PATH
- Rewrite Dockerfile.sandbox: Nix replaces ~50 lines of manual installs
- Update entrypoint.sh to source Nix profile PATH
- Delete deprecated root Dockerfile.sandbox

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: sandbox image runs as non-root user with wmdev

- Rewrite entrypoint.sh to start PostgreSQL as current user (no
  chown/su needed), fixing "Operation not permitted" when wmdev
  runs containers with --user
- Add chmod -R 777 /root and passwd entry for UID 1000 so non-root
  containers can access bashrc, nix-profile, and tool configs
- Remove apt postgresql server (Nix profile provides it)
- Fix bash history expansion errors from literal `!` in system prompt
- Fix asciinema path reference (available on PATH, not hardcoded)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: wrap pkg-config in sandbox profiles to bake in Nix search path

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add openssh-client and sudo to sandbox image for full root access

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use useradd instead of manual passwd entry for sandbox agent user

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 17:42:33 +00:00
Ruben Fiszel
6eca08480a chore: remove legacy wmill_pg python client (#8155)
The wmill_pg package (psycopg2 wrapper for running PostgreSQL queries)
has been fully replaced by Windmill's native PostgreSQL support.
Remove the package directory and all references from build, publish,
install, version, LSP, and dependabot configs.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 08:52:35 +00:00
Ruben Fiszel
36353359f6 chore(main): release 1.647.0 (#8127)
* chore(main): release 1.647.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-03-01 07:00:37 +00:00
Fred Reimer
7d6f4fdabb chore: bump Go in container images to 1.26.0 (#8135) 2026-03-01 06:53:33 +00:00
Ruben Fiszel
7a32abec96 feat: slow stream warnings, batch size control, and fix result/skipped filters (#8154)
- Show recurring toast every 15s (8s duration) when loading takes long, with stop button
- When streaming by batches of 25 and a batch takes >4s, offer to stream 1 by 1
- Expose batch size in progress bar with editable input to customize on the fly
- Make stop button more prominent (destructive Button component)
- Fix list_jobs UNION: exclude queue jobs when filtering by result or is_skipped=true
- Add "Show skipped" preset to runs filter

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 06:39:24 +00:00
Ruben Fiszel
4f5a804091 perf: batch large job list requests and fix loadExtraJobs cursor (#8151)
* perf: batch large job list requests and fix loadExtraJobs cursor

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: replace timeout toast with batch progress banner for large job lists

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: show loading indicator on Load more buttons

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: distinguish load-more vs auto-refresh loading indicators

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: offer to stream by batches of 25 when loading is slow

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove refreshing text on auto-refresh and clean up unused loading prop

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: batch progress race condition when restreaming with small batches

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 22:57:06 +00:00
Ruben Fiszel
faf190f12d fix: sync flow on_behalf_of_email on load (#8149) 2026-02-28 22:30:15 +00:00
Ruben Fiszel
86182ed2e9 fix: validate tarball URL host against registry to prevent SSRF and token exfiltration (#8153)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-28 22:03:19 +00:00
Ruben Fiszel
7f6e9fec0c bun-types 2026-02-28 21:47:41 +00:00
Ruben Fiszel
13daebf88a fix: restore email domain (MX) setting in instance settings UI (#8152)
The email_domain setting was accidentally removed from the frontend
instance settings in a recent onboarding cleanup. The backend still
fully supports it. This restores the setting in the Core section.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 21:46:01 +00:00
Ruben Fiszel
c98db016b6 nit claude settings 2026-02-28 21:39:15 +00:00
Ruben Fiszel
d4673c2e91 fix: add partial index for fast failure filtering on runs page (#8150)
When failures are sparse (<1%), filtering by failure status on the runs
page required scanning millions of success rows. Add a partial index on
v2_job_completed (workspace_id, completed_at DESC) WHERE status IN
('failure', 'canceled') and switch ORDER BY to completed_at when
filtering failures, so Postgres walks the small partial index directly.

Benchmarked at 5.2M rows / 1% failure rate:
- LIMIT 30:   800ms -> 0.4ms (2000x faster)
- LIMIT 1000: 550ms -> 21ms  (26x faster)

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 18:18:34 +00:00
Ruben Fiszel
59e51ac097 nit workmux cli 2026-02-28 18:06:39 +00:00
Ruben Fiszel
278983c4fd fix: process deletes before adds in CLI sync push to avoid conflicts (#8148)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 18:05:33 +00:00
Ruben Fiszel
d933446a9e .npmrc nit 2026-02-28 09:16:55 +00:00
Ruben Fiszel
ba48d70157 perf: lazy-load heavy deps (graphql, openapi-parser, sha256) (#8145)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 09:11:00 +00:00
Ruben Fiszel
cd2cf0c39e copy .npmrc in Dockerfiles so npm ci resolves legacy-peer-deps (#8146)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 09:08:03 +00:00
Ruben Fiszel
bd9ff03010 perf: lazy-load markdown in Tooltip components (#8143)
* perf: lazy-load markdown in Tooltip to reduce stores2 chunk by 335KB

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: migrate TooltipInner to Svelte 5 runes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* perf: remove markdown rendering from Tooltip components

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: use HTML tables for date format tooltips to preserve formatting

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 08:52:10 +00:00
Ruben Fiszel
c424b1a961 chore: update vite to 8, vite-plugin-svelte to 7 (#8141)
* chore: update vite to 8 beta, vite-plugin-svelte to 7, vitest to 4.1 beta

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: add .npmrc with legacy-peer-deps for vite 8 beta

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 07:50:31 +00:00
Ruben Fiszel
0776de6b21 fix: copy deps and remove user auto-add on workspace fork (#8142)
* fix: copy deps and remove user auto-add on workspace fork

Clone workspace_dependencies to forked workspaces and remove
automatic workspace_invite creation for parent workspace users.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update sqlx offline cache

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 07:39:27 +00:00
Diego Imbert
762fd3d993 Fix python datatable client requiring explicit types (#8086)
* Support arg type decl in postgres

* Python datatable client no longer requires explicit arg typing

* compilation fix

* Set correct type in statement exec

* reset to main

* Explicit pg arg types

* remove code duplication

* update parser js

* FLOAT8 doesn't have space

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-02-28 07:08:02 +00:00
claude[bot]
83aee49978 add Google triggers doc link in workspace native triggers settings (#8091)
Add docsUrl to the Google service config in WorkspaceIntegrations so a
"Docs" button appears next to the Google integration, linking to
https://www.windmill.dev/docs/core_concepts/native_triggers#google-triggers.
This follows the same pattern already used for Nextcloud.

Closes #8090

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Henri Courdent <122811744+hcourdent@users.noreply.github.com>
2026-02-28 07:04:40 +00:00
Diego Imbert
095505136c fix: Handle CTEs and local tables in SQL asset parser (#8131)
* Handle CTEs and local tables in SQL asset parser

* also handle CREATE VIEW

* Update package regex version
2026-02-28 07:04:19 +00:00
claude[bot]
257734b9ab prevent dropdown from switching to top when less space is available above (#8126) 2026-02-28 07:03:44 +00:00
hugocasa
5d58a87a7f feat: populate baseUrl and userId in Nextcloud resource from OAuth (#8132)
When connecting Nextcloud via workspace integration OAuth, the resource
now includes baseUrl (from OAuth config) and userId (fetched from
Nextcloud OCS API) alongside the token, making it immediately usable
by scripts. Falls back to token + baseUrl if user info fetch fails.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 07:03:15 +00:00
Diego Imbert
b68ff965dd fix: fix custom TS Monaco worker not reloading on file uri change (#8130) 2026-02-28 07:01:23 +00:00
centdix
ff180de4de refactor: slim down claude instructions for lean context and fast iteration (#8136)
* refactor: slim down claude instructions for lean context and fast iteration

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add private and license feature flags to enterprise validation docs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add /refine skill for end-of-session doc evolution

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: remove architecture.md overview doc per research findings

General codebase overviews distract agents and trigger unnecessary
exploration. Keep only operational docs (validation, enterprise).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add autonomous mode doc for bypass permission workflows

Covers: plan-first requirement, tmux pane usage for checking
backend/frontend logs, manual testing via Playwright MCP,
Playwright gotchas, and end-of-task summary expectations.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add mermaid, playwright, and asciinema tools to autonomous mode doc

Claude should use mmdc for diagrams during planning, playwright CLI for
screenshots of frontend changes, and asciinema for terminal recordings
of CLI changes. All attached to the PR.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: use pastebin for screenshot/recording uploads

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review findings

- Remove stale docs/architecture.md reference from /refine skill
- Fix script name: ./update-sqlx -> ./update_sqlx.sh
- Remove .claude/settings.local.json mention from enterprise doc

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 07:00:19 +00:00
centdix
7728475fc9 refactor: rewrite flake.nix for clarity and modularity (#8137)
* refactor: rewrite flake.nix from scratch for clarity and modularity

Rewrite the Nix flake with clean separation of concerns, organized
let-bindings, and 4 purpose-specific devShells instead of a monolithic
default shell with broken package outputs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add CLI tools to default devShell (gh, aws, playwright, mermaid, asciinema)

Add tools needed for AI agent workflows and dev tooling:
- gh (GitHub CLI)
- awscli2
- asciinema (terminal recording)
- playwright-driver with Nix-managed browsers
- mermaid-cli (diagram generation)

Playwright browsers are provided via nixpkgs' playwright-driver.browsers.
Mermaid/Puppeteer reuses the headless_shell from the same browser set.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: move wm-minio scripts to default devShell

MinIO (local S3) is needed for regular development, not just the full
profile.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: use playwright wrapper + chromium for browser tools

Replace playwright-driver (library, no CLI) with:
- A `playwright` wrapper script that calls the Nix playwright-core CLI
  (version-matched to its own Nix-provided browsers)
- pkgs.chromium for Mermaid/Puppeteer (which respects PUPPETEER_EXECUTABLE_PATH)

This fixes playwright screenshot and mermaid diagram generation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: auto-load .env.local from main worktree in all devShells

Gitignored files like .env.local don't exist in git worktrees.
Add a shared shellHook that resolves back to the main tree via
git-common-dir and sources .env.local if present. This ensures
AWS credentials and other secrets are available in worktrees.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: replace deprecated pkgs.hostPlatform with stdenv.hostPlatform

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: remove AWS CLI from flake and sandbox images

Pastebin is sufficient for screenshot sharing; AWS credentials
add unnecessary complexity.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review — ruby mismatch, quoting, shell dedup

- Fix pkgs.ruby → pkgs.ruby_3_4 in extraRuntimeVars to match extraRuntimes
- Replace $* with "$@" in all helper scripts (wm, wm-build, wm-caddy,
  wm-bench, wm-cli) to correctly preserve argument boundaries
- Extract coreBuildInputs, browserVars, and playwrightWrapper as shared
  let-bindings to eliminate duplication between default and full shells

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: remove .env.local auto-loading from devShells

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 06:51:02 +00:00
451 changed files with 15433 additions and 8885 deletions

View File

@@ -0,0 +1,21 @@
#!/usr/bin/env bash
# PreToolUse hook: block destructive git operations when on the main branch.
# Non-git tool calls and read-only git commands pass through silently.
set -euo pipefail
input="$(cat)"
tool_name="$(echo "$input" | jq -r '.tool_name // empty')"
# Only care about Bash tool calls
[[ "$tool_name" == "Bash" ]] || exit 0
command="$(echo "$input" | jq -r '.tool_input.command // empty')"
# Only care about git write commands
if [[ "$command" =~ ^git\ (push|reset|revert|checkout|merge|rebase|commit|add) ]]; then
branch="$(git rev-parse --abbrev-ref HEAD 2>/dev/null || true)"
if [[ "$branch" == "main" ]]; then
echo "BLOCK: You are on the main branch. Create or switch to a feature branch first."
fi
fi

View File

@@ -30,7 +30,15 @@
"Bash(cargo check:*)",
"mcp__ide__getDiagnostics",
"Bash(npm run generate-backend-client:*)",
"Bash(npm run check:*)"
"Bash(npm run check:*)",
"Bash(git push:*)",
"Bash(git reset:*)",
"Bash(git revert:*)",
"Bash(git checkout:*)",
"Bash(git merge:*)",
"Bash(git rebase:*)",
"Bash(git add:*)",
"Bash(git commit:*)"
],
"deny": [
"Read(.env)",
@@ -55,17 +63,23 @@
"Bash(chown:*)",
"Bash(truncate:*)",
"Bash(shred:*)",
"Bash(unlink:*)",
"Bash(git push:*)",
"Bash(git reset:*)",
"Bash(git revert:*)",
"Bash(git checkout:*)",
"Bash(git merge:*)",
"Bash(git rebase:*)"
"Bash(unlink:*)"
]
},
"enableAllProjectMcpServers": true,
"hooks": {
"PreToolUse": [
{
"matcher": "Bash",
"hooks": [
{
"type": "command",
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/guard-main-branch.sh",
"timeout": 5
}
]
}
],
"PostToolUse": [
{
"matcher": "Edit|Write",
@@ -96,7 +110,6 @@
]
},
"enabledPlugins": {
"rust-analyzer-lsp@claude-plugins-official": true,
"typescript-lsp@claude-plugins-official": true,
"code-review@claude-plugins-official": true
}

View File

@@ -0,0 +1,39 @@
---
name: refine
user_invocable: true
description: End-of-session reflection. Reviews friction encountered during the session and proposes updates to docs/ to capture lessons learned.
---
# Refine Skill
Reflect on the current session and update documentation with lessons learned.
## Instructions
1. **Identify friction**: Review what happened in this session:
- Run `git diff main...HEAD --stat` to see what files were touched
- Think about: what was slow, what failed, what required multiple attempts, what information was missing or hard to find
2. **Read current docs**: Read the docs that were relevant to this session:
- `docs/validation.md`
- `docs/enterprise.md`
- `docs/autonomous-mode.md`
- Any skills that were invoked
3. **Propose updates**: For each piece of friction, decide if it warrants a doc update:
- **Missing knowledge**: Information you had to discover that should be documented
- **Wrong guidance**: Instructions that led you astray
- **Missing validation rule**: A check that should be in the validation matrix
- **New pattern**: A codebase pattern worth capturing for next time
4. **Apply updates**: Edit the relevant `docs/` files. Keep changes minimal and specific — add only what would have saved time this session.
5. **Report**: Summarize what was added/changed and why.
## Rules
- Only add knowledge confirmed by this session — no speculative additions
- Keep docs concise — add a line or two, not a paragraph
- If a whole new doc is needed, create it in `docs/` and add a pointer in `CLAUDE.md`
- Don't update skills unless a coding pattern was genuinely wrong
- Don't add things Claude already knows — only Windmill-specific knowledge

View File

@@ -3,493 +3,105 @@ name: rust-backend
description: Rust coding guidelines for the Windmill backend. MUST use when writing or modifying Rust code in the backend directory.
---
# Rust Backend Coding Guidelines
# Windmill Rust Patterns
Apply these patterns when writing or modifying Rust code in the `backend/` directory.
## Data Structure Design
Choose between `struct`, `enum`, or `newtype` based on domain needs:
- Use `enum` for state machines instead of boolean flags or loosely related fields
- Model invariants explicitly using types (e.g., `NonZeroU32`, `Duration`, custom enums)
- Consider ownership of each field:
- Use `&str` vs `String`, slices vs vectors
- Use `Arc<T>` when sharing across threads
- Use `Cow<'a, T>` for flexible ownership
```rust
// State machine with enum
enum JobState {
Pending { scheduled_for: DateTime<Utc> },
Running { started_at: DateTime<Utc>, worker: String },
Completed { result: JobResult, duration_ms: i64 },
Failed { error: String, retries: u32 },
}
// Avoid multiple booleans
struct Job {
is_pending: bool, // Don't do this
is_running: bool,
is_completed: bool,
}
```
## Impl Block Organization
Place `impl` blocks immediately below the struct/enum they modify. Group methods logically:
```rust
struct JobQueue {
jobs: Vec<Job>,
capacity: usize,
}
impl JobQueue {
// Constructors first
pub fn new(capacity: usize) -> Self { ... }
pub fn with_jobs(jobs: Vec<Job>) -> Self { ... }
// Getters
pub fn len(&self) -> usize { ... }
pub fn is_empty(&self) -> bool { ... }
// Mutation methods
pub fn push(&mut self, job: Job) -> Result<()> { ... }
pub fn pop(&mut self) -> Option<Job> { ... }
// Domain logic
pub fn next_scheduled(&self) -> Option<&Job> { ... }
}
```
## Iterator Chains Over For-Loops
Prefer functional iterator chains (`.filter().map().collect()`) over imperative for-loops:
```rust
// Preferred
let results: Vec<_> = items
.iter()
.filter(|item| item.is_valid())
.map(|item| item.transform())
.collect();
// Avoid
let mut results = Vec::new();
for item in items.iter() {
if item.is_valid() {
results.push(item.transform());
}
}
```
Apply these Windmill-specific patterns when writing Rust code in `backend/`.
## Error Handling
Use the `Error` type from `windmill_common::error`. Return `Result<T, Error>` or `JsonResult<T>` for fallible functions:
Use `Error` from `windmill_common::error`. Return `Result<T, Error>` or `JsonResult<T>`:
```rust
use windmill_common::error::{Error, Result};
// Use ? operator for propagation
pub async fn get_job(db: &DB, id: Uuid) -> Result<Job> {
let job = sqlx::query_as!(Job, "SELECT ... WHERE id = $1", id)
sqlx::query_as!(Job, "SELECT id, workspace_id FROM v2_job WHERE id = $1", id)
.fetch_optional(db)
.await?
.ok_or_else(|| Error::NotFound("job not found".to_string()))?;
Ok(job)
}
```
Prefer `if let` for optional handling. Use `let...else` when early return makes code clearer:
Never panic in library code. Reserve `.unwrap()` for compile-time guarantees.
## SQLx Patterns
**Never use `SELECT *`** — always list columns explicitly. Critical for backwards compatibility when workers lag behind API version:
```rust
let Some(config) = get_config() else {
return Err(Error::MissingConfig);
};
// Correct
sqlx::query_as!(Job, "SELECT id, workspace_id, path FROM v2_job WHERE id = $1", id)
// Wrong — breaks when columns are added
sqlx::query_as!(Job, "SELECT * FROM v2_job WHERE id = $1", id)
```
Never panic in library code. Reserve `.unwrap()` for cases with compile-time guarantees. Keep functions short to help lifetime inference and clarity.
## Early Returns
Return early to avoid deep nesting. Handle error cases and edge conditions first:
Use batch operations to avoid N+1:
```rust
// Preferred - early returns
fn process_job(job: Option<Job>) -> Result<Output> {
let Some(job) = job else {
return Ok(Output::default());
};
if !job.is_valid() {
return Err(Error::InvalidJob);
}
if job.is_cached() {
return Ok(job.cached_result());
}
// Main logic at the end, not nested
execute_job(job)
}
// Avoid - deep nesting
fn process_job(job: Option<Job>) -> Result<Output> {
if let Some(job) = job {
if job.is_valid() {
if !job.is_cached() {
execute_job(job)
} else {
Ok(job.cached_result())
}
} else {
Err(Error::InvalidJob)
}
} else {
Ok(Output::default())
}
}
// Preferred — single query with IN clause
sqlx::query!("SELECT ... WHERE id = ANY($1)", &ids[..]).fetch_all(db).await?
```
## Variable Shadowing
Shadow variables instead of creating new names with prefixes:
```rust
// Preferred
let data = fetch_raw_data();
let data = parse(data);
let data = validate(data)?;
// Avoid
let raw_data = fetch_raw_data();
let parsed_data = parse(raw_data);
let validated_data = validate(parsed_data)?;
```
## Minimal Comments
- No inline comments explaining obvious code
- No TODO/FIXME comments in committed code
- Doc comments (`///`) only on public items
- Let code be self-documenting through clear naming
## Type Safety
Use enums over boolean flags for clarity:
```rust
// Preferred
enum JobStatus {
Pending,
Running,
Completed,
}
// Avoid
struct Job {
is_running: bool,
is_completed: bool,
}
```
## Pattern Matching
Prefer explicit matching. Use wildcards strategically for fallback cases or ignored fields:
```rust
// Explicit matching preferred
match status {
JobStatus::Pending => handle_pending(),
JobStatus::Running => handle_running(),
JobStatus::Completed => handle_completed(),
}
// Wildcards OK for fallback
match result {
Ok(value) => process(value),
Err(_) => return default_value(),
}
// Wildcards OK for ignoring fields in destructuring
let Point { x, y, .. } = point;
```
## Destructuring in Function Signatures
Destructure structs directly in function parameters:
```rust
// Preferred
async fn process_job(
Extension(db): Extension<DB>,
Path((workspace, job_id)): Path<(String, Uuid)>,
Query(pagination): Query<Pagination>,
) -> Result<Json<Job>> {
// ...
}
// Avoid
async fn process_job(
db_ext: Extension<DB>,
path: Path<(String, Uuid)>,
query: Query<Pagination>,
) -> Result<Json<Job>> {
let Extension(db) = db_ext;
let Path((workspace, job_id)) = path;
// ...
}
```
## Trait Implementations
Use standard trait implementations to simplify conversions and reduce boilerplate:
```rust
// Implement From/Into for type conversions
impl From<DbJob> for ApiJob {
fn from(db: DbJob) -> Self {
ApiJob {
id: db.id,
status: db.status.into(),
}
}
}
// Use TryFrom for fallible conversions
impl TryFrom<String> for JobKind {
type Error = Error;
fn try_from(s: String) -> Result<Self, Self::Error> { ... }
}
```
Apply `derive` macros to reduce boilerplate:
```rust
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Job { ... }
```
## Module Structure
- Use `pub(crate)` instead of `pub` when possible; expose only what needs exposing
- Keep APIs small and expressive; avoid leaking internal types
- Organize code into modules reflecting ownership and domain boundaries
```rust
// Prefer restricted visibility
pub(crate) fn internal_helper() { ... }
// Only pub for external API
pub fn create_job(...) -> Result<Job> { ... }
```
## Code Navigation
Always use rust-analyzer LSP for:
- Go to definition
- Find references
- Type information
- Import resolution
Do not guess at module paths or type definitions.
Use transactions for multi-step operations. Parameterize all queries.
## JSON Handling
Prefer `Box<serde_json::value::RawValue>` over `serde_json::Value` when:
- Storing JSON in the database (JSONB columns)
- Passing JSON through without modification
- The JSON structure doesn't need inspection
Prefer `Box<serde_json::value::RawValue>` over `serde_json::Value` when storing/passing JSON without inspection:
```rust
// Preferred - avoids parsing/serialization overhead
pub struct Job {
pub id: Uuid,
pub args: Option<Box<serde_json::value::RawValue>>,
}
// Only use Value when you need to inspect/modify JSON
let value: serde_json::Value = serde_json::from_str(&json)?;
if let Some(field) = value.get("field") {
// modify or inspect
}
```
## Serde Optimizations
Only use `serde_json::Value` when you need to inspect or modify the JSON.
Use serde attributes to optimize serialization:
## Serde Optimizations
```rust
#[derive(Serialize, Deserialize)]
pub struct Job {
#[serde(rename = "jobId")]
pub id: Uuid,
#[serde(default)]
pub priority: i32,
#[serde(skip_serializing_if = "Option::is_none")]
pub parent_job: Option<Uuid>,
#[serde(skip_serializing_if = "Vec::is_empty")]
pub tags: Vec<String>,
#[serde(default)]
pub priority: i32,
}
```
Prefer borrowing for zero-copy deserialization when lifetimes allow:
## Async & Concurrency
Never block the async runtime. Use `spawn_blocking` for CPU-intensive work:
```rust
#[derive(Deserialize)]
pub struct JobInput<'a> {
#[serde(borrow)]
pub workspace_id: Cow<'a, str>,
#[serde(borrow)]
pub script_path: &'a str,
}
let result = tokio::task::spawn_blocking(move || expensive_computation(&data)).await?;
```
## SQLx Patterns
**Mutex selection**: Prefer `std::sync::Mutex` (or `parking_lot::Mutex`) for data protection. Only use `tokio::sync::Mutex` when holding locks across `.await` points.
**Never use `SELECT *`** - always list columns explicitly. This is critical for backwards compatibility when workers run behind the API server version:
Use `tokio::sync::mpsc` (bounded) for channels. Avoid `std::thread::sleep` in async contexts.
## Module Structure & Visibility
- Use `pub(crate)` instead of `pub` when possible
- Place new code in the appropriate crate based on functionality
- API endpoints go in `windmill-api/src/` organized by domain
- Shared functionality goes in `windmill-common/src/`
## Code Navigation
Always use rust-analyzer LSP for go-to-definition, find-references, and type info. Do not guess at module paths.
## Axum Handlers
Destructure extractors directly in function signatures:
```rust
// Preferred - explicit columns
sqlx::query_as!(
Job,
"SELECT id, workspace_id, path, created_at FROM v2_job WHERE id = $1",
job_id
)
// Avoid - breaks when columns are added
sqlx::query_as!(Job, "SELECT * FROM v2_job WHERE id = $1", job_id)
async fn process_job(
Extension(db): Extension<DB>,
Path((workspace, job_id)): Path<(String, Uuid)>,
Query(pagination): Query<Pagination>,
) -> Result<Json<Job>> { ... }
```
Use batch operations to minimize round trips:
```rust
// Preferred - single query with multiple values
sqlx::query!(
"INSERT INTO job_logs (job_id, logs) VALUES ($1, $2), ($3, $4)",
id1, log1, id2, log2
)
// Avoid N+1 queries
for id in ids {
sqlx::query!("SELECT ... WHERE id = $1", id).fetch_one(db).await?;
}
// Preferred - single query with IN clause
sqlx::query!("SELECT ... WHERE id = ANY($1)", &ids[..]).fetch_all(db).await?
```
Use transactions for multi-step operations and parameterize all queries.
## Async & Tokio Patterns
Never block the async runtime. Use `spawn_blocking` for CPU-intensive or blocking I/O:
```rust
// Preferred - offload blocking work
let result = tokio::task::spawn_blocking(move || {
expensive_computation(&data)
}).await?;
// Avoid - blocks the runtime
let result = expensive_computation(&data); // Don't do this in async
```
Use tokio primitives for sleep and channels:
```rust
use tokio::sync::mpsc;
use tokio::time::sleep;
// Avoid in async contexts
use std::thread::sleep; // Blocks the runtime
```
Use bounded channels for backpressure:
```rust
// Preferred - bounded channel prevents overwhelming
let (tx, rx) = tokio::sync::mpsc::channel(100);
// Be careful with unbounded
let (tx, rx) = tokio::sync::mpsc::unbounded_channel();
```
## Mutex Selection in Async Code
**Prefer `std::sync::Mutex` (or `parking_lot::Mutex`) over `tokio::sync::Mutex`** for protecting data in async code. The async mutex is more expensive and only needed when holding locks across `.await` points.
```rust
// Preferred for data protection - std mutex is faster
use std::sync::Mutex;
struct Cache {
data: Mutex<HashMap<String, Value>>,
}
impl Cache {
fn get(&self, key: &str) -> Option<Value> {
self.data.lock().unwrap().get(key).cloned()
}
fn insert(&self, key: String, value: Value) {
self.data.lock().unwrap().insert(key, value);
}
}
```
**Use `tokio::sync::Mutex` only when you must hold the lock across `.await` points**, typically for IO resources like database connections:
```rust
use tokio::sync::Mutex;
use std::sync::Arc;
// Async mutex for IO resources held across await points
let conn = Arc::new(Mutex::new(db_connection));
async fn execute_query(conn: Arc<Mutex<DbConn>>, query: &str) {
let mut lock = conn.lock().await;
lock.execute(query).await; // Lock held across .await
}
```
**Common pattern**: Wrap `Arc<Mutex<...>>` in a struct with non-async methods that lock internally, keeping lock scope minimal:
```rust
struct SharedState {
inner: std::sync::Mutex<StateInner>,
}
impl SharedState {
fn update(&self, value: i32) {
self.inner.lock().unwrap().value = value;
}
fn get(&self) -> i32 {
self.inner.lock().unwrap().value
}
}
```
**Alternative for IO resources**: Spawn a dedicated task to manage the resource and communicate via message passing:
```rust
let (tx, mut rx) = tokio::sync::mpsc::channel(32);
tokio::spawn(async move {
while let Some(cmd) = rx.recv().await {
handle_io_command(&mut resource, cmd).await;
}
});
```
## Build & Tooling
Build speed tips:
- Use `cargo check` during rapid iteration over `cargo build`
- Minimize unnecessary dependencies and feature flags

View File

@@ -3,316 +3,78 @@ name: svelte-frontend
description: Svelte coding guidelines for the Windmill frontend. MUST use when writing or modifying code in the frontend directory.
---
# Svelte 5 Best Practices
# Windmill Svelte Patterns
This guide outlines best practices for developing with Svelte 5, incorporating the new Runes API and other modern Svelte features. These rules MUST NOT be applied on svelte 4 files unless explicitly asked to do so.
Apply these Windmill-specific patterns when writing Svelte code in `frontend/`. For general Svelte 5 syntax (runes, snippets, event handling), use the Svelte MCP server.
## Reactivity with Runes
## Windmill UI Components (MUST use)
Svelte 5 introduces Runes for more explicit and flexible reactivity.
Always use Windmill's design-system components. Never use raw HTML elements.
1. **Embrace Runes for State Management**:
* Use `$state` for reactive local component state.
```svelte
<script>
let count = $state(0);
function increment() {
count += 1;
}
</script>
<button onclick={increment}>
Clicked {count} {count === 1 ? 'time' : 'times'}
</button>
```
* Use `$derived` for computed values based on other reactive state.
```svelte
<script>
let count = $state(0);
const doubled = $derived(count * 2);
</script>
<p>{count} * 2 = {doubled}</p>
```
* Use `$effect` for side effects that need to run when reactive values change (e.g., logging, manual DOM manipulation, data fetching). Remember `$effect` does not run on the server.
```svelte
<script>
let count = $state(0);
$effect(() => {
console.log('The count is now', count);
if (count > 5) {
alert('Count is too high!');
}
});
</script>
```
2. **Props with `$props`**:
* Declare component props using `$props()`. This offers better clarity and flexibility compared to `export let`.
```svelte
<script>
// ChildComponent.svelte
let { name, age = $state(30) } = $props();
</script>
<p>Name: {name}</p>
<p>Age: {age}</p>
```
* For bindable props, use `$bindable`.
```svelte
<script>
// MyInput.svelte
let { value = $bindable() } = $props();
</script>
<input bind:value />
```
## Event Handling
* **Use direct event attributes**: Svelte 5 moves away from `on:` directives for DOM events.
* **Do**: `<button onclick={handleClick}>...</button>`
* **Don't**: `<button on:click={handleClick}>...</button>`
* **For component events, prefer callback props**: Instead of `createEventDispatcher`, pass functions as props.
```svelte
<!-- Parent.svelte -->
<script>
import Child from './Child.svelte';
let message = $state('');
function handleChildEvent(detail) {
message = detail;
}
</script>
<Child onCustomEvent={handleChildEvent} />
<p>Message from child: {message}</p>
<!-- Child.svelte -->
<script>
let { onCustomEvent } = $props();
function emitEvent() {
onCustomEvent('Hello from child!');
}
</script>
<button onclick={emitEvent}>Send Event</button>
```
## Snippets for Content Projection
* **Use `{#snippet ...}` and `{@render ...}` instead of slots**: Snippets are more powerful and flexible.
```svelte
<!-- Parent.svelte -->
<script>
import Card from './Card.svelte';
</script>
<Card>
{#snippet title()}
My Awesome Title
{/snippet}
{#snippet content()}
<p>Some interesting content here.</p>
{/snippet}
</Card>
<!-- Card.svelte -->
<script>
let { title, content } = $props();
</script>
<article>
<header>{@render title()}</header>
<div>{@render content()}</div>
</article>
```
* Default content is passed via the `children` prop (which is a snippet).
```svelte
<!-- Wrapper.svelte -->
<script>
let { children } = $props();
</script>
<div>
{@render children?.()}
</div>
```
## Component Design
1. **Create Small, Reusable Components**: Break down complex UIs into smaller, focused components. Each component should have a single responsibility. This also aids performance by limiting the scope of reactivity updates.
2. **Descriptive Naming**: Use clear and descriptive names for variables, functions, and components.
3. **Minimize Logic in Components**: Move complex business logic to utility functions or services. Keep components focused on presentation and interaction.
## State Management (Stores)
1. **Segment Stores**: Avoid a single global store. Create multiple stores, each responsible for a specific piece of global state (e.g., `userStore.js`, `themeStore.js`). This can help limit reactivity updates to only the parts of the UI that depend on specific state segments.
2. **Use Custom Stores for Complex Logic**: For stores with related methods, create custom stores.
```javascript
// counterStore.js
import { writable } from 'svelte/store';
function createCounter() {
const { subscribe, set, update } = writable(0);
return {
subscribe,
increment: () => update(n => n + 1),
decrement: () => update(n => n - 1),
reset: () => set(0)
};
}
export const counter = createCounter();
```
3. **Use Context API for Localized State**: For state shared within a component subtree, consider Svelte's context API (`setContext`, `getContext`) instead of global stores when the state doesn't need to be truly global.
## Performance Optimizations (Svelte 5)
When generating Svelte 5 code, prioritize frontend performance by applying the following principles:
### General Svelte 5 Principles
- **Leverage the Compiler:** Trust Svelte's compiler to generate optimized JavaScript. Avoid manual DOM manipulation (`document.querySelector`, etc.) unless absolutely necessary for integrating third-party libraries that lack Svelte adapters.
- **Keep Components Small and Focused:** Reinforcing from Component Design, smaller components lead to less complex reactivity graphs and more targeted, efficient updates.
### Reactivity & State Management
- **Optimize Computations with `$derived`:** Always use `$derived` for computed values that depend on other state. This ensures the computation only runs when its specific dependencies change, avoiding unnecessary work compared to recomputing derived values in `$effect` or less efficient methods.
- **Minimize `$effect` Usage:** Use `$effect` sparingly and only for true side effects that interact with the outside world or non-Svelte state. Avoid putting complex logic or state updates *within* an `$effect` unless those updates are explicitly intended as a reaction to external changes or non-Svelte state. Excessive or complex effects can impact rendering performance.
- **Structure State for Fine-Grained Updates:** Design your `$state` objects or variables such that updates affect only the necessary parts of the UI. Avoid putting too much unrelated state into a single large object that gets frequently updated, as this can potentially trigger broader updates than necessary. Consider normalizing complex, nested state.
### List Rendering (`{#each}`)
- **Mandate `key` Attribute:** Always use a `key` attribute (`{#each items as item (item.id)}`) that refers to a unique, stable identifier for each item in a list. This is critical for allowing Svelte to efficiently update, reorder, add, or remove list items without destroying and re-creating unnecessary DOM elements and component instances.
### Component Loading & Bundling
- **Implement Lazy Loading/Code Splitting:** For routes, components, or modules that are not immediately needed on page load, use dynamic imports (`import(...)`) to split the code bundle. SvelteKit handles this automatically for routes, but it can be applied manually to components using helper patterns if needed.
- **Be Mindful of Third-Party Libraries:** When incorporating external libraries, import only the necessary functions or components to minimize the final bundle size. Prefer libraries designed to be tree-shakeable.
### Rendering & DOM
- **Use CSS for Animations/Transitions:** Prefer CSS animations or transitions where possible for performance. Svelte's built-in `transition:` directive is also highly optimized and should be used for complex state-driven transitions, but simple cases can often use plain CSS.
- **Optimize Image Loading:** Implement best practices for images: use optimized formats (WebP, AVIF), lazy loading (`loading="lazy"`), and responsive images (`<picture>`, `srcset`) to avoid loading unnecessarily large images.
### Server-Side Rendering (SSR) & Hydration
- **Ensure SSR Compatibility:** Write components that can be rendered on the server for faster initial page loads. Avoid relying on browser-specific APIs (like `window` or `document`) in the main `<script>` context. If necessary, use `$effect` or check `if (browser)` inside effects to run browser-specific code only on the client.
- **Minimize Work During Hydration:** Structure components and data fetching such that minimal complex setup or computation is required when the client-side Svelte code takes over from the server-rendered HTML. Heavy synchronous work during hydration can block the main thread.
## General Clean Code Practices
1. **Organized File Structure**: Group related files together. A common structure:
```
/src
|-- /routes // Page components (if using a router like SvelteKit)
|-- /lib // Utility functions, services, constants (SvelteKit often uses this)
| |-- /stores
| |-- /utils
| |-- /services
| |-- /components // Reusable UI components
|-- App.svelte
|-- main.js (or main.ts)
```
2. **Scoped Styles**: Keep CSS scoped to components to avoid unintended side effects and improve maintainability. Avoid `:global` where possible.
3. **Immutability**: With Svelte 5 and `$state`, direct assignments to properties of `$state` objects (`obj.prop = value;`) are generally fine as Svelte's reactivity system handles updates. However, for non-rune state or when interacting with other systems, understanding and sometimes preferring immutable updates (creating new objects/arrays) can still be relevant.
4. **Use `class:` and `style:` directives**: For dynamic classes and styles, use Svelte's built-in directives for cleaner templates and potentially optimized updates.
```svelte
<script>
let isActive = $state(true);
let color = $state('blue');
</script>
<div class:active={isActive} style:color={color}>
Hello
</div>
```
5. **Stay Updated**: Keep Svelte and its related packages up to date to benefit from the latest features, performance improvements, and security fixes.
## Windmill UI Component Rules (MUST follow)
Always use Windmill's own design-system components instead of raw HTML elements. Using raw HTML elements produces inconsistent styling and breaks the design language.
### Icons — use `lucide-svelte`
**Never** write inline SVGs. Import icons from `lucide-svelte`.
```svelte
<script>
import { ChevronLeft, ChevronRight, X } from 'lucide-svelte'
</script>
<ChevronLeft size={16} />
```
### Buttons — use `<Button>`
**Never** use `<button>`. Import and use `Button` from `$lib/components/common`.
### Buttons — `<Button>`
```svelte
<script>
import { Button } from '$lib/components/common'
import { ChevronLeft, ChevronRight } from 'lucide-svelte'
import { ChevronLeft } from 'lucide-svelte'
</script>
<!-- Regular button -->
<Button variant="default" onclick={handleClick}>Label</Button>
<!-- Icon-only button (no label) -->
<Button startIcon={{ icon: ChevronLeft }} iconOnly onclick={prevMonth} />
<Button startIcon={{ icon: ChevronRight }} iconOnly onclick={nextMonth} />
<Button startIcon={{ icon: ChevronLeft }} iconOnly onclick={prev} />
```
Key `Button` props:
- `variant?: 'accent' | 'accent-secondary' | 'default' | 'subtle'`
- `unifiedSize?: 'sm' | 'md' | 'lg'`
- `startIcon?: { icon: SvelteComponent }` — renders an icon before the label
- `iconOnly?: boolean` — renders icon with no surrounding label text
- `disabled?: boolean`
Props: `variant?: 'accent' | 'accent-secondary' | 'default' | 'subtle'`, `unifiedSize?: 'sm' | 'md' | 'lg'`, `startIcon?: { icon: SvelteComponent }`, `iconOnly?: boolean`, `disabled?: boolean`
### Text inputs — use `<TextInput>`
**Never** use `<input>`. Import and use `TextInput` from `$lib/components/common`.
### Text inputs — `<TextInput>`
```svelte
<script>
import { TextInput } from '$lib/components/common'
let val = $state('')
</script>
<TextInput bind:value={val} placeholder="Enter value" />
```
Key `TextInput` props:
- `value?: string | number` (bindable)
- `placeholder?: string`
- `disabled?: boolean`
- `error?: string | boolean`
- `size?: 'sm' | 'md' | 'lg'`
- `inputProps?` — forwarded to the underlying `<input>`
Props: `value?: string | number` (bindable), `placeholder?: string`, `disabled?: boolean`, `error?: string | boolean`, `size?: 'sm' | 'md' | 'lg'`
### Selects — use `<Select>`
**Never** use `<select>`. Import and use `Select` from `$lib/components/select/Select.svelte`.
### Selects — `<Select>`
```svelte
<script>
import Select from '$lib/components/select/Select.svelte'
const monthItems = [
{ label: 'January', value: 1 },
{ label: 'February', value: 2 },
// ...
]
let selectedMonth = $state(1)
</script>
<Select items={monthItems} bind:value={selectedMonth} />
<Select items={[{ label: 'Jan', value: 1 }]} bind:value={selected} />
```
Key `Select` props:
- `items?: Array<{ label?: string; value: any; subtitle?: string; disabled?: boolean }>`
- `value` (bindable) — the currently selected `.value`
- `placeholder?: string`
- `clearable?: boolean`
- `disabled?: boolean`
- `size?: 'sm' | 'md' | 'lg'`
Props: `items?: Array<{ label?: string; value: any }>`, `value` (bindable), `placeholder?: string`, `clearable?: boolean`, `size?: 'sm' | 'md' | 'lg'`
### Icons — `lucide-svelte`
Never write inline SVGs. Import from `lucide-svelte`:
```svelte
<script>
import { ChevronLeft, X } from 'lucide-svelte'
</script>
<ChevronLeft size={16} />
```
## Form Components
Form components (TextInput, Toggle, Select, etc.) should use the unified size system when placed together.
## Styling
- Use Tailwind CSS for all styling — no custom CSS
- Use Windmill's theming classes for colors/surfaces (see `frontend/brand-guidelines.md`)
- Read component props JSDoc before using them
## Svelte MCP Server
Use the Svelte MCP tools when working on Svelte code:
1. **list-sections**: Call first to discover available docs
2. **get-documentation**: Fetch relevant sections based on use_cases
3. **svelte-autofixer**: MUST use on all Svelte code before finalizing — keep calling until no issues
4. **playground-link**: Only after user confirms and code was NOT written to project files

View File

@@ -42,7 +42,7 @@ RUN wget https://www.python.org/ftp/python/${PYTHON_VERSION}/Python-${PYTHON_VER
RUN /usr/local/bin/python3 -m pip install pip-tools
# Bun
COPY --from=oven/bun:1.3.8 /usr/local/bin/bun /usr/bin/bun
COPY --from=oven/bun:1.3.10 /usr/local/bin/bun /usr/bin/bun
# Install windmill CLI
RUN bun install -g windmill-cli \

View File

@@ -15,11 +15,8 @@ sed -i '' -e "/\"version\": /s/: .*,/: \"$VERSION\",/" ${root_dirpath}/typescrip
sed -i '' -e "/\"version\": /s/: .*,/: \"$VERSION\",/" ${root_dirpath}/frontend/package.json
sed -i '' -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/python-client/wmill/pyproject.toml
sed -i '' -e "/^windmill-api =/s/= .*/= \"\\^$VERSION\"/" ${root_dirpath}/python-client/wmill/pyproject.toml
sed -i '' -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/python-client/wmill_pg/pyproject.toml
sed -i '' -e "/^[[:space:]]*ModuleVersion[[:space:]]*=/s/= .*/= '$VERSION'/" ${root_dirpath}/powershell-client/WindmillClient/WindmillClient.psd1
# sed -i '' -e "/^wmill =/s/= .*/= \"\\^$VERSION\"/" python-client/wmill_pg/pyproject.toml
sed -i '' -e "/^wmill =/s/= .*/= \">=$VERSION\"/" ${root_dirpath}/lsp/Pipfile
sed -i '' -e "/^wmill_pg =/s/= .*/= \">=$VERSION\"/" ${root_dirpath}/lsp/Pipfile
sed -i '' -E "s/name = \"windmill\"\nversion = \"[^\"]*\"\\n(.*)/name = \"windmill\"\nversion = \"$VERSION\"\\n\\1/" ${root_dirpath}/backend/Cargo.lock

View File

@@ -16,11 +16,8 @@ sed -i -e "/\"version\": /s/: .*,/: \"$VERSION\",/" ${root_dirpath}/typescript-c
sed -i -e "/\"version\": /s/: .*,/: \"$VERSION\",/" ${root_dirpath}/frontend/package.json
sed -i -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/python-client/wmill/pyproject.toml
sed -i -e "/^windmill-api =/s/= .*/= \"\\^$VERSION\"/" ${root_dirpath}/python-client/wmill/pyproject.toml
sed -i -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/python-client/wmill_pg/pyproject.toml
sed -i -e "/^[[:space:]]*ModuleVersion[[:space:]]*=/s/= .*/= '$VERSION'/" ${root_dirpath}/powershell-client/WindmillClient/WindmillClient.psd1
# sed -i -e "/^wmill =/s/= .*/= \"\\^$VERSION\"/" ${root_dirpath}/python-client/wmill_pg/pyproject.toml
sed -i -e "/^wmill =/s/= .*/= \">=$VERSION\"/" ${root_dirpath}/lsp/Pipfile
sed -i -e "/^wmill_pg =/s/= .*/= \">=$VERSION\"/" ${root_dirpath}/lsp/Pipfile
sed -i -zE "s/name = \"windmill\"\nversion = \"[^\"]*\"\\n(.*)/name = \"windmill\"\nversion = \"$VERSION\"\\n\\1/" ${root_dirpath}/backend/Cargo.lock

View File

@@ -31,9 +31,3 @@ updates:
directory: "/python-client/wmill"
schedule:
interval: "weekly"
# Maintain dependencies for wmill_pg python client
- package-ecosystem: "pip"
directory: "/python-client/wmill_pg"
schedule:
interval: "weekly"

View File

@@ -0,0 +1,165 @@
name: Backend integration tests (Windows)
on:
workflow_dispatch:
push:
branches:
- "ci-windows-tests"
env:
CARGO_INCREMENTAL: 0
SQLX_OFFLINE: true
DISABLE_EMBEDDING: true
jobs:
cargo_test_windows:
runs-on: blacksmith-16vcpu-windows-2025
steps:
- uses: actions/checkout@v4
- name: Read EE repo commit hash
shell: pwsh
run: |
$ee_repo_ref = Get-Content .\backend\ee-repo-ref.txt
echo "ee_repo_ref=$ee_repo_ref" | Out-File -FilePath $env:GITHUB_ENV -Append
- name: Checkout windmill-ee-private repository
uses: actions/checkout@v4
with:
repository: windmill-labs/windmill-ee-private
path: ./windmill-ee-private
ref: ${{ env.ee_repo_ref }}
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
fetch-depth: 0
- name: Substitute EE code
shell: bash
run: |
./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
- name: Setup PostgreSQL
uses: ikalnytskyi/action-setup-postgres@v6
with:
username: postgres
password: changeme
database: windmill
port: 5432
- uses: actions-rust-lang/setup-rust-toolchain@v1
with:
cache-workspaces: backend
toolchain: 1.93.0
- uses: actions/setup-dotnet@v4
with:
dotnet-version: "9.0.x"
- uses: denoland/setup-deno@v2
with:
deno-version: v2.x
- uses: actions/setup-go@v2
with:
go-version: 1.21.5
- uses: oven-sh/setup-bun@v2
with:
bun-version: 1.3.10
- uses: actions/setup-node@v4
with:
node-version: "20"
- uses: astral-sh/setup-uv@v6.2.1
with:
version: "0.9.24"
- uses: shivammathur/setup-php@v2
with:
php-version: "8.3"
tools: composer
- name: Install windmill CLI
shell: bash
run: |
cd cli
bash gen_wm_client.sh
bun install
mkdir -p "$HOME/.local/bin"
printf '#!/bin/sh\nexec bun run "%s/cli/src/main.ts" "$@"\n' "$GITHUB_WORKSPACE" > "$HOME/.local/bin/wmill"
chmod +x "$HOME/.local/bin/wmill"
echo "$HOME/.local/bin" >> $GITHUB_PATH
- name: Install OpenSSL via vcpkg
run: |
vcpkg.exe install openssl-windows:x64-windows
vcpkg.exe install openssl:x64-windows-static
vcpkg.exe integrate install
- name: Get runtime paths
id: runtime-paths
shell: pwsh
run: |
echo "DENO_PATH=$($(Get-Command deno).Source)" >> $env:GITHUB_OUTPUT
echo "BUN_PATH=$($(Get-Command bun).Source)" >> $env:GITHUB_OUTPUT
echo "NODE_BIN_PATH=$($(Get-Command node).Source)" >> $env:GITHUB_OUTPUT
echo "GO_PATH=$($(Get-Command go).Source)" >> $env:GITHUB_OUTPUT
echo "UV_PATH=$($(Get-Command uv).Source)" >> $env:GITHUB_OUTPUT
echo "PHP_PATH=$($(Get-Command php).Source)" >> $env:GITHUB_OUTPUT
echo "COMPOSER_PATH=$($(Get-Command composer).Source)" >> $env:GITHUB_OUTPUT
echo "POWERSHELL_PATH=$($(Get-Command pwsh).Source)" >> $env:GITHUB_OUTPUT
echo "DOTNET_PATH=$($(Get-Command dotnet).Source)" >> $env:GITHUB_OUTPUT
- name: Build DuckDB FFI module
working-directory: backend/windmill-duckdb-ffi-internal
timeout-minutes: 30
run: |
cargo build --release -p windmill_duckdb_ffi_internal
New-Item -ItemType Directory -Path ..\target\debug -Force
Copy-Item target\release\windmill_duckdb_ffi_internal.dll ..\target\debug\
- name: Print runtime versions and env
shell: pwsh
run: |
deno --version
bun -v
node --version
go version
python3 --version
php --version
pwsh --version
dotnet --version
echo "TEMP=$env:TEMP"
echo "TMP=$env:TMP"
echo "USERPROFILE=$env:USERPROFILE"
echo "HOME=$env:HOME"
- name: cargo test
working-directory: backend
timeout-minutes: 60
env:
DATABASE_URL: postgres://postgres:changeme@localhost:5432/windmill
RUST_LOG: "off"
RUST_LOG_STYLE: never
CARGO_NET_GIT_FETCH_WITH_CLI: true
CARGO_BUILD_JOBS: 12
VCPKGRS_DYNAMIC: 1
OPENSSL_DIR: ${{ env.VCPKG_INSTALLATION_ROOT }}\installed\x64-windows-static
DENO_PATH: ${{ steps.runtime-paths.outputs.DENO_PATH }}
BUN_PATH: ${{ steps.runtime-paths.outputs.BUN_PATH }}
NODE_BIN_PATH: ${{ steps.runtime-paths.outputs.NODE_BIN_PATH }}
GO_PATH: ${{ steps.runtime-paths.outputs.GO_PATH }}
UV_PATH: ${{ steps.runtime-paths.outputs.UV_PATH }}
PHP_PATH: ${{ steps.runtime-paths.outputs.PHP_PATH }}
COMPOSER_PATH: ${{ steps.runtime-paths.outputs.COMPOSER_PATH }}
POWERSHELL_PATH: ${{ steps.runtime-paths.outputs.POWERSHELL_PATH }}
DOTNET_PATH: ${{ steps.runtime-paths.outputs.DOTNET_PATH }}
WMDEBUG_FORCE_V0_WORKSPACE_DEPENDENCIES: 1
WMDEBUG_FORCE_RUNNABLE_SETTINGS_V0: 1
WMDEBUG_FORCE_NO_LEGACY_DEBOUNCING_COMPAT: 1
run: >
cargo test
--no-fail-fast
--features enterprise,deno_core,duckdb,license,python,rust,scoped_cache,parquet,private,csharp,php,quickjs,mcp,run_inline
--all
-- --nocapture --test-threads=10

View File

@@ -55,7 +55,7 @@ jobs:
go-version: 1.21.5
- uses: oven-sh/setup-bun@v2
with:
bun-version: 1.3.8
bun-version: 1.3.10
- uses: actions/setup-node@v4
with:
node-version: "20"

View File

@@ -4,13 +4,13 @@ on:
push:
branches: [main]
paths:
- 'cli/**'
- '.github/workflows/cli-tests.yml'
- "cli/**"
- ".github/workflows/cli-tests.yml"
pull_request:
branches: [main]
paths:
- 'cli/**'
- '.github/workflows/cli-tests.yml'
- "cli/**"
- ".github/workflows/cli-tests.yml"
env:
CARGO_TERM_COLOR: always
@@ -26,7 +26,7 @@ jobs:
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
node-version: "20"
- name: Setup Bun
uses: oven-sh/setup-bun@v2
@@ -72,7 +72,7 @@ jobs:
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
node-version: "20"
- name: Setup Bun
uses: oven-sh/setup-bun@v2
@@ -126,7 +126,7 @@ jobs:
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
node-version: "20"
- name: Setup Bun
uses: oven-sh/setup-bun@v2
@@ -163,11 +163,6 @@ jobs:
NODE_BIN_PATH: ${{ steps.runtime-paths.outputs.NODE_BIN_PATH }}
run: bun test --timeout 120000 test/
- name: Keep runner alive for SSH debug
if: failure()
shell: pwsh
run: Start-Sleep -Seconds 3600
# Combined summary job for branch protection
test-summary:
runs-on: ubuntu-latest

View File

@@ -1,3 +1,10 @@
name: Windmill
startupEnvs:
CARGO_FEATURES: "quickjs"
WM_CLONE_DB: false
USE_RUST_PLUGIN: false
services:
- name: BE
portEnv: BACKEND_PORT
@@ -53,12 +60,11 @@ profiles:
--endpoint-url "$(printenv R2_ENDPOINT)"
3) The public URL will be:
$(printenv R2_PUBLIC_URL)/<branch>/screenshot.png
4) Include screenshots in PR descriptions as markdown images:
![description]($(printenv R2_PUBLIC_URL)/<branch>/screenshot.png)
4) Include in PR descriptions using markdown image syntax.
--- Terminal Recordings (asciinema) ---
You can record terminal sessions and upload them for sharing.
asciinema is pre-installed at /usr/local/bin/asciinema.
asciinema is available on PATH.
1) Write a shell script with the commands to demo. Add sleep
delays for readable pacing:
@@ -98,8 +104,9 @@ profiles:
4) The public URL will be:
$(printenv R2_PUBLIC_URL)/<branch>/diagram.svg
5) Include in PR descriptions as markdown images:
![description]($(printenv R2_PUBLIC_URL)/<branch>/diagram.svg)
5) Include in PR descriptions using markdown image syntax.
IMPORTANT: Read docs/autonomous-mode.md before starting any work.
linkedRepos:
- repo: windmill-labs/windmill-ee-private

View File

@@ -55,7 +55,8 @@ panes:
- Pane 2: frontend (npm run dev)\n\n
To check logs, use: \`tmux capture-pane -t .1 -p -S -50\` (backend) or \`tmux capture-pane -t .2 -p -S -50\` (frontend).\n
When restarting backend or frontend, make sure to use the ports listed in .env.local.\n
Because we are running backend with cargo watch, to verify your changes, just check the logs in the backend pane. No need for cargo check."
Because we are running backend with cargo watch, to verify your changes, just check the logs in the backend pane. No need for cargo check.\n\n
IMPORTANT: Read docs/autonomous-mode.md before starting any work."
focus: true
- command: 'ROOT="$(git rev-parse --show-toplevel)"; [ -f "$ROOT/.env.local" ] && source "$ROOT/.env.local"; cd "$ROOT/backend" && PORT=${BACKEND_PORT:-8000} cargo watch -x "run ${CARGO_FEATURES:+--features $CARGO_FEATURES}"'
split: horizontal

View File

@@ -1,5 +1,133 @@
# Changelog
## [1.651.1](https://github.com/windmill-labs/windmill/compare/v1.651.0...v1.651.1) (2026-03-05)
### Bug Fixes
* prevent slow loading toast interval from leaking on promise cancellation ([#8240](https://github.com/windmill-labs/windmill/issues/8240)) ([2e582b1](https://github.com/windmill-labs/windmill/commit/2e582b1bc1c299388a3c97cfddff9d0eb92858f2))
* suppress unused variable warnings on windows builds ([#8241](https://github.com/windmill-labs/windmill/issues/8241)) ([2d58382](https://github.com/windmill-labs/windmill/commit/2d583826dc065c05684d4cd1d1510f0d1f2d9ae9))
## [1.651.0](https://github.com/windmill-labs/windmill/compare/v1.650.0...v1.651.0) (2026-03-05)
### Features
* add sandbox annotations, volume mounts, for AI sandbox starting with claude ([#8058](https://github.com/windmill-labs/windmill/issues/8058)) ([5f0ef93](https://github.com/windmill-labs/windmill/commit/5f0ef936d1d5d07d01c8e07e26ec254feebef8fb))
* hash-based MCP tool names for long paths ([#8133](https://github.com/windmill-labs/windmill/issues/8133)) ([ce041e8](https://github.com/windmill-labs/windmill/commit/ce041e8a5e7ff105df389875d9981f3843d4ce39))
### Bug Fixes
* **python-client:** add delete_s3_object ([#8216](https://github.com/windmill-labs/windmill/issues/8216)) ([90f4c64](https://github.com/windmill-labs/windmill/commit/90f4c64ee12e1d04ce846ff88d6658f667e194e0))
* update CLI bun template to match UI template ([#8238](https://github.com/windmill-labs/windmill/issues/8238)) ([a8cbe93](https://github.com/windmill-labs/windmill/commit/a8cbe9396ffc51140dce5582d57f4dc59873304e))
* write fallback package.json for codebase mode nsjail ([#8239](https://github.com/windmill-labs/windmill/issues/8239)) ([d46913b](https://github.com/windmill-labs/windmill/commit/d46913b74a0ffd41d2323e0355cc81954f09e29d))
## [1.650.0](https://github.com/windmill-labs/windmill/compare/v1.649.0...v1.650.0) (2026-03-05)
### Features
* add move, delete, and duplicate to flow node context menu ([#8050](https://github.com/windmill-labs/windmill/issues/8050)) ([c0c9388](https://github.com/windmill-labs/windmill/commit/c0c9388415716ce77d841bd08a46f94e0a529685))
* add variable and resource types to flow env variables ([#8214](https://github.com/windmill-labs/windmill/issues/8214)) ([164e499](https://github.com/windmill-labs/windmill/commit/164e499c64dc5eb76fcfb0f8cefbad2df244f610))
* Ducklake typechecker ([#8118](https://github.com/windmill-labs/windmill/issues/8118)) ([53caecf](https://github.com/windmill-labs/windmill/commit/53caecf1da8d76e246178dfb9b86d330f0ec52fd))
* make WINDMILL_DIR configurable via environment variable ([#8215](https://github.com/windmill-labs/windmill/issues/8215)) ([424ca59](https://github.com/windmill-labs/windmill/commit/424ca59dfe3e730f5388d9cac4ea7e69773614d3))
* make WM_END_USER_EMAIL display users from different workspaces ([#8208](https://github.com/windmill-labs/windmill/issues/8208)) ([baf2bcf](https://github.com/windmill-labs/windmill/commit/baf2bcf14da0c8c95bdbbf511fcaee48be33948b))
* persistent Db manager state in URI ([#8134](https://github.com/windmill-labs/windmill/issues/8134)) ([4bf827b](https://github.com/windmill-labs/windmill/commit/4bf827bea4d44aca8c5ff7aa67ad449dbcf00673))
* replace hub error toasts with warning alerts and add disable hub setting ([#8225](https://github.com/windmill-labs/windmill/issues/8225)) ([63ebae8](https://github.com/windmill-labs/windmill/commit/63ebae8829a6dc47a4e23c8670b514f042c9d4be))
* token expiration notifications ([#8190](https://github.com/windmill-labs/windmill/issues/8190)) ([e56ccd2](https://github.com/windmill-labs/windmill/commit/e56ccd200be29e6ac8ea2b04a341b1ce78a307f6))
### Bug Fixes
* handle multipart stream errors gracefully instead of panicking ([#8226](https://github.com/windmill-labs/windmill/issues/8226)) ([19c065b](https://github.com/windmill-labs/windmill/commit/19c065bed5468c484c8e7a50a6b79ab90153cc0e))
* improve windows compatibility ([077779e](https://github.com/windmill-labs/windmill/commit/077779ec52f7d3e5fcc93951544bf47bd6dc30b6))
* wrap set_encryption_key in a single database transaction ([#8212](https://github.com/windmill-labs/windmill/issues/8212)) ([62382fd](https://github.com/windmill-labs/windmill/commit/62382fd2869ea0190dd0c0b714f9cbd35ceddd7a))
## [1.649.0](https://github.com/windmill-labs/windmill/compare/v1.648.0...v1.649.0) (2026-03-03)
### Features
* **frontend:** add script recorder for offline replay ([#8200](https://github.com/windmill-labs/windmill/issues/8200)) ([c97d8b4](https://github.com/windmill-labs/windmill/commit/c97d8b4715f86ea83ab2c0223ba859ced690829a))
* move index management out of /srch/, add storage size reporting ([#8169](https://github.com/windmill-labs/windmill/issues/8169)) ([ee01acd](https://github.com/windmill-labs/windmill/commit/ee01acd9a6a2cd68a3f226988bfb46f6a6e64c08))
### Bug Fixes
* clean up slow-load toast interval on component destroy ([#8207](https://github.com/windmill-labs/windmill/issues/8207)) ([26f4f2b](https://github.com/windmill-labs/windmill/commit/26f4f2b399b828185b553289d6560e12261030a3))
* **frontend:** prevent subflow expansion from hiding all insertion points ([#8203](https://github.com/windmill-labs/windmill/issues/8203)) ([e97da86](https://github.com/windmill-labs/windmill/commit/e97da860672171e33054a77d71f4824bb09e540d))
* gracefully handle malformed OAuth entries in instance config ([#8205](https://github.com/windmill-labs/windmill/issues/8205)) ([cac4bdd](https://github.com/windmill-labs/windmill/commit/cac4bdd54f0c3ea80844ac31f7597f418ff7d8ae))
* skip stop_after_if evaluation for skipped (identity) flow steps ([#8201](https://github.com/windmill-labs/windmill/issues/8201)) ([e6f7775](https://github.com/windmill-labs/windmill/commit/e6f7775d4d9a052aefc37260c6ed161146841cd7))
* use exact matching for python requirements directive parsing ([#8199](https://github.com/windmill-labs/windmill/issues/8199)) ([2b2be38](https://github.com/windmill-labs/windmill/commit/2b2be38f129bbe58b6bb3815c4bd94aa03a3da90))
### Performance Improvements
* use two-step query in input history to leverage v2_job index ([#8197](https://github.com/windmill-labs/windmill/issues/8197)) ([50defdd](https://github.com/windmill-labs/windmill/commit/50defdded113b4d2cf0991b3fb642d1cd9a462b7))
## [1.648.0](https://github.com/windmill-labs/windmill/compare/v1.647.2...v1.648.0) (2026-03-02)
### Features
* add right-click context menu to ObjectViewer ([#8181](https://github.com/windmill-labs/windmill/issues/8181)) ([1855204](https://github.com/windmill-labs/windmill/commit/18552046c29878b5cf115b9364c2ce829ab7aa59))
* **frontend:** add drag-and-drop node movement in flow editor ([#8076](https://github.com/windmill-labs/windmill/issues/8076)) ([7a5e487](https://github.com/windmill-labs/windmill/commit/7a5e48787860c38aa3589c49ea9a70654d479c8a))
### Bug Fixes
* don't insert underscore after digit in PascalCase to snake_case conversion ([#8184](https://github.com/windmill-labs/windmill/issues/8184)) ([a111653](https://github.com/windmill-labs/windmill/commit/a111653c6d32fd1a3d2f45351eceb8d8d7df6f41))
* **frontend:** preserve keycloak realm url between instance settings saves ([#8189](https://github.com/windmill-labs/windmill/issues/8189)) ([cfd9541](https://github.com/windmill-labs/windmill/commit/cfd9541ab1daf635c7d801cd3a7788db57b98257))
* preserve debouncing settings for post-preprocessing arg accumulation ([#8191](https://github.com/windmill-labs/windmill/issues/8191)) ([9e92445](https://github.com/windmill-labs/windmill/commit/9e92445faed1a10b2406b97562e8df7a5b2dfd76))
## [1.647.2](https://github.com/windmill-labs/windmill/compare/v1.647.1...v1.647.2) (2026-03-02)
### Bug Fixes
* update oracle instant client arm64 download url ([#8179](https://github.com/windmill-labs/windmill/issues/8179)) ([758b35f](https://github.com/windmill-labs/windmill/commit/758b35f8ebbf78e1473a8fd83dbc795d58b23b80))
## [1.647.1](https://github.com/windmill-labs/windmill/compare/v1.647.0...v1.647.1) (2026-03-02)
### Bug Fixes
* add missing display_name and tenant fields to instance config OAuthClient ([#8176](https://github.com/windmill-labs/windmill/issues/8176)) ([db44b8b](https://github.com/windmill-labs/windmill/commit/db44b8be74e1709dbf759dd391bdb3861b3c711b))
* add missing grant_types field to instance config OAuth structs ([#8175](https://github.com/windmill-labs/windmill/issues/8175)) ([fca94f8](https://github.com/windmill-labs/windmill/commit/fca94f88dd796db66e0c5bd0225e23b92efce4a7))
* show sync endpoint timeout setting on all instances ([#8170](https://github.com/windmill-labs/windmill/issues/8170)) ([c70307d](https://github.com/windmill-labs/windmill/commit/c70307d3f2dfe61a0250dd12234470a25baf2d1b))
## [1.647.0](https://github.com/windmill-labs/windmill/compare/v1.646.0...v1.647.0) (2026-03-01)
### Features
* populate baseUrl and userId in Nextcloud resource from OAuth ([#8132](https://github.com/windmill-labs/windmill/issues/8132)) ([5d58a87](https://github.com/windmill-labs/windmill/commit/5d58a87a7f02c4f7775bd02c885071495a5f686d))
* runScript inline for path and hash ([#8019](https://github.com/windmill-labs/windmill/issues/8019)) ([7d9d16a](https://github.com/windmill-labs/windmill/commit/7d9d16a6a3357981e5692023982ca1e670acfaae))
* slow stream warnings, batch size control, and fix result/skipped filters ([#8154](https://github.com/windmill-labs/windmill/issues/8154)) ([7a32abe](https://github.com/windmill-labs/windmill/commit/7a32abec96124f96a1dbac11e03162cca68f3286))
### Bug Fixes
* : persist show schedules and show future jobs toggles in local storage ([#8125](https://github.com/windmill-labs/windmill/issues/8125)) ([f1d8568](https://github.com/windmill-labs/windmill/commit/f1d8568831bf69ee790def4f90df8f32c59a94e0)), closes [#8123](https://github.com/windmill-labs/windmill/issues/8123)
* add partial index for fast failure filtering on runs page ([#8150](https://github.com/windmill-labs/windmill/issues/8150)) ([d4673c2](https://github.com/windmill-labs/windmill/commit/d4673c2e91168dcdb0aca9d6c039df0d9c52bb28))
* copy deps and remove user auto-add on workspace fork ([#8142](https://github.com/windmill-labs/windmill/issues/8142)) ([0776de6](https://github.com/windmill-labs/windmill/commit/0776de6b2173075f533fd59a49efb111000da5df))
* fix custom TS Monaco worker not reloading on file uri change ([#8130](https://github.com/windmill-labs/windmill/issues/8130)) ([b68ff96](https://github.com/windmill-labs/windmill/commit/b68ff965dd4f67046fae7e8cf756c8b3e15c2643))
* Handle CTEs and local tables in SQL asset parser ([#8131](https://github.com/windmill-labs/windmill/issues/8131)) ([0955051](https://github.com/windmill-labs/windmill/commit/095505136c2b3e03f656ace20a5c1bbe142fa63f))
* prevent wm-cursor from hanging on stale cursor IPC sockets ([b9e3e05](https://github.com/windmill-labs/windmill/commit/b9e3e053e4914e753bbb806e6b748c791edb92d2))
* process deletes before adds in CLI sync push to avoid conflicts ([#8148](https://github.com/windmill-labs/windmill/issues/8148)) ([278983c](https://github.com/windmill-labs/windmill/commit/278983c4fd38d67a14a8c208178c04db05ee1880))
* remove review comments from discord notifications and support comment edits ([cdc0543](https://github.com/windmill-labs/windmill/commit/cdc0543747680267e30974037a2eb180a19062d9))
* restore email domain (MX) setting in instance settings UI ([#8152](https://github.com/windmill-labs/windmill/issues/8152)) ([13daebf](https://github.com/windmill-labs/windmill/commit/13daebf88ac1abcb833646490073f922ac7c050e))
* sync flow on_behalf_of_email on load ([#8149](https://github.com/windmill-labs/windmill/issues/8149)) ([faf190f](https://github.com/windmill-labs/windmill/commit/faf190f12d96cd75ba9eda10ab3e6f26d2eed813))
* validate tarball URL host against registry to prevent SSRF and token exfiltration ([#8153](https://github.com/windmill-labs/windmill/issues/8153)) ([86182ed](https://github.com/windmill-labs/windmill/commit/86182ed2e999f018fc72343308e7df8e9de6c189))
### Performance Improvements
* batch large job list requests and fix loadExtraJobs cursor ([#8151](https://github.com/windmill-labs/windmill/issues/8151)) ([4f5a804](https://github.com/windmill-labs/windmill/commit/4f5a8040912e18f34401a6e3a95dea6f97d1d24c))
* lazy-load heavy deps (graphql, openapi-parser, sha256) ([#8145](https://github.com/windmill-labs/windmill/issues/8145)) ([ba48d70](https://github.com/windmill-labs/windmill/commit/ba48d7015741eb6bbbe04088a957c37499cd8471))
* lazy-load markdown in Tooltip components ([#8143](https://github.com/windmill-labs/windmill/issues/8143)) ([bd9ff03](https://github.com/windmill-labs/windmill/commit/bd9ff03010f75557dcc315d10e9208b4e9cafece))
## [1.646.0](https://github.com/windmill-labs/windmill/compare/v1.645.0...v1.646.0) (2026-02-26)

View File

@@ -1,68 +1,33 @@
# Windmill Development Guide
# Windmill
## Overview
Open-source platform for internal tools, workflows, API integrations, background jobs, and UIs. Rust backend + Svelte 5 frontend.
Windmill is an open-source developer platform for building internal tools, workflows, API integrations, background jobs, workflows, and user interfaces. See @windmill-overview.mdc for full platform details.
## Workflow
## New Feature Implementation Guidelines
1. **Understand**: Before coding, read relevant docs from `docs/` to understand the area you're changing
2. **Plan**: For non-trivial changes, use plan mode. For large features, break into reviewable stages
3. **Execute**: Follow coding patterns from skills (`rust-backend`, `svelte-frontend`)
4. **Validate**: After every change, run the appropriate checks per `docs/validation.md`
When implementing new features in Windmill, follow these best practices:
## Documentation
- **Clean Code First**: Write clean, readable, and maintainable code. Prioritize clarity over cleverness.
- **Avoid Duplication at All Costs**: Before writing new code, thoroughly search for existing implementations that can be reused or extended.
- **Adapt Existing Code**: Refactor and generalize existing code when necessary to avoid logic duplication. Extract common patterns into reusable utilities.
- **Follow Established Patterns**: Study existing code patterns in the codebase and maintain consistency with established conventions.
- **Single Responsibility**: Each function, component, and module should have a single, well-defined responsibility.
- **Incremental Implementation**: Break large features into smaller, reviewable chunks that can be implemented and tested incrementally.
## Language-Specific Guides
- Backend (Rust): see `backend/CLAUDE.md` and the `rust-backend` skill: `.claude/skills/rust-backend/SKILL.md`
- Frontend (Svelte 5): see `frontend/CLAUDE.md` and the `svelte-frontend` skill: `.claude/skills/svelte-frontend/SKILL.md`
- **Validation**: `docs/validation.md` — what checks to run based on what you changed
- **Enterprise**: `docs/enterprise.md` — EE file conventions and PR workflow
- **Backend patterns**: use the `rust-backend` skill when writing Rust code
- **Frontend patterns**: use the `svelte-frontend` skill when writing Svelte code
- **Domain guides**: `.claude/skills/native-trigger/` and `frontend/tutorial-system-guide.mdc`
- **Brand/UI guidelines**: `frontend/brand-guidelines.md`
## Dev Environment
- **Backend**: `cargo run` from `backend/` (API at http://localhost:8000)
- **Frontend**: `REMOTE=http://localhost:8000 npm run dev` from `frontend/`
- The `REMOTE` env var configures the Vite proxy target. Without it, API calls proxy to `https://app.windmill.dev` instead of the local backend.
- The dev server starts on port 3000 (or 3001+ if 3000 is in use).
- **Default login**: `admin@windmill.dev` / `changeme`
- **Instance settings**: navigate to `/#superadmin-settings` (opens the drawer overlay)
- **Frontend**: `REMOTE=http://localhost:8000 npm run dev` from `frontend/` (port 3000+)
- **DB**: `psql postgres://postgres:changeme@localhost:5432/windmill`
- **Login**: `admin@windmill.dev` / `changeme`
- **Instance settings**: navigate to `/#superadmin-settings`
## UI Testing with Playwright MCP
## Core Principles
When testing the frontend with the Playwright MCP tools:
1. **Start servers**: Launch backend (`cargo run`) and frontend (`REMOTE=http://localhost:8000 npm run dev`) as background tasks
2. **Wait for readiness**: Backend takes ~60s to compile; check output for `health check completed`. Frontend starts in ~5s.
3. **Login flow**: Navigate to `/user/login`, click "Log in without third-party", fill email/password, submit
4. **Instance settings drawer**: Navigate to `/#superadmin-settings` to open the drawer directly
5. **Toggle components**: The YAML toggle uses a custom `<Toggle>` component where the checkbox is visually hidden (`sr-only`). Click the wrapper `<label>` element (the parent container with `cursor=pointer`), not the checkbox ref directly.
6. **Console errors to ignore**: `critical_alerts` 404s are expected on CE builds (EE-only endpoint). VSCode worker 404s are dev-mode artifacts.
## Code Validation (MUST DO)
After making code changes, you MUST run the appropriate checks and fix all errors before considering the work done:
- **Backend**: Run `cargo check` from the `backend/` directory. Only enable the feature flags needed for the code you changed — check `backend/Cargo.toml` `[features]` section to identify which flags gate the crates/modules you modified. For example: `cargo check --features enterprise,parquet` if you only touched enterprise and parquet code.
- **Frontend**: Run `npm run check` from the `frontend/` directory.
## Querying the Database
`backend/summarized_schema.txt` provides a compact overview of all tables, columns, types, ENUMs, and foreign keys. Use it to quickly understand the data model and relationships. Note: this file is a simplified summary — it omits indexes, constraints details, and other metadata.
For exact table definitions (indexes, constraints, column defaults, etc.), query the database directly:
```bash
psql postgres://postgres:changeme@localhost:5432/windmill
```
Useful psql commands:
- `\d <table_name>` — full table definition with indexes and constraints
- `\di <table_name>*` — list indexes for a table
- `\d+ <table_name>` — extended table info including storage and descriptions
This is also helpful for:
- Inspecting database state during development
- Testing queries before implementing them in Rust
- Debugging data-related issues
- Search for existing code to reuse before writing new code
- Follow established patterns in the codebase
- Keep changes focused — don't refactor beyond what's asked

View File

@@ -58,7 +58,7 @@ FROM node:24-alpine as frontend
# install dependencies
WORKDIR /frontend
COPY ./frontend/package.json ./frontend/package-lock.json ./
COPY ./frontend/package.json ./frontend/package-lock.json ./frontend/.npmrc ./
COPY ./frontend/scripts/ ./scripts/
RUN npm ci
@@ -126,7 +126,7 @@ ARG POWERSHELL_DEB_VERSION=7.5.0-1
ARG KUBECTL_VERSION=1.28.7
ARG HELM_VERSION=3.14.3
# NOTE: If changing, also change go version in workspace dependencies template at WorkspaceDependenciesEditor.svelte
ARG GO_VERSION=1.25.0
ARG GO_VERSION=1.26.0
ARG APP=/usr/src/app
ARG WITH_POWERSHELL=true
ARG WITH_KUBECTL=true
@@ -256,12 +256,18 @@ COPY --from=windmill_duckdb_ffi_internal_builder /windmill-duckdb-ffi-internal/t
COPY --from=denoland/deno:2.2.1 --chmod=755 /usr/bin/deno /usr/bin/deno
COPY --from=oven/bun:1.3.8 /usr/local/bin/bun /usr/bin/bun
COPY --from=oven/bun:1.3.10 /usr/local/bin/bun /usr/bin/bun
# Install windmill CLI
RUN bun install -g windmill-cli \
&& ln -s $(bun pm bin -g)/wmill /usr/bin/wmill
# Install Claude Code CLI (used by claude sandbox scripts)
# The installer puts the binary in ~/.local/bin/claude (symlink to ~/.local/share/claude/versions/*)
# Copy it to /usr/bin/claude so it's accessible inside nsjail sandbox (which mounts /usr but not /root)
RUN curl -fsSL https://claude.ai/install.sh | bash \
&& cp /root/.local/share/claude/versions/* /usr/bin/claude
COPY --from=php:8.3.7-cli /usr/local/bin/php /usr/bin/php
COPY --from=composer:2.7.6 /usr/bin/composer /usr/bin/composer

View File

@@ -1,238 +0,0 @@
FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
curl \
ca-certificates \
git \
iptables \
gosu \
sudo \
unzip \
# Rust native build deps (for cargo check)
pkg-config \
cmake \
clang \
mold \
libtool \
libssl-dev \
libxml2-dev \
libxmlsec1-dev \
libxslt1-dev \
libffi-dev \
zlib1g-dev \
libcurl4-openssl-dev \
libclang-dev \
libkrb5-dev \
libsasl2-dev \
# PostgreSQL (for local DB during development)
postgresql \
postgresql-client \
# Node.js 22 (for npm run check / frontend dev)
&& curl -fsSL https://deb.nodesource.com/setup_22.x | bash - \
&& apt-get install -y --no-install-recommends nodejs \
&& rm -rf /var/lib/apt/lists/* \
# Container runs as arbitrary UIDs (--user uid:gid). These three lines make
# sudo work for any UID:
# 1) NOPASSWD rule so sudo never prompts for a password
# 2) Writable passwd/group so the entrypoint can register the dynamic UID
# 3) Writable shadow so unix_chkpwd can validate the account (without this,
# sudo fails with "account validation failure, is your account locked?")
&& echo "ALL ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/sandbox \
&& chmod 0440 /etc/sudoers.d/sandbox \
&& chmod 666 /etc/passwd /etc/group /etc/shadow
# ── GitHub CLI (for PR creation) ──────────────────────────────────────────────
RUN curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg \
-o /usr/share/keyrings/githubcli-archive-keyring.gpg \
&& echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" \
> /etc/apt/sources.list.d/github-cli.list \
&& apt-get update && apt-get install -y --no-install-recommends gh \
&& rm -rf /var/lib/apt/lists/*
# ── Rust toolchain ────────────────────────────────────────────────────────────
# Install under /usr/local/lib/ so bins are world-readable with default umask.
# CARGO_HOME is overridden to /tmp/.cargo at the end for mutable runtime state.
ENV RUSTUP_HOME=/usr/local/lib/rustup CARGO_HOME=/usr/local/lib/cargo
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | \
sh -s -- -y --default-toolchain stable --profile minimal && \
ln -s /usr/local/lib/cargo/bin/* /usr/local/bin/
RUN cargo install sqlx-cli --no-default-features --features native-tls,postgres && \
cargo install cargo-watch && \
cargo install --locked --git https://github.com/asciinema/asciinema && \
ln -sf /usr/local/lib/cargo/bin/sqlx /usr/local/bin/sqlx && \
ln -sf /usr/local/lib/cargo/bin/cargo-watch /usr/local/bin/cargo-watch && \
ln -sf /usr/local/lib/cargo/bin/asciinema /usr/local/bin/asciinema
# ── Register dynamic runtime users ───────────────────────────────────────────
RUN cat <<'SCRIPT' > /usr/local/bin/register-dynamic-user.sh
#!/bin/sh
set -eu
uid="${1:-}"
gid="${2:-}"
if [ -z "$uid" ] || [ -z "$gid" ]; then
echo "register-dynamic-user: usage: register-dynamic-user <uid> <gid>" >&2
exit 1
fi
if ! getent group "$gid" >/dev/null 2>&1; then
echo "sandbox:x:${gid}:" >> /etc/group
fi
if ! getent passwd "$uid" >/dev/null 2>&1; then
echo "sandbox:x:${uid}:${gid}:sandbox:/tmp:/bin/sh" >> /etc/passwd
fi
# Add a shadow entry ("*" = no password) so unix_chkpwd doesn't reject sudo.
if ! grep -q "^sandbox:" /etc/shadow 2>/dev/null; then
echo "sandbox:*:19000:0:99999:7:::" >> /etc/shadow
fi
SCRIPT
RUN chmod +x /usr/local/bin/register-dynamic-user.sh
# ── Network init script (iptables firewall + privilege drop) ──────────────────
RUN cat <<'SCRIPT' > /usr/local/bin/network-init.sh
#!/bin/bash
set -euo pipefail
if [ -n "${WM_PROXY_HOST:-}" ] && [ -n "${WM_PROXY_PORT:-}" ]; then
# Resolve hostnames to ALL IPs (multi-A records, round-robin DNS)
PROXY_IPS=$(getent ahostsv4 "$WM_PROXY_HOST" | awk '{print $1}' | sort -u)
RPC_HOST="${WM_RPC_HOST:-$WM_PROXY_HOST}"
RPC_IPS=$(getent ahostsv4 "$RPC_HOST" | awk '{print $1}' | sort -u)
if [ -z "$PROXY_IPS" ] || [ -z "$RPC_IPS" ]; then
echo "network-init: failed to resolve proxy/RPC host" >&2
exit 1
fi
# IPv4: default deny outbound
iptables -P OUTPUT DROP
iptables -A OUTPUT -o lo -j ACCEPT
iptables -A OUTPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
# Allow DNS (UDP/TCP 53) to configured nameservers.
if [ -f /etc/resolv.conf ]; then
grep '^nameserver' /etc/resolv.conf | awk '{print $2}' | while read -r ns; do
iptables -A OUTPUT -d "$ns" -p udp --dport 53 -j ACCEPT
iptables -A OUTPUT -d "$ns" -p tcp --dport 53 -j ACCEPT
done
fi
# Allow ALL resolved proxy IPs (handles multi-A DNS)
for ip in $PROXY_IPS; do
iptables -A OUTPUT -d "$ip" -p tcp --dport "$WM_PROXY_PORT" -j ACCEPT
done
# Allow ALL resolved RPC IPs
if [ -n "${WM_RPC_PORT:-}" ]; then
for ip in $RPC_IPS; do
iptables -A OUTPUT -d "$ip" -p tcp --dport "$WM_RPC_PORT" -j ACCEPT
done
fi
# Reject (not drop) everything else to fail fast instead of hanging
iptables -A OUTPUT -j REJECT
# IPv6: block entirely to prevent leaks (fail closed)
if ip6tables -L -n >/dev/null 2>&1; then
ip6tables -P OUTPUT DROP
ip6tables -A OUTPUT -o lo -j ACCEPT
ip6tables -A OUTPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
ip6tables -A OUTPUT -j REJECT
else
if ! sysctl -w net.ipv6.conf.all.disable_ipv6=1 2>/dev/null; then
echo "network-init: failed to block IPv6 (neither ip6tables nor sysctl available)" >&2
exit 1
fi
fi
fi
# Add sandbox user/group so sudo works after dropping privileges.
if [ -z "${WM_TARGET_UID:-}" ] || [ -z "${WM_TARGET_GID:-}" ]; then
echo "network-init: WM_TARGET_UID and WM_TARGET_GID are required" >&2
exit 1
fi
/usr/local/bin/register-dynamic-user.sh "${WM_TARGET_UID}" "${WM_TARGET_GID}"
# Fix PTY ownership so the unprivileged user can read/write the terminal.
if [ -t 0 ]; then
chown "${WM_TARGET_UID}:${WM_TARGET_GID}" "$(tty)"
fi
# Drop privileges and exec the user command.
exec gosu "${WM_TARGET_UID}:${WM_TARGET_GID}" env HOME=/tmp "$@"
SCRIPT
RUN chmod +x /usr/local/bin/network-init.sh
# ── workmux (sandbox RPC) ────────────────────────────────────────────────────
RUN curl -fsSL https://raw.githubusercontent.com/raine/workmux/main/scripts/install.sh | bash
# ── Claude Code ───────────────────────────────────────────────────────────────
RUN curl -fsSL https://claude.ai/install.sh | bash && \
target="$(readlink -f /root/.local/bin/claude)" && \
mv /root/.local/share/claude /usr/local/lib/claude && \
ln -s "/usr/local/lib/claude/versions/$(basename "$target")" /usr/local/bin/claude && \
mkdir -p /tmp/.local/bin && \
ln -s /usr/local/bin/claude /tmp/.local/bin/claude && \
chmod -R a+rwX /tmp/.local
# ── Codex ─────────────────────────────────────────────────────────────────────
RUN npm i -g @openai/codex
# ── Bun ───────────────────────────────────────────────────────────────────────
ENV BUN_INSTALL=/usr/local/lib/bun
RUN curl -fsSL https://bun.sh/install | bash && \
ln -s /usr/local/lib/bun/bin/bun /usr/local/bin/bun && \
ln -s /usr/local/lib/bun/bin/bunx /usr/local/bin/bunx
# ── Playwright + Chromium (for screenshots) ──────────────────────────────────
ENV PLAYWRIGHT_BROWSERS_PATH=/usr/local/lib/playwright-browsers
RUN bun add -g @playwright/test \
&& bunx playwright install chromium --with-deps \
&& chmod -R a+rwX /usr/local/lib/playwright-browsers \
&& chmod -R a+rwX /usr/local/lib/bun/install \
&& rm -rf /var/lib/apt/lists/* /tmp/bunx-*
# ── AWS CLI (for S3-compatible uploads to R2) ─────────────────────────────────
RUN curl -fsSL "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o /tmp/awscliv2.zip \
&& unzip -q /tmp/awscliv2.zip -d /tmp \
&& /tmp/aws/install \
&& rm -rf /tmp/aws /tmp/awscliv2.zip
ENV AWS_DEFAULT_REGION=auto
# ── Runtime env for arbitrary UID ─────────────────────────────────────────────
# Mutable state goes to /tmp (writable by any UID). Toolchains stay read-only.
ENV CARGO_HOME=/tmp/.cargo BUN_TMPDIR=/tmp
# ── Entrypoint ────────────────────────────────────────────────────────────────
RUN cat <<'ENTRY' > /usr/local/bin/entrypoint.sh
#!/bin/sh
/usr/local/bin/register-dynamic-user.sh "$(id -u)" "$(id -g)"
# Start PostgreSQL (unix socket in /tmp, owned by postgres user)
mkdir -p /tmp/pgdata && sudo chown postgres:postgres /tmp/pgdata
if [ ! -f /tmp/pgdata/PG_VERSION ]; then
sudo -u postgres /usr/lib/postgresql/15/bin/initdb -D /tmp/pgdata --auth=trust
fi
sudo -u postgres /usr/lib/postgresql/15/bin/pg_ctl -D /tmp/pgdata -l /tmp/pg.log start -o "-k /tmp"
sudo -u postgres psql -h /tmp -c "CREATE ROLE sandbox SUPERUSER LOGIN" 2>/dev/null || true
sudo -u postgres createdb -h /tmp windmill 2>/dev/null || true
# Run database migrations so sqlx compile-time checks work
if [ -d "$PWD/backend/migrations" ]; then
DATABASE_URL="postgres://sandbox@localhost/windmill?host=/tmp" \
sqlx migrate run --source "$PWD/backend/migrations" 2>/dev/null || true
fi
# Install frontend dependencies and generate backend client
if [ -d "$PWD/frontend" ]; then
(cd "$PWD/frontend" && npm install && npm run generate-backend-client) 2>/dev/null || true
fi
exec "$@"
ENTRY
RUN chmod +x /usr/local/bin/entrypoint.sh
ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]

View File

@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE volume SET lease_until = now() + interval '60 seconds'\n WHERE workspace_id = $1 AND name = $2 AND leased_by = $3 AND lease_until > now()",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "00bf3dbd9d3f51dd7fdefcbd654d55e0379cc84188954037165cbe2d198ef71f"
}

View File

@@ -1,11 +1,11 @@
{
"db_name": "PostgreSQL",
"query": "SELECT value FROM variable WHERE workspace_id = $1 AND path = $2",
"query": "SELECT group_ FROM usr_to_group WHERE usr = $1 AND workspace_id = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "value",
"name": "group_",
"type_info": "Varchar"
}
],
@@ -19,5 +19,5 @@
false
]
},
"hash": "2c0ab7571e1a7c4290315bc3efccb4db9e0c9aee05596a594f81975a0cdb74d1"
"hash": "015a8551c646f9b027fc23752c5c5c81e520e3ca97dd1cd1e4ebfe3e46c4ad11"
}

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT large_file_storage->>'volume_storage' FROM workspace_settings WHERE workspace_id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "?column?",
"type_info": "Text"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
null
]
},
"hash": "083d69abc8a662bb364cf43b8ffc6e9b159a54c179cecb108068597536835f7e"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT extra_perms FROM volume WHERE workspace_id = $1 AND name = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "extra_perms",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false
]
},
"hash": "0afd4ae50ff7e1b0dcca4b483816c595401dd2e1f7699a28bf3b79db5e3841f4"
}

View File

@@ -1,16 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT token\n FROM token\n WHERE token LIKE concat($1::text, '%')\n LIMIT 1\n ",
"query": "SELECT created_by FROM volume WHERE name = $1 AND workspace_id = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "token",
"name": "created_by",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
@@ -18,5 +19,5 @@
false
]
},
"hash": "90092c0b3f7612373fcc8fb7a966200118ab308430d4a0cbb5cb16c397246492"
"hash": "0eb54f04a8185085b3f80772f5c28e666f6fbd1ec5ee9d30ee0cdb5e30a68750"
}

View File

@@ -0,0 +1,25 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO volume (workspace_id, name, size_bytes, created_by, lease_until, leased_by)\n VALUES ($1, $2, 0, $3, now() + interval '60 seconds', $4)\n ON CONFLICT (workspace_id, name) DO UPDATE\n SET lease_until = now() + interval '60 seconds', leased_by = $4\n WHERE volume.lease_until IS NULL OR volume.lease_until < now()\n RETURNING name",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "name",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar",
"Varchar"
]
},
"nullable": [
false
]
},
"hash": "14004a7c1641a3157eddd571fea11a1dfb1422187200119268b2342b47a960c6"
}

View File

@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO v2_job (id, kind, tag, created_by, permissioned_as, permissioned_as_email, workspace_id, runnable_path, preprocessed)\n VALUES ($1, 'flow', 'flow', 'test-user', 'u/test-user', 'test@windmill.dev', $2, $3, $4)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid",
"Varchar",
"Varchar",
"Bool"
]
},
"nullable": []
},
"hash": "181e6fca7e0d0fd88eccd79303f0339b1f2194c52f6bd1245dfa8ff3f0db4051"
}

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT email FROM token WHERE token = $1 AND (expiration > NOW() OR expiration IS NULL)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "email",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
true
]
},
"hash": "19a7ebb2e7e8e57b6e7c974da8eb7c6841a5c4ff12ba7c12c73d691c49dd99ed"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE volume SET last_used_at = now() WHERE workspace_id = $1 AND name = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": []
},
"hash": "1d2f765c2a71e1154ca5d9f5e52ef31e6d647377d37747f7bdc834748a59419e"
}

View File

@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO volume (workspace_id, name, size_bytes, created_by, last_used_at)\n VALUES ($1, $2, $3, $4, now())\n ON CONFLICT (workspace_id, name) DO UPDATE\n SET size_bytes = $3, last_used_at = now()",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Int8",
"Varchar"
]
},
"nullable": []
},
"hash": "1e9b9a02f45e6200f4d101bd5336fc8ce983f857339e6fccf799dc6587964aab"
}

View File

@@ -0,0 +1,25 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO volume (workspace_id, name, size_bytes, created_by, lease_until, leased_by)\n VALUES ($1, $2, 0, $3, now() + interval '60 seconds', $4)\n ON CONFLICT (workspace_id, name) DO UPDATE\n SET lease_until = now() + interval '60 seconds', leased_by = $4\n WHERE volume.lease_until IS NULL OR volume.lease_until < now()\n RETURNING name",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "name",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar",
"Varchar"
]
},
"nullable": [
false
]
},
"hash": "23f47f5207abe0cfaede197aeee485957990eb92fa3ce515895eab0d3f28bfdc"
}

View File

@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE volume SET lease_until = NULL, leased_by = NULL\n WHERE workspace_id = $1 AND name = $2 AND leased_by = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "28df7bbe1f54f69640bc76def9e580b4c7ba25f279644e3233b63f4f6db0ad98"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO v2_job_queue (id, workspace_id, scheduled_for, tag, running)\n VALUES ($1, $2, now(), 'flow', false)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid",
"Varchar"
]
},
"nullable": []
},
"hash": "2c503e1e8ee0863b3a6274874ef9b9a10b31dbbe2a676a50d1bbfb2e9e0ab7e0"
}

View File

@@ -0,0 +1,24 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT\n CASE\n WHEN flow_version.id IS NOT NULL THEN\n flow_version.value -> 'flow_env' -> $3\n ELSE\n root_job.raw_flow -> 'flow_env' -> $3\n END AS \"flow_env: sqlx::types::Json<Box<RawValue>>\"\n FROM\n v2_job current_job\n JOIN\n v2_job root_job ON root_job.id = COALESCE(current_job.root_job, current_job.flow_innermost_root_job, current_job.parent_job, current_job.id)\n AND root_job.workspace_id = current_job.workspace_id\n LEFT JOIN\n flow_version ON flow_version.id = root_job.runnable_id\n AND flow_version.path = root_job.runnable_path\n AND flow_version.workspace_id = root_job.workspace_id\n WHERE\n current_job.id = $1 AND\n current_job.workspace_id = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "flow_env: sqlx::types::Json<Box<RawValue>>",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Uuid",
"Text",
"Text"
]
},
"nullable": [
null
]
},
"hash": "2f53576c2ad58abc24617e911e486d7c4b9bdb1e8fb1f7725060990ef8984943"
}

View File

@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO global_settings (name, value) VALUES ('indexer_settings', $1)\n ON CONFLICT (name) DO UPDATE SET value = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Jsonb"
]
},
"nullable": []
},
"hash": "380ca9ebea53d5c016e4e76797cc103178ac4a25fc2842a13ce19b1ec4445c9d"
}

View File

@@ -0,0 +1,18 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE volume\n SET size_bytes = $3, file_count = $4,\n updated_at = now(), updated_by = $5, last_used_at = now(),\n lease_until = NULL, leased_by = NULL\n WHERE workspace_id = $1 AND name = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Int8",
"Int4",
"Varchar"
]
},
"nullable": []
},
"hash": "3955e57e216d169c30b1548a2252eb169329116cba57780fa90ecf2bdb910f34"
}

View File

@@ -0,0 +1,26 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO worker_ping (worker_instance, worker, ip, custom_tags, worker_group, dedicated_worker, dedicated_workers, wm_version, vcpus, memory, job_isolation, native_mode, uses_batch_http_pull) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13) ON CONFLICT (worker)\n DO UPDATE set ip = EXCLUDED.ip, custom_tags = EXCLUDED.custom_tags, worker_group = EXCLUDED.worker_group, dedicated_workers = EXCLUDED.dedicated_workers, native_mode = EXCLUDED.native_mode, uses_batch_http_pull = EXCLUDED.uses_batch_http_pull",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar",
"TextArray",
"Varchar",
"Varchar",
"TextArray",
"Varchar",
"Int8",
"Int8",
"Text",
"Bool",
"Bool"
]
},
"nullable": []
},
"hash": "3e8afd021088a99a24f27fa6f0a1b7f3edba3e9b834c814b464305bc2eb6ba80"
}

View File

@@ -0,0 +1,76 @@
{
"db_name": "PostgreSQL",
"query": "SELECT\n name as \"name!\",\n size_bytes as \"size_bytes!\",\n file_count as \"file_count!\",\n created_at as \"created_at!\",\n created_by as \"created_by!\",\n updated_at,\n updated_by,\n description as \"description!\",\n last_used_at,\n extra_perms as \"extra_perms!\"\n FROM (\n SELECT\n COALESCE(v.name, a.path) as name,\n COALESCE(v.size_bytes, 0) as size_bytes,\n COALESCE(v.file_count, 0) as file_count,\n COALESCE(v.created_at, a.min_created_at) as created_at,\n COALESCE(v.created_by, 'unknown') as created_by,\n v.updated_at,\n v.updated_by,\n COALESCE(v.description, '') as description,\n v.last_used_at,\n COALESCE(v.extra_perms, '{}'::jsonb) as extra_perms\n FROM (\n SELECT path, MIN(created_at) as min_created_at\n FROM asset\n WHERE workspace_id = $1 AND kind = 'volume'\n GROUP BY path\n ) a\n FULL OUTER JOIN volume v ON v.workspace_id = $1 AND v.name = a.path\n WHERE v.workspace_id = $1 OR a.path IS NOT NULL\n ) combined\n ORDER BY name",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "name!",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "size_bytes!",
"type_info": "Int8"
},
{
"ordinal": 2,
"name": "file_count!",
"type_info": "Int4"
},
{
"ordinal": 3,
"name": "created_at!",
"type_info": "Timestamptz"
},
{
"ordinal": 4,
"name": "created_by!",
"type_info": "Varchar"
},
{
"ordinal": 5,
"name": "updated_at",
"type_info": "Timestamptz"
},
{
"ordinal": 6,
"name": "updated_by",
"type_info": "Varchar"
},
{
"ordinal": 7,
"name": "description!",
"type_info": "Text"
},
{
"ordinal": 8,
"name": "last_used_at",
"type_info": "Timestamptz"
},
{
"ordinal": 9,
"name": "extra_perms!",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
null,
null,
null,
null,
null,
true,
true,
null,
true,
null
]
},
"hash": "40d0f6dca30456514cb85e36c6e367b27171894016c714e41497e69115be1468"
}

View File

@@ -15,7 +15,7 @@
]
},
"nullable": [
null
true
]
},
"hash": "5a219a2532517869578c4504ff3153c43903f929ae5d62fbba12610f89c36d55"

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM volume WHERE workspace_id = $1 AND name = $2\n AND (lease_until IS NULL OR lease_until < now())\n RETURNING name",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "name",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false
]
},
"hash": "5af44b46a2e2f1a9adeb39013790be7046cf8789d842717b6c793c22a2a05daa"
}

View File

@@ -0,0 +1,29 @@
{
"db_name": "PostgreSQL",
"query": "SELECT created_by, extra_perms FROM volume WHERE workspace_id = $1 AND name = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "created_by",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "extra_perms",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
false
]
},
"hash": "6086849bb08e1b37d6693d2808767cd897dca4722e4f2076308afdb7ee9fc147"
}

View File

@@ -0,0 +1,26 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE worker_ping SET ping_at = now(), jobs_executed = $1, custom_tags = $2,\n occupancy_rate = $3, memory_usage = $4, wm_memory_usage = $5, vcpus = COALESCE($7, vcpus),\n memory = COALESCE($8, memory), occupancy_rate_15s = $9, occupancy_rate_5m = $10, occupancy_rate_30m = $11, native_mode = $12, uses_batch_http_pull = $13 WHERE worker = $6",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Int4",
"TextArray",
"Float4",
"Int8",
"Int8",
"Text",
"Int8",
"Int8",
"Float4",
"Float4",
"Float4",
"Bool",
"Bool"
]
},
"nullable": []
},
"hash": "6cd099d458ac380d5da27b9e69da035755496ea50f2b78fb9b1cd3a2eb7e7625"
}

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT count(*) FROM volume WHERE workspace_id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "count",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
null
]
},
"hash": "712092e5033bc6894025a55ebc58bca8450d09982e582266d215dff521256fa6"
}

View File

@@ -0,0 +1,18 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE volume\n SET size_bytes = $3, file_count = $4,\n updated_at = now(), last_used_at = now(),\n lease_until = NULL, leased_by = NULL\n WHERE workspace_id = $1 AND name = $2 AND leased_by = $5",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Int8",
"Int4",
"Text"
]
},
"nullable": []
},
"hash": "75a03e9e4cba350a104e2e3a95de919cd25538c0b433bc29bb052c7a7b8568ca"
}

View File

@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE volume SET lease_until = now() + interval '60 seconds'\n WHERE workspace_id = $1 AND name = $2 AND leased_by = $3 AND lease_until > now()",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "769035629df5a5034f64bf38992e142006825a3911addacdf1a026660b5e2b7f"
}

View File

@@ -0,0 +1,24 @@
{
"db_name": "PostgreSQL",
"query": "SELECT EXISTS(SELECT 1 FROM volume WHERE workspace_id = $1 AND name = $2 AND lease_until > now() AND leased_by = $3)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "exists",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": [
null
]
},
"hash": "78af8bdb6a3ee6396c54f87ff6403b566fc75e16e0b7a81204816fd50b3346a5"
}

View File

@@ -1,19 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "WITH job_result AS (\n SELECT result\n FROM v2_job_completed\n WHERE id = $1\n ),\n updated_queue AS (\n UPDATE v2_job_queue\n SET running = false,\n tag = COALESCE($3, tag),\n scheduled_for = COALESCE($6, scheduled_for)\n WHERE id = $2\n )\n UPDATE v2_job\n SET\n tag = COALESCE($3, tag),\n concurrent_limit = COALESCE($4, concurrent_limit),\n concurrency_time_window_s = COALESCE($5, concurrency_time_window_s),\n args = COALESCE(\n CASE\n WHEN job_result.result IS NULL THEN NULL\n WHEN jsonb_typeof(job_result.result) = 'object'\n THEN job_result.result\n WHEN jsonb_typeof(job_result.result) = 'null'\n THEN NULL\n ELSE jsonb_build_object('value', job_result.result)\n END,\n '{}'::jsonb\n ),\n preprocessed = TRUE\n FROM job_result\n WHERE v2_job.id = $2;\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid",
"Uuid",
"Varchar",
"Int4",
"Int4",
"Timestamptz"
]
},
"nullable": []
},
"hash": "79b437ad31ddab94310989b8fb6a1c130b9be1ab4b6a100fffffd687677b9c92"
}

View File

@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE volume SET lease_until = NULL, leased_by = NULL\n WHERE workspace_id = $1 AND name = $2 AND leased_by = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "7ce06d4f623932fce12352be3a09ba8973a2ef1defa36c6d46d9c1c6406a7c33"
}

View File

@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO volume (workspace_id, name, size_bytes, created_by)\n VALUES ($1, $2, $3, $4)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Int8",
"Varchar"
]
},
"nullable": []
},
"hash": "7e8e79a7d140be511cedbfe9ff8eea76a8a3079ce80c035087f797cdc410f35b"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT last_used_at FROM volume WHERE workspace_id = $1 AND name = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "last_used_at",
"type_info": "Timestamptz"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
true
]
},
"hash": "803abdcd3614437b26c5d2e4f1ad75ca7014b431239ac1b681f2b26380c719c4"
}

View File

@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE volume SET extra_perms = extra_perms - $1\n WHERE workspace_id = $2 AND name = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "82b3bd95e5d28c4cd4eedcae8cf050ba7b7e4d9eabba03be251ae9a8017b317d"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT leased_by FROM volume WHERE workspace_id = $1 AND name = $2 AND lease_until > now()",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "leased_by",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
true
]
},
"hash": "88e25dc24bb06237b3677c947ee53fd6e9c7606231ad3c522e98cb1fcc14361a"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT count(*) FROM volume WHERE workspace_id = $1 AND name = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "count",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
null
]
},
"hash": "907241c195fea227e4a945ee472425e5f7600e28c728a06235f7ff430a4bd77a"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT size_bytes FROM volume WHERE workspace_id = $1 AND name = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "size_bytes",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false
]
},
"hash": "94d6f598076ad67d68e6f01926c9fc2c73e855790e17abf5461b96ea30fbbdb7"
}

View File

@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE volume SET lease_until = NULL, leased_by = NULL\n WHERE workspace_id = $1 AND name = $2 AND leased_by = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "9662f1e304124fa52db4aa1e80e03b2601630f2d31458bdaf70c2702b2998d89"
}

View File

@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO v2_job (id, kind, tag, created_by, permissioned_as, permissioned_as_email, workspace_id, runnable_path, args)\n VALUES ($1, 'script', 'deno', 'test-user', 'u/test-user', 'test@windmill.dev', $2, $3, $4)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid",
"Varchar",
"Varchar",
"Jsonb"
]
},
"nullable": []
},
"hash": "9c76a980bf1e3b79ab26c79aee19e5552aa16eb3626618da4dbb44ed18efee60"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM volume WHERE workspace_id = $1 AND name = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": []
},
"hash": "9e30b5545a51453205a713a6276156ada29ae320465d9790dce7e1e8a436d4de"
}

View File

@@ -0,0 +1,29 @@
{
"db_name": "PostgreSQL",
"query": "SELECT extra_perms, created_by FROM volume WHERE workspace_id = $1 AND name = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "extra_perms",
"type_info": "Jsonb"
},
{
"ordinal": 1,
"name": "created_by",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
false
]
},
"hash": "9f64d6ed0adb609ced1551563062550919fcac56deaf1b3cb36b3e15117936e7"
}

View File

@@ -0,0 +1,24 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO volume (workspace_id, name, size_bytes, created_by)\n VALUES ($1, $2, 0, $3)\n ON CONFLICT (workspace_id, name) DO NOTHING\n RETURNING name",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "name",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar"
]
},
"nullable": [
false
]
},
"hash": "a3970c15271a124307301c0dafa263e7168fa325c5ceb44e9dd1595bdb7e7ce6"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO token_expiry_notification (token, expiration) VALUES ($1, $2) ON CONFLICT DO NOTHING",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Timestamptz"
]
},
"nullable": []
},
"hash": "a4d973d0f1c293345ad2bfd2472da8d6a3b425ea0590a66f1db6692dd2ddb437"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM token_expiry_notification WHERE expiration <= now()",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "a6b1c8808c892e62ae4ba04171d856a39c89cdc658b09c478050de5145a45ca4"
}

View File

@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO volume (workspace_id, name, size_bytes, created_by)\n VALUES ($1, $2, $3, $4)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Int8",
"Varchar"
]
},
"nullable": []
},
"hash": "ab8daa93bc66d0142b9e9e8d7fa6719fc41b2ca5cb0b7ac5ad73ab01b650c935"
}

View File

@@ -0,0 +1,38 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM token WHERE expiration <= now()\n RETURNING substring(token for 10) as token_prefix, label, email, workspace_id",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "token_prefix",
"type_info": "Text"
},
{
"ordinal": 1,
"name": "label",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "email",
"type_info": "Varchar"
},
{
"ordinal": 3,
"name": "workspace_id",
"type_info": "Varchar"
}
],
"parameters": {
"Left": []
},
"nullable": [
null,
true,
true,
true
]
},
"hash": "bb446cbb20166f274a7ee6e88abaa27e233e60e18b3d35545005eb680701241f"
}

View File

@@ -0,0 +1,29 @@
{
"db_name": "PostgreSQL",
"query": "SELECT size_bytes, last_used_at FROM volume WHERE workspace_id = $1 AND name = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "size_bytes",
"type_info": "Int8"
},
{
"ordinal": 1,
"name": "last_used_at",
"type_info": "Timestamptz"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
true
]
},
"hash": "bc61ca62d8f71880facb5d701a6e78697414b35618c50f8693f4e804bf1d7dbb"
}

View File

@@ -0,0 +1,34 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id, last_locked_at, owner FROM concurrency_locks WHERE id = ANY($1)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "last_locked_at",
"type_info": "Timestamp"
},
{
"ordinal": 2,
"name": "owner",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"TextArray"
]
},
"nullable": [
false,
false,
true
]
},
"hash": "bcefd1ce47d05f2ce14493f0e7c4d4fea16c0cf71ddc233f6431cf624ecdfe60"
}

View File

@@ -1,25 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT\n CASE\n WHEN flow_version.id IS NOT NULL THEN\n (flow_version.value -> 'flow_env' -> $3) #> $4\n ELSE\n (root_job.raw_flow -> 'flow_env' -> $3) #> $4\n END AS \"flow_env: sqlx::types::Json<Box<RawValue>>\"\n FROM\n v2_job current_job\n JOIN\n v2_job root_job ON root_job.id = COALESCE(current_job.root_job, current_job.flow_innermost_root_job, current_job.parent_job, current_job.id)\n AND root_job.workspace_id = current_job.workspace_id\n LEFT JOIN\n flow_version ON flow_version.id = root_job.runnable_id\n AND flow_version.path = root_job.runnable_path\n AND flow_version.workspace_id = root_job.workspace_id\n WHERE\n current_job.id = $1 AND\n current_job.workspace_id = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "flow_env: sqlx::types::Json<Box<RawValue>>",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Uuid",
"Text",
"Text",
"TextArray"
]
},
"nullable": [
null
]
},
"hash": "c23bea7db9623a60683596b7d6e689e2c0100c1569436a01b207876aaa470154"
}

View File

@@ -0,0 +1,20 @@
{
"db_name": "PostgreSQL",
"query": "WITH job_result AS (\n SELECT result\n FROM v2_job_completed\n WHERE id = $1\n ),\n updated_queue AS (\n UPDATE v2_job_queue\n SET running = false,\n tag = COALESCE($3, tag),\n scheduled_for = COALESCE($6, scheduled_for),\n runnable_settings_handle = COALESCE($7, runnable_settings_handle)\n WHERE id = $2\n )\n UPDATE v2_job\n SET\n tag = COALESCE($3, tag),\n concurrent_limit = COALESCE($4, concurrent_limit),\n concurrency_time_window_s = COALESCE($5, concurrency_time_window_s),\n args = COALESCE(\n CASE\n WHEN job_result.result IS NULL THEN NULL\n WHEN jsonb_typeof(job_result.result) = 'object'\n THEN job_result.result\n WHEN jsonb_typeof(job_result.result) = 'null'\n THEN NULL\n ELSE jsonb_build_object('value', job_result.result)\n END,\n '{}'::jsonb\n ),\n preprocessed = TRUE\n FROM job_result\n WHERE v2_job.id = $2;\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid",
"Uuid",
"Varchar",
"Int4",
"Int4",
"Timestamptz",
"Int8"
]
},
"nullable": []
},
"hash": "c2a0605b07f5df8d972bc02cc23fe7def5e1ee8fdf6dfb68576d3b72aa03f666"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE v2_job SET args = $2 WHERE id = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid",
"Jsonb"
]
},
"nullable": []
},
"hash": "c31cf6239044615e1cc3743aa1c82cce96e1a23ada28107ffffc8b5546d48101"
}

View File

@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_invite (workspace_id, email, is_admin, operator)\n SELECT $1, email, is_admin, operator\n FROM usr\n WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "cd399a3a797d1733fb9071ebca3f5928a3c7eba2983431844581fd2393312a2e"
}

View File

@@ -0,0 +1,47 @@
{
"db_name": "PostgreSQL",
"query": "SELECT workspace_id, name, size_bytes, created_by, last_used_at\n FROM volume WHERE workspace_id = $1 AND name = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "workspace_id",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "name",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "size_bytes",
"type_info": "Int8"
},
{
"ordinal": 3,
"name": "created_by",
"type_info": "Varchar"
},
{
"ordinal": 4,
"name": "last_used_at",
"type_info": "Timestamptz"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
false,
false,
false,
true
]
},
"hash": "d0869a340c8f34ca7a560d3b4c0070c9f117da3dd00ce3247c54a61052a6809c"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT leased_by FROM volume WHERE workspace_id = $1 AND name = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "leased_by",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
true
]
},
"hash": "d1ad2baf5e3a6f45f1f079d494e8d6affad03a1f388024806a5de3f9cc939c04"
}

View File

@@ -0,0 +1,38 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM token_expiry_notification n\n USING token t\n WHERE n.token = t.token\n AND n.expiration > now()\n AND n.expiration <= now() + interval '7 days'\n RETURNING substring(t.token for 10) as token_prefix, t.label, t.email, t.workspace_id",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "token_prefix",
"type_info": "Text"
},
{
"ordinal": 1,
"name": "label",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "email",
"type_info": "Varchar"
},
{
"ordinal": 3,
"name": "workspace_id",
"type_info": "Varchar"
}
],
"parameters": {
"Left": []
},
"nullable": [
null,
true,
true,
true
]
},
"hash": "d7e9b69fef8369117ce057d01d87288b39ea7c802007f112eb3d62230d07abb6"
}

View File

@@ -0,0 +1,28 @@
{
"db_name": "PostgreSQL",
"query": "SELECT name, size_bytes FROM volume WHERE workspace_id = $1 ORDER BY name",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "name",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "size_bytes",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false,
false
]
},
"hash": "dc18db954239c4ebdd3b46cfd34f33554794444f0dc4e2d2fec158eca5ebe865"
}

View File

@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE volume SET extra_perms = jsonb_set(extra_perms, $1, to_jsonb($2::bool), true)\n WHERE workspace_id = $3 AND name = $4",
"describe": {
"columns": [],
"parameters": {
"Left": [
"TextArray",
"Bool",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "eb79db2aeac7bf246ad56a5f116511b9d3183cb91b740a86944a77a2a964b57d"
}

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT token as \"token!\"\n FROM token\n WHERE token LIKE concat($1::text, '%')\n LIMIT 1\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "token!",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false
]
},
"hash": "eba16eb819e2644284fb073c891706d78a6f24cb0e614d7d81ba1b643805bf06"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT permissioned_as FROM v2_job WHERE id = $1 AND workspace_id = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "permissioned_as",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Uuid",
"Text"
]
},
"nullable": [
false
]
},
"hash": "f0ac12b66c5d3cca680541aed04359b064baf73b890efdc25426261d4eadfee0"
}

View File

@@ -0,0 +1,41 @@
{
"db_name": "PostgreSQL",
"query": "SELECT size_bytes, file_count, leased_by, lease_until\n FROM volume WHERE workspace_id = $1 AND name = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "size_bytes",
"type_info": "Int8"
},
{
"ordinal": 1,
"name": "file_count",
"type_info": "Int4"
},
{
"ordinal": 2,
"name": "leased_by",
"type_info": "Varchar"
},
{
"ordinal": 3,
"name": "lease_until",
"type_info": "Timestamptz"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
false,
true,
true
]
},
"hash": "f7ba87d5804b9bc05e7156c7c18c5a30037abef63efb5b44dc535c5f45d62a06"
}

View File

@@ -1,98 +1,8 @@
# Backend Development (Rust)
# Backend (Rust)
## Project Structure
Windmill uses a workspace-based architecture with multiple crates:
- **windmill-api**: API server functionality
- **windmill-worker**: Job execution
- **windmill-common**: Shared code used by all crates
- **windmill-queue**: Job & flow queuing
- **windmill-audit**: Audit logging
- Other specialized crates (git-sync, autoscaling, etc.)
## Key References (MUST FOLLOW THESE)
- You MUST follow best-practices by using the `rust-backend` skill, everytime you write RUST code.
- When working with the database: read `summarized_schema.txt` before starting
- When working with the API routes: you can read `windmill-api/src/lib.rs` to get started
## Adding New Code
### Module Organization
- Place new code in the appropriate crate based on functionality
- For API endpoints, create or modify files in `windmill-api/src/` organized by domain
- For shared functionality, use `windmill-common/src/`
- Follow existing patterns for file structure and organization
### API Endpoints
- Follow existing patterns in the `windmill-api` crate
- Use axum's routing system and extractors
- Update `backend/windmill-api/openapi.yaml` after modifying API endpoints
### Database Changes
- Update database schema with migration if necessary
- Use `sqlx` for database operations with prepared statements
- Use transactions for multi-step operations
- To apply pending migrations: `sqlx migrate run` (never manually run .sql files)
- **Never use `SQLX_OFFLINE=true`** — a live database is always available for compilation
- After all code changes are done, run `./update-sqlx` to regenerate the offline query cache
## Enterprise Features
- Enterprise files use the `*_ee.rs` suffix
- Enterprise source is in `windmill-ee-private` folder (sibling directory at `../../windmill-ee-private` or `~/windmill-ee-private`), symlinked into each crate's `src/`
- The `_ee.rs` files are gitignored in the main repo — they are tracked only in the `windmill-ee-private` repo
- You can and should modify `windmill-ee-private` directly when needed (e.g., when creating new crates that need EE code, mirror the package structure there)
- Use feature flags: `#[cfg(feature = "enterprise")]`
- Isolate enterprise code in separate modules
### EE PR Workflow (MUST DO when modifying `*_ee.rs` files)
When you modify any `*_ee.rs` file and create a PR on the windmill repo, you **MUST** also:
1. **Create a matching branch** in the `windmill-ee-private` repo (use the same branch name). If using worktrees, the EE worktree is at `~/windmill-ee-private__worktrees/<branch-name>/`
2. **Commit and push** the `_ee.rs` changes in that branch
3. **Create a PR** on `windmill-ee-private` with a link to the companion windmill PR
4. **Update `ee-repo-ref.txt`**: Run `bash write_latest_ee_ref.sh` from `backend/` to write the latest EE commit hash. **Important**: the script may fall back to `~/windmill-ee-private` (main branch) instead of the worktree — verify it wrote the correct commit hash from your branch, not from main. If wrong, manually write the correct hash.
5. **Commit `ee-repo-ref.txt`** in the windmill repo so CI picks up the correct EE ref
## Code Validation (MUST DO)
After making backend changes, you MUST run `cargo check` and fix all errors and warnings before considering the work done.
Only enable the feature flags relevant to your changes — do NOT use `all_sqlx_features` as it compiles the entire codebase and is very slow. Check the `[features]` section in `Cargo.toml` to identify which flags gate the crates/modules you modified.
Examples:
```bash
# Changed core code (no feature-gated modules)
cargo check
# Changed code behind the enterprise feature
cargo check --features enterprise
# Changed kafka trigger code
cargo check --features kafka
```
## Git Workflow
- **Never push directly to main** — always create a branch and open a pull request
## Testing
- Write unit tests for core functionality
- Use the `#[cfg(test)]` module for test code
- For database tests, use the existing test utilities
## Common Crates
- **tokio**: Async runtime
- **axum**: Web server and routing
- **sqlx**: Database operations
- **serde**: Serialization/deserialization
- **tracing**: Logging and diagnostics
- **reqwest**: HTTP client
- **Coding patterns**: MUST use the `rust-backend` skill when writing Rust code
- **Validation**: `docs/validation.md` — which `cargo check` flags to use
- **Enterprise**: `docs/enterprise.md` — EE file conventions and PR workflow
- **DB schema**: `backend/summarized_schema.txt`
- **API routes entry point**: `windmill-api/src/lib.rs`
- **OpenAPI spec**: `windmill-api/openapi.yaml`

308
backend/Cargo.lock generated
View File

@@ -860,9 +860,9 @@ dependencies = [
[[package]]
name = "aws-lc-rs"
version = "1.16.0"
version = "1.16.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d9a7b350e3bb1767102698302bc37256cbd48422809984b98d292c40e2579aa9"
checksum = "94bffc006df10ac2a68c83692d734a465f8ee6c5b384d8545a636f81d858f4bf"
dependencies = [
"aws-lc-sys",
"zeroize",
@@ -870,9 +870,9 @@ dependencies = [
[[package]]
name = "aws-lc-sys"
version = "0.37.1"
version = "0.38.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b092fe214090261288111db7a2b2c2118e5a7f30dc2569f1732c4069a6840549"
checksum = "4321e568ed89bb5a7d291a7f37997c2c0df89809d7b6d12062c81ddb54aa782e"
dependencies = [
"cc",
"cmake",
@@ -1334,9 +1334,9 @@ dependencies = [
[[package]]
name = "aws-smithy-xml"
version = "0.60.14"
version = "0.60.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b53543b4b86ed43f051644f704a98c7291b3618b67adf057ee77a366fa52fcaa"
checksum = "0ce02add1aa3677d022f8adf81dcbe3046a95f17a1b1e8979c145cd21d3d22b3"
dependencies = [
"xmlparser",
]
@@ -1900,7 +1900,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0686c856aa6aac0c4498f936d7d6a02df690f614c03e4d906d1018062b5c5e2c"
dependencies = [
"once_cell",
"proc-macro-crate 3.4.0",
"proc-macro-crate",
"proc-macro2",
"quote",
"syn 2.0.117",
@@ -2550,6 +2550,15 @@ dependencies = [
"unicode-segmentation",
]
[[package]]
name = "convert_case"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "633458d4ef8c78b72454de2d54fd6ab2e60f9e02be22f3c6104cdc8a4e0fceb9"
dependencies = [
"unicode-segmentation",
]
[[package]]
name = "cooked-waker"
version = "5.0.0"
@@ -6173,20 +6182,20 @@ dependencies = [
"cfg-if",
"js-sys",
"libc",
"r-efi",
"r-efi 5.3.0",
"wasip2",
"wasm-bindgen",
]
[[package]]
name = "getrandom"
version = "0.4.1"
version = "0.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "139ef39800118c7683f2fd3c98c1b23c09ae076556b435f8e9064ae108aaeeec"
checksum = "0de51e6874e94e7bf76d726fc5d13ba782deca734ff60d5bb2fb2607c7406555"
dependencies = [
"cfg-if",
"libc",
"r-efi",
"r-efi 6.0.0",
"wasip2",
"wasip3",
]
@@ -7421,9 +7430,9 @@ dependencies = [
[[package]]
name = "ipnet"
version = "2.11.0"
version = "2.12.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "469fb0b9cefa57e3ef31275ee7cacb78f2fdca44e4765491884a2b119d4eb130"
checksum = "d98f6fed1fde3f8c21bc40a1abb88dd75e67924f9cffc3ef95607bad8017f8e2"
[[package]]
name = "ipnetwork"
@@ -8049,13 +8058,14 @@ dependencies = [
[[package]]
name = "libredox"
version = "0.1.12"
version = "0.1.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3d0b95e02c851351f877147b7deea7b1afb1df71b63aa5f8270716e0c5720616"
checksum = "1744e39d1d6a9948f4f388969627434e31128196de472883b39f148769bfe30a"
dependencies = [
"bitflags 2.9.4",
"libc",
"redox_syscall 0.7.2",
"plain",
"redox_syscall 0.7.3",
]
[[package]]
@@ -8599,9 +8609,9 @@ dependencies = [
[[package]]
name = "moka"
version = "0.12.13"
version = "0.12.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b4ac832c50ced444ef6be0767a008b02c106a909ba79d1d830501e94b96f6b7e"
checksum = "85f8024e1c8e71c778968af91d43700ce1d11b219d127d79fb2934153b82b42b"
dependencies = [
"async-lock",
"crossbeam-channel",
@@ -8682,7 +8692,7 @@ dependencies = [
"darling 0.20.11",
"heck 0.5.0",
"num-bigint",
"proc-macro-crate 3.4.0",
"proc-macro-crate",
"proc-macro-error2",
"proc-macro2",
"quote",
@@ -9251,7 +9261,7 @@ version = "0.7.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ff32365de1b6743cb203b710788263c44a03de03802daf96092f2da4fe6ba4d7"
dependencies = [
"proc-macro-crate 3.4.0",
"proc-macro-crate",
"proc-macro2",
"quote",
"syn 2.0.117",
@@ -10085,18 +10095,18 @@ dependencies = [
[[package]]
name = "pin-project"
version = "1.1.10"
version = "1.1.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "677f1add503faace112b9f1373e43e9e054bfdd22ff1a63c1bc485eaec6a6a8a"
checksum = "f1749c7ed4bcaf4c3d0a3efc28538844fb29bcdd7d2b67b2be7e20ba861ff517"
dependencies = [
"pin-project-internal",
]
[[package]]
name = "pin-project-internal"
version = "1.1.10"
version = "1.1.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6e918e4ff8c4549eb882f14b3a4bc8c8bc93de829416eacf579f1207a8fbf861"
checksum = "d9b20ed30f105399776b9c883e68e536ef602a16ae6f596d2c473591d6ad64c6"
dependencies = [
"proc-macro2",
"quote",
@@ -10105,9 +10115,9 @@ dependencies = [
[[package]]
name = "pin-project-lite"
version = "0.2.16"
version = "0.2.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3b3cff922bd51709b605d9ead9aa71031d81447142d828eb4a6eba76fe619f9b"
checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd"
[[package]]
name = "pin-utils"
@@ -10159,6 +10169,12 @@ version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7edddbd0b52d732b21ad9a5fab5c704c14cd949e5e9a1ec5929a24fded1b904c"
[[package]]
name = "plain"
version = "0.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b4596b6d070b27117e987119b4dac604f3c58cfb0b191112e24771b2faeac1a6"
[[package]]
name = "png"
version = "0.17.16"
@@ -10324,16 +10340,6 @@ dependencies = [
"elliptic-curve",
]
[[package]]
name = "proc-macro-crate"
version = "1.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f4c021e1093a56626774e81216a4ce732a735e5bad4868a03f3ed65ca0c3919"
dependencies = [
"once_cell",
"toml_edit 0.19.15",
]
[[package]]
name = "proc-macro-crate"
version = "3.4.0"
@@ -10703,9 +10709,9 @@ dependencies = [
[[package]]
name = "quote"
version = "1.0.44"
version = "1.0.45"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "21b2ebcf727b7760c461f091f9f0f539b77b8e87f2fd88131e7f1b433b3cece4"
checksum = "41f2619966050689382d2b44f664f4bc593e129785a36d6ee376ddf37259b924"
dependencies = [
"proc-macro2",
]
@@ -10716,6 +10722,12 @@ version = "5.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "69cdb34c158ceb288df11e18b4bd39de994f6657d83847bdffdbd7f346754b0f"
[[package]]
name = "r-efi"
version = "6.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf"
[[package]]
name = "radium"
version = "0.7.0"
@@ -10855,9 +10867,9 @@ dependencies = [
[[package]]
name = "range-alloc"
version = "0.1.4"
version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c3d6831663a5098ea164f89cff59c6284e95f4e3c76ce9848d4529f5ccca9bde"
checksum = "ca45419789ae5a7899559e9512e58ca889e41f04f1f2445e9f4b290ceccd1d08"
[[package]]
name = "raw-cpuid"
@@ -10989,9 +11001,9 @@ dependencies = [
[[package]]
name = "redox_syscall"
version = "0.7.2"
version = "0.7.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6d94dd2f7cd932d4dc02cc8b2b50dfd38bd079a4e5d79198b99743d7fcf9a4b4"
checksum = "6ce70a74e890531977d37e532c34d45e9055d2409ed08ddba14529471ed0be16"
dependencies = [
"bitflags 2.9.4",
]
@@ -11081,9 +11093,12 @@ checksum = "dc897dd8d9e8bd1ed8cdad82b5966c3e0ecae09fb1907d58efaa013543185d0a"
[[package]]
name = "relative-path"
version = "1.9.3"
version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ba39f3699c378cd8970968dcbff9c43159ea4cfbd88d43c00b22f2ef10a435d2"
checksum = "bca40a312222d8ba74837cb474edef44b37f561da5f773981007a10bbaa992b0"
dependencies = [
"serde",
]
[[package]]
name = "rend"
@@ -11377,9 +11392,9 @@ dependencies = [
[[package]]
name = "rquickjs"
version = "0.8.1"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d16661bff09e9ed8e01094a188b463de45ec0693ade55b92ed54027d7ba7c40c"
checksum = "c50dc6d6c587c339edb4769cf705867497a2baf0eca8b4645fa6ecd22f02c77a"
dependencies = [
"rquickjs-core",
"rquickjs-macro",
@@ -11387,26 +11402,27 @@ dependencies = [
[[package]]
name = "rquickjs-core"
version = "0.8.1"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6c8db6379e204ef84c0811e90e7cc3e3e4d7688701db68a00d14a6db6849087b"
checksum = "b8bf7840285c321c3ab20e752a9afb95548c75cd7f4632a0627cea3507e310c1"
dependencies = [
"async-lock",
"hashbrown 0.16.0",
"relative-path",
"rquickjs-sys",
]
[[package]]
name = "rquickjs-macro"
version = "0.8.1"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6041104330c019fcd936026ae05e2446f5e8a2abef329d924f25424b7052a2f3"
checksum = "7106215ff41a5677b104906a13e1a440b880f4b6362b5dc4f3978c267fad2b80"
dependencies = [
"convert_case 0.6.0",
"convert_case 0.10.0",
"fnv",
"ident_case",
"indexmap 2.11.1",
"proc-macro-crate 1.3.1",
"proc-macro-crate",
"proc-macro2",
"quote",
"rquickjs-core",
@@ -11415,9 +11431,9 @@ dependencies = [
[[package]]
name = "rquickjs-sys"
version = "0.8.1"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4bc352c6b663604c3c186c000cfcc6c271f4b50bc135a285dd6d4f2a42f9790a"
checksum = "27344601ef27460e82d6a4e1ecb9e7e99f518122095f3c51296da8e9be2b9d83"
dependencies = [
"cc",
]
@@ -12586,9 +12602,9 @@ checksum = "1b6709c7b6754dca1311b3c73e79fcce40dd414c782c66d88e8823030093b02b"
[[package]]
name = "sketches-ddsketch"
version = "0.3.0"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c1e9a774a6c28142ac54bb25d25562e6bcf957493a184f15ad4eebccb23e410a"
checksum = "0c6f73aeb92d671e0cc4dca167e59b2deb6387c375391bc99ee743f326994a2b"
dependencies = [
"serde",
]
@@ -13843,7 +13859,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "82a72c767771b47409d2345987fda8628641887d5466101319899796367354a0"
dependencies = [
"fastrand",
"getrandom 0.4.1",
"getrandom 0.4.2",
"once_cell",
"rustix 1.1.4",
"windows-sys 0.61.2",
@@ -15725,7 +15741,7 @@ dependencies = [
[[package]]
name = "windmill"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-nats",
@@ -15757,6 +15773,7 @@ dependencies = [
"sql-builder",
"sqlx",
"strum 0.27.2",
"tar",
"tempfile",
"tikv-jemalloc-ctl",
"tikv-jemalloc-sys",
@@ -15782,14 +15799,16 @@ dependencies = [
"windmill-queue",
"windmill-runtime-nativets",
"windmill-test-utils",
"windmill-types",
"windmill-worker",
"windmill-worker-volumes",
"windows-service",
"windows-sys 0.52.0",
]
[[package]]
name = "windmill-alerting"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -15802,7 +15821,7 @@ dependencies = [
[[package]]
name = "windmill-api"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"argon2",
@@ -15936,11 +15955,12 @@ dependencies = [
"windmill-trigger-websocket",
"windmill-types",
"windmill-worker",
"windmill-worker-volumes",
]
[[package]]
name = "windmill-api-agent-workers"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -15963,7 +15983,7 @@ dependencies = [
[[package]]
name = "windmill-api-assets"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -15976,7 +15996,7 @@ dependencies = [
[[package]]
name = "windmill-api-auth"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -16002,7 +16022,7 @@ dependencies = [
[[package]]
name = "windmill-api-client"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"reqwest 0.12.28",
"serde",
@@ -16012,7 +16032,7 @@ dependencies = [
[[package]]
name = "windmill-api-configs"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16029,7 +16049,7 @@ dependencies = [
[[package]]
name = "windmill-api-debug"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"axum 0.7.9",
"base64 0.22.1",
@@ -16052,7 +16072,7 @@ dependencies = [
[[package]]
name = "windmill-api-embeddings"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -16075,7 +16095,7 @@ dependencies = [
[[package]]
name = "windmill-api-flow-conversations"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16091,7 +16111,7 @@ dependencies = [
[[package]]
name = "windmill-api-flows"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16111,7 +16131,7 @@ dependencies = [
[[package]]
name = "windmill-api-groups"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16131,7 +16151,7 @@ dependencies = [
[[package]]
name = "windmill-api-inputs"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16145,7 +16165,7 @@ dependencies = [
[[package]]
name = "windmill-api-integration-tests"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-nats",
@@ -16172,7 +16192,7 @@ dependencies = [
[[package]]
name = "windmill-api-jobs"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -16197,7 +16217,7 @@ dependencies = [
[[package]]
name = "windmill-api-npm-proxy"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"axum 0.7.9",
"flate2",
@@ -16208,13 +16228,14 @@ dependencies = [
"tar",
"tower-http",
"tracing",
"url",
"windmill-api-auth",
"windmill-common",
]
[[package]]
name = "windmill-api-openapi"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -16235,7 +16256,7 @@ dependencies = [
[[package]]
name = "windmill-api-schedule"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16255,7 +16276,7 @@ dependencies = [
[[package]]
name = "windmill-api-scripts"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16285,7 +16306,7 @@ dependencies = [
[[package]]
name = "windmill-api-settings"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -16312,7 +16333,7 @@ dependencies = [
[[package]]
name = "windmill-api-sse"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"lazy_static",
"serde",
@@ -16324,7 +16345,7 @@ dependencies = [
[[package]]
name = "windmill-api-users"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"argon2",
"axum 0.7.9",
@@ -16347,7 +16368,7 @@ dependencies = [
[[package]]
name = "windmill-api-workers"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16361,7 +16382,7 @@ dependencies = [
[[package]]
name = "windmill-api-workspaces"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16369,6 +16390,7 @@ dependencies = [
"http 1.4.0",
"hyper 1.8.1",
"lazy_static",
"magic-crypt",
"regex",
"serde",
"serde_json",
@@ -16391,7 +16413,7 @@ dependencies = [
[[package]]
name = "windmill-audit"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"chrono",
"lazy_static",
@@ -16405,7 +16427,7 @@ dependencies = [
[[package]]
name = "windmill-autoscaling"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -16424,7 +16446,7 @@ dependencies = [
[[package]]
name = "windmill-common"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"aes-gcm",
"anyhow",
@@ -16523,7 +16545,7 @@ dependencies = [
[[package]]
name = "windmill-dep-map"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"chrono",
"itertools 0.14.0",
@@ -16542,7 +16564,7 @@ dependencies = [
[[package]]
name = "windmill-git-sync"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"regex",
"serde",
@@ -16557,7 +16579,7 @@ dependencies = [
[[package]]
name = "windmill-indexer"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"astral-tokio-tar",
@@ -16581,7 +16603,7 @@ dependencies = [
[[package]]
name = "windmill-jseval"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"futures",
@@ -16598,7 +16620,7 @@ dependencies = [
[[package]]
name = "windmill-macros"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"itertools 0.14.0",
"lazy_static",
@@ -16614,7 +16636,7 @@ dependencies = [
[[package]]
name = "windmill-mcp"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-trait",
@@ -16635,7 +16657,7 @@ dependencies = [
[[package]]
name = "windmill-native-triggers"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-trait",
@@ -16666,7 +16688,7 @@ dependencies = [
[[package]]
name = "windmill-oauth"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-oauth2",
@@ -16690,7 +16712,7 @@ dependencies = [
[[package]]
name = "windmill-object-store"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-stream",
@@ -16724,7 +16746,7 @@ dependencies = [
[[package]]
name = "windmill-operator"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"futures",
@@ -16742,7 +16764,7 @@ dependencies = [
[[package]]
name = "windmill-parser"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"convert_case 0.6.0",
"serde",
@@ -16751,7 +16773,7 @@ dependencies = [
[[package]]
name = "windmill-parser-bash"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"lazy_static",
@@ -16763,7 +16785,7 @@ dependencies = [
[[package]]
name = "windmill-parser-csharp"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"serde_json",
@@ -16775,7 +16797,7 @@ dependencies = [
[[package]]
name = "windmill-parser-go"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"gosyn",
@@ -16787,7 +16809,7 @@ dependencies = [
[[package]]
name = "windmill-parser-graphql"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"lazy_static",
@@ -16799,7 +16821,7 @@ dependencies = [
[[package]]
name = "windmill-parser-java"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"serde_json",
@@ -16811,7 +16833,7 @@ dependencies = [
[[package]]
name = "windmill-parser-nu"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"nu-parser",
@@ -16822,7 +16844,7 @@ dependencies = [
[[package]]
name = "windmill-parser-php"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"itertools 0.14.0",
@@ -16833,7 +16855,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"itertools 0.14.0",
@@ -16846,7 +16868,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py-imports"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-recursion",
@@ -16870,7 +16892,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ruby"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"lazy_static",
@@ -16884,7 +16906,7 @@ dependencies = [
[[package]]
name = "windmill-parser-rust"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"convert_case 0.6.0",
@@ -16901,7 +16923,7 @@ dependencies = [
[[package]]
name = "windmill-parser-sql"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"lazy_static",
@@ -16916,7 +16938,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ts"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"lazy_static",
@@ -16935,7 +16957,7 @@ dependencies = [
[[package]]
name = "windmill-parser-yaml"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"serde",
@@ -16946,7 +16968,7 @@ dependencies = [
[[package]]
name = "windmill-queue"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-recursion",
@@ -16983,7 +17005,7 @@ dependencies = [
[[package]]
name = "windmill-runtime-nativets"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"const_format",
@@ -17021,8 +17043,9 @@ dependencies = [
[[package]]
name = "windmill-sql-datatype-parser-wasm"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"getrandom 0.3.4",
"wasm-bindgen",
"wasm-bindgen-test",
"windmill-parser",
@@ -17031,7 +17054,7 @@ dependencies = [
[[package]]
name = "windmill-store"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-recursion",
@@ -17060,7 +17083,7 @@ dependencies = [
[[package]]
name = "windmill-test-utils"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -17083,7 +17106,7 @@ dependencies = [
[[package]]
name = "windmill-trigger"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-trait",
@@ -17116,7 +17139,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-email"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-trait",
@@ -17136,7 +17159,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-gcp"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-trait",
@@ -17170,7 +17193,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-http"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-trait",
@@ -17205,7 +17228,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-kafka"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-trait",
@@ -17228,7 +17251,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-mqtt"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-trait",
@@ -17252,7 +17275,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-nats"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-nats",
@@ -17276,7 +17299,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-postgres"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-trait",
@@ -17311,7 +17334,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-sqs"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-trait",
@@ -17339,7 +17362,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-websocket"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-trait",
@@ -17362,7 +17385,7 @@ dependencies = [
[[package]]
name = "windmill-types"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"bitflags 2.9.4",
@@ -17380,7 +17403,7 @@ dependencies = [
[[package]]
name = "windmill-worker"
version = "1.646.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-once-cell",
@@ -17480,9 +17503,28 @@ dependencies = [
"windmill-queue",
"windmill-runtime-nativets",
"windmill-types",
"windmill-worker-volumes",
"yaml-rust",
]
[[package]]
name = "windmill-worker-volumes"
version = "1.651.1"
dependencies = [
"bytes",
"futures",
"lazy_static",
"md-5 0.10.6",
"object_store",
"regex",
"serde",
"serde_json",
"tempfile",
"tokio",
"tracing",
"windmill-common",
]
[[package]]
name = "windows"
version = "0.56.0"
@@ -18350,18 +18392,18 @@ dependencies = [
[[package]]
name = "zerocopy"
version = "0.8.39"
version = "0.8.40"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "db6d35d663eadb6c932438e763b262fe1a70987f9ae936e60158176d710cae4a"
checksum = "a789c6e490b576db9f7e6b6d661bcc9799f7c0ac8352f56ea20193b2681532e5"
dependencies = [
"zerocopy-derive",
]
[[package]]
name = "zerocopy-derive"
version = "0.8.39"
version = "0.8.40"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4122cd3169e94605190e77839c9a40d40ed048d305bfdc146e7df40ab0f3e517"
checksum = "f65c489a7071a749c849713807783f70672b28094011623e200cb86dcb835953"
dependencies = [
"proc-macro2",
"quote",
@@ -18456,9 +18498,9 @@ dependencies = [
[[package]]
name = "zlib-rs"
version = "0.6.2"
version = "0.6.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c745c48e1007337ed136dc99df34128b9faa6ed542d80a1c673cf55a6d7236c8"
checksum = "3be3d40e40a133f9c916ee3f9f4fa2d9d63435b5fbe1bfc6d9dae0aa0ada1513"
[[package]]
name = "zstd"

View File

@@ -1,6 +1,6 @@
[package]
name = "windmill"
version = "1.646.0"
version = "1.651.1"
authors.workspace = true
edition.workspace = true
@@ -70,13 +70,14 @@ members = [
"./parsers/windmill-parser-py-imports",
"./parsers/windmill-sql-datatype-parser-wasm",
"./parsers/windmill-parser-yaml", "windmill-macros", "parsers/windmill-parser-nu",
"./windmill-worker-volumes",
"./windmill-test-utils",
"./windmill-api-integration-tests",
]
exclude = ["./windmill-duckdb-ffi-internal"]
[workspace.package]
version = "1.646.0"
version = "1.651.1"
authors = ["Ruben Fiszel <ruben@windmill.dev>"]
edition = "2021"
@@ -250,10 +251,13 @@ reqwest.workspace = true
windmill-queue = { workspace = true, features = ["failpoints"] }
windmill-dep-map.workspace = true
windmill-test-utils.workspace = true
windmill-worker-volumes.workspace = true
windmill-types.workspace = true
axum.workspace = true
serde.workspace = true
windmill-api-client.workspace = true
tempfile.workspace = true
tar.workspace = true
windmill-parser-ts.workspace = true
rumqttc.workspace = true
rdkafka.workspace = true
@@ -267,6 +271,7 @@ aws-credential-types.workspace = true
windmill-api = { path = "./windmill-api", default-features = false }
windmill-queue = { path = "./windmill-queue" }
windmill-worker = { path = "./windmill-worker" }
windmill-worker-volumes = { path = "./windmill-worker-volumes" }
windmill-dep-map = { path = "./windmill-dep-map" }
windmill-types = { path = "./windmill-types" }
windmill-common = { path = "./windmill-common", default-features = false }
@@ -351,7 +356,7 @@ tower-cookies = "^0.10"
serde = "=1.0.219"
serde_json = { version = "^1", features = ["preserve_order", "raw_value"] }
serde_yml = "0.0.12"
uuid = { version = "^1", features = ["serde", "v4"] }
uuid = { version = "^1", features = ["serde", "v4", "js"] }
thiserror = "^2"
anyhow = "^1"
chrono = { version = "^0.4", features = ["serde"] }
@@ -439,6 +444,7 @@ base64 = "^0.22.1"
base32 = "^0"
hmac = "0.12.1"
sha2 = "0.10.6"
md-5 = "0.10.6"
sha1 = "0.10.6"
sqlx = { version = "0.8.0", features = [
"macros",
@@ -512,7 +518,7 @@ nu-parser = { version = "0.101.0", default-features = false }
globset = "0.4.16"
croner = "2.2.0"
rmcp = { version = "=0.15.0", features = ["client", "transport-streamable-http-client", "transport-streamable-http-client-reqwest"] }
rquickjs = { version = "0.8", features = ["futures", "parallel", "macro"] }
rquickjs = { version = "0.11", features = ["futures", "parallel", "macro"] }
process-wrap = { version = "8.2.1", features = ["tokio1"] }
systemstat = "0.2.4"

View File

@@ -1 +1 @@
8ffae1f43b31dc8136714fa612d22b6301773e27
c3c543f4c60a8c4dfe0d912c79a051376fb091a9

View File

@@ -0,0 +1 @@
DROP TABLE IF EXISTS volume;

View File

@@ -0,0 +1,22 @@
-- Add 'volume' to the asset_kind enum
ALTER TYPE asset_kind ADD VALUE IF NOT EXISTS 'volume';
-- Volume metadata table
CREATE TABLE volume (
workspace_id VARCHAR(50) NOT NULL REFERENCES workspace(id) ON DELETE CASCADE,
name VARCHAR(255) NOT NULL,
size_bytes BIGINT NOT NULL DEFAULT 0,
file_count INTEGER NOT NULL DEFAULT 0,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
created_by VARCHAR(255) NOT NULL,
updated_at TIMESTAMPTZ,
updated_by VARCHAR(255),
description TEXT NOT NULL DEFAULT '',
lease_until TIMESTAMPTZ,
leased_by VARCHAR(255),
last_used_at TIMESTAMPTZ,
extra_perms JSONB NOT NULL DEFAULT '{}',
PRIMARY KEY (workspace_id, name)
);
CREATE INDEX idx_volume_last_used ON volume(workspace_id, last_used_at);

View File

@@ -0,0 +1 @@
DROP INDEX IF EXISTS ix_v2_job_completed_failure_workspace;

View File

@@ -0,0 +1,7 @@
-- Partial index for fast failure/canceled filtering on the runs page.
-- When failures are sparse (<1%) this avoids scanning millions of successful jobs.
-- The query orders by completed_at DESC (switched from created_at when success=false),
-- so this index provides both filtering and ordering in a single scan.
CREATE INDEX IF NOT EXISTS ix_v2_job_completed_failure_workspace
ON v2_job_completed (workspace_id, completed_at DESC)
WHERE status IN ('failure', 'canceled');

View File

@@ -0,0 +1 @@
DROP TABLE IF EXISTS token_expiry_notification;

View File

@@ -0,0 +1,8 @@
-- Tracks pending expiry notifications: row exists = not yet notified.
-- Deleted once the notification is sent. Orphaned rows are harmless (filtered out by the join).
CREATE TABLE token_expiry_notification (
token VARCHAR(255) PRIMARY KEY,
expiration TIMESTAMPTZ NOT NULL
);
CREATE INDEX idx_token_expiry_notification_expiration ON token_expiry_notification (expiration);

View File

@@ -0,0 +1 @@
ALTER TABLE worker_ping DROP COLUMN IF EXISTS uses_batch_http_pull;

View File

@@ -0,0 +1 @@
ALTER TABLE worker_ping ADD COLUMN IF NOT EXISTS uses_batch_http_pull BOOLEAN NOT NULL DEFAULT false;

View File

@@ -1,4 +1,4 @@
use std::collections::BTreeMap;
use std::collections::{BTreeMap, HashSet};
use sqlparser::{
ast::{
@@ -45,6 +45,10 @@ struct AssetCollector {
var_identifiers: BTreeMap<String, (AssetKind, String)>,
// e.g USE dl;
currently_used_asset: Option<(AssetKind, String)>,
// CTE names in scope (stack for nested queries)
cte_name_stack: Vec<HashSet<String>>,
// Locally created tables (not attached to an asset)
local_table_names: HashSet<String>,
}
impl AssetCollector {
@@ -54,9 +58,30 @@ impl AssetCollector {
current_access_type_stack: Vec::with_capacity(8),
var_identifiers: BTreeMap::new(),
currently_used_asset: None,
cte_name_stack: Vec::new(),
local_table_names: HashSet::new(),
}
}
/// If the name resolves to an attached asset, record it. Otherwise, register it as a local
/// table/view so that subsequent references are not mistakenly attributed to the active asset.
fn track_table_definition(&mut self, name: &ObjectName) {
if let Some(asset) = self.get_associated_asset_from_obj_name(name, Some(W)) {
self.assets.push(asset);
} else if let Some(simple_name) = get_trivial_obj_name(name) {
self.local_table_names.insert(simple_name.to_lowercase());
}
}
fn is_locally_defined(&self, name: &str) -> bool {
let name_lower = name.to_lowercase();
self.local_table_names.contains(&name_lower)
|| self
.cte_name_stack
.iter()
.any(|set| set.contains(&name_lower))
}
// Detect when we do 'a.b' and 'a' is associated with an asset in var_identifiers
// Or when we access 'b' and we did USE a;
fn get_associated_asset_from_obj_name(
@@ -72,6 +97,14 @@ impl AssetCollector {
return None;
}
if name.0.len() == 1 {
if let Some(ident) = name.0.first().and_then(|id| id.as_ident()) {
if self.is_locally_defined(&ident.value) {
return None;
}
}
}
if name.0.len() == 1 || name.0.len() == 2 {
if name
.0
@@ -452,6 +485,7 @@ impl Visitor for AssetCollector {
) -> std::ops::ControlFlow<Self::Break> {
match statement {
sqlparser::ast::Statement::Query(q) => {
self.cte_name_stack.push(collect_cte_names(q));
if let Some(select) = q.body.as_select() {
// First, handle table references (adds table-level assets)
for t in &select.from {
@@ -612,17 +646,11 @@ impl Visitor for AssetCollector {
}
sqlparser::ast::Statement::CreateTable(create_table) => {
if let Some(asset) =
self.get_associated_asset_from_obj_name(&create_table.name, Some(W))
{
self.assets.push(asset);
}
self.track_table_definition(&create_table.name);
}
sqlparser::ast::Statement::CreateView { name, .. } => {
if let Some(asset) = self.get_associated_asset_from_obj_name(name, Some(W)) {
self.assets.push(asset);
}
self.track_table_definition(name);
}
sqlparser::ast::Statement::Copy { target: CopyTarget::File { filename }, .. } => {
@@ -672,16 +700,20 @@ impl Visitor for AssetCollector {
fn post_visit_statement(
&mut self,
_statement: &sqlparser::ast::Statement,
statement: &sqlparser::ast::Statement,
) -> std::ops::ControlFlow<Self::Break> {
if matches!(statement, sqlparser::ast::Statement::Query(_)) {
self.cte_name_stack.pop();
}
std::ops::ControlFlow::Continue(())
}
fn pre_visit_query(
&mut self,
_query: &sqlparser::ast::Query,
query: &sqlparser::ast::Query,
) -> std::ops::ControlFlow<Self::Break> {
self.current_access_type_stack.push(R);
self.cte_name_stack.push(collect_cte_names(query));
std::ops::ControlFlow::Continue(())
}
@@ -690,12 +722,22 @@ impl Visitor for AssetCollector {
_query: &sqlparser::ast::Query,
) -> std::ops::ControlFlow<Self::Break> {
self.current_access_type_stack.pop();
self.cte_name_stack.pop();
std::ops::ControlFlow::Continue(())
}
// We do not use pre_visit_relation because we cannot know if an ObjectName is a table or a function
}
fn collect_cte_names(query: &sqlparser::ast::Query) -> HashSet<String> {
query.with.as_ref().map_or_else(HashSet::new, |with| {
with.cte_tables
.iter()
.map(|cte| cte.alias.name.value.to_lowercase())
.collect()
})
}
fn is_read_fn(fname: &str) -> bool {
fname.eq_ignore_ascii_case("read_parquet")
|| fname.eq_ignore_ascii_case("read_csv")
@@ -1509,6 +1551,235 @@ mod tests {
assert!(result[0].columns.is_none());
}
#[test]
fn test_sql_asset_parser_cte_not_treated_as_asset() {
let input = r#"
ATTACH 'ducklake://my_dl' AS dl;
USE dl;
WITH tmp AS (SELECT 1 AS x)
SELECT * FROM tmp;
SELECT * FROM real_table;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "my_dl/real_table".to_string(),
access_type: Some(R),
columns: None
},])
);
}
#[test]
fn test_sql_asset_parser_cte_scope_does_not_leak() {
let input = r#"
ATTACH 'ducklake://my_dl' AS dl;
USE dl;
WITH tmp AS (SELECT 1) SELECT * FROM tmp;
SELECT * FROM tmp;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "my_dl/tmp".to_string(),
access_type: Some(R),
columns: None
},])
);
}
#[test]
fn test_sql_asset_parser_multiple_ctes() {
let input = r#"
ATTACH 'ducklake://my_dl' AS dl;
USE dl;
WITH cte1 AS (SELECT 1), cte2 AS (SELECT 2)
SELECT * FROM cte1 JOIN cte2 ON true;
SELECT * FROM real_table;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "my_dl/real_table".to_string(),
access_type: Some(R),
columns: None
},])
);
}
#[test]
fn test_sql_asset_parser_local_create_table_overrides_asset() {
let input = r#"
CREATE TABLE local_tbl (id INT);
ATTACH 'ducklake://my_dl' AS dl;
USE dl;
SELECT * FROM local_tbl;
SELECT * FROM asset_table;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "my_dl/asset_table".to_string(),
access_type: Some(R),
columns: None
},])
);
}
#[test]
fn test_sql_asset_parser_create_table_with_use_is_still_asset() {
let input = r#"
ATTACH 'ducklake' AS dl; USE dl;
CREATE TABLE friends (
name text,
age int
);
INSERT INTO friends VALUES ($name, $age);
SELECT * FROM friends;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "main/friends".to_string(),
access_type: Some(RW),
columns: None
},])
);
}
#[test]
fn test_sql_asset_parser_local_create_view_overrides_asset() {
let input = r#"
CREATE VIEW my_view AS SELECT 1;
ATTACH 'ducklake://my_dl' AS dl;
USE dl;
SELECT * FROM my_view;
SELECT * FROM asset_table;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "my_dl/asset_table".to_string(),
access_type: Some(R),
columns: None
},])
);
}
#[test]
fn test_sql_asset_parser_create_view_with_use_is_still_asset() {
let input = r#"
ATTACH 'ducklake://my_dl' AS dl;
USE dl;
CREATE VIEW my_view AS SELECT 1;
SELECT * FROM my_view;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "my_dl/my_view".to_string(),
access_type: Some(RW),
columns: None
},])
);
}
#[test]
fn test_sql_asset_parser_cte_mixed_with_asset_tables() {
let input = r#"
ATTACH 'ducklake://my_dl' AS dl;
USE dl;
WITH tmp AS (SELECT 1 AS x)
SELECT * FROM tmp JOIN real_table ON true;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "my_dl/real_table".to_string(),
access_type: Some(R),
columns: None
},])
);
}
#[test]
fn test_sql_asset_parser_local_table_insert_and_select() {
let input = r#"
CREATE TABLE staging (id INT, val TEXT);
ATTACH 'ducklake://my_dl' AS dl;
USE dl;
INSERT INTO staging VALUES (1, 'a');
SELECT * FROM staging;
INSERT INTO real_table VALUES (2, 'b');
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "my_dl/real_table".to_string(),
access_type: Some(W),
columns: None
},])
);
}
#[test]
fn test_sql_asset_parser_qualified_ref_bypasses_local() {
// Even if 'tbl' is local, 'dl.tbl' is an explicit asset reference
let input = r#"
CREATE TABLE tbl (id INT);
ATTACH 'ducklake://my_dl' AS dl;
SELECT * FROM dl.tbl;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "my_dl/tbl".to_string(),
access_type: Some(R),
columns: None
},])
);
}
#[test]
fn test_sql_asset_parser_cte_case_insensitive() {
let input = r#"
ATTACH 'ducklake://my_dl' AS dl;
USE dl;
WITH MyTable AS (SELECT 1)
SELECT * FROM mytable;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "my_dl".to_string(),
access_type: None,
columns: None
},])
);
}
#[test]
fn test_sql_asset_parser_s3_read_csv_columns() {
let input = r#"

View File

@@ -58,16 +58,23 @@ pub fn parse_oracledb_sig(code: &str) -> anyhow::Result<MainArgSignature> {
}
pub fn parse_pgsql_sig(code: &str) -> anyhow::Result<MainArgSignature> {
let (sig, _) = parse_pgsql_sig_with_typed_schema(code)?;
Ok(sig)
}
pub fn parse_pgsql_sig_with_typed_schema(code: &str) -> anyhow::Result<(MainArgSignature, bool)> {
let parsed = parse_pg_file(&code)?;
if let Some(x) = parsed {
let args = x;
Ok(MainArgSignature {
star_args: false,
star_kwargs: false,
args,
no_main_func: None,
has_preprocessor: None,
})
if let Some((args, typed_schema)) = parsed {
Ok((
MainArgSignature {
star_args: false,
star_kwargs: false,
args,
no_main_func: None,
has_preprocessor: None,
},
typed_schema,
))
} else {
Err(anyhow!("Error parsing sql".to_string()))
}
@@ -216,7 +223,7 @@ lazy_static::lazy_static! {
static ref RE_ARG_MYSQL: Regex = Regex::new(r#"(?m)^-- \? (\w+) \((\w+)\)(?: ?\= ?(.+))? *(?:\r|\n|$)"#).unwrap();
pub static ref RE_ARG_MYSQL_NAMED: Regex = Regex::new(r#"(?m)^-- :([a-z_][a-z0-9_]*) \((\w+(?:\([\w, ]+\))?)\)(?: ?\= ?(.+))? *(?:\r|\n|$)"#).unwrap();
static ref RE_ARG_PGSQL: Regex = Regex::new(r#"(?m)^-- \$(\d+) (\w+)(?: ?\= ?(.+))? *(?:\r|\n|$)"#).unwrap();
static ref RE_ARG_PGSQL: Regex = Regex::new(r#"(?m)^-- \$(\d+) (\w+)(?: \(([A-Za-z0-9_\[\]]+)\))?(?: ?\= ?(.+))? *(?:\r|\n|$)"#).unwrap();
// -- @name (type) = default
static ref RE_ARG_BIGQUERY: Regex = Regex::new(r#"(?m)^-- @(\w+) \((\w+(?:\[\])?)\)(?: ?\= ?(.+))? *(?:\r|\n|$)"#).unwrap();
@@ -478,21 +485,62 @@ pub fn parse_pg_statement_arg_indices(code: &str) -> HashSet<i32> {
arg_indices
}
fn parse_pg_file(code: &str) -> anyhow::Result<Option<Vec<Arg>>> {
fn parse_pg_file(code: &str) -> anyhow::Result<Option<(Vec<Arg>, bool)>> {
let mut args = vec![];
// Track which args have explicit types in declaration comments
let mut explicitly_typed_args: HashSet<i32> = HashSet::new();
// First pass: collect args from declaration comments (-- $1 argName (type))
for cap in RE_ARG_PGSQL.captures_iter(code) {
let idx = cap
.get(1)
.and_then(|x| x.as_str().parse::<i32>().ok())
.ok_or_else(|| anyhow!("Impossible to parse arg digit"))?;
let name = cap.get(2).map(|x| x.as_str().to_string()).unwrap();
let explicit_type = cap.get(3).map(|x| x.as_str().to_string().to_lowercase());
let default = cap.get(4).map(|x| x.as_str().to_string());
let has_default = default.is_some();
if let Some(typ) = explicit_type {
// If explicitly typed, use that type and don't infer from usage
explicitly_typed_args.insert(idx);
let parsed_typ = parse_pg_typ(typ.as_str());
let parsed_default = default.and_then(|x| parsed_default(&parsed_typ, x));
args.push(Arg {
name,
typ: parsed_typ,
default: parsed_default,
otyp: Some(typ),
has_default,
oidx: Some(idx),
});
}
}
// Second pass: infer types from usage for non-explicitly-typed args
let mut hm: HashMap<i32, String> = HashMap::new();
for cap in RE_CODE_PGSQL.captures_iter(code) {
let idx = cap
.get(1)
.and_then(|x| x.as_str().parse::<i32>().ok())
.ok_or_else(|| anyhow!("Impossible to parse arg digit"))?;
// Skip if this arg was explicitly typed in declaration
if explicitly_typed_args.contains(&idx) {
continue;
}
let typ = cap
.get(2)
.map(|cap| transform_types_with_spaces(&cap, &code))
.unwrap_or("text");
hm.insert(
cap.get(1)
.and_then(|x| x.as_str().parse::<i32>().ok())
.ok_or_else(|| anyhow!("Impossible to parse arg digit"))?,
typ.to_string(),
);
hm.insert(idx, typ.to_string());
}
// Add inferred args
for (i, v) in hm.iter() {
let typ = v.to_lowercase();
args.push(Arg {
@@ -504,19 +552,28 @@ fn parse_pg_file(code: &str) -> anyhow::Result<Option<Vec<Arg>>> {
oidx: Some(*i),
});
}
// Sort by index
args.sort_by(|a, b| a.oidx.unwrap().cmp(&b.oidx.unwrap()));
// Third pass: update names and defaults for inferred args
for cap in RE_ARG_PGSQL.captures_iter(code) {
let i = cap
.get(1)
.and_then(|x| x.as_str().parse::<i32>().ok())
.map(|x| x);
// Skip explicitly typed args (already handled)
if i.is_some_and(|idx| explicitly_typed_args.contains(&idx)) {
continue;
}
if let Some(arg_pos) = args
.iter()
.position(|x| i.is_some_and(|i| x.oidx.unwrap() == i))
{
let name = cap.get(2).map(|x| x.as_str().to_string()).unwrap();
let default = cap.get(3).map(|x| x.as_str().to_string());
let default = cap.get(4).map(|x| x.as_str().to_string());
let has_default = default.is_some();
let oarg = args[arg_pos].clone();
let parsed_default = default.and_then(|x| parsed_default(&oarg.typ, x));
@@ -532,8 +589,10 @@ fn parse_pg_file(code: &str) -> anyhow::Result<Option<Vec<Arg>>> {
}
}
let typed_schema = !explicitly_typed_args.is_empty();
args.append(&mut parse_sql_sanitized_interpolation(code));
Ok(Some(args))
Ok(Some((args, typed_schema)))
}
// The regex doesn't parse types with space such as "character varying"
@@ -1306,4 +1365,186 @@ SELECT * FROM table_name WHERE thing = :name4;
Ok(())
}
#[test]
fn test_parse_pgsql_explicit_type_at_declaration() -> anyhow::Result<()> {
let code = r#"
-- $1 user_id (bigint)
-- $2 email
SELECT * FROM users WHERE id = $1 AND email = $2::text;
"#;
assert_eq!(
parse_pgsql_sig(code)?,
MainArgSignature {
star_args: false,
star_kwargs: false,
args: vec![
Arg {
otyp: Some("bigint".to_string()),
name: "user_id".to_string(),
typ: Typ::Int,
default: None,
has_default: false,
oidx: Some(1),
},
Arg {
otyp: Some("text".to_string()),
name: "email".to_string(),
typ: Typ::Str(None),
default: None,
has_default: false,
oidx: Some(2),
},
],
no_main_func: None,
has_preprocessor: None
}
);
Ok(())
}
#[test]
fn test_parse_pgsql_explicit_type_with_default() -> anyhow::Result<()> {
let code = r#"
-- $1 limit (integer) = 10
-- $2 offset (bigint) = 0
SELECT * FROM users LIMIT $1 OFFSET $2;
"#;
assert_eq!(
parse_pgsql_sig(code)?,
MainArgSignature {
star_args: false,
star_kwargs: false,
args: vec![
Arg {
otyp: Some("integer".to_string()),
name: "limit".to_string(),
typ: Typ::Int,
default: Some(json!(10)),
has_default: true,
oidx: Some(1),
},
Arg {
otyp: Some("bigint".to_string()),
name: "offset".to_string(),
typ: Typ::Int,
default: Some(json!(0)),
has_default: true,
oidx: Some(2),
},
],
no_main_func: None,
has_preprocessor: None
}
);
Ok(())
}
#[test]
fn test_parse_pgsql_mixed_explicit_and_inferred() -> anyhow::Result<()> {
let code = r#"
-- $1 user_id (bigint)
-- $2 status
-- $3 created_at (timestamptz)
SELECT * FROM users
WHERE id = $1
AND status = $2::text
AND created_at > $3;
"#;
assert_eq!(
parse_pgsql_sig(code)?,
MainArgSignature {
star_args: false,
star_kwargs: false,
args: vec![
Arg {
otyp: Some("bigint".to_string()),
name: "user_id".to_string(),
typ: Typ::Int,
default: None,
has_default: false,
oidx: Some(1),
},
Arg {
otyp: Some("text".to_string()),
name: "status".to_string(),
typ: Typ::Str(None),
default: None,
has_default: false,
oidx: Some(2),
},
Arg {
otyp: Some("timestamptz".to_string()),
name: "created_at".to_string(),
typ: Typ::Datetime,
default: None,
has_default: false,
oidx: Some(3),
},
],
no_main_func: None,
has_preprocessor: None
}
);
Ok(())
}
#[test]
fn test_parse_pgsql_explicit_type_array() -> anyhow::Result<()> {
let code = r#"
-- $1 ids (bigint[])
SELECT * FROM users WHERE id = ANY($1);
"#;
assert_eq!(
parse_pgsql_sig(code)?,
MainArgSignature {
star_args: false,
star_kwargs: false,
args: vec![Arg {
otyp: Some("bigint[]".to_string()),
name: "ids".to_string(),
typ: Typ::List(Box::new(Typ::Int)),
default: None,
has_default: false,
oidx: Some(1),
},],
no_main_func: None,
has_preprocessor: None
}
);
Ok(())
}
#[test]
fn test_parse_pgsql_explicit_type_does_not_infer_from_usage() -> anyhow::Result<()> {
// Even though $1 is used as ::integer in the query,
// the explicit type (text) should take precedence
let code = r#"
-- $1 value (text)
SELECT $1::integer;
"#;
assert_eq!(
parse_pgsql_sig(code)?,
MainArgSignature {
star_args: false,
star_kwargs: false,
args: vec![Arg {
otyp: Some("text".to_string()),
name: "value".to_string(),
typ: Typ::Str(None),
default: None,
has_default: false,
oidx: Some(1),
},],
no_main_func: None,
has_preprocessor: None
}
);
Ok(())
}
}

View File

@@ -18,6 +18,7 @@ pub enum AssetKind {
Resource,
Ducklake,
DataTable,
Volume,
}
#[derive(Serialize, Debug, PartialEq, Clone)]
@@ -148,4 +149,5 @@ pub const ASSET_KINDS: &[(&str, AssetKind)] = &[
("$res:", AssetKind::Resource),
("ducklake://", AssetKind::Ducklake),
("datatable://", AssetKind::DataTable),
("volume://", AssetKind::Volume),
];

View File

@@ -126,6 +126,7 @@ pub fn json_to_typ(js: &Value, precise_arrays: bool) -> Typ {
pub fn to_snake_case(s: &str) -> String {
s.with_boundaries(&Boundary::defaults())
.without_boundaries(&Boundary::letter_digit())
.without_boundaries(&[Boundary::DigitLower])
.to_case(Case::Snake)
}
@@ -138,8 +139,8 @@ mod test {
assert_eq!("s3", to_snake_case("S3"));
assert_eq!("s3", to_snake_case("s3"));
assert_eq!("s3_object", to_snake_case("S3Object"));
assert_eq!("s3_object", to_snake_case("S3object"));
assert_eq!("s3_object", to_snake_case("s3object"));
assert_eq!("s3object", to_snake_case("S3object"));
assert_eq!("s3object", to_snake_case("s3object"));
assert_eq!("abc", to_snake_case("ABC"));
assert_eq!("aa_bc", to_snake_case("AaBC"));
assert_eq!("a_b_c", to_snake_case("A_B_C"));
@@ -181,6 +182,9 @@ mod test {
fn test_mixed_case_with_numbers() {
assert_eq!(to_snake_case("testCase1"), "test_case1");
assert_eq!(to_snake_case("Test123Case"), "test123_case");
// digit followed by lowercase should NOT insert underscore (issue #7934)
assert_eq!(to_snake_case("Connect2allApi"), "connect2all_api");
assert_eq!(to_snake_case("Foo2barApi"), "foo2bar_api");
}
#[test]

View File

@@ -16,4 +16,7 @@ wasm-bindgen-test.workspace = true
[dependencies]
windmill-parser.workspace = true
windmill-parser-sql.workspace = true
wasm-bindgen.workspace = true
wasm-bindgen.workspace = true
# getrandom 0.3 is pulled in transitively by rand 0.9 (via windmill-types).
# It requires the "wasm_js" feature to work on wasm32-unknown-unknown.
getrandom3 = { package = "getrandom", version = "0.3", features = ["wasm_js"] }

View File

@@ -38,11 +38,11 @@ use windmill_common::{
agent_workers::AgentConfig,
global_settings::{
APP_WORKSPACED_ROUTE_SETTING, BASE_URL_SETTING, BUNFIG_INSTALL_SCOPES_SETTING,
CRITICAL_ALERTS_ON_DB_OVERSIZE_SETTING, CRITICAL_ALERT_MUTE_UI_SETTING,
CRITICAL_ERROR_CHANNELS_SETTING, CUSTOM_TAGS_SETTING, DEFAULT_TAGS_PER_WORKSPACE_SETTING,
DEFAULT_TAGS_WORKSPACES_SETTING, EMAIL_DOMAIN_SETTING, ENV_SETTINGS,
EXPOSE_DEBUG_METRICS_SETTING, EXPOSE_METRICS_SETTING, EXTRA_PIP_INDEX_URL_SETTING,
HUB_API_SECRET_SETTING, HUB_BASE_URL_SETTING, INDEXER_SETTING,
CRITICAL_ALERTS_ON_DB_OVERSIZE_SETTING, CRITICAL_ALERTS_ON_TOKEN_EXPIRY_SETTING,
CRITICAL_ALERT_MUTE_UI_SETTING, CRITICAL_ERROR_CHANNELS_SETTING, CUSTOM_TAGS_SETTING,
DEFAULT_TAGS_PER_WORKSPACE_SETTING, DEFAULT_TAGS_WORKSPACES_SETTING, EMAIL_DOMAIN_SETTING,
ENV_SETTINGS, EXPOSE_DEBUG_METRICS_SETTING, EXPOSE_METRICS_SETTING,
EXTRA_PIP_INDEX_URL_SETTING, HUB_API_SECRET_SETTING, HUB_BASE_URL_SETTING, INDEXER_SETTING,
INSTANCE_PYTHON_VERSION_SETTING, JOB_DEFAULT_TIMEOUT_SECS_SETTING, JOB_ISOLATION_SETTING,
JWT_SECRET_SETTING, KEEP_JOB_DIR_SETTING, LICENSE_KEY_SETTING, MAVEN_REPOS_SETTING,
MAVEN_SETTINGS_XML_SETTING, MONITOR_LOGS_ON_OBJECT_STORE_SETTING, NO_DEFAULT_MAVEN_SETTING,
@@ -61,8 +61,9 @@ use windmill_common::{
MODE_AND_ADDONS,
},
worker::{
is_native_mode_from_env, reload_custom_tags_setting, Connection, HUB_CACHE_DIR,
HUB_RT_CACHE_DIR, NATIVE_MODE_RESOLVED, TMP_DIR, TMP_LOGS_DIR, WORKER_GROUP,
is_native_mode_from_env, reload_custom_tags_setting, Connection, HttpClient, HUB_CACHE_DIR,
HUB_RT_CACHE_DIR, NATIVE_MODE_RESOLVED, TMP_LOGS_DIR, USES_BATCH_HTTP_PULL, WINDMILL_DIR,
WORKER_GROUP,
},
KillpillSender, DEFAULT_HUB_BASE_URL, METRICS_ENABLED,
};
@@ -99,10 +100,10 @@ use crate::monitor::{
load_tag_per_workspace_enabled, load_tag_per_workspace_workspaces, monitor_db,
reload_app_workspaced_route_setting, reload_base_url_setting,
reload_bunfig_install_scopes_setting, reload_critical_alert_mute_ui_setting,
reload_critical_error_channels_setting, reload_extra_pip_index_url_setting,
reload_hub_api_secret_setting, reload_hub_base_url_setting, reload_job_default_timeout_setting,
reload_job_isolation_setting, reload_jwt_secret_setting, reload_license_key,
reload_npm_config_registry_setting, reload_otel_tracing_proxy_setting,
reload_critical_alerts_on_token_expiry_setting, reload_critical_error_channels_setting,
reload_extra_pip_index_url_setting, reload_hub_api_secret_setting, reload_hub_base_url_setting,
reload_job_default_timeout_setting, reload_job_isolation_setting, reload_jwt_secret_setting,
reload_license_key, reload_npm_config_registry_setting, reload_otel_tracing_proxy_setting,
reload_pip_index_url_setting, reload_retention_period_setting, reload_scim_token_setting,
reload_smtp_config, reload_uv_index_strategy_setting, reload_worker_config, MonitorIteration,
};
@@ -238,8 +239,8 @@ async fn cache_hub_scripts(file_path: Option<String>) -> anyhow::Result<()> {
)
})?;
create_dir_all(HUB_CACHE_DIR)?;
create_dir_all(BUN_BUNDLE_CACHE_DIR)?;
create_dir_all(&*HUB_CACHE_DIR)?;
create_dir_all(&*BUN_BUNDLE_CACHE_DIR)?;
for path in paths.values() {
tracing::info!("Caching hub script at {path}");
@@ -249,7 +250,7 @@ async fn cache_hub_scripts(file_path: Option<String>) -> anyhow::Result<()> {
.as_ref()
.is_some_and(|x| x == &ScriptLang::Deno)
{
let job_dir = format!("{}/cache_init/{}", TMP_DIR, Uuid::new_v4());
let job_dir = format!("{}/cache_init/{}", *WINDMILL_DIR, Uuid::new_v4());
create_dir_all(&job_dir)?;
let _ = windmill_worker::generate_deno_lock(
&Uuid::nil(),
@@ -267,7 +268,7 @@ async fn cache_hub_scripts(file_path: Option<String>) -> anyhow::Result<()> {
tokio::fs::remove_dir_all(job_dir).await?;
} else if res.language.as_ref().is_some_and(|x| x == &ScriptLang::Bun) {
let job_id = Uuid::new_v4();
let job_dir = format!("{}/cache_init/{}", TMP_DIR, job_id);
let job_dir = format!("{}/cache_init/{}", *WINDMILL_DIR, job_id);
create_dir_all(&job_dir)?;
if let Some(lock) = res.lockfile {
let _ = windmill_worker::prepare_job_dir(&lock, &job_dir).await?;
@@ -384,9 +385,9 @@ async fn cache_hub_resource_types() -> anyhow::Result<()> {
println!("Fetched {} resource types from hub", resource_types.len());
create_dir_all(HUB_RT_CACHE_DIR)?;
create_dir_all(&*HUB_RT_CACHE_DIR)?;
let cache_path = format!("{}/{}", HUB_RT_CACHE_DIR, HUB_RT_CACHE_FILE);
let cache_path = format!("{}/{}", *HUB_RT_CACHE_DIR, HUB_RT_CACHE_FILE);
let content = serde_json::to_string_pretty(&resource_types)
.with_context(|| "Failed to serialize resource types")?;
@@ -398,7 +399,7 @@ async fn cache_hub_resource_types() -> anyhow::Result<()> {
}
pub async fn sync_cached_resource_types(db: &sqlx::Pool<sqlx::Postgres>) -> anyhow::Result<()> {
let cache_path = format!("{}/{}", HUB_RT_CACHE_DIR, HUB_RT_CACHE_FILE);
let cache_path = format!("{}/{}", *HUB_RT_CACHE_DIR, HUB_RT_CACHE_FILE);
if tokio::fs::metadata(&cache_path).await.is_err() {
tracing::info!(
@@ -920,6 +921,20 @@ Windmill Community Edition {GIT_VERSION}
default_base_internal_url.clone()
};
// BATCH_PULL_URL: explicit URL for native workers to pull jobs via HTTP.
// In standalone mode (server_mode=true), defaults to the local server.
let batch_pull_url: Option<String> = if is_native_mode_from_env() {
if let Ok(url) = std::env::var("BATCH_PULL_URL") {
Some(url)
} else if server_mode {
Some(default_base_internal_url.clone())
} else {
None
}
} else {
None
};
initial_load(
&conn,
killpill_tx.clone(),
@@ -969,7 +984,7 @@ Windmill Community Edition {GIT_VERSION}
DirBuilder::new()
.recursive(true)
.create("/tmp/windmill")
.create(&*WINDMILL_DIR)
.expect("could not create initial server dir");
#[cfg(feature = "tantivy")]
@@ -1130,6 +1145,30 @@ Windmill Community Edition {GIT_VERSION}
)?;
let mut workers = vec![];
// For native workers, create a self-signed JWT for batch pulling via HTTP.
// Enabled when BATCH_PULL_URL is set (explicitly or auto-detected in standalone mode).
let batch_pull_client = if let Some(ref pull_url) = batch_pull_url {
match create_native_batch_pull_client(pull_url).await {
Ok(client) => {
tracing::info!(
"Native batch pull client created for HTTP pull at {}",
pull_url
);
USES_BATCH_HTTP_PULL
.store(true, std::sync::atomic::Ordering::Relaxed);
Some(client)
}
Err(e) => {
tracing::warn!(
"Failed to create native batch pull client, falling back to SQL pull: {e:#}"
);
None
}
}
} else {
None
};
for i in 0..num_workers {
let suffix = if i == 0 && first_suffix.is_some() {
first_suffix.as_ref().unwrap().clone()
@@ -1153,6 +1192,7 @@ Windmill Community Edition {GIT_VERSION}
WORKER_GROUP.as_str(),
&suffix,
),
batch_pull_client: batch_pull_client.clone(),
};
workers.push(worker_conn);
}
@@ -1717,6 +1757,11 @@ async fn process_notify_event(
tracing::error!(error = %e, "Could not reload critical alert UI setting");
}
}
CRITICAL_ALERTS_ON_TOKEN_EXPIRY_SETTING => {
if let Err(e) = reload_critical_alerts_on_token_expiry_setting(conn).await {
tracing::error!(error = %e, "Could not reload critical alerts on token expiry setting");
}
}
"workspace_telemetry_enabled" => {
// Read the new value from the database and log it
let enabled = sqlx::query_scalar!(
@@ -1761,6 +1806,7 @@ fn display_config(envs: &[&str]) {
pub struct WorkerConn {
conn: Connection,
worker_name: String,
batch_pull_client: Option<HttpClient>,
}
pub async fn run_workers(
@@ -1794,27 +1840,27 @@ pub async fn run_workers(
let mut handles = Vec::with_capacity(num_workers as usize);
for x in [
TMP_LOGS_DIR,
UV_CACHE_DIR,
DENO_CACHE_DIR,
DENO_CACHE_DIR_DEPS,
DENO_CACHE_DIR_NPM,
BUN_CACHE_DIR,
PY310_CACHE_DIR,
PY311_CACHE_DIR,
PY312_CACHE_DIR,
PY313_CACHE_DIR,
BUN_BUNDLE_CACHE_DIR,
GO_CACHE_DIR,
GO_BIN_CACHE_DIR,
RUST_CACHE_DIR,
CSHARP_CACHE_DIR,
NU_CACHE_DIR,
HUB_CACHE_DIR,
POWERSHELL_CACHE_DIR,
JAVA_CACHE_DIR,
RUBY_CACHE_DIR,
TAR_JAVA_CACHE_DIR, // for related places search: ADD_NEW_LANG
&*TMP_LOGS_DIR,
&*UV_CACHE_DIR,
&*DENO_CACHE_DIR,
&*DENO_CACHE_DIR_DEPS,
&*DENO_CACHE_DIR_NPM,
&*BUN_CACHE_DIR,
&*PY310_CACHE_DIR,
&*PY311_CACHE_DIR,
&*PY312_CACHE_DIR,
&*PY313_CACHE_DIR,
&*BUN_BUNDLE_CACHE_DIR,
&*GO_CACHE_DIR,
&*GO_BIN_CACHE_DIR,
&*RUST_CACHE_DIR,
&*CSHARP_CACHE_DIR,
&*NU_CACHE_DIR,
&*HUB_CACHE_DIR,
&*POWERSHELL_CACHE_DIR,
&*JAVA_CACHE_DIR,
&*RUBY_CACHE_DIR,
&*TAR_JAVA_CACHE_DIR, // for related places search: ADD_NEW_LANG
] {
DirBuilder::new()
.recursive(true)
@@ -1831,6 +1877,7 @@ pub async fn run_workers(
let wk_conf = &workers[i as usize - 1];
let conn1 = wk_conf.conn.clone();
let worker_name = wk_conf.worker_name.clone();
let batch_pull_client = wk_conf.batch_pull_client.clone();
WORKERS_NAMES.write().await.push(worker_name.clone());
let ip = ip.clone();
let rx = killpill_rxs.pop().unwrap();
@@ -1853,6 +1900,7 @@ pub async fn run_workers(
rx,
tx,
&base_internal_url,
batch_pull_client.as_ref(),
);
// #[cfg(tokio_unstable)]
@@ -1871,6 +1919,41 @@ pub async fn run_workers(
Ok(())
}
/// Create an HTTP client for native workers to pull jobs from the local server's batch buffer.
/// Self-signs a JWT with native_mode=true using the same JWT secret the server uses.
async fn create_native_batch_pull_client(base_internal_url: &str) -> anyhow::Result<HttpClient> {
use windmill_common::agent_workers::{build_agent_http_client, AGENT_JWT_PREFIX};
use windmill_common::jwt::encode_with_internal_secret;
#[derive(serde::Serialize)]
struct NativeAgentAuth {
worker_group: String,
tags: Vec<String>,
native_mode: Option<bool>,
exp: usize,
}
let worker_config = windmill_common::worker::WORKER_CONFIG.read().await;
let tags = worker_config.worker_tags.clone();
drop(worker_config);
// Token expires in 30 days — renewed on restart
let exp = (chrono::Utc::now() + chrono::Duration::days(30)).timestamp() as usize;
let claims = NativeAgentAuth {
worker_group: WORKER_GROUP.to_string(),
tags,
native_mode: Some(true),
exp,
};
let jwt = encode_with_internal_secret(claims).await?;
let token = format!("{}{}", AGENT_JWT_PREFIX, jwt);
let suffix = create_default_worker_suffix(&HOSTNAME);
Ok(build_agent_http_client(&suffix, &token, base_internal_url))
}
async fn send_delayed_killpill(tx: &KillpillSender, mut max_delay_secs: u64, context: &str) {
if max_delay_secs == 0 {
max_delay_secs = 1;

View File

@@ -44,19 +44,20 @@ use windmill_common::{
apps::APP_WORKSPACED_ROUTE,
auth::create_token_for_owner,
ee_oss::CriticalErrorChannel,
email_oss::send_email_if_possible,
error,
flow_status::{FlowStatus, FlowStatusModule},
global_settings::{
BASE_URL_SETTING, BUNFIG_INSTALL_SCOPES_SETTING, CRITICAL_ALERTS_ON_DB_OVERSIZE_SETTING,
CRITICAL_ALERT_MUTE_UI_SETTING, CRITICAL_ERROR_CHANNELS_SETTING,
DEFAULT_TAGS_PER_WORKSPACE_SETTING, DEFAULT_TAGS_WORKSPACES_SETTING,
EXPOSE_DEBUG_METRICS_SETTING, EXPOSE_METRICS_SETTING, EXTRA_PIP_INDEX_URL_SETTING,
HUB_API_SECRET_SETTING, HUB_BASE_URL_SETTING, INSTANCE_PYTHON_VERSION_SETTING,
JOB_DEFAULT_TIMEOUT_SECS_SETTING, JOB_ISOLATION_SETTING, JWT_SECRET_SETTING,
KEEP_JOB_DIR_SETTING, LICENSE_KEY_SETTING, MONITOR_LOGS_ON_OBJECT_STORE_SETTING,
NPMRC_SETTING, NPM_CONFIG_REGISTRY_SETTING, NUGET_CONFIG_SETTING, OTEL_SETTING,
OTEL_TRACING_PROXY_SETTING, PIP_INDEX_URL_SETTING, POWERSHELL_REPO_PAT_SETTING,
POWERSHELL_REPO_URL_SETTING, REQUEST_SIZE_LIMIT_SETTING,
CRITICAL_ALERTS_ON_TOKEN_EXPIRY_SETTING, CRITICAL_ALERT_MUTE_UI_SETTING,
CRITICAL_ERROR_CHANNELS_SETTING, DEFAULT_TAGS_PER_WORKSPACE_SETTING,
DEFAULT_TAGS_WORKSPACES_SETTING, EXPOSE_DEBUG_METRICS_SETTING, EXPOSE_METRICS_SETTING,
EXTRA_PIP_INDEX_URL_SETTING, HUB_API_SECRET_SETTING, HUB_BASE_URL_SETTING,
INSTANCE_PYTHON_VERSION_SETTING, JOB_DEFAULT_TIMEOUT_SECS_SETTING, JOB_ISOLATION_SETTING,
JWT_SECRET_SETTING, KEEP_JOB_DIR_SETTING, LICENSE_KEY_SETTING,
MONITOR_LOGS_ON_OBJECT_STORE_SETTING, NPMRC_SETTING, NPM_CONFIG_REGISTRY_SETTING,
NUGET_CONFIG_SETTING, OTEL_SETTING, OTEL_TRACING_PROXY_SETTING, PIP_INDEX_URL_SETTING,
POWERSHELL_REPO_PAT_SETTING, POWERSHELL_REPO_URL_SETTING, REQUEST_SIZE_LIMIT_SETTING,
REQUIRE_PREEXISTING_USER_FOR_OAUTH_SETTING, RETENTION_PERIOD_SECS_SETTING,
SAML_METADATA_SETTING, SCIM_TOKEN_SETTING, TIMEOUT_WAIT_RESULT_SETTING,
UV_INDEX_STRATEGY_SETTING,
@@ -73,13 +74,14 @@ use windmill_common::{
load_periodic_bash_script_interval_from_env, load_whitelist_env_vars_from_env,
load_worker_config, reload_custom_tags_setting, store_pull_query,
store_suspended_pull_query, Connection, WorkerConfig, DEFAULT_TAGS_PER_WORKSPACE,
DEFAULT_TAGS_WORKSPACES, INDEXER_CONFIG, SCRIPT_TOKEN_EXPIRY, SMTP_CONFIG, TMP_DIR,
DEFAULT_TAGS_WORKSPACES, INDEXER_CONFIG, SCRIPT_TOKEN_EXPIRY, SMTP_CONFIG, WINDMILL_DIR,
WORKER_CONFIG, WORKER_GROUP,
},
KillpillSender, BASE_URL, CRITICAL_ALERTS_ON_DB_OVERSIZE, CRITICAL_ALERT_MUTE_UI_ENABLED,
CRITICAL_ERROR_CHANNELS, DB, DEFAULT_HUB_BASE_URL, HUB_BASE_URL, JOB_RETENTION_SECS,
METRICS_DEBUG_ENABLED, METRICS_ENABLED, MONITOR_LOGS_ON_OBJECT_STORE, OTEL_LOGS_ENABLED,
OTEL_METRICS_ENABLED, OTEL_TRACING_ENABLED, SERVICE_LOG_RETENTION_SECS,
KillpillSender, BASE_URL, CRITICAL_ALERTS_ON_DB_OVERSIZE, CRITICAL_ALERTS_ON_TOKEN_EXPIRY,
CRITICAL_ALERT_MUTE_UI_ENABLED, CRITICAL_ERROR_CHANNELS, DB, DEFAULT_HUB_BASE_URL,
HUB_BASE_URL, JOB_RETENTION_SECS, METRICS_DEBUG_ENABLED, METRICS_ENABLED,
MONITOR_LOGS_ON_OBJECT_STORE, OTEL_LOGS_ENABLED, OTEL_METRICS_ENABLED, OTEL_TRACING_ENABLED,
SERVICE_LOG_RETENTION_SECS,
};
use windmill_common::{client::AuthedClient, global_settings::APP_WORKSPACED_ROUTE_SETTING};
#[cfg(feature = "parquet")]
@@ -207,6 +209,10 @@ pub async fn initial_load(
tracing::error!("Error loading critical alert mute ui setting: {e:#}");
}
if let Err(e) = reload_critical_alerts_on_token_expiry_setting(conn).await {
tracing::error!("Error loading critical alerts on token expiry setting: {e:#}");
}
if let Some(db) = conn.as_sql() {
if let Err(e) = load_tag_per_workspace_enabled(db).await {
tracing::error!("Error loading default tag per workpsace: {e:#}");
@@ -477,6 +483,21 @@ pub async fn reload_critical_alert_mute_ui_setting(conn: &Connection) -> error::
Ok(())
}
pub async fn reload_critical_alerts_on_token_expiry_setting(
conn: &Connection,
) -> error::Result<()> {
if let Ok(Some(serde_json::Value::Bool(t))) = load_value_from_global_settings_with_conn(
conn,
CRITICAL_ALERTS_ON_TOKEN_EXPIRY_SETTING,
true,
)
.await
{
CRITICAL_ALERTS_ON_TOKEN_EXPIRY.store(t, Ordering::Relaxed);
}
Ok(())
}
pub async fn load_metrics_debug_enabled(conn: &Connection) -> error::Result<()> {
let metrics_enabled =
load_value_from_global_settings_with_conn(conn, EXPOSE_DEBUG_METRICS_SETTING, true).await;
@@ -595,7 +616,7 @@ async fn sleep_until_next_minute_start_plus_one_s() {
use windmill_common::tracing_init::TMP_WINDMILL_LOGS_SERVICE;
async fn find_two_highest_files(hostname: &str) -> (Option<String>, Option<String>) {
let log_dir = format!("{}/{}/", TMP_WINDMILL_LOGS_SERVICE, hostname);
let log_dir = format!("{}/{}/", *TMP_WINDMILL_LOGS_SERVICE, hostname);
let rd_dir = tokio::fs::read_dir(log_dir).await;
if let Ok(mut log_files) = rd_dir {
let mut highest_file: Option<String> = None;
@@ -614,7 +635,8 @@ async fn find_two_highest_files(hostname: &str) -> (Option<String>, Option<Strin
(highest_file, second_highest_file)
} else {
tracing::error!(
"Error reading log files: {TMP_WINDMILL_LOGS_SERVICE}, {:#?}",
"Error reading log files: {}, {:#?}",
*TMP_WINDMILL_LOGS_SERVICE,
rd_dir.unwrap_err()
);
(None, None)
@@ -716,7 +738,7 @@ async fn send_log_file_to_object_store(
let s3_client = windmill_object_store::get_object_store().await;
#[cfg(feature = "parquet")]
if let Some(s3_client) = s3_client {
let path = std::path::Path::new(TMP_WINDMILL_LOGS_SERVICE)
let path = std::path::Path::new(&*TMP_WINDMILL_LOGS_SERVICE)
.join(hostname)
.join(&highest_file);
@@ -844,18 +866,82 @@ struct LogFile {
hostname: String,
}
struct TokenRow {
token_prefix: Option<String>,
label: Option<String>,
email: Option<String>,
workspace_id: Option<String>,
}
fn is_user_token(label: Option<&str>) -> bool {
match label {
None => true,
Some(l) => l != "session" && !l.starts_with("ephemeral") && !l.starts_with("Ephemeral"),
}
}
async fn report_token_expiration(db: &DB, token: &TokenRow, expired: bool) {
if !is_user_token(token.label.as_deref()) {
return;
}
let prefix = token.token_prefix.as_deref().unwrap_or("??????????");
let email_addr = token.email.as_deref().unwrap_or("unknown");
let token_desc = match token.label.as_deref() {
Some(l) if !l.is_empty() => format!("'{l}' ({prefix}****)"),
_ => format!("{prefix}****"),
};
let (alert_message, email_subject, email_body) = if expired {
(
format!(
"API token {token_desc} of '{email_addr}' has expired and been deleted"
),
"Windmill: Your API token has expired",
format!(
"Your API token {token_desc} has expired and been deleted.\n\nPlease create a new token if you still need API access."
),
)
} else {
(
format!("API token {token_desc} of '{email_addr}' is expiring soon"),
"Windmill: Your API token is expiring soon",
format!(
"Your API token {token_desc} is expiring soon.\n\nPlease rotate or renew your token to avoid service disruption."
),
)
};
tracing::info!("{}", alert_message);
if CRITICAL_ALERTS_ON_TOKEN_EXPIRY.load(Ordering::Relaxed) {
report_critical_error(
alert_message,
db.clone(),
token.workspace_id.as_deref(),
None,
)
.await;
}
if let Some(email) = &token.email {
send_email_if_possible(email_subject, &email_body, email);
}
}
pub async fn delete_expired_items(db: &DB) -> () {
let tokens_deleted_r: std::result::Result<Vec<String>, _> = sqlx::query_scalar(
let expired_tokens_r = sqlx::query_as!(
TokenRow,
"DELETE FROM token WHERE expiration <= now()
RETURNING concat(substring(token for 10), '*****')",
RETURNING substring(token for 10) as token_prefix, label, email, workspace_id",
)
.fetch_all(db)
.await;
match tokens_deleted_r {
match expired_tokens_r {
Ok(tokens) => {
if tokens.len() > 0 {
tracing::info!("deleted {} tokens: {:?}", tokens.len(), tokens)
if !tokens.is_empty() {
tracing::info!("deleted {} expired tokens", tokens.len());
for t in &tokens {
report_token_expiration(db, t, true).await;
}
}
}
Err(e) => tracing::error!("Error deleting token: {}", e.to_string()),
@@ -935,7 +1021,7 @@ pub async fn delete_expired_items(db: &DB) -> () {
.iter()
.map(|f| format!("{}/{}", f.hostname, f.file_path))
.collect();
delete_log_files_from_disk_and_store(paths, TMP_WINDMILL_LOGS_SERVICE, windmill_common::tracing_init::LOGS_SERVICE).await;
delete_log_files_from_disk_and_store(paths, &*TMP_WINDMILL_LOGS_SERVICE, windmill_common::tracing_init::LOGS_SERVICE).await;
}
Err(e) => tracing::error!("Error deleting log file: {:?}", e),
@@ -1064,6 +1150,41 @@ pub async fn delete_expired_items(db: &DB) -> () {
}
}
pub async fn check_expiring_tokens(db: &DB) {
// Find tokens expiring within 7 days that still have a pending notification row
let expiring_tokens_r = sqlx::query_as!(
TokenRow,
"DELETE FROM token_expiry_notification n
USING token t
WHERE n.token = t.token
AND n.expiration > now()
AND n.expiration <= now() + interval '7 days'
RETURNING substring(t.token for 10) as token_prefix, t.label, t.email, t.workspace_id",
)
.fetch_all(db)
.await;
match expiring_tokens_r {
Ok(tokens) => {
for t in &tokens {
report_token_expiration(db, t, false).await;
}
if !tokens.is_empty() {
tracing::info!("Sent expiration warnings for {} token(s)", tokens.len());
}
}
Err(e) => tracing::error!("Error checking expiring tokens: {}", e),
}
// Clean up notification rows whose expiration has passed
if let Err(e) = sqlx::query!("DELETE FROM token_expiry_notification WHERE expiration <= now()")
.execute(db)
.await
{
tracing::error!("Error cleaning up expired token notifications: {}", e);
}
}
/// Delete a batch of expired jobs with LIMIT and SKIP LOCKED for high-scale environments.
/// Uses a single transaction per batch to minimize lock duration.
/// Returns the number of jobs deleted in this batch.
@@ -1140,7 +1261,7 @@ async fn delete_expired_jobs_batch(
.filter_map(|opt| opt)
.flat_map(|inner_vec| inner_vec.into_iter())
.collect();
delete_log_files_from_disk_and_store(paths, TMP_DIR, "").await;
delete_log_files_from_disk_and_store(paths, &*WINDMILL_DIR, "").await;
}
Err(e) => tracing::error!("Error deleting job logs: {:?}", e),
}
@@ -1367,7 +1488,7 @@ pub async fn reload_maven_settings_xml_setting(conn: &Connection) {
let settings_xml = MAVEN_SETTINGS_XML.read().await.clone();
match settings_xml {
Some(ref content) if !content.trim().is_empty() => {
let m2_dir = format!("{JAVA_HOME_DIR}/.m2");
let m2_dir = format!("{}/.m2", *JAVA_HOME_DIR);
if let Err(e) = tokio::fs::create_dir_all(&m2_dir).await {
tracing::error!("Failed to create .m2 directory: {e:#}");
return;
@@ -1378,7 +1499,7 @@ pub async fn reload_maven_settings_xml_setting(conn: &Connection) {
}
}
_ => {
let settings_path = format!("{JAVA_HOME_DIR}/.m2/settings.xml");
let settings_path = format!("{}/.m2/settings.xml", *JAVA_HOME_DIR);
let _ = tokio::fs::remove_file(&settings_path).await;
}
}
@@ -2051,6 +2172,16 @@ pub async fn monitor_db(
}
};
// Run every hour (10 iterations * 30s = 5 minutes)
// Check for tokens expiring within 7 days and send alerts
let check_expiring_tokens_f = async {
if server_mode && iteration.is_some() && iteration.as_ref().unwrap().should_run(10) {
if let Some(db) = conn.as_sql() {
check_expiring_tokens(&db).await;
}
}
};
join!(
expired_items_f,
zombie_jobs_f,
@@ -2072,6 +2203,7 @@ pub async fn monitor_db(
cleanup_worker_group_stats_f,
native_triggers_sync_f,
cleanup_notify_events_f,
check_expiring_tokens_f,
);
}

View File

@@ -151,6 +151,8 @@ sqs_trigger: path(char), queue_url(char), aws_resource_path(char), message_attri
FK: (workspace_id) -> workspace(id)
token: token(char), label(char), expiration(ts), workspace_id(char), owner(char), email(char), super_admin(bool), created_at(ts), last_used_at(ts), scopes(text[]), job(uuid)
FK: (workspace_id) -> workspace(id)
token_expiry_notification: token(char), expiration(ts)
INDEX: idx_token_expiry_notification_expiration (expiration)
tutorial_progress: email(char), progress(bit64), skipped_all(bool)
unique_ext_jwt_token: jwt_hash(bigint), last_used_at(ts)
usage: id(char), is_workspace(bool), month_(int), usage(int)
@@ -172,7 +174,7 @@ websocket_trigger: path(char), url(char), script_path(char), is_flow(bool), work
windmill_migrations: name(text), created_at(ts)
worker_group_job_stats: hour(bigint), worker_group(text), script_lang(char), workspace_id(char), job_count(int), total_duration_ms(bigint)
FK: (workspace_id) -> workspace(id)
worker_ping: worker(char), worker_instance(char), ping_at(ts), started_at(ts), ip(char), jobs_executed(int), custom_tags(text[]), worker_group(char), dedicated_worker(char), wm_version(char), current_job_id(uuid), current_job_workspace_id(char), vcpus(bigint), memory(bigint), occupancy_rate(float), memory_usage(bigint), wm_memory_usage(bigint), occupancy_rate_15s(float), occupancy_rate_5m(float), occupancy_rate_30m(float), job_isolation(text), dedicated_workers(text[])
worker_ping: worker(char), worker_instance(char), ping_at(ts), started_at(ts), ip(char), jobs_executed(int), custom_tags(text[]), worker_group(char), dedicated_worker(char), wm_version(char), current_job_id(uuid), current_job_workspace_id(char), vcpus(bigint), memory(bigint), occupancy_rate(float), memory_usage(bigint), wm_memory_usage(bigint), occupancy_rate_15s(float), occupancy_rate_5m(float), occupancy_rate_30m(float), job_isolation(text), dedicated_workers(text[]), native_mode(bool), uses_batch_http_pull(bool)
workspace: id(char), name(char), owner(char), deleted(bool), premium(bool), parent_workspace_id(char)
FK: (parent_workspace_id) -> workspace(id)
workspace_dependencies: id(bigint), name(char), content(text), language(script_lang), description(text), archived(bool), workspace_id(char), created_at(ts)

View File

@@ -1,12 +1,12 @@
#![cfg(all(feature = "private", feature = "agent_worker_server"))]
use windmill_test_utils::*;
use serde_json::json;
use sqlx::{Pool, Postgres};
use windmill_common::{
jobs::{JobPayload, RawCode},
scripts::ScriptLang,
};
use windmill_test_utils::*;
fn bun_code(code: &str) -> RawCode {
RawCode {
@@ -18,8 +18,8 @@ fn bun_code(code: &str) -> RawCode {
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings:
windmill_common::runnable_settings::ConcurrencySettings::default().into(),
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default()
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
}
}
@@ -223,7 +223,10 @@ async fn test_agent_worker_token_and_ping(db: Pool<Postgres>) -> anyhow::Result<
.fetch_one(&db)
.await?;
assert!(worker_count > 0, "worker ping should be recorded in database");
assert!(
worker_count > 0,
"worker ping should be recorded in database"
);
// MainLoop ping updates the existing record
let resp = http_client
@@ -265,3 +268,319 @@ async fn test_agent_worker_multiple_jobs_sequential(db: Pool<Postgres>) -> anyho
Ok(())
}
/// Test the volume HTTP proxy endpoints that agent workers use.
///
/// Exercises the full volume lifecycle via HTTP:
/// 1. Configure workspace S3 storage (FilesystemStorage)
/// 2. Pre-populate a volume with a file
/// 3. POST /begin — acquire lease, get manifest
/// 4. GET /file/* — download existing file
/// 5. PUT /file/* — upload a new file
/// 6. POST /commit — finalize with stats, release lease
/// 7. Verify DB state and storage
#[cfg(feature = "parquet")]
#[sqlx::test(fixtures("base"))]
async fn test_agent_worker_volume_e2e(db: Pool<Postgres>) -> anyhow::Result<()> {
let (client, _port, _server) = init_client_agent_mode(db.clone()).await;
// 1. Set up filesystem-based object storage in a temp dir
let storage_dir = tempfile::tempdir()?;
let storage_root = storage_dir.path().to_string_lossy().to_string();
let lfs_config = json!({
"type": "FilesystemStorage",
"root_path": storage_root,
"public_resource": null,
"advanced_permissions": null
});
sqlx::query!(
"UPDATE workspace_settings SET large_file_storage = $1 WHERE workspace_id = $2",
lfs_config,
"test-workspace"
)
.execute(&db)
.await?;
// 2. Pre-populate the volume with a file
let vol_dir = storage_dir.path().join("volumes").join("test-vol");
std::fs::create_dir_all(&vol_dir)?;
std::fs::write(vol_dir.join("hello.txt"), b"hello from volume")?;
let base = client.baseurl();
let http = client.client();
let vol_base = format!("{base}/w/test-workspace/volumes/test-vol");
// 3. POST /begin — acquire lease, get manifest + permissions
let resp = http
.post(format!("{vol_base}/begin"))
.json(&json!({
"worker_name": "test-worker-1",
"permissioned_as": "u/test-user"
}))
.send()
.await?;
assert!(
resp.status().is_success(),
"begin should succeed, got: {}",
resp.status()
);
let begin_body: serde_json::Value = resp.json().await?;
assert!(
begin_body["writable"].as_bool().unwrap(),
"should be writable"
);
let manifest = begin_body["manifest"].as_object().unwrap();
assert!(
manifest.contains_key("hello.txt"),
"manifest should contain hello.txt, got: {manifest:?}"
);
// 4. GET /file/* — download the existing file
let resp = http
.get(format!("{vol_base}/file/hello.txt"))
.send()
.await?;
assert!(
resp.status().is_success(),
"file download should succeed, got: {}",
resp.status()
);
let file_bytes = resp.bytes().await?;
assert_eq!(
file_bytes.as_ref(),
b"hello from volume",
"downloaded file content should match"
);
// 5. PUT /file/* — upload a new file
let resp = http
.put(format!("{vol_base}/file/output.txt"))
.body(b"written by agent worker".to_vec())
.send()
.await?;
assert!(
resp.status().is_success(),
"file upload should succeed, got: {}",
resp.status()
);
// 6. POST /commit — finalize: report stats, release lease
let resp = http
.post(format!("{vol_base}/commit"))
.json(&json!({
"worker_name": "test-worker-1",
"deleted_keys": [],
"symlinks": {},
"file_count": 2,
"size_bytes": 39
}))
.send()
.await?;
assert!(
resp.status().is_success(),
"commit should succeed, got: {}",
resp.status()
);
// 7. Verify volume DB row was updated
let vol_row = sqlx::query!(
"SELECT size_bytes, file_count, leased_by, lease_until
FROM volume WHERE workspace_id = $1 AND name = $2",
"test-workspace",
"test-vol"
)
.fetch_optional(&db)
.await?;
let vol_row = vol_row.expect("volume row should exist");
assert_eq!(vol_row.file_count, 2, "file_count should be 2");
assert_eq!(vol_row.size_bytes, 39, "size_bytes should match");
assert!(vol_row.leased_by.is_none(), "lease should be released");
assert!(
vol_row.lease_until.is_none() || vol_row.lease_until.unwrap() < chrono::Utc::now(),
"lease_until should be cleared or in the past"
);
// 8. Verify the uploaded file was persisted in storage
let output_path = vol_dir.join("output.txt");
assert!(output_path.exists(), "output.txt should be in storage");
let output_content = std::fs::read_to_string(&output_path)?;
assert_eq!(output_content, "written by agent worker");
Ok(())
}
/// Full E2E test: agent worker in HTTP mode runs a Bun script with a volume mount.
///
/// The worker pulls the job via HTTP, downloads volume files via the server-side
/// volume proxy endpoints, executes the script, and syncs changes back.
#[cfg(all(feature = "parquet", feature = "enterprise"))]
#[sqlx::test(fixtures("base"))]
async fn test_agent_worker_volume_http_worker_e2e(db: Pool<Postgres>) -> anyhow::Result<()> {
let (_client, port, _server) = init_client_agent_mode(db.clone()).await;
// 1. Set up filesystem-based object storage in a temp dir
let storage_dir = tempfile::tempdir()?;
let storage_root = storage_dir.path().to_string_lossy().to_string();
let lfs_config = json!({
"type": "FilesystemStorage",
"root_path": storage_root,
"public_resource": null,
"advanced_permissions": null
});
sqlx::query!(
"UPDATE workspace_settings SET large_file_storage = $1 WHERE workspace_id = $2",
lfs_config,
"test-workspace"
)
.execute(&db)
.await?;
// 2. Pre-populate the volume with a file
let vol_dir = storage_dir.path().join("volumes").join("test-vol");
std::fs::create_dir_all(&vol_dir)?;
std::fs::write(vol_dir.join("hello.txt"), b"hello from volume")?;
// 3. Push the job, then run worker with HTTP connection (bun tag)
let code = r#"// volume: test-vol /tmp/data
import { readFileSync, writeFileSync, existsSync } from "fs";
export function main() {
const content = readFileSync("/tmp/data/hello.txt", "utf-8");
writeFileSync("/tmp/data/output.txt", "written by agent worker");
return {
read_content: content,
output_exists: existsSync("/tmp/data/output.txt"),
};
}"#;
let uuid = RunJob::from(JobPayload::Code(bun_code(code)))
.push(&db)
.await;
let listener = listen_for_completed_jobs(&db).await;
let conn = testing_http_connection_with_tags(
port,
vec!["bun".into(), "flow".into(), "dependency".into()],
)
.await;
in_test_worker(conn, listener.find(&uuid), port).await;
let result = completed_job(uuid, &db).await;
assert!(result.success, "job should succeed: {:?}", result.result);
let json = result.json_result().expect("should have JSON result");
assert_eq!(json["read_content"], json!("hello from volume"));
assert_eq!(json["output_exists"], json!(true));
// 4. Verify volume DB row was updated
let vol_row = sqlx::query!(
"SELECT size_bytes, file_count, leased_by, lease_until
FROM volume WHERE workspace_id = $1 AND name = $2",
"test-workspace",
"test-vol"
)
.fetch_optional(&db)
.await?;
let vol_row = vol_row.expect("volume row should exist");
assert!(
vol_row.file_count >= 2,
"should have at least 2 files (hello.txt + output.txt), got: {}",
vol_row.file_count
);
assert!(vol_row.size_bytes > 0, "size_bytes should be > 0");
assert!(vol_row.leased_by.is_none(), "lease should be released");
assert!(
vol_row.lease_until.is_none() || vol_row.lease_until.unwrap() < chrono::Utc::now(),
"lease_until should be cleared or in the past"
);
// 5. Verify the new file was written back to the storage
let output_path = vol_dir.join("output.txt");
assert!(
output_path.exists(),
"output.txt should be synced back to storage"
);
let output_content = std::fs::read_to_string(&output_path)?;
assert_eq!(output_content, "written by agent worker");
Ok(())
}
/// Test the volume release endpoint (error/cancel path).
#[cfg(feature = "parquet")]
#[sqlx::test(fixtures("base"))]
async fn test_agent_worker_volume_release(db: Pool<Postgres>) -> anyhow::Result<()> {
let (client, _port, _server) = init_client_agent_mode(db.clone()).await;
// Set up filesystem storage
let storage_dir = tempfile::tempdir()?;
let storage_root = storage_dir.path().to_string_lossy().to_string();
let lfs_config = json!({
"type": "FilesystemStorage",
"root_path": storage_root,
"public_resource": null,
"advanced_permissions": null
});
sqlx::query!(
"UPDATE workspace_settings SET large_file_storage = $1 WHERE workspace_id = $2",
lfs_config,
"test-workspace"
)
.execute(&db)
.await?;
let base = client.baseurl();
let http = client.client();
let vol_base = format!("{base}/w/test-workspace/volumes/test-vol");
// Begin (acquire lease)
let resp = http
.post(format!("{vol_base}/begin"))
.json(&json!({
"worker_name": "test-worker-2",
"permissioned_as": "u/test-user"
}))
.send()
.await?;
assert!(resp.status().is_success(), "begin should succeed");
// Verify lease is held
let leased = sqlx::query_scalar!(
"SELECT leased_by FROM volume WHERE workspace_id = $1 AND name = $2",
"test-workspace",
"test-vol"
)
.fetch_optional(&db)
.await?
.flatten();
assert_eq!(leased.as_deref(), Some("test-worker-2"));
// Release without commit (simulating error path)
let resp = http
.post(format!("{vol_base}/release"))
.json(&json!({ "worker_name": "test-worker-2" }))
.send()
.await?;
assert!(resp.status().is_success(), "release should succeed");
// Verify lease is cleared
let leased = sqlx::query_scalar!(
"SELECT leased_by FROM volume WHERE workspace_id = $1 AND name = $2",
"test-workspace",
"test-vol"
)
.fetch_optional(&db)
.await?
.flatten();
assert!(leased.is_none(), "lease should be released");
Ok(())
}

View File

@@ -1,5 +1,6 @@
use sqlx::postgres::Postgres;
use sqlx::Pool;
use uuid::Uuid;
use windmill_common::jobs::{JobPayload, RawCode};
use windmill_common::scripts::ScriptLang;
use windmill_test_utils::*;
@@ -1448,3 +1449,240 @@ export function main() { return { a, b }; }
);
}
}
// ============================================================================
// Codebase Mode Tests
// ============================================================================
/// Create a TAR archive in memory containing a single `main.js` file.
fn create_codebase_tar(main_js_content: &str) -> Vec<u8> {
let mut builder = tar::Builder::new(Vec::new());
let content = main_js_content.as_bytes();
let mut header = tar::Header::new_gnu();
header.set_path("main.js").unwrap();
header.set_size(content.len() as u64);
header.set_mode(0o644);
header.set_cksum();
builder.append(&header, content).unwrap();
builder.into_inner().unwrap()
}
/// Place a TAR codebase at the expected cache path for the given job ID and hash.
fn place_codebase_in_cache(job_id: &Uuid, tar_bytes: &[u8], is_esm: bool) {
let codebase_id = if is_esm {
format!("{}.esm.tar", job_id)
} else {
format!("{}.tar", job_id)
};
let bundle_path = format!("script_bundle/test-workspace/{}", codebase_id);
let cache_path = format!(
"{}/{}.tar",
*windmill_common::worker::ROOT_CACHE_NOMOUNT_DIR,
bundle_path,
);
let parent = std::path::Path::new(&cache_path).parent().unwrap();
std::fs::create_dir_all(parent).unwrap();
std::fs::write(&cache_path, tar_bytes).unwrap();
}
#[sqlx::test(fixtures("base"))]
async fn test_cjs_codebase_tar(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let main_js = r#"
module.exports.main = function() {
return "cjs codebase ok";
};
"#;
let inner_content = r#"export function main() { return "cjs codebase ok"; }"#;
let job_id = Uuid::new_v4();
let tar_bytes = create_codebase_tar(main_js);
place_codebase_in_cache(&job_id, &tar_bytes, false);
let job = JobPayload::Code(RawCode {
hash: Some(-43), // PREVIEW_IS_TAR_CODEBASE_HASH
content: inner_content.to_string(),
path: None,
language: ScriptLang::Bun,
lock: None,
concurrency_settings: Default::default(),
debouncing_settings: Default::default(),
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
});
let result = RunJob::from(job)
.job_id(job_id)
.run_until_complete(&db, false, port)
.await
.json_result()
.unwrap();
assert_eq!(result, serde_json::json!("cjs codebase ok"));
Ok(())
}
#[sqlx::test(fixtures("base"))]
async fn test_esm_codebase_tar(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let main_js = r#"
export function main() {
return "esm codebase ok";
}
"#;
let inner_content = r#"export function main() { return "esm codebase ok"; }"#;
let job_id = Uuid::new_v4();
let tar_bytes = create_codebase_tar(main_js);
place_codebase_in_cache(&job_id, &tar_bytes, true);
let job = JobPayload::Code(RawCode {
hash: Some(-45), // PREVIEW_IS_TAR_ESM_CODEBASE_HASH
content: inner_content.to_string(),
path: None,
language: ScriptLang::Bun,
lock: None,
concurrency_settings: Default::default(),
debouncing_settings: Default::default(),
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
});
let result = RunJob::from(job)
.job_id(job_id)
.run_until_complete(&db, false, port)
.await
.json_result()
.unwrap();
assert_eq!(result, serde_json::json!("esm codebase ok"));
Ok(())
}
#[sqlx::test(fixtures("base"))]
async fn test_cjs_codebase_tar_nsjail(db: Pool<Postgres>) -> anyhow::Result<()> {
if std::process::Command::new("nsjail")
.arg("--help")
.output()
.is_err()
{
eprintln!("nsjail not found, skipping test");
return Ok(());
}
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let main_js = r#"
module.exports.main = function() {
return "cjs nsjail ok";
};
"#;
let inner_content = r#"export function main() { return "cjs nsjail ok"; }"#;
let job_id = Uuid::new_v4();
let tar_bytes = create_codebase_tar(main_js);
place_codebase_in_cache(&job_id, &tar_bytes, false);
let job = JobPayload::Code(RawCode {
hash: Some(-43),
content: inner_content.to_string(),
path: None,
language: ScriptLang::Bun,
lock: None,
concurrency_settings: Default::default(),
debouncing_settings: Default::default(),
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
});
use std::sync::atomic::Ordering;
windmill_worker::JOB_ISOLATION.store(
windmill_worker::JobIsolationLevel::NsjailSandboxing as u8,
Ordering::Relaxed,
);
let result = RunJob::from(job)
.job_id(job_id)
.run_until_complete(&db, false, port)
.await;
windmill_worker::JOB_ISOLATION.store(
windmill_worker::JobIsolationLevel::Undefined as u8,
Ordering::Relaxed,
);
let json = result.json_result().unwrap();
assert_eq!(json, serde_json::json!("cjs nsjail ok"));
Ok(())
}
#[sqlx::test(fixtures("base"))]
async fn test_esm_codebase_tar_nsjail(db: Pool<Postgres>) -> anyhow::Result<()> {
if std::process::Command::new("nsjail")
.arg("--help")
.output()
.is_err()
{
eprintln!("nsjail not found, skipping test");
return Ok(());
}
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let main_js = r#"
export function main() {
return "esm nsjail ok";
}
"#;
let inner_content = r#"export function main() { return "esm nsjail ok"; }"#;
let job_id = Uuid::new_v4();
let tar_bytes = create_codebase_tar(main_js);
place_codebase_in_cache(&job_id, &tar_bytes, true);
let job = JobPayload::Code(RawCode {
hash: Some(-45),
content: inner_content.to_string(),
path: None,
language: ScriptLang::Bun,
lock: None,
concurrency_settings: Default::default(),
debouncing_settings: Default::default(),
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
});
use std::sync::atomic::Ordering;
windmill_worker::JOB_ISOLATION.store(
windmill_worker::JobIsolationLevel::NsjailSandboxing as u8,
Ordering::Relaxed,
);
let result = RunJob::from(job)
.job_id(job_id)
.run_until_complete(&db, false, port)
.await;
windmill_worker::JOB_ISOLATION.store(
windmill_worker::JobIsolationLevel::Undefined as u8,
Ordering::Relaxed,
);
let json = result.json_result().unwrap();
assert_eq!(json, serde_json::json!("esm nsjail ok"));
Ok(())
}

View File

@@ -0,0 +1,323 @@
//! Tests for WM_END_USER_EMAIL environment variable.
//!
//! These tests verify that WM_END_USER_EMAIL is populated with the authenticated
//! user's email when executing app components.
//!
//! TODO: Add tests for scripts and flows once public execution endpoints are identified.
//! Currently only apps support non-workspace-member execution via OptAuthed + token lookup.
use serde_json::json;
use sqlx::{Pool, Postgres};
use windmill_common::worker::Connection;
use windmill_test_utils::*;
const SAME_WS_TOKEN: &str = "SECRET_TOKEN";
const OTHER_WS_TOKEN: &str = "OTHER_WS_TOKEN";
const NO_WS_TOKEN: &str = "NO_WS_TOKEN";
const SAME_WS_EMAIL: &str = "test@windmill.dev";
const OTHER_WS_EMAIL: &str = "other-ws@windmill.dev";
const NO_WS_EMAIL: &str = "no-ws@windmill.dev";
fn client() -> reqwest::Client {
reqwest::Client::new()
}
fn authed(builder: reqwest::RequestBuilder, token: &str) -> reqwest::RequestBuilder {
builder.header("Authorization", format!("Bearer {}", token))
}
// TODO: Script tests - need to identify public execution endpoints for non-workspace-members
// async fn run_script(port: u16, token: &str) -> anyhow::Result<String> {
// let url = format!(
// "http://localhost:{}/api/w/test-workspace/jobs/run_wait_result/p/f/test/get_end_user_email",
// port
// );
// let resp = authed(client().post(&url), token)
// .json(&json!({}))
// .send()
// .await?;
// if !resp.status().is_success() {
// anyhow::bail!("script run failed: {} - {}", resp.status(), resp.text().await?);
// }
// Ok(resp.json::<serde_json::Value>().await?
// .as_str().unwrap_or("").to_string())
// }
// TODO: Flow tests - need to identify public execution endpoints for non-workspace-members
// async fn run_flow(port: u16, token: &str) -> anyhow::Result<String> {
// let url = format!(
// "http://localhost:{}/api/w/test-workspace/jobs/run_wait_result/f/f/test/get_end_user_email_flow",
// port
// );
// let resp = authed(client().post(&url), token)
// .json(&json!({}))
// .send()
// .await?;
// if !resp.status().is_success() {
// anyhow::bail!("flow run failed: {} - {}", resp.status(), resp.text().await?);
// }
// Ok(resp.json::<serde_json::Value>().await?
// .as_str().unwrap_or("").to_string())
// }
/// Create an app with inline script via API
async fn create_app_with_inline_script(port: u16, path: &str) -> anyhow::Result<()> {
let url = format!(
"http://localhost:{}/api/w/test-workspace/apps/create",
port
);
let resp = authed(client().post(&url), SAME_WS_TOKEN)
.json(&json!({
"path": path,
"summary": "Test app for WM_END_USER_EMAIL",
"value": {
"type": "app",
"grid": [],
"subgrids": {},
"hiddenInlineScripts": [{
"name": "get_email",
"language": "deno",
"content": "export function main() { return Deno.env.get(\"WM_END_USER_EMAIL\") || \"\"; }",
"path": "f/test/email_app/get_email"
}]
},
"policy": {
"execution_mode": "anonymous",
"on_behalf_of": null,
"on_behalf_of_email": null,
"triggerables_v2": {
"get_email": {
"static_inputs": {},
"one_of_inputs": {}
},
// SHA256 hash of raw_code content for anonymous execution
"rawscript/6428aba5aa2d3ea8e1215bfdccbedd3718b18da7a239e3778a9787bb9a0ea606": {
"static_inputs": {},
"one_of_inputs": {}
}
}
}
}))
.send()
.await?;
if !resp.status().is_success() {
anyhow::bail!("create app failed: {} - {}", resp.status(), resp.text().await?);
}
Ok(())
}
/// Create a raw app with inline script via API (uses regular app endpoint with rawapp type)
async fn create_raw_app_with_inline_script(port: u16, path: &str) -> anyhow::Result<()> {
let url = format!(
"http://localhost:{}/api/w/test-workspace/apps/create",
port
);
let resp = authed(client().post(&url), SAME_WS_TOKEN)
.json(&json!({
"path": path,
"summary": "Test raw app for WM_END_USER_EMAIL",
"value": {
"type": "rawapp",
"css": "",
"inlineScripts": [{
"name": "get_email",
"language": "deno",
"content": "export function main() { return Deno.env.get(\"WM_END_USER_EMAIL\") || \"\"; }"
}]
},
"policy": {
"execution_mode": "anonymous",
"on_behalf_of": null,
"on_behalf_of_email": null,
"triggerables_v2": {
"get_email": {
"static_inputs": {},
"one_of_inputs": {}
},
// SHA256 hash of raw_code content for anonymous execution
"rawscript/6428aba5aa2d3ea8e1215bfdccbedd3718b18da7a239e3778a9787bb9a0ea606": {
"static_inputs": {},
"one_of_inputs": {}
}
}
}
}))
.send()
.await?;
if !resp.status().is_success() {
anyhow::bail!("create raw app failed: {} - {}", resp.status(), resp.text().await?);
}
Ok(())
}
async fn run_app_inline_script(port: u16, token: &str, app_path: &str, force_viewer: bool) -> anyhow::Result<String> {
let url = format!(
"http://localhost:{}/api/w/test-workspace/apps_u/execute_component/{}",
port, app_path
);
let mut payload = json!({
"args": {},
"component": "get_email",
"raw_code": {
"language": "deno",
"content": "export function main() { return Deno.env.get(\"WM_END_USER_EMAIL\") || \"\"; }",
"path": format!("{}/get_email", app_path)
}
});
if force_viewer {
payload["force_viewer_static_fields"] = json!({});
}
let resp = authed(client().post(&url), token)
.json(&payload)
.send()
.await?;
if !resp.status().is_success() {
anyhow::bail!("app inline script run failed: {} - {}", resp.status(), resp.text().await?);
}
let job_id = resp.text().await?;
wait_for_job_result(port, token, &job_id).await
}
async fn run_raw_app_inline_script(port: u16, token: &str, app_path: &str, force_viewer: bool) -> anyhow::Result<String> {
let url = format!(
"http://localhost:{}/api/w/test-workspace/apps_u/execute_component/{}",
port, app_path
);
let mut payload = json!({
"args": {},
"component": "get_email",
"raw_code": {
"language": "deno",
"content": "export function main() { return Deno.env.get(\"WM_END_USER_EMAIL\") || \"\"; }"
}
});
if force_viewer {
payload["force_viewer_static_fields"] = json!({});
}
let resp = authed(client().post(&url), token)
.json(&payload)
.send()
.await?;
if !resp.status().is_success() {
anyhow::bail!("raw app inline script run failed: {} - {}", resp.status(), resp.text().await?);
}
let job_id = resp.text().await?;
wait_for_job_result(port, token, &job_id).await
}
async fn wait_for_job_result(port: u16, token: &str, job_id: &str) -> anyhow::Result<String> {
let url = format!(
"http://localhost:{}/api/w/test-workspace/jobs_u/completed/get_result/{}",
port, job_id
);
for _ in 0..100 {
tokio::time::sleep(std::time::Duration::from_millis(100)).await;
let resp = authed(client().get(&url), token).send().await?;
if resp.status().is_success() {
return Ok(resp.json::<serde_json::Value>().await?
.as_str().unwrap_or("").to_string());
}
}
anyhow::bail!("timeout waiting for job result")
}
// TODO: Script tests - need to identify public execution endpoints for non-workspace-members
// #[cfg(feature = "deno_core")]
// #[sqlx::test(fixtures("base", "end_user_email"))]
// async fn test_script_wm_end_user_email(db: Pool<Postgres>) -> anyhow::Result<()> {
// initialize_tracing().await;
// set_jwt_secret().await;
// let server = ApiServer::start(db.clone()).await?;
// let port = server.addr.port();
//
// in_test_worker(Connection::Sql(db.clone()), async move {
// let result = run_script(port, SAME_WS_TOKEN).await?;
// assert_eq!(result, SAME_WS_EMAIL, "same workspace user should get their email");
// Ok::<(), anyhow::Error>(())
// }, port).await?;
//
// Ok(())
// }
// TODO: Flow tests - need to identify public execution endpoints for non-workspace-members
// #[cfg(feature = "deno_core")]
// #[sqlx::test(fixtures("base", "end_user_email"))]
// async fn test_flow_wm_end_user_email(db: Pool<Postgres>) -> anyhow::Result<()> {
// initialize_tracing().await;
// set_jwt_secret().await;
// let server = ApiServer::start(db.clone()).await?;
// let port = server.addr.port();
//
// in_test_worker(Connection::Sql(db.clone()), async move {
// let result = run_flow(port, SAME_WS_TOKEN).await?;
// assert_eq!(result, SAME_WS_EMAIL, "same workspace user should get their email");
// Ok::<(), anyhow::Error>(())
// }, port).await?;
//
// Ok(())
// }
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base", "end_user_email"))]
async fn test_app_wm_end_user_email(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
set_jwt_secret().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let app_path = "f/test/email_app";
in_test_worker(Connection::Sql(db.clone()), async move {
// Create the app with inline script first
create_app_with_inline_script(port, app_path).await?;
// Same workspace user (force_viewer mode works for workspace members)
let result = run_app_inline_script(port, SAME_WS_TOKEN, app_path, true).await?;
assert_eq!(result, SAME_WS_EMAIL, "same workspace user should get their email");
// Other workspace user (uses app's anonymous policy + token lookup)
let result = run_app_inline_script(port, OTHER_WS_TOKEN, app_path, false).await?;
assert_eq!(result, OTHER_WS_EMAIL, "other workspace user should get their email");
// No workspace user (uses app's anonymous policy + token lookup)
let result = run_app_inline_script(port, NO_WS_TOKEN, app_path, false).await?;
assert_eq!(result, NO_WS_EMAIL, "no workspace user should get their email");
Ok::<(), anyhow::Error>(())
}, port).await?;
Ok(())
}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base", "end_user_email"))]
async fn test_raw_app_wm_end_user_email(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
set_jwt_secret().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let app_path = "f/test/email_raw_app";
in_test_worker(Connection::Sql(db.clone()), async move {
// Create the raw app with inline script first
create_raw_app_with_inline_script(port, app_path).await?;
// Same workspace user (force_viewer mode works for workspace members)
let result = run_raw_app_inline_script(port, SAME_WS_TOKEN, app_path, true).await?;
assert_eq!(result, SAME_WS_EMAIL, "same workspace user should get their email");
// Other workspace user (uses app's anonymous policy + token lookup)
let result = run_raw_app_inline_script(port, OTHER_WS_TOKEN, app_path, false).await?;
assert_eq!(result, OTHER_WS_EMAIL, "other workspace user should get their email");
// No workspace user (uses app's anonymous policy + token lookup)
let result = run_raw_app_inline_script(port, NO_WS_TOKEN, app_path, false).await?;
assert_eq!(result, NO_WS_EMAIL, "no workspace user should get their email");
Ok::<(), anyhow::Error>(())
}, port).await?;
Ok(())
}

View File

@@ -0,0 +1,63 @@
-- Fixture for WM_END_USER_EMAIL tests
-- Sets up 3 users with different workspace memberships:
-- 1. test@windmill.dev - in test-workspace (from base.sql)
-- 2. other-ws@windmill.dev - in other-workspace only
-- 3. no-ws@windmill.dev - not in any workspace
-- Second workspace for cross-workspace user
INSERT INTO workspace (id, name, owner)
VALUES ('other-workspace', 'other-workspace', 'other-ws-user');
INSERT INTO workspace_key(workspace_id, kind, key)
VALUES ('other-workspace', 'cloud', 'other-key');
INSERT INTO workspace_settings (workspace_id)
VALUES ('other-workspace');
INSERT INTO group_ (workspace_id, name, summary, extra_perms)
VALUES ('other-workspace', 'all', 'All users', '{}');
-- User in other-workspace only (not in test-workspace)
INSERT INTO password(email, password_hash, login_type, super_admin, verified, name)
VALUES ('other-ws@windmill.dev', 'hash', 'password', false, true, 'Other WS User');
INSERT INTO usr(workspace_id, email, username, is_admin, role)
VALUES ('other-workspace', 'other-ws@windmill.dev', 'other-ws-user', true, 'Admin');
INSERT INTO token(token, email, label, super_admin)
VALUES ('OTHER_WS_TOKEN', 'other-ws@windmill.dev', 'other ws token', false);
-- User not in any workspace
INSERT INTO password(email, password_hash, login_type, super_admin, verified, name)
VALUES ('no-ws@windmill.dev', 'hash', 'password', false, true, 'No WS User');
INSERT INTO token(token, email, label, super_admin)
VALUES ('NO_WS_TOKEN', 'no-ws@windmill.dev', 'no ws token', false);
-- Script that returns WM_END_USER_EMAIL (public via extra_perms)
INSERT INTO script (workspace_id, created_by, content, schema, summary, description, path, hash, language, lock, kind, extra_perms)
VALUES (
'test-workspace', 'test-user',
'export function main() { return Deno.env.get("WM_END_USER_EMAIL") || ""; }',
'{"$schema":"https://json-schema.org/draft/2020-12/schema","properties":{},"required":[],"type":"object"}',
'Returns WM_END_USER_EMAIL', '', 'f/test/get_end_user_email', 900001, 'deno', '', 'script',
'{"g/all": true}'
);
-- Flow that returns WM_END_USER_EMAIL (public via extra_perms)
INSERT INTO flow (workspace_id, summary, description, path, versions, schema, value, edited_by, extra_perms)
VALUES (
'test-workspace', 'Returns WM_END_USER_EMAIL', '', 'f/test/get_end_user_email_flow', '{900002}',
'{"$schema":"https://json-schema.org/draft/2020-12/schema","properties":{},"required":[],"type":"object"}',
'{"modules": [{"id": "a", "value": {"type": "rawscript", "language": "deno", "content": "export function main() { return Deno.env.get(\"WM_END_USER_EMAIL\") || \"\"; }", "input_transforms": {}}}]}',
'test-user',
'{"g/all": true}'
);
INSERT INTO flow_version (id, workspace_id, path, schema, value, created_by)
VALUES (
900002, 'test-workspace', 'f/test/get_end_user_email_flow',
'{"$schema":"https://json-schema.org/draft/2020-12/schema","properties":{},"required":[],"type":"object"}',
'{"modules": [{"id": "a", "value": {"type": "rawscript", "language": "deno", "content": "export function main() { return Deno.env.get(\"WM_END_USER_EMAIL\") || \"\"; }", "input_transforms": {}}}]}',
'test-user'
);

View File

@@ -206,7 +206,7 @@ fn spawn_workers(
std::fs::DirBuilder::new()
.recursive(true)
.create(windmill_worker::GO_BIN_CACHE_DIR)
.create(&*windmill_worker::GO_BIN_CACHE_DIR)
.expect("could not create initial worker dir");
let (tx, _) = KillpillSender::new(n + 1);
@@ -241,6 +241,7 @@ fn spawn_workers(
rx,
tx2,
&base_internal_url,
None,
)
.await;
};

Some files were not shown because too many files have changed in this diff Show More