Compare commits

...

116 Commits

Author SHA1 Message Date
wendrul
ba775edef2 Merge remote-tracking branch 'origin/main' into folder-deploy
# Conflicts:
#	frontend/src/lib/utils_workspace_deploy.ts
2026-04-02 15:39:36 +02:00
wendrul
91f0a564b9 fix: deployment UIs for folders 2026-04-02 15:33:12 +02:00
Diego Imbert
c3a1c26be1 nit: revert ee.rs in substitute_ee_code.sh (#8672) 2026-04-02 10:16:28 +00:00
Ruben Fiszel
c87a6a0f2c fix: support branch-specific folder.meta.yaml in missing-meta check (#8661)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-04-02 10:05:02 +00:00
hugocasa
350ffdce29 fix: pre-fix trigger edited_by for superadmins not in workspace (#8669)
Add a migration that runs just before 20260318000000 (add_permissioned_as).
For each trigger table, if the email column still exists, update edited_by
to the trigger's email when the user is not in the workspace but is a
superadmin. This ensures the subsequent permissioned_as migration stores
the raw email instead of an invalid u/{username} reference.

If 20260318000000 was already applied, the migration is a no-op (email
column is gone, guarded by information_schema check).

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 08:39:44 +00:00
centdix
28c073056c fix: correct raw app flow inputs (#8667)
* fix: correct raw app flow inputs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: remove raw app legacy migration

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-04-02 01:57:56 +00:00
Ruben Fiszel
c86846ac19 rate limit token creation on CLOUD_HOSTED (10/min per user) (#8664)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 01:52:02 +00:00
Ruben Fiszel
7ab0ea581d fix: strip f/ prefix from folder paths when deploying from workspace forks (#8662)
* fix: strip f/ prefix from folder paths when deploying from workspace forks

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: extract folderName helper for f/ prefix stripping

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 21:06:19 +00:00
Ruben Fiszel
bcce627387 fix: validate rd redirect on login with same rules as logout (#8655)
* fix: validate rd redirect on login with same rules as logout

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: sanitize rd at source in login callback to prevent leaking to goto

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: validate rd redirect in Login component for fresh login flow

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 19:44:22 +00:00
Ruben Fiszel
175af8032f chore(main): release 1.672.0 (#8654)
* chore(main): release 1.672.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-04-01 12:05:44 -04:00
Diego Imbert
1784bed4ac Fix WAC RunForm layout (#8658) 2026-04-01 12:01:29 -04:00
Ruben Fiszel
a46aa641f9 feat: add R language support (#8263)
* feat: add R language support

Add R as a new supported scripting language in Windmill, following the
same pattern used for Ruby. Includes:

- Backend: ScriptLang::Rlang enum variant, DB migration, tree-sitter-r
  parser crate with tests, WASM parser binding, R executor with NSJail
  sandboxing, job dispatch and signature parsing
- Frontend: language picker, R icon, syntax highlighting, editor bar
  insertions (Sys.getenv, get_variable, get_resource), schema inference,
  init code template, BETA badge
- CLI: .r extension mapping, sync support, bootstrap template

R scripts use `main <- function(...)` syntax, jsonlite for JSON
serialization, and system curl for the Windmill client helper.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add R package resolution and installation

Parse library()/require() calls from R scripts to extract dependencies.
Resolve versions from CRAN, cache lockfiles in pip_resolution_cache,
and install packages to a shared R library cache. The run step sets
R_LIBS_USER so installed packages are available to the script.

- Parser: parse_r_requirements() extracts package names from AST
- Executor: resolve() generates lockfile, install() installs from CRAN
- Worker lockfiles: wire up R resolve for dependency jobs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add nsjail sandboxing for R resolve and install phases

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: fix R get_variable/get_resource and add sandbox annotation + e2e tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: fix R arg inference with JS fallback parser and get_variable/get_resource

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix flake

* nsjail

* nits

* fix: R install improvements - suppress verbose output, flat lockfile logging, Dockerfile R support, rlimits

- Suppress renv verbose output during resolve and install (controlled by #verbose annotation)
- Filter renv from install list (already loaded, causes noisy restart message)
- Log compact "resolved N packages" instead of full renv.lock JSON
- Add R (r-base, r-cran-renv) to DockerfileFull and DockerfileFullEe
- Use disable_rl for nsjail install config (R compiles from source)
- Reduce default concurrency from 20 to 5
- Add rlang to openflow.openapi.yaml
- Fix MainArgSignature (no_main_func -> auto_kind) after main merge

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* final

* fix: remove accidental R install from multiplayer Dockerfile

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: remove R from Windows build and DockerfileExtra

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: rename R migration to avoid timestamp collision with trigger_filter_logic

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* all

* fix: R install improvements - suppress verbose output, flat lockfile logging, Dockerfile R support, rlimits

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: add clear error when Rscript binary is missing

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: fix type errors in R fallback parser, use format! in wrap(), add R system prompts

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: pyranota <pyra@duck.com>
2026-04-01 06:11:37 +00:00
Alexander Petric
7069202190 fix: approval page freeze, stale state, and missing approval link (#8653)
* fix: prevent browser freeze when approval form number field has no default value

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: disable approval buttons and keep polling after approve/deny action

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: restore approval page link and prevent double resume in flow viewer

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: guard against NaN fallback in Range and reset actionTaken on new approval step

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix approval page url

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-04-01 05:25:22 +02:00
Ruben Fiszel
df7a8eebcf chore(main): release 1.671.0 (#8650)
* chore(main): release 1.671.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-03-31 21:26:34 +00:00
centdix
2862c1cf56 add codex PR review workflow (#8626)
* feat: add codex PR review workflow

* refactor: simplify codex PR review comments

* chore: use ubicloud for codex review

* fix: harden codex review workflow

* chore: use chatgpt auth for codex review
2026-03-31 19:21:39 +00:00
centdix
d67223de9b chore: use fully qualified tmux pane targets in webmux systemPrompt (#8651)
* fix: use fully qualified tmux pane targets in webmux systemPrompt

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: anchor tmux pane targets to $TMUX_PANE for stability across window switches

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 19:16:38 +00:00
Ruben Fiszel
da8886be85 feat: add configurable preview job tag override in default tags settings (#8649)
* feat: add configurable preview job tag override in default tags settings

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: skip re-tagging for FlowPreview jobs when preview override is active

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 18:59:23 +00:00
centdix
040a199685 feat: support hub flows in raw app runnables (#8627)
* feat: support hub flows in raw app runnables

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: support hub flow previews in app ui

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: move trigger context into flow graph viewer

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use script viewer for hub flow steps

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: stretch raw app flow previews to pane height

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: improve hub flow run links

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: stabilize hub flow preview drawer

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: align hub flow id validation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* style: fix runnable panel indentation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-31 18:26:56 +00:00
Alexander Petric
6c3c971af5 feat: improve CLI flow log streaming and job inspection (#8644)
* fix: improve CLI flow log streaming, sub-job listing, and failure handling

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add hierarchical flow status in job get and aggregated flow logs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove duplicate ansi color hint in job logs output

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: update cli-commands skill with new job/flow features

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add integration tests for flow job inspection and log aggregation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: remove internal friction discovery doc from branch

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: trim cli-commands skill to reduce context bloat

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: update job command descriptions and regenerate skills.ts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: commit auto-generated files from system_prompts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address review comments on flow streaming and test assertions

- Move for-loop waiting logic outside --silent guard (Cubic #2)
- Break outer loop when for-loop module fails (Cubic #3)
- Strengthen test assertion: toContain("a") -> toContain("a: Generate data") (Cubic #1)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: generator regex truncating descriptions with parentheses

The .command() regex used [^)]+ for the second arg, stopping at the
first ')' inside description strings like "(machine-friendly)".
Now matches quoted strings properly before falling back.

Fixes 6 truncated descriptions across job, flow, and script commands.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 18:22:18 +00:00
Ruben Fiszel
852c59efbb fix: return default_args/enums in approval info and fix subflow resume buttons (#8648)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 18:16:47 +00:00
Ruben Fiszel
89d1acda24 chore(main): release 1.670.0 (#8625)
* chore(main): release 1.670.0

* update
2026-03-31 16:01:05 +00:00
Ruben Fiszel
12ea7e7423 fix: resolve missing form schema for nested suspend steps in FlowNode sub-flows (#8643)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 15:36:46 +00:00
Ruben Fiszel
375fb66abe feat: support sensitive/secret fields for non-string types (#8635)
* feat: support sensitive/secret fields for non-string types

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: restrict sensitive toggle to object type, move after showExpr

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: show sensitive toggle in PropertyEditor at bottom, after children

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: gate sensitive toggle with showSensitiveToggle prop

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: process secret args in flow test and script test paths

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: inline SecretArgInput into ArgInput, delete component

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address CI review feedback

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: pass showSensitiveToggle to flow input schema editors

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use explicit prop syntax to satisfy svelte-check

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: narrow try/catch to only processSecretArgs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 15:13:23 +00:00
Ruben Fiszel
52a04d210f fix: preserve flow notes/groups and field ordering in generate-metadata (#8641) (#8642)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 13:48:22 +00:00
Ruben Fiszel
cf30bcf3f9 5x request size limit for raw app bundle uploads (#8640)
* feat: 5x request size limit for raw app bundle uploads

Raw app bundle endpoints (create_raw, update_raw) now get 5x the
configured request size limit. Also improves error messages when
multipart uploads exceed the limit to include the actual limit
and mention it's adjustable in instance settings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: mention size limit as possible cause, not definitive

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 11:14:20 +00:00
Ruben Fiszel
b69d63eeb0 warn about linked item deletion in variable/resource delete modal (#8639)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 11:11:10 +00:00
Ruben Fiszel
26050f96c3 feat: support multiline secrets in resource password fields (#8637)
* feat: support multiline secrets in resource password fields

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: wire minRows through to password textarea instead of disabling it

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: auto-detect multiline in password field instead of always using textarea

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: switch to textarea on Enter keypress in password field

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: focus textarea after switching from single-line password input

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: simplify Password multiline logic and fix cursor position bug

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 09:33:26 +00:00
Diego Imbert
c5fccd2f69 fix: remove timeout on python client httpx to prevent ducklake query timeouts (#8636)
* fix: set 300s timeout on python client httpx to prevent ducklake query timeouts

The httpx.Client was using the default 5s timeout, causing ducklake SQL
queries (which run synchronously via run_inline_preview_script) to timeout
for any query taking longer than 5 seconds.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: disable timeout on python client httpx

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Update client.py

* Update client.py

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 09:31:59 +00:00
Ruben Fiszel
92b9ac72c5 fix: compute highest workspace role across all instance groups (#8633)
* fix: compute highest workspace role across all instance groups when adding user to group

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref.txt

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: add regression tests for instance group role precedence

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 15dd361793564ab50bc485b71b042da4cfc1d32c

This commit updates the EE repository reference after PR #501 was merged in windmill-ee-private.

Previous ee-repo-ref: 7e9d0e7ec0b52585cab368b7351e3a5654da3589

New ee-repo-ref: 15dd361793564ab50bc485b71b042da4cfc1d32c

Automated by sync-ee-ref workflow.

* refactor: extract compute_highest_workspace_role helper and clean up role logic

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref.txt

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to e08a87450627bef9013498e40ee93a47bedda7ee

This commit updates the EE repository reference after PR #502 was merged in windmill-ee-private.

Previous ee-repo-ref: a4ba20109813332320839488da5ecf83ca8f70b4

New ee-repo-ref: e08a87450627bef9013498e40ee93a47bedda7ee

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-31 09:08:48 +00:00
Ruben Fiszel
fbb6170650 rm cpus limit 2026-03-31 07:19:38 +00:00
Ruben Fiszel
d6957aaf31 remove NUN_WORKERS 2026-03-31 07:16:33 +00:00
Ruben Fiszel
cd8edcd94f feat: expose getJob and getJobLogs as MCP tools (#8632)
* feat: expose getJob and getJobLogs as MCP tools

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: add integration test for getJob/getJobLogs MCP endpoint tools

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: add MCP client integration test for getJob and getJobLogs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 07:09:34 +00:00
Ruben Fiszel
9d85768287 local-review to create PR 2026-03-31 07:04:32 +00:00
Ruben Fiszel
0520b11d5d nit improve local review 2026-03-31 07:02:09 +00:00
Alexander Petric
c41e3adcc6 db health page improvements (#8631)
* db health page improvements

* Update SQLx metadata

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-31 05:55:38 +00:00
Diego Imbert
ef1757f5d7 fix: Ducklake UI Nits (#8628) 2026-03-30 20:00:14 +00:00
Ruben Fiszel
bfc2aefdb8 fix: smarter secret masking based on secret length (#8629)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 19:59:08 +00:00
Ruben Fiszel
3876902a7b feat: add OR logic support to kafka/websocket trigger filters (#8580)
* feat: add OR logic support to kafka/websocket trigger filters

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref for OR logic filter support

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add filter_logic to OpenAPI spec/save utils, fix websocket derive, show capture group ID

- Add filter_logic field to all 6 Kafka/WebSocket OpenAPI schemas so it
  is included in the generated frontend client types
- Include filter_logic in save request bodies (kafka/utils.ts, websocket/utils.ts)
- Fix misplaced #[derive(FromRow)] on WebsocketConfig (was on the default fn)
- Show copyable "Test group ID" in Kafka capture UI
- Remove capture event-loss warning for Kafka (uses separate consumer group)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* update sqlx

* update ee ref

* chore: regenerate system prompts for filter_logic schema changes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove banned $bindable(default_value) pattern in TriggerFilters

Use $bindable() without default and $derived with ?? for the effective
value, per CLAUDE.md rules.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: make filterLogic prop required in TriggerFilters

All callers always pass it, no need for optional + derived fallback.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 5ee1382dfb23b6a1516e3c7586058cec8240fdf2

This commit updates the EE repository reference after PR #498 was merged in windmill-ee-private.

Previous ee-repo-ref: bbd674991c07bff1cb2f3744e71fda10df53f09d

New ee-repo-ref: 5ee1382dfb23b6a1516e3c7586058cec8240fdf2

Automated by sync-ee-ref workflow.

* fix: reset filterLogic to 'and' in openNew for kafka/websocket editors

Prevents stale OR logic from carrying over when creating a new trigger
after editing one with OR filters.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: hugocasa <hugo@casademont.ch>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-30 19:32:24 +00:00
Ruben Fiszel
8e973c892d fix: cap input history per_page to 100 on cloud (#8624)
Prevents excessive memory/query load from large per_page values on the
inputs/history endpoint in cloud environments.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 18:44:51 +00:00
Ruben Fiszel
2d27b17a05 chore(main): release 1.669.1 (#8621)
* chore(main): release 1.669.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-03-30 16:56:21 +00:00
hugocasa
96d4b3f123 add plain_emails_telemetry setting (#8622)
* feat: add plain_emails_telemetry notification handler

Listen for plain_emails_telemetry setting changes via the global settings
notification channel, matching the workspace_telemetry_enabled pattern.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref.txt

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 780857855e231c9d71f02fefd8253c254542ef32

This commit updates the EE repository reference after PR #500 was merged in windmill-ee-private.

Previous ee-repo-ref: 393c9c4ebd317d09466866ab17cc1ee9fc25582a

New ee-repo-ref: 780857855e231c9d71f02fefd8253c254542ef32

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-30 16:42:58 +00:00
hugocasa
4817913f0c fix: avoid doubled /oauth2 path in Okta custom authorization server URLs (#8620)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 16:00:40 +00:00
Alex Petric
d532c1d470 fix: improve db health UI text and prevent label wrapping
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-30 11:54:56 -04:00
Ruben Fiszel
b5185b0e12 chore(main): release 1.669.0 (#8606)
* chore(main): release 1.669.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-03-30 15:54:17 +00:00
Ruben Fiszel
abc6b12d68 feat: WAC workflow diagram visualization via WASM (#8604)
* feat: WAC workflow diagram visualization in script editor

Add WASM-powered workflow diagram for WAC scripts in the script editor,
inspired by Cloudflare's workflow diagrams approach. Parses WAC code
client-side via WASM and renders an interactive DAG using @xyflow/svelte.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: show WAC diagram on script detail page

Show the workflow diagram below the run form on the script detail page
for WAC scripts, matching how flows display their graph.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: handle try/catch and while loops in WAC diagram

Instead of rejecting these patterns with validation errors, render them
as graph nodes:
- try/catch → Branch node with "try"/"catch" edge labels
- while loops → LoopStart/LoopEnd with condition as iter_source

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: remove dead code from WAC parser and add pkg-wac to publish script

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: use published windmill-parser-wasm-wac@1.668.5

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: handle undefined language prop in WacDiagram usage

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: resolve windmill-parser-wasm-wac from npm registry in lockfile

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: create actual merge nodes for branch/try-catch convergence points

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 15:34:07 +00:00
Ruben Fiszel
ee6231590e fix: enable S3 bundle cache for PHP previews without lock file (#8608)
* perf: enable S3 bundle cache for PHP previews without lock file

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: resolve borrow-after-move of lock in php cache save

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: use DB-based lockfile cache for PHP previews instead of requirements-only key

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: prevent stale lockfile TTL refresh in PHP preview cache

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: add sqlx offline cache entry for PHP lockfile resolution query

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-30 15:16:38 +00:00
centdix
ee1afb2415 chore: add missing native-trigger skill frontmatter (#8616)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-30 14:43:36 +00:00
Ruben Fiszel
adfb0bd5ff add suggestIntegration and suggestScript to flow builder customUI (#8615)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-30 12:36:37 +00:00
centdix
98934d59c5 fix: use route_service instead of fallback_service for MCP router (#8614)
fallback_service on a router with no explicit routes is invisible to
axum's nest() — requests never reach the nested fallback, resulting in
404s. route_service("/", service) registers an actual route so nest()
forwards correctly.

Also reverts layer back to route_layer for the ApiAuthed extractor
since there is now a real route to match against.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 11:54:07 +00:00
Ruben Fiszel
33032ed297 fix: enforce workspace isolation on flow resume endpoint (#8612)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-30 10:54:18 +00:00
Diego Imbert
22da5bd9ea fix: handle DuckDB boolean types in ColumnDef deserializers (#8610)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 10:27:46 +00:00
Diego Imbert
f3012ee7cc fix: cast DuckDB IS_NULLABLE to string in metadata query (#8607)
DuckDB's information_schema.columns returns IS_NULLABLE as a boolean,
but the ColumnDef struct expects a string ('YES'/'NO'). This caused
deserialization to fail with "invalid type: boolean, expected a string"
when expanding WM_INTERNAL_DB markers (e.g. COUNT) for Ducklake tables.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 09:14:26 +00:00
Ruben Fiszel
5f2d3e6812 fix: add path traversal check in service_logs get_log_file endpoint (#8605)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 08:18:08 +00:00
Ruben Fiszel
9bcda7023f chore(main): release 1.668.5 (#8594)
* chore(main): release 1.668.5

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-03-29 15:25:16 +00:00
Ruben Fiszel
9f86c72133 widen login rate limit defaults to reduce false positives (#8603)
* fix: widen login rate limit defaults to reduce false positives

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: increase rate limits further for large-scale deployments

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-29 14:01:13 +00:00
Ruben Fiszel
754b88a52c fix: harden login rate limiting with CLOUD_HOSTED gating and memory eviction (#8602)
* fix: harden login rate limiting with CLOUD_HOSTED gating, memory eviction, and race fix

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: replace global DashMap with atomics and move extract_client_ip inside conditional

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-29 13:52:13 +00:00
Ruben Fiszel
06bbe7b94b fix: add per-IP and per-account brute force protection on login endpoint (#8601)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-29 13:16:11 +00:00
Ruben Fiszel
970e859a41 fix: validate JSON before sql_builder bind to prevent injection via JSONB queries (#8599)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-29 12:51:15 +00:00
Ruben Fiszel
845db72b73 fix: prevent SSRF and local file read via git repository resource URLs (#8600)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-29 12:48:55 +00:00
Ruben Fiszel
f5fc9f8485 fix: require mcp: scope for MCP endpoints instead of blanket bypass (#8597)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-29 12:39:48 +00:00
Ruben Fiszel
be7fbeb8b1 fix: disable workspace webhook events when CLOUD_HOSTED (#8598)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-29 12:34:00 +00:00
Ruben Fiszel
8c770a206a fix: rename snippet param to avoid svelte compiler shadowing bug in asset usages drawer (#8595)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-29 12:21:37 +00:00
Ruben Fiszel
74fba2abf3 fix: add timestamp validation to webhook signature verification (#8596)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-29 12:19:59 +00:00
Ruben Fiszel
b4d1f2aac7 fix: use constant-time comparison for API key and basic auth validation (#8593)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-29 12:01:52 +00:00
Ruben Fiszel
8baa7f8a20 chore(main): release 1.668.4 (#8592)
* chore(main): release 1.668.4

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-03-29 07:21:52 +00:00
Ruben Fiszel
0549f682fe fix: update git sync version to latest cli 2026-03-29 06:38:53 +00:00
Ruben Fiszel
73f649c152 chore(main): release 1.668.3 (#8591)
* chore(main): release 1.668.3

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-03-28 21:04:44 +00:00
Ruben Fiszel
c6ce3197a7 fix(cli): phantom diffs, flow safety, trigger DX, lint watch, error clarity (#8588)
* fix(cli): phantom diffs, flow push safety, error messages, digest stability

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): differentiate stale vs missing metadata warnings on script push

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): job list --limit off-by-one, deps push double error

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): flow get shows nested steps, lint works on specific directories

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(cli): add lint --watch mode for continuous validation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): email trigger template missing local_part, trigger get shows all fields

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): fix CI — flow push warns instead of failing, lint subdir detection

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 20:56:10 +00:00
Ruben Fiszel
37799574d8 chore(main): release 1.668.2 (#8586)
* chore(main): release 1.668.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-03-28 15:56:53 +00:00
Ruben Fiszel
78ac28b4e0 fix(cli): address review — createBundle appDir, shared arg validation (#8587)
* fix(cli): address review — createBundle appDir, shared validateRequiredArgs, warn on fetch failure

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(cli): add coverage for exit codes, arg validation, variable add, job logs, push --message

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): fix test — create script with required schema, relax push --message assertion

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 15:55:40 +00:00
Ruben Fiszel
f40cdaf434 fix(cli): app push crash, lint path, push --message, run validation, history timestamps (#8585)
* fix(cli): app push crash, lint entry point, push --message, run arg validation, history timestamps

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): update sqlx cache and fix second history query missing created_at

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore(cli): regenerate system prompts after new CLI options

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 14:33:49 +00:00
Ruben Fiszel
0ea9b945e0 chore(main): release 1.668.1 (#8583)
* chore(main): release 1.668.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-03-28 10:51:49 +00:00
Ruben Fiszel
38acaa3653 fix(cli): fix 13 CLI bugs — exit codes, sync tar fallback, variable encryption, JSON output (#8582)
* fix(cli): fix 13 CLI bugs — exit codes, sync tar fallback, variable encryption, JSON output, parent dirs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): address PR review — TarAsZip.folder(), retry timeout, stderr hint

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): update resource-type list test to handle empty state message

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 10:46:01 +00:00
Ruben Fiszel
e34acafce5 chore(main): release 1.668.0 (#8575)
* chore(main): release 1.668.0

* Apply automatic changes

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <217088191+windmill-internal-app[bot]@users.noreply.github.com>
2026-03-28 09:43:55 +00:00
Alexander Petric
9ceab730d7 feat: add DB health diagnostic dashboard for superadmins (#8574)
* feat: add DB health diagnostic dashboard for superadmins

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Update SQLx metadata

* fix: improve db health query performance

Bound large_results scan to last N jobs (configurable via scan_limit
query param, default 10K) instead of full-table pg_column_size sort.
Replace N+1 datatable size queries with single batched pg_class lookup.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Update SQLx metadata

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* sqlx

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-03-28 09:32:10 +00:00
Ruben Fiszel
d29cb234db feat(cli): add job, group, audit, token commands and schedule enable/disable (#8581)
* feat(cli): add job, group, audit, token commands and schedule enable/disable

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore(cli): regenerate system prompts after new commands

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): address PR review feedback

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore(cli): regenerate system prompts after review fixes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(cli): extract shared formatTimestamp util and remove unused resolveWorkspace in token

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 09:18:55 +00:00
Ruben Fiszel
820f28f879 fix: trigger capture filter and focus issues (#8579)
* fix: replace label with div for filter value editor to fix focus stealing

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref.txt

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 02c0d34e54e71c9293f9cefb56f68652cf0db8a5

This commit updates the EE repository reference after PR #497 was merged in windmill-ee-private.

Previous ee-repo-ref: 44d665af35ad23cd3549b1d094f5d6633237deb4

New ee-repo-ref: 02c0d34e54e71c9293f9cefb56f68652cf0db8a5

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-28 08:53:40 +00:00
Ruben Fiszel
501a4ff2a9 fix: Improve CLI developer experience: error handling, sync workflow, JSON output, workspace forks (#8578)
* fix(cli): address 28 DX friction points across CLI commands

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore(cli): regenerate system prompts after help text updates

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): address PR review feedback

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): update removeType tests to match lenient behavior

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): address CE/EE sync friction and improve JSON output

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): revert instance config masking to avoid breaking push flow

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): mask instance secrets by default with interactive prompt

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore(cli): regenerate system prompts

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): use stderr for errors, optimize skipped-files scan, rename --auto to --auto-metadata

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(cli): improve workspace fork lifecycle — delete-fork fallback, list-forks, --workspace override

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): update fork merge instructions to reference all merge methods

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): clarify skipped-files warning comment re DynFSElement traversal

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-28 08:41:52 +00:00
Ruben Fiszel
95688884ce update ee-repo-ref to fix deprecated rand API in CI (#8577)
* [ee] fix: update ee-repo-ref to fix deprecated rand API in CI

Updates ee-repo-ref.txt to point to a commit that replaces deprecated
rand::thread_rng().gen() with rand::rng().random() in the MITM proxy
cert generation, fixing the check_ee_full CI failure.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 9316adc693d7f1a668df661e000109bb48b93375

This commit updates the EE repository reference after PR #495 was merged in windmill-ee-private.

Previous ee-repo-ref: d311a3c6ecb50c086fb86b1f4fa3f9e62ff40df5

New ee-repo-ref: 9316adc693d7f1a668df661e000109bb48b93375

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-28 00:09:38 +00:00
Ruben Fiszel
ce2e6c8c01 fix: add Authority Key Identifier to MITM proxy leaf certs (#8576)
* test: add x509-parser dev-dep for MITM proxy cert tests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref.txt for ssl-verify-fix branch

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref.txt

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to a90b083660b372bf1da1c18769cbd50936ea8040

This commit updates the EE repository reference after PR #494 was merged in windmill-ee-private.

Previous ee-repo-ref: db665a09d5b9a485977d73c22908629e3dda6200

New ee-repo-ref: a90b083660b372bf1da1c18769cbd50936ea8040

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-27 23:49:40 +00:00
Alexander Petric
56253c04cb feat: IAM RDS auth for PostgreSQL worker resources (#8573)
* feat: add IAM RDS auth support for PostgreSQL worker resources

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: use Config builder for IAM RDS connections

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review feedback for IAM RDS auth

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref to ebea6ef1e5bfcfc3f0151da9687dac6c61bbfab6

This commit updates the EE repository reference after PR #493 was merged in windmill-ee-private.

Previous ee-repo-ref: 1228561a98c5195bb97a81d4a57ce2bb2ecfca79

New ee-repo-ref: ebea6ef1e5bfcfc3f0151da9687dac6c61bbfab6

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-27 21:50:29 +00:00
Ruben Fiszel
522da50c97 chore(main): release 1.667.0 (#8549)
* chore(main): release 1.667.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-03-27 20:03:54 +00:00
Ruben Fiszel
80cf26bb61 nit npm checks 2026-03-27 19:39:55 +00:00
Pyra
248188aaa2 nit: add workflow_dispatch to cli tests (#8479) 2026-03-27 19:28:48 +00:00
centdix
a8b651da9f fix(cli): preserve inline script files during flow generate-locks (#8561)
* fix(cli): preserve inline script files during flow generate-locks

Three bugs caused `wmill flow generate-locks` to destroy inline script
content and rename files:

1. YAML parser stripped unquoted `!inline` tags (treated as YAML tag,
   not string prefix), leaving just the filename as script content.
   Fix: register custom YAML tags for `!inline` and `!inline_fileset`.

2. Inline script files were renamed based on step summaries because
   `extractInlineScriptsForFlows` was called with empty mapping `{}`.
   Fix: call existing `extractCurrentMapping()` before replacement and
   pass the mapping to preserve original filenames.

3. Lock file paths were derived from the assigner instead of the mapped
   content path, causing inconsistent naming.
   Fix: derive lock base path from mapped content path when available.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(cli): add unit tests for !inline YAML tag and mapping preservation

- YAML tag tests: unquoted/quoted !inline parsing, !inline_fileset,
  nested structures, round-trip stability
- Mapping tests: path preservation with mapping, fallthrough without
  mapping, lock path derivation from mapped content path, mixed
  mapped/unmapped modules, dotted path handling

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): correct yaml parse type cast and inline prefix check

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): harden lock path for extensionless files and merge customTags

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 19:27:56 +00:00
Ruben Fiszel
3959fe8297 feat: add workspace-level service accounts (#8560)
* feat: add workspace-level service accounts (EE)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* sqlx

* sqlx

* chore: update ee-repo-ref

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 19:23:03 +00:00
Alexander Petric
dc75b73edc improve logging for github app operations (#8568)
* improve logging for github app operations

* ee ref

* chore: update ee-repo-ref to 0b9e92f9e089293c6d523b77ed2c11edbc7a99c0

This commit updates the EE repository reference after PR #489 was merged in windmill-ee-private.

Previous ee-repo-ref: b259642e7f36b83a991034d5b28ae616f94ee5fc

New ee-repo-ref: 0b9e92f9e089293c6d523b77ed2c11edbc7a99c0

Automated by sync-ee-ref workflow.

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-03-27 18:41:10 +00:00
Ruben Fiszel
5e5da4f7ef test: add OTEL coverage tests (#8558)
* test: add OTEL coverage tests

Add 38 unit tests covering OpenTelemetry infrastructure:
- OtelSettings serde (empty, partial, full, roundtrip, skip_serializing)
- OtelTracingProxySettings serde (defaults, languages, dedup, rejection)
- ScriptLang rename cases
- LogCounter initialization and CountingLayer event counting
- Targets filter suppression of windmill:job_log
- get_otel_context_envs traceparent format verification
- Worker OtelTracingProxySettings (HashSet variant)

Companion EE PR adds tests for span_cx_from_job_id, metric functions,
proto conversion, SpanBuilder, and tracing proxy handler.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: add E2E OTEL tests with in-memory exporters

Add integration tests that verify metrics and spans flow correctly
through the OpenTelemetry pipeline using in-memory exporters:

Metrics (1 comprehensive test):
- All 20 metric names registered correctly
- Counter values (push/delete/pull/zombie/execution/failed/started)
- Gauge values with attributes (queue count by tag, worker busy, db pool, health)
- Histogram values (execution duration, pull duration)
- Health status phase encoding (healthy=1, degraded=0, unhealthy=0)

Spans (6 tests):
- Root job span created with "full_job" name and Ok status
- Error status with "Job failed" description on failure
- trace_id derived from job UUID
- span_id derived from job UUID low bits
- Child jobs (with parent_job) produce no span
- Attribute values (job_id, workspace_id, script_path) match job data

Also:
- Add testing feature to opentelemetry_sdk for InMemoryMetricExporter
- Update otel_oss.rs for SdkTracer type rename in 0.30
- Add opentelemetry/opentelemetry_sdk to dev-dependencies

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: remove unit tests in favor of E2E OTEL tests

The E2E integration tests in backend/tests/otel.rs cover the same
ground more thoroughly with in-memory exporters.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 18:28:59 +00:00
Ruben Fiszel
7a14d38d4a use layer instead of route_layer for MCP router to prevent axum 0.8 panic (#8572)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 18:12:52 +00:00
hugocasa
63a3573951 fix: multi-script dedicated workers race on shared job_dir (#8551) (#8569)
* [ee] fix: update ee-repo-ref for dedicated worker job_dir fix

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* [ee] fix: update ee-repo-ref

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 5e8b1bcfc2c9ade9db39c839f2faed4f82da5efc

This commit updates the EE repository reference after PR #490 was merged in windmill-ee-private.

Previous ee-repo-ref: d958cd3b8a9a17b5f3cb6cb411c8ebba0c380fdd

New ee-repo-ref: 5e8b1bcfc2c9ade9db39c839f2faed4f82da5efc

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-27 17:57:57 +00:00
Ruben Fiszel
b592996eee feat: add schedule support to CLI branch-specific items (#8570)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 17:52:53 +00:00
Ruben Fiszel
bc7007bb42 fix: include importer_kind in dependency debounce key to prevent cross-kind collisions (#8567)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 16:22:35 +00:00
Ruben Fiszel
99b0ebd677 use fallback_service instead of nest_service for MCP router (#8566)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-27 16:14:47 +00:00
centdix
5fd2c1a129 chore(cli): separate unit tests from integration tests and fix test cleanup (#8562)
* fix(cli): separate unit tests from integration tests and fix test cleanup

- Rename 14 non-backend test files to *_unit.test.ts convention
- Add UNIT_ONLY env var guard in setup.ts to skip cargo build/backend startup
- Add test:unit and test:integration scripts to package.json
- Use setsid on Linux for process group management so stop() kills both
  cargo and the windmill child process
- Fix exit handler to kill process group instead of just the direct child
- Add cleanupStaleTestResources() to drop orphaned windmill_test_* databases
  and kill orphaned backend processes on startup
- Rewrite TESTING.md with current bun-based instructions

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): fix process group approach - kill by db name instead of setsid

The setsid approach didn't work because setsid forks, making the PID
we get from Bun.spawn ephemeral. Instead, kill orphaned windmill child
processes by matching our unique database name in /proc/pid/environ.

Also add afterAll hook in setup.ts so full async cleanup (process kill
+ database drop) runs when all tests complete normally, not just on
SIGINT/SIGTERM.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): address PR review feedback

- Remove duplicate cleanupStaleTestResources() call in getTestBackend()
  (already called in setup.ts)
- Add regex guard on database names before SQL interpolation
- Extract shared killWindmillProcessesByEnvMatch() helper to deduplicate
  process-killing logic
- Remove redundant test:integration script (test already runs everything)
- Flip setup.ts to if/else pattern for readability

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 16:13:33 +00:00
centdix
70f3ee5ed4 fix: use admin db pool in get_copilot_settings_state (#8564)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 15:21:42 +00:00
Ruben Fiszel
8df1d8ec17 test nits 2026-03-27 12:28:54 +00:00
Ruben Fiszel
2f32675801 feat: DB-coordinated graceful restart staggering for settings changes (#8555)
* feat: add DB-coordinated graceful restart staggering for settings changes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: preserve original instance names in restart coordination record

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: remove randomness, add drain delay for in-flight requests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: spawn restart in background, deduplicate entries, clarify stale filter

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 11:59:17 +00:00
Ruben Fiszel
ab868e9ebc perf: enable bun bundle caching for WAC v2 scripts (#8556)
WAC v2 scripts previously disabled bundle caching, forcing every execution
to resolve windmill-client from node_modules at runtime (~74ms overhead per
bun launch). This makes both the prebundle and execution paths WAC-aware by
including WorkflowCtx/StepSuspend/setWorkflowCtx re-exports in the bundle,
so the wrapper can import them from the cached bundle instead of node_modules.

Benchmarked improvement: wac_inline_2 12→38 wf/s (3.2x), wac_seq_2 6→17 wf/s
(2.8x) with no regression on plain bun scripts or flows.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 11:58:24 +00:00
centdix
ad19ac9b37 feat: support multiple folder selection in MCP scope selector (#8557)
* feat: support multiple folder selection in MCP scope selector

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add per-folder caching for multi-folder runnables loading

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address PR review — workspace prop, length check, empty folder state

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: cache folder names per workspace and reload on workspace change

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 11:57:47 +00:00
Ruben Fiszel
0fb115304a fix: preserve notes on nodes inside collapsed groups (#8552)
* fix: preserve notes on nodes inside collapsed groups

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: hide notes for nodes inside collapsed groups instead of repositioning

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 11:55:47 +00:00
Ruben Fiszel
79cc4a92d8 fix: emit 0 for OTEL queue metrics when tag queue is empty (#8559)
Previously, windmill.queue.count and windmill.queue.running_count OTEL
metrics would report no data instead of 0 when a tag's queue emptied.
This was because the SQL query uses GROUP BY tag, so empty tags are
absent from results. The Prometheus path already handled this by tracking
previously-seen tags and emitting 0, but the OTEL path was missing this
logic.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 11:55:09 +00:00
Ruben Fiszel
943fe9c6cc fix: handle inline script deletion in sync push + flow new nonDottedPaths (#8553)
* fix: handle inline script file deletions in app/flow folders during sync push

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: add regression test for app inline script deletion during sync push

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: flow new respects nonDottedPaths setting

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: add flow new nonDottedPaths test

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: separate stat from pushObj in delete handler to avoid masking errors

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 11:54:45 +00:00
Ruben Fiszel
e15bfbf91e fix: sanitize flow step summaries for filesystem-safe names (#8554)
* fix: sanitize flow step summaries for filesystem-safe names

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: bump windmill-utils-internal to 1.3.6

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: handle Windows reserved device names in flow step sanitization

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: collapse consecutive underscores in sanitized flow step names

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: bump windmill-utils-internal to 1.3.7

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* bump

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-27 11:38:20 +00:00
centdix
d06b42613f feat(cli): generate commented wmill.yaml and add config reference command (#8546)
* feat: generate commented wmill.yaml template and add config reference command

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add missing options to config reference (promotion, skipBranchValidation, commonSpecificItems)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: generate YAML template from CONFIG_REFERENCE instead of handwritten string

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: preserve YAML comments when binding workspace profile during init

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: simplify to `wmill config` and reorder table columns

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: generate JSON Schema for wmill.yaml editor autocomplete and validation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: remove redundant templateValue fields and make specificItemsSchema data-driven

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: use native JSON Schema types in CONFIG_REFERENCE, strip non-schema keys for generation

Eliminates typeToJsonSchema, specificItemsSchema, codebaseItemSchema,
branchConfigSchema, and the complex generateJsonSchema body. Each
CONFIG_REFERENCE entry is now a JSON Schema property with extra metadata.
Schema generation just iterates and strips non-schema keys.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: remove typeLabel and displayType — use schema types directly

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: remove hidden entries, auto-expand nested schemas in reference table

Sub-fields (codebases[], gitBranches.<branch>.*) are now derived from
the parent's inline schema instead of being maintained as duplicate
hidden entries. Removes 29 entries and the hidden field entirely.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use console.log for JSON output and quote YAML-special branch names

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: regenerate system prompts to include new config command

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: review feedback + add tests for template, schema, and config reference

- Use console.log for --json output (no ANSI escape codes)
- Quote branch names with YAML-special characters
- Add 28 tests covering template generation, JSON Schema validation,
  config reference formatting, and CONFIG_REFERENCE integrity

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add generate-schema script and commit wmill.schema.json to repo

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: remove schema.json generation from wmill init

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: eliminate read-back cycle, harden yamlKey, fix triple negation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 11:35:28 +00:00
Ruben Fiszel
0389d9601c chore: upgrade axum 0.7 to 0.8 (#8539)
* chore: upgrade axum 0.7 to 0.8 and related dependencies

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: add route reachability tests for ~80 previously untested endpoints

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: switch feature-gated trigger handlers from axum::async_trait to async_trait crate

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: update new trash routes to axum 0.8 path syntax

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to latest EE commit

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: upgrade route tests to assert 2xx responses with proper data setup

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: restore npm_proxy and ai_routes tests using local echo servers

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: gate workspace fork test behind enterprise feature flag

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: add ~40 more endpoint tests (jobs authed, health, favorites, ACLs, reachability)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address review findings from axum 0.8 upgrade

- Use cookie value_trimmed() instead of value() for cookie 0.18 compat
- Update comments still referencing old :workspace_id syntax

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 61ae055ea31481f1899953e9d5f65566b8c707b1

This commit updates the EE repository reference after PR #486 was merged in windmill-ee-private.

Previous ee-repo-ref: 0059d175a6fdddf52998b183bf91059b224704ac

New ee-repo-ref: 61ae055ea31481f1899953e9d5f65566b8c707b1

Automated by sync-ee-ref workflow.

* test: add test for new get_imports endpoint

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove unused import in raw_apps test

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-27 09:55:04 +00:00
Ruben Fiszel
9e235937ce add WAC v2 benchmarks and improve benchmark infrastructure (#8550)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 08:53:46 +00:00
Ruben Fiszel
e2cc6e4709 nit sqlx 2026-03-26 20:58:23 +00:00
Tristan TR
c0aafee9a9 feat: improve-replay-ui (#8250)
* Improve UI of script record

* Improve UI for scripts

* Remove Result & Logs loading container while flow not finised

* Improve Graph view

* Add click on a step mention

* Fix spacing when empty

* Fix step duration disappearing in recorded flows

* Modernize timeline tab

* Improve Script recording result UI

* feat: externalize recording player controls for fake-window embedding

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: reorder FlowViewer tab sync effects for clarity

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: eliminate tab sync effects in FlowViewer, use selectedTab directly

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: remove unnecessary untrack in FlowViewer tab init

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: skip tab auto-selection when selectedTab is controlled externally

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: export recording types from package

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: non-null assertion for recording.flow in FlowGraphViewer

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: replace banned $bindable(default_value) pattern and simplify tab sync

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use svelte 5 onclick syntax on replay page

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: skip db clock endpoint during replay mode

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove line numbers from script recording code display

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: hugocasa <hugo@casademont.ch>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 18:52:15 +00:00
Ruben Fiszel
264fa33917 chore(main): release 1.666.0 (#8543)
* chore(main): release 1.666.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-03-26 18:46:25 +00:00
wendrul
d760ea5eaf fix: add relative imports to the dependency list in deploymentUI (#8548)
* prepare sqlx

* Add relative imports to getDependencies of deployUI

* nit

* fix: correct get_imports doc comment, add tracing, use Set for dedup

- Fix copy-pasted doc comment on get_imports (said "get dependents")
- Add tracing::debug to get_imports handler to match get_dependents
- Use Set for O(1) duplicate detection in deploy dependency traversal

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 18:28:18 +00:00
Ruben Fiszel
8866bd44cf nit backend tests 2026-03-26 18:20:46 +00:00
Ruben Fiszel
71549c3db0 fix: resolve parent_hash race condition in sync push with auto_parent (#8545)
* fix: resolve parent_hash race condition in sync push with auto_parent

During concurrent sync push operations (parallel CLI groups or separate
CI pipelines), multiple requests could read the same remote script hash
and both try to create a new version with the same parent_hash, causing
"the lineage must be linear" errors.

Adds an opt-in `auto_parent` field to the create_script API. When set,
the backend resolves the parent_hash to the current head script at that
path within the transaction, atomically. This eliminates the client-side
race window where the parent could change between read and write.

The CLI now sends `auto_parent: true` when updating existing scripts,
so sync push is resilient to concurrent deployments.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add missing auto_parent field in clone_script NewScript initializer

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: add advisory lock to serialize concurrent auto_parent script creates

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* sqlx

* fix: add sqlx anchor for CE-only user count query

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 18:14:10 +00:00
Diego Imbert
1fa4d919b3 fix: upload_s3_file not working in VS Code extension (#8547) 2026-03-26 17:40:51 +00:00
centdix
1a73012e07 fix: filter null entries in FileUpload initialValue to prevent s3 access error (#8544)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-26 14:59:45 +01:00
centdix
e44504c6e9 feat: add PDF input support to AI agent (#8525)
* feat: add PDF input support to AI agent with user_attachments field

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: add integration tests for PDF input and backward compat

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add ContentPart::File variant for PDF support across all providers

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: address review feedback on PDF support

- Extract parse_data_url_bytes and mime_to_document_format helpers in Bedrock
- Add is_document_mime helper in ai_types for centralized MIME routing
- Extract s3_object_to_content_part helper to deduplicate image_handler/openai
- Rename AnthropicImageSource to AnthropicBase64Source
- Derive Bedrock DocumentFormat from MIME type instead of hardcoding Pdf

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: merge user message and attachments into single message for Bedrock

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 13:55:10 +00:00
Ruben Fiszel
d7f4b950ce fix: pass pre-bound TcpListener to run_server to fix Windows CI test race (#8542)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 12:42:02 +00:00
487 changed files with 22435 additions and 3712 deletions

View File

@@ -1,3 +1,8 @@
---
name: native-trigger
description: Guidance for adding native trigger services to Windmill. Use when implementing or modifying native trigger integrations across the backend and frontend.
---
# Skill: Adding Native Trigger Services
This skill provides comprehensive guidance for adding new native trigger services to Windmill. Native triggers allow external services (like Nextcloud, Google Drive, etc.) to trigger Windmill scripts/flows via webhooks or push notifications.

25
.claude/review-prompt.md Normal file
View File

@@ -0,0 +1,25 @@
# Code Review Instructions
Review this pull request and provide comprehensive feedback.
## Focus Areas
- **Code quality and best practices** — does the code follow established patterns?
- **Potential bugs or issues** — will this code work correctly in all cases?
- **Performance considerations** — are there unnecessary allocations, N+1 queries, or bottlenecks?
- **Security implications** — injection, auth bypass, data exposure?
## CLAUDE.md Compliance
Read all relevant CLAUDE.md files (root and in directories containing changed files). Check each rule against the changed code. Quote the exact rule when flagging a violation.
## Review Guidelines
- Provide detailed feedback using inline comments for specific issues
- Use top-level comments for general observations or praise
- Only flag issues introduced by this PR, not pre-existing problems
- Self-validate each finding: "Is this definitely a real issue?" If uncertain, discard it
## Testing Instructions
At the end of your review, add complete instructions to reproduce the added changes through the app interface. These instructions will be given to a tester so they can verify the changes. It should be a short descriptive text (not a step-by-step or a list) on how to navigate the app (what page, what action, what input, etc.) to see the changes.

View File

@@ -6,53 +6,24 @@ description: Code review a pull request for bugs and CLAUDE.md compliance. MUST
# Local Code Review Skill
Review a pull request for real bugs and CLAUDE.md compliance violations. This review targets HIGH SIGNAL issues only.
## Review Philosophy
- **Only flag issues you are certain about.** If you are not sure an issue is real, do not flag it. False positives erode trust and waste reviewer time.
- Think like a senior engineer doing a final review — flag things that would cause incidents, not things that are merely imperfect.
## What to Flag
- Code that won't compile or parse (syntax errors, type errors, missing imports)
- Code that will definitely produce wrong results regardless of inputs
- Clear, unambiguous CLAUDE.md violations (quote the exact rule being violated)
- Security issues in introduced code (injection, auth bypass, data exposure)
- Incorrect logic that will fail in production
## What NOT to Flag
- Code style or quality concerns
- Potential issues that depend on specific inputs or runtime state
- Subjective suggestions or improvements
- Pre-existing issues not introduced by this PR
- Pedantic nitpicks a senior engineer wouldn't flag
- Issues a linter or type checker will catch
- General quality concerns unless explicitly prohibited in CLAUDE.md
- Issues silenced via lint ignore comments
Run the same review locally that the GitHub Claude Auto Review action runs on PRs. The shared review instructions live in `.claude/review-prompt.md` — read that file first and follow its instructions.
## Execution Steps
1. **Determine the PR scope**:
1. **Read `.claude/review-prompt.md`** for the review criteria and focus areas
2. **Determine the PR scope**:
- If an argument is provided, use it as the PR number or branch
- Otherwise, detect from the current branch vs main
- Run `gh pr view` if a PR exists, or use `git diff main...HEAD`
2. **Find relevant CLAUDE.md files**:
- Read the root `CLAUDE.md`
- Check for CLAUDE.md files in directories containing changed files
3. **Get the diff and metadata**:
- `gh pr diff` or `git diff main...HEAD` for the full diff
- `gh pr view` or `git log main..HEAD --oneline` for context
4. **Read changed files** where the diff alone is insufficient to understand context
5. **Review for**:
- CLAUDE.md compliance — check each rule against the changed code
- Bugs and logic errors — will this code work correctly?
- Security issues — injection, auth, data exposure in new code
5. **Apply the review instructions from `.claude/review-prompt.md`**
6. **Self-validate each finding**: Before reporting, ask yourself:
- "Is this definitely a real issue, not a false positive?"

View File

@@ -1,3 +1,8 @@
---
name: native-trigger
description: Guidance for adding native trigger services to Windmill. Use when implementing or modifying native trigger integrations across the backend and frontend.
---
# Skill: Adding Native Trigger Services
This skill provides comprehensive guidance for adding new native trigger services to Windmill. Native triggers allow external services (like Nextcloud, Google Drive, etc.) to trigger Windmill scripts/flows via webhooks or push notifications.

View File

@@ -61,12 +61,13 @@ Generated with [Claude Code](https://claude.com/claude-code)
1. Run `git status` to check for uncommitted changes
2. Run `git log main..HEAD --oneline` to see all commits in this branch
3. Run `git diff main...HEAD` to see the full diff against main
4. Check if remote branch exists and is up to date:
4. **Run `/local-review`** before creating the PR. If issues are found, fix them and commit before proceeding. Do not skip this step.
5. Check if remote branch exists and is up to date:
```bash
git rev-parse --abbrev-ref --symbolic-full-name @{u} 2>/dev/null || echo "no upstream"
```
5. Push to remote if needed: `git push -u origin HEAD`
6. Create draft PR using gh CLI:
6. Push to remote if needed: `git push -u origin HEAD`
7. Create draft PR using gh CLI:
```bash
gh pr create --draft --title "<type>: <description>" --body "$(cat <<'EOF'
## Summary
@@ -85,7 +86,7 @@ Generated with [Claude Code](https://claude.com/claude-code)
EOF
)"
```
7. Return the PR URL to the user
8. Return the PR URL to the user
## EE Companion PR (when `*_ee.rs` files were modified)

23
.github/codex/pr-review.prompt.md vendored Normal file
View File

@@ -0,0 +1,23 @@
You are reviewing a GitHub pull request for this repository.
Review policy:
- Read `CLAUDE.md` before reviewing code.
- Only report issues you are confident are real and introduced by this pull request.
- Focus on bugs, security problems, and clear `CLAUDE.md` violations.
- Do not report style nits, speculative concerns, pre-existing issues, or problems that a normal linter/typechecker would obviously catch.
- Keep the review high signal. If there is no clear issue, return no findings.
Repository context:
- Read `./.github/codex/pr-review-context.md` for the PR metadata and the exact diff commands to use.
- Review only the changes introduced by this PR.
- Read additional files only when the diff is not enough to validate a finding.
- Do not modify any files.
Output requirements:
- Return a GitHub PR comment in markdown, not JSON.
- Start with `## Codex Review`.
- Give a short overall summary first.
- If you found high-signal issues, list them in a short numbered list with file paths and line numbers when you know them confidently.
- If you found no high-signal issues, say that explicitly.
- End with a `### Reproduction instructions` section containing a short descriptive paragraph for a tester explaining how to navigate the app to observe the change. Do not make it a numbered list. If the diff is not enough to infer this safely, say that plainly.
- Prefer at most 10 findings.

View File

@@ -290,6 +290,49 @@ jobs:
path: |
*.json
benchmark_wac:
runs-on: ubicloud-standard-8
services:
postgres:
image: postgres
env:
POSTGRES_DB: windmill
POSTGRES_PASSWORD: changeme
POSTGRES_INITDB_ARGS: "-c shared_buffers=2GB -c work_mem=32MB -c effective_cache_size=4GB"
options: >-
--health-cmd pg_isready --health-interval 10s --health-timeout 5s
--health-retries 5
--shm-size=2g
windmill:
image: ghcr.io/windmill-labs/windmill-ee:main
env:
DATABASE_URL: postgres://postgres:changeme@postgres:5432/windmill
LICENSE_KEY: ${{ secrets.WM_LICENSE_KEY_CI }}
WORKER_GROUP: main
WORKER_TAGS: deno,bun,go,python3,bash,dependency,flow,nativets
options: >-
--pull always --health-interval 10s --health-timeout 5s
--health-retries 5 --health-cmd "curl
http://localhost:8000/api/version"
ports:
- 8000:8000
steps:
- uses: denoland/setup-deno@v2
with:
deno-version: v2.x
- name: benchmark
timeout-minutes: 30
run: deno run -A -r
https://raw.githubusercontent.com/windmill-labs/windmill/${GITHUB_REF##ref/head/}/benchmarks/benchmark_suite.ts
-c
https://raw.githubusercontent.com/windmill-labs/windmill/${GITHUB_REF##ref/head/}/benchmarks/suite_wac.json
- name: Save benchmark results
uses: actions/upload-artifact@v4
with:
name: benchmark_wac
path: |
*.json
benchmark_graphs:
runs-on: ubicloud
needs:
@@ -297,6 +340,7 @@ jobs:
- benchmark_dedicated
- benchmark_4workers
- benchmark_8workers
- benchmark_wac
steps:
- uses: denoland/setup-deno@v2
with:

View File

@@ -1,6 +1,7 @@
name: CLI Tests
on:
workflow_dispatch:
push:
branches: [main]
paths:

145
.github/workflows/codex-pr-review.yml vendored Normal file
View File

@@ -0,0 +1,145 @@
name: Codex Auto Review
on:
pull_request:
types: [ready_for_review, opened]
concurrency:
group: codex-review-${{ github.event.pull_request.number }}
cancel-in-progress: true
jobs:
codex-review:
runs-on: ubicloud-standard-2
timeout-minutes: 30
if: github.event.pull_request.draft == false && github.event.pull_request.head.repo.fork == false
permissions:
contents: read
issues: write
steps:
- name: Check Codex configuration
id: codex_config
env:
CODEX_AUTH_JSON: ${{ secrets.CODEX_AUTH_JSON }}
run: |
if [ -n "$CODEX_AUTH_JSON" ]; then
echo "enabled=true" >> "$GITHUB_OUTPUT"
else
echo "enabled=false" >> "$GITHUB_OUTPUT"
echo "CODEX_AUTH_JSON is not configured; skipping Codex review."
fi
- name: Checkout repository
if: steps.codex_config.outputs.enabled == 'true'
uses: actions/checkout@v5
with:
ref: refs/pull/${{ github.event.pull_request.number }}/merge
fetch-depth: 1
- name: Set up Node.js
if: steps.codex_config.outputs.enabled == 'true'
uses: actions/setup-node@v4
with:
node-version: 22
- name: Install Codex CLI
if: steps.codex_config.outputs.enabled == 'true'
run: npm install --global @openai/codex@0.117.0
- name: Configure file-backed Codex auth
if: steps.codex_config.outputs.enabled == 'true'
env:
CODEX_AUTH_JSON: ${{ secrets.CODEX_AUTH_JSON }}
run: |
CODEX_HOME="$HOME/.codex"
echo "CODEX_HOME=$CODEX_HOME" >> "$GITHUB_ENV"
mkdir -p "$CODEX_HOME"
chmod 700 "$CODEX_HOME"
cat > "$CODEX_HOME/config.toml" <<'EOF'
cli_auth_credentials_store = "file"
EOF
printf '%s' "$CODEX_AUTH_JSON" > "$CODEX_HOME/auth.json"
chmod 600 "$CODEX_HOME/auth.json"
node -e 'JSON.parse(require("fs").readFileSync(process.argv[1], "utf8"))' "$CODEX_HOME/auth.json"
- name: Pre-fetch base and head refs for the PR
if: steps.codex_config.outputs.enabled == 'true'
env:
PR_BASE_REF: ${{ github.event.pull_request.base.ref }}
PR_NUMBER: ${{ github.event.pull_request.number }}
run: |
git fetch --no-tags origin \
"$PR_BASE_REF" \
"+refs/pull/$PR_NUMBER/head"
- name: Write Codex review context
if: steps.codex_config.outputs.enabled == 'true'
env:
PR_REPOSITORY: ${{ github.repository }}
PR_NUMBER: ${{ github.event.pull_request.number }}
PR_BASE_SHA: ${{ github.event.pull_request.base.sha }}
PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
PR_TITLE: ${{ github.event.pull_request.title }}
PR_BODY: ${{ github.event.pull_request.body || '' }}
run: |
mkdir -p .github/codex
node <<'NODE'
const fs = require('fs');
const lines = [
`Repository: ${process.env.PR_REPOSITORY}`,
`PR number: ${process.env.PR_NUMBER}`,
`Base SHA: ${process.env.PR_BASE_SHA}`,
`Head SHA: ${process.env.PR_HEAD_SHA}`,
'',
'PR title:',
process.env.PR_TITLE || '(empty)',
'',
'PR body:',
process.env.PR_BODY || '(empty)',
'',
'Changed commits command:',
`git log --oneline ${process.env.PR_BASE_SHA}...${process.env.PR_HEAD_SHA}`,
'',
'Changed files command:',
`git diff --stat ${process.env.PR_BASE_SHA}...${process.env.PR_HEAD_SHA}`,
'',
'Full review diff command:',
`git diff --unified=0 ${process.env.PR_BASE_SHA}...${process.env.PR_HEAD_SHA}`
];
fs.writeFileSync('.github/codex/pr-review-context.md', `${lines.join('\n')}\n`);
NODE
- name: Run Codex review
if: steps.codex_config.outputs.enabled == 'true'
run: |
codex exec \
-C "$GITHUB_WORKSPACE" \
-m gpt-5.4 \
-c 'model_reasoning_effort="xhigh"' \
-s read-only \
-o codex-final-message.md \
- < .github/codex/pr-review.prompt.md
- name: Post Codex review comment
if: steps.codex_config.outputs.enabled == 'true'
uses: actions/github-script@v7
with:
github-token: ${{ github.token }}
script: |
const fs = require('fs');
const path = `${process.env.GITHUB_WORKSPACE}/codex-final-message.md`;
if (!fs.existsSync(path)) {
core.info('Codex did not produce a final message; skipping PR comment.');
return;
}
const body = fs.readFileSync(path, 'utf8').trim();
if (!body) {
core.info('Codex final message was empty; skipping PR comment.');
return;
}
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.payload.pull_request.number,
body,
});

View File

@@ -22,6 +22,15 @@ jobs:
with:
fetch-depth: 1
- name: Read review prompt
id: review-prompt
run: |
{
echo 'REVIEW_PROMPT<<EOF'
cat .claude/review-prompt.md
echo 'EOF'
} >> "$GITHUB_ENV"
- name: Automatic PR Review
uses: anthropics/claude-code-action@v1
with:
@@ -31,18 +40,7 @@ jobs:
REPO: ${{ github.repository }}
PR NUMBER: ${{ github.event.pull_request.number }}
Please review this pull request and provide comprehensive feedback.
Focus on:
- Code quality and best practices
- Potential bugs or issues
- Performance considerations
- Security implications
Provide detailed feedback using inline comments for specific issues.
Use top-level comments for general observations or praise.
At the end of your review, add complete instructions to reproduce the added changes through the app interface. These instructions will be given to a tester so he can verify the changes. It should be a short descriptive text (not a step by step or a list) on how to navigate the app (what page, what action, what input, etc) to see the changes.
${{ env.REVIEW_PROMPT }}
claude_args: |
--allowedTools "mcp__github_inline_comment__create_inline_comment,Bash(gh pr comment:*),Bash(gh pr diff:*),Bash(gh pr view:*)"
--model opus

View File

@@ -43,7 +43,7 @@ profiles:
- Pane 0: this pane (claude agent)
- Pane 1: backend (cargo watch -x run)
- Pane 2: frontend (npm run dev)
To check logs, use: \`tmux capture-pane -t .1 -p -S -50\` (backend) or \`tmux capture-pane -t .2 -p -S -50\` (frontend).
To check logs, use: \`tmux capture-pane -t $(tmux display-message -t "$TMUX_PANE" -p '#{session_name}:#{window_name}').1 -p -S -50\` (backend) or \`tmux capture-pane -t $(tmux display-message -t "$TMUX_PANE" -p '#{session_name}:#{window_name}').2 -p -S -50\` (frontend).
For this window specifically, backend is running on: ${BACKEND_PORT} and frontend is running on: ${FRONTEND_PORT}.
To connect to the database, use this connection string: ${DATABASE_URL}
Because we are running backend with cargo watch, to verify your changes, just check the logs in the backend pane. No need for cargo check.
@@ -72,7 +72,7 @@ profiles:
Pane layout (current window):
- Pane 0: this pane (claude agent)
- Pane 1: frontend (npm run dev)
To check logs, use: \`tmux capture-pane -t .1 -p -S -50\` (frontend).
To check logs, use: \`tmux capture-pane -t $(tmux display-message -t "$TMUX_PANE" -p '#{session_name}:#{window_name}').1 -p -S -50\` (frontend).
On this window specifically, frontend is running on: ${FRONTEND_PORT}.
To connect to the database, use this connection string: ${DATABASE_URL}
Because we are running frontend with npm run dev, to verify your changes, just check the logs in the frontend pane. No need for npm run build.

View File

@@ -1,5 +1,181 @@
# Changelog
## [1.672.0](https://github.com/windmill-labs/windmill/compare/v1.671.0...v1.672.0) (2026-04-01)
### Features
* add R language support ([#8263](https://github.com/windmill-labs/windmill/issues/8263)) ([a46aa64](https://github.com/windmill-labs/windmill/commit/a46aa641f9d72809c52a0eb11a877a0f2d587c32))
### Bug Fixes
* approval page freeze, stale state, and missing approval link ([#8653](https://github.com/windmill-labs/windmill/issues/8653)) ([7069202](https://github.com/windmill-labs/windmill/commit/70692021909443b86ed61fa621fe49f28742fb54))
## [1.671.0](https://github.com/windmill-labs/windmill/compare/v1.670.0...v1.671.0) (2026-03-31)
### Features
* add configurable preview job tag override in default tags settings ([#8649](https://github.com/windmill-labs/windmill/issues/8649)) ([da8886b](https://github.com/windmill-labs/windmill/commit/da8886be8575dd925b6d24c55ab379bc6984c5f8))
* improve CLI flow log streaming and job inspection ([#8644](https://github.com/windmill-labs/windmill/issues/8644)) ([6c3c971](https://github.com/windmill-labs/windmill/commit/6c3c971af5aa1362632ee0deeddf91b8bc47c853))
* support hub flows in raw app runnables ([#8627](https://github.com/windmill-labs/windmill/issues/8627)) ([040a199](https://github.com/windmill-labs/windmill/commit/040a199685cea5c99c944bacb5584a381d6ec829))
### Bug Fixes
* return default_args/enums in approval info and fix subflow resume buttons ([#8648](https://github.com/windmill-labs/windmill/issues/8648)) ([852c59e](https://github.com/windmill-labs/windmill/commit/852c59efbb04510e5e6f99919707effcf6769a2f))
## [1.670.0](https://github.com/windmill-labs/windmill/compare/v1.669.1...v1.670.0) (2026-03-31)
### Features
* add OR logic support to kafka/websocket trigger filters ([#8580](https://github.com/windmill-labs/windmill/issues/8580)) ([3876902](https://github.com/windmill-labs/windmill/commit/3876902a7be798fd5ef208bc5756b28fb55e569e))
* expose getJob and getJobLogs as MCP tools ([#8632](https://github.com/windmill-labs/windmill/issues/8632)) ([cd8edcd](https://github.com/windmill-labs/windmill/commit/cd8edcd94f2bf44c3e771000cb0bbad08accc0e7))
* support multiline secrets in resource password fields ([#8637](https://github.com/windmill-labs/windmill/issues/8637)) ([26050f9](https://github.com/windmill-labs/windmill/commit/26050f96c34f14826298760174a45f3559d3266c))
* support sensitive/secret fields for non-string types ([#8635](https://github.com/windmill-labs/windmill/issues/8635)) ([375fb66](https://github.com/windmill-labs/windmill/commit/375fb66abe2d1861b53dc2b36d2cf0e2eb82c3a8))
### Bug Fixes
* cap input history per_page to 100 on cloud ([#8624](https://github.com/windmill-labs/windmill/issues/8624)) ([8e973c8](https://github.com/windmill-labs/windmill/commit/8e973c892d768be2da2e6b4b7af9e40b62333052))
* compute highest workspace role across all instance groups ([#8633](https://github.com/windmill-labs/windmill/issues/8633)) ([92b9ac7](https://github.com/windmill-labs/windmill/commit/92b9ac72c5fc9a5085fcb2e9d835ccbb53bcd4b0))
* Ducklake UI Nits ([#8628](https://github.com/windmill-labs/windmill/issues/8628)) ([ef1757f](https://github.com/windmill-labs/windmill/commit/ef1757f5d747e513d201eb6fa48918dba8248abe))
* preserve flow notes/groups and field ordering in generate-metadata ([#8641](https://github.com/windmill-labs/windmill/issues/8641)) ([#8642](https://github.com/windmill-labs/windmill/issues/8642)) ([52a04d2](https://github.com/windmill-labs/windmill/commit/52a04d210f476f4598007f67770bc6520b045950))
* remove timeout on python client httpx to prevent ducklake query timeouts ([#8636](https://github.com/windmill-labs/windmill/issues/8636)) ([c5fccd2](https://github.com/windmill-labs/windmill/commit/c5fccd2f69ad8a6e46c514cf89b9aa21b380e6fe))
* resolve missing form schema for nested suspend steps in FlowNode sub-flows ([#8643](https://github.com/windmill-labs/windmill/issues/8643)) ([12ea7e7](https://github.com/windmill-labs/windmill/commit/12ea7e74237560a9dfc99b6bc1338e3343b57640))
* smarter secret masking based on secret length ([#8629](https://github.com/windmill-labs/windmill/issues/8629)) ([bfc2aef](https://github.com/windmill-labs/windmill/commit/bfc2aefdb8ab92b7284de7f9e485a5504502d944))
## [1.669.1](https://github.com/windmill-labs/windmill/compare/v1.669.0...v1.669.1) (2026-03-30)
### Bug Fixes
* avoid doubled /oauth2 path in Okta custom authorization server URLs ([#8620](https://github.com/windmill-labs/windmill/issues/8620)) ([4817913](https://github.com/windmill-labs/windmill/commit/4817913f0cab49980bfeb442089631d7953955ff))
* improve db health UI text and prevent label wrapping ([d532c1d](https://github.com/windmill-labs/windmill/commit/d532c1d470fcb0ef02ebc5342ad1cf22e58b1f4d))
## [1.669.0](https://github.com/windmill-labs/windmill/compare/v1.668.5...v1.669.0) (2026-03-30)
### Features
* WAC workflow diagram visualization via WASM ([#8604](https://github.com/windmill-labs/windmill/issues/8604)) ([abc6b12](https://github.com/windmill-labs/windmill/commit/abc6b12d6815edc4dda3ddf5f0572ecedcb670dd))
### Bug Fixes
* add path traversal check in service_logs get_log_file endpoint ([#8605](https://github.com/windmill-labs/windmill/issues/8605)) ([5f2d3e6](https://github.com/windmill-labs/windmill/commit/5f2d3e6812f01fe6194bcfd976970a6e3c4186cc))
* cast DuckDB IS_NULLABLE to string in metadata query ([#8607](https://github.com/windmill-labs/windmill/issues/8607)) ([f3012ee](https://github.com/windmill-labs/windmill/commit/f3012ee7ccc7a8947b5f6bd7c7df77984437f91e))
* enable S3 bundle cache for PHP previews without lock file ([#8608](https://github.com/windmill-labs/windmill/issues/8608)) ([ee62315](https://github.com/windmill-labs/windmill/commit/ee6231590ed91063f104e6d054b52e88b569986f))
* enforce workspace isolation on flow resume endpoint ([#8612](https://github.com/windmill-labs/windmill/issues/8612)) ([33032ed](https://github.com/windmill-labs/windmill/commit/33032ed297cf9ea867388d4ea2ece607c9d36dc7))
* handle DuckDB boolean types in ColumnDef deserializers ([#8610](https://github.com/windmill-labs/windmill/issues/8610)) ([22da5bd](https://github.com/windmill-labs/windmill/commit/22da5bd9ea1ca000cfab3eecf1e3fb0fc01200cb))
* use route_service instead of fallback_service for MCP router ([#8614](https://github.com/windmill-labs/windmill/issues/8614)) ([98934d5](https://github.com/windmill-labs/windmill/commit/98934d59c552325fcf88c016e31ae977970e8c9a))
## [1.668.5](https://github.com/windmill-labs/windmill/compare/v1.668.4...v1.668.5) (2026-03-29)
### Bug Fixes
* add per-IP and per-account brute force protection on login endpoint ([#8601](https://github.com/windmill-labs/windmill/issues/8601)) ([06bbe7b](https://github.com/windmill-labs/windmill/commit/06bbe7b94bfb846bd73aaf6abdc83e4c14e70adc))
* add timestamp validation to webhook signature verification ([#8596](https://github.com/windmill-labs/windmill/issues/8596)) ([74fba2a](https://github.com/windmill-labs/windmill/commit/74fba2abf3dc68b682777c01da360258786fded8))
* disable workspace webhook events when CLOUD_HOSTED ([#8598](https://github.com/windmill-labs/windmill/issues/8598)) ([be7fbeb](https://github.com/windmill-labs/windmill/commit/be7fbeb8b1f31d15e33b0783b2a504d6a01e532e))
* harden login rate limiting with CLOUD_HOSTED gating and memory eviction ([#8602](https://github.com/windmill-labs/windmill/issues/8602)) ([754b88a](https://github.com/windmill-labs/windmill/commit/754b88a52c4e76421cb21c1eed87ad9d8385e9aa))
* prevent SSRF and local file read via git repository resource URLs ([#8600](https://github.com/windmill-labs/windmill/issues/8600)) ([845db72](https://github.com/windmill-labs/windmill/commit/845db72b7344fb87ac9c5e24697750549665c7bf))
* rename snippet param to avoid svelte compiler shadowing bug in asset usages drawer ([#8595](https://github.com/windmill-labs/windmill/issues/8595)) ([8c770a2](https://github.com/windmill-labs/windmill/commit/8c770a206a3b0704642c0bda2ab2aeb199d8af3f))
* require mcp: scope for MCP endpoints instead of blanket bypass ([#8597](https://github.com/windmill-labs/windmill/issues/8597)) ([f5fc9f8](https://github.com/windmill-labs/windmill/commit/f5fc9f8485d2ec3e20f8b451305195446b90e5a3))
* use constant-time comparison for API key and basic auth validation ([#8593](https://github.com/windmill-labs/windmill/issues/8593)) ([b4d1f2a](https://github.com/windmill-labs/windmill/commit/b4d1f2aac789306c2e35e123ac93e12c47c26f99))
* validate JSON before sql_builder bind to prevent injection via JSONB queries ([#8599](https://github.com/windmill-labs/windmill/issues/8599)) ([970e859](https://github.com/windmill-labs/windmill/commit/970e859a410b0144847a1a30d7059955effdd402))
## [1.668.4](https://github.com/windmill-labs/windmill/compare/v1.668.3...v1.668.4) (2026-03-29)
### Bug Fixes
* update git sync version to latest cli ([0549f68](https://github.com/windmill-labs/windmill/commit/0549f682fe14f4d4b2f67941362ed2cc29d974a1))
## [1.668.3](https://github.com/windmill-labs/windmill/compare/v1.668.2...v1.668.3) (2026-03-28)
### Bug Fixes
* **cli:** phantom diffs, flow safety, trigger DX, lint watch, error clarity ([#8588](https://github.com/windmill-labs/windmill/issues/8588)) ([c6ce319](https://github.com/windmill-labs/windmill/commit/c6ce3197a72ceeffd702cf2263b1074ecbf1ca33))
## [1.668.2](https://github.com/windmill-labs/windmill/compare/v1.668.1...v1.668.2) (2026-03-28)
### Bug Fixes
* **cli:** app push crash, lint path, push --message, run validation, history timestamps ([#8585](https://github.com/windmill-labs/windmill/issues/8585)) ([f40cdaf](https://github.com/windmill-labs/windmill/commit/f40cdaf43453d2643800ed730d6abe6873bbe8e7))
## [1.668.1](https://github.com/windmill-labs/windmill/compare/v1.668.0...v1.668.1) (2026-03-28)
### Bug Fixes
* **cli:** fix 13 CLI bugs — exit codes, sync tar fallback, variable encryption, JSON output ([#8582](https://github.com/windmill-labs/windmill/issues/8582)) ([38acaa3](https://github.com/windmill-labs/windmill/commit/38acaa3653728bf9e0ae6f746edf433703b4ab63))
## [1.668.0](https://github.com/windmill-labs/windmill/compare/v1.667.0...v1.668.0) (2026-03-28)
### Features
* add DB health diagnostic dashboard for superadmins ([#8574](https://github.com/windmill-labs/windmill/issues/8574)) ([9ceab73](https://github.com/windmill-labs/windmill/commit/9ceab730d7def09c2b46527f8a586789d14f2ce0))
* **cli:** add job, group, audit, token commands and schedule enable/disable ([#8581](https://github.com/windmill-labs/windmill/issues/8581)) ([d29cb23](https://github.com/windmill-labs/windmill/commit/d29cb234dbff07473b911e5e75e362def8a47650))
* IAM RDS auth for PostgreSQL worker resources ([#8573](https://github.com/windmill-labs/windmill/issues/8573)) ([56253c0](https://github.com/windmill-labs/windmill/commit/56253c04cb679c58d00750da699a6cb62ed52aca))
### Bug Fixes
* add Authority Key Identifier to MITM proxy leaf certs ([#8576](https://github.com/windmill-labs/windmill/issues/8576)) ([ce2e6c8](https://github.com/windmill-labs/windmill/commit/ce2e6c8c015110d0385e6afecdc8313aabca1364))
* Improve CLI developer experience: error handling, sync workflow, JSON output, workspace forks ([#8578](https://github.com/windmill-labs/windmill/issues/8578)) ([501a4ff](https://github.com/windmill-labs/windmill/commit/501a4ff2a94510145952686d24ccc639781beefe))
* trigger capture filter and focus issues ([#8579](https://github.com/windmill-labs/windmill/issues/8579)) ([820f28f](https://github.com/windmill-labs/windmill/commit/820f28f8799f8dad5cfab94b51ac9921d664f04a))
## [1.667.0](https://github.com/windmill-labs/windmill/compare/v1.666.0...v1.667.0) (2026-03-27)
### Features
* add schedule support to CLI branch-specific items ([#8570](https://github.com/windmill-labs/windmill/issues/8570)) ([b592996](https://github.com/windmill-labs/windmill/commit/b592996eee98ddb664f1b007b95a2096d5d4e3a6))
* add workspace-level service accounts ([#8560](https://github.com/windmill-labs/windmill/issues/8560)) ([3959fe8](https://github.com/windmill-labs/windmill/commit/3959fe82974f5f0383e94fd83a5d78fe4212d56a))
* **cli:** generate commented wmill.yaml and add config reference command ([#8546](https://github.com/windmill-labs/windmill/issues/8546)) ([d06b426](https://github.com/windmill-labs/windmill/commit/d06b42613f73c4a7b31c990be22b0c97efab2666))
* DB-coordinated graceful restart staggering for settings changes ([#8555](https://github.com/windmill-labs/windmill/issues/8555)) ([2f32675](https://github.com/windmill-labs/windmill/commit/2f326758013dd1f1e6ae732e5784a32f1fb6e4bd))
* improve-replay-ui ([#8250](https://github.com/windmill-labs/windmill/issues/8250)) ([c0aafee](https://github.com/windmill-labs/windmill/commit/c0aafee9a9923d5dc2fa3b99da4378e923933a06))
* support multiple folder selection in MCP scope selector ([#8557](https://github.com/windmill-labs/windmill/issues/8557)) ([ad19ac9](https://github.com/windmill-labs/windmill/commit/ad19ac9b37b04591c921f93f180bdda961af6cef))
### Bug Fixes
* **cli:** preserve inline script files during flow generate-locks ([#8561](https://github.com/windmill-labs/windmill/issues/8561)) ([a8b651d](https://github.com/windmill-labs/windmill/commit/a8b651da9ff86766119e14c0b61652be8a7b453a))
* emit 0 for OTEL queue metrics when tag queue is empty ([#8559](https://github.com/windmill-labs/windmill/issues/8559)) ([79cc4a9](https://github.com/windmill-labs/windmill/commit/79cc4a92d88486c999799826bd0c9663767103f5))
* handle inline script deletion in sync push + flow new nonDottedPaths ([#8553](https://github.com/windmill-labs/windmill/issues/8553)) ([943fe9c](https://github.com/windmill-labs/windmill/commit/943fe9c6cc9b046e24007e45b5c37afc4804256a))
* include importer_kind in dependency debounce key to prevent cross-kind collisions ([#8567](https://github.com/windmill-labs/windmill/issues/8567)) ([bc7007b](https://github.com/windmill-labs/windmill/commit/bc7007bb4265e1f1375c1f0678b74325882a4e92))
* multi-script dedicated workers race on shared job_dir ([#8551](https://github.com/windmill-labs/windmill/issues/8551)) ([#8569](https://github.com/windmill-labs/windmill/issues/8569)) ([63a3573](https://github.com/windmill-labs/windmill/commit/63a3573951d1f724cc63728ed973d039a5468072))
* preserve notes on nodes inside collapsed groups ([#8552](https://github.com/windmill-labs/windmill/issues/8552)) ([0fb1153](https://github.com/windmill-labs/windmill/commit/0fb115304afc49812420e9ce24e5048502621059))
* sanitize flow step summaries for filesystem-safe names ([#8554](https://github.com/windmill-labs/windmill/issues/8554)) ([e15bfbf](https://github.com/windmill-labs/windmill/commit/e15bfbf91ee1517432a6861ebb48e129485006aa))
* use admin db pool in get_copilot_settings_state ([#8564](https://github.com/windmill-labs/windmill/issues/8564)) ([70f3ee5](https://github.com/windmill-labs/windmill/commit/70f3ee5ed4470e9993be822874f2b38e83a96611))
### Performance Improvements
* enable bun bundle caching for WAC v2 scripts ([#8556](https://github.com/windmill-labs/windmill/issues/8556)) ([ab868e9](https://github.com/windmill-labs/windmill/commit/ab868e9ebceadaa55e54770d9d59dc5524da13ff))
## [1.666.0](https://github.com/windmill-labs/windmill/compare/v1.665.0...v1.666.0) (2026-03-26)
### Features
* add PDF input support to AI agent ([#8525](https://github.com/windmill-labs/windmill/issues/8525)) ([e44504c](https://github.com/windmill-labs/windmill/commit/e44504c6e93e7a4ee94ced03ab626b79a4fd0754))
### Bug Fixes
* add relative imports to the dependency list in deploymentUI ([#8548](https://github.com/windmill-labs/windmill/issues/8548)) ([d760ea5](https://github.com/windmill-labs/windmill/commit/d760ea5eaf4dc33007f1fd3e5e07b86925a0aa11))
* filter null entries in FileUpload initialValue to prevent s3 access error ([#8544](https://github.com/windmill-labs/windmill/issues/8544)) ([1a73012](https://github.com/windmill-labs/windmill/commit/1a73012e0737a6ebea8307013dc0f79982269d91))
* pass pre-bound TcpListener to run_server to fix Windows CI test race ([#8542](https://github.com/windmill-labs/windmill/issues/8542)) ([d7f4b95](https://github.com/windmill-labs/windmill/commit/d7f4b950ce6e966ed1b410e03d48fe96bc036e73))
* resolve parent_hash race condition in sync push with auto_parent ([#8545](https://github.com/windmill-labs/windmill/issues/8545)) ([71549c3](https://github.com/windmill-labs/windmill/commit/71549c3db053bcc209c7065ac8cd42f1e8047cc3))
* upload_s3_file not working in VS Code extension ([#8547](https://github.com/windmill-labs/windmill/issues/8547)) ([1fa4d91](https://github.com/windmill-labs/windmill/commit/1fa4d919b30ac9eff2d1789fba2695450ba115e7))
## [1.665.0](https://github.com/windmill-labs/windmill/compare/v1.664.0...v1.665.0) (2026-03-26)

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO email_to_igroup (email, igroup) VALUES ('alice@example.com', 'admins') ON CONFLICT DO NOTHING",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "077467cd813d5af161cb1cc232724f26984822d4c28ba36c0a9331273b10edc0"
}

View File

@@ -0,0 +1,20 @@
{
"db_name": "PostgreSQL",
"query": "SELECT setting::bigint as \"max!\" FROM pg_settings WHERE name = 'max_connections'",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "max!",
"type_info": "Int8"
}
],
"parameters": {
"Left": []
},
"nullable": [
null
]
},
"hash": "07770a002a49428c4f956cfc7262d6b6792ae5b97ed90b0ee07d17480b2dffe2"
}

View File

@@ -0,0 +1,20 @@
{
"db_name": "PostgreSQL",
"query": "SELECT EXISTS(SELECT 1 FROM pg_extension WHERE extname = 'pg_stat_statements') as \"exists!\"",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "exists!",
"type_info": "Bool"
}
],
"parameters": {
"Left": []
},
"nullable": [
null
]
},
"hash": "143acebe5d815c5d828013ebe46274f891f953c75f821499552ab7794f75063d"
}

View File

@@ -0,0 +1,20 @@
{
"db_name": "PostgreSQL",
"query": "SELECT igroup FROM email_to_igroup WHERE email = 'alice@example.com'",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "igroup",
"type_info": "Varchar"
}
],
"parameters": {
"Left": []
},
"nullable": [
false
]
},
"hash": "1721f8b52ea265c0537fd7c742deddf0afbe5cf0d81b15e487c411ae169d3a89"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT usr.*, password.super_admin, password.name FROM usr LEFT JOIN password ON usr.email = password.email Where usr.username = $1 AND workspace_id = $2\n ",
"query": "SELECT usr.*, COALESCE(password.super_admin, false) as \"super_admin!\", password.name FROM usr LEFT JOIN password ON usr.email = password.email Where usr.username = $1 AND workspace_id = $2\n ",
"describe": {
"columns": [
{
@@ -50,11 +50,16 @@
},
{
"ordinal": 9,
"name": "super_admin",
"name": "is_service_account",
"type_info": "Bool"
},
{
"ordinal": 10,
"name": "super_admin!",
"type_info": "Bool"
},
{
"ordinal": 11,
"name": "name",
"type_info": "Varchar"
}
@@ -76,8 +81,9 @@
true,
true,
false,
null,
true
]
},
"hash": "6aabe704395c9be30c86d15a5d22f3509b4fcea56227b019588837132b64d58b"
"hash": "1cf8597b9d37ec5a924aff8cbc0a05768ed9a679ba908ab16497a9bd55578ba1"
}

View File

@@ -0,0 +1,32 @@
{
"db_name": "PostgreSQL",
"query": "SELECT\n schemaname || '.' || relname as \"table_name!\",\n pg_total_relation_size(relid) as \"total_size_bytes!\",\n pg_size_pretty(pg_total_relation_size(relid)) as \"total_size_pretty!\"\n FROM pg_catalog.pg_statio_user_tables\n ORDER BY pg_total_relation_size(relid) DESC\n LIMIT 15",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "table_name!",
"type_info": "Text"
},
{
"ordinal": 1,
"name": "total_size_bytes!",
"type_info": "Int8"
},
{
"ordinal": 2,
"name": "total_size_pretty!",
"type_info": "Text"
}
],
"parameters": {
"Left": []
},
"nullable": [
null,
null,
null
]
},
"hash": "1dd73eff0e89b84c0316af2760a136afdd19dc34f9f31c4f9de6b0f74bc386a6"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO usr (workspace_id, username, email, is_admin, operator, added_via)\n VALUES ($1, 'alice', 'alice@example.com', false, true, $2)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Jsonb"
]
},
"nullable": []
},
"hash": "250a4e3f1a1f95296f7075bf8780e9c7407e89c8f7636484895e99f5a5e71297"
}

View File

@@ -0,0 +1,20 @@
{
"db_name": "PostgreSQL",
"query": "SELECT value FROM global_settings WHERE name = 'retention_period_secs'",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "value",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": []
},
"nullable": [
false
]
},
"hash": "26e62b4509e44a7548957ad4ef217fd46bc03d5dca19344cd3bf7b131fa40ed2"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO usr (workspace_id, username, email, is_admin, operator, added_via)\n VALUES ($1, 'alice', 'alice@example.com', true, false, $2)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Jsonb"
]
},
"nullable": []
},
"hash": "2ba03e555d2e09dbd0e2ae5ddfd9a268a675bdb23615c78904cebe7f1e31f400"
}

View File

@@ -0,0 +1,32 @@
{
"db_name": "PostgreSQL",
"query": "SELECT\n ws.workspace_id as \"workspace_id!\",\n dt.key as \"name!\",\n dt.value->>'table_name' as \"table_name\"\n FROM workspace_settings ws,\n jsonb_each(ws.datatable) dt\n WHERE dt.value->>'resource_type' = 'instance'\n AND dt.value->>'table_name' IS NOT NULL",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "workspace_id!",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "name!",
"type_info": "Text"
},
{
"ordinal": 2,
"name": "table_name",
"type_info": "Text"
}
],
"parameters": {
"Left": []
},
"nullable": [
false,
null,
null
]
},
"hash": "2d4ccf3ee19a70cbb5bd034c74703bbb30f217cd3673821e11bae3bf9f925720"
}

View File

@@ -0,0 +1,32 @@
{
"db_name": "PostgreSQL",
"query": "SELECT\n COUNT(*) as \"total!\",\n COUNT(*) FILTER (WHERE state = 'active') as \"active!\",\n COUNT(*) FILTER (WHERE state = 'idle') as \"idle!\"\n FROM pg_stat_activity\n WHERE backend_type = 'client backend'",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "total!",
"type_info": "Int8"
},
{
"ordinal": 1,
"name": "active!",
"type_info": "Int8"
},
{
"ordinal": 2,
"name": "idle!",
"type_info": "Int8"
}
],
"parameters": {
"Left": []
},
"nullable": [
null,
null,
null
]
},
"hash": "2d95191e899d60385b32f36f2e38137e4173a34c54344ee522745640d48b8813"
}

View File

@@ -0,0 +1,44 @@
{
"db_name": "PostgreSQL",
"query": "SELECT\n table_name as \"table_name!\",\n SUM(live_tuples)::bigint as \"live_tuples!\",\n SUM(dead_tuples)::bigint as \"dead_tuples!\",\n MAX(last_autovacuum) as \"last_autovacuum\",\n MAX(last_autoanalyze) as \"last_autoanalyze\"\n FROM (\n SELECT\n CASE\n WHEN i.inhparent IS NOT NULL THEN schemaname || '.' || p.relname\n ELSE schemaname || '.' || s.relname\n END as table_name,\n COALESCE(n_live_tup, 0) as live_tuples,\n COALESCE(n_dead_tup, 0) as dead_tuples,\n last_autovacuum,\n last_autoanalyze\n FROM pg_stat_user_tables s\n LEFT JOIN pg_class c ON c.relname = s.relname AND c.relnamespace = (\n SELECT oid FROM pg_namespace WHERE nspname = s.schemaname\n )\n LEFT JOIN pg_inherits i ON i.inhrelid = c.oid\n LEFT JOIN pg_class p ON p.oid = i.inhparent\n ) sub\n GROUP BY table_name\n ORDER BY SUM(dead_tuples) DESC",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "table_name!",
"type_info": "Text"
},
{
"ordinal": 1,
"name": "live_tuples!",
"type_info": "Int8"
},
{
"ordinal": 2,
"name": "dead_tuples!",
"type_info": "Int8"
},
{
"ordinal": 3,
"name": "last_autovacuum",
"type_info": "Timestamptz"
},
{
"ordinal": 4,
"name": "last_autoanalyze",
"type_info": "Timestamptz"
}
],
"parameters": {
"Left": []
},
"nullable": [
null,
null,
null,
null,
null
]
},
"hash": "30930bfb0513f1a70194a900011b2e890bc4146bb0419210cd76743cacda8bfa"
}

View File

@@ -0,0 +1,38 @@
{
"db_name": "PostgreSQL",
"query": "WITH all_audit AS (SELECT username, operation, timestamp FROM audit_partitioned UNION ALL SELECT username, operation, timestamp FROM audit),\n active_users as (SELECT distinct username as email FROM all_audit WHERE timestamp > NOW() - INTERVAL '1 month' AND (operation = 'users.login' OR operation = 'oauth.login' OR operation = 'users.token.refresh')),\n active_authors as (SELECT distinct email FROM usr WHERE usr.operator IS false AND email IN (SELECT email FROM active_users)),\n active_authors_agg as (SELECT array_agg(email) as authors FROM active_authors),\n active_ops_agg as (SELECT array_agg(email) as operators from active_users WHERE email NOT IN (SELECT email FROM active_authors))\n SELECT active_authors_agg.authors, active_ops_agg.operators, array_length(active_authors_agg.authors, 1) as author_count, array_length(active_ops_agg.operators, 1) as operator_count FROM active_authors_agg, active_ops_agg",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "authors",
"type_info": "VarcharArray"
},
{
"ordinal": 1,
"name": "operators",
"type_info": "VarcharArray"
},
{
"ordinal": 2,
"name": "author_count",
"type_info": "Int4"
},
{
"ordinal": 3,
"name": "operator_count",
"type_info": "Int4"
}
],
"parameters": {
"Left": []
},
"nullable": [
null,
null,
null,
null
]
},
"hash": "359cd29f531d263a8cf7205e0869229a610767087f01e0154be8da0620fa114b"
}

View File

@@ -0,0 +1,26 @@
{
"db_name": "PostgreSQL",
"query": "SELECT pg_database_size(current_database()) as size_bytes, pg_size_pretty(pg_database_size(current_database())) as size_pretty",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "size_bytes",
"type_info": "Int8"
},
{
"ordinal": 1,
"name": "size_pretty",
"type_info": "Text"
}
],
"parameters": {
"Left": []
},
"nullable": [
null,
null
]
},
"hash": "384f5e9b2ab8e430141e28ea58854cbcfbcf96fd2adbf0513ce942cfe9bceaf0"
}

View File

@@ -0,0 +1,34 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT workspace_id,\n auto_invite->'instance_groups_roles' as instance_groups_roles,\n auto_invite->'instance_groups' as instance_groups_json\n FROM workspace_settings\n WHERE auto_invite->'instance_groups' ? $1\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "workspace_id",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "instance_groups_roles",
"type_info": "Jsonb"
},
{
"ordinal": 2,
"name": "instance_groups_json",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false,
null,
null
]
},
"hash": "3bd4f38a1629a69ddda622b6b436198b47c2fe1a507358d49f05031e7beedab6"
}

View File

@@ -21,7 +21,7 @@
{
"ordinal": 3,
"name": "item_path",
"type_info": "Varchar"
"type_info": "Text"
},
{
"ordinal": 4,

View File

@@ -21,7 +21,7 @@
{
"ordinal": 3,
"name": "item_path",
"type_info": "Varchar"
"type_info": "Text"
},
{
"ordinal": 4,

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT DISTINCT imported_path as \"imported_path!\"\n FROM dependency_map\n WHERE workspace_id = $1\n AND importer_path = $2\n AND imported_path NOT LIKE 'dependencies/%'\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "imported_path!",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false
]
},
"hash": "52d765c87cb8da0ca71fb53156820e383a998a54c95355bb85fe7e762a0d9765"
}

View File

@@ -0,0 +1,35 @@
{
"db_name": "PostgreSQL",
"query": "SELECT email, is_service_account, disabled FROM usr WHERE username = $1 AND workspace_id = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "email",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "is_service_account",
"type_info": "Bool"
},
{
"ordinal": 2,
"name": "disabled",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
false,
false
]
},
"hash": "544a02447bb2cbe8354a5c4ae93685848af38a3461257a9734c43cbd7bd905cb"
}

View File

@@ -15,7 +15,7 @@
]
},
"nullable": [
true
null
]
},
"hash": "5a219a2532517869578c4504ff3153c43903f929ae5d62fbba12610f89c36d55"

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT igroup FROM email_to_igroup WHERE email = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "igroup",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false
]
},
"hash": "5d160ba4958583f1ad42de846c544d8d8e81e1b54925a0c5f2cedc1817d99a1b"
}

View File

@@ -47,6 +47,11 @@
"ordinal": 8,
"name": "added_via",
"type_info": "Jsonb"
},
{
"ordinal": 9,
"name": "is_service_account",
"type_info": "Bool"
}
],
"parameters": {
@@ -63,7 +68,8 @@
false,
false,
true,
true
true,
false
]
},
"hash": "5d6adbe21b9f8dd984d1bfc750fb81763d8650c1316bb0b20816f1a5d61a678c"

View File

@@ -47,6 +47,11 @@
"ordinal": 8,
"name": "added_via",
"type_info": "Jsonb"
},
{
"ordinal": 9,
"name": "is_service_account",
"type_info": "Bool"
}
],
"parameters": {
@@ -64,7 +69,8 @@
false,
false,
true,
true
true,
false
]
},
"hash": "60b3a59805d463a61eed68072d1ea032b00fc9bd7a6db22f530f67eb9730fa3b"

View File

@@ -0,0 +1,32 @@
{
"db_name": "PostgreSQL",
"query": "SELECT is_admin, operator, added_via FROM usr WHERE workspace_id = 'ws-multi-group' AND email = 'alice@example.com'",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "is_admin",
"type_info": "Bool"
},
{
"ordinal": 1,
"name": "operator",
"type_info": "Bool"
},
{
"ordinal": 2,
"name": "added_via",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": []
},
"nullable": [
false,
false,
true
]
},
"hash": "62e8e443cf063fcb30799d9c8971c00d761d54811936deb87a0315ca9cdc9769"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT\n workspace_id,\n auto_invite->'instance_groups_roles' as instance_groups_roles\n FROM workspace_settings\n WHERE\n auto_invite->'instance_groups' IS NOT NULL\n AND auto_invite->'instance_groups' ? $1\n ",
"query": "\n SELECT\n workspace_id,\n auto_invite->'instance_groups_roles' as instance_groups_roles,\n auto_invite->'instance_groups' as instance_groups_json\n FROM workspace_settings\n WHERE\n auto_invite->'instance_groups' IS NOT NULL\n AND auto_invite->'instance_groups' ? $1\n ",
"describe": {
"columns": [
{
@@ -12,6 +12,11 @@
"ordinal": 1,
"name": "instance_groups_roles",
"type_info": "Jsonb"
},
{
"ordinal": 2,
"name": "instance_groups_json",
"type_info": "Jsonb"
}
],
"parameters": {
@@ -21,8 +26,9 @@
},
"nullable": [
false,
null,
null
]
},
"hash": "7e01ef5799168c0fc2779d42ce352827e2fda6711c0a1b104ca6435ddb14b47d"
"hash": "66e2f8468ba64f22b7a7caa18639d7c833ac2ec573bd89d878b5c8b1afc74d3a"
}

View File

@@ -0,0 +1,30 @@
{
"db_name": "PostgreSQL",
"query": "\n UPDATE kafka_trigger\n SET\n kafka_resource_path = $1,\n group_id = $2,\n topics = $3,\n filters = $4,\n filter_logic = $5,\n auto_offset_reset = $6,\n auto_commit = $7,\n script_path = $8,\n path = $9,\n is_flow = $10,\n edited_by = $11,\n permissioned_as = $12,\n edited_at = now(),\n server_id = NULL,\n error = NULL,\n error_handler_path = $15,\n error_handler_args = $16,\n retry = $17\n WHERE\n workspace_id = $13 AND path = $14\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"VarcharArray",
"JsonbArray",
"Varchar",
"Varchar",
"Bool",
"Varchar",
"Varchar",
"Bool",
"Varchar",
"Varchar",
"Text",
"Text",
"Varchar",
"Jsonb",
"Jsonb"
]
},
"nullable": []
},
"hash": "68c19cb0e18b94870bbe81f9aab92ba37da67cd2a56834c9d1378eab7551284d"
}

View File

@@ -0,0 +1,24 @@
{
"db_name": "PostgreSQL",
"query": "SELECT EXISTS(SELECT 1 FROM usr WHERE workspace_id = $1 AND (username = $2 OR email = $3))",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "exists",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": [
null
]
},
"hash": "68d1370fa02f4fe585684a91e898c4aed45e6b8f409bb33c2681f92265922040"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO websocket_trigger (\n workspace_id,\n path,\n url,\n script_path,\n is_flow,\n mode,\n filters,\n initial_messages,\n url_runnable_args,\n edited_by,\n can_return_message,\n can_return_error_result,\n permissioned_as,\n edited_at,\n error_handler_path,\n error_handler_args,\n retry\n ) VALUES (\n $1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, now(), $14, $15, $16\n )\n ",
"query": "\n INSERT INTO websocket_trigger (\n workspace_id,\n path,\n url,\n script_path,\n is_flow,\n mode,\n filters,\n filter_logic,\n initial_messages,\n url_runnable_args,\n edited_by,\n can_return_message,\n can_return_error_result,\n permissioned_as,\n edited_at,\n error_handler_path,\n error_handler_args,\n retry\n ) VALUES (\n $1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, now(), $15, $16, $17\n )\n ",
"describe": {
"columns": [],
"parameters": {
@@ -23,6 +23,7 @@
}
},
"JsonbArray",
"Varchar",
"JsonbArray",
"Jsonb",
"Varchar",
@@ -36,5 +37,5 @@
},
"nullable": []
},
"hash": "942c0abb55c910862fd45d3fa56a4eb6729f1a658101bda2d0b0fca96b3cfee5"
"hash": "6948eb5aabf82f2f4a08dd4410eb472080ecab3ed652912397245e5216ae0389"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO kafka_trigger (\n workspace_id,\n path,\n kafka_resource_path,\n group_id,\n topics,\n filters,\n auto_offset_reset,\n auto_commit,\n script_path,\n is_flow,\n mode,\n edited_by,\n permissioned_as,\n edited_at,\n error_handler_path,\n error_handler_args,\n retry\n ) VALUES (\n $1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, now(), $14, $15, $16\n )\n ",
"query": "\n INSERT INTO kafka_trigger (\n workspace_id,\n path,\n kafka_resource_path,\n group_id,\n topics,\n filters,\n filter_logic,\n auto_offset_reset,\n auto_commit,\n script_path,\n is_flow,\n mode,\n edited_by,\n permissioned_as,\n edited_at,\n error_handler_path,\n error_handler_args,\n retry\n ) VALUES (\n $1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, now(), $15, $16, $17\n )\n ",
"describe": {
"columns": [],
"parameters": {
@@ -12,6 +12,7 @@
"VarcharArray",
"JsonbArray",
"Varchar",
"Varchar",
"Bool",
"Varchar",
"Bool",
@@ -36,5 +37,5 @@
},
"nullable": []
},
"hash": "a0a545fda5f3ebea0113d5daaf13358c964d9fb0f41bf2a1c834305b4d2398f2"
"hash": "6a8f4ed9946bb2a3c5e90695c90b70aa2e83fcb5aa0c953febdd9bac2d95bbec"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT pg_advisory_xact_lock(hashtext($1 || '/' || $2))",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "pg_advisory_xact_lock",
"type_info": "Void"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
null
]
},
"hash": "6d070f476538aa6fcd6227fe5312561a7d098f2af5287e1e6c339e15080378be"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO usr (workspace_id, username, email, is_admin, operator)\n VALUES ('ws-multi-group', 'alice', 'alice@example.com', true, false)",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "6f941e4454f736b32eaef80cdfb9582d6e75af3dc159e7c5f12497d3957f1eef"
}

View File

@@ -0,0 +1,40 @@
{
"db_name": "PostgreSQL",
"query": "SELECT\n c.relname as \"table_name!\",\n pg_total_relation_size(c.oid) as \"size_bytes!\",\n pg_size_pretty(pg_total_relation_size(c.oid)) as \"size_pretty!\",\n COALESCE(c.reltuples, 0) as \"estimated_rows!\"\n FROM pg_class c\n JOIN pg_namespace n ON n.oid = c.relnamespace\n WHERE n.nspname = 'public' AND c.relname = ANY($1)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "table_name!",
"type_info": "Name"
},
{
"ordinal": 1,
"name": "size_bytes!",
"type_info": "Int8"
},
{
"ordinal": 2,
"name": "size_pretty!",
"type_info": "Text"
},
{
"ordinal": 3,
"name": "estimated_rows!",
"type_info": "Float4"
}
],
"parameters": {
"Left": [
"NameArray"
]
},
"nullable": [
false,
null,
null,
null
]
},
"hash": "7c5db0b3bd1dd1f766e1841ca620871a468033e05b6e0188ea4775b63fc66e84"
}

View File

@@ -0,0 +1,26 @@
{
"db_name": "PostgreSQL",
"query": "SELECT is_admin, operator FROM usr WHERE workspace_id = 'ws-multi-group' AND email = 'alice@example.com'",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "is_admin",
"type_info": "Bool"
},
{
"ordinal": 1,
"name": "operator",
"type_info": "Bool"
}
],
"parameters": {
"Left": []
},
"nullable": [
false,
false
]
},
"hash": "88a467f3c943b134a81ac69c3c6686d1ce1ff2f5aafc15ff1b63cfa86c09c4f0"
}

View File

@@ -13,7 +13,7 @@
"Left": [
"Varchar",
"Varchar",
"Varchar",
"Text",
"Jsonb",
"Varchar"
]

View File

@@ -21,7 +21,7 @@
{
"ordinal": 3,
"name": "item_path",
"type_info": "Varchar"
"type_info": "Text"
},
{
"ordinal": 4,

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT s.hash as hash, dm.deployment_msg as deployment_msg \n FROM script s LEFT JOIN deployment_metadata dm ON s.hash = dm.script_hash\n WHERE s.workspace_id = $1 AND s.path = $2\n ORDER by s.created_at DESC",
"query": "SELECT s.hash as hash, dm.deployment_msg as deployment_msg, s.created_at as created_at\n FROM script s LEFT JOIN deployment_metadata dm ON s.hash = dm.script_hash\n WHERE s.workspace_id = $1 AND s.path = $2\n ORDER by s.created_at DESC",
"describe": {
"columns": [
{
@@ -12,6 +12,11 @@
"ordinal": 1,
"name": "deployment_msg",
"type_info": "Text"
},
{
"ordinal": 2,
"name": "created_at",
"type_info": "Timestamptz"
}
],
"parameters": {
@@ -22,8 +27,9 @@
},
"nullable": [
false,
true
true,
false
]
},
"hash": "726e956cfcd3ac7c07abeecdf92cf0996efe7fa7b671ac2b3b000ead0ea307de"
"hash": "9a1483a81f5b086e0765d3d69483e29b09f66090e1f9d394564c16d921d2e66c"
}

View File

@@ -0,0 +1,20 @@
{
"db_name": "PostgreSQL",
"query": "SELECT COUNT(*) FROM usr WHERE is_service_account = true",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "count",
"type_info": "Int8"
}
],
"parameters": {
"Left": []
},
"nullable": [
null
]
},
"hash": "a37c2c4d5656d4b44433de84c454046f7586e36b7bd6a4679d70c359d4aacfcf"
}

View File

@@ -1,29 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n UPDATE kafka_trigger\n SET\n kafka_resource_path = $1,\n group_id = $2,\n topics = $3,\n filters = $4,\n auto_offset_reset = $5,\n auto_commit = $6,\n script_path = $7,\n path = $8,\n is_flow = $9,\n edited_by = $10,\n permissioned_as = $11,\n edited_at = now(),\n server_id = NULL,\n error = NULL,\n error_handler_path = $14,\n error_handler_args = $15,\n retry = $16\n WHERE\n workspace_id = $12 AND path = $13\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"VarcharArray",
"JsonbArray",
"Varchar",
"Bool",
"Varchar",
"Varchar",
"Bool",
"Varchar",
"Varchar",
"Text",
"Text",
"Varchar",
"Jsonb",
"Jsonb"
]
},
"nullable": []
},
"hash": "a37cfc632dd37cf37c06743239b5ebc784e5da5ee25d47af187a75220d8fded7"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO pip_resolution_cache (hash, lockfile, expiration) VALUES ($1, $2, now() + ('7 days')::interval) ON CONFLICT (hash) DO UPDATE SET lockfile = EXCLUDED.lockfile, expiration = EXCLUDED.expiration",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "a860dd9722f608184c4b1ef5e609b20cd61f9967a2012fc1c8fe352ee7596358"
}

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT AVG(pg_column_size(result))::bigint as \"avg_size\"\n FROM (\n SELECT result FROM v2_job_completed\n WHERE completed_at > now() - interval '30 days'\n AND result IS NOT NULL\n ORDER BY completed_at DESC\n LIMIT $1\n ) sub",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "avg_size",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Int8"
]
},
"nullable": [
null
]
},
"hash": "a9c3461ca3053f699c957f61780d1e889ad53dc5bf1669c24c0666c290656c00"
}

View File

@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO usr_to_group (workspace_id, usr, group_) VALUES ($1, $2, $3)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar"
]
},
"nullable": []
},
"hash": "add01e9e31d64e88b84c9505fe3de553031e581b1bb173413a9a3e3eb0817b43"
}

View File

@@ -0,0 +1,18 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE usr SET is_admin = $1, operator = $2, added_via = $3 WHERE workspace_id = $4 AND email = $5 AND added_via->>'source' = 'instance_group'",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Bool",
"Bool",
"Jsonb",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "b38bd869477a729279cac3ccd4825191fb49e17e5f7e7297c0c819f52b486f49"
}

View File

@@ -0,0 +1,26 @@
{
"db_name": "PostgreSQL",
"query": "SELECT MIN(completed_at) as oldest, COUNT(*) as total FROM v2_job_completed",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "oldest",
"type_info": "Timestamptz"
},
{
"ordinal": 1,
"name": "total",
"type_info": "Int8"
}
],
"parameters": {
"Left": []
},
"nullable": [
null,
null
]
},
"hash": "b760be4a0a80853073a061f7c9ebc2d411294d57b07d54d15d178db3c6ee2a30"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT s.hash as hash, dm.deployment_msg as deployment_msg \n FROM script s LEFT JOIN deployment_metadata dm ON s.hash = dm.script_hash\n WHERE s.workspace_id = $1 AND s.path = $2\n ORDER by s.created_at DESC LIMIT 1",
"query": "SELECT s.hash as hash, dm.deployment_msg as deployment_msg, s.created_at as created_at\n FROM script s LEFT JOIN deployment_metadata dm ON s.hash = dm.script_hash\n WHERE s.workspace_id = $1 AND s.path = $2\n ORDER by s.created_at DESC LIMIT 1",
"describe": {
"columns": [
{
@@ -12,6 +12,11 @@
"ordinal": 1,
"name": "deployment_msg",
"type_info": "Text"
},
{
"ordinal": 2,
"name": "created_at",
"type_info": "Timestamptz"
}
],
"parameters": {
@@ -22,8 +27,9 @@
},
"nullable": [
false,
true
true,
false
]
},
"hash": "cf2a6ad6471a40b6298775cda9300aeecdd75503bed59d80cd62091d1642d1ec"
"hash": "c73e98e5a937f44724a96ee1b74d31fa71a7be3b8ba3dec9f59f54a6c4030462"
}

View File

@@ -1,29 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n UPDATE\n websocket_trigger\n SET\n url = $1,\n script_path = $2,\n path = $3,\n is_flow = $4,\n filters = $5,\n initial_messages = $6,\n url_runnable_args = $7,\n edited_by = $8,\n permissioned_as = $9,\n can_return_message = $10,\n can_return_error_result = $11,\n edited_at = now(),\n server_id = NULL,\n error = NULL,\n error_handler_path = $14,\n error_handler_args = $15,\n retry = $16\n WHERE\n workspace_id = $12 AND path = $13\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar",
"Bool",
"JsonbArray",
"JsonbArray",
"Jsonb",
"Varchar",
"Varchar",
"Bool",
"Bool",
"Text",
"Text",
"Varchar",
"Jsonb",
"Jsonb"
]
},
"nullable": []
},
"hash": "c7aed7fe3b6774477d403bc3e7fcbce7cdbdd1feb553718cbde60bb8ccff4733"
}

View File

@@ -1,38 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "WITH active_users as (SELECT distinct username as email FROM audit WHERE timestamp > NOW() - INTERVAL '1 month' AND (operation = 'users.login' OR operation = 'oauth.login' OR operation = 'users.token.refresh')),\n active_authors as (SELECT distinct email FROM usr WHERE usr.operator IS false AND email IN (SELECT email FROM active_users)),\n active_authors_agg as (SELECT array_agg(email) as authors FROM active_authors),\n active_ops_agg as (SELECT array_agg(email) as operators from active_users WHERE email NOT IN (SELECT email FROM active_authors))\n SELECT active_authors_agg.authors, active_ops_agg.operators, array_length(active_authors_agg.authors, 1) as author_count, array_length(active_ops_agg.operators, 1) as operator_count FROM active_authors_agg, active_ops_agg",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "authors",
"type_info": "VarcharArray"
},
{
"ordinal": 1,
"name": "operators",
"type_info": "VarcharArray"
},
{
"ordinal": 2,
"name": "author_count",
"type_info": "Int4"
},
{
"ordinal": 3,
"name": "operator_count",
"type_info": "Int4"
}
],
"parameters": {
"Left": []
},
"nullable": [
null,
null,
null,
null
]
},
"hash": "cb3862634f18160207ee2621ddfca43f00456a27fda32583846497116f92f96c"
}

View File

@@ -0,0 +1,20 @@
{
"db_name": "PostgreSQL",
"query": "SELECT value FROM global_settings WHERE name = 'plain_emails_telemetry'",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "value",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": []
},
"nullable": [
false
]
},
"hash": "d43a4ff78e48580815fb912c98639a08d45596a9f11a2dcf5b1e0d135844ecda"
}

View File

@@ -0,0 +1,46 @@
{
"db_name": "PostgreSQL",
"query": "SELECT\n c.id as \"id!\",\n c.workspace_id as \"workspace_id!\",\n j.runnable_path as \"runnable_path\",\n pg_column_size(c.result) as \"result_size_bytes!\",\n c.completed_at as \"completed_at!\"\n FROM (\n SELECT id, workspace_id, result, completed_at\n FROM v2_job_completed\n WHERE completed_at > now() - interval '30 days'\n AND result IS NOT NULL\n ORDER BY completed_at DESC\n LIMIT $1\n ) c\n LEFT JOIN v2_job j ON j.id = c.id\n WHERE pg_column_size(c.result) > 1024\n ORDER BY pg_column_size(c.result) DESC\n LIMIT 10",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id!",
"type_info": "Uuid"
},
{
"ordinal": 1,
"name": "workspace_id!",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "runnable_path",
"type_info": "Varchar"
},
{
"ordinal": 3,
"name": "result_size_bytes!",
"type_info": "Int4"
},
{
"ordinal": 4,
"name": "completed_at!",
"type_info": "Timestamptz"
}
],
"parameters": {
"Left": [
"Int8"
]
},
"nullable": [
false,
false,
true,
null,
false
]
},
"hash": "dbc5924bca3aa0b32e296b73f8a967bed68332caf526216597f10ffa5fa951c7"
}

View File

@@ -0,0 +1,30 @@
{
"db_name": "PostgreSQL",
"query": "\n UPDATE\n websocket_trigger\n SET\n url = $1,\n script_path = $2,\n path = $3,\n is_flow = $4,\n filters = $5,\n filter_logic = $6,\n initial_messages = $7,\n url_runnable_args = $8,\n edited_by = $9,\n permissioned_as = $10,\n can_return_message = $11,\n can_return_error_result = $12,\n edited_at = now(),\n server_id = NULL,\n error = NULL,\n error_handler_path = $15,\n error_handler_args = $16,\n retry = $17\n WHERE\n workspace_id = $13 AND path = $14\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar",
"Bool",
"JsonbArray",
"Varchar",
"JsonbArray",
"Jsonb",
"Varchar",
"Varchar",
"Bool",
"Bool",
"Text",
"Text",
"Varchar",
"Jsonb",
"Jsonb"
]
},
"nullable": []
},
"hash": "e3d4f89ce36337af15d237b543eaca47771b480ff194884f9c947dcaf71d6cf9"
}

View File

@@ -0,0 +1,26 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT auto_invite->'instance_groups_roles' as instance_groups_roles,\n auto_invite->'instance_groups' as instance_groups_json\n FROM workspace_settings WHERE workspace_id = 'ws-multi-group'\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "instance_groups_roles",
"type_info": "Jsonb"
},
{
"ordinal": 1,
"name": "instance_groups_json",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": []
},
"nullable": [
null,
null
]
},
"hash": "e58ef252b0d2b81e9cd76f394a396abefd791906ada29dd5a7a9148157635ca5"
}

View File

@@ -47,6 +47,11 @@
"ordinal": 8,
"name": "added_via",
"type_info": "Jsonb"
},
{
"ordinal": 9,
"name": "is_service_account",
"type_info": "Bool"
}
],
"parameters": {
@@ -63,7 +68,8 @@
false,
false,
true,
true
true,
false
]
},
"hash": "e5fb3531f8bc7ef1f7484524f8c3bc9c48f71a44827ba0d01ac5588dc31082a2"

View File

@@ -0,0 +1,20 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO token\n (token_hash, token_prefix, token, email, label, expiration, super_admin, owner)\n VALUES ($1, $2, $3, $4, $5, $6, false, $7)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar",
"Varchar",
"Varchar",
"Timestamptz",
"Varchar"
]
},
"nullable": []
},
"hash": "f4ad2cf2438c2ae31e388517d09a2c1a2f63ab88cdbc79ffad96c6f9ffb5764b"
}

View File

@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO usr\n (workspace_id, email, username, is_admin, operator, is_service_account)\n VALUES ($1, $2, $3, false, true, true)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar"
]
},
"nullable": []
},
"hash": "f8654d5f50a80d862edbf57355502a9bd039d16f7dfb11e22d16ff9090456853"
}

838
backend/Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,6 @@
[package]
name = "windmill"
version = "1.665.0"
version = "1.672.0"
authors.workspace = true
edition.workspace = true
@@ -66,10 +66,13 @@ members = [
"./parsers/windmill-parser-nu",
"./parsers/windmill-parser-java",
"./parsers/windmill-parser-ruby",
"./parsers/windmill-parser-r",
"./parsers/windmill-parser-bash",
"./parsers/windmill-parser-py",
"./parsers/windmill-parser-py-asset",
"./parsers/windmill-parser-py-imports",
# Uncomment to build wasm parsers:
# "./parsers/windmill-parser-wasm",
"./parsers/windmill-parser-wac",
"./parsers/windmill-parser-sql",
"./parsers/windmill-parser-sql-asset",
@@ -79,10 +82,10 @@ members = [
"./windmill-test-utils",
"./windmill-api-integration-tests",
]
exclude = ["./windmill-duckdb-ffi-internal"]
exclude = ["./windmill-duckdb-ffi-internal", "./parsers/windmill-parser-wasm"]
[workspace.package]
version = "1.665.0"
version = "1.672.0"
authors = ["Ruben Fiszel <ruben@windmill.dev>"]
edition = "2021"
@@ -163,7 +166,8 @@ csharp = ["windmill-worker/csharp"]
nu = ["windmill-worker/nu"]
java = ["windmill-worker/java"]
ruby = ["windmill-worker/ruby"]
all_languages = ["python", "deno_core", "rust", "mysql", "oracledb", "duckdb", "mssql-kerberos", "bigquery", "csharp", "nu", "php", "java", "ruby"]
rlang = ["windmill-worker/rlang"]
all_languages = ["python", "deno_core", "rust", "mysql", "oracledb", "duckdb", "mssql-kerberos", "bigquery", "csharp", "nu", "php", "java", "ruby", "rlang"]
# For windows we have another set of languages enabled
all_languages_windows = ["python", "deno_core", "rust", "mysql", "oracledb", "duckdb", "mssql-winauth", "bigquery", "csharp", "nu", "php", "java"]
# Edition meta-features: shared groups
@@ -260,6 +264,8 @@ windmill-dep-map.workspace = true
windmill-test-utils.workspace = true
windmill-worker-volumes.workspace = true
windmill-types.workspace = true
opentelemetry = { workspace = true }
opentelemetry_sdk = { workspace = true }
windmill-trigger.workspace = true
windmill-trigger-websocket.workspace = true
windmill-trigger-postgres.workspace = true
@@ -345,6 +351,7 @@ windmill-parser-yaml = { path = "./parsers/windmill-parser-yaml" }
windmill-parser-csharp = { path = "./parsers/windmill-parser-csharp" }
windmill-parser-java = { path = "./parsers/windmill-parser-java" }
windmill-parser-ruby = { path = "./parsers/windmill-parser-ruby" }
windmill-parser-r = { path = "./parsers/windmill-parser-r" }
windmill-parser-nu = { path = "./parsers/windmill-parser-nu" }
windmill-parser-bash = { path = "./parsers/windmill-parser-bash" }
windmill-parser-sql = { path = "./parsers/windmill-parser-sql" }
@@ -362,7 +369,7 @@ reqwest-middleware = { version = "^0", features = ["json"] }
bitflags = "2.9.4"
memchr = "2.7.4"
axum = { version = "^0.7", features = ["multipart", "macros"] }
axum = { version = "^0.8", features = ["multipart", "macros"] }
headers = "^0"
hyper = { version = "^1", features = ["full"] }
hyper-tls = "^0.6"
@@ -371,7 +378,7 @@ tokio = { version = "=1.46.1", features = ["full", "tracing", "time"] }
tokio-stream = { version = "0.1.17" }
tower = "^0"
tower-http = { version = "^0.6", features = ["trace", "cors", "catch-panic"] }
tower-cookies = "^0.10"
tower-cookies = "^0.11"
#stuck because of swc for now
serde = "=1.0.220"
serde_json = { version = "^1", features = ["preserve_order", "raw_value"] }
@@ -386,7 +393,7 @@ tracing = "^0"
tracing-subscriber = { version = "^0", features = ["env-filter", "json"] }
tracing-appender = "^0"
prometheus = { version = "^0", default-features = false }
cookie = { version = "0.17.0" }
cookie = { version = "0.18.0" }
phf = { version = "0.11", features = ["macros"] }
rust-embed = { version = "^6", features = ["interpolate-folder-path"] }
mime_guess = "^2"
@@ -415,6 +422,7 @@ time = "^0"
serde_urlencoded = "^0"
astral-tokio-tar = "^0.5.6"
tempfile = "^3"
x509-parser = "^0.16"
tokio-util = { version = "=0.7.17", features = ["io"] }
json-pointer = "^0"
itertools = "^0.14.0"
@@ -566,18 +574,18 @@ flate2 = "^1"
http = "^1"
async-stream = "^0"
opentelemetry = "0.27.0"
tracing-opentelemetry = "0.28.0"
opentelemetry_sdk = { version = "0.27.1", features = ["rt-tokio"] }
opentelemetry-otlp = { version = "0.27.0", features = ["grpc-tonic", "tls"] }
opentelemetry-appender-tracing = "0.27.0"
opentelemetry-semantic-conventions = { version = "0.27.0", features = ["semconv_experimental"] }
opentelemetry-proto = { version = "0.29.0", features = ["with-serde", "gen-tonic"] }
opentelemetry = "0.30.0"
tracing-opentelemetry = "0.31.0"
opentelemetry_sdk = { version = "0.30.0", features = ["rt-tokio", "testing"] }
opentelemetry-otlp = { version = "0.30.0", features = ["grpc-tonic", "tls"] }
opentelemetry-appender-tracing = "0.30.0"
opentelemetry-semantic-conventions = { version = "0.30.0", features = ["semconv_experimental"] }
opentelemetry-proto = { version = "0.30.0", features = ["with-serde", "gen-tonic"] }
prost = "0.13"
bollard = "0.18.1"
tonic = { version = "=0.12.3", features = ["tls-native-roots"] }
tonic = { version = "^0.13", features = ["tls-native-roots"] }
byteorder = "1.5.0"
tikv-jemallocator = { version = "0.5" }
@@ -610,6 +618,7 @@ tree-sitter = { version = "0.23.0", features = [] }
tree-sitter-c-sharp = "0.23.0"
tree-sitter-java = "0.23.0"
tree-sitter-ruby = "0.23.0"
tree-sitter-r = "1.2.0"
oracle = { version = "0.6.3", features = ["chrono"] }
rumqttc = { version = "0.24.0", features = ["use-native-tls"]}
strum = { version = "0.27", features = ["derive"] }

View File

@@ -1 +1 @@
6db424512b0d02f86489e85f0026581b7637d6e6
e08a87450627bef9013498e40ee93a47bedda7ee

View File

@@ -0,0 +1 @@
-- No-op: this migration is a data fixup and cannot be reversed.

View File

@@ -0,0 +1,48 @@
-- Pre-fix: before permissioned_as migration drops the email column, update edited_by
-- for triggers where the user (edited_by) is not in the workspace but is a superadmin.
-- This ensures the subsequent 20260318000000 migration stores the raw email as permissioned_as
-- (via the `edited_by LIKE '%@%'` branch).
-- For instances that already applied 20260318000000, this is a no-op (email column is gone);
-- the 20260401000000 migration handles those as a fallback.
DO $$
DECLARE
trigger_table TEXT;
has_email BOOLEAN;
BEGIN
FOREACH trigger_table IN ARRAY ARRAY[
'http_trigger',
'websocket_trigger',
'postgres_trigger',
'mqtt_trigger',
'kafka_trigger',
'nats_trigger',
'sqs_trigger',
'gcp_trigger',
'email_trigger'
]
LOOP
SELECT EXISTS (
SELECT 1 FROM information_schema.columns
WHERE table_name = trigger_table AND column_name = 'email'
) INTO has_email;
IF has_email THEN
EXECUTE format($q$
UPDATE %I t
SET edited_by = t.email
WHERE NOT EXISTS (
SELECT 1 FROM usr u
WHERE u.username = t.edited_by
AND u.workspace_id = t.workspace_id
)
AND EXISTS (
SELECT 1 FROM password p
WHERE p.email = t.email
AND p.super_admin = true
)
$q$, trigger_table);
END IF;
END LOOP;
END;
$$;

View File

@@ -0,0 +1 @@
ALTER TABLE usr DROP COLUMN is_service_account;

View File

@@ -0,0 +1 @@
ALTER TABLE usr ADD COLUMN IF NOT EXISTS is_service_account BOOLEAN NOT NULL DEFAULT FALSE;

View File

@@ -0,0 +1,2 @@
ALTER TABLE magic_link ALTER COLUMN email TYPE VARCHAR(50);
ALTER TABLE schedule ALTER COLUMN email TYPE VARCHAR(50);

View File

@@ -0,0 +1,2 @@
ALTER TABLE magic_link ALTER COLUMN email TYPE VARCHAR(255);
ALTER TABLE schedule ALTER COLUMN email TYPE VARCHAR(255);

View File

@@ -0,0 +1,2 @@
ALTER TABLE kafka_trigger DROP COLUMN filter_logic;
ALTER TABLE websocket_trigger DROP COLUMN filter_logic;

View File

@@ -0,0 +1,2 @@
ALTER TABLE kafka_trigger ADD COLUMN filter_logic VARCHAR(3) NOT NULL DEFAULT 'and';
ALTER TABLE websocket_trigger ADD COLUMN filter_logic VARCHAR(3) NOT NULL DEFAULT 'and';

View File

@@ -0,0 +1,2 @@
ALTER TYPE SCRIPT_LANG ADD VALUE IF NOT EXISTS 'rlang';
UPDATE config SET config = jsonb_set(config, '{worker_tags}', config->'worker_tags' || '["rlang"]'::jsonb) WHERE name = 'worker__default' AND config @> '{"worker_tags": ["deno", "python3", "go", "bash", "powershell", "dependency", "flow", "hub", "other", "bun", "php", "rust", "ansible", "csharp", "nu", "java", "duckdb", "ruby"]}'::jsonb AND NOT config->'worker_tags' @> '"rlang"'::jsonb;

View File

@@ -0,0 +1,17 @@
[package]
name = "windmill-parser-r"
version.workspace = true
edition.workspace = true
authors.workspace = true
[lib]
name = "windmill_parser_r"
path = "./src/lib.rs"
[dependencies]
windmill-parser.workspace = true
tree-sitter.workspace = true
tree-sitter-r.workspace = true
anyhow.workspace = true
wasm-bindgen.workspace = true
serde_json.workspace = true

View File

@@ -0,0 +1,363 @@
#![cfg_attr(target_arch = "wasm32", feature(c_variadic))]
#[cfg(target_arch = "wasm32")]
pub mod wasm_libc;
use anyhow::anyhow;
use serde_json::Value;
use tree_sitter::Node;
use tree_sitter::Range;
use windmill_parser::json_to_typ;
use windmill_parser::Arg;
use windmill_parser::MainArgSignature;
pub fn parse_r_sig_meta(code: &str) -> anyhow::Result<MainArgSignature> {
let mut parser = tree_sitter::Parser::new();
let language = tree_sitter_r::LANGUAGE;
parser
.set_language(&language.into())
.map_err(|e| anyhow!("Error setting R as language: {e}"))?;
let tree = parser
.parse(code, None)
.ok_or(anyhow!("Failed to parse code"))?;
let root_node = tree.root_node();
let args = find_main_signature(root_node, code)?;
let main_sig = MainArgSignature {
star_args: false,
star_kwargs: false,
args: args.unwrap_or_default(),
has_preprocessor: None,
auto_kind: None,
};
Ok(main_sig)
}
pub fn parse_r_signature(code: &str) -> anyhow::Result<MainArgSignature> {
Ok(parse_r_sig_meta(code)?)
}
/// Extract package names from `library(...)` and `require(...)` calls in R code.
/// Returns a newline-separated list of package names.
pub fn parse_r_requirements(code: &str) -> anyhow::Result<String> {
let mut parser = tree_sitter::Parser::new();
let language = tree_sitter_r::LANGUAGE;
parser
.set_language(&language.into())
.map_err(|e| anyhow!("Error setting R as language: {e}"))?;
let tree = parser
.parse(code, None)
.ok_or(anyhow!("Failed to parse code"))?;
let root_node = tree.root_node();
let mut packages = vec![];
find_library_calls(root_node, code, &mut packages);
// Deduplicate and exclude base packages
packages.sort();
packages.dedup();
packages.retain(|p| !is_base_package(p));
Ok(packages.join("\n"))
}
fn find_library_calls(node: Node, code: &str, packages: &mut Vec<String>) {
let mut cursor = node.walk();
for child in node.children(&mut cursor) {
if child.kind() == "call" {
// call node: child 0 is the function name, child 1 is arguments
if let (Some(func_node), Some(args_node)) = (child.child(0), child.child(1)) {
let func_name = func_node.utf8_text(code.as_bytes()).unwrap_or("");
if func_name == "library" || func_name == "require" {
// AST: arguments → ( + argument → identifier/string + )
if args_node.kind() == "arguments" {
let mut args_cursor = args_node.walk();
for arg in args_node.children(&mut args_cursor) {
if arg.kind() == "argument" {
// The argument node wraps the actual value
if let Some(value_node) = arg.child(0) {
let pkg = value_node
.utf8_text(code.as_bytes())
.unwrap_or("")
.trim_matches('"')
.trim_matches('\'');
if !pkg.is_empty() {
packages.push(pkg.to_string());
}
}
break; // only first arg
}
}
}
}
}
}
// Recurse into children to find nested library() calls
find_library_calls(child, code, packages);
}
}
fn is_base_package(pkg: &str) -> bool {
matches!(
pkg,
"base"
| "compiler"
| "datasets"
| "grDevices"
| "graphics"
| "grid"
| "methods"
| "parallel"
| "splines"
| "stats"
| "stats4"
| "tcltk"
| "tools"
| "utils"
)
}
/// Find the main function signature in R code.
/// R function definitions look like: `main <- function(x, y = 10) { ... }`
/// In the tree-sitter-r AST, this is a `binary_operator` node with:
/// - child 0: identifier "main"
/// - child 1: "<-" or "="
/// - child 2: function_definition node
fn find_main_signature<'a>(root_node: Node<'a>, code: &str) -> anyhow::Result<Option<Vec<Arg>>> {
let mut cursor = root_node.walk();
for x in root_node.children(&mut cursor) {
if x.kind() == "binary_operator" {
let child_count = x.child_count();
if child_count < 3 {
continue;
}
// First child should be identifier "main"
let ident_node = x.child(0).unwrap();
if ident_node.kind() != "identifier" {
continue;
}
let ident = ident_node.utf8_text(code.as_bytes()).unwrap_or("");
if ident != "main" {
continue;
}
// Second child should be "<-" or "="
let op_node = x.child(1).unwrap();
let op = op_node.utf8_text(code.as_bytes()).unwrap_or("");
if op != "<-" && op != "=" {
continue;
}
// Third child should be the function_definition
let func_node = x.child(2).unwrap();
if func_node.kind() != "function_definition" {
continue;
}
return Ok(Some(parse_function_params(func_node, code)?));
}
}
Ok(None)
}
/// Parse parameters from a function_definition node.
/// function_definition has children: "function", parameters, body
/// Each parameter node has:
/// - 1 child (identifier) for positional args
/// - 3 children (identifier, "=", value) for default args
fn parse_function_params(func_node: Node, code: &str) -> anyhow::Result<Vec<Arg>> {
let mut args = vec![];
let mut func_cursor = func_node.walk();
for child in func_node.children(&mut func_cursor) {
if child.kind() == "parameters" {
let mut param_cursor = child.walk();
for param in child.children(&mut param_cursor) {
if param.kind() != "parameter" {
continue;
}
let param_child_count = param.child_count();
if param_child_count == 1 {
// Simple parameter: just identifier
let ident_node = param.child(0).unwrap();
let name = ident_node.utf8_text(code.as_bytes())?;
args.push(Arg { name: name.to_owned(), ..Default::default() });
} else if param_child_count >= 3 {
// Default parameter: identifier = value
let ident_node = param.child(0).unwrap();
let value_node = param.child(2).unwrap();
let name = ident_node.utf8_text(code.as_bytes())?;
let Range { start_byte, end_byte, .. } = value_node.range();
let raw = &code[start_byte..end_byte];
// Convert R literals to JSON
let unparsed = raw
.replace("NULL", "null")
.replace("TRUE", "true")
.replace("FALSE", "false");
match serde_json::from_str::<Value>(&unparsed) {
Ok(default) => {
args.push(Arg {
name: name.to_owned(),
typ: json_to_typ(&default, true),
default: Some(default),
has_default: true,
..Default::default()
});
}
Err(_) => {
args.push(Arg {
name: name.to_owned(),
has_default: true,
..Default::default()
});
}
}
}
}
}
}
Ok(args)
}
#[cfg(test)]
mod test {
use serde_json::json;
use windmill_parser::Typ;
use super::parse_r_sig_meta as parse;
#[test]
fn test_parse_r_no_main() {
let code = r#"
not_main <- function() {}
helper <- function(x) { x + 1 }
"#;
let sig = parse(code).unwrap();
assert_eq!(
sig,
windmill_parser::MainArgSignature { auto_kind: None, ..Default::default() }
);
}
#[test]
fn test_parse_r_no_args() {
let code = r#"
main <- function() {
return(42)
}
"#;
let sig = parse(code).unwrap();
assert_eq!(
sig,
windmill_parser::MainArgSignature { auto_kind: None, ..Default::default() }
);
}
#[test]
fn test_parse_r_positional_args() {
let code = r#"main <- function(a, b, c) { a + b + c }"#;
let sig = parse(code).unwrap();
assert_eq!(
sig,
windmill_parser::MainArgSignature {
args: vec![
windmill_parser::Arg { name: "a".into(), ..Default::default() },
windmill_parser::Arg { name: "b".into(), ..Default::default() },
windmill_parser::Arg { name: "c".into(), ..Default::default() },
],
auto_kind: None,
..Default::default()
}
);
}
#[test]
fn test_parse_r_default_args() {
let code = r#"main <- function(a = 10, b = "hey", c = FALSE) { }"#;
let sig = parse(code).unwrap();
assert_eq!(sig.args.len(), 3);
assert_eq!(sig.args[0].name, "a");
assert_eq!(sig.args[0].default, Some(json!(10)));
assert_eq!(sig.args[0].typ, Typ::Int);
assert_eq!(sig.args[1].name, "b");
assert_eq!(sig.args[1].default, Some(json!("hey")));
assert_eq!(sig.args[1].typ, Typ::Str(None));
assert_eq!(sig.args[2].name, "c");
assert_eq!(sig.args[2].default, Some(json!(false)));
assert_eq!(sig.args[2].typ, Typ::Bool);
}
#[test]
fn test_parse_r_equals_assignment() {
let code = r#"main = function(x, y = 5) { x + y }"#;
let sig = parse(code).unwrap();
assert_eq!(sig.args.len(), 2);
assert_eq!(sig.args[0].name, "x");
assert_eq!(sig.args[1].name, "y");
assert_eq!(sig.args[1].default, Some(json!(5)));
}
#[test]
fn test_parse_r_null_default() {
let code = r#"main <- function(x = NULL) { x }"#;
let sig = parse(code).unwrap();
assert_eq!(sig.args.len(), 1);
assert_eq!(sig.args[0].name, "x");
assert_eq!(sig.args[0].default, Some(json!(null)));
}
#[test]
fn test_parse_r_requirements() {
use super::parse_r_requirements;
let code = r#"
library(dplyr)
library(ggplot2)
require(tidyr)
library(stats)
main <- function(x) {
library(stringr)
x
}
"#;
let reqs = parse_r_requirements(code).unwrap();
let pkgs: Vec<&str> = reqs.lines().collect();
assert!(pkgs.contains(&"dplyr"));
assert!(pkgs.contains(&"ggplot2"));
assert!(pkgs.contains(&"tidyr"));
assert!(pkgs.contains(&"stringr"));
assert!(!pkgs.contains(&"stats")); // base package excluded
}
#[test]
fn test_parse_r_requirements_string_args() {
use super::parse_r_requirements;
let code = r#"
library("data.table")
require("jsonlite")
main <- function() { }
"#;
let reqs = parse_r_requirements(code).unwrap();
let pkgs: Vec<&str> = reqs.lines().collect();
assert!(pkgs.contains(&"data.table"));
assert!(pkgs.contains(&"jsonlite"));
}
#[test]
fn test_parse_r_requirements_no_deps() {
use super::parse_r_requirements;
let code = r#"main <- function(x) { x + 1 }"#;
let reqs = parse_r_requirements(code).unwrap();
assert!(reqs.is_empty());
}
}

View File

@@ -0,0 +1,293 @@
use std::collections::BTreeMap;
use std::sync::{Mutex, OnceLock};
use std::{
alloc::{self, Layout},
ffi::{c_char, c_int, c_void},
mem::align_of,
ptr,
};
use wasm_bindgen::prelude::*;
/* -------------------------------- stdlib.h -------------------------------- */
#[no_mangle]
pub unsafe extern "C" fn abort() {
panic!("Aborted from C");
}
macro_rules! console_log {
($($t:tt)*) => (unsafe { log(&format_args!($($t)*).to_string()) })
}
#[wasm_bindgen]
extern "C" {
#[wasm_bindgen(js_namespace = console)]
fn log(a: &str);
}
#[no_mangle]
pub unsafe extern "C" fn malloc(size: usize) -> *mut c_void {
if size == 0 {
return ptr::null_mut();
}
let (layout, offset_to_data) = layout_for_size_prepended(size);
let buf = alloc::alloc(layout);
store_layout(buf, layout, offset_to_data)
}
#[no_mangle]
pub unsafe extern "C" fn calloc(count: usize, size: usize) -> *mut c_void {
if count == 0 || size == 0 {
return ptr::null_mut();
}
let (layout, offset_to_data) = layout_for_size_prepended(size * count);
let buf = alloc::alloc_zeroed(layout);
store_layout(buf, layout, offset_to_data)
}
#[no_mangle]
pub unsafe extern "C" fn realloc(buf: *mut c_void, new_size: usize) -> *mut c_void {
if buf.is_null() {
malloc(new_size)
} else if new_size == 0 {
free(buf);
ptr::null_mut()
} else {
let (old_buf, old_layout) = retrieve_layout(buf);
let (new_layout, offset_to_data) = layout_for_size_prepended(new_size);
let new_buf = alloc::realloc(old_buf, old_layout, new_layout.size());
store_layout(new_buf, new_layout, offset_to_data)
}
}
#[no_mangle]
pub unsafe extern "C" fn free(buf: *mut c_void) {
if buf.is_null() {
return;
}
let (buf, layout) = retrieve_layout(buf);
alloc::dealloc(buf, layout);
}
// In all these allocations, we store the layout before the data for later retrieval.
// This is because we need to know the layout when deallocating the memory.
// Here are some helper methods for that:
/// Given a pointer to the data, retrieve the layout and the pointer to the layout.
unsafe fn retrieve_layout(buf: *mut c_void) -> (*mut u8, Layout) {
let (_, layout_offset) = Layout::new::<Layout>()
.extend(Layout::from_size_align(0, align_of::<*const u8>() * 2).unwrap())
.unwrap();
let buf = (buf as *mut u8).offset(-(layout_offset as isize));
let layout = *(buf as *mut Layout);
(buf, layout)
}
/// Calculate a layout for a given size with space for storing a layout at the start.
/// Returns the layout and the offset to the data.
fn layout_for_size_prepended(size: usize) -> (Layout, usize) {
Layout::new::<Layout>()
.extend(Layout::from_size_align(size, align_of::<*const u8>() * 2).unwrap())
.unwrap()
}
/// Store a layout in the pointer, returning a pointer to where the data should be stored.
unsafe fn store_layout(buf: *mut u8, layout: Layout, offset_to_data: usize) -> *mut c_void {
*(buf as *mut Layout) = layout;
(buf as *mut u8).offset(offset_to_data as isize) as *mut c_void
}
/* -------------------------------- string.h -------------------------------- */
#[no_mangle]
pub unsafe extern "C" fn strncmp(ptr1: *const c_void, ptr2: *const c_void, n: usize) -> c_int {
let s1 = std::slice::from_raw_parts(ptr1 as *const u8, n);
let s2 = std::slice::from_raw_parts(ptr2 as *const u8, n);
for (a, b) in s1.iter().zip(s2.iter()) {
if *a != *b || *a == 0 {
return (*a as i32) - (*b as i32);
}
}
0
}
// Implementation by AI:
pub type size_t = usize;
use std::slice;
#[no_mangle]
pub unsafe extern "C" fn memchr(haystack: *const c_void, needle: c_int, len: usize) -> *mut c_void {
if haystack.is_null() || len == 0 {
return ptr::null_mut(); // Return null if the input pointer is null or length is zero
}
let needle_byte = needle as u8; // Convert needle to a byte
// Create a pointer to the start of the haystack
let mut current = haystack as *const u8;
// Iterate through the memory block
for _ in 0..len {
if *current == needle_byte {
return current as *mut c_void; // Return the pointer to the found byte
}
current = current.add(1); // Move to the next byte
}
ptr::null_mut() // Return null if the byte was not found
}
#[no_mangle]
pub unsafe extern "C" fn strchr(mut s: *const c_char, c: c_int) -> *mut c_char {
if s.is_null() {
return std::ptr::null_mut(); // Return null if the input string is null
}
let target = c as u8 as char; // Convert c to a char
let mut current = s;
// Iterate through the string until we find the character or reach the end
while *current != 0 {
if *current as u8 as char == target {
return current as *mut c_char; // Return the pointer to the found character
}
current = current.add(1); // Move to the next character
}
std::ptr::null_mut() // Return null if the character was not found
}
// End of AI implemetation
/* -------------------------------- wctype.h -------------------------------- */
#[no_mangle]
pub unsafe extern "C" fn iswspace(c: c_int) -> bool {
char::from_u32(c as u32).map_or(false, |c| c.is_whitespace())
}
#[no_mangle]
pub unsafe extern "C" fn iswalnum(c: c_int) -> bool {
char::from_u32(c as u32).map_or(false, |c| c.is_alphanumeric())
}
// Implementation by AI:
pub type wint_t = u32;
#[no_mangle]
pub extern "C" fn iswdigit(wc: wint_t) -> c_int {
// Check if the character is a digit ('0' to '9')
if wc >= '0' as wint_t && wc <= '9' as wint_t {
return 1; // Return true (1)
}
0 // Return false (0)
}
#[no_mangle]
pub extern "C" fn iswupper(wc: wint_t) -> c_int {
// Check if the character is an uppercase letter ('A' to 'Z')
if wc >= 'A' as wint_t && wc <= 'Z' as wint_t {
return 1; // Return true (1)
}
0 // Return false (0)
}
#[no_mangle]
pub extern "C" fn iswalpha(wc: wint_t) -> c_int {
// Check if the character is an alphabetic character ('A' to 'Z' or 'a' to 'z')
if (wc >= 'A' as wint_t && wc <= 'Z' as wint_t) || (wc >= 'a' as wint_t && wc <= 'z' as wint_t)
{
return 1; // Return true (1)
}
0 // Return false (0)
}
#[no_mangle]
pub extern "C" fn iswlower(wc: wint_t) -> c_int {
// Check if the character is a lowercase letter ('a' to 'z')
if wc >= 'a' as wint_t && wc <= 'z' as wint_t {
return 1; // Return true (1)
}
0 // Return false (0)
}
// End of AI implemetation
/* --------------------------------- time.h --------------------------------- */
#[no_mangle]
pub unsafe extern "C" fn clock() -> u64 {
panic!("clock is not supported");
}
/* --------------------------------- ctype.h -------------------------------- */
#[no_mangle]
pub unsafe extern "C" fn isprint(c: c_int) -> bool {
c >= 32 && c <= 126
}
/* --------------------------------- stdio.h -------------------------------- */
#[no_mangle]
pub unsafe extern "C" fn fprintf(_file: *mut c_void, _format: *const c_void, _args: ...) -> c_int {
panic!("fprintf is not supported");
}
#[no_mangle]
pub unsafe extern "C" fn fputs(_s: *const c_void, _file: *mut c_void) -> c_int {
panic!("fputs is not supported");
}
#[no_mangle]
pub unsafe extern "C" fn fputc(_c: c_int, _file: *mut c_void) -> c_int {
panic!("fputc is not supported");
}
#[no_mangle]
pub unsafe extern "C" fn fdopen(_fd: c_int, _mode: *const c_void) -> *mut c_void {
panic!("fdopen is not supported");
}
#[no_mangle]
pub unsafe extern "C" fn fclose(_file: *mut c_void) -> c_int {
panic!("fclose is not supported");
}
#[no_mangle]
pub unsafe extern "C" fn fwrite(
_ptr: *const c_void,
_size: usize,
_nmemb: usize,
_stream: *mut c_void,
) -> usize {
panic!("fwrite is not supported");
}
#[no_mangle]
pub unsafe extern "C" fn vsnprintf(
_buf: *mut c_char,
_size: usize,
_format: *const c_char,
_args: ...
) -> c_int {
panic!("vsnprintf is not supported");
}
#[no_mangle]
pub extern "C" fn clock_gettime(ptr: usize, new_size: usize) {
panic!("clock_gettime is not supported");
}
// int snprintf( char* restrict buffer, size_t bufsz, const char* restrict format, ... );
#[no_mangle]
pub extern "C" fn snprintf() {
panic!("snprintf is not supported");
}
#[no_mangle]
pub extern "C" fn __assert_fail(_: *const i32, _: *const i32, _: *const i32, _: *const i32) {
panic!("oh no");
}

View File

@@ -27,11 +27,15 @@ pub struct DagNode {
#[serde(tag = "type")]
pub enum DagNodeType {
Step { name: String, script: String },
InlineStep { name: String },
Sleep { seconds: String },
WaitForApproval,
Branch { condition_source: String },
ParallelStart,
ParallelEnd,
LoopStart { iter_source: String },
LoopEnd,
Merge,
Return,
}

View File

@@ -37,7 +37,8 @@ impl LineIndex {
/// Maps task function name → optional external path (from `@task(path="...")`)
type TaskFunctions = HashMap<String, Option<String>>;
/// First pass: scan top-level `@task async def foo(...)` declarations.
/// First pass: scan top-level `@task async def foo(...)` declarations
/// and `foo = task_script("path")` / `foo = task_flow("path")` assignments.
fn collect_task_functions(stmts: &[Stmt]) -> TaskFunctions {
let mut tasks = HashMap::new();
for stmt in stmts {
@@ -61,6 +62,30 @@ fn collect_task_functions(stmts: &[Stmt]) -> TaskFunctions {
}
}
}
// foo = task_script("path") or foo = task_flow("path")
if let Stmt::Assign(assign) = stmt {
if let Expr::Call(call) = assign.value.as_ref() {
if let Expr::Name(ExprName { id, .. }) = call.func.as_ref() {
if id.as_str() == "task_script" || id.as_str() == "task_flow" {
// Extract the path from the first positional argument
let path = call.args.first().and_then(|arg| {
if let Expr::Constant(c) = arg {
if let rustpython_parser::ast::Constant::Str(s) = &c.value {
return Some(s.to_string());
}
}
None
});
// Extract variable name from target
if let Some(Expr::Name(ExprName { id: var_name, .. })) =
assign.targets.first()
{
tasks.insert(var_name.to_string(), path);
}
}
}
}
}
}
tasks
}
@@ -88,8 +113,6 @@ struct WacWalker {
node_counter: usize,
line_index: LineIndex,
task_functions: TaskFunctions,
in_try: bool,
in_while: bool,
in_nested_func: bool,
in_comprehension: bool,
}
@@ -103,8 +126,6 @@ impl WacWalker {
node_counter: 0,
line_index: LineIndex::new(source),
task_functions,
in_try: false,
in_while: false,
in_nested_func: false,
in_comprehension: false,
}
@@ -292,6 +313,9 @@ impl WacWalker {
if self.is_task_fn_call(expr) {
return true;
}
if Self::is_sdk_call(expr) {
return true;
}
match expr {
Expr::Await(ExprAwait { value, .. }) => self.expr_contains_step(value),
Expr::Call(call) => {
@@ -307,6 +331,17 @@ impl WacWalker {
}
}
/// Check if expr is a call to a known SDK function (step, sleep, wait_for_approval)
fn is_sdk_call(expr: &Expr) -> bool {
if let Expr::Call(call) = expr {
if let Expr::Name(ExprName { id, .. }) = call.func.as_ref() {
let name = id.as_str();
return name == "step" || name == "sleep" || name == "wait_for_approval";
}
}
false
}
/// Walk a list of statements, returning (first_node_id, last_node_id)
fn walk_body(&mut self, body: &[Stmt]) -> Option<(String, String)> {
let mut first_id: Option<String> = None;
@@ -353,13 +388,17 @@ impl WacWalker {
}
fn walk_expr_stmt(&mut self, expr: &Expr) -> Option<(String, String)> {
// await task_fn(...)
// await task_fn(...) / await step(...) / await sleep(...) / await wait_for_approval(...)
if let Expr::Await(ExprAwait { value, .. }) = expr {
// await task_fn(...)
if let Expr::Call(call) = value.as_ref() {
if self.is_task_fn_call(&Expr::Call(call.clone())) {
return self.emit_step(call, expr);
}
// Check for SDK-level calls: step(), sleep(), wait_for_approval()
if let Some(result) = self.try_emit_sdk_call(call, expr) {
return Some(result);
}
}
// await asyncio.gather(task_fn(...), task_fn(...), ...)
if Self::is_asyncio_gather_call(value) {
@@ -378,17 +417,69 @@ impl WacWalker {
None
}
/// Try to emit a node for SDK-level calls: step(), sleep(), wait_for_approval()
fn try_emit_sdk_call(&mut self, call: &ExprCall, expr: &Expr) -> Option<(String, String)> {
let callee_name = match call.func.as_ref() {
Expr::Name(ExprName { id, .. }) => Some(id.as_str()),
_ => None,
}?;
let line = self.line_of_expr(expr);
match callee_name {
"step" => {
// step("name", fn) — extract the name from the first string argument
let name = call
.args
.first()
.and_then(|arg| {
if let Expr::Constant(c) = arg {
if let rustpython_parser::ast::Constant::Str(s) = &c.value {
return Some(s.to_string());
}
}
None
})
.unwrap_or_else(|| "step".to_string());
let id = self.next_id();
let node_id = self.add_node(DagNode {
id: id.clone(),
node_type: DagNodeType::InlineStep { name: name.clone() },
label: name,
line,
});
Some((node_id.clone(), node_id))
}
"sleep" => {
let seconds = call
.args
.first()
.map(|arg| Self::expr_to_source(arg))
.unwrap_or_else(|| "?".to_string());
let id = self.next_id();
let node_id = self.add_node(DagNode {
id: id.clone(),
node_type: DagNodeType::Sleep { seconds: seconds.clone() },
label: format!("sleep({seconds})"),
line,
});
Some((node_id.clone(), node_id))
}
"wait_for_approval" => {
let id = self.next_id();
let node_id = self.add_node(DagNode {
id: id.clone(),
node_type: DagNodeType::WaitForApproval,
label: "wait_for_approval".to_string(),
line,
});
Some((node_id.clone(), node_id))
}
_ => None,
}
}
fn emit_step(&mut self, call: &ExprCall, expr: &Expr) -> Option<(String, String)> {
if self.in_try {
self.errors
.push(validation::error_step_in_try(self.line_of_expr(expr)));
return None;
}
if self.in_while {
self.errors
.push(validation::error_step_in_while(self.line_of_expr(expr)));
return None;
}
if self.in_nested_func {
self.errors.push(validation::error_step_in_nested_function(
self.line_of_expr(expr),
@@ -416,17 +507,6 @@ impl WacWalker {
}
fn emit_parallel(&mut self, gather_call: &ExprCall, expr: &Expr) -> Option<(String, String)> {
if self.in_try {
self.errors
.push(validation::error_step_in_try(self.line_of_expr(expr)));
return None;
}
if self.in_while {
self.errors
.push(validation::error_step_in_while(self.line_of_expr(expr)));
return None;
}
let line = self.line_of_expr(expr);
let start_id = self.next_id();
let start_node_id = self.add_node(DagNode {
@@ -491,8 +571,6 @@ impl WacWalker {
line,
});
let merge_id = format!("{branch_id}_merge");
let mut last_ids = Vec::new();
if let Some((true_first, true_last)) = self.walk_body(&if_stmt.body) {
@@ -514,7 +592,17 @@ impl WacWalker {
if last_ids.len() == 1 {
Some((branch_node_id, last_ids.into_iter().next().unwrap()))
} else {
Some((branch_node_id, merge_id))
let merge_id = format!("{branch_id}_merge");
let merge_node_id = self.add_node(DagNode {
id: merge_id,
node_type: DagNodeType::Merge,
label: "merge".to_string(),
line,
});
for last in last_ids {
self.add_edge(&last, &merge_node_id, None);
}
Some((branch_node_id, merge_node_id))
}
}
@@ -552,11 +640,36 @@ impl WacWalker {
}
fn walk_while(&mut self, while_stmt: &StmtWhile) -> Option<(String, String)> {
if self.body_contains_step(&while_stmt.body) {
let line = self.line_index.line_of(while_stmt.range.start().to_usize());
self.errors.push(validation::error_step_in_while(line));
if !self.body_contains_step(&while_stmt.body) {
return None;
}
None
let line = self.line_index.line_of(while_stmt.range.start().to_usize());
let condition = Self::expr_to_source(&while_stmt.test);
let start_id = self.next_id();
let start_node_id = self.add_node(DagNode {
id: start_id.clone(),
node_type: DagNodeType::LoopStart { iter_source: condition },
label: "while".to_string(),
line,
});
if let Some((body_first, body_last)) = self.walk_body(&while_stmt.body) {
self.add_edge(&start_node_id, &body_first, None);
self.add_edge(&body_last, &start_node_id, Some("next".to_string()));
}
let end_id = self.next_id();
let end_node_id = self.add_node(DagNode {
id: end_id.clone(),
node_type: DagNodeType::LoopEnd,
label: "end while".to_string(),
line,
});
self.add_edge(&start_node_id, &end_node_id, Some("done".to_string()));
Some((start_node_id, end_node_id))
}
fn walk_try(&mut self, try_stmt: &StmtTry) -> Option<(String, String)> {
@@ -569,11 +682,17 @@ impl WacWalker {
}
});
if has_steps {
let line = self.line_index.line_of(try_stmt.range.start().to_usize());
self.errors.push(validation::error_step_in_try(line));
if !has_steps {
return None;
}
None
let line = self.line_index.line_of(try_stmt.range.start().to_usize());
self.emit_try_catch_branch(
&try_stmt.body,
&try_stmt.handlers,
&try_stmt.finalbody,
line,
)
}
fn walk_try_star(&mut self, try_stmt: &StmtTryStar) -> Option<(String, String)> {
@@ -586,11 +705,81 @@ impl WacWalker {
}
});
if has_steps {
let line = self.line_index.line_of(try_stmt.range.start().to_usize());
self.errors.push(validation::error_step_in_try(line));
if !has_steps {
return None;
}
None
let line = self.line_index.line_of(try_stmt.range.start().to_usize());
self.emit_try_catch_branch(
&try_stmt.body,
&try_stmt.handlers,
&try_stmt.finalbody,
line,
)
}
fn emit_try_catch_branch(
&mut self,
try_body: &[Stmt],
handlers: &[rustpython_parser::ast::ExceptHandler],
finally_body: &[Stmt],
line: usize,
) -> Option<(String, String)> {
let branch_id = self.next_id();
let branch_node_id = self.add_node(DagNode {
id: branch_id.clone(),
node_type: DagNodeType::Branch { condition_source: "try/except".to_string() },
label: "try".to_string(),
line,
});
let mut last_ids = Vec::new();
// Try body
if let Some((try_first, try_last)) = self.walk_body(try_body) {
self.add_edge(&branch_node_id, &try_first, Some("try".to_string()));
last_ids.push(try_last);
} else {
last_ids.push(branch_node_id.clone());
}
// Except handlers
for handler in handlers {
match handler {
rustpython_parser::ast::ExceptHandler::ExceptHandler(eh) => {
if let Some((catch_first, catch_last)) = self.walk_body(&eh.body) {
self.add_edge(&branch_node_id, &catch_first, Some("except".to_string()));
last_ids.push(catch_last);
}
}
}
}
// Finally body — sequential after merge
let merge_last = if last_ids.len() == 1 {
last_ids.into_iter().next().unwrap()
} else {
let merge_id = format!("{branch_id}_merge");
let merge_node_id = self.add_node(DagNode {
id: merge_id,
node_type: DagNodeType::Merge,
label: "merge".to_string(),
line,
});
for last in last_ids {
self.add_edge(&last, &merge_node_id, None);
}
merge_node_id
};
if !finally_body.is_empty() {
if let Some((finally_first, finally_last)) = self.walk_body(finally_body) {
self.add_edge(&merge_last, &finally_first, None);
return Some((branch_node_id, finally_last));
}
}
Some((branch_node_id, merge_last))
}
fn walk_return(&mut self, ret: &StmtReturn) -> Option<(String, String)> {

View File

@@ -51,22 +51,29 @@ fn extract_var_name(pat: &Pat) -> Option<String> {
}
}
/// Check if expr is `task(async fn)` or `task("path", async fn)`.
/// Returns Some(optional_path) if it is a task() call.
/// Check if expr is `task(async fn)`, `task("path", async fn)`,
/// `taskScript("path")`, or `taskFlow("path")`.
/// Returns Some(optional_path) if it is a task/taskScript/taskFlow call.
fn extract_task_call_info(expr: &Expr) -> Option<Option<String>> {
if let Expr::Call(call) = expr {
if let Callee::Expr(callee) = &call.callee {
if let Expr::Ident(ident) = callee.as_ref() {
if ident.sym.as_ref() == "task" {
let name = ident.sym.as_ref();
if name == "task" {
// task("f/path", async fn) or task(async fn)
if call.args.len() == 2 {
// task("f/path", async fn)
let path = extract_string_lit(&call.args[0].expr);
return Some(path);
} else if call.args.len() == 1 {
// task(async fn)
return Some(None);
}
} else if name == "taskScript" || name == "taskFlow" {
// taskScript("./helper.ts") or taskFlow("f/my_flow")
if let Some(first_arg) = call.args.first() {
let path = extract_string_lit(&first_arg.expr);
return Some(path);
}
return Some(None);
}
}
}
@@ -81,8 +88,6 @@ struct TsWacWalker {
node_counter: usize,
cm: Lrc<SourceMap>,
task_functions: TaskFunctions,
in_try: bool,
in_while: bool,
in_nested_func: bool,
}
@@ -95,8 +100,6 @@ impl TsWacWalker {
node_counter: 0,
cm,
task_functions,
in_try: false,
in_while: false,
in_nested_func: false,
}
}
@@ -224,6 +227,9 @@ impl TsWacWalker {
if self.is_task_call(expr) {
return true;
}
if Self::is_sdk_call(expr) {
return true;
}
match expr {
Expr::Await(await_expr) => self.expr_contains_step(&await_expr.arg),
Expr::Call(call) => {
@@ -237,6 +243,19 @@ impl TsWacWalker {
}
}
/// Check if expr is a call to a known SDK function (step, sleep, waitForApproval)
fn is_sdk_call(expr: &Expr) -> bool {
if let Expr::Call(call) = expr {
if let Callee::Expr(callee) = &call.callee {
if let Expr::Ident(ident) = callee.as_ref() {
let name = ident.sym.as_ref();
return name == "step" || name == "sleep" || name == "waitForApproval";
}
}
}
false
}
fn walk_body(&mut self, stmts: &[Stmt]) -> Option<(String, String)> {
let mut first_id: Option<String> = None;
let mut prev_id: Option<String> = None;
@@ -294,12 +313,16 @@ impl TsWacWalker {
}
fn walk_expr_stmt(&mut self, expr: &Expr) -> Option<(String, String)> {
// await task_fn(...)
// await task_fn(...) / await step(...) / await sleep(...) / await waitForApproval(...)
if let Expr::Await(await_expr) = expr {
if let Expr::Call(call) = await_expr.arg.as_ref() {
if self.is_task_call(&Expr::Call(call.clone())) {
return self.emit_step(call, expr);
}
// Check for SDK-level calls: step(), sleep(), waitForApproval()
if let Some(result) = self.try_emit_sdk_call(call, expr) {
return Some(result);
}
}
// await Promise.all([task_fn(...), ...])
if Self::is_promise_all(&await_expr.arg) {
@@ -318,17 +341,70 @@ impl TsWacWalker {
None
}
/// Try to emit a node for SDK-level calls: step(), sleep(), waitForApproval()
fn try_emit_sdk_call(&mut self, call: &CallExpr, expr: &Expr) -> Option<(String, String)> {
let callee_name = match &call.callee {
Callee::Expr(callee) => match callee.as_ref() {
Expr::Ident(ident) => Some(ident.sym.as_ref().to_string()),
_ => None,
},
_ => None,
}?;
let line = self.span_line(expr.span());
match callee_name.as_str() {
"step" => {
// step("name", fn) — extract the name from the first string argument
let name = call
.args
.first()
.and_then(|a| extract_string_lit(&a.expr))
.unwrap_or_else(|| "step".to_string());
let id = self.next_id();
let node_id = self.add_node(DagNode {
id: id.clone(),
node_type: DagNodeType::InlineStep { name: name.clone() },
label: name,
line,
});
Some((node_id.clone(), node_id))
}
"sleep" => {
// sleep(N) — extract the duration from the first argument
let seconds = call
.args
.first()
.map(|a| {
self.cm
.span_to_snippet(a.expr.span())
.unwrap_or_else(|_| "?".to_string())
})
.unwrap_or_else(|| "?".to_string());
let id = self.next_id();
let node_id = self.add_node(DagNode {
id: id.clone(),
node_type: DagNodeType::Sleep { seconds: seconds.clone() },
label: format!("sleep({seconds})"),
line,
});
Some((node_id.clone(), node_id))
}
"waitForApproval" => {
let id = self.next_id();
let node_id = self.add_node(DagNode {
id: id.clone(),
node_type: DagNodeType::WaitForApproval,
label: "waitForApproval".to_string(),
line,
});
Some((node_id.clone(), node_id))
}
_ => None,
}
}
fn emit_step(&mut self, call: &CallExpr, expr: &Expr) -> Option<(String, String)> {
if self.in_try {
self.errors
.push(validation::error_step_in_catch(self.span_line(expr.span())));
return None;
}
if self.in_while {
self.errors
.push(validation::error_step_in_while(self.span_line(expr.span())));
return None;
}
if self.in_nested_func {
self.errors.push(validation::error_step_in_nested_function(
self.span_line(expr.span()),
@@ -350,17 +426,6 @@ impl TsWacWalker {
}
fn emit_parallel(&mut self, promise_call: &CallExpr, expr: &Expr) -> Option<(String, String)> {
if self.in_try {
self.errors
.push(validation::error_step_in_catch(self.span_line(expr.span())));
return None;
}
if self.in_while {
self.errors
.push(validation::error_step_in_while(self.span_line(expr.span())));
return None;
}
let line = self.span_line(expr.span());
let start_id = self.next_id();
let start_node_id = self.add_node(DagNode {
@@ -457,7 +522,16 @@ impl TsWacWalker {
Some((branch_node_id, last_ids.into_iter().next().unwrap()))
} else {
let merge_id = format!("{branch_id}_merge");
Some((branch_node_id, merge_id))
let merge_node_id = self.add_node(DagNode {
id: merge_id,
node_type: DagNodeType::Merge,
label: "merge".to_string(),
line,
});
for last in last_ids {
self.add_edge(&last, &merge_node_id, None);
}
Some((branch_node_id, merge_node_id))
}
}
@@ -473,7 +547,7 @@ impl TsWacWalker {
return None;
}
let iter_source = self.expr_to_source(&for_in.right);
self.walk_loop_body_with_iter(&for_in.body, for_in.span, &iter_source)
self.walk_loop_body_with_iter(&for_in.body, for_in.span, &iter_source, "for")
}
fn walk_for_of(&mut self, for_of: &ForOfStmt) -> Option<(String, String)> {
@@ -481,7 +555,7 @@ impl TsWacWalker {
return None;
}
let iter_source = self.expr_to_source(&for_of.right);
self.walk_loop_body_with_iter(&for_of.body, for_of.span, &iter_source)
self.walk_loop_body_with_iter(&for_of.body, for_of.span, &iter_source, "for")
}
fn walk_loop_body(
@@ -490,7 +564,7 @@ impl TsWacWalker {
span: swc_common::Span,
_label: &str,
) -> Option<(String, String)> {
self.walk_loop_body_with_iter(body, span, "...")
self.walk_loop_body_with_iter(body, span, "...", "for")
}
fn walk_loop_body_with_iter(
@@ -498,13 +572,14 @@ impl TsWacWalker {
body: &Stmt,
span: swc_common::Span,
iter_source: &str,
loop_label: &str,
) -> Option<(String, String)> {
let line = self.span_line(span);
let start_id = self.next_id();
let start_node_id = self.add_node(DagNode {
id: start_id.clone(),
node_type: DagNodeType::LoopStart { iter_source: iter_source.to_string() },
label: "for".to_string(),
label: loop_label.to_string(),
line,
});
@@ -526,12 +601,11 @@ impl TsWacWalker {
}
fn walk_while(&mut self, while_stmt: &WhileStmt) -> Option<(String, String)> {
if self.stmt_contains_step(&while_stmt.body) {
self.errors.push(validation::error_step_in_while(
self.span_line(while_stmt.span),
));
if !self.stmt_contains_step(&while_stmt.body) {
return None;
}
None
let condition = self.expr_to_source(&while_stmt.test);
self.walk_loop_body_with_iter(&while_stmt.body, while_stmt.span, &condition, "while")
}
fn walk_try(&mut self, try_stmt: &TryStmt) -> Option<(String, String)> {
@@ -545,12 +619,62 @@ impl TsWacWalker {
.as_ref()
.map_or(false, |f| self.body_contains_step(&f.stmts));
if has_steps {
self.errors.push(validation::error_step_in_catch(
self.span_line(try_stmt.span),
));
if !has_steps {
return None;
}
None
let line = self.span_line(try_stmt.span);
let branch_id = self.next_id();
let branch_node_id = self.add_node(DagNode {
id: branch_id.clone(),
node_type: DagNodeType::Branch { condition_source: "try/catch".to_string() },
label: "try".to_string(),
line,
});
let mut last_ids = Vec::new();
// Try body
if let Some((try_first, try_last)) = self.walk_body(&try_stmt.block.stmts) {
self.add_edge(&branch_node_id, &try_first, Some("try".to_string()));
last_ids.push(try_last);
} else {
last_ids.push(branch_node_id.clone());
}
// Catch body
if let Some(handler) = &try_stmt.handler {
if let Some((catch_first, catch_last)) = self.walk_body(&handler.body.stmts) {
self.add_edge(&branch_node_id, &catch_first, Some("catch".to_string()));
last_ids.push(catch_last);
}
}
// Finally body — sequential after merge
let merge_last = if last_ids.len() == 1 {
last_ids.into_iter().next().unwrap()
} else {
let merge_id = format!("{branch_id}_merge");
let merge_node_id = self.add_node(DagNode {
id: merge_id,
node_type: DagNodeType::Merge,
label: "merge".to_string(),
line,
});
for last in last_ids {
self.add_edge(&last, &merge_node_id, None);
}
merge_node_id
};
if let Some(finalizer) = &try_stmt.finalizer {
if let Some((finally_first, finally_last)) = self.walk_body(&finalizer.stmts) {
self.add_edge(&merge_last, &finally_first, None);
return Some((branch_node_id, finally_last));
}
}
Some((branch_node_id, merge_last))
}
fn walk_return(&mut self, ret: &ReturnStmt) -> Option<(String, String)> {
@@ -632,6 +756,19 @@ pub fn parse_ts_workflow(code: &str) -> Result<WorkflowDag, Vec<CompileError>> {
}
}
}
// export const main = workflow(async (...) => { ... })
if let ModuleItem::ModuleDecl(ModuleDecl::ExportDecl(export)) = item {
if let Decl::Var(var_decl) = &export.decl {
for decl in &var_decl.decls {
if let Some(init) = &decl.init {
if let Some(result) = find_workflow_call(init, &cm) {
workflow_body = Some(result);
break;
}
}
}
}
}
}
let (stmts, params) = workflow_body.ok_or_else(|| {

View File

@@ -12,23 +12,6 @@ impl std::fmt::Display for CompileError {
}
}
pub fn error_step_in_try(line: usize) -> CompileError {
CompileError {
message:
"Task calls inside try/except are not allowed. Steps have built-in error handling."
.to_string(),
line,
}
}
pub fn error_step_in_while(line: usize) -> CompileError {
CompileError {
message: "Task calls inside while loops are not allowed. Use for loops instead."
.to_string(),
line,
}
}
pub fn error_step_in_nested_function(line: usize) -> CompileError {
CompileError {
message: "Task calls inside nested functions, closures, or lambdas are not allowed."
@@ -53,12 +36,3 @@ pub fn error_missing_await(line: usize) -> CompileError {
line,
}
}
pub fn error_step_in_catch(line: usize) -> CompileError {
CompileError {
message:
"Task calls inside catch blocks are not allowed. Steps have built-in error handling."
.to_string(),
line,
}
}

View File

@@ -147,7 +147,7 @@ async def my_etl(items: list):
}
#[test]
fn test_reject_step_in_try() {
fn test_step_in_try_except() {
let code = r#"
import asyncio
from wmill import workflow, task
@@ -155,39 +155,52 @@ from wmill import workflow, task
@task
async def extract_data(): ...
@task
async def handle_error(): ...
@workflow
async def my_etl():
try:
await extract_data()
except Exception:
pass
await handle_error()
"#;
let result = parse_python_workflow(code);
assert!(result.is_err());
let errors = result.unwrap_err();
assert!(errors[0].message.contains("try/except"));
let dag = parse_python_workflow(code).expect("should parse try/except");
// Branch(try/except), extract_data, handle_error, merge = 4
assert_eq!(dag.nodes.len(), 4);
assert!(matches!(dag.nodes[0].node_type, DagNodeType::Branch { .. }));
assert_eq!(dag.nodes[0].label, "try");
assert!(matches!(dag.nodes[1].node_type, DagNodeType::Step { .. }));
assert!(matches!(dag.nodes[2].node_type, DagNodeType::Step { .. }));
assert!(matches!(dag.nodes[3].node_type, DagNodeType::Merge));
}
#[test]
fn test_reject_step_in_while() {
fn test_step_in_while() {
let code = r#"
import asyncio
from wmill import workflow, task
@task
async def extract_data(): ...
async def poll_status(): ...
@workflow
async def my_etl():
while True:
await extract_data()
await poll_status()
"#;
let result = parse_python_workflow(code);
assert!(result.is_err());
let errors = result.unwrap_err();
assert!(errors[0].message.contains("while"));
let dag = parse_python_workflow(code).expect("should parse while loop");
// LoopStart, poll_status, LoopEnd = 3
assert_eq!(dag.nodes.len(), 3);
assert!(matches!(
dag.nodes[0].node_type,
DagNodeType::LoopStart { .. }
));
assert_eq!(dag.nodes[0].label, "while");
assert!(matches!(dag.nodes[1].node_type, DagNodeType::Step { .. }));
assert!(matches!(dag.nodes[2].node_type, DagNodeType::LoopEnd));
}
#[test]
@@ -264,3 +277,89 @@ async def my_wf(x: int):
_ => panic!("expected Step node"),
}
}
#[test]
fn test_task_script_and_task_flow_py() {
let code = r#"
from wmill import workflow, task, task_script, task_flow
helper = task_script("./helper.py")
pipeline = task_flow("f/etl/pipeline")
@task()
async def process(x: str) -> str:
return f"processed: {x}"
@workflow
async def main(x: str):
a = await process(x=x)
b = await helper(a=a)
c = await pipeline(b=b)
return {"a": a, "b": b, "c": c}
"#;
let dag = parse_python_workflow(code).expect("should parse");
assert_eq!(dag.nodes.len(), 4); // 3 steps + 1 return
match &dag.nodes[1].node_type {
DagNodeType::Step { name, script } => {
assert_eq!(name, "helper");
assert_eq!(script, "./helper.py");
}
_ => panic!("expected Step node for task_script"),
}
match &dag.nodes[2].node_type {
DagNodeType::Step { name, script } => {
assert_eq!(name, "pipeline");
assert_eq!(script, "f/etl/pipeline");
}
_ => panic!("expected Step node for task_flow"),
}
}
#[test]
fn test_full_template_with_sdk_calls_py() {
let code = r#"
from wmill import workflow, task, task_script, step, sleep, wait_for_approval, get_resume_urls
helper = task_script("./helper.py")
@task()
async def process(x: str) -> str:
return f"processed: {x}"
@workflow
async def main(x: str):
a = await process(x=x)
b = await helper(a=a)
urls = await step("get_urls", lambda: get_resume_urls())
await sleep(1)
approval = await wait_for_approval(timeout=3600)
return {"processed": a, "helper_result": b, "approval": approval}
"#;
let dag = parse_python_workflow(code).expect("should parse");
// process, helper, step("get_urls"), sleep(1), wait_for_approval, return = 6
assert_eq!(dag.nodes.len(), 6);
match &dag.nodes[2].node_type {
DagNodeType::InlineStep { name } => {
assert_eq!(name, "get_urls");
}
_ => panic!("expected InlineStep node, got {:?}", dag.nodes[2].node_type),
}
match &dag.nodes[3].node_type {
DagNodeType::Sleep { seconds } => {
assert_eq!(seconds, "1");
}
_ => panic!("expected Sleep node, got {:?}", dag.nodes[3].node_type),
}
assert!(matches!(
dag.nodes[4].node_type,
DagNodeType::WaitForApproval
));
assert!(matches!(dag.nodes[5].node_type, DagNodeType::Return));
}

View File

@@ -129,45 +129,56 @@ export default workflow(async (items: string[]) => {
}
#[test]
fn test_reject_step_in_try_catch() {
fn test_step_in_try_catch() {
let code = r#"
import { workflow, task } from "windmill-client";
const extract_data = task(async () => {});
const handle_error = task(async (e: any) => {});
export default workflow(async () => {
try {
await extract_data();
} catch (e) {
console.log(e);
await handle_error(e);
}
});
"#;
let result = parse_ts_workflow(code);
assert!(result.is_err());
let errors = result.unwrap_err();
assert!(errors[0].message.contains("catch"));
let dag = parse_ts_workflow(code).expect("should parse try/catch");
// Branch(try/catch), extract_data, handle_error, merge = 4
assert_eq!(dag.nodes.len(), 4);
assert!(matches!(dag.nodes[0].node_type, DagNodeType::Branch { .. }));
assert_eq!(dag.nodes[0].label, "try");
assert!(matches!(dag.nodes[1].node_type, DagNodeType::Step { .. }));
assert!(matches!(dag.nodes[2].node_type, DagNodeType::Step { .. }));
assert!(matches!(dag.nodes[3].node_type, DagNodeType::Merge));
}
#[test]
fn test_reject_step_in_while_ts() {
fn test_step_in_while_ts() {
let code = r#"
import { workflow, task } from "windmill-client";
const extract_data = task(async () => {});
const poll_status = task(async () => {});
export default workflow(async () => {
while (true) {
await extract_data();
await poll_status();
}
});
"#;
let result = parse_ts_workflow(code);
assert!(result.is_err());
let errors = result.unwrap_err();
assert!(errors[0].message.contains("while"));
let dag = parse_ts_workflow(code).expect("should parse while loop");
// LoopStart, poll_status, LoopEnd = 3
assert_eq!(dag.nodes.len(), 3);
assert!(matches!(
dag.nodes[0].node_type,
DagNodeType::LoopStart { .. }
));
assert_eq!(dag.nodes[0].label, "while");
assert!(matches!(dag.nodes[1].node_type, DagNodeType::Step { .. }));
assert!(matches!(dag.nodes[2].node_type, DagNodeType::LoopEnd));
}
#[test]
@@ -243,3 +254,158 @@ export default workflow(async (x: number) => {
_ => panic!("expected Step node"),
}
}
#[test]
fn test_task_script_and_task_flow() {
let code = r#"
import { workflow, task, taskScript, taskFlow } from "windmill-client";
const helper = taskScript("./helper.ts");
const pipeline = taskFlow("f/etl/pipeline");
const process = task(async (x: string) => {});
export default workflow(async (x: string) => {
const a = await process(x);
const b = await helper({ a });
const c = await pipeline({ b });
return { a, b, c };
});
"#;
let dag = parse_ts_workflow(code).expect("should parse");
assert_eq!(dag.nodes.len(), 4); // 3 steps + 1 return
match &dag.nodes[0].node_type {
DagNodeType::Step { name, script } => {
assert_eq!(name, "process");
assert_eq!(script, "process");
}
_ => panic!("expected Step node"),
}
match &dag.nodes[1].node_type {
DagNodeType::Step { name, script } => {
assert_eq!(name, "helper");
assert_eq!(script, "./helper.ts");
}
_ => panic!("expected Step node for taskScript"),
}
match &dag.nodes[2].node_type {
DagNodeType::Step { name, script } => {
assert_eq!(name, "pipeline");
assert_eq!(script, "f/etl/pipeline");
}
_ => panic!("expected Step node for taskFlow"),
}
}
#[test]
fn test_full_template_with_sdk_calls() {
let code = r#"
import { task, taskScript, step, sleep, waitForApproval, getResumeUrls, workflow } from "windmill-client";
const helper = taskScript("./helper.ts");
const process = task(async (x: string): Promise<string> => {
return `processed: ${x}`;
});
export const main = workflow(async (x: string) => {
const a = await process(x);
const b = await helper({ a });
const urls = await step("get_urls", () => getResumeUrls());
await sleep(1);
const approval = await waitForApproval({ timeout: 3600 });
return { processed: a, helper_result: b, approval };
});
"#;
let dag = parse_ts_workflow(code).expect("should parse");
// process, helper, step("get_urls"), sleep(1), waitForApproval, return = 6
assert_eq!(dag.nodes.len(), 6);
assert_eq!(dag.edges.len(), 5);
assert!(matches!(dag.nodes[0].node_type, DagNodeType::Step { .. }));
match &dag.nodes[1].node_type {
DagNodeType::Step { name, script } => {
assert_eq!(name, "helper");
assert_eq!(script, "./helper.ts");
}
_ => panic!("expected Step node"),
}
match &dag.nodes[2].node_type {
DagNodeType::InlineStep { name } => {
assert_eq!(name, "get_urls");
}
_ => panic!("expected InlineStep node, got {:?}", dag.nodes[2].node_type),
}
match &dag.nodes[3].node_type {
DagNodeType::Sleep { seconds } => {
assert_eq!(seconds, "1");
}
_ => panic!("expected Sleep node, got {:?}", dag.nodes[3].node_type),
}
assert!(matches!(
dag.nodes[4].node_type,
DagNodeType::WaitForApproval
));
assert!(matches!(dag.nodes[5].node_type, DagNodeType::Return));
}
#[test]
fn test_complex_mixed_workflow() {
let code = r#"
import { workflow, task, step, sleep } from "windmill-client";
const validate = task(async (data: any) => {});
const process_csv = task(async (data: any) => {});
const process_json = task(async (data: any) => {});
const enrich = task(async (item: any) => {});
const store = task(async (data: any) => {});
export default workflow(async (data: any) => {
const validated = await validate(data);
if (validated.format === "csv") {
const parsed = await process_csv(validated);
for (const row of parsed.rows) {
await enrich(row);
}
} else {
await process_json(validated);
}
await sleep(5);
const ts = await step("timestamp", () => new Date().toISOString());
await store(validated);
return { done: true };
});
"#;
let dag = parse_ts_workflow(code).expect("should parse");
// validate, Branch, process_csv, LoopStart, enrich, LoopEnd, process_json,
// merge, sleep(5), step("timestamp"), store, return = 12
assert_eq!(dag.nodes.len(), 12);
assert!(matches!(dag.nodes[0].node_type, DagNodeType::Step { .. }));
assert!(matches!(dag.nodes[1].node_type, DagNodeType::Branch { .. }));
assert!(matches!(dag.nodes[2].node_type, DagNodeType::Step { .. })); // process_csv
assert!(matches!(
dag.nodes[3].node_type,
DagNodeType::LoopStart { .. }
));
assert!(matches!(dag.nodes[4].node_type, DagNodeType::Step { .. })); // enrich
assert!(matches!(dag.nodes[5].node_type, DagNodeType::LoopEnd));
assert!(matches!(dag.nodes[6].node_type, DagNodeType::Step { .. })); // process_json
assert!(matches!(dag.nodes[7].node_type, DagNodeType::Merge));
assert!(matches!(dag.nodes[8].node_type, DagNodeType::Sleep { .. }));
assert!(matches!(
dag.nodes[9].node_type,
DagNodeType::InlineStep { .. }
)); // timestamp
assert!(matches!(dag.nodes[10].node_type, DagNodeType::Step { .. })); // store
assert!(matches!(dag.nodes[11].node_type, DagNodeType::Return));
}

View File

@@ -38,6 +38,7 @@ csharp-parser = [ "dep:windmill-parser-csharp"]
nu-parser = [ "dep:windmill-parser-nu"]
java-parser = [ "dep:windmill-parser-java"]
ruby-parser = [ "dep:windmill-parser-ruby"]
r-parser = [ "dep:windmill-parser-r"]
wac-parser = [ "dep:windmill-parser-wac"]
asset-parser = [ "dep:windmill-parser-ts-asset", "dep:windmill-parser-py-asset", "dep:windmill-parser-sql-asset"]
py-imports-parser = [ "dep:windmill-parser-py-imports"]
@@ -58,6 +59,7 @@ windmill-parser-csharp = { workspace = true, optional = true }
windmill-parser-nu = { workspace = true, optional = true }
windmill-parser-java = { workspace = true, optional = true }
windmill-parser-ruby = { workspace = true, optional = true }
windmill-parser-r = { workspace = true, optional = true }
windmill-parser-wac = { workspace = true, optional = true }
windmill-parser-ts-asset = { workspace = true, optional = true }
windmill-parser-py-asset = { workspace = true, optional = true }

View File

@@ -55,6 +55,11 @@ const targets = [
desc: "Ruby",
features: "ruby-parser",
env: "tree-sitter",
}, {
ident: "r",
desc: "R",
features: "r-parser",
env: "tree-sitter",
},
{
ident: "wac",

View File

@@ -39,3 +39,6 @@ popd
pushd "pkg-py-imports" && npm publish ${args}
popd
pushd "pkg-wac" && npm publish ${args}
popd

View File

@@ -198,6 +198,12 @@ pub fn parse_ruby(code: &str) -> String {
wrap_sig(windmill_parser_ruby::parse_ruby_signature(code))
}
#[cfg(feature = "r-parser")]
#[wasm_bindgen]
pub fn parse_r(code: &str) -> String {
wrap_sig(windmill_parser_r::parse_r_signature(code))
}
#[cfg(feature = "asset-parser")]
#[wasm_bindgen]
pub fn parse_assets_sql(code: &str) -> String {

View File

@@ -1,5 +1,7 @@
#pragma once
#include <stdint.h>
void *memcpy(void *dest, const void *src, unsigned long n);
void *memmove(void *dest, const void *src, unsigned long n);
void *memset(void *s, int c, unsigned long n);

View File

@@ -51,10 +51,11 @@ use windmill_common::{
MONITOR_LOGS_ON_OBJECT_STORE_SETTING, NO_DEFAULT_MAVEN_SETTING,
NPM_CONFIG_REGISTRY_SETTING, NUGET_CONFIG_SETTING, OAUTH_SETTING, OTEL_SETTING,
OTEL_TRACING_PROXY_SETTING, PIP_INDEX_URL_SETTING, POWERSHELL_REPO_PAT_SETTING,
POWERSHELL_REPO_URL_SETTING, REQUEST_SIZE_LIMIT_SETTING,
REQUIRE_PREEXISTING_USER_FOR_OAUTH_SETTING, RETENTION_PERIOD_SECS_SETTING,
RUBY_REPOS_SETTING, SAML_METADATA_SETTING, SCIM_TOKEN_SETTING, SMTP_SETTING, TEAMS_SETTING,
TIMEOUT_WAIT_RESULT_SETTING, UV_INDEX_STRATEGY_SETTING, WORKSPACE_REGISTRIES_SETTING,
POWERSHELL_REPO_URL_SETTING, PREVIEW_TAGS_OVERRIDE_SETTING, REQUEST_SIZE_LIMIT_SETTING,
REQUIRE_PREEXISTING_USER_FOR_OAUTH_SETTING, RESTART_COORDINATION_SETTING,
RETENTION_PERIOD_SECS_SETTING, RUBY_REPOS_SETTING, SAML_METADATA_SETTING,
SCIM_TOKEN_SETTING, SMTP_SETTING, TEAMS_SETTING, TIMEOUT_WAIT_RESULT_SETTING,
UV_INDEX_STRATEGY_SETTING, WORKSPACE_REGISTRIES_SETTING,
},
scripts::ScriptLang,
stats_oss::schedule_stats,
@@ -67,7 +68,7 @@ use windmill_common::{
is_native_mode_from_env, reload_custom_tags_setting, Connection, HUB_CACHE_DIR,
HUB_RT_CACHE_DIR, NATIVE_MODE_RESOLVED, TMP_LOGS_DIR, WINDMILL_DIR, WORKER_GROUP,
},
KillpillSender, DEFAULT_HUB_BASE_URL, METRICS_ENABLED,
KillpillSender, DEFAULT_HUB_BASE_URL, INSTANCE_NAME, METRICS_ENABLED,
};
#[cfg(feature = "enterprise")]
@@ -94,20 +95,21 @@ use windmill_worker::{
BUN_BUNDLE_CACHE_DIR, BUN_CACHE_DIR, CSHARP_CACHE_DIR, DENO_CACHE_DIR, DENO_CACHE_DIR_DEPS,
DENO_CACHE_DIR_NPM, GO_BIN_CACHE_DIR, GO_CACHE_DIR, JAVA_CACHE_DIR, NU_CACHE_DIR,
POWERSHELL_CACHE_DIR, PY310_CACHE_DIR, PY311_CACHE_DIR, PY312_CACHE_DIR, PY313_CACHE_DIR,
RUBY_CACHE_DIR, RUST_CACHE_DIR, TAR_JAVA_CACHE_DIR, UV_CACHE_DIR,
RUBY_CACHE_DIR, RUST_CACHE_DIR, R_CACHE_DIR, TAR_JAVA_CACHE_DIR, UV_CACHE_DIR,
};
use crate::monitor::{
initial_load, load_keep_job_dir, load_metrics_debug_enabled, load_require_preexisting_user,
load_tag_per_workspace_enabled, load_tag_per_workspace_workspaces, monitor_db,
reload_app_workspaced_route_setting, reload_audit_log_retention_days_setting,
reload_base_url_setting, reload_bunfig_install_scopes_setting,
reload_critical_alert_mute_ui_setting, reload_critical_alerts_on_token_expiry_setting,
reload_critical_error_channels_setting, reload_extra_pip_index_url_setting,
reload_http_route_workspaced_route_setting, reload_hub_api_secret_setting,
reload_hub_base_url_setting, reload_instance_events_webhook_setting,
reload_job_default_timeout_setting, reload_job_isolation_setting, reload_jwt_secret_setting,
reload_license_key, reload_npm_config_registry_setting, reload_otel_tracing_proxy_setting,
initial_load, load_keep_job_dir, load_metrics_debug_enabled, load_preview_tags_override,
load_require_preexisting_user, load_tag_per_workspace_enabled,
load_tag_per_workspace_workspaces, monitor_db, reload_app_workspaced_route_setting,
reload_audit_log_retention_days_setting, reload_base_url_setting,
reload_bunfig_install_scopes_setting, reload_critical_alert_mute_ui_setting,
reload_critical_alerts_on_token_expiry_setting, reload_critical_error_channels_setting,
reload_extra_pip_index_url_setting, reload_http_route_workspaced_route_setting,
reload_hub_api_secret_setting, reload_hub_base_url_setting,
reload_instance_events_webhook_setting, reload_job_default_timeout_setting,
reload_job_isolation_setting, reload_jwt_secret_setting, reload_license_key,
reload_npm_config_registry_setting, reload_otel_tracing_proxy_setting,
reload_pip_index_url_setting, reload_retention_period_setting, reload_scim_token_setting,
reload_smtp_config, reload_uv_index_strategy_setting, reload_worker_config, MonitorIteration,
};
@@ -1099,6 +1101,9 @@ Windmill Community Edition {GIT_VERSION}
}
let addr = SocketAddr::from((server_bind_address, port));
let listener = tokio::net::TcpListener::bind(addr)
.await
.context("binding main windmill server")?;
let (base_internal_tx, base_internal_rx) = tokio::sync::oneshot::channel::<String>();
@@ -1232,7 +1237,7 @@ Windmill Community Edition {GIT_VERSION}
db.clone(),
index_reader,
log_index_reader,
addr,
listener,
server_killpill_rx,
base_internal_tx,
server_mode,
@@ -1738,6 +1743,11 @@ async fn process_notify_event(
);
}
}
PREVIEW_TAGS_OVERRIDE_SETTING => {
if let Err(e) = load_preview_tags_override(db).await {
tracing::error!("Error loading preview tags override: {e:#}");
}
}
SMTP_SETTING => {
reload_smtp_config(db).await;
}
@@ -1788,7 +1798,8 @@ async fn process_notify_event(
reload_otel_tracing_proxy_setting(conn).await;
if worker_mode {
tracing::info!("OTEL tracing proxy setting changed, restarting worker");
send_delayed_killpill(tx, 4, "OTEL tracing proxy setting change").await;
spawn_graceful_killpill(tx, db, 10, "OTEL tracing proxy setting change")
.await;
}
}
REQUIRE_PREEXISTING_USER_FOR_OAUTH_SETTING => {
@@ -1796,12 +1807,12 @@ async fn process_notify_event(
}
EXPOSE_METRICS_SETTING => {
tracing::info!("Metrics setting changed, restarting");
send_delayed_killpill(tx, 40, "metrics setting change").await;
spawn_graceful_killpill(tx, db, 10, "metrics setting change").await;
}
EMAIL_DOMAIN_SETTING => {
tracing::info!("Email domain setting changed");
if server_mode {
send_delayed_killpill(tx, 4, "email domain setting change").await;
spawn_graceful_killpill(tx, db, 10, "email domain setting change").await;
}
}
EXPOSE_DEBUG_METRICS_SETTING => {
@@ -1837,19 +1848,19 @@ async fn process_notify_event(
}
OTEL_SETTING => {
tracing::info!("OTEL setting changed, restarting");
send_delayed_killpill(tx, 4, "OTEL setting change").await;
spawn_graceful_killpill(tx, db, 10, "OTEL setting change").await;
}
REQUEST_SIZE_LIMIT_SETTING => {
if server_mode {
tracing::info!("Request limit size change detected, killing server expecting to be restarted");
send_delayed_killpill(tx, 4, "request size limit change").await;
spawn_graceful_killpill(tx, db, 10, "request size limit change").await;
}
}
SAML_METADATA_SETTING => {
tracing::info!(
"SAML metadata change detected, killing server expecting to be restarted"
);
send_delayed_killpill(tx, 0, "SAML metadata change").await;
spawn_graceful_killpill(tx, db, 10, "SAML metadata change").await;
}
HUB_BASE_URL_SETTING => {
if let Err(e) = reload_hub_base_url_setting(conn, server_mode).await {
@@ -1898,6 +1909,24 @@ async fn process_notify_event(
.unwrap_or(false);
tracing::info!("Workspace telemetry setting changed: enabled={}", enabled);
}
RESTART_COORDINATION_SETTING => {
// Internal coordination key for staggered restarts, no action needed
}
"plain_emails_telemetry" => {
let enabled = sqlx::query_scalar!(
"SELECT value FROM global_settings WHERE name = 'plain_emails_telemetry'"
)
.fetch_optional(db)
.await
.ok()
.flatten()
.and_then(|v| v.as_bool())
.unwrap_or(false);
tracing::info!(
"Plain emails telemetry setting changed: enabled={}",
enabled
);
}
_ => {
tracing::info!("Unrecognized Global Setting Change Payload: {:?}", payload);
}
@@ -1982,6 +2011,7 @@ pub async fn run_workers(
&*POWERSHELL_CACHE_DIR,
&*JAVA_CACHE_DIR,
&*RUBY_CACHE_DIR,
&*R_CACHE_DIR,
&*TAR_JAVA_CACHE_DIR, // for related places search: ADD_NEW_LANG
] {
DirBuilder::new()
@@ -2039,14 +2069,145 @@ pub async fn run_workers(
Ok(())
}
async fn send_delayed_killpill(tx: &KillpillSender, mut max_delay_secs: u64, context: &str) {
if max_delay_secs == 0 {
max_delay_secs = 1;
}
// Random delay to avoid all servers/workers shutting down simultaneously
let rd_delay = rand::rng().random_range(0..max_delay_secs);
tracing::info!("Scheduling {context} shutdown in {rd_delay}s");
tokio::time::sleep(Duration::from_secs(rd_delay)).await;
/// Schedule a graceful restart with DB-coordinated staggering.
///
/// Uses a PostgreSQL advisory lock to serialize restart scheduling across server instances.
/// Each instance records its planned restart time in the `_restart_coordination` global setting;
/// subsequent instances read existing schedules and shift their restart to maintain at least
/// `safety_margin_secs` between consecutive restarts (must exceed the server startup time).
///
/// Every server waits at least `DRAIN_DELAY_SECS` to let in-flight requests complete.
/// Each subsequent server waits an additional `safety_margin_secs` after the previous one,
/// guaranteeing zero downtime overlap.
///
/// The DB coordination is done synchronously (fast, ~ms) to reserve our restart slot,
/// then the sleep+kill is spawned in the background so the notification handler is not blocked.
///
/// Falls back to drain-only delay if DB coordination fails.
async fn spawn_graceful_killpill(
tx: &KillpillSender,
db: &Pool<Postgres>,
safety_margin_secs: u64,
context: &str,
) {
// Minimum delay before any restart to let in-flight requests drain
const DRAIN_DELAY_SECS: u64 = 3;
tx.send();
let delay = match coordinate_restart_delay(db, safety_margin_secs, DRAIN_DELAY_SECS).await {
Ok(d) => d,
Err(e) => {
tracing::warn!(
"Failed to coordinate restart for {context}: {e:#}, \
falling back to drain delay of {DRAIN_DELAY_SECS}s"
);
DRAIN_DELAY_SECS
}
};
tracing::info!("Scheduling {context} graceful shutdown in {delay}s");
let tx = tx.clone();
tokio::spawn(async move {
tokio::time::sleep(Duration::from_secs(delay)).await;
tx.send();
});
}
/// Coordinate a restart delay with other instances via the DB.
///
/// Returns the delay (in seconds from now) at which this instance should restart.
/// The first server gets `drain_delay_secs` (to let in-flight requests complete).
/// Each subsequent server is spaced `safety_margin_secs` after the latest scheduled restart.
async fn coordinate_restart_delay(
db: &Pool<Postgres>,
safety_margin_secs: u64,
drain_delay_secs: u64,
) -> anyhow::Result<u64> {
const RESTART_LOCK_ID: i64 = 737_483_920;
// Stale threshold: ignore coordination entries older than this
const STALE_THRESHOLD_SECS: i64 = 120;
let now = chrono::Utc::now();
let mut tx = db.begin().await.context("begin restart coordination tx")?;
// Serialize access across all instances
sqlx::query("SELECT pg_advisory_xact_lock($1)")
.bind(RESTART_LOCK_ID)
.execute(&mut *tx)
.await
.context("acquire restart coordination lock")?;
// Read existing coordination record
let existing: Option<serde_json::Value> =
sqlx::query_scalar("SELECT value FROM global_settings WHERE name = $1")
.bind(RESTART_COORDINATION_SETTING)
.fetch_optional(&mut *tx)
.await
.context("read restart coordination")?;
// Parse existing scheduled restarts, filtering out stale entries
// Each entry is (instance_name, restart_at)
let mut scheduled: Vec<(String, chrono::DateTime<chrono::Utc>)> = Vec::new();
if let Some(val) = &existing {
if let Some(arr) = val.get("restarts").and_then(|v| v.as_array()) {
for entry in arr {
let instance = entry
.get("instance")
.and_then(|v| v.as_str())
.unwrap_or("unknown")
.to_string();
if let Some(ts_str) = entry.get("restart_at").and_then(|v| v.as_str()) {
if let Ok(dt) = chrono::DateTime::parse_from_rfc3339(ts_str) {
let dt = dt.with_timezone(&chrono::Utc);
let stale_cutoff = now - chrono::Duration::seconds(STALE_THRESHOLD_SECS);
if dt > stale_cutoff {
scheduled.push((instance, dt));
}
}
}
}
}
}
// Find the latest scheduled restart
let latest = scheduled.iter().map(|(_, dt)| *dt).max();
let earliest_allowed = now + chrono::Duration::seconds(drain_delay_secs as i64);
// Our restart time: drain_delay from now, or safety_margin after the latest existing restart
let our_restart = match latest {
Some(last) => {
let after_last = last + chrono::Duration::seconds(safety_margin_secs as i64);
// Use whichever is later: drain delay or staggered position
earliest_allowed.max(after_last)
}
None => earliest_allowed,
};
// Record our restart time (deduplicate: remove any prior entry for this instance)
scheduled.retain(|(inst, _)| inst != &*INSTANCE_NAME);
scheduled.push((INSTANCE_NAME.clone(), our_restart));
let new_value = serde_json::json!({
"restarts": scheduled.iter().map(|(inst, dt)| {
serde_json::json!({
"instance": inst,
"restart_at": dt.to_rfc3339()
})
}).collect::<Vec<_>>()
});
sqlx::query(
"INSERT INTO global_settings (name, value, updated_at) \
VALUES ($1, $2, now()) \
ON CONFLICT (name) DO UPDATE SET value = $2, updated_at = now()",
)
.bind(RESTART_COORDINATION_SETTING)
.bind(&new_value)
.execute(&mut *tx)
.await
.context("write restart coordination")?;
tx.commit().await.context("commit restart coordination")?;
let delay = (our_restart - now).num_seconds().max(0) as u64;
Ok(delay)
}

View File

@@ -62,7 +62,7 @@ use windmill_common::{
KEEP_JOB_DIR_SETTING, LICENSE_KEY_SETTING, MONITOR_LOGS_ON_OBJECT_STORE_SETTING,
NPMRC_SETTING, NPM_CONFIG_REGISTRY_SETTING, NUGET_CONFIG_SETTING, OTEL_SETTING,
OTEL_TRACING_PROXY_SETTING, PIP_INDEX_URL_SETTING, POWERSHELL_REPO_PAT_SETTING,
POWERSHELL_REPO_URL_SETTING, REQUEST_SIZE_LIMIT_SETTING,
POWERSHELL_REPO_URL_SETTING, PREVIEW_TAGS_OVERRIDE_SETTING, REQUEST_SIZE_LIMIT_SETTING,
REQUIRE_PREEXISTING_USER_FOR_OAUTH_SETTING, RETENTION_PERIOD_SECS_SETTING,
SAML_METADATA_SETTING, SCIM_TOKEN_SETTING, TIMEOUT_WAIT_RESULT_SETTING,
UV_INDEX_STRATEGY_SETTING,
@@ -79,8 +79,8 @@ use windmill_common::{
load_periodic_bash_script_interval_from_env, load_whitelist_env_vars_from_env,
load_worker_config, reload_custom_tags_setting, store_pull_query,
store_suspended_pull_query, Connection, WorkerConfig, DEFAULT_TAGS_PER_WORKSPACE,
DEFAULT_TAGS_WORKSPACES, INDEXER_CONFIG, SCRIPT_TOKEN_EXPIRY, SMTP_CONFIG, WINDMILL_DIR,
WORKER_CONFIG, WORKER_GROUP,
DEFAULT_TAGS_WORKSPACES, INDEXER_CONFIG, PREVIEW_TAGS_OVERRIDE, SCRIPT_TOKEN_EXPIRY,
SMTP_CONFIG, WINDMILL_DIR, WORKER_CONFIG, WORKER_GROUP,
},
KillpillSender, AUDIT_LOG_RETENTION_DAYS, BASE_URL, CRITICAL_ALERTS_ON_DB_OVERSIZE,
CRITICAL_ALERTS_ON_TOKEN_EXPIRY, CRITICAL_ALERT_MUTE_UI_ENABLED, CRITICAL_ERROR_CHANNELS, DB,
@@ -169,6 +169,8 @@ lazy_static::lazy_static! {
static ref QUEUE_COUNT_TAGS: Arc<RwLock<Vec<String>>> = Arc::new(RwLock::new(Vec::new()));
static ref QUEUE_RUNNING_COUNT_TAGS: Arc<RwLock<Vec<String>>> = Arc::new(RwLock::new(Vec::new()));
static ref OTEL_QUEUE_COUNT_TAGS: Arc<RwLock<Vec<String>>> = Arc::new(RwLock::new(Vec::new()));
static ref OTEL_QUEUE_RUNNING_COUNT_TAGS: Arc<RwLock<Vec<String>>> = Arc::new(RwLock::new(Vec::new()));
static ref DISABLE_CONCURRENCY_LIMIT: bool = std::env::var("DISABLE_CONCURRENCY_LIMIT").is_ok_and(|s| s == "true");
//legacy typo
@@ -233,6 +235,10 @@ pub async fn initial_load(
if let Err(e) = load_tag_per_workspace_workspaces(db).await {
tracing::error!("Error loading default tag per workpsace workspaces: {e:#}");
}
if let Err(e) = load_preview_tags_override(db).await {
tracing::error!("Error loading preview tags override: {e:#}");
}
}
if server_mode {
@@ -497,6 +503,16 @@ pub async fn load_tag_per_workspace_workspaces(db: &DB) -> error::Result<()> {
Ok(())
}
pub async fn load_preview_tags_override(db: &DB) -> error::Result<()> {
let value = load_value_from_global_settings(db, PREVIEW_TAGS_OVERRIDE_SETTING).await;
match value {
Ok(Some(serde_json::Value::Bool(t))) => PREVIEW_TAGS_OVERRIDE.store(t, Ordering::Relaxed),
_ => (),
};
Ok(())
}
pub async fn reload_critical_alert_mute_ui_setting(conn: &Connection) -> error::Result<()> {
if let Ok(Some(serde_json::Value::Bool(t))) =
load_value_from_global_settings_with_conn(conn, CRITICAL_ALERT_MUTE_UI_SETTING, true).await
@@ -2372,8 +2388,20 @@ pub async fn expose_queue_metrics(db: &Pool<Postgres>) {
}
}
let otel_enabled = OTEL_METRICS_ENABLED.load(Ordering::Relaxed);
if otel_enabled {
for q in OTEL_QUEUE_COUNT_TAGS.read().await.iter() {
if queue_counts.get(q).is_none() {
otel_set_queue_count(q, 0);
}
}
}
#[allow(unused_mut)]
let mut tags_to_watch = vec![];
#[allow(unused_mut)]
let mut otel_tags_to_watch = vec![];
for q in queue_counts {
let count = q.1;
let tag = q.0;
@@ -2385,6 +2413,9 @@ pub async fn expose_queue_metrics(db: &Pool<Postgres>) {
tags_to_watch.push(tag.to_string());
}
if otel_enabled {
otel_tags_to_watch.push(tag.to_string());
}
otel_set_queue_count(&tag, count as i64);
// save queue_count and delay metrics per tag
@@ -2419,9 +2450,13 @@ pub async fn expose_queue_metrics(db: &Pool<Postgres>) {
let mut w = QUEUE_COUNT_TAGS.write().await;
*w = tags_to_watch;
}
if otel_enabled {
let mut w = OTEL_QUEUE_COUNT_TAGS.write().await;
*w = otel_tags_to_watch;
}
// Single DB query for running counts, shared by Prometheus and OTel
let otel_running = OTEL_METRICS_ENABLED.load(Ordering::Relaxed);
let otel_running = otel_enabled;
#[cfg(feature = "prometheus")]
let need_running_counts = metrics_enabled || otel_running;
#[cfg(not(feature = "prometheus"))]
@@ -2439,8 +2474,18 @@ pub async fn expose_queue_metrics(db: &Pool<Postgres>) {
}
}
if otel_running {
for q in OTEL_QUEUE_RUNNING_COUNT_TAGS.read().await.iter() {
if queue_running_counts.get(q).is_none() {
otel_set_queue_running_count(q, 0);
}
}
}
#[allow(unused_mut, unused_variables)]
let mut running_tags_to_watch: Vec<String> = vec![];
#[allow(unused_mut, unused_variables)]
let mut otel_running_tags_to_watch: Vec<String> = vec![];
for (tag, count) in &queue_running_counts {
#[cfg(feature = "prometheus")]
if metrics_enabled {
@@ -2451,6 +2496,7 @@ pub async fn expose_queue_metrics(db: &Pool<Postgres>) {
if otel_running {
otel_set_queue_running_count(tag, *count as i64);
otel_running_tags_to_watch.push(tag.to_string());
}
}
@@ -2459,6 +2505,10 @@ pub async fn expose_queue_metrics(db: &Pool<Postgres>) {
let mut w = QUEUE_RUNNING_COUNT_TAGS.write().await;
*w = running_tags_to_watch;
}
if otel_running {
let mut w = OTEL_QUEUE_RUNNING_COUNT_TAGS.write().await;
*w = otel_running_tags_to_watch;
}
}
}

View File

@@ -84,7 +84,7 @@ fi
if [ "$REVERT" == "YES" ]; then
backend_dirpath="${root_dirpath}/backend/"
for ce_file in $(find "${root_dirpath}/backend" -name "*_ee.rs"); do
for ce_file in $(find "${root_dirpath}/backend" \( -name "*_ee.rs" -o -name "ee.rs" \)); do
if [ -L "${ce_file}" ]; then
rm "${ce_file}"
echo "Deleted symlink '${ce_file}'"

View File

@@ -26,7 +26,7 @@ native_trigger_service: nextcloud
request_type: sync, async, sync_sse
runnable_type: ScriptHash, ScriptPath, FlowPath
script_kind: script, trigger, failure, command, approval, preprocessor
script_lang: python3, deno, go, bash, postgresql, nativets, bun, mysql, bigquery, snowflake, graphql, powershell, mssql, php, bunnative, rust, ansible, csharp, oracledb, nu, java, duckdb, ruby
script_lang: python3, deno, go, bash, postgresql, nativets, bun, mysql, bigquery, snowflake, graphql, powershell, mssql, php, bunnative, rust, ansible, csharp, oracledb, nu, java, duckdb, ruby, rlang
trigger_kind: webhook, http, websocket, kafka, email, nats, postgres, sqs, mqtt, gcp, default_email, nextcloud
trigger_mode: enabled, disabled, suspended
workspace_key_kind: cloud

504
backend/tests/otel.rs Normal file
View File

@@ -0,0 +1,504 @@
//! E2E tests for OpenTelemetry integration.
//!
//! Verify that metrics are recorded with correct names/values/attributes and
//! spans are created with correct trace IDs, attributes, and status codes.
//!
//! Run with: cargo test --features enterprise,private,otel --test otel -- --test-threads=1
#![cfg(all(feature = "otel", feature = "enterprise"))]
use std::sync::{atomic::Ordering, Arc};
use opentelemetry::global;
use opentelemetry::trace::TracerProvider as _;
use opentelemetry_sdk::{
metrics::{InMemoryMetricExporter, PeriodicReader, SdkMeterProvider},
trace::{InMemorySpanExporter, SdkTracerProvider, SimpleSpanProcessor},
};
use windmill_common::otel_ee::*;
use windmill_common::{OTEL_METRICS_ENABLED, OTEL_TRACING_ENABLED};
// ── Global test infrastructure ──────────────────────────────────────────
struct OtelTestState {
metric_exporter: InMemoryMetricExporter,
span_exporter: InMemorySpanExporter,
meter_provider: SdkMeterProvider,
}
static STATE: tokio::sync::OnceCell<Arc<OtelTestState>> = tokio::sync::OnceCell::const_new();
async fn ensure_setup() -> Arc<OtelTestState> {
STATE
.get_or_init(|| async {
// Metrics: InMemoryMetricExporter + PeriodicReader (needs async tokio context)
let metric_exporter = InMemoryMetricExporter::default();
let reader = PeriodicReader::builder(metric_exporter.clone()).build();
let meter_provider = SdkMeterProvider::builder().with_reader(reader).build();
global::set_meter_provider(meter_provider.clone());
OTEL_METRICS_ENABLED.store(true, Ordering::SeqCst);
// Tracing: InMemorySpanExporter + SimpleSpanProcessor
let span_exporter = InMemorySpanExporter::default();
let tracer_provider = SdkTracerProvider::builder()
.with_span_processor(SimpleSpanProcessor::new(span_exporter.clone()))
.build();
let tracer = tracer_provider.tracer("windmill");
*TRACER.write().unwrap() = Some(tracer);
OTEL_TRACING_ENABLED.store(true, Ordering::SeqCst);
Arc::new(OtelTestState { metric_exporter, span_exporter, meter_provider })
})
.await
.clone()
}
// ── Metric helper: flush + collect ──────────────────────────────────────
fn flush_and_get_metrics(
state: &OtelTestState,
) -> Vec<opentelemetry_sdk::metrics::data::ResourceMetrics> {
state.meter_provider.force_flush().expect("flush failed");
state
.metric_exporter
.get_finished_metrics()
.expect("get_finished_metrics failed")
}
fn find_metric<'a>(
all: &'a [opentelemetry_sdk::metrics::data::ResourceMetrics],
name: &str,
) -> Option<&'a opentelemetry_sdk::metrics::data::Metric> {
all.iter()
.flat_map(|rm| rm.scope_metrics())
.flat_map(|sm| sm.metrics())
.find(|m| m.name() == name)
}
fn metric_names(all: &[opentelemetry_sdk::metrics::data::ResourceMetrics]) -> Vec<String> {
all.iter()
.flat_map(|rm| rm.scope_metrics())
.flat_map(|sm| sm.metrics())
.map(|m| m.name().to_string())
.collect()
}
// ── Counter value helpers ───────────────────────────────────────────────
fn sum_u64_value(metric: &opentelemetry_sdk::metrics::data::Metric) -> Option<u64> {
use opentelemetry_sdk::metrics::data::{AggregatedMetrics, MetricData};
match metric.data() {
AggregatedMetrics::U64(MetricData::Sum(sum)) => {
Some(sum.data_points().map(|dp| dp.value()).sum())
}
_ => None,
}
}
fn gauge_i64_values(
metric: &opentelemetry_sdk::metrics::data::Metric,
) -> Vec<(Vec<opentelemetry::KeyValue>, i64)> {
use opentelemetry_sdk::metrics::data::{AggregatedMetrics, MetricData};
match metric.data() {
AggregatedMetrics::I64(MetricData::Gauge(gauge)) => gauge
.data_points()
.map(|dp| (dp.attributes().cloned().collect(), dp.value()))
.collect(),
_ => panic!("expected I64 Gauge metric"),
}
}
fn gauge_f64_value(metric: &opentelemetry_sdk::metrics::data::Metric) -> Option<f64> {
use opentelemetry_sdk::metrics::data::{AggregatedMetrics, MetricData};
match metric.data() {
AggregatedMetrics::F64(MetricData::Gauge(gauge)) => {
gauge.data_points().next().map(|dp| dp.value())
}
_ => None,
}
}
fn histogram_f64_count(metric: &opentelemetry_sdk::metrics::data::Metric) -> Option<u64> {
use opentelemetry_sdk::metrics::data::{AggregatedMetrics, MetricData};
match metric.data() {
AggregatedMetrics::F64(MetricData::Histogram(hist)) => {
Some(hist.data_points().map(|dp| dp.count()).sum())
}
_ => None,
}
}
fn histogram_f64_sum(metric: &opentelemetry_sdk::metrics::data::Metric) -> Option<f64> {
use opentelemetry_sdk::metrics::data::{AggregatedMetrics, MetricData};
match metric.data() {
AggregatedMetrics::F64(MetricData::Histogram(hist)) => {
Some(hist.data_points().map(|dp| dp.sum()).sum())
}
_ => None,
}
}
// ═══════════════════════════════════════════════════════════════════════
// METRICS E2E TEST
//
// All metric assertions live in one test function because the PeriodicReader's
// background task is tied to the tokio runtime that created it. Separate
// #[tokio::test] functions each get their own runtime, and the reader becomes
// disconnected after the first test's runtime is dropped.
// ═══════════════════════════════════════════════════════════════════════
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_all_metrics_e2e() {
let state = ensure_setup().await;
// ── Counters ────────────────────────────────────────────────────
otel_incr_queue_push_count();
otel_incr_queue_push_count();
otel_incr_queue_push_count();
otel_incr_queue_delete_count();
otel_incr_queue_pull_count();
otel_incr_zombie_restart_count(7);
otel_incr_zombie_delete_count(3);
otel_incr_worker_execution_count("bun");
otel_incr_worker_execution_count("bun");
otel_incr_worker_execution_failed("go");
otel_incr_worker_started();
// ── Gauges ──────────────────────────────────────────────────────
otel_set_queue_count("python3", 42);
otel_set_queue_running_count("deno", 5);
otel_set_worker_busy("worker-test-1", 1);
otel_set_db_pool(5, 10, 20);
otel_set_health_db_latency(2.5);
otel_set_worker_uptime("w-uptime", 3600.0);
otel_set_health_status_phase("healthy");
otel_set_health_db_unresponsive(true);
// ── Histograms ──────────────────────────────────────────────────
otel_record_worker_execution_duration("python3", 1.5);
otel_record_worker_execution_duration("python3", 2.5);
otel_record_worker_pull_duration("w1", true, 0.05);
otel_record_worker_pull_duration("w1", false, 0.01);
// ── Flush and collect ───────────────────────────────────────────
let metrics = flush_and_get_metrics(&state);
let names = metric_names(&metrics);
// ── Verify all 20 metric names are present ──────────────────────
let expected = [
"windmill.queue.push_count",
"windmill.queue.delete_count",
"windmill.queue.pull_count",
"windmill.queue.zombie_restart_count",
"windmill.queue.zombie_delete_count",
"windmill.queue.count",
"windmill.queue.running_count",
"windmill.worker.execution_count",
"windmill.worker.execution_duration",
"windmill.worker.busy",
"windmill.worker.pull_duration",
"windmill.worker.execution_failed",
"windmill.db.pool.active",
"windmill.db.pool.idle",
"windmill.db.pool.max",
"windmill.health.db_latency",
"windmill.worker.started",
"windmill.worker.uptime",
"windmill.health.status",
"windmill.health.db_unresponsive",
];
for name in expected {
assert!(
names.iter().any(|n| n == name),
"metric '{}' not found in {:?}",
name,
names
);
}
// ── Counter values ──────────────────────────────────────────────
let m = find_metric(&metrics, "windmill.queue.push_count").unwrap();
assert!(sum_u64_value(m).unwrap() >= 3, "push_count should be >= 3");
let m = find_metric(&metrics, "windmill.queue.delete_count").unwrap();
assert!(sum_u64_value(m).unwrap() >= 1);
let m = find_metric(&metrics, "windmill.queue.pull_count").unwrap();
assert!(sum_u64_value(m).unwrap() >= 1);
let m = find_metric(&metrics, "windmill.queue.zombie_restart_count").unwrap();
assert!(sum_u64_value(m).unwrap() >= 7);
let m = find_metric(&metrics, "windmill.queue.zombie_delete_count").unwrap();
assert!(sum_u64_value(m).unwrap() >= 3);
let m = find_metric(&metrics, "windmill.worker.execution_count").unwrap();
assert!(sum_u64_value(m).unwrap() >= 2);
let m = find_metric(&metrics, "windmill.worker.execution_failed").unwrap();
assert!(sum_u64_value(m).unwrap() >= 1);
let m = find_metric(&metrics, "windmill.worker.started").unwrap();
assert!(sum_u64_value(m).unwrap() >= 1);
// ── Gauge values ────────────────────────────────────────────────
let m = find_metric(&metrics, "windmill.queue.count").unwrap();
let values = gauge_i64_values(m);
let dp = values
.iter()
.find(|(attrs, _)| {
attrs
.iter()
.any(|kv| kv.key.as_str() == "tag" && kv.value.as_str() == "python3")
})
.expect("queue.count data point with tag=python3 not found");
assert_eq!(dp.1, 42);
let m = find_metric(&metrics, "windmill.queue.running_count").unwrap();
let values = gauge_i64_values(m);
let dp = values
.iter()
.find(|(attrs, _)| {
attrs
.iter()
.any(|kv| kv.key.as_str() == "tag" && kv.value.as_str() == "deno")
})
.expect("running_count data point with tag=deno not found");
assert_eq!(dp.1, 5);
let m = find_metric(&metrics, "windmill.worker.busy").unwrap();
let values = gauge_i64_values(m);
let dp = values
.iter()
.find(|(attrs, _)| {
attrs
.iter()
.any(|kv| kv.key.as_str() == "worker" && kv.value.as_str() == "worker-test-1")
})
.expect("worker.busy data point with worker=worker-test-1 not found");
assert_eq!(dp.1, 1);
let m = find_metric(&metrics, "windmill.db.pool.active").unwrap();
assert_eq!(gauge_i64_values(m)[0].1, 5);
let m = find_metric(&metrics, "windmill.db.pool.idle").unwrap();
assert_eq!(gauge_i64_values(m)[0].1, 10);
let m = find_metric(&metrics, "windmill.db.pool.max").unwrap();
assert_eq!(gauge_i64_values(m)[0].1, 20);
let m = find_metric(&metrics, "windmill.health.db_latency").unwrap();
assert!((gauge_f64_value(m).unwrap() - 2.5).abs() < f64::EPSILON);
let m = find_metric(&metrics, "windmill.worker.uptime").unwrap();
assert!((gauge_f64_value(m).unwrap() - 3600.0).abs() < f64::EPSILON);
let m = find_metric(&metrics, "windmill.health.db_unresponsive").unwrap();
assert_eq!(gauge_i64_values(m)[0].1, 1);
// ── Health status phase (all 3 phases) ──────────────────────────
let m = find_metric(&metrics, "windmill.health.status").unwrap();
let values = gauge_i64_values(m);
let healthy = values
.iter()
.find(|(attrs, _)| {
attrs
.iter()
.any(|kv| kv.key.as_str() == "phase" && kv.value.as_str() == "healthy")
})
.expect("phase=healthy");
let degraded = values
.iter()
.find(|(attrs, _)| {
attrs
.iter()
.any(|kv| kv.key.as_str() == "phase" && kv.value.as_str() == "degraded")
})
.expect("phase=degraded");
let unhealthy = values
.iter()
.find(|(attrs, _)| {
attrs
.iter()
.any(|kv| kv.key.as_str() == "phase" && kv.value.as_str() == "unhealthy")
})
.expect("phase=unhealthy");
assert_eq!(healthy.1, 1);
assert_eq!(degraded.1, 0);
assert_eq!(unhealthy.1, 0);
// ── Histogram values ────────────────────────────────────────────
let m = find_metric(&metrics, "windmill.worker.execution_duration").unwrap();
assert!(histogram_f64_count(m).unwrap() >= 2);
assert!(histogram_f64_sum(m).unwrap() >= 4.0);
let m = find_metric(&metrics, "windmill.worker.pull_duration").unwrap();
assert!(histogram_f64_count(m).unwrap() >= 2);
}
// ═══════════════════════════════════════════════════════════════════════
// SPAN E2E TESTS
// ═══════════════════════════════════════════════════════════════════════
fn make_test_job(id: uuid::Uuid, parent: Option<uuid::Uuid>) -> windmill_queue::MiniPulledJob {
use windmill_types::jobs::JobKind;
let mut job = windmill_queue::MiniPulledJob::new_inline(
"test-workspace".to_string(),
None,
"test-user".to_string(),
"u/test-user".to_string(),
"test@example.com".to_string(),
Some("f/test/script".to_string()),
JobKind::Script,
None,
"deno".to_string(),
None,
);
job.id = id;
job.parent_job = parent;
job.started_at = Some(chrono::Utc::now());
job
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_root_job_span_created_on_success() {
let state = ensure_setup().await;
state.span_exporter.reset();
let job_id = uuid::Uuid::new_v4();
let job = make_test_job(job_id, None);
windmill_worker::otel_ee::add_root_flow_job_to_otlp(&job, true);
let spans = state.span_exporter.get_finished_spans().unwrap();
let span = spans
.iter()
.find(|s| s.name == "full_job")
.expect("full_job span not found");
assert_eq!(span.status, opentelemetry::trace::Status::Ok,);
// Verify attributes
let attrs: Vec<_> = span.attributes.iter().map(|kv| kv.key.as_str()).collect();
assert!(attrs.contains(&"job_id"), "missing job_id attribute");
assert!(
attrs.contains(&"workspace_id"),
"missing workspace_id attribute"
);
assert!(
attrs.contains(&"script_path"),
"missing script_path attribute"
);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_root_job_span_error_on_failure() {
let state = ensure_setup().await;
state.span_exporter.reset();
let job_id = uuid::Uuid::new_v4();
let job = make_test_job(job_id, None);
windmill_worker::otel_ee::add_root_flow_job_to_otlp(&job, false);
let spans = state.span_exporter.get_finished_spans().unwrap();
let span = spans
.iter()
.find(|s| s.name == "full_job")
.expect("full_job span not found");
match &span.status {
opentelemetry::trace::Status::Error { description } => {
assert_eq!(description.as_ref(), "Job failed");
}
other => panic!("expected Error status, got {:?}", other),
}
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_root_job_trace_id_matches_uuid() {
let state = ensure_setup().await;
state.span_exporter.reset();
let job_id = uuid::Uuid::parse_str("550e8400-e29b-41d4-a716-446655440000").unwrap();
let job = make_test_job(job_id, None);
windmill_worker::otel_ee::add_root_flow_job_to_otlp(&job, true);
let spans = state.span_exporter.get_finished_spans().unwrap();
let span = spans
.iter()
.find(|s| s.name == "full_job")
.expect("full_job span not found");
let expected_trace_id =
opentelemetry::trace::TraceId::from_bytes(job_id.as_u128().to_be_bytes());
assert_eq!(span.span_context.trace_id(), expected_trace_id);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_root_job_span_id_matches_uuid() {
let state = ensure_setup().await;
state.span_exporter.reset();
let job_id = uuid::Uuid::parse_str("550e8400-e29b-41d4-a716-446655440000").unwrap();
let job = make_test_job(job_id, None);
windmill_worker::otel_ee::add_root_flow_job_to_otlp(&job, true);
let spans = state.span_exporter.get_finished_spans().unwrap();
let span = spans
.iter()
.find(|s| s.name == "full_job")
.expect("full_job span not found");
let expected_span_id =
opentelemetry::trace::SpanId::from_bytes(job_id.as_u64_pair().1.to_be_bytes());
assert_eq!(span.span_context.span_id(), expected_span_id);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_child_job_produces_no_span() {
let state = ensure_setup().await;
state.span_exporter.reset();
let parent_id = uuid::Uuid::new_v4();
let job_id = uuid::Uuid::new_v4();
let job = make_test_job(job_id, Some(parent_id));
windmill_worker::otel_ee::add_root_flow_job_to_otlp(&job, true);
let spans = state.span_exporter.get_finished_spans().unwrap();
let found = spans.iter().any(|s| s.name == "full_job");
assert!(!found, "child job should not produce a span");
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_root_job_span_attributes_values() {
let state = ensure_setup().await;
state.span_exporter.reset();
let job_id = uuid::Uuid::new_v4();
let job = make_test_job(job_id, None);
windmill_worker::otel_ee::add_root_flow_job_to_otlp(&job, true);
let spans = state.span_exporter.get_finished_spans().unwrap();
let span = spans
.iter()
.find(|s| s.name == "full_job")
.expect("full_job span not found");
let get_attr = |key: &str| -> String {
span.attributes
.iter()
.find(|kv| kv.key.as_str() == key)
.map(|kv| kv.value.as_str().to_string())
.unwrap_or_default()
};
assert_eq!(get_attr("job_id"), job_id.to_string());
assert_eq!(get_attr("workspace_id"), "test-workspace");
assert_eq!(get_attr("script_path"), "f/test/script");
}

View File

@@ -1081,6 +1081,115 @@ echo "$result"
Ok(())
}
#[cfg(feature = "rlang")]
#[sqlx::test(fixtures("base"))]
async fn test_r_job(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let content = r#"
main <- function(msg) {
return(paste("hello", msg))
}
"#
.to_owned();
let result = RunJob::from(JobPayload::Code(RawCode {
hash: None,
content,
path: None,
lock: None,
language: ScriptLang::Rlang,
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default()
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
}))
.arg("msg", json!("world"))
.run_until_complete(&db, false, port)
.await
.json_result()
.unwrap();
assert_eq!(result, json!("hello world"));
Ok(())
}
#[cfg(feature = "rlang")]
#[sqlx::test(fixtures("base", "wmill_cli_test"))]
async fn test_r_get_variable(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let content = r#"
main <- function() {
return(get_variable("u/test-user/test_var"))
}
"#
.to_owned();
let result = RunJob::from(JobPayload::Code(RawCode {
hash: None,
content,
path: None,
lock: None,
language: ScriptLang::Rlang,
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default()
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
}))
.run_until_complete(&db, false, port)
.await
.json_result()
.unwrap();
assert_eq!(result, json!("hello from variable"));
Ok(())
}
#[cfg(feature = "rlang")]
#[sqlx::test(fixtures("base", "wmill_cli_test"))]
async fn test_r_get_resource(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let content = r#"
main <- function() {
return(get_resource("u/test-user/test_res"))
}
"#
.to_owned();
let result = RunJob::from(JobPayload::Code(RawCode {
hash: None,
content,
path: None,
lock: None,
language: ScriptLang::Rlang,
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default()
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
}))
.run_until_complete(&db, false, port)
.await
.json_result()
.unwrap();
assert_eq!(result, json!({"host": "localhost", "port": 5432}));
Ok(())
}
#[cfg(feature = "nu")]
#[sqlx::test(fixtures("base"))]
async fn test_nu_job(db: Pool<Postgres>) -> anyhow::Result<()> {

View File

@@ -1,7 +1,6 @@
#[cfg(feature = "enterprise")]
use crate::ee_oss::ExternalJwks;
use axum::{
async_trait,
extract::{FromRequestParts, OriginalUri, Query},
Extension, Json,
};
@@ -226,7 +225,15 @@ impl AuthCache {
t_hash,
w_id.as_ref(),
)
.map(|x| (x.owner, x.email, x.super_admin, x.scopes, x.label))
.map(|x| {
(
x.owner,
x.email,
x.super_admin,
x.scopes,
x.label,
)
})
.fetch_optional(&self.db)
.await
.ok()
@@ -235,7 +242,13 @@ impl AuthCache {
if let Some(user) = user_o {
let authed_o = {
match user {
(Some(owner), Some(email), super_admin, _, label) if w_id.is_some() => {
(
Some(owner),
Some(email),
super_admin,
_,
label,
) if w_id.is_some() => {
let username_override = username_override_from_label(label);
if let Some((prefix, name)) = owner.split_once('/') {
if prefix == "u" {
@@ -451,7 +464,11 @@ pub(crate) async fn extract_token<S: Send + Sync>(parts: &mut Parts, state: &S)
None => Extension::<Cookies>::from_request_parts(parts, state)
.await
.ok()
.and_then(|cookies| cookies.get(COOKIE_NAME).map(|c| c.value().to_owned())),
.and_then(|cookies| {
cookies
.get(COOKIE_NAME)
.map(|c| c.value_trimmed().to_owned())
}),
};
#[derive(Deserialize)]
@@ -504,7 +521,6 @@ impl BruteForceCounter {
}
}
#[async_trait]
impl<S> FromRequestParts<S> for Tokened
where
S: Send + Sync,
@@ -535,7 +551,6 @@ where
}
}
#[async_trait]
impl<S> FromRequestParts<S> for OptTokened
where
S: Send + Sync,

View File

@@ -12,8 +12,7 @@ pub mod ee;
pub mod ee_oss;
pub mod scopes;
use axum::async_trait;
use axum::extract::FromRequestParts;
use axum::extract::{FromRequestParts, OptionalFromRequestParts};
use http::request::Parts;
use windmill_audit::audit_oss::AuditAuthorable;
@@ -345,7 +344,6 @@ pub async fn maybe_refresh_folders(
// ------------ FromRequestParts impls (direct call to auth module) ------------
#[async_trait]
impl<S> FromRequestParts<S> for ApiAuthed
where
S: Send + Sync,
@@ -361,7 +359,24 @@ where
}
}
#[async_trait]
impl<S> OptionalFromRequestParts<S> for ApiAuthed
where
S: Send + Sync,
{
type Rejection = std::convert::Infallible;
async fn from_request_parts(
parts: &mut Parts,
state: &S,
) -> std::result::Result<Option<Self>, Self::Rejection> {
Ok(
<Self as FromRequestParts<S>>::from_request_parts(parts, state)
.await
.ok(),
)
}
}
impl<S> FromRequestParts<S> for OptJobAuthed
where
S: Send + Sync,
@@ -397,7 +412,6 @@ fn empty_parts() -> Parts {
#[derive(Clone, Debug)]
pub struct OptAuthed(pub Option<ApiAuthed>);
#[async_trait]
impl<S> FromRequestParts<S> for OptAuthed
where
S: Send + Sync,
@@ -408,7 +422,7 @@ where
parts: &mut Parts,
state: &S,
) -> std::result::Result<Self, Self::Rejection> {
ApiAuthed::from_request_parts(parts, state)
<ApiAuthed as FromRequestParts<S>>::from_request_parts(parts, state)
.await
.map(|authed| Self(Some(authed)))
.or_else(|_| Ok(Self(None)))

Some files were not shown because too many files have changed in this diff Show More