Commit Graph

516 Commits

Author SHA1 Message Date
Diego Imbert
5d1c54d9b3 feat: Debounce node (#8324)
* Debounce node works

* sqlx prepare

* sqlx prepare

* fix: address PR review issues for flow node debouncing

- Add sibling check in parent-walking loop to avoid killing branchall siblings
- Remove stale .sqlx cache files from earlier iterations
- Remove single-variant FlowNodeDebounceResult enum, use Result<()>
- Parse flow value once in version guard, recurse into nested modules
- Fix Svelte reactivity when switching selected flow modules
- Fix Tab indentation in FlowModuleComponent
- Use integer types in OpenAPI spec for debounce fields

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* ee repo ref

* nit sqlx

* add Debouncing: None

* ee repo ref

* ee repo

* sqlx update

* fix: reject node-level debouncing inside branches (branchall/branchone)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Revert "fix: reject node-level debouncing inside branches (branchall/branchone)"

This reverts commit fa4820dde2.

* ee repo

* sqlx prepare

* sqlx prepare

* feat: add MIN_VERSION_SUPPORTS_NODE_DEBOUNCING (1.658.0) version guard

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: mark node-level debouncing as EE only in openflow schema

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: guard node debouncing against parallel steps (len > 1)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* generate system prompts

* system prompts

* chore: update ee-repo-ref to c04f3851c03758662e4936ff4b6e71bc56dbae7e

This commit updates the EE repository reference after PR #451 was merged in windmill-ee-private.

Previous ee-repo-ref: d140bb8944dfe3efb23cf8c12f556eacf30e2f87

New ee-repo-ref: c04f3851c03758662e4936ff4b6e71bc56dbae7e

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-24 09:47:41 +00:00
Pyra
010753c73a fix: skip debounce arg accumulation when batch table is empty (CE) (#8485)
On CE (without private feature), v2_job_debounce_batch is never
populated because maybe_debounce_post_preprocessing is EE-only.
The accumulation query returns zero rows, producing an empty array
that replaces the original nodes_to_relock value. This causes flow
modules to never get relocked when triggered by relative imports.

Fix: only replace the original value when the batch query actually
returned entries to accumulate.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-23 17:59:46 +00:00
hugocasa
efb4a27d51 fix: replace email with permissioned_as for triggers/schedules (#8439)
* refactor: replace email with permissioned_as for triggers/schedules

Add a new `permissioned_as` column (format: `u/{username}`, `g/{group}`,
or raw email) to all trigger tables and schedule. This value is used
directly for job permission checks, removing the need for email lookups
when creating/updating triggers.

- Migration: add permissioned_as to all 9 trigger tables + schedule,
  drop email from trigger tables (schedule keeps it for backwards compat)
- Backend: resolve_email() (async, DB) -> resolve_permissioned_as() (sync)
- Email cache: get_email_from_permissioned_as() with quick_cache for
  places that still need email (fetch_api_authed, schedule backwards compat)
- Frontend: rename email/preserve_email -> permissioned_as/preserve_permissioned_as
  in deploy data and OpenAPI schemas
- Tests updated for new field names and u/{username} format

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix sqlx/build

* update ee ref

* refactor: simplify resolve_edited_by to always use authed username

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix compile + migration

* update ref

* test: add trigger trait method tests for permissioned_as queries

Add tests that call TriggerCrud and Listener trait methods directly
to verify dynamic SQL correctly references the permissioned_as column.
Covers get_trigger_by_path, list_triggers, set_trigger_mode, and
fetch_enabled_unlistened_triggers for all trigger types.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* update sqlx

* fix: use permissioned_as directly for schedules and fix audit RLS for groups

- Schedule: permissioned_as only set on create, not on edit/set_enabled
- Schedule: stop reading email column, use get_email_from_permissioned_as
- Triggers: use fetch_api_authed_from_permissioned_as instead of edited_by
- Triggers: rename listener fields for clarity (username -> edited_by)
- Fix audit author username for group permissioned_as (g/test -> group-test)
  to match session.user, preventing RLS policy violations on audit_partitioned
- OpenAPI: remove permissioned_as/preserve_permissioned_as from EditSchedule
- Add backwards-compat comments for schedule email writes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: regenerate system prompts for permissioned_as field

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix build

* refactor: generalize onBehalfOf naming, add permissioned_as to EditSchedule

- Frontend: rename onBehalfOfPermissionedAs -> onBehalfOf with comments
  explaining it carries emails for flows/scripts and permissioned_as for
  triggers/schedules
- Frontend: rename getOnBehalfOfEmail -> getOnBehalfOf,
  getOnBehalfOfPermissionedAsForDeploy -> getOnBehalfOfForDeploy,
  customOnBehalfOfEmails -> customOnBehalfOf
- Backend: add optional permissioned_as/preserve_permissioned_as to
  EditSchedule with COALESCE (only updates when provided)
- Backend: add on_behalf_of audit log for schedule edit
- Backend: remove unused resolve_on_behalf_of_permissioned_as
- Tests: remove email assertions from schedule update test (email is
  just backwards compat, only permissioned_as matters)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: preserve email column when permissioned_as is preserved on schedule edit

Derive email from the preserved permissioned_as via cache lookup instead
of always writing authed.email. This keeps the email column consistent
with the old behavior for backwards compat with old workers.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: update deploy UI labels from "edited by" to "run as" for triggers

Triggers now use permissioned_as (not edited_by) for permissions, so
update the deploy UI wording to reflect this. Also update wm_deployers
group description to mention schedules and permissioned_as.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use u/username format for custom trigger/schedule deploy selection

When picking a custom user for trigger/schedule deployment, store
u/${username} (permissioned_as format) instead of the email. Flows/scripts
continue to use email format for on_behalf_of_email.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: show u/username format for "me" option in trigger deploy selector

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: simplify OnBehalfOfSelector to return the right format per kind

OnBehalfOfSelector now handles the email vs permissioned_as format
internally based on kind:
- triggers: returns u/username, displays u/username in all options
- flows/scripts/apps: returns email, displays username

The onSelect callback now takes (choice, value?) where value is already
in the correct format. Parent components just store it directly without
needing to know about the format difference.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: always show u/username format in OnBehalfOfSelector for all kinds

Display is now consistent: all kinds show u/username in the selector.
The returned value still differs (email for flows/scripts, u/username
for triggers) since the backend APIs expect different formats.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: replace email with permissioned_as in http_trigger test insert

The email column was dropped from trigger tables in the migration.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: review fixes — migration, app policy, capture cleanup, naming

- Migration: remove DEFAULT '', use nullable → populate → SET NOT NULL
- App policy: set both on_behalf_of and on_behalf_of_email for all choices
- OnBehalfOfSelector: return OnBehalfOfDetails {email, permissionedAs} instead of ambiguous value
- Remove unused email field from Capture struct and query
- Rename getSourceEmail/getTargetEmail → getSourceOnBehalfOf/getTargetOnBehalfOf
- Rename test functions from preserve_email to preserve_permissioned_as

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add permissioned_as to all test schedule INSERTs

Since the migration no longer uses DEFAULT '', all INSERTs must
explicitly provide permissioned_as. Updated test fixtures and
schedule_push tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: strip permissioned_as from exports/sync, fix OpenAPI required field

- Add permissioned_as to workspace export strip list (like edited_by)
- Add permissioned_as to CLI TriggerFile Omit list
- Fix TriggerExtraProperty.required: email → permissioned_as
- Regenerate frontend and CLI types

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove accidentally committed generated files

These directories are gitignored and should not be tracked.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: regenerate system prompts for permissioned_as schema changes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove permissioned_as from CLI TriggerFile Omit list

Already stripped in workspace export, no need to also omit from the type.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: optimize email cache key and revert TriggerFile Omit change

- Use single concatenated string for cache key instead of (String, String) tuple
- Remove permissioned_as from CLI TriggerFile Omit (already stripped in export)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: zero-allocation email cache lookups using Equivalent trait

Use a borrowed EmailCacheKey(&str, &str) for cache lookups via
quick_cache's Equivalent support. Only allocates (String, String)
on cache miss for insert. This is called on every trigger fire
and schedule push.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add permissioned_as to Schedule required fields in OpenAPI spec

The backend always returns permissioned_as (non-optional String),
so the schema should reflect that.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: handle group- prefix in migration UPDATE statements

edited_by can be 'group-{name}' for group-owned triggers/schedules.
The migration now correctly maps these to 'g/{name}' format instead
of incorrectly producing 'u/group-{name}'.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Revert "fix: handle group- prefix in migration UPDATE statements"

This reverts commit 0971392b38.

* fix: use superadmin email to resolve permissioned_as in schedule migration

For users upgrading from older versions where edited_by may not reflect
the actual schedule owner, check if the email belongs to a superadmin
and look up their username. Otherwise fall back to edited_by.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: fall back to superadmin email when not in workspace usr table

If the superadmin isn't a member of the workspace, use their email
as raw permissioned_as instead of falling back to edited_by.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: always update permissioned_as and email on schedule edit

Consistent with pre-refactor behavior where email and edited_by
were always updated on every edit. permissioned_as is now always
set (to editing user or preserved value), removing the COALESCE
that previously preserved it when not provided.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add schedule permission tests and centralize group prefix constants

Tests: schedule create/update for normal user, workspace admin, and
superadmin not in workspace. Verifies schedule fields (email,
permissioned_as, edited_by) and pushed job fields (permissioned_as,
permissioned_as_email).

Constants: centralize "u/", "g/", "group-" as PERMISSIONED_AS_USER_PREFIX,
PERMISSIONED_AS_GROUP_PREFIX, USERNAME_GROUP_PREFIX.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use @unknown.windmill.dev for synthetic email fallback

Prevents privilege escalation: a user with username like
'superadmin_secret' would get superadmin via the synthetic
email matching SUPERADMIN_SECRET_EMAIL. Using a different
subdomain avoids any collision with hardcoded @windmill.dev emails.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* update ee ref

* sqlx

* chore: regenerate system prompts after main merge

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to bda51bc33bcb573659e7ff07d0a23ff6e23b8148

This commit updates the EE repository reference after PR #468 was merged in windmill-ee-private.

Previous ee-repo-ref: 8cf1802f8fe183f430830590b4f3172a50207843

New ee-repo-ref: bda51bc33bcb573659e7ff07d0a23ff6e23b8148

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-20 16:28:38 +00:00
Ruben Fiszel
7de98c0df4 feat: add OTel metrics support (#8442)
* [ee] feat: add OTel metrics support

Add OpenTelemetry metrics export for Windmill operational metrics.
When the OTel metrics toggle is enabled in instance settings (EE),
Windmill exports 16 metrics to any OTLP-compatible collector, letting
users observe queue depths, worker execution, DB pool state, and health
without a separate Prometheus setup.

Changes:
- otel_oss.rs: no-op stubs for OSS builds
- monitor.rs: queue count/running count gauges, zombie counters, DB pool
  monitoring (shared single DB query and loop with Prometheus)
- worker.rs: execution count/duration, worker busy, pull duration
- jobs.rs: queue push/delete/pull counters
- health.rs: DB latency gauge
- main.rs: call monitor_pool_otel unconditionally
- InstanceSetting.svelte: enable metrics toggle for EE licenses

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref.txt for OTel metrics

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add worker_started, worker_uptime, health_status, health_db_unresponsive OTel metrics

Wire up 5 additional metrics to reach parity with Prometheus:
- worker_execution_failed: wired in add_completed_job_error (was defined but unused)
- worker.started: incremented on worker startup
- worker.uptime: recorded each loop iteration
- health.status: phase gauge (healthy/degraded/unhealthy)
- health.db_unresponsive: flag (0/1)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref.txt

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to fbe68e4aa621e30378995cfd328a6ccf74176614

This commit updates the EE repository reference after PR #469 was merged in windmill-ee-private.

Previous ee-repo-ref: 6fa1881aafdfb60f4abf11a37f01f6fedaecb3ec

New ee-repo-ref: fbe68e4aa621e30378995cfd328a6ccf74176614

Automated by sync-ee-ref workflow.

* fix: remove duplicate cfg attr and duplicate OTel pool reporting

- Remove duplicate #[cfg(feature = "prometheus")] on monitor_pool
- Remove OTel block from monitor_pool; monitor_pool_otel is the sole
  OTel reporter, eliminating duplicate windmill.db.pool.* metrics in
  EE builds
- Simplify monitor_pool back to its original Prometheus-only structure

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-19 08:10:37 +00:00
Ruben Fiszel
bee928276e fix: show cancelled WAC jobs as done in workflow timeline (#8436)
* fix: show cancelled WAC jobs as done in workflow timeline

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: keep polling after cancel so WAC timeline updates to completed state

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: treat WAC as done in LogPanel when loader stops after cancel

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: show preview badge and hide _MODULES arg in run history

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: show preview badge alongside status dot, not instead of it

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 12:50:50 +00:00
Diego Imbert
ad03a5dbd7 fix: cleanup job debounce batch (#8420)
* delete from v2_job_debounce on accumulate

* sqlx prepare

* Unconditionally remove from v2_job_debounce_batch

* sqlx prepare
2026-03-18 08:18:58 +00:00
Ruben Fiszel
31d6660d56 feat: script module mode with CLI sync, preview, and WAC UI improvements (#8380)
* feat: add script module mode with folder model for Bun and Python

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add missing modules field to RawCode in bun_executor

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* sqlx

* feat: enrich WAC templates with checkpoint and replay semantics

Add prominent comments explaining that all computation must happen
inside task/step/taskScript or it will be replayed on resume/retry.
Clarify that waitForApproval does not hold a worker and that
approve/reject URLs are available in the timeline step details.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(cli): script module sync idempotency, per-module hash tracking, and preview support

- Fix pull→push idempotency: use `??` instead of `||` for module lock
  field so empty strings are preserved (matches API's `lock: ""`)
- Add per-module hash tracking in wmill-lock.yaml following the flow
  inline script pattern (SCRIPT_TOP_HASH + per-module subpath hashes)
- Selective module lock regeneration: only regenerate locks for modules
  whose content actually changed, not all modules
- Use unfiltered rawWorkspaceDependencies for module hashes to match
  what updateModuleLocks passes to fetchScriptLock
- Show changed module names in stale script output for clarity
- Add module support to `script preview` command: read modules from
  __mod/ folder and pass them in the preview API request
- Add preview tests for taskScript pattern (flat and folder layout)
- Update test assertion for module stale detection output

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(frontend): WAC UI improvements — reorder templates, module tab rename, import consolidation

- Reorder WAC template buttons: TypeScript before Python in
  ScriptBuilder, CreateActionsScript, and CreateActionsFlow
- Remove dropdown items from +Script button (simplify to direct link)
- Move "Import Workflow-as-Code" to +Flow dropdown with dedicated drawer
- Add module tab rename: pencil icon on hover opens popover with
  validation, fixed-width icon container prevents layout shift

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: remaining module-mode changes from working branch

- Backend parser updates for WAC detection
- CLI sync/types updates for raw app path and module support
- Frontend UI polish (Dev.svelte, ScriptRow, script hash page)
- Test fixture updates

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(cli): add test for module modification detection in generate-metadata

Verifies that modifying a single module file re-triggers stale
detection and only the changed module is listed, not all modules.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backend): critical fixes from PR review

- Fix hardcoded dev path in bun_executor.rs WAC v2 wrapper — use
  "windmill-client" import instead of absolute filesystem path
- Fix missed no_main_func → auto_kind rename in parser TS test
- Add modules column to clone_script SQL (windmill-common and
  windmill-api-workspaces) so cloned scripts retain their modules
- Add modules: None to RawCode structs in worker tests
- Restore complete sqlx cache (merge main's cache + our new queries)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backend): fix clone warning treated as error in CI

Change `.clone()` on double reference to `*k` dereference in
scripts.rs hash implementation. Update sqlx cache with new query
hashes from modified clone_script SQL.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(frontend): use published parser wasm versions for CI build

The local file:// paths for windmill-parser-wasm-py and
windmill-parser-wasm-ts don't exist in the Cloudflare Pages build
environment. Revert to published npm versions (1.655.0).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(frontend): update parser wasm packages to 1.657.2

Use newly published windmill-parser-wasm-ts and windmill-parser-wasm-py
v1.657.2 which include auto_kind/WAC detection changes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(frontend): regenerate package-lock.json for npm ci compatibility

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(frontend): use main's lockfile as base, update only parser wasm packages

Regenerating package-lock.json from scratch pulled different dependency
versions causing svelte-check type errors. Instead, start from main's
lockfile and only update the two changed packages.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backend): add modules column to fetch_script_for_update query

The Script<SR> struct has a modules field (FromRow), but
fetch_script_for_update didn't SELECT modules, causing a runtime
error "no column found for name: modules" when the worker processed
dependency jobs. This was the root cause of the relock_skip test
timeout.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backend): fix script module execution for Python and Bun

- Fix modules not passed through job queue: inject _MODULES into
  PushArgs.extra when pushing Code jobs so worker can extract them
- Fix Python module imports: use relative imports (from .helper)
  and add sys.path.insert for module directory in wrapper
- Fix Python tests: use relative imports and empty lock to prevent
  pip from resolving module names as packages
- Add local file check in Bun loader for module resolution
- Ignore Bun module test (bundle mode loader integration tracked
  separately)
- Add missing modules column to fetch_script_for_update query

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backend): remove unnecessary empty lock in Python module tests

Relative imports (from .helper) are not parsed as pip packages,
so the empty lock workaround is not needed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backend): fix module execution for Python and Bun — all tests pass

Python modules:
- Use relative imports (from .helper import greet) since scripts run
  as packages
- Add sys.path.insert for module directory in wrapper to ensure local
  modules take precedence over pip packages with same name

Bun modules:
- Use bundled output (./out/main.js) as wrapper import when modules
  are present — the bundled output has module content inlined by
  Bun.build, avoiding runtime loader resolution issues
- Add local file check in loader.bun.js onResolve to short-circuit
  API URL resolution for module files on disk

Job queue:
- Inject _MODULES into PushArgs.extra when pushing Code jobs so
  the worker can extract them at execution time

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: address PR review — simplify, fix correctness, remove dead code

Critical fixes:
- Replace all CLI `no_main_func` references with `auto_kind` (string)
  to match the backend migration and API changes
- Remove duplicated `compute_python_module_dir` in worker.rs, use
  the canonical version from python_executor.rs

High priority:
- Auto-create `__init__.py` in intermediate directories for nested
  Python modules so imports like `from .utils.math import add` work
  without users manually creating __init__.py files
- Remove redundant `sys_path_insert` — relative imports use Python's
  package system, not sys.path

Medium:
- Fix lock file base name extraction: use regex to strip only the
  final extension (`.replace(/\.[^.]+$/, '')`) instead of `indexOf(".")`
  which breaks for files like `helper.test.ts`

Simplification:
- Remove dead `{#if false}` Popover block in ScriptEditor.svelte
- Guard loader.bun.js local file check to only run for relative paths
  (matching the Windows loader pattern)
- Add clarifying comment on Bun dual mechanism (build + run phases)
- Add maintenance comment on manual Hash impl for NewScript

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: final review fixes — stale cleanup, baseName, auto_kind export

- Fix sync.ts baseName extraction using indexOf(".") → regex
  (same fix as script.ts/metadata.ts, missed this instance)
- Add stale module file cleanup in writeModulesToDisk: removes files
  from __mod/ that are no longer in the modules map before writing,
  fixing the pull→push cycle that couldn't delete modules
- Log warning when _MODULES serialization fails in job push instead
  of silently dropping modules
- Use strict equality (===) for auto_kind comparison
- Exclude auto_kind from workspace export — it is auto-detected by
  the parser at deploy time from script content

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): remove auto_kind from push, comparison, and metadata

auto_kind is auto-detected by the parser at deploy time, so the CLI
should not send it, compare it, or write it to script.yaml.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove erroneously added backend/backend/.sqlx directory

Duplicate .sqlx cache was committed at the wrong nested path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address PR review feedback + fix CI dead_code warning

Frontend (ScriptEditor.svelte):
- Fix switchToMain() missing lastSyncedCode update — prevents stale
  code sync on external changes while editing a module tab
- Fix formatAction saving module code to main script's localStorage
  draft — now saves main code when on a module tab
- Fix non-null assertion on inferModuleLang in renameModule — fall
  back to original language instead of force unwrap
- Remove redundant activeModuleTab truthy check in runTest

CLI (script.ts):
- Clean up empty directories after removing stale module files in
  writeModulesToDisk

Backend:
- Add path traversal guard in write_module_files — reject module
  paths containing ".."
- Fix dead_code warning on auto_kind field in workspace export struct

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(frontend): improve auto_kind UX + address review findings

- Rename "Include without main function" toggle to "Include library
  scripts" in script list (ItemsList.svelte)
- Update NoMainFuncBadge: "No main" → "Library" with clearer tooltip
- Filter module file extensions by main script language — Python
  scripts only allow .py modules, TypeScript only .ts, etc.
- Split flushModuleState into flushModuleContent (no UI side-effect)
  and flushModuleState (flush + reset tab), reducing duplication
- Dynamic placeholder and hint text in add module popover based on
  main script language

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 01:20:09 +00:00
Ruben Fiszel
f2be625348 feat: store hashed tokens instead of plaintext (#8217)
* feat: store hashed tokens in the token table instead of plaintext

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address review issues in token hash migration

- Update all base.sql fixtures to include token_hash/token_prefix columns
- Keep plaintext token for webhook tokens (needed for URL reconstruction)
- Restore get_token_by_prefix to query DB for webhook tokens
- Fix down migration to delete NULL-token rows before restoring NOT NULL
- Update parser fixture standalone schema
- Update EE dedicated_worker_ee.rs to use token_hash/token_prefix

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: restore sqlx offline cache (only add new query files)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: keep writing plaintext token column for backward compat

Write to token column alongside token_hash until MIN_VERSION_SUPPORTS_TOKEN_HASH
(1.649.0) is reached. This ensures older workers can still authenticate
during rolling upgrades. Remove the separate UPDATE in new_webhook_token
since create_token_internal now writes plaintext directly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: branch on MIN_VERSION to write plaintext token or null

Check MIN_VERSION_SUPPORTS_TOKEN_HASH at runtime: write plaintext to
token column while old workers exist, switch to NULL once all workers
are >= 1.649.0.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: set MIN_VERSION_SUPPORTS_TOKEN_HASH to 1.650.0

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: use token_hash for email lookup and expiry notifications

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: rotate webhook tokens instead of recovering plaintext from DB

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: use token_hash for native trigger token lookups and deletes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* sqlx

* refactor: drop webhook_token_prefix from native_trigger table

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: backward compat for token rotation and make webhook_token_hash NOT NULL

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: prevent panic on short superadmin secret token prefix

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: prevent panic on short superadmin secret token prefix

Replace all `token[0..TOKEN_PREFIX_LEN]` slicing with
`token.get(..TOKEN_PREFIX_LEN).unwrap_or(token)` to prevent
panics when a token shorter than 10 chars is provided (e.g.
malformed Authorization header, short superadmin secret).

Co-authored-by: hugocasa <hugocasa@users.noreply.github.com>

* fix: prevent panic on short token prefix slicing

Replace all `token[0..TOKEN_PREFIX_LEN]` with safe
`token.get(..TOKEN_PREFIX_LEN).unwrap_or(token)` to prevent panics
on malformed tokens shorter than 10 characters.

Co-authored-by: hugocasa <hugocasa@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Revert "fix: prevent panic on short superadmin secret token prefix"

This reverts commit 37ec2e5ad5.

* revert: remove unnecessary defensive token prefix slicing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add token_hash to end_user_email test fixture

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: add integration tests for token hash migration

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: correct token_hash test assertions for cache and version

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: add plaintext column removal reminder to test fixtures

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: log count of orphaned triggers deleted during migration

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: preserve orphaned triggers with error instead of deleting

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: rename token_expiry_notification.token to token_hash and copy owner/expiration in rotate

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: hash existing plaintext values before renaming token_expiry_notification column

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: remove unnecessary length check in token_expiry_notification migration

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* update dates and version

* updat ee ref + sqlx

* improve mcp migration

* fix: atomic token rotation with rollback on trigger update failure

rotate_webhook_token now atomically inserts the new token and deletes
the old one in a single transaction, preventing token leaks.

Returns new_token_hash so callers can clean up the new token if their
subsequent trigger update fails (which involves external HTTP calls
and cannot be in the same DB transaction).

- Handler: wraps post-rotation work; deletes new token on failure
- Google renewal: deletes new token if service_config update fails
- Tests updated to match new atomic semantics

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* higher min version

* fix: defer old token deletion to avoid breaking triggers on update failure

rotate_webhook_token now keeps the old token alive and returns
old_token_hash. Callers delete it only after the trigger row has been
successfully updated. If the external service call or DB update fails,
the trigger keeps working with the old token.

Worst case: if the best-effort delete fails, the old token leaks as an
extra DB row — harmless compared to breaking the trigger.

Also update summarized_schema.txt for renamed columns.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 2d0823a471014e2bc2d898c63518323946b7474f

This commit updates the EE repository reference after PR #437 was merged in windmill-ee-private.

Previous ee-repo-ref: 7aef8b06cb6f54c2bc89dd57b70947deed72553c

New ee-repo-ref: 2d0823a471014e2bc2d898c63518323946b7474f

Automated by sync-ee-ref workflow.

* fix: prevent panic on short tokens by using safe prefix extraction

Add safe_token_prefix() helper that uses .get(..TOKEN_PREFIX_LEN).unwrap_or(token)
instead of direct slice indexing, preventing panics when tokens are shorter than
10 characters (e.g., short superadmin secrets or malformed Bearer tokens).

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: HugoCasa <hugo@casademont.ch>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: hugocasa <hugocasa@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
2026-03-17 01:15:38 +00:00
Diego Imbert
e0e78442b7 Support T | T[] in debounce (#8340)
* Detect union types in TS

* display union type arguments

* Handle single values at accumulation time

* nit propagate otyp

* Python support

* npm package update
2026-03-13 20:31:25 +00:00
Ruben Fiszel
48bc3e2445 fix: prevent zombie jobs from looping forever (#8313)
* fix: always increment zombie job counter to prevent infinite loop at restart limit

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: force-complete zombie jobs when handle_job_error fails

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: revert counter increment change, keep re-detection for robustness

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: disable schedule when permissioned_as user is not found

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 23:46:24 +00:00
Ruben Fiszel
83be59e0e8 fix: debounce webhook arg accumulation with max_count/max_time limits (#8307)
* fix: correct debounce max_total_debounces_amount semantics and complete previous job on limit exceeded

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: enable debounce arg accumulation for post-preprocessing flows

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add debounce accumulation tests for max_count and max_time limits

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add push-time max_count and max_time accumulation tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* sqlx

* sqlx

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 17:22:39 +00:00
Ruben Fiszel
438f609a78 fix: delete debounce_key on post-preprocessing limit exceeded (#8299)
* fix: delete debounce_key entry when post-preprocessing limits exceeded

For preprocessor flows, the runnable_settings_handle has
debounce_delay_s = None, so maybe_apply_debouncing at pull-time
won't clean up stale debounce_key entries. Previously we only
reset the entry (UPDATE), but since the flow executes immediately
without rescheduling, a stale entry would cause the next incoming
flow to incorrectly try to debounce against an already-executing job.

Change from UPDATE (reset) to DELETE so the entry is fully removed.
Update tests to expect deletion instead of reset.

Companion EE PR: https://github.com/windmill-labs/windmill-ee-private/pull/448

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: persist accumulated debounce args to v2_job for flows

The in-memory arg accumulation in maybe_apply_debouncing was not
persisted back to v2_job. For scripts this is fine (single execution),
but for flows, subsequent steps re-read args from the DB via
get_mini_pulled_job and would see the original (non-accumulated) value.

Also improve the job log message to show both original and accumulated
argument values for clarity.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref to d1c14355026151ecdd31adda8e2c60ecd1b5ad65

This commit updates the EE repository reference after PR #448 was merged in windmill-ee-private.

Previous ee-repo-ref: bff784002a3335af7c10982599c8f03e536d5abf

New ee-repo-ref: d1c14355026151ecdd31adda8e2c60ecd1b5ad65

Automated by sync-ee-ref workflow.

* test: assert accumulated debounce args are persisted to v2_job

Add DB persistence assertions to accumulation tests to prevent
regressions on the fix that writes accumulated args back to v2_job.
Without this, flow steps re-reading args from the DB would see
the original (non-accumulated) value.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* sqlx

* chore: update ee-repo-ref.txt to ee-private main

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-10 13:01:58 +00:00
Ruben Fiszel
a6d4390790 feat: workflow-as-code (WAC) v2 (#8172)
* feat: workflow-as-code v2 with @task decorator API

Replace ctx.step("name", "script") API with @task decorators where
functions are called directly. Users no longer need to pass WorkflowCtx
or use string-based step names/script paths.

Python: @task decorator with contextvars-based implicit context
TypeScript: task() wrapper with module-level context variable
Parsers: detect @task function calls instead of ctx.step() calls
Worker: updated wrappers to set implicit context

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: WAC v2 checkpoint/replay with _executing_key child dispatch

- Rust-side orchestration: parent dispatches child jobs, suspends, resumes on completion
- _executing_key in checkpoint tells child which step to execute directly
- task() throws StepSuspend(mode="step_complete") after executing target step
- result_processor handles child completion and updates parent checkpoint
- WacGraph.svelte for runtime execution visualization
- Sequential and parallel workflows tested end-to-end

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: WAC v2 bundle cache, globalThis ctx sharing, description optional

- Disable bun bundle caching for WAC v2 scripts (wrapper needs
  windmill-client from node_modules, not available in bundle mode)
- Use Reflect.set/get(globalThis, "__wmill_wf_ctx") to share workflow
  context across dual module instances (wrapper vs user script)
- Never-resolving thenable for non-matching steps in child job mode
  prevents Promise.all race conditions
- Make description field optional in NewScript API (defaults to "")

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add step() primitive for inline checkpointed steps

step() executes a function inline (no child job) and persists the result
to the checkpoint. On replay, the cached value is returned — ensuring
deterministic behavior for non-deterministic operations like Date.now()
or Math.random().

- TypeScript: step(name, fn) — executes inline, throws StepSuspend with
  mode "inline_checkpoint" to persist before continuing
- Rust: InlineCheckpoint variant in WacOutput, saves to checkpoint and
  resets running=false for immediate re-pickup (no zombie wait)
- Shared step counter between task() and step() via _allocKey()

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add Python WAC v2 support with task(), step(), workflow()

- Python SDK: WorkflowCtx with _executing_key child mode, _alloc_key
  shared counter, _run_inline_step for step(), _execute_directly and
  _never_resolve for child mode, step() async function
- Python executor: WAC v2 detection, checkpoint.json writing, WAC
  wrapper.py generation calling _run_workflow(), post-execution hook
  into shared handle_wac_v2_output()
- Make handle_wac_v2_output pub so both bun and python executors share
  the same dispatch/suspend/inline-checkpoint logic
- 17 Python tests covering dispatch, replay, parallel, conditional,
  inline checkpoint, and child mode

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update sqlx prepared queries

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: WacGraph Tooltip→Popover, simplify wacToFlow parsers

- Fix type error: Tooltip doesn't accept text snippet, use Popover
- Extract shared helpers for task matching and block collection
- Replace linear tasks.find() with Map lookups
- Remove mutable module-level counter

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: Box::pin WAC v2 output handler to prevent stack overflow

handle_python_job's async state machine was too large when combined
with handle_wac_v2_output. Box::pin heap-allocates the future.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: merge WAC v1 and v2 task decorators to preserve backward compat

The v2 @task decorator was shadowing the v1 one, breaking WAC v1
scripts that rely on HTTP-based dispatch via /workflow_as_code/ API.

The merged decorator handles three modes:
- v2: inside @workflow context → checkpoint/replay dispatch
- v1: WM_JOB_ID set, no @workflow → HTTP API dispatch + wait_job
- standalone: no Windmill env → execute function body directly

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: skip no_main_func detection for WAC v2 scripts in TS and Python parsers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: prevent empty/noop dispatch causing infinite requeue loop

- Validate steps.len() > 0 in WAC dispatch handler (issue 3)
- Replace noop StepSuspend throw with never-resolving promise so it
  can't reach the backend as an empty dispatch (issue 4)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: Python task wrapper now converts positional args to kwargs in v2 mode

Previously only **kwargs were passed to _next_step(), silently dropping
positional arguments. Extract shared _merge_args() helper used by both
v1 and v2 paths.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: replace unwrap() with proper error propagation in WAC arg serialization

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add workspace_id filter to v2_job queries in WAC dispatch

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: prevent race condition in WAC child dispatch

Restructure dispatch to save checkpoint + suspend parent + seed child
checkpoints in a single transaction BEFORE pushing child jobs. This
ensures a fast child can't complete before the parent is suspended.

Also wrap InlineCheckpoint save + running reset in a transaction to
prevent corrupted state on crash.

Use ULID for pre-generated child job IDs (consistent with rest of API).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: include step key and child job ID in WAC error propagation

Move step_key lookup before the success check so failed child errors
include which task failed, the child job ID, and the original error.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: document WAC determinism contract and step dispatch semantics

- Document that workflow functions must be deterministic across replays
- Document that WacStepDispatch.script/args are metadata, not dispatch targets
- Add comments on counter-based key allocation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: tighten WAC v2 detection to reduce false positives

Replace naive substring matching with line-aware checks that skip
comments and look for specific patterns:
- TS: import from "windmill-client" containing workflow/task
- Python: @workflow and @task decorators with wmill import

Extracted shared helpers in wac_executor.rs used by both executors.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: show failed steps in WacGraph when workflow completes with errors

When flowDone is true and a pending step isn't in completedSteps,
mark it as 'failed' instead of 'running'. The failed state CSS and
XCircle icon were already defined but never triggered.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: unsuspend and fail parent when WAC child push fails

Previously if a child push failed mid-batch, the parent remained
suspended with suspend = num_steps but fewer children, hanging until
the 14-day timeout. Now the push loop catches errors and unsuspends
the parent before returning the error.

Also adds source hash validation: if the script content changes between
replays, the job fails with a clear error instead of silently feeding
stale checkpoint data into wrong steps.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: clear suspend_until when unsuspending WAC parent

Set suspend_until = NULL alongside suspend = 0 in both the child
failure and all-children-complete paths, so the parent doesn't rely
on subtle pull query invariants to be re-picked-up.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add exhaustive edge case tests for WAC v2 SDK

fix: make TS task wrapper non-async to fix unawaited task flush

The async wrapper caused microtask-based thenable auto-resolution that
fired .then() and threw StepSuspend before _flushPending() could capture
unawaited steps — making the flush mechanism completely broken. Now the
thenable is returned directly without async wrapping. Backward compatible
with v1 (all code paths still return awaitables).

Tests added (59 TS + 66 Python) covering: full sequential lifecycle,
step after parallel, parallel after parallel, conditional on step result,
empty/single-task workflows, 10+ steps, falsy value preservation, inline
steps, mixed step/task, unawaited flush, child mode with parallel,
key determinism, large parallel groups, and complex mixed patterns.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: atomic checkpoint updates to prevent parallel child race condition

Replace read-modify-write pattern in handle_wac_child_completion with
atomic SQL operations:
- completed_steps merged via jsonb_set(... || jsonb_build_object(...))
  so concurrent children on different workers don't overwrite each other
- suspend counter decremented atomically with RETURNING to determine
  "all done" condition (instead of checking completed_steps in memory)
- suspend_until cleared in the same atomic decrement statement

Before this fix, two parallel children completing simultaneously could
both load the same checkpoint, each add their step, and save — the
second write would overwrite the first, silently losing a child result
and leaving the parent suspended forever.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: cancel already-pushed children on partial WAC dispatch failure

When pushing child jobs sequentially, if pushing child N fails, children
1..N-1 are already running. Previously the error handler only unsuspended
the parent, leaving orphaned children that would complete and corrupt the
checkpoint state (decrementing suspend on an already-unsuspended parent,
potentially causing duplicate step execution on re-run).

Now on partial failure:
1. Cancel all already-pushed children (prevents them from completing
   and corrupting checkpoint state)
2. Clear pending_steps from checkpoint (so parent doesn't think
   children are outstanding on re-run)
3. Then unsuspend parent (so the error propagates)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: skip WAC duration write and child check for non-WAC parents

The duration write to workflow_as_code_status was running for every
non-flow child with a parent (error handlers, success handlers,
run_script children), even though it was only intended for WAC jobs.

Add WHERE workflow_as_code_status IS NOT NULL to skip non-WAC parents
entirely. Piggyback RETURNING pending_steps.job_ids on the same query
so WAC v2 child completion needs zero extra DB round-trips on the
success path.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: seed child checkpoint in same transaction as push

The child checkpoint insert was happening before the child job was
pushed, violating the FK constraint on v2_job_status. Move it into
the push transaction so the job row exists and the child can't be
picked up before its checkpoint is ready.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: set running=false when WAC parent suspends for child dispatch

The parent job kept running=true after suspending, so workers wouldn't
pick it up when children completed and suspend reached 0. The parent
only advanced when the zombie job detector reset it (~90s). Now the
dispatch suspend sets running=false so the parent is immediately
eligible for pickup.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: WAC parent suspend/unsuspend lifecycle

Keep running=true when suspending the parent so the normal pull query
(WHERE running=false) never picks it up. Keep suspend_until non-null
when decrementing suspend to 0 so the suspended pull query
(WHERE suspend_until IS NOT NULL AND suspend<=0) picks it up.

Previously: setting running=false caused infinite restart loops because
the normal pull query has no suspend check and would immediately re-pick
the parent. Clearing suspend_until on the last child prevented the
suspended pull from ever seeing it, requiring the 90s zombie detector.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add approval primitive, flow child completion, timeline fixes for WAC v2

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add error propagation, task options, sleep, and parallel for WAC v2

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: fix python SDK tests to use name-based keys and add new test coverage

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address WAC v2 review findings (sleep timing, error marker, atomicity)

- Fix sleep using suspend=1 instead of 0 to enforce actual delay
- Add approval/sleep resume injection to Python executor
- Fix TS SDK concurrency_limit mapping (was reading wrong property)
- Namespace error marker as __wmill_error to avoid user data collision
- Wrap child completion SQL in transaction for atomicity
- Decrement suspend even when step key is missing (prevents hang)
- Expand TASK_RE to handle export const, let, var, generics
- Validate step key uniqueness before dispatch
- Log warning on checkpoint deserialization failure
- Remove unimplemented delete_after_use from SDKs
- Add TaskError exception class to Python SDK with diagnostic context
- Fix extra positional args handling and add functools.wraps
- Improve getParamNames to handle typed/destructured params

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* sqlx

* sqlx

* test: add WAC v1 e2e integration tests for TS and Python

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: revert fake test versions in typescript-client

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: remove unused WacGraph component and strip wacToFlow to isWorkflowAsCode

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: extract shared approval/sleep resume logic into wac_executor

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 19:39:24 +00:00
Ruben Fiszel
9e92445fae fix: preserve debouncing settings for post-preprocessing arg accumulation (#8191)
* fix: preserve debouncing settings for post-preprocessing arg accumulation

After preprocessing completes, store the flow's debouncing settings in
runnable_settings_handle on v2_job_queue so that maybe_apply_debouncing
can find them when the surviving job is pulled. Without this, the handle
is NULL and arg accumulation silently does nothing for flows with
preprocessors.

Also adds a debouncing badge in flow settings and 4 focused accumulation
tests covering scripts, flows without preprocessor, flows with
preprocessor (with and without the fix).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update sqlx prepared query for worker_flow.rs change

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 16:01:36 +00:00
Diego Imbert
7d9d16a6a3 feat: runScript inline for path and hash (#8019)
* runScript inline for path and hash

* Update backend/windmill-api/src/jobs.rs

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* refactor: unify inline script param structs and deduplicate closures

- Replace RunInlineScriptByPathFnParams and RunInlineScriptByHashFnParams
  with a single RunInlineScriptFnParams using InlineScriptTarget enum
- Collapse two nearly-identical closures in worker.rs into one
- Merge duplicate InlineByPath/InlineByHash into InlineScriptArgs
- Extract shared run_inline_script_inner helper in API handler
- Add missing check_scopes to run_inline_script_by_hash endpoint
- Fix duplicate lines from prior commit in run_inline_script_by_path
- Change tag from "inline_preview" to "inline" for deployed scripts

Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com>

* Integration tests

* rm

* rename feature to run_inline

* Run inline integration tests

* Fix tests

* check path scope

* openapi fix

* nits

* remove register_potential_assets_on_inline_execution

* unused variable

* refactor

* Pass user_db to check script permission

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com>
2026-02-27 13:59:14 +01:00
Ruben Fiszel
a00927b300 fix: preserve debouncing settings for flows with preprocessors (#8043)
* fix: preserve debouncing settings for flows with preprocessors

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Revert "fix: preserve debouncing settings for flows with preprocessors"

This reverts commit 3452c1657c.

* feat: add post-preprocessing debounce for flows with preprocessors

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* perf: reuse caller tx for push-time debounce and add stress test

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add exhaustive edge case tests for debouncing behavior

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* perf: optimize debouncing to reduce DB round-trips

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: replace legacy debounce compat with error logging

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: add debounce args accumulation tests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: add end-to-end test for maybe_apply_debouncing arg accumulation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: update sqlx offline query cache

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: make workmux pane commands idempotent for replay

Use git rev-parse --show-toplevel to resolve absolute paths instead of
relative cd, so commands work when replayed from within backend/frontend.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add e2e debounce test script for backend API

Comprehensive end-to-end test covering:
- Deploy & run scripts rapidly (no debounce with different args)
- Redeploy without lock in rapid succession
- Debounce with same args (should consolidate)
- Debounce with different args (should not consolidate)
- Custom debounce key behavior
- Git sync debounce + item aggregation (using glob-style ** path filter)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref to 0fede4b1086bc1456be9cc55b203228c979c5c5e

This commit updates the EE repository reference after PR #426 was merged in windmill-ee-private.

Previous ee-repo-ref: b5d333370603a6cc7ef70842354cf3be734241b4

New ee-repo-ref: 0fede4b1086bc1456be9cc55b203228c979c5c5e

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-02-22 14:16:52 +00:00
Ruben Fiszel
a91c532eca fix: make WM_FLOW_PATH available in flow step previews (#8042)
* fix: pass flow path in flow step preview for AI agent modules

JobLoader.runFlowPreview was missing the path parameter, causing
WM_FLOW_PATH to be unavailable when using the Run button on individual
flow steps. Test up to here worked correctly because it uses a
different code path (utils.svelte.ts) that already passed the path.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: make WM_FLOW_PATH available for rawscript/script step previews

Inject the flow path as `_flow_path` in the job args when running a
script preview from the flow editor. The SQL pull queries now use
COALESCE to fall back to this arg when no parent runnable path exists,
making WM_FLOW_PATH available for individual step "Run" previews.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: rename _flow_path args key to _FLOW_PATH

Match existing convention used by _ENTRYPOINT_OVERRIDE.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 21:56:32 +00:00
Ruben Fiszel
fd5ebc2fda fix: tag bunnative dependency jobs as bun instead of nativets (#8045)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-21 20:59:48 +00:00
hugocasa
aaa1b92300 fix: hash long dedicated worker tags (#7914)
* fix: hash long dedicated worker tags

* Update frontend/src/lib/components/dedicated_worker.ts

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
2026-02-11 19:29:44 +00:00
Ruben Fiszel
ea52a8b8ce refactor: move integration tests to subcrates to reduce recompilation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 14:04:54 +00:00
Ruben Fiszel
1456f73c16 nit test 2026-02-09 22:51:57 +00:00
Ruben Fiszel
4343b73485 fix: reduce DB pool contention by eliminating dual-connection patterns (#7861) 2026-02-09 22:48:29 +00:00
Ruben Fiszel
894d8a94f8 reuse existing transaction in push to reduce pool pressure (#7858)
* fix: reuse existing transaction in push instead of acquiring new connection

In push_inner, fetch_authed_from_permissioned_as was acquiring a new
connection from the pool to fetch job permissions, even though a
transaction was already open. Use fetch_authed_from_permissioned_as_conn
with the existing transaction instead, reducing pool pressure when many
jobs are pushed concurrently.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* improve contention

* improve contention

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 18:07:33 +00:00
Ruben Fiszel
5e63d31b92 nit 2026-02-08 15:52:19 +00:00
Ruben Fiszel
67c8aef9fa fix: box push() future to prevent stack overflow in nested async chains
The push() function generates a ~13KB async state machine that gets inlined
into every caller's future. In deeply nested async chains (e.g. flow execution),
this causes stack overflows. Boxing the future at the definition site via a thin
wrapper reduces each caller's stack footprint to a single pointer.

This also reverts the RUST_MIN_STACK workaround from CI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 15:51:44 +00:00
Ruben Fiszel
ccc4806b9a refactor: extract windmill-types crate from windmill-common (#7851)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 14:15:07 +00:00
Ruben Fiszel
c580572252 refactor: extract windmill-api-scripts and windmill-api-users subcrates (#7850)
* refactor: extract windmill-api-scripts and windmill-api-users subcrates

Split the monolithic windmill-api crate by extracting scripts.rs, flows.rs,
users.rs, and users_oss.rs into dedicated subcrates. This reduces incremental
rebuild times when editing these modules.

Changes:
- Create windmill-api-scripts crate (scripts.rs + flows.rs, ~4.3K lines)
- Create windmill-api-users crate (users.rs + users_oss.rs, ~2.4K lines)
- Move clear_schedule to windmill-queue (shared by scripts, flows, workspaces)
- Move username utilities (VALID_USERNAME, INVALID_USERNAME_CHARS,
  generate_instance_wide_unique_username) to windmill-common/src/usernames.rs
- Move COOKIE_DOMAIN, IS_SECURE, WithStarredInfoQuery, BulkDeleteRequest,
  WebhookShared to windmill-common for cross-crate access
- Original files in windmill-api become thin stubs with pub use re-exports
- EE-dependent route handlers remain in windmill-api (create_user, rename_user,
  set_password, reset_password, etc.)
- Feature forwarding for enterprise, private, parquet, no_auth

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: extract windmill-api-workspaces subcrate (Step 3)

Move workspaces.rs, workspaces_extra.rs, workspaces_oss.rs, and
workspaces_ee.rs into a new windmill-api-workspaces crate (~7K lines).

Routes that depend on windmill-api internals (AI copilot, teams,
tarball export, critical alerts, stripe) remain in the windmill-api
stub. The subcrate handles all other workspace management routes.

Also moved send_email_if_possible to windmill-common/email_oss.rs
to make it available across subcrates without circular deps.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

* refactor: extract windmill-api-groups subcrate (groups.rs + folders.rs)

Extract groups.rs (1,093 lines) and folders.rs (833 lines) into a new
windmill-api-groups subcrate. Both modules had clean dependencies on
already-extracted crates (windmill-api-auth, windmill-common,
windmill-api-workspaces). Also removes unused re-exports of
get_instance_username_or_create_pending and INVALID_USERNAME_CHARS
from windmill-api/src/utils.rs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: add granular_acls.rs and folder_history.rs to windmill-api-groups

Extract granular_acls.rs (395 lines) and folder_history.rs (68 lines) into
the windmill-api-groups subcrate. Both modules only depend on already-extracted
crates and belong to the same access-control domain as groups and folders.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove unused imports and dead code from subcrate extraction

- Remove unused BASE_URL import from lib.rs
- Remove workspaces_extra.rs and workspaces_oss.rs re-export stubs (no consumers in windmill-api)
- Remove dead send_email_if_possible OSS stub (callers moved to windmill-api-users)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

* chore: bust CI cargo cache for subcrate split

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: re-export BASE_URL for EE files that use crate::BASE_URL

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: forward no_auth feature to windmill-api-users, remove dead code

- Add "windmill-api-users/no_auth" to windmill-api's no_auth feature
  so the login bypass in users.rs:1600 activates correctly
- Remove dead send_email_if_possible from windmill-api-users/users_oss.rs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: re-enable cargo cache for backend tests

Cache was disabled to bust stale entries from before subcrate split.
Now that a clean build has run, re-enable for faster CI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: install mold+clang in CI workflows

The .cargo/config.toml uses mold linker for x86_64-linux.
Build scripts require linking even during cargo check.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: increase cargo test timeout to 30 min

Exit code 143 (SIGTERM) means the 20-min timeout was hit during
compilation without cache. Bump to 30 min as safety net.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: limit cargo build jobs to 4 to prevent OOM in CI

Exit code 143 (SIGTERM) after 8 min = OOM kill during compilation.
8 parallel LLVM codegen jobs exhaust memory on ubicloud-standard-8.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 12:12:03 +00:00
Ruben Fiszel
bbb397b6ad fix: improve scheduling reliability in extreme pool contention conditions (#7825)
* fix: reuse outer tx for schedule push in commit_completed_job

Instead of calling handle_maybe_scheduled_job(db) which opens its own
connections (peak=3), inline the schedule push using a savepoint on the
outer transaction. Auth is fetched via the tx connection using
fetch_authed_from_permissioned_as_conn, and push_scheduled_job runs
on a savepoint so failures roll back only the push, not the completion.

On push failure: savepoint rolls back, schedule is disabled on the outer
tx, and the zombie return path is preserved if disabling also fails.

Peak connections drop from 3 to 1 (or 2 on cold RunnableSettings cache).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

* fix: extract shared try_schedule_next_job to unify schedule push paths

Replace the two diverging schedule-push implementations (inlined in
commit_completed_job and standalone handle_maybe_scheduled_job) with a
single try_schedule_next_job that reuses the caller's transaction via
savepoints. This eliminates extra pool connection usage in the
worker_flow.rs path and ensures consistent retry/error semantics.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add failpoint markers to try_schedule_next_job

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: remove plan.md

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove inner retry loop from try_schedule_next_job, add caller-level retries

The 10-retry x 5s-sleep loop inside try_schedule_next_job held locks on
v2_job_completed/v2_job_queue for up to ~45s when running inside the
outer commit_completed_job transaction.

Now try_schedule_next_job makes a single attempt and returns errors to
the caller. Non-retryable errors (QuotaExceeded, NotFound) disable the
schedule immediately inside the function. Transient errors are returned
for the caller to retry:

- commit_completed_job path: outer backon retry (10x3s) retries the
  entire transaction including the schedule push, so no locks are held
  during sleep.
- handle_flow path: new backon retry (10x3s) wraps begin/push/commit
  with a fresh transaction per attempt.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: clear push_err after successful schedule disable to prevent stuck schedules

When try_schedule_next_job disables the schedule for non-retryable errors
(NotFound, QuotaExceeded), clear the error so the caller commits the tx
(persisting the disable). Previously, the error propagated up, causing the
tx to be dropped and rolling back the disable — leaving the schedule
permanently enabled but broken.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add 5s timeout on push_scheduled_job, clean up handle_flow error handling

- Add tokio::time::timeout(5s) around push_scheduled_job inside
  try_schedule_next_job to bound worst-case lock holding per attempt
- Remove unreachable QuotaExceeded/NotFound match arms in handle_flow
  (these errors are handled internally by try_schedule_next_job)
- Add report_error_to_workspace_handler_or_critical_side_channel in
  handle_flow when post-exhaustion schedule disable fails

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: return SchedulePushZombieError when both schedule push and disable fail

When handle_flow cannot push the next scheduled job AND cannot disable the
schedule, return a SchedulePushZombieError so the worker leaves the flow job
in the queue for zombie detection to restart. This prevents stuck schedules
where neither the next tick was pushed nor the schedule was disabled.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 13:16:32 +00:00
Ruben Fiszel
e655a06563 fix: prevent schedule pool connection exhaustion (#7821)
* fix: prevent schedule pool connection exhaustion

Schedules were disabling themselves after upgrading from v1.605.0 to
v1.614.0 due to pool connection deadlock. The root cause was
fetch_authed_from_permissioned_as acquiring a pool connection inside
push() while a transaction already held one, exhausting the pool under
pressure.

Fix: pre-compute Authed before db.begin() for the normal path, and
reuse the transaction connection via fetch_authed_from_permissioned_as_conn
for the on_behalf_of_email path. Peak pool usage drops from 2 to 1 for
all schedule push paths.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: replace pool exhaustion tests with comprehensive schedule push tests

Replace the 16 pool-specific tests with 19 tests covering all schedule
push code paths: script/flow scheduling, on_behalf_of_email (script and
flow), retry wrapping, duplicate detection, invalid timezone/cron/args,
script/flow not found, paused schedules, clock shift detection, disabled
schedule, path mismatch, push failure disabling schedule, and trigger
metadata.

Also simplify the obo_authed pattern in push_scheduled_job to use a
single match assignment instead of two bindings with .or() chaining.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-05 23:25:42 +00:00
claude[bot]
05fa3cd013 feat: add workspace setting to disable error handler for u/ scripts/flows (#7634)
* feat: add workspace setting to disable error handler for u/ scripts/flows

Add a new workspace setting `error_handler_muted_on_user_path` that allows
disabling the error handler for jobs related to scripts/flows that start
with the "u/" prefix (user scripts/flows).

Changes:
- Add migration to add `error_handler_muted_on_user_path` column to workspace_settings
- Update WorkspaceSettings struct and get_settings query
- Update EditErrorHandler struct and edit_error_handler API endpoint
- Update error handler cache to include the new setting
- Add skip logic in send_error_to_workspace_handler for u/ paths
- Add toggle in workspace settings UI

The implementation uses the existing cached settings to avoid additional
database queries, making it efficient.

Closes #7633

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* all

* update ee private

* combine migrations

* sqlx

* fix: migration fixes for auto_invite JSONB column

- Fix TEXT[] to JSONB conversion using to_jsonb()
- Add GIN index on auto_invite for efficient ? operator queries
- Add index cleanup to down migration

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* sqlx

* all

* all

* all

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-24 19:26:06 +00:00
Pyra
7249b82dba feat: better mixed versions handling (#7628) 2026-01-22 23:16:57 +00:00
Ruben Fiszel
73c4ce3012 feat: add workspace success handler with 60s TTL caching (#7598)
* feat: add workspace success handler with 60s TTL caching

Add a new workspace-level success handler that triggers when jobs complete
successfully, similar to the existing error handler. Key features:

- Database migration adds success_handler and success_handler_extra_args columns
- API endpoint for configuring success handler in workspace settings
- 60s TTL caching to avoid DB trips when checking handler existence
- Handler only triggers for top-level Script/Flow/SingleStepFlow jobs (not previews)
- Frontend UI with ScriptPicker and "Create from template" button for bunnative scripts
- EE warning alerts for CE users on success handler and critical alerts sections
- Backend tests for cache behavior and settings CRUD operations

The success handler receives: path, email, result, job_id, is_flow, workspace_id,
and started_at. It runs as the g/success_handler group.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: update RawCode struct in tests to use new field names

Update test to use ConcurrencySettingsWithCustom and DebouncingSettings
instead of the deprecated individual fields.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Update backend/windmill-api/src/workspaces.rs

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* test: improve success handler test to verify job creation

Update test_success_handler_triggered_on_success to actually verify that:
- A success handler job is created in v2_job table
- The job has the correct runnable_path (configured script)
- The job runs as success_handler@windmill.dev
- The job has correct parent_job and root_job references
- The trigger starts with 'success.handler.'

This provides better test coverage for the success handler feature.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: fix success handler test and root_job parameter

- Fix test to use JobPayload::ScriptHash (JobKind::Script) instead of
  JobPayload::Code (JobKind::Preview) since previews don't trigger handlers
- Add required 'lock' column to script inserts for deployed scripts
- Fix duplicate lines in workspaces.rs
- Set root_job correctly in push_success_handler
- Update test assertions to match actual behavior (root_job optimization)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* sqlx

* refactor: reduce duplication in workspace handler cache logic

- Extract database queries into fetch_error_handler_from_db and
  fetch_success_handler_from_db helper functions
- Use shared WORKSPACE_HANDLER_CACHE_TTL_SECONDS constant for 60s TTL
- Reduces code from ~116 lines to ~54 lines while maintaining identical behavior

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Add cache propagation warning to workspace success handler UI

Note that changes may take up to 60 seconds to propagate due to caching.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
2026-01-17 13:24:58 +00:00
Pyra
b31d8dffc3 feat(git-sync): sync jobs debouncing for greatly improved perf (#7489)
* feat(git-sync): sync jobs debouncing [merge-ee-first]

* add compat

* ee repo

Signed-off-by: pyranota <pyra@duck.com>

* nit

Signed-off-by: pyranota <pyra@duck.com>

* change is_script_meets_min_version

Signed-off-by: pyranota <pyra@duck.com>

* update version

Signed-off-by: pyranota <pyra@duck.com>

* chore: update ee-repo-ref to bc904859dd66c55ebad002e8526103c73de841cd

This commit updates the EE repository reference after PR #371 was merged in windmill-ee-private.

Previous ee-repo-ref: 346d38e91776e987f88866543adbf78a9c721073

New ee-repo-ref: bc904859dd66c55ebad002e8526103c73de841cd

Automated by sync-ee-ref workflow.

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-01-07 04:07:26 +00:00
Pyra
1bbf8c251e nit: better logs for debouncing (#7501)
* nit: better logs for debouncing

Signed-off-by: pyranota <pyra@duck.com>

* ee ref

Signed-off-by: pyranota <pyra@duck.com>

* ee ref

Signed-off-by: pyranota <pyra@duck.com>

* remove default featrues

Signed-off-by: pyranota <pyra@duck.com>

* chore: update ee-repo-ref to ea468d0b673257c694203517b0c5df52d9cb70dd

This commit updates the EE repository reference after PR #373 was merged in windmill-ee-private.

Previous ee-repo-ref: 1196ee50bfdbc8c9fd5220d6e324334dd41293a5

New ee-repo-ref: ea468d0b673257c694203517b0c5df52d9cb70dd

Automated by sync-ee-ref workflow.

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-01-06 18:16:53 +00:00
Pyra
3cf5604d7b fix: properly construct concurrency key (#7504)
Signed-off-by: pyranota <pyra@duck.com>
2026-01-06 18:03:07 +00:00
Pyra
0f51f9702c fix: clone script by path instead of hash (#7439)
* fix: clone script by path instead of hash

Signed-off-by: pyranota <pyra@duck.com>

* Update backend/windmill-common/src/scripts.rs

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
2025-12-23 16:05:58 +00:00
Pyra
9d698dabb4 feat: v2 job debouncing (#7411)
* rework everything again

Signed-off-by: pyranota <pyra@duck.com>

* updcate sqlx

Signed-off-by: pyranota <pyra@duck.com>

* update ref

Signed-off-by: pyranota <pyra@duck.com>

* fix things

Signed-off-by: pyranota <pyra@duck.com>

* fix function

Signed-off-by: pyranota <pyra@duck.com>

* final fixes

Signed-off-by: pyranota <pyra@duck.com>

* update sqlx

Signed-off-by: pyranota <pyra@duck.com>

* fix script creation

Signed-off-by: pyranota <pyra@duck.com>

* address todo

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* remove dbg

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fix

Signed-off-by: pyranota <pyra@duck.com>

* fixups

Signed-off-by: pyranota <pyra@duck.com>

* fix cargo.toml

Signed-off-by: pyranota <pyra@duck.com>

* update ee repo

Signed-off-by: pyranota <pyra@duck.com>

* fix ci

Signed-off-by: pyranota <pyra@duck.com>

* nit

Signed-off-by: pyranota <pyra@duck.com>

* ref

Signed-off-by: pyranota <pyra@duck.com>

* fix

Signed-off-by: pyranota <pyra@duck.com>

* ee repo

Signed-off-by: pyranota <pyra@duck.com>

* sqlx

Signed-off-by: pyranota <pyra@duck.com>

* ee ref

Signed-off-by: pyranota <pyra@duck.com>

* remove dbg

Signed-off-by: pyranota <pyra@duck.com>

* sqlx

Signed-off-by: pyranota <pyra@duck.com>

* feat: v2 job debouncing

Signed-off-by: pyranota <pyra@duck.com>

* v2 debouncing

Signed-off-by: pyranota <pyra@duck.com>

* compat

Signed-off-by: pyranota <pyra@duck.com>

* fix all

Signed-off-by: pyranota <pyra@duck.com>

* remove legacy logic

Signed-off-by: pyranota <pyra@duck.com>

* nit

Signed-off-by: pyranota <pyra@duck.com>

* finish

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* stage import

Signed-off-by: pyranota <pyra@duck.com>

* remove unused import

Signed-off-by: pyranota <pyra@duck.com>

* remove comments

Signed-off-by: pyranota <pyra@duck.com>

* ee ref

Signed-off-by: pyranota <pyra@duck.com>

* ee ref

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* Update backend/windmill-queue/src/jobs.rs

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* Update backend/windmill-common/src/runnable_settings/settings.rs

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* Update frontend/src/lib/components/flows/DebounceLimit.svelte

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* fix nits

Signed-off-by: pyranota <pyra@duck.com>

* force no compat in ci

Signed-off-by: pyranota <pyra@duck.com>

* better default key handling

Signed-off-by: pyranota <pyra@duck.com>

* simplify

Signed-off-by: pyranota <pyra@duck.com>

* sqlx

Signed-off-by: pyranota <pyra@duck.com>

* optimize

Signed-off-by: pyranota <pyra@duck.com>

* sqlx

Signed-off-by: pyranota <pyra@duck.com>

* fix flow step

Signed-off-by: pyranota <pyra@duck.com>

* add openapi

Signed-off-by: pyranota <pyra@duck.com>

* fix ci

Signed-off-by: pyranota <pyra@duck.com>

* ee ref

Signed-off-by: pyranota <pyra@duck.com>

* rename WMDEBUG flag

Signed-off-by: pyranota <pyra@duck.com>

* ee ref

Signed-off-by: pyranota <pyra@duck.com>

* remove gate for is_flow_step on pull

Signed-off-by: pyranota <pyra@duck.com>

* remove dead code

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* chore: update ee-repo-ref to 71d872d27de44227b82f75a9c0879d8428684880

This commit updates the EE repository reference after PR #369 was merged in windmill-ee-private.

Previous ee-repo-ref: 4c7642dc9f7f4467fc055136bd56558b66ef2d54

New ee-repo-ref: 71d872d27de44227b82f75a9c0879d8428684880

Automated by sync-ee-ref workflow.

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-23 13:24:04 +00:00
Pyra
6723a6a04b fix: better timeouts on job pull (#7434)
* fix: better timeouts on job pull

Signed-off-by: pyranota <pyra@duck.com>

* increase timeouts

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
2025-12-22 21:44:35 +00:00
hugocasa
f89fb292da fix(backend): put for loop itered in a separate table (#7419)
* fix(backend): put for loop itered in a separate table

* Update SQLx metadata

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-19 19:46:46 +00:00
hugocasa
a6993823af feat: restart flow from step with different flow version (#7409)
* feat: restart flow from step with different flow version

* fix tests

* fix tests
2025-12-18 20:27:51 +00:00
Pyra
fe56191422 feat(internal): runnable settings (#7298)
* rework everything again

Signed-off-by: pyranota <pyra@duck.com>

* updcate sqlx

Signed-off-by: pyranota <pyra@duck.com>

* update ref

Signed-off-by: pyranota <pyra@duck.com>

* fix things

Signed-off-by: pyranota <pyra@duck.com>

* fix function

Signed-off-by: pyranota <pyra@duck.com>

* final fixes

Signed-off-by: pyranota <pyra@duck.com>

* update sqlx

Signed-off-by: pyranota <pyra@duck.com>

* fix script creation

Signed-off-by: pyranota <pyra@duck.com>

* address todo

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* remove dbg

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fix

Signed-off-by: pyranota <pyra@duck.com>

* fixups

Signed-off-by: pyranota <pyra@duck.com>

* fix cargo.toml

Signed-off-by: pyranota <pyra@duck.com>

* update ee repo

Signed-off-by: pyranota <pyra@duck.com>

* fix ci

Signed-off-by: pyranota <pyra@duck.com>

* nit

Signed-off-by: pyranota <pyra@duck.com>

* ref

Signed-off-by: pyranota <pyra@duck.com>

* fix

Signed-off-by: pyranota <pyra@duck.com>

* ee repo

Signed-off-by: pyranota <pyra@duck.com>

* sqlx

Signed-off-by: pyranota <pyra@duck.com>

* ee ref

Signed-off-by: pyranota <pyra@duck.com>

* remove dbg

Signed-off-by: pyranota <pyra@duck.com>

* sqlx

Signed-off-by: pyranota <pyra@duck.com>

* chore: update ee-repo-ref to 505eadbff32d102ea5245a2bef88ce6f1bb95395

This commit updates the EE repository reference after PR #348 was merged in windmill-ee-private.

Previous ee-repo-ref: 195243e56cc0eab55f8890fa57297206bfe2c18c

New ee-repo-ref: 505eadbff32d102ea5245a2bef88ce6f1bb95395

Automated by sync-ee-ref workflow.

* ci: force runnable settings

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-16 21:17:06 +00:00
Ruben Fiszel
431074d249 fix: add history to raw app builder (#7362)
* appHistory

* appHistory

* all

* all

* all

* all

* all

* all

* all

* all

* improvements
2025-12-13 13:32:52 +00:00
Ruben Fiszel
651681b7ef fix(backend): add presigned url support for object storage (#7328)
* presigned

* all

* all

* all

* all

* all

* all

* all

* nit

* nit

* ee-ref

* presigned

* presigned
2025-12-10 15:35:26 +00:00
hugocasa
eb284dfabd feat: triggers suspended mode (#7297)
* first commit

* base

* ok

* remove print

* rm packe json

* fix discard

* fix

* update .sqlx

* update

* update ref

* jobs update

* nits

* update ref

* big

* update sqlx

* fix

* update

* nits and fix

* nits

* ok

* sqlx + fix import

* better

* nit

* suspended mode draft

* UI and http triggers

* nits

* done for http triggers

* feat: better retry/error handler check

* generalize to all triggers

* fix merge conflicts

* fix listener

* nits

* nit

* fix merge conflict

* update ee ref

---------

Co-authored-by: dieriba <dieriba.pro@gmail.com>
2025-12-08 16:20:49 +00:00
Diego Imbert
7b1a8b0dce fix: improve S3 etag caching (#7301) 2025-12-05 18:01:27 +00:00
Pyra
1dfb0f0078 refactor: move concurrency and debouncing settings to their own structs (#7286)
* move concurrency and debouncing settings to it's own structs

Signed-off-by: pyranota <pyra@duck.com>

* cleanup queue::jobs::pull

Signed-off-by: pyranota <pyra@duck.com>

* done

Signed-off-by: pyranota <pyra@duck.com>

* migrated the rest

Signed-off-by: pyranota <pyra@duck.com>

* implement todo!

* impmenet todo! v2

* more cleanup

* fix all

Signed-off-by: pyranota <pyra@duck.com>

* remove unrelated code

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* do not serialize rawcode

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fixes

Signed-off-by: pyranota <pyra@duck.com>

* update ee repo ref

Signed-off-by: pyranota <pyra@duck.com>

* update ee repo

Signed-off-by: pyranota <pyra@duck.com>

* fix tests

Signed-off-by: pyranota <pyra@duck.com>

* Update ee-repo-ref.txt

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-04 10:11:54 +00:00
claude[bot]
e9f13065bf fix: show related job when deployment is in progress (#7294)
* feat: show related job when deployment is in progress

- Added job_id column to deployment_metadata table to track current deployment jobs
- Updated backend to store job_id when creating dependency jobs for scripts and flows
- Modified deployment status API endpoints to include job_id in responses
- Updated frontend to display clickable job link in "Deployment in progress" badge
- Added OpenAPI schema updates for new job_id field

Resolves #7293

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* update

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-12-04 09:50:07 +00:00
Pyra
d38c96db36 feat: workspace dependencies (#7124)
* commit raw requirements

Signed-off-by: pyranota <pyra@duck.com>

* raw requirements

Signed-off-by: pyranota <pyra@duck.com>

* implement `parse_annotation`

Signed-off-by: pyranota <pyra@duck.com>

* more progress on wdeps

Signed-off-by: pyranota <pyra@duck.com>

* more progress

Signed-off-by: pyranota <pyra@duck.com>

* fixes

Signed-off-by: pyranota <pyra@duck.com>

* more progress

Signed-off-by: pyranota <pyra@duck.com>

* fixes

Signed-off-by: pyranota <pyra@duck.com>

* cli improvements + raw deps

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fix python versions

Signed-off-by: pyranota <pyra@duck.com>

* progress

Signed-off-by: pyranota <pyra@duck.com>

* update :)

Signed-off-by: pyranota <pyra@duck.com>

* add MaybeLock

Signed-off-by: pyranota <pyra@duck.com>

* go WIP

Signed-off-by: pyranota <pyra@duck.com>

* fix python ignoring py version from requirements

Signed-off-by: pyranota <pyra@duck.com>

* optimize php

Signed-off-by: pyranota <pyra@duck.com>

* require admin to alter

Signed-off-by: pyranota <pyra@duck.com>

* fix(cli): flow generateLocks raw deps

Signed-off-by: pyranota <pyra@duck.com>

* progress in checklist

Signed-off-by: pyranota <pyra@duck.com>

* fix agent workers

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* nit: remove default features

Signed-off-by: pyranota <pyra@duck.com>

* oh-wow

Signed-off-by: pyranota <pyra@duck.com>

* remove dbg!

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* add indexes

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* remove todos

Signed-off-by: pyranota <pyra@duck.com>

* fix cli

Signed-off-by: pyranota <pyra@duck.com>

* add debug flag

Signed-off-by: pyranota <pyra@duck.com>

* cli: remove noise

Signed-off-by: pyranota <pyra@duck.com>

* fix cli

Signed-off-by: pyranota <pyra@duck.com>

* remove todos

Signed-off-by: pyranota <pyra@duck.com>

* trigger deps correctly

Signed-off-by: pyranota <pyra@duck.com>

* fix frontend

Signed-off-by: pyranota <pyra@duck.com>

* fix frontend again

Signed-off-by: pyranota <pyra@duck.com>

* finally fix frontend

Signed-off-by: pyranota <pyra@duck.com>

* ee repo ref

Signed-off-by: pyranota <pyra@duck.com>

* fix all

Signed-off-by: pyranota <pyra@duck.com>

* more fixes...

Signed-off-by: pyranota <pyra@duck.com>

* remove test

Signed-off-by: pyranota <pyra@duck.com>

* Update backend-test.yml

* comment out legacy test

Signed-off-by: pyranota <pyra@duck.com>

* fix ci

Signed-off-by: pyranota <pyra@duck.com>

* fix ci?

Signed-off-by: pyranota <pyra@duck.com>

* comment out thing

Signed-off-by: pyranota <pyra@duck.com>

* ignore test

Signed-off-by: pyranota <pyra@duck.com>

* ci

Signed-off-by: pyranota <pyra@duck.com>

* base fixture

Signed-off-by: pyranota <pyra@duck.com>

* fix regression

Signed-off-by: pyranota <pyra@duck.com>

* fix docs links

Signed-off-by: pyranota <pyra@duck.com>

* update min version

Signed-off-by: pyranota <pyra@duck.com>

* simplify

* implement cache for get_latest

Signed-off-by: pyranota <pyra@duck.com>

* move to workspace settings

Signed-off-by: pyranota <pyra@duck.com>

* sqlx + migration

Signed-off-by: pyranota <pyra@duck.com>

* more migrations

Signed-off-by: pyranota <pyra@duck.com>

* use box pin

Signed-off-by: pyranota <pyra@duck.com>

* nit

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-11-28 16:00:11 +00:00
hugocasa
93c34c6d76 feat: flow for loop squashing (#7107)
* feat: flow for loop optimization

* fix: make dedicated flows work with flow nodes

* fix: transform dedicated inputs + better error handling + differentiate squash from same worker

* fix builds

* fix build

* fix build

* make dedicated worker private

* update ee ref

* frontend nits

* nit

* add trace to dedicated worker

* update ee ref

* Update ee-repo-ref.txt

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-11-25 17:25:34 +01:00
Ruben Fiszel
478e19379f buffer cloud hosted usage 2025-11-18 09:40:57 +00:00