Commit Graph

1640 Commits

Author SHA1 Message Date
hugocasa
9686608355 fix(backend): decimal between 0 and -1 in mssql (#8051) 2026-02-23 09:38:22 +00:00
Ruben Fiszel
a00927b300 fix: preserve debouncing settings for flows with preprocessors (#8043)
* fix: preserve debouncing settings for flows with preprocessors

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Revert "fix: preserve debouncing settings for flows with preprocessors"

This reverts commit 3452c1657c.

* feat: add post-preprocessing debounce for flows with preprocessors

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* perf: reuse caller tx for push-time debounce and add stress test

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add exhaustive edge case tests for debouncing behavior

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* perf: optimize debouncing to reduce DB round-trips

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: replace legacy debounce compat with error logging

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: add debounce args accumulation tests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: add end-to-end test for maybe_apply_debouncing arg accumulation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: update sqlx offline query cache

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: make workmux pane commands idempotent for replay

Use git rev-parse --show-toplevel to resolve absolute paths instead of
relative cd, so commands work when replayed from within backend/frontend.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add e2e debounce test script for backend API

Comprehensive end-to-end test covering:
- Deploy & run scripts rapidly (no debounce with different args)
- Redeploy without lock in rapid succession
- Debounce with same args (should consolidate)
- Debounce with different args (should not consolidate)
- Custom debounce key behavior
- Git sync debounce + item aggregation (using glob-style ** path filter)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref to 0fede4b1086bc1456be9cc55b203228c979c5c5e

This commit updates the EE repository reference after PR #426 was merged in windmill-ee-private.

Previous ee-repo-ref: b5d333370603a6cc7ef70842354cf3be734241b4

New ee-repo-ref: 0fede4b1086bc1456be9cc55b203228c979c5c5e

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-02-22 14:16:52 +00:00
Ruben Fiszel
9eb15312f6 feat: add .npmrc support for private npm registries (#8039)
* feat: add .npmrc support for private npm registries

Add a new `npmrc` instance setting that accepts full .npmrc file content
for configuring private npm registries. Works with bun (native .npmrc
support since 1.1.18), deno (native .npmrc support in 2.x), and the npm
proxy (parses default registry + auth token from .npmrc).

Legacy `npm_config_registry` and `bunfig_install_scopes` fields are now
hidden when empty, so new users only see the .npmrc field. Also fixes a
pre-existing race condition where gen_bunfig was called after
start_child_process.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 05:29:36 +00:00
Diego Imbert
e3460aba89 Fix duckdb tests (#8035) 2026-02-20 17:34:23 +00:00
hugocasa
37c9acb232 feat: dedicated nativets (#8021)
* feat: dedicated nativets

* review nits

* prewarm isolates

* ref

* chore: update ee-repo-ref to 5f8105b808f3f0186fdf5132d2ee602d8a14aa17

This commit updates the EE repository reference after PR #424 was merged in windmill-ee-private.

Previous ee-repo-ref: b7906acabb8ce359230bbd3e30dbb3bba4c42adb

New ee-repo-ref: 5f8105b808f3f0186fdf5132d2ee602d8a14aa17

Automated by sync-ee-ref workflow.

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-02-20 16:28:40 +00:00
Diego Imbert
f05b00aa8a Nit better test and comment (#8030) 2026-02-20 12:06:05 +00:00
Diego Imbert
ff6c49b43e Fix remove_comments with multi bytes (#8029)
* Fix remove_comments with multi bytes

* change names
2026-02-20 11:53:10 +00:00
Diego Imbert
90b1a7a531 fix: Fix DuckDB incorrect pg password encoding (#8028) 2026-02-20 10:35:27 +00:00
Ruben Fiszel
6bf544f507 refactor: extract object store into dedicated crate with filesystem backend (#7996)
* refactor: extract object store code into windmill-object-store crate with filesystem backend

Consolidate all object_store-dependent code from windmill-common into a new
windmill-object-store crate. Add a filesystem-backed object store implementation
using LocalFileSystem for dev/testing without cloud credentials. Includes 30
comprehensive tests covering render_endpoint, lfs_to_object_store_resource,
duckdb_connection_settings, error mapping, and filesystem-backed integration tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

* all

* all

* all

* fix: fix raw_app hardcoded path, add missing ObjectStoreResource import, and add tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: move S3ModeFormat to windmill-types, make windmill-parser-sql optional, restore debug logs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 12:24:45 +00:00
hugocasa
90fa5b3ced native mode nits (#7981)
* native mode UI nits

* accept flow jobs on native workers

* limit native mode to non-dep jobs + flow tag infobox
2026-02-17 16:32:52 +00:00
hugocasa
535e108cbf feat: native mode (#7939)
* feat: native mode

* improve

* fix build

* review fixes

* tracing nit
2026-02-16 23:36:41 +00:00
Ruben Fiszel
423e07376b feat: gate private registry configuration behind enterprise feature (#7953)
Add read_ee_registry<T> helper that returns None and emits a job log
in CE builds when a private registry is configured. Applied to all
language executors: pip, npm/bun, deno, cargo, go, maven, nuget,
powershell, and ruby. Maven settings.xml file write in monitor.rs
is also gated.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 14:02:14 +00:00
Ruben Fiszel
e4a34d031b feat: handle $var: and $res: in arrays for transform_json_value (#7949)
* feat: handle $var: and $res: in arrays for transform_json_value

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: resolve $var/$res arrays inside objects in transform_json_value

Replace top_level: bool with depth: u8 to fix the main bug where arrays
nested inside objects were never resolved. The Object arm recursed with
top_level: false, which caused the Array arm guard to reject the value.

With depth <= 2, arrays inside objects (depth 1-2) are now processed.
Also: transform all array elements unconditionally (handles $encrypted:
in worker, reserved $vars, nested structures), and warn on >1000 items.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 08:35:06 +00:00
Zhiyuan Zheng
8410b59a8f fix(go): preserve proxy envs for go mod tidy/download (#7946) 2026-02-15 07:46:15 +00:00
Ruben Fiszel
581dde8d0b feat: add maven settings.xml support for java private registries (#7940)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 16:57:30 +00:00
Ruben Fiszel
a9e4a5c8e7 windows build nit 2026-02-13 15:16:13 +00:00
Ruben Fiszel
7d88676b15 rust nits 2026-02-13 14:13:11 +00:00
Ruben Fiszel
50f04fe8d4 rust nits 2026-02-13 14:08:42 +00:00
Ruben Fiszel
e144432a16 fix: rust + java works with just /tmp mounted 2026-02-13 14:03:45 +00:00
Ruben Fiszel
791cb3e225 fix: java + rust only relies on /tmp, + https proxy improvement for java 2026-02-13 10:47:59 +00:00
Pyra
d6c72df99a fix: conditionally skip relock on dep job (#7860)
* perf: conditionally skip relock on dep job

Signed-off-by: pyranota <pyra@duck.com>

* fix

* stage

Signed-off-by: pyranota <pyra@duck.com>

* final

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* add index

Signed-off-by: pyranota <pyra@duck.com>

* fix bug

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* final (for real)

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* merge migrations into one

Signed-off-by: pyranota <pyra@duck.com>

* add test

Signed-off-by: pyranota <pyra@duck.com>

* fix test

Signed-off-by: pyranota <pyra@duck.com>

* fix test

Signed-off-by: pyranota <pyra@duck.com>

* update sqlx

Signed-off-by: pyranota <pyra@duck.com>

* remove unused

Signed-off-by: pyranota <pyra@duck.com>

* sqlx

Signed-off-by: pyranota <pyra@duck.com>

* increase timeout

Signed-off-by: pyranota <pyra@duck.com>

* optimize the hell out of tests

Signed-off-by: pyranota <pyra@duck.com>

* simplify test

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-02-12 23:02:36 +00:00
Ruben Fiszel
2646629194 feat: add force_sandboxing global setting and #sandbox bash annotation (#7816)
* feat: add force_sandboxing global setting and #sandbox bash annotation

Add a new global setting `force_sandboxing` to enable nsjail sandboxing
for all jobs, overriding the DISABLE_NSJAIL environment variable.
Also add a #sandbox annotation for bash scripts to enable sandboxing
on a per-script basis.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: decouple nsjail probing from DISABLE_NSJAIL and apply force_sandboxing to all executors

NSJAIL_AVAILABLE now always probes for the nsjail binary at startup
regardless of the DISABLE_NSJAIL policy flag, fixing #sandbox annotation
and force_sandboxing being unusable in default (DISABLE_NSJAIL=true)
configuration. All language executors now use is_sandboxing_enabled()
instead of checking DISABLE_NSJAIL directly, so force_sandboxing
applies to all job types, not just bash.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: clarify force_sandboxing and DISABLE_NSJAIL relationship

Neither setting overrides the other — sandboxing is enabled when either
force_sandboxing=true OR DISABLE_NSJAIL=false. Updated frontend
description and backend comments to reflect this.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

* all

* revert typo

* all

* all

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-12 18:20:22 +00:00
Ruben Fiszel
e2c28e42db fix: add private registries support for RUST + java home nit 2026-02-12 14:23:52 +00:00
Alexander Petric
7df4aa4fec feat: /health endpoints (#7727)
* feat: /health endpoints

* Update SQLx metadata

* use different paths + caching + prometheus

* update prom metrics

* all

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-02-12 10:53:27 +00:00
centdix
d95e4db8f3 feat(ai): add AWS bedrock session token support (#7908)
* Add AWS Bedrock session token support in API and worker

* Add Bedrock auth mode integration tests for AI agents

* Split Bedrock integration test env vars for IAM and session creds

* cleaning

* Add masked Bedrock bearer-token debug info logs

* Revert "Add masked Bedrock bearer-token debug info logs"

This reverts commit 6b2fc5e7c2d5b1c6db81f416a4439941a084108c.

* cleaning
2026-02-11 19:29:58 +00:00
Devdatta Talele
5effb87a36 fix: strip unsupported schema fields for Google AI (#7894)
Closes #7759
2026-02-11 13:26:08 +00:00
centdix
f22eb964e4 feat(ai): support 1M context window for Anthropic resources (#7891)
* feat(ai): support 1M context window for Anthropic resources

Add `enable_1m_context` boolean field to Anthropic resource configuration.
When enabled (and not using Vertex AI), sends the `anthropic-beta: context-1m-2025-08-07`
header in both the API proxy layer and the AI agent worker layer.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(ai): add serde alias for enable_1M_context DB field name

The resource_type schema uses `enable_1M_context` (uppercase M) but
serde only matched `enable_1m_context` and `enable1mContext`, causing
the field to always deserialize as false.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 12:01:46 +00:00
Pyra
097d9288c5 feat(uv): index resolve strategy (#7885)
* fix

* fix

Signed-off-by: pyranota <pyra@duck.com>

* Update backend/windmill-worker/src/python_executor.rs

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* Update backend/windmill-worker/src/python_executor.rs

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
2026-02-11 11:18:51 +00:00
Ruben Fiszel
9e8d77436b dotNet (#7900) 2026-02-11 10:23:31 +00:00
Ruben Fiszel
4c87e7ac2e fix: retry js eval up to 3 times on timeout from slow DB (#7890)
When eval_timeout_quickjs hits the timeout (typically due to slow DB
result retrieval), retry up to 2 more times with a 5s interval between
attempts. Non-timeout errors are returned immediately without retry.

Also extract the eval timeout duration as EVAL_TIMEOUT_MS const (set to
20000ms, up from 10000ms) in windmill-jseval.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Alexander Petric <alpetric@users.noreply.github.com>
2026-02-10 17:50:41 +00:00
centdix
6272cd17a4 feat(aiagent): add prompt caching for Anthropic models (#7878)
* feat: add prompt caching support for Anthropic API

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* exclude vertex

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-10 17:42:49 +00:00
Ruben Fiszel
af02e200a5 add benchmark variants and fix db pool connection starvation (#7884)
Add new BENCHMARK_KIND variants (sequentialflow, scriptlogs, concurrencylimit,
concurrencykey, mixed, mixed_no_cc) for targeted performance testing. Fix shared
iteration counting across workers using a global atomic counter. Add job_perms
inserts and queue diagnostics for benchmark mode.

Move db connection setup to dedicated module and drop the initial connection pool
before creating the main one, preventing connection starvation when PostgreSQL
max_connections is low.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 16:41:47 +00:00
Ruben Fiszel
45980f0220 resolve Windows build warnings treated as errors (#7870)
* fix: resolve Windows build warnings treated as errors

- Gate UV_PATH import behind #[cfg(unix)] in python_versions.rs
- Remove unused tokio::time::sleep import in worker.rs (use fully qualified path)
- Fix unused `file` variable warnings in ansible_executor.rs on Windows

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* ci: add Windows cargo check workflow

Runs cargo check with ee_windows features on push to backend/**
using the blacksmith-16vcpu-windows-2025 runner.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* ci: add cargo check step to Windows build, remove separate check workflow

Add a cargo check step with -D warnings before the full build to fail
fast on any warnings. Remove the separate windows-check.yml workflow.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 09:58:56 +00:00
Ruben Fiszel
1456f73c16 nit test 2026-02-09 22:51:57 +00:00
Ruben Fiszel
4343b73485 fix: reduce DB pool contention by eliminating dual-connection patterns (#7861) 2026-02-09 22:48:29 +00:00
hugocasa
b1d6ac91bd fix: restart after empty branchone + improve UI (#7838)
* improve flow restart UI

* fix: better restart UI + fix restart avec empty branch one
2026-02-09 18:13:18 +00:00
Ruben Fiszel
ff70a4e9d1 fix: parse Python datetime.datetime and datetime.date type annotations (#7856)
* fix: parse Python datetime.datetime and datetime.date type annotations correctly

The Python parser only matched ExprKind::Name for type annotations, so
`datetime.datetime` (an Attribute expression) silently fell through to
Typ::Unknown and no datetime picker was shown in the UI.

- Extend parse_expr to resolve `datetime.*` attribute access (alongside
  the existing `wmill.*` handling)
- Add Typ::Date variant for `datetime.date` → JSON schema format "date"
- Update python worker to import and convert `date.fromisoformat()`
- Update argSigToJsonSchemaType, AI types, schema validation, and SQL
  datatype wasm for the new Date variant

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

* all

* all

* all

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 15:17:26 +00:00
Ruben Fiszel
623407de6a chore: make rquickjs, kube, otel proxy deps optional behind feature flags
- rquickjs: gated behind `quickjs` feature in windmill-jseval, propagated through windmill-worker/windmill-api, added to oss_core
- windmill-autoscaling: made optional in windmill-api (was unconditional), enabled via enterprise feature
- opentelemetry-proto, prost, hudsucker, rcgen, hyper-http-proxy, hyper-tls, hyper-util: made optional in windmill-worker, enabled via enterprise feature

This significantly reduces compilation time for vanilla `cargo check` without features.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 16:13:16 +00:00
Ruben Fiszel
ccc4806b9a refactor: extract windmill-types crate from windmill-common (#7851)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 14:15:07 +00:00
Ruben Fiszel
bb57f8cd29 remove unecessary deps for vanilla cargo run 2026-02-08 14:08:36 +00:00
Ruben Fiszel
3f68e3a58e refactor: isolate deno_core into windmill-runtime-nativets subcrate (#7848)
* refactor: isolate deno_core into windmill-runtime-nativets subcrate

Remove deno_core from flow eval and isolate nativets V8 runtime into a
dedicated subcrate so deno_core compilation no longer blocks
windmill-worker or windmill-api.

- Create windmill-jseval crate: QuickJS-based JS eval for flow
  expressions and batch rerun, extracted from windmill-worker
- Create windmill-runtime-nativets crate: all deno_core/V8 deps and
  nativets script execution, with build.rs snapshot generation
- Simplify windmill-worker: remove all deno_* direct deps, empty
  build.rs, gate nativets behind optional dep
- Update windmill-api: use windmill-jseval for batch rerun instead of
  deno_core, remove deno_core feature entirely
- Add nativets integration tests (nativets_jobs.rs) and parallel
  stress test (nativets_stress.rs, 8 workers x 200 jobs)
- Remove dead code: deno flow eval path, USE_QUICKJS env var,
  parity tests (replaced with 63 standalone expected-value tests)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review feedback for deno_core isolation

- Deduplicate unsafe_raw() into windmill-common/src/utils.rs (single source)
- Delete orphaned runtime.js and windmill-client.js from windmill-worker/src/
- Fix operator precedence in windmill-jseval with explicit parentheses
- Remove unnecessary return keyword in heap limit callback
- Remove redundant as usize casts
- Remove ~150 lines of commented-out code from runtime.js
- Remove commented-out #[cfg] in build.rs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* otel ee

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 09:16:37 +00:00
Ruben Fiszel
e1a815f6a0 refactor: extract windmill-dep-map crate for parallel api/worker compilation (#7846)
* refactor: extract windmill-dep-map crate for parallel api/worker compilation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: resolve WebhookShared type mismatch and missing enterprise propagation

- Make windmill-api webhook_util re-export from windmill-common instead of
  duplicating types, fixing Extension<WebhookShared> mismatch between
  windmill-store and windmill-api
- Add windmill-api-jobs/enterprise to windmill-trigger enterprise feature
  so check_license_key_valid is available when trigger subcrates enable
  enterprise on windmill-trigger

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: stop trigger features from unconditionally enabling enterprise

Move enterprise propagation for all trigger subcrates from individual
trigger feature definitions to the enterprise feature itself, so
enterprise is only enabled when explicitly requested.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: remove unused pub use re-exports and disable CI cargo cache

- Remove unused re-exports from windmill-worker/src/lib.rs:
  trigger_dependents_to_recompute_dependencies, handle_job_error,
  and unused bun/otel items
- Fix callers to use direct module paths instead
- Add windmill-dep-map as dev-dependency for tests
- Disable cargo cache in backend-check CI (faster from-scratch builds)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: restore bun re-exports used by tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

* chore: re-enable cargo cache for check_ee_full CI job

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 00:39:56 +00:00
Ruben Fiszel
bbb397b6ad fix: improve scheduling reliability in extreme pool contention conditions (#7825)
* fix: reuse outer tx for schedule push in commit_completed_job

Instead of calling handle_maybe_scheduled_job(db) which opens its own
connections (peak=3), inline the schedule push using a savepoint on the
outer transaction. Auth is fetched via the tx connection using
fetch_authed_from_permissioned_as_conn, and push_scheduled_job runs
on a savepoint so failures roll back only the push, not the completion.

On push failure: savepoint rolls back, schedule is disabled on the outer
tx, and the zombie return path is preserved if disabling also fails.

Peak connections drop from 3 to 1 (or 2 on cold RunnableSettings cache).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

* fix: extract shared try_schedule_next_job to unify schedule push paths

Replace the two diverging schedule-push implementations (inlined in
commit_completed_job and standalone handle_maybe_scheduled_job) with a
single try_schedule_next_job that reuses the caller's transaction via
savepoints. This eliminates extra pool connection usage in the
worker_flow.rs path and ensures consistent retry/error semantics.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add failpoint markers to try_schedule_next_job

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: remove plan.md

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove inner retry loop from try_schedule_next_job, add caller-level retries

The 10-retry x 5s-sleep loop inside try_schedule_next_job held locks on
v2_job_completed/v2_job_queue for up to ~45s when running inside the
outer commit_completed_job transaction.

Now try_schedule_next_job makes a single attempt and returns errors to
the caller. Non-retryable errors (QuotaExceeded, NotFound) disable the
schedule immediately inside the function. Transient errors are returned
for the caller to retry:

- commit_completed_job path: outer backon retry (10x3s) retries the
  entire transaction including the schedule push, so no locks are held
  during sleep.
- handle_flow path: new backon retry (10x3s) wraps begin/push/commit
  with a fresh transaction per attempt.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: clear push_err after successful schedule disable to prevent stuck schedules

When try_schedule_next_job disables the schedule for non-retryable errors
(NotFound, QuotaExceeded), clear the error so the caller commits the tx
(persisting the disable). Previously, the error propagated up, causing the
tx to be dropped and rolling back the disable — leaving the schedule
permanently enabled but broken.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add 5s timeout on push_scheduled_job, clean up handle_flow error handling

- Add tokio::time::timeout(5s) around push_scheduled_job inside
  try_schedule_next_job to bound worst-case lock holding per attempt
- Remove unreachable QuotaExceeded/NotFound match arms in handle_flow
  (these errors are handled internally by try_schedule_next_job)
- Add report_error_to_workspace_handler_or_critical_side_channel in
  handle_flow when post-exhaustion schedule disable fails

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: return SchedulePushZombieError when both schedule push and disable fail

When handle_flow cannot push the next scheduled job AND cannot disable the
schedule, return a SchedulePushZombieError so the worker leaves the flow job
in the queue for zombie detection to restart. This prevents stuck schedules
where neither the next tick was pushed nor the schedule was disabled.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 13:16:32 +00:00
Alexander Petric
153dd32187 feat: customer portal extra workspace stats (#7841)
* iam

* fix occupancy + log settings change

* ee ref

* ee ref

* sqlx

* chore: update ee-repo-ref to 7f93a13e96c77292ed4b1e63bc1e8ff1e341d283

This commit updates the EE repository reference after PR #408 was merged in windmill-ee-private.

Previous ee-repo-ref: 5b6a4b2f990b7e5bdf6dea14645c787b42a4d9a6

New ee-repo-ref: 7f93a13e96c77292ed4b1e63bc1e8ff1e341d283

Automated by sync-ee-ref workflow.

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-02-06 22:21:19 +00:00
Alexander Petric
afa6e7ab5b feat: mssql integrated auth (gssapi) (#7760)
* feat: mssql integrated auth (kerberos/ntlm)

* install krb5 headers

* also make it work for windows

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-02-05 18:35:31 +00:00
hugocasa
476e6fd4bd fix: allow unauthed private pwsh repo (#7817) 2026-02-05 17:26:31 +00:00
Ruben Fiszel
6d184c0795 lower vacuum period 2026-02-05 15:23:28 +00:00
Ruben Fiszel
149da9b763 feat: make nsjail available in all standard images (CE) (#7793)
* feat: make nsjail available in all standard images (CE)

Include nsjail binary and runtime deps in the main Dockerfile and
DockerfileSlim so sandboxing is available out of the box. Flip
DISABLE_NSJAIL default to false so nsjail is enabled by default.

Remove DockerfileNsjail (now redundant) and the build_ee_nsjail CI job,
pointing publish_ecr_s3 at the base EE image instead. Add iptables to
DockerfileFullEe to preserve the functionality from the removed nsjail
image.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* revert: keep DISABLE_NSJAIL default as true

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: pin publish_ecr_s3 to exact commit hash

Add type=sha tag to build_ee so it pushes a commit-pinned image tag.
Restore git hash lookup in publish_ecr_s3 to reference the exact image
for that commit, avoiding race conditions with the mutable dev tag.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: publish_ecr_s3 depends on build_ee_full, uses release tag

Only publish to S3 on tag releases, extracting static frontend from the
ee-full image using the semver tag.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: remove stale windmill-ee-nsjail references, add nsjail to EE slim

The windmill-ee-nsjail image is no longer published since DockerfileNsjail
was deleted. Update all references to use the base EE image (which now
includes nsjail), remove redundant nsjail deps from DockerfileExtra, and
add nsjail build to DockerfileSlimEe for consistency with CE slim.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 18:48:16 +00:00
Ruben Fiszel
8220a0bd8d nits 2026-02-04 17:44:46 +00:00
Diego Imbert
0caa533fbd feat: column-level asset tracking for ducklake and datatables (#7774)
* Refactor 1

* claude tmp1

* fixes1

* support for insert and update

* Fix returning

* docs: add nuanced mutex selection guidance to Rust backend skill (#7737)

Add "Mutex Selection in Async Code" section explaining when to use
std::sync::Mutex vs tokio::sync::Mutex based on official Tokio docs.
std::sync::Mutex is preferred for data protection as it's faster;
tokio::sync::Mutex only needed when holding locks across .await points.

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): revert findCodebase change that broke ../shared codebases (#7740)

* fix(cli): revert findCodebase relative_path check that broke ../shared codebases

The previous change added a check to ensure script paths start with the
codebase's relative_path. However, this broke cases where relative_path
uses parent directory references (e.g., "../shared") because:

1. path.join normalizes paths, so "/project/../shared/f/script.ts" becomes
   "/shared/f/script.ts"
2. FSFSElement strips the cwd prefix, resulting in "f/script.ts"
3. The check "f/script.ts".startsWith("../shared/") failed

The original behavior was correct - relative_path indicates where to find
codebase files, while includes/excludes patterns match against the normalized
paths that get passed during sync.

Fixes regression reported in #7729 comments.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test(cli): add preview test for codebase with imports

Tests that codebase bundling correctly includes imported modules,
which is the key functionality needed for ../shared codebases.
The test creates a helper module and a main script that imports
from it, then verifies the bundled script executes correctly.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* chore(main): release 1.621.2 (#7735)

* chore(main): release 1.621.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* feat: workspace dedicated workers (#7741)

* feat: workspace dedicated workers

* ref

* chore: update ee-repo-ref to a18ac31062ac092cb9a5fc87629e217d97f4911d

This commit updates the EE repository reference after PR #398 was merged in windmill-ee-private.

Previous ee-repo-ref: 98cfe3fef764d9d815d326d5056c734a03689d33

New ee-repo-ref: a18ac31062ac092cb9a5fc87629e217d97f4911d

Automated by sync-ee-ref workflow.

* fix(frontend): workspace script in flow steps

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* fix: remove uuid-ossp extension requirement for RDS compatibility

The uuid-ossp extension was created in the first migration but never
actually used - the codebase uses gen_random_uuid() which is built-in
to PostgreSQL 13+. This allows Windmill to run on AWS RDS where
application users may not have CREATE SCHEMA privileges.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: indexer build error (#7744)

* fix: indexer build error

* prepare sqlx

* Remove changes from Cargo.toml

* refactor: remove seed parameter from AI chat completions (#7745)

* better claude

* refactor: remove seed parameter from AI chat completions

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* fix: visibility bug on deployment UI (issue when renaming items) + add tracking of folders and resource types (#7739)

* fix: Raw apps deployment UI (and merge UI)

* Add folders and resource tpyes to merge UI

* claude first pass on adding the new arg for h_deploy_metadata

* Add missing argument to handle_deployment_metadata in all its calls

* Add support for folders and resource types in merge UI

* Update eereporef for CI

* Update ee repo

* Add migration to reset cached diff with potential artifacts

* fix type in frontend

* Preapare sqlx

* Remove unused import and logs

* update ee-repo

* Update eerepo

* chore: update ee-repo-ref to aca38475afd2cafaf63f4bbffc65be9437d57d86

This commit updates the EE repository reference after PR #397 was merged in windmill-ee-private.

Previous ee-repo-ref: 19c64cf8c61d83f45047b37660054b29658cd403

New ee-repo-ref: aca38475afd2cafaf63f4bbffc65be9437d57d86

Automated by sync-ee-ref workflow.

* Make integration  test for workspace comparisons

* Update SQLx metadata

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* fix: require AGENT_TOKEN and BASE_INTERNAL_URL for agent mode

- Add AgentConfig struct to validate required env vars on startup
- Change build_agent_http_client to require explicit token and URL
- Remove DEFAULT_BASE_INTERNAL_URL fallback (no more silent localhost:8000)
- Exit immediately if agent cannot connect to server on initial load
- Update integration tests to use dynamic port for BASE_INTERNAL_URL

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: forward teams error to client (#7746)

* fix: forward teams error to client

* chore: update ee-repo-ref to 9a3d71f2c6a41ed4d17111a8c05d8e1d4933898d

This commit updates the EE repository reference after PR #400 was merged in windmill-ee-private.

Previous ee-repo-ref: 25d35a8de1cd70e281dc876e51cd30402580b5c0

New ee-repo-ref: 9a3d71f2c6a41ed4d17111a8c05d8e1d4933898d

Automated by sync-ee-ref workflow.

* fix

* fix

* fix

* al

* sqlx

* sqlx

* all

* all

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>

* feat: add token usage tracking to AI agent output (#7738)

* feat: add token usage tracking to AI agent output

Add TokenUsage struct to track input/output/cache tokens from AI providers.
Currently implemented for Bedrock provider, with infrastructure in place
for other providers. Usage is included in the AI agent result alongside
output and messages when available.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add token usage extraction for Anthropic provider

Extract usage from message_delta SSE event and convert to TokenUsage.
Includes input_tokens, output_tokens, cache_read_input_tokens, and
cache_creation_input_tokens (mapped to cache_write_input_tokens).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add token usage extraction for Google AI/Gemini provider

Extract usage from usageMetadata in Gemini SSE events and convert to TokenUsage.
Maps promptTokenCount -> input_tokens, candidatesTokenCount -> output_tokens,
totalTokenCount -> total_tokens.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add token usage extraction for OpenAI Responses API provider

Extract usage from response.completed SSE event and convert to TokenUsage.
Maps input_tokens, output_tokens, and total_tokens directly.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add token usage extraction for Azure OpenAI / Chat Completions API

Add stream_options.include_usage to request and parse usage from final
SSE chunk for providers using the standard OpenAI Chat Completions API
(Azure OpenAI, Mistral, DeepSeek, Groq, TogetherAI, CustomAI).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: cleanup token usage tracking - remove unused Image usage field and accumulate across iterations

- Remove unused `usage` field from ParsedResponse::Image variant
- Add TokenUsage::accumulate() method to sum usage across agent iterations
- Accumulate input/output/total/cache tokens instead of replacing with last iteration

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: remove verbose debug logging from AI providers

Remove tracing::info!("[debug] ...") statements that were too verbose
for production. These logged raw events on every streaming event.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: add retry mechanism for OpenAI-compatible providers without stream_options support

Some OpenAI-compatible providers don't support the stream_options parameter
for usage tracking. This adds a retry mechanism that:
- First attempts the request with stream_options.include_usage
- If it fails with 400 and error mentions stream_options/include_usage,
  automatically retries without the parameter

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: remove unnecessary text parsing overhead in image response handlers

Revert debugging changes that read response as text before parsing JSON.
Using response.json() directly is more efficient.

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: centralize TokenUsage conversion with constructor methods

Add new(), from_input_output(), and with_cache() constructors to TokenUsage
to eliminate duplicate conversion logic across providers. Also fixes potential
truncation in Bedrock cache token conversion by using i32::try_from with
fallback to i32::MAX.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: simplify Anthropic usage extraction and add Default derive

- Use idiomatic `if let` pattern instead of `is_some()` check for usage extraction
- Add Default derive to OpenAIChatUsage for consistency with other usage structs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use saturating_add to prevent overflow in token accumulation

In long-running agents with many iterations, token counts could
potentially overflow. Using saturating_add ensures values cap at
i32::MAX instead of wrapping around.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* better claude

* nit

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* chore(main): release 1.622.0 (#7742)

* chore(main): release 1.622.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* fix npm check

* fix: add schema compatibility layer for MCP clients like n8n (#7747)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* nit ui nextcloud triggers (#7749)

* feat: add PocketID OAuth provider support (#7318)

* feat(oauth): add Pocket-ID OAuth provider component

- Implements PocketIdSetting.svelte following Keycloak pattern
- Configures OIDC endpoints for Pocket-ID (/authorize, /api/oidc/*)
- Supports standard OIDC scopes (openid, profile, email)
- Uses passkey-only authentication via Pocket-ID

Refs #5678

* feat(oauth): register Pocket-ID in SSO provider list

- Import PocketIdSetting component
- Add Pocket-ID to provider list in SSO tab
- Update exclusion filter to prevent duplicate custom entries

Refs #5678

* fix(oauth): add missing PocketID icon and fix component integration

- Create PocketIdIcon.svelte component with user profile icon
- Register pocket-id in APP_TO_ICON_COMPONENT mapping
- Fix PocketIdSetting to use IconedResourceType pattern matching other OAuth providers

This resolves the issue where PocketID toggle was not appearing in SSO settings.

Refs #5678

* refactor: migrate PocketIdSetting to Svelte 5 runes syntax

- Use $props() with $bindable() for reactive prop binding
- Use $state() for local reactive state
- Use $derived() for computed values
- Use $effect() for reactive side effects
- Replace on:change with onchange event handler
- Pre-populate base URL from existing config when editing
- Clean up bracket notation to dot notation for value properties

Addresses reviewer feedback

* fix: rename pocket-id to pocketid for naming convention compliance

Change identifier from 'pocket-id' to 'pocketid' to match Windmill's naming convention.
No OAuth provider uses hyphens - all custom SSO providers (keycloak, authentik, authelia,
kanidm, zitadel) use no separator.

Changes:
- AuthSettings.svelte: oauths['pocket-id'] → oauths['pocketid'] (2 locations)
- PocketIdSetting.svelte: name={'pocket-id'} → name={'pocketid'}
- icons/index.ts: 'pocket-id': PocketIdIcon → pocketid: PocketIdIcon

Note: PocketID does not need oauth_connect.json entry as it's a custom SSO provider
with user-configured endpoints, similar to Keycloak/Authentik.

Addresses reviewer feedback

* fix: use TextInput component for consistency

---------

Co-authored-by: hugocasa <hugo@casademont.ch>

* fix: preserve script envs field during sync push

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit frontend fix

* chore(main): release 1.623.0 (#7748)

* chore(main): release 1.623.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* fix: resolve infinite effect loop in PocketIdSetting component (#7753)

* fix: prevent retention cleanup from deleting jobs of active flows (#7755)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* chore(main): release 1.623.1 (#7754)

* chore(main): release 1.623.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* feat: default to quickjs on ce for flow eval (#7756)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* feat: runtime assets (#7656)

* Runtime assets

* Nits

* Revert "Nits"

This reverts commit 3031a2ddd1.

* detection_kinds

* don't delete runtime assets

* Show latest executions

* conditional unique idx

* nit status

* refactor

* nit refactor

* prepare sql

* Detect assets in complex JSON input objects

* false positive prevent

* nit

* redundant idx

* Update frontend/src/lib/components/assets/AssetsUsageDrawer.svelte

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* Update backend/migrations/20260122134517_runtime_assets.up.sql

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* runtime assets are inserted in a loop

* nit

* nit fix

* Don't use lazy static

* fix compilation

* nits

* missing on conflict do nothing

* add index

* Fix max n logic

* created at

* nits

* remove pagination

* sqlx prepare

* Only detect resource assets in input

* get_runtime_asset_sender()

* use global get_runtime_asset_sender to avoid prop drilling

* nit refactor : register_runtime_asset

* get job_id from token

* job as a usage kind

* fixes

* ee

* nit refactor

* merge access types when same job uses same asset multiple times

* Refactor to support wmill s3 API

* nit

* parse_wmill_sdk_sql_assets refactor

* Detect datatable and ducklake usage

* nit order by

* Join with v2_job

* better UI

* add sequential id for cursor pagination

* useInfiniteQuery

* useScrollToBottom

* sql index

* claude code stash

* migration fixes

* Infinite scroll UI

* nit

* style nit

* runtime asset created at

* Asset filters

* fix usage kind filter

* also check runnable_path for jobs when filtering

* better filters

* avoid flickering

* debounced filters

* nit

* tooltips

* fix: update AssetUsage type to match new ListAssetsResponse structure

The ListAssetsResponse changed from an array to an object with an 'assets' property.
Updated the type extraction accordingly.

Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com>

* sqlx prepare

* Delete .claude/hooks/.symlink-manifest

* unnecessary dep

* nit refactor

* nit comment

* nit naming

* CI fix attempt 1

* ee ref

* nit remove alerts

* nit

* chore: update ee-repo-ref to 138a4f5f868f3bded5bb7cb77b222b532c07e4af

This commit updates the EE repository reference after PR #395 was merged in windmill-ee-private.

Previous ee-repo-ref: 7d3a21d53066726e97dfea9f117373299bc9318c

New ee-repo-ref: 138a4f5f868f3bded5bb7cb77b222b532c07e4af

Automated by sync-ee-ref workflow.

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* fix: remove $schema field from Google AI output schema requests (#7765)

* fix: remove $schema field from Google AI output schema requests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: add $schema field to all output schema integration tests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: remove $schema field from Google AI tool parameter schemas

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: add workspace script tool test for AI agents

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* fix: Fix app multiselect not refreshing result when creating element (#7766)

* Fix returning

* asset columns are saved and displayed in the assets page

* runtime assets column detectionz

* frontend nits

* update regex parsers

* UI nits

* Display asset columns in flow graph

* Column hint directly in asset node

* nit bg

* sqlx prepare

* ee repo ref

* chore: update ee-repo-ref to 66a68df97e8c65c498b28f302a365ab8687cad9e

This commit updates the EE repository reference after PR #402 was merged in windmill-ee-private.

Previous ee-repo-ref: 0a32bc104cbaec9664a4d7cb1565823722c875a1

New ee-repo-ref: 66a68df97e8c65c498b28f302a365ab8687cad9e

Automated by sync-ee-ref workflow.

---------

Co-authored-by: centdix <40307056+centdix@users.noreply.github.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
Co-authored-by: hugocasa <hugo@casademont.ch>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: wendrul <53628737+wendrul@users.noreply.github.com>
Co-authored-by: Alexander Petric <alpetric@users.noreply.github.com>
Co-authored-by: Devdatta Talele <50290838+devdattatalele@users.noreply.github.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com>
2026-02-04 16:27:53 +00:00