Commit Graph

832 Commits

Author SHA1 Message Date
HugoCasa
980cbcccf0 Merge remote-tracking branch 'origin/main' into batch-pulling 2026-03-05 17:14:53 +01:00
HugoCasa
876a9cfc8e feat: batch job pulling for native workers
Reduce DB polling overhead for native workers by batch-fetching jobs
server-side and serving them from an in-memory buffer via HTTP.

- Add batch_pull() in windmill-queue: single SELECT...FOR UPDATE SKIP LOCKED LIMIT N
- Add batch pull SQL helpers (make_batch_pull_query, format_batch_pull_query)
- OSS stubs for agent-workers accept batch_buffer parameter (4-tuple return)
- Native workers self-sign JWT and pull jobs via HTTP when co-located with server
- Add uses_batch_http_pull column to worker_ping for server-side tracking
- Worker pull loop: HTTP batch pull when client available, SQL otherwise

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 16:08:28 +01:00
Ruben Fiszel
5f0ef936d1 feat: add sandbox annotations, volume mounts, for AI sandbox starting with claude (#8058) 2026-03-05 06:19:51 +00:00
hugocasa
e56ccd200b feat: token expiration notifications (#8190)
* feat: add token expiration notifications via email, critical alerts, and webhooks

- Monitor loop checks for tokens expiring within 7 days and sends
  email notifications to token owners. Tracks notification state via
  new `expiry_notified` column on the token table to avoid duplicates.
- When tokens expire and are deleted, owners are also notified.
- Critical alerts (in-app UI) are gated behind a new instance setting
  `critical_alerts_on_token_expiry` (off by default); emails are
  always sent regardless of the setting.
- Add TokenExpiringSoon and TokenExpired webhook message variants for
  workspace webhook integrations.
- Frontend: show expiration badges and a warning banner on the tokens
  table for tokens expiring within 30 days.
- Exclude session and ephemeral tokens from all notifications.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: use separate token_expiry_notification table for dedup

- Replace `expiry_notified` column on token table with a dedicated
  `token_expiry_notification` table (token, expiration)
- Insert notification row on token creation via shared
  `register_token_expiry_notification()` helper
- Delete notification row atomically when sending the notification
- Clean up orphaned rows in `delete_expired_items()`
- No FK constraint to avoid cascade overhead on token deletions
- Add index on expiration column for efficient range queries

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: calendar-based expiration badge and move notification cleanup

- Fix daysUntilExpiration to compare calendar dates instead of time diff
- Move notification row cleanup from delete_expired_items to
  check_expiring_tokens to keep it off the hot path
- Use simple expiration <= now() index scan instead of NOT EXISTS join

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 05:22:46 +00:00
Ruben Fiszel
077779ec52 fix: improve windows compatibility
* ci: add Windows backend integration test workflow

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* ci: temporarily add push trigger for testing

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* ci: add --no-fail-fast to run all test binaries

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: Windows path handling for backend integration tests

- WINDMILL_DIR: use std::env::temp_dir() on Windows instead of /tmp/windmill
- HOME_ENV: fall back to USERPROFILE on Windows when HOME is not set
- loader.bun.js: normalize paths to forward slashes for consistent
  comparison with Bun's resolver output on Windows
- bun_executor.rs: convert job_dir to forward slashes in JS template
  strings to avoid backslash escape issues (\t -> tab, etc.)
- go_executor.rs: fix windows_gopath() double backslash bug (r"\\" -> "\\")
- bash_executor.rs: default to "bash" (in PATH) on Windows instead of /bin/bash

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: improve Windows diagnostics and fix onLoad handler

- Include path in create_directory_async/sync panic messages
- Add WINDMILL_DIR initialization debug output
- Fix loader.bun.js onLoad: use properly escaped regex instead of
  returning undefined (Bun requires onLoad to return an object)
- Add env var debug output to CI workflow

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: sanitize Windows-invalid characters in test worker names and fix cargo path

- Replace :: with __ in worker names (colons illegal in Windows dir names)
- Fix HOME_DIR to fall back to USERPROFILE on Windows
- Add PATH fallback for cargo discovery on Windows
- Add debug logging to bun loader for fetch errors

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: handle single colons in worker names, pass MSVC linker env vars, revert bun debug

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use .exe binary name on Windows and normalize bun import URL paths

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use absolute path for rust binary, normalize bun resolve paths

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use .wurl extension instead of .url for bun import resolution on Windows

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use custom namespace for bun plugin to bypass default file resolution

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use virtual namespace for bun import resolution to avoid Windows path issues

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: handle Windows 8.3 paths and namespace-prefixed importers in bun loader

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: strip namespace prefix from args.path and handle absolute imports without leading slash in bun loader

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: simplify bun loader and remove redundant cargo path lookups

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use platform-specific cargo binary path with .exe on Windows

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: replace HOME_DIR with HOME_ENV in rust_executor to remove duplication

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: keep original bun loader on linux, use virtual namespace loader only on windows

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-04 20:20:18 +00:00
hugocasa
63ebae8829 feat: replace hub error toasts with warning alerts and add disable hub setting (#8225)
* feat: replace hub error toasts with warning alerts and add disable hub setting

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: guard hub script cache refresh when hub is disabled

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 15:12:00 +00:00
Ruben Fiszel
424ca59dfe feat: make WINDMILL_DIR configurable via environment variable (#8215)
* fix: auto-heal corrupted python runtime cache on remote workers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Revert "fix: auto-heal corrupted python runtime cache on remote workers"

This reverts commit 0ea013a554.

* feat: make WINDMILL_DIR configurable via environment variable

Allow users to configure the base directory for Windmill's tmp/cache files
via the WINDMILL_DIR env var (default: /tmp/windmill). This fixes Python
runtime cache corruption on RHEL systems where systemd-tmpfiles-clean
removes files from /tmp.

Converts TMP_DIR (renamed to WINDMILL_DIR) and all derived cache directory
constants from compile-time const &str (concatcp!) to runtime lazy_static
String values.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee ref

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee ref

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: deref ERROR_DIR lazy_static for AsRef<Path> and Display traits

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee ref to branch name for CI compatibility

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: deref lazy_static constants in all executor files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee ref

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee ref

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee ref

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: panic if WINDMILL_DIR has trailing slash

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: also reject trailing backslash in WINDMILL_DIR for Windows

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: deref GO_BIN_CACHE_DIR in test utils

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: replace remaining hardcoded /tmp/windmill paths and validate empty WINDMILL_DIR

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: nsjail powershell mount dst, Windows path assumptions, pwsh deref consistency

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: restore Windows /tmp path translation in go and bun executors

The Windows path translation replaces /tmp with the Windows temp dir
(e.g. C:\tmp) before normalizing slashes. Without this, the default
WINDMILL_DIR=/tmp/windmill produces paths without a drive letter on
Windows.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref to 6fd5a2ce908235a17975ad4dbdf0051cd89334f3

This commit updates the EE repository reference after PR #436 was merged in windmill-ee-private.

Previous ee-repo-ref: e8c03e16720833230ebd1878b4c63642ecc6c80f

New ee-repo-ref: 6fd5a2ce908235a17975ad4dbdf0051cd89334f3

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-04 08:53:25 +00:00
Ruben Fiszel
cac4bdd54f fix: gracefully handle malformed OAuth entries in instance config (#8205)
When an OAuth provider entry in instance settings has unexpected types
(e.g. `"true"` instead of `true` for req_body_auth), the entire
/api/settings/instance_config endpoint would fail with a deserialization
error, preventing access to any instance settings.

Introduce OAuthClientEntry enum that tries typed OAuthClient
deserialization first and falls back to raw JSON, logging the
deserialization error. This allows the settings page to load even when
individual OAuth entries are malformed.

Also show a user-visible error toast in SaveButton on save failure
instead of only logging to console.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 12:37:37 +00:00
Ruben Fiszel
2b2be38f12 fix: use exact matching for python requirements directive parsing (#8199)
* fix: use exact matching for python requirements directive parsing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: apply same exact matching fix to CLI parser

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 21:21:28 +00:00
Diego Imbert
cf3ddce68a Fix data tables not working with non-secret pg variables (#8195) 2026-03-02 18:01:18 +00:00
Ruben Fiszel
cfd9541ab1 fix(frontend): preserve keycloak realm url between instance settings saves (#8189)
* fix(frontend): preserve keycloak realm url between instance settings saves

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(backend): preserve provider-specific oauth fields through round-trip

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 14:26:34 +00:00
Ruben Fiszel
db44b8be74 fix: add missing display_name and tenant fields to instance config OAuthClient (#8176)
* fix: add missing grant_types field to instance config OAuth structs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add missing display_name and tenant fields to instance config OAuthClient

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 07:35:40 +00:00
Ruben Fiszel
fca94f88dd fix: add missing grant_types field to instance config OAuth structs (#8175)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 07:30:36 +00:00
Fred Reimer
7d6f4fdabb chore: bump Go in container images to 1.26.0 (#8135) 2026-03-01 06:53:33 +00:00
Diego Imbert
7d9d16a6a3 feat: runScript inline for path and hash (#8019)
* runScript inline for path and hash

* Update backend/windmill-api/src/jobs.rs

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* refactor: unify inline script param structs and deduplicate closures

- Replace RunInlineScriptByPathFnParams and RunInlineScriptByHashFnParams
  with a single RunInlineScriptFnParams using InlineScriptTarget enum
- Collapse two nearly-identical closures in worker.rs into one
- Merge duplicate InlineByPath/InlineByHash into InlineScriptArgs
- Extract shared run_inline_script_inner helper in API handler
- Add missing check_scopes to run_inline_script_by_hash endpoint
- Fix duplicate lines from prior commit in run_inline_script_by_path
- Change tag from "inline_preview" to "inline" for deployed scripts

Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com>

* Integration tests

* rm

* rename feature to run_inline

* Run inline integration tests

* Fix tests

* check path scope

* openapi fix

* nits

* remove register_potential_assets_on_inline_execution

* unused variable

* refactor

* Pass user_db to check script permission

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com>
2026-02-27 13:59:14 +01:00
hugocasa
4e1ae276b0 feat: add force_branch parameter to git sync settings (#8089)
* feat: add force branch param to git sync settings

* update hub path

* chore: update ee-repo-ref to a797dd4d619cdab737e133ce593f2f8582ba21de

This commit updates the EE repository reference after PR #430 was merged in windmill-ee-private.

Previous ee-repo-ref: 373c5870ae5499c8c8a22cb92b2fd3a64a651183

New ee-repo-ref: a797dd4d619cdab737e133ce593f2f8582ba21de

Automated by sync-ee-ref workflow.

* UI nits

* nit

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-02-26 18:23:44 +01:00
Diego Imbert
16a6d5e7af feat: Broad filters for search (#8112)
* Default filters

* nit style

* Nit fixes

* broadFilter for runs

* unused param

* Remove debounce from assets page

* Escape ILIKE patterns

* fix pg enum cast to text for ILIKE

* nit error
2026-02-26 14:03:46 +00:00
wendrul
7ac93f6ee3 feat: option to preserve on_behalf_of and edited_by for admins and users in the new wm_deployers group (#8079) 2026-02-25 12:05:22 +00:00
Ruben Fiszel
a91c532eca fix: make WM_FLOW_PATH available in flow step previews (#8042)
* fix: pass flow path in flow step preview for AI agent modules

JobLoader.runFlowPreview was missing the path parameter, causing
WM_FLOW_PATH to be unavailable when using the Run button on individual
flow steps. Test up to here worked correctly because it uses a
different code path (utils.svelte.ts) that already passed the path.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: make WM_FLOW_PATH available for rawscript/script step previews

Inject the flow path as `_flow_path` in the job args when running a
script preview from the flow editor. The SQL pull queries now use
COALESCE to fall back to this arg when no parent runnable path exists,
making WM_FLOW_PATH available for individual step "Run" previews.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: rename _flow_path args key to _FLOW_PATH

Match existing convention used by _ENTRYPOINT_OVERRIDE.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 21:56:32 +00:00
Ruben Fiszel
9eb15312f6 feat: add .npmrc support for private npm registries (#8039)
* feat: add .npmrc support for private npm registries

Add a new `npmrc` instance setting that accepts full .npmrc file content
for configuring private npm registries. Works with bun (native .npmrc
support since 1.1.18), deno (native .npmrc support in 2.x), and the npm
proxy (parses default registry + auth token from .npmrc).

Legacy `npm_config_registry` and `bunfig_install_scopes` fields are now
hidden when empty, so new users only see the .npmrc field. Also fixes a
pre-existing race condition where gen_bunfig was called after
start_child_process.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 05:29:36 +00:00
Diego Imbert
90b1a7a531 fix: Fix DuckDB incorrect pg password encoding (#8028) 2026-02-20 10:35:27 +00:00
Ruben Fiszel
6bf544f507 refactor: extract object store into dedicated crate with filesystem backend (#7996)
* refactor: extract object store code into windmill-object-store crate with filesystem backend

Consolidate all object_store-dependent code from windmill-common into a new
windmill-object-store crate. Add a filesystem-backed object store implementation
using LocalFileSystem for dev/testing without cloud credentials. Includes 30
comprehensive tests covering render_endpoint, lfs_to_object_store_resource,
duckdb_connection_settings, error mapping, and filesystem-backed integration tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

* all

* all

* all

* fix: fix raw_app hardcoded path, add missing ObjectStoreResource import, and add tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: move S3ModeFormat to windmill-types, make windmill-parser-sql optional, restore debug logs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 12:24:45 +00:00
Ruben Fiszel
71608bf669 fix: yaml settings UI mask rsa_keys and jwt_secret 2026-02-17 08:54:58 +00:00
Ruben Fiszel
4b8bb72857 operator nits 2026-02-17 08:40:46 +00:00
Ruben Fiszel
b7bec1a83d fix(operator): improve configmap handling of older license keys 2026-02-17 08:38:12 +00:00
Ruben Fiszel
b3eeee4131 feat: show all settings in YAML UI and protect from empty overwrites (#7976)
- Show custom_instance_pg_databases, ducklake_settings, ducklake_user_pg_pwd
  and rsa_keys in frontend YAML editor (remove from excludedKeys)
- Redact sensitive values: add ducklake_user_pg_pwd and rsa_keys to
  sensitiveKeys, add custom_instance_pg_databases.user_pwd to
  nestedSensitiveFields
- Remove rsa_keys from HIDDEN_SETTINGS so it appears in YAML export
- Hide automate_username_creation from export (add to HIDDEN_SETTINGS)
- Add ducklake_user_pg_pwd and rsa_keys to SENSITIVE_SETTINGS for log
  redaction
- Generalize empty/null protection for all PROTECTED_SETTINGS: operator
  diff skips empty values when DB has existing data, direct API rejects
  delete/empty for protected settings

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 08:52:31 +01:00
Ruben Fiszel
2019aecf42 fix: improve operator ConfigMap settings handling (#7975)
* feat: improve operator ConfigMap settings handling

- Protect jwt_secret and min_keep_alive_version from deletion (add to
  PROTECTED_SETTINGS)
- Expose jwt_secret in config exports (remove from HIDDEN_SETTINGS)
- Reject empty/null jwt_secret values with warning
- Clamp retention_period_secs to 30 days max on CE builds
- Improve apply_settings_diff logging: distinguish Created/Updated/Deleted
  with from/to values and unchanged count summary
- Add sensitive value masking in logs with partial redaction (prefix/suffix)
  for top-level secrets and nested sub-field masking for oauths, smtp,
  object_store_cache_config, custom_instance_pg_databases
- Sort global_settings keys alphabetically in YAML export
- Order worker_configs with "default" and "native" first in YAML export
- Add tests for sorted YAML serializer

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review feedback

- Fix redact_string panic on multi-byte UTF-8 by using chars() instead
  of byte-length slicing
- Protect jwt_secret from deletion via direct API
  (set_global_setting_internal rejects empty/null with BadRequest)
- Add code comment documenting jwt_secret visibility trade-off

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 08:34:08 +01:00
Ruben Fiszel
3e313cc4e8 feat: add native_mode as typed field on WorkerGroupConfig
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 00:17:39 +00:00
Ruben Fiszel
f02ef6d03c refactor: switch operator from CRD to ConfigMap (#7972)
* refactor: switch operator from CRD to ConfigMap

Replace the WindmillInstance CRD with a plain ConfigMap for the K8s
operator. This simplifies deployment (no CRD to install/manage, no
ClusterRole for custom API groups) while keeping the same config schema.

- Replace crd_ee.rs with configmap_ee.rs (parses data.spec YAML key)
- Rewrite reconciler_ee.rs: ConfigMap watcher + Event recorder instead
  of CRD Controller + status subresource
- Add license_key preservation: if absent/empty in ConfigMap but present
  in DB, the DB value is kept
- Remove print_crd_yaml() and "operator crd" subcommand
- Drop schemars, chrono, instance_config_schema dependencies
- Delete manifests/crd.yaml
- Update K8s example and README for ConfigMap approach
- RBAC now only needs a namespace-scoped Role (not ClusterRole)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add superadmin YAML export endpoint and remove cache_clear from operator config

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 00:06:56 +00:00
hugocasa
535e108cbf feat: native mode (#7939)
* feat: native mode

* improve

* fix build

* review fixes

* tracing nit
2026-02-16 23:36:41 +00:00
hugocasa
6f24f1939d feat: google native triggers (#7837)
* feat: google native triggers

* nit skill

* better native trigger abstraction

* use resources for workspace integrations

* better and better

* better tests

* update native trigger skill

* sqlx

* less tx and google update fix

* refactor a bit the external logic

* nits

* fix

* fix google native trigger update

* fix oauth

* review fixes

* sqlx fix

* nit

* chore: update ee-repo-ref to a10eda4251610cceee67fbe05463b8be82ffa9e0

This commit updates the EE repository reference after PR #416 was merged in windmill-ee-private.

Previous ee-repo-ref: bf3696d5f2a39a3cb84dbbee81e092155f2a8c75

New ee-repo-ref: a10eda4251610cceee67fbe05463b8be82ffa9e0

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-02-16 15:52:02 +00:00
Ruben Fiszel
714f713a04 only update license key via IaC when new expiry is posterior (#7959)
When using infrastructure-as-code to update settings, if the desired
license key matches the current one (same client ID and signature) but
differs only in the expiration date, only apply the update if the new
key has a later expiry. This prevents accidental downgrades when an
older license key is present in the IaC configuration.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 17:13:37 +00:00
Ruben Fiszel
dbe576406e add type annotations to opaque CRD schema fields (#7952)
* fix: add type annotations to opaque CRD schema fields

The WindmillInstance CRD failed K8s structural schema validation because
four `Option<serde_json::Value>` fields (`object_store_cache_config`,
`secret_backend`, `slack`, `teams`) generated schemas without a `type`
property. Kubernetes requires all specified object fields to declare a
type.

Add a `schemars(schema_with)` annotation that emits
`{"type": "object", "nullable": true, "x-kubernetes-preserve-unknown-fields": true}`
for these opaque settings fields.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: enable operator in CE builds

The operator feature was only included in `ee_core`, meaning CE images
couldn't run `windmill operator` as a proper k8s controller — the
subcommand silently fell through to standalone mode.

Add `operator` to `ce_core` so CE builds include the k8s controller
that watches and reconciles WindmillInstance CRDs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 12:26:21 +00:00
Ruben Fiszel
82e5f6de48 feat: add Kubernetes operator and instance settings YAML editor (#7836)
* Add windmill-operator crate for Kubernetes CRD-based instance config

Introduces a new `windmill-operator` crate that enables declarative
management of Windmill instance configuration via a Kubernetes
`WindmillInstance` CRD. The operator watches CRD resources and performs
full declarative sync of global_settings and worker configs to the
database, supporting GitOps workflows for instance-level configuration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add tests for windmill-operator CRD and db_sync

- 9 unit tests for CRD serialization, deserialization, metadata, and
  status field behavior
- 15 integration tests for db_sync using #[sqlx::test] with full
  declarative sync coverage: upsert, delete, protected keys,
  idempotency, worker config prefix handling, and end-to-end sync

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Replace untyped BTreeMap CRD fields with typed structs for schema validation

GlobalSettings, SmtpSettings, IndexerSettings, and WorkerGroupConfig now
have explicit typed fields with serde(flatten) catch-all for forward
compatibility. The generated CRD YAML includes a full OpenAPI v3 schema
that Kubernetes validates on kubectl apply.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Type opaque serde_json::Value CRD fields with real structs

Replace most remaining serde_json::Value fields in WindmillInstance CRD
with properly typed structs derived from the codebase:
- oauths: BTreeMap<String, OAuthClient>
- otel: OtelSettings
- otel_tracing_proxy: OtelTracingProxySettings with ScriptLang enum
- critical_error_channels: Vec<CriticalErrorChannel> (untagged enum)
- critical_alerts_on_db_oversize: DbOversizeAlert
- ducklake_settings: DucklakeSettings with nested catalog/storage types
- custom_instance_pg_databases: CustomInstancePgDatabases
- autoscaling (worker config): AutoscalingConfig with integration struct
- custom_tags, default_tags_workspaces: Vec<String>
- default_tags_per_workspace: bool

Still opaque (serde_json::Value): object_store_cache_config (kube-core
can't generate schemas for internally-tagged enums), secret_backend
(EE-private), slack, teams (no clear struct definitions).

Regenerated CRD YAML with full OpenAPI schema (352→703 lines).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Unify instance config types and add bulk GET/PUT API

Move all typed settings (GlobalSettings, WorkerGroupConfig, etc.) from
windmill-operator/crd.rs into windmill-common/instance_config.rs so both
the API server and operator share a single source of truth. Add diff/apply
logic (Merge mode for UI, Replace mode for operator) and InstanceConfig::from_db().

Add GET/PUT /settings/instance_config endpoints so the frontend loads all
settings in 1 call instead of 42, and saves with a single bulk PUT. The
backend handles the diff internally, running pre-write hooks for changed keys.

Refactor windmill-operator/db_sync.rs to use the shared diff+apply functions
and slim crd.rs down to the CRD wrapper with re-exports.

Includes 32 unit tests and 30 integration tests covering serialization,
diff logic, DB roundtrips, protected settings, and edge cases.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add Form/YAML toggle to instance settings UI

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: show Form/YAML toggle regardless of hideTabs prop

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: replace toggle button group with simple YAML toggle

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: mask sensitive fields in YAML view with show/hide toggle

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: hide internal settings and mask sensitive fields in YAML view

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: hide jwt_secret and min_keep_alive_version from API and config exports

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* all

* feat: add secretKeyRef support for sensitive fields in operator CRD

Allow sensitive fields (license_key, hub_api_secret, scim_token,
smtp_password, OAuthClient.secret, custom PG user_pwd) to reference
Kubernetes Secrets via the standard secretKeyRef pattern instead of
inlining values as plaintext YAML. The reconciler resolves all refs
by reading K8s Secrets before syncing to the database.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

* all

* all

* fix: merge main and update dev environment docs

Resolve merge conflicts from origin/main, fix duplicate
UV_INDEX_STRATEGY_SETTING import, and add Playwright MCP
testing instructions to CLAUDE.md.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

* fix: init tracing for CLI subcommands and deduplicate setting side-effects

Initialize tracing subscriber before early-return CLI paths (sync-config,
operator) so tracing calls are not silently dropped. Refactor
set_global_setting_internal to call run_setting_pre_write_hook instead of
duplicating the side-effect logic.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add `wmill instance get-config` CLI command

Dumps the current instance config (global settings + worker configs) as
YAML. Supports --output-file to write to a file instead of stdout.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 22:42:01 +00:00
Ruben Fiszel
581dde8d0b feat: add maven settings.xml support for java private registries (#7940)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 16:57:30 +00:00
Ruben Fiszel
beeb19db04 fix: fix hub schedule not set at on-boarding 2026-02-13 00:33:40 +00:00
Pyra
d6c72df99a fix: conditionally skip relock on dep job (#7860)
* perf: conditionally skip relock on dep job

Signed-off-by: pyranota <pyra@duck.com>

* fix

* stage

Signed-off-by: pyranota <pyra@duck.com>

* final

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* add index

Signed-off-by: pyranota <pyra@duck.com>

* fix bug

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* final (for real)

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* merge migrations into one

Signed-off-by: pyranota <pyra@duck.com>

* add test

Signed-off-by: pyranota <pyra@duck.com>

* fix test

Signed-off-by: pyranota <pyra@duck.com>

* fix test

Signed-off-by: pyranota <pyra@duck.com>

* update sqlx

Signed-off-by: pyranota <pyra@duck.com>

* remove unused

Signed-off-by: pyranota <pyra@duck.com>

* sqlx

Signed-off-by: pyranota <pyra@duck.com>

* increase timeout

Signed-off-by: pyranota <pyra@duck.com>

* optimize the hell out of tests

Signed-off-by: pyranota <pyra@duck.com>

* simplify test

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-02-12 23:02:36 +00:00
Ruben Fiszel
2646629194 feat: add force_sandboxing global setting and #sandbox bash annotation (#7816)
* feat: add force_sandboxing global setting and #sandbox bash annotation

Add a new global setting `force_sandboxing` to enable nsjail sandboxing
for all jobs, overriding the DISABLE_NSJAIL environment variable.
Also add a #sandbox annotation for bash scripts to enable sandboxing
on a per-script basis.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: decouple nsjail probing from DISABLE_NSJAIL and apply force_sandboxing to all executors

NSJAIL_AVAILABLE now always probes for the nsjail binary at startup
regardless of the DISABLE_NSJAIL policy flag, fixing #sandbox annotation
and force_sandboxing being unusable in default (DISABLE_NSJAIL=true)
configuration. All language executors now use is_sandboxing_enabled()
instead of checking DISABLE_NSJAIL directly, so force_sandboxing
applies to all job types, not just bash.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: clarify force_sandboxing and DISABLE_NSJAIL relationship

Neither setting overrides the other — sandboxing is enabled when either
force_sandboxing=true OR DISABLE_NSJAIL=false. Updated frontend
description and backend comments to reflect this.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

* all

* revert typo

* all

* all

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-12 18:20:22 +00:00
Ruben Fiszel
e2c28e42db fix: add private registries support for RUST + java home nit 2026-02-12 14:23:52 +00:00
hugocasa
aaa1b92300 fix: hash long dedicated worker tags (#7914)
* fix: hash long dedicated worker tags

* Update frontend/src/lib/components/dedicated_worker.ts

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
2026-02-11 19:29:44 +00:00
Ruben Fiszel
ea52a8b8ce refactor: move integration tests to subcrates to reduce recompilation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 14:04:54 +00:00
Pyra
097d9288c5 feat(uv): index resolve strategy (#7885)
* fix

* fix

Signed-off-by: pyranota <pyra@duck.com>

* Update backend/windmill-worker/src/python_executor.rs

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* Update backend/windmill-worker/src/python_executor.rs

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
2026-02-11 11:18:51 +00:00
Ruben Fiszel
e26e437dd1 refactor: extract 12 leaf crates from windmill-api (#7899)
* feat(backend): extract 12 leaf crates from windmill-api to improve incremental compilation

Extract independent modules from windmill-api (90k LOC monolith) into
separate leaf crates to reduce incremental compilation times. Modules
extracted: assets, configs, debug, flow-conversations, inputs,
npm-proxy, openapi, schedule, settings, workers, agent-workers, and
alerting (from windmill-common).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add windmill-api-settings dep to root crate, make ee_oss public

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add agent_workers integration tests

8 tests covering agent worker lifecycle:
- Simple script execution (bun)
- Script with arguments
- Script with logs (verified in job_logs table)
- Script failure handling
- Complex result (nested objects/arrays)
- Agent token creation via API
- Token creation + Initial/MainLoop ping cycle
- Multiple sequential job execution

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 10:31:03 +00:00
Alexander Petric
7147dde511 fix: enforce self-approval check on flow resume owner endpoint (#7886)
* fix: enforce self-approval check on flow resume owner endpoint

* sqlx

* fix test

* fix mcp fields

* test self aproval allowed + other user approves flow
2026-02-10 23:14:40 +00:00
wendrul
2851b6b7ca feat: restriction rulesets for workspaces (#7879) 2026-02-10 16:49:25 +00:00
Ruben Fiszel
af02e200a5 add benchmark variants and fix db pool connection starvation (#7884)
Add new BENCHMARK_KIND variants (sequentialflow, scriptlogs, concurrencylimit,
concurrencykey, mixed, mixed_no_cc) for targeted performance testing. Fix shared
iteration counting across workers using a global atomic counter. Add job_perms
inserts and queue diagnostics for benchmark mode.

Move db connection setup to dedicated module and drop the initial connection pool
before creating the main one, preventing connection starvation when PostgreSQL
max_connections is low.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 16:41:47 +00:00
hugocasa
8363ff1eee feat: download encrypted usage (#7804) 2026-02-10 09:50:24 +00:00
Ruben Fiszel
cf596f370a fix: gate Permissions import behind #[cfg(unix)] for Windows build
Move `use std::fs::Permissions` and `use std::os::unix::fs::PermissionsExt`
inside the #[cfg(unix)] block to avoid unused import error on Windows.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 09:02:09 +00:00
Ruben Fiszel
4343b73485 fix: reduce DB pool contention by eliminating dual-connection patterns (#7861) 2026-02-09 22:48:29 +00:00
Ruben Fiszel
894d8a94f8 reuse existing transaction in push to reduce pool pressure (#7858)
* fix: reuse existing transaction in push instead of acquiring new connection

In push_inner, fetch_authed_from_permissioned_as was acquiring a new
connection from the pool to fetch job permissions, even though a
transaction was already open. Use fetch_authed_from_permissioned_as_conn
with the existing transaction instead, reducing pool pressure when many
jobs are pushed concurrently.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* improve contention

* improve contention

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 18:07:33 +00:00