Commit Graph

12066 Commits

Author SHA1 Message Date
Ruben Fiszel
d5a5e54da0 test(sandbox_setup): add overlayfs and crane integration tests
Replace placeholder #[ignore] tests with real integration tests:

- overlay_integration: test overlayfs read/write semantics via unshare
  (no root required), and overlayfs rootfs as nsjail sandbox root
- crane_integration: test crane export of alpine image → tar/gz
  roundtrip, and running the extracted rootfs in nsjail

Also fix tar_gz to preserve symlinks (follow_symlinks=false) instead
of following them to absolute paths that may not exist on the host.

All tests skip gracefully when tools (nsjail, crane, unshare) are
unavailable, so they won't break CI.

Total: 32 passing (with parquet), 14 parser tests = 46 tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 21:14:40 +00:00
Ruben Fiszel
c69dd024e4 test(sandbox_setup): add nsjail integration tests, fix overlay mount ordering
Add 6 nsjail integration tests that verify actual sandbox execution:
- basic bash execution
- volume read (bind-mount data into sandbox)
- volume write-back (sandbox writes to bind-mounted volume)
- overlay root (snapshot rootfs as sandbox root)
- overlay root + volume mount combined
- result.json output from sandbox

Fix a bug in finalize_nsjail_config where the overlay root mount
(dst: "/") was placed last in the config (via {SHARED_MOUNT}). nsjail
applies mounts in order, so a root bind mount after tmpfs/bind mounts
would overwrite them. The root mount is now moved to be the first mount
block, ensuring subsequent mounts layer correctly on top.

Also add 2 #[ignore] tests for mount_overlay (needs root) and
build_snapshot (needs crane CLI).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 19:01:57 +00:00
Ruben Fiszel
999530e1ee refactor(sandbox_setup): extract testable S3 helpers, add InMemory integration tests
Refactor S3-dependent functions into testable core helpers that accept
Arc<dyn ObjectStore>, enabling real integration tests with InMemory
object store instead of requiring a live S3/MinIO instance.

- Extract: tar_gz, untar_gz, fetch_bytes_from_store, put_bytes_to_store,
  upload_volume_to_store, download_volume_from_store
- Add 6 S3 integration tests using object_store::memory::InMemory:
  volume upload/download roundtrip, overwrite semantics, snapshot tar
  lifecycle, missing key handling, put/fetch bytes roundtrip
- Rename misleading "e2e" tests to "config_pipeline" tests
- Total: 36 tests (14 parser + 22 worker)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 18:45:25 +00:00
Ruben Fiszel
94dc2df312 feat(backend): add sandbox snapshots & volumes for nsjail
Add support for custom rootfs snapshots and persistent volumes in nsjail
sandboxes, enabling users to run scripts with pre-built environments and
shared persistent storage.

- DB migration: sandbox_snapshot + sandbox_volume tables
- API: CRUD endpoints under /api/w/{ws}/sandbox/
- Common: annotation parser for # sandbox: and # volume: directives
- Worker: S3-backed snapshot cache, overlayfs mount, volume sync
- Executors: nsjail config post-processing to swap rootfs when overlay active
- Tests: 28 tests covering parsing, config generation, and end-to-end flows

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 18:30:49 +00:00
hugocasa
90fa5b3ced native mode nits (#7981)
* native mode UI nits

* accept flow jobs on native workers

* limit native mode to non-dep jobs + flow tag infobox
2026-02-17 16:32:52 +00:00
Henri Courdent
45b959711e New Raw App tab name (#7984) 2026-02-17 16:32:40 +00:00
Guilhem
a46924a0f2 fix(frontend): improve folder picker with sticky create button and drawer flow (#7985)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 16:32:25 +00:00
Ruben Fiszel
907ed41093 chore(main): release 1.638.3 (#7983)
* chore(main): release 1.638.3

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
v1.638.3
2026-02-17 13:42:25 +00:00
centdix
f387daa2a6 fix: always create guidance files during wmill init (#7974)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 13:39:14 +00:00
Ruben Fiszel
b094649586 fix(frontend): preserve user config when trimming oneOf non-selected keys
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 13:37:28 +00:00
Guilhem
3ed86816fb fix flow rename (#7978)
* fix(frontend): preserve flow settings when updating summary/path from detail page

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(frontend): type builders prop with ReturnType<typeof createDropdownMenu>

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(frontend): extract shared updateItemPathAndSummary utility to deduplicate move/rename logic

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(frontend): enable inline summary/path editing on script detail page

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* improve layout

* feat(frontend): add dirty tracking to MoveDrawer

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* nit move drawer

* fix(frontend): drop on_behalf_of_email from move/rename and warn user about redeployment

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(frontend): hide on_behalf_of warning in MoveDrawer when user is not owner

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(frontend): only reload script when path unchanged in onSaved callback

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-17 12:48:58 +00:00
hugocasa
2d5393941c fix(frontend): incorrect job result on the runs page (#7982) 2026-02-17 12:22:14 +00:00
Ruben Fiszel
6d1d1f162b chore(main): release 1.638.2 (#7980)
* chore(main): release 1.638.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
v1.638.2
2026-02-17 09:07:08 +00:00
hugocasa
5b7bb2fb84 fix(backend): gcp private key parsing (#7979) 2026-02-17 09:02:52 +00:00
Ruben Fiszel
71608bf669 fix: yaml settings UI mask rsa_keys and jwt_secret 2026-02-17 08:54:58 +00:00
Ruben Fiszel
47c7fe83f4 chore(main): release 1.638.1 (#7977)
* chore(main): release 1.638.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
v1.638.1
2026-02-17 08:44:00 +00:00
Ruben Fiszel
4b8bb72857 operator nits 2026-02-17 08:40:46 +00:00
Ruben Fiszel
b7bec1a83d fix(operator): improve configmap handling of older license keys 2026-02-17 08:38:12 +00:00
Ruben Fiszel
8971dd660c chore(main): release 1.638.0 (#7973)
* chore(main): release 1.638.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
v1.638.0
2026-02-17 08:01:25 +00:00
Ruben Fiszel
b3eeee4131 feat: show all settings in YAML UI and protect from empty overwrites (#7976)
- Show custom_instance_pg_databases, ducklake_settings, ducklake_user_pg_pwd
  and rsa_keys in frontend YAML editor (remove from excludedKeys)
- Redact sensitive values: add ducklake_user_pg_pwd and rsa_keys to
  sensitiveKeys, add custom_instance_pg_databases.user_pwd to
  nestedSensitiveFields
- Remove rsa_keys from HIDDEN_SETTINGS so it appears in YAML export
- Hide automate_username_creation from export (add to HIDDEN_SETTINGS)
- Add ducklake_user_pg_pwd and rsa_keys to SENSITIVE_SETTINGS for log
  redaction
- Generalize empty/null protection for all PROTECTED_SETTINGS: operator
  diff skips empty values when DB has existing data, direct API rejects
  delete/empty for protected settings

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 08:52:31 +01:00
Pyra
bba319b282 fix: download audit logs (#7965)
* feat: download audit logs

Signed-off-by: pyranota <pyra@duck.com>

* add on hover

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
2026-02-17 07:36:59 +00:00
hugocasa
bb03c62c28 fix: add missing google native triggers to triggers panel (#7966) 2026-02-17 07:34:45 +00:00
Ruben Fiszel
2019aecf42 fix: improve operator ConfigMap settings handling (#7975)
* feat: improve operator ConfigMap settings handling

- Protect jwt_secret and min_keep_alive_version from deletion (add to
  PROTECTED_SETTINGS)
- Expose jwt_secret in config exports (remove from HIDDEN_SETTINGS)
- Reject empty/null jwt_secret values with warning
- Clamp retention_period_secs to 30 days max on CE builds
- Improve apply_settings_diff logging: distinguish Created/Updated/Deleted
  with from/to values and unchanged count summary
- Add sensitive value masking in logs with partial redaction (prefix/suffix)
  for top-level secrets and nested sub-field masking for oauths, smtp,
  object_store_cache_config, custom_instance_pg_databases
- Sort global_settings keys alphabetically in YAML export
- Order worker_configs with "default" and "native" first in YAML export
- Add tests for sorted YAML serializer

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review feedback

- Fix redact_string panic on multi-byte UTF-8 by using chars() instead
  of byte-length slicing
- Protect jwt_secret from deletion via direct API
  (set_global_setting_internal rejects empty/null with BadRequest)
- Add code comment documenting jwt_secret visibility trade-off

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 08:34:08 +01:00
Ruben Fiszel
3e313cc4e8 feat: add native_mode as typed field on WorkerGroupConfig
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 00:17:39 +00:00
Ruben Fiszel
c3a76c2cc5 chore(main): release 1.637.0 (#7971)
* chore(main): release 1.637.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
v1.637.0
2026-02-17 00:14:06 +00:00
Guilhem
eb5a8dab74 feat(frontend): inline edit summary & path from header (#7968)
* allow editing flow/script summary

* feat(frontend): wire up edit summary/path on flow detail page

- Fix on:click → onclick (Svelte 5) and add title on Save button
- Make can_write reactive ($state) so onEdit prop updates correctly
- Wire onEdit in flow detail page to call FlowService.updateFlow

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(frontend): use Path component for path editing in detail page header

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(frontend): extract SummaryPathDisplay component with edit popover

Consolidate the summary+path display and edit popover into a reusable
SummaryPathDisplay component, used in both the detail page header and
the flow editor toolbar.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(frontend): add size prop to Path/FolderPicker, compact popover

Add size prop ('sm' | 'md') to Path and FolderPicker components,
passed through to ToggleButton, TextInput, and Button children.
Use hideFullPath and size="sm" in the SummaryPathDisplay popover
for a compact inline path editor. Widen popover to 480px.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix add folder in path editor

* fix(frontend): disable focus trap on edit popover for drawer access

Disable melt-ui's focus trap on the SummaryPathDisplay popover so
that inputs inside drawers (e.g. New Folder) can receive focus.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* nit

* feat(frontend): auto-create folder and render drawer above popover

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(frontend): show placeholder and hover-reveal pencil in SummaryPathDisplay

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(frontend): click-to-edit SummaryPathDisplay with inline layout

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(frontend): move undo/redo and tutorials into dropdown submenu with notification dot

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(frontend): stack path above summary in SummaryPathDisplay

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(frontend): bind summary/path directly in flow builder popover

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit

* chore: add PR screenshots (to be removed before merge)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: remove PR screenshots (moved to release assets)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 00:10:05 +00:00
Ruben Fiszel
f02ef6d03c refactor: switch operator from CRD to ConfigMap (#7972)
* refactor: switch operator from CRD to ConfigMap

Replace the WindmillInstance CRD with a plain ConfigMap for the K8s
operator. This simplifies deployment (no CRD to install/manage, no
ClusterRole for custom API groups) while keeping the same config schema.

- Replace crd_ee.rs with configmap_ee.rs (parses data.spec YAML key)
- Rewrite reconciler_ee.rs: ConfigMap watcher + Event recorder instead
  of CRD Controller + status subresource
- Add license_key preservation: if absent/empty in ConfigMap but present
  in DB, the DB value is kept
- Remove print_crd_yaml() and "operator crd" subcommand
- Drop schemars, chrono, instance_config_schema dependencies
- Delete manifests/crd.yaml
- Update K8s example and README for ConfigMap approach
- RBAC now only needs a namespace-scoped Role (not ClusterRole)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add superadmin YAML export endpoint and remove cache_clear from operator config

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 00:06:56 +00:00
hugocasa
535e108cbf feat: native mode (#7939)
* feat: native mode

* improve

* fix build

* review fixes

* tracing nit
2026-02-16 23:36:41 +00:00
Ruben Fiszel
0940d70a2b chore(main): release 1.636.0 (#7962)
* chore(main): release 1.636.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
v1.636.0 pr-assets
2026-02-16 17:09:46 +00:00
centdix
e7b0b00f56 fix: mark base_url as unsaved when using browser fallback (#7964)
* fix: mark base_url as unsaved when using browser fallback in instance settings

When base_url is not set in the database, the frontend silently fills in
window.location.origin but also snapshots it as the initial value. This
makes the dirty-check see no change, so the Save button stays disabled
and the user cannot persist the auto-detected value.

Fix by snapshotting initialValues before applying the fallback, and show
a yellow warning indicating the value is auto-detected and unsaved.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: render base_url fallback warning inside the setting card

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 17:01:18 +00:00
Ruben Fiszel
4587cd5f3e fix sqlx 2026-02-16 17:00:35 +00:00
Ruben Fiszel
7a5719217b fix sqlx 2026-02-16 16:49:42 +00:00
Ruben Fiszel
ec17b29ae2 fix private check issue 2026-02-16 16:39:59 +00:00
Ruben Fiszel
713c39040b fix private check issue 2026-02-16 16:12:02 +00:00
centdix
5db6a405ad fix review comments (#7963) 2026-02-16 15:53:52 +00:00
hugocasa
6f24f1939d feat: google native triggers (#7837)
* feat: google native triggers

* nit skill

* better native trigger abstraction

* use resources for workspace integrations

* better and better

* better tests

* update native trigger skill

* sqlx

* less tx and google update fix

* refactor a bit the external logic

* nits

* fix

* fix google native trigger update

* fix oauth

* review fixes

* sqlx fix

* nit

* chore: update ee-repo-ref to a10eda4251610cceee67fbe05463b8be82ffa9e0

This commit updates the EE repository reference after PR #416 was merged in windmill-ee-private.

Previous ee-repo-ref: bf3696d5f2a39a3cb84dbbee81e092155f2a8c75

New ee-repo-ref: a10eda4251610cceee67fbe05463b8be82ffa9e0

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-02-16 15:52:02 +00:00
centdix
5b8ec502fe feat: allow adding workspace scripts and flows as AI chat context (#7882)
* feat: allow adding workspace scripts and flows as AI chat context

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* cleaning

* cleaning

* cleaning

* better

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-16 15:46:17 +00:00
Ruben Fiszel
b2128a0109 add dynamic tls support for email 2026-02-16 14:57:28 +00:00
Ruben Fiszel
ba80570357 chore(main): release 1.635.1 (#7958)
* chore(main): release 1.635.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
v1.635.1
2026-02-15 17:14:04 +00:00
Ruben Fiszel
714f713a04 only update license key via IaC when new expiry is posterior (#7959)
When using infrastructure-as-code to update settings, if the desired
license key matches the current one (same client ID and signature) but
differs only in the expiration date, only apply the update if the new
key has a later expiry. This prevents accidental downgrades when an
older license key is present in the IaC configuration.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 17:13:37 +00:00
Ruben Fiszel
68f766e1ae fix: pin strum_macros to 0.27 to match strum version (#7957)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 16:56:39 +00:00
Ruben Fiszel
d9a5cb64b8 chore(main): release 1.635.0 (#7945)
* chore(main): release 2.0.0

* Apply automatic changes

* chore(main): release 1.635.0

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
v1.635.0
2026-02-15 16:48:38 +00:00
Ruben Fiszel
2ca3c8e409 nit 2026-02-15 16:44:43 +00:00
Ruben Fiszel
0cc4e2650c make api for setting instance config more consistent 2026-02-15 16:38:07 +00:00
Ruben Fiszel
bdffba53ed expose 'Set via API' in worker group config drawer 2026-02-15 16:32:38 +00:00
Ruben Fiszel
64532a1d12 list to record for worker config 2026-02-15 16:21:16 +00:00
Ruben Fiszel
8b8e33e2dc feat: IaC hints, YAML editor for worker configs (#7956)
* feat: add CRD configuration hints to instance settings and workers pages

Add subtle text notes with doc links to help K8s users discover that
instance settings and worker group configs can be managed via the
WindmillInstance CRD.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: IaC hints, YAML editor for worker configs, and EE badges

- Move IaC hint from form view to YAML mode in Instance Settings
- Add YAML editor drawer for worker group configs with diff review
- Add EE-only badges to gated fields in worker config drawer
- Make init_bash available in CE (frontend + backend gate)
- Promote Init/Periodic scripts to top-level sections
- Add "YAML editor" button to worker config drawer action bar
- Strip cache_clear and empty fields from YAML output
- Sort default/native groups first in YAML editor

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review feedback on worker YAML editor

- Extract serializeWorkerGroupsAsYaml() shared helper (dedup)
- Fix name:undefined leak using proper destructuring
- Protect default/native groups from accidental deletion
- Add loading indicator during YAML save
- Add restart warning for periodic script changes
- Use consistent on:click event syntax in YAML drawer actions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: use onClick prop instead of on:click for Button components

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 15:53:03 +00:00
Ruben Fiszel
c5d870f480 make K8s operator private and add registry EE warning (#7955)
Move K8s operator source code (crd, db_sync, reconciler, resolve) to
windmill-ee-private and gate behind feature = "private". OSS stubs
provide error messages when the feature is disabled.

Add an info Alert banner in the Registries settings section when no
enterprise license is active.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 14:32:15 +00:00
Ruben Fiszel
6cf3f5f4a3 feat: open-source worker group configuration UI (#7954)
* feat: gate private registry configuration behind enterprise feature

Add read_ee_registry<T> helper that returns None and emits a job log
in CE builds when a private registry is configured. Applied to all
language executors: pip, npm/bun, deno, cargo, go, maven, nuget,
powershell, and ruby. Maven settings.xml file write in monitor.rs
is also gated.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: open-source worker group configuration UI

Allow CE users to create worker group configs and edit worker_tags.
EE-only features (dedicated workers, custom env vars, init/periodic
scripts, autoscaling, priority tags, etc.) remain gated behind the
enterprise license in the UI. Backend now filters CE config updates
to only allow worker_tags and cache_clear fields instead of rejecting
the entire request.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 14:05:23 +00:00
Ruben Fiszel
423e07376b feat: gate private registry configuration behind enterprise feature (#7953)
Add read_ee_registry<T> helper that returns None and emits a job log
in CE builds when a private registry is configured. Applied to all
language executors: pip, npm/bun, deno, cargo, go, maven, nuget,
powershell, and ruby. Maven settings.xml file write in monitor.rs
is also gated.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 14:02:14 +00:00