* refactor: extract object store code into windmill-object-store crate with filesystem backend
Consolidate all object_store-dependent code from windmill-common into a new
windmill-object-store crate. Add a filesystem-backed object store implementation
using LocalFileSystem for dev/testing without cloud credentials. Includes 30
comprehensive tests covering render_endpoint, lfs_to_object_store_resource,
duckdb_connection_settings, error mapping, and filesystem-backed integration tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* all
* all
* all
* all
* fix: fix raw_app hardcoded path, add missing ObjectStoreResource import, and add tests
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: move S3ModeFormat to windmill-types, make windmill-parser-sql optional, restore debug logs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* all
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Move nsjail config, overlayfs, tar utilities, S3 operations, and sandbox
types from windmill-common and windmill-worker into a dedicated
windmill-sandbox crate. S3 logic lives in windmill-ee-private (symlinked
s3_ee.rs). Both windmill-api and windmill-worker depend on the new crate.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit updates the EE repository reference after PR #420 was merged in windmill-ee-private.
Previous ee-repo-ref: 931813b75b8260faa13ddc07f36a11607b7e3bf6
New ee-repo-ref: 592848d59ca2304926fb2bd85d000668a7f46a77
Automated by sync-ee-ref workflow.
mount_overlay now tries kernel `mount -t overlay` first (needs root or
CAP_SYS_ADMIN), then falls back to fuse-overlayfs (works unprivileged).
unmount_overlay uses fusermount3 -u for FUSE mounts, umount for kernel.
OverlayMount gains an `is_fuse` field to track which backend was used.
The overlay integration tests now exercise the actual mount_overlay()
and unmount_overlay() Rust functions (via fuse-overlayfs on non-root),
verifying read-through, copy-on-write, and cleanup semantics.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace placeholder #[ignore] tests with real integration tests:
- overlay_integration: test overlayfs read/write semantics via unshare
(no root required), and overlayfs rootfs as nsjail sandbox root
- crane_integration: test crane export of alpine image → tar/gz
roundtrip, and running the extracted rootfs in nsjail
Also fix tar_gz to preserve symlinks (follow_symlinks=false) instead
of following them to absolute paths that may not exist on the host.
All tests skip gracefully when tools (nsjail, crane, unshare) are
unavailable, so they won't break CI.
Total: 32 passing (with parquet), 14 parser tests = 46 tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add 6 nsjail integration tests that verify actual sandbox execution:
- basic bash execution
- volume read (bind-mount data into sandbox)
- volume write-back (sandbox writes to bind-mounted volume)
- overlay root (snapshot rootfs as sandbox root)
- overlay root + volume mount combined
- result.json output from sandbox
Fix a bug in finalize_nsjail_config where the overlay root mount
(dst: "/") was placed last in the config (via {SHARED_MOUNT}). nsjail
applies mounts in order, so a root bind mount after tmpfs/bind mounts
would overwrite them. The root mount is now moved to be the first mount
block, ensuring subsequent mounts layer correctly on top.
Also add 2 #[ignore] tests for mount_overlay (needs root) and
build_snapshot (needs crane CLI).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add support for custom rootfs snapshots and persistent volumes in nsjail
sandboxes, enabling users to run scripts with pre-built environments and
shared persistent storage.
- DB migration: sandbox_snapshot + sandbox_volume tables
- API: CRUD endpoints under /api/w/{ws}/sandbox/
- Common: annotation parser for # sandbox: and # volume: directives
- Worker: S3-backed snapshot cache, overlayfs mount, volume sync
- Executors: nsjail config post-processing to swap rootfs when overlay active
- Tests: 28 tests covering parsing, config generation, and end-to-end flows
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Show custom_instance_pg_databases, ducklake_settings, ducklake_user_pg_pwd
and rsa_keys in frontend YAML editor (remove from excludedKeys)
- Redact sensitive values: add ducklake_user_pg_pwd and rsa_keys to
sensitiveKeys, add custom_instance_pg_databases.user_pwd to
nestedSensitiveFields
- Remove rsa_keys from HIDDEN_SETTINGS so it appears in YAML export
- Hide automate_username_creation from export (add to HIDDEN_SETTINGS)
- Add ducklake_user_pg_pwd and rsa_keys to SENSITIVE_SETTINGS for log
redaction
- Generalize empty/null protection for all PROTECTED_SETTINGS: operator
diff skips empty values when DB has existing data, direct API rejects
delete/empty for protected settings
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: improve operator ConfigMap settings handling
- Protect jwt_secret and min_keep_alive_version from deletion (add to
PROTECTED_SETTINGS)
- Expose jwt_secret in config exports (remove from HIDDEN_SETTINGS)
- Reject empty/null jwt_secret values with warning
- Clamp retention_period_secs to 30 days max on CE builds
- Improve apply_settings_diff logging: distinguish Created/Updated/Deleted
with from/to values and unchanged count summary
- Add sensitive value masking in logs with partial redaction (prefix/suffix)
for top-level secrets and nested sub-field masking for oauths, smtp,
object_store_cache_config, custom_instance_pg_databases
- Sort global_settings keys alphabetically in YAML export
- Order worker_configs with "default" and "native" first in YAML export
- Add tests for sorted YAML serializer
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address PR review feedback
- Fix redact_string panic on multi-byte UTF-8 by using chars() instead
of byte-length slicing
- Protect jwt_secret from deletion via direct API
(set_global_setting_internal rejects empty/null with BadRequest)
- Add code comment documenting jwt_secret visibility trade-off
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: switch operator from CRD to ConfigMap
Replace the WindmillInstance CRD with a plain ConfigMap for the K8s
operator. This simplifies deployment (no CRD to install/manage, no
ClusterRole for custom API groups) while keeping the same config schema.
- Replace crd_ee.rs with configmap_ee.rs (parses data.spec YAML key)
- Rewrite reconciler_ee.rs: ConfigMap watcher + Event recorder instead
of CRD Controller + status subresource
- Add license_key preservation: if absent/empty in ConfigMap but present
in DB, the DB value is kept
- Remove print_crd_yaml() and "operator crd" subcommand
- Drop schemars, chrono, instance_config_schema dependencies
- Delete manifests/crd.yaml
- Update K8s example and README for ConfigMap approach
- RBAC now only needs a namespace-scoped Role (not ClusterRole)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add superadmin YAML export endpoint and remove cache_clear from operator config
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
When using infrastructure-as-code to update settings, if the desired
license key matches the current one (same client ID and signature) but
differs only in the expiration date, only apply the update if the new
key has a later expiry. This prevents accidental downgrades when an
older license key is present in the IaC configuration.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add CRD configuration hints to instance settings and workers pages
Add subtle text notes with doc links to help K8s users discover that
instance settings and worker group configs can be managed via the
WindmillInstance CRD.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: IaC hints, YAML editor for worker configs, and EE badges
- Move IaC hint from form view to YAML mode in Instance Settings
- Add YAML editor drawer for worker group configs with diff review
- Add EE-only badges to gated fields in worker config drawer
- Make init_bash available in CE (frontend + backend gate)
- Promote Init/Periodic scripts to top-level sections
- Add "YAML editor" button to worker config drawer action bar
- Strip cache_clear and empty fields from YAML output
- Sort default/native groups first in YAML editor
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address PR review feedback on worker YAML editor
- Extract serializeWorkerGroupsAsYaml() shared helper (dedup)
- Fix name:undefined leak using proper destructuring
- Protect default/native groups from accidental deletion
- Add loading indicator during YAML save
- Add restart warning for periodic script changes
- Use consistent on:click event syntax in YAML drawer actions
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: use onClick prop instead of on:click for Button components
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Move K8s operator source code (crd, db_sync, reconciler, resolve) to
windmill-ee-private and gate behind feature = "private". OSS stubs
provide error messages when the feature is disabled.
Add an info Alert banner in the Registries settings section when no
enterprise license is active.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: gate private registry configuration behind enterprise feature
Add read_ee_registry<T> helper that returns None and emits a job log
in CE builds when a private registry is configured. Applied to all
language executors: pip, npm/bun, deno, cargo, go, maven, nuget,
powershell, and ruby. Maven settings.xml file write in monitor.rs
is also gated.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: open-source worker group configuration UI
Allow CE users to create worker group configs and edit worker_tags.
EE-only features (dedicated workers, custom env vars, init/periodic
scripts, autoscaling, priority tags, etc.) remain gated behind the
enterprise license in the UI. Backend now filters CE config updates
to only allow worker_tags and cache_clear fields instead of rejecting
the entire request.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Add read_ee_registry<T> helper that returns None and emits a job log
in CE builds when a private registry is configured. Applied to all
language executors: pip, npm/bun, deno, cargo, go, maven, nuget,
powershell, and ruby. Maven settings.xml file write in monitor.rs
is also gated.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
