Reduce DB polling overhead for native workers by batch-fetching jobs
server-side and serving them from an in-memory buffer via HTTP.
- Add batch_pull() in windmill-queue: single SELECT...FOR UPDATE SKIP LOCKED LIMIT N
- Add batch pull SQL helpers (make_batch_pull_query, format_batch_pull_query)
- OSS stubs for agent-workers accept batch_buffer parameter (4-tuple return)
- Native workers self-sign JWT and pull jobs via HTTP when co-located with server
- Add uses_batch_http_pull column to worker_ping for server-side tracking
- Worker pull loop: HTTP batch pull when client available, SQL otherwise
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: replace hub error toasts with warning alerts and add disable hub setting
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: guard hub script cache refresh when hub is disabled
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Prevent workspace corruption when re-encryption fails mid-loop by
wrapping the key update and variable re-encryption in a single
transaction. If any step fails, the entire operation rolls back.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add variable and resource types to flow env variables
Flow env variables can now reference workspace variables ($var:path)
and resources ($res:path) that are resolved at runtime. Adds Variable
and Resource type options to the flow env editor with ItemPicker and
ResourcePicker components, and resolves references in both the flow
worker (via transform_json) and the API fallback endpoint.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(frontend): use inline DollarSign icon for variable picker
Replace the separate "Pick" button with the standard inline DollarSign
icon overlay that appears on hover, matching the existing ArgInput
pattern. Also add the icon to the string type input for quick variable
linking from any string field.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: simplify flow env var resolution and json_path handling in API
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(frontend): always show flow env variables in property picker
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: update flow_env openapi type to allow any JSON value
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* refactor(frontend): remove redundant variable type from env var dropdown
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(frontend): use Label component and fix alert text in flow env vars editor
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(frontend): avoid redundant stringify/parse roundtrip in env type switch
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: address PR review comments for flow env vars
- Deduplicate db_authed in jobs.rs $var/$res resolution
- Add warn logging on variable/resource resolution failures
- Consolidate $effect blocks and remove auto-type-correction effect
- Make linked variable text a clickable link to variable editor
- Add hash-based variable editor opening on variables page
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* perf: avoid cloning entire FlowValue to resolve flow_env references
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: auto-heal corrupted python runtime cache on remote workers
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Revert "fix: auto-heal corrupted python runtime cache on remote workers"
This reverts commit 0ea013a554.
* feat: make WINDMILL_DIR configurable via environment variable
Allow users to configure the base directory for Windmill's tmp/cache files
via the WINDMILL_DIR env var (default: /tmp/windmill). This fixes Python
runtime cache corruption on RHEL systems where systemd-tmpfiles-clean
removes files from /tmp.
Converts TMP_DIR (renamed to WINDMILL_DIR) and all derived cache directory
constants from compile-time const &str (concatcp!) to runtime lazy_static
String values.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: update ee ref
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: update ee ref
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: deref ERROR_DIR lazy_static for AsRef<Path> and Display traits
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: update ee ref to branch name for CI compatibility
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: deref lazy_static constants in all executor files
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: update ee ref
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: update ee ref
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: update ee ref
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: panic if WINDMILL_DIR has trailing slash
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: also reject trailing backslash in WINDMILL_DIR for Windows
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: deref GO_BIN_CACHE_DIR in test utils
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: replace remaining hardcoded /tmp/windmill paths and validate empty WINDMILL_DIR
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: nsjail powershell mount dst, Windows path assumptions, pwsh deref consistency
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: restore Windows /tmp path translation in go and bun executors
The Windows path translation replaces /tmp with the Windows temp dir
(e.g. C:\tmp) before normalizing slashes. Without this, the default
WINDMILL_DIR=/tmp/windmill produces paths without a drive letter on
Windows.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: update ee-repo-ref to 6fd5a2ce908235a17975ad4dbdf0051cd89334f3
This commit updates the EE repository reference after PR #436 was merged in windmill-ee-private.
Previous ee-repo-ref: e8c03e16720833230ebd1878b4c63642ecc6c80f
New ee-repo-ref: 6fd5a2ce908235a17975ad4dbdf0051cd89334f3
Automated by sync-ee-ref workflow.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* feat: move index management endpoints out of /srch/, add storage size reporting
- Mount management_service() at /api/indexer (authenticated)
- Add management_service() OSS stub in indexer_oss.rs
- Update OpenAPI: /indexer/delete/{idx_name} and /indexer/storage
- Show disk + S3 storage sizes in IndexerMemorySettings UI
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add index storage section with refresh button
Move storage sizes into a dedicated "Index storage" section with a
refresh button to reload sizes after clearing an index.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add indexer status endpoint with liveness detection and improve settings UI
Add GET /indexer/status endpoint that combines lock-based liveness
detection with storage sizes. Frontend now shows running/stopped
indicators with last-active timestamps for each indexer.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* update ee ref
* fix
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
When an OAuth provider entry in instance settings has unexpected types
(e.g. `"true"` instead of `true` for req_body_auth), the entire
/api/settings/instance_config endpoint would fail with a deserialization
error, preventing access to any instance settings.
Introduce OAuthClientEntry enum that tries typed OAuthClient
deserialization first and falls back to raw JSON, logging the
deserialization error. This allows the settings page to load even when
individual OAuth entries are malformed.
Also show a user-visible error toast in SaveButton on save failure
instead of only logging to console.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: skip stop_after_if evaluation for skipped (identity) flow steps
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: relax is_identity_job guard to only require skip_if
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: use exact matching for python requirements directive parsing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: apply same exact matching fix to CLI parser
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: preserve debouncing settings for post-preprocessing arg accumulation
After preprocessing completes, store the flow's debouncing settings in
runnable_settings_handle on v2_job_queue so that maybe_apply_debouncing
can find them when the surviving job is pulled. Without this, the handle
is NULL and arg accumulation silently does nothing for flows with
preprocessors.
Also adds a debouncing badge in flow settings and 4 focused accumulation
tests covering scripts, flows without preprocessor, flows with
preprocessor (with and without the fix).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: update sqlx prepared query for worker_flow.rs change
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix(frontend): preserve keycloak realm url between instance settings saves
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(backend): preserve provider-specific oauth fields through round-trip
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: add missing grant_types field to instance config OAuth structs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: add missing display_name and tenant fields to instance config OAuthClient
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
- Show recurring toast every 15s (8s duration) when loading takes long, with stop button
- When streaming by batches of 25 and a batch takes >4s, offer to stream 1 by 1
- Expose batch size in progress bar with editable input to customize on the fly
- Make stop button more prominent (destructive Button component)
- Fix list_jobs UNION: exclude queue jobs when filtering by result or is_skipped=true
- Add "Show skipped" preset to runs filter
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
When failures are sparse (<1%), filtering by failure status on the runs
page required scanning millions of success rows. Add a partial index on
v2_job_completed (workspace_id, completed_at DESC) WHERE status IN
('failure', 'canceled') and switch ORDER BY to completed_at when
filtering failures, so Postgres walks the small partial index directly.
Benchmarked at 5.2M rows / 1% failure rate:
- LIMIT 30: 800ms -> 0.4ms (2000x faster)
- LIMIT 1000: 550ms -> 21ms (26x faster)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: copy deps and remove user auto-add on workspace fork
Clone workspace_dependencies to forked workspaces and remove
automatic workspace_invite creation for parent workspace users.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: update sqlx offline cache
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Support arg type decl in postgres
* Python datatable client no longer requires explicit arg typing
* compilation fix
* Set correct type in statement exec
* reset to main
* Explicit pg arg types
* remove code duplication
* update parser js
* FLOAT8 doesn't have space
---------
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
When connecting Nextcloud via workspace integration OAuth, the resource
now includes baseUrl (from OAuth config) and userId (fetched from
Nextcloud OCS API) alongside the token, making it immediately usable
by scripts. Falls back to token + baseUrl if user info fetch fails.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: slim down claude instructions for lean context and fast iteration
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: add private and license feature flags to enterprise validation docs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add /refine skill for end-of-session doc evolution
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: remove architecture.md overview doc per research findings
General codebase overviews distract agents and trigger unnecessary
exploration. Keep only operational docs (validation, enterprise).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add autonomous mode doc for bypass permission workflows
Covers: plan-first requirement, tmux pane usage for checking
backend/frontend logs, manual testing via Playwright MCP,
Playwright gotchas, and end-of-task summary expectations.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add mermaid, playwright, and asciinema tools to autonomous mode doc
Claude should use mmdc for diagrams during planning, playwright CLI for
screenshots of frontend changes, and asciinema for terminal recordings
of CLI changes. All attached to the PR.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: use pastebin for screenshot/recording uploads
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address PR review findings
- Remove stale docs/architecture.md reference from /refine skill
- Fix script name: ./update-sqlx -> ./update_sqlx.sh
- Remove .claude/settings.local.json mention from enterprise doc
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: replace SELECT * with explicit columns in teams command query
- Update sqlx offline cache for the changed query
- Fix write_latest_ee_ref.sh to prefer matching EE worktree branch
- Update ee-repo-ref.txt
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: update ee-repo-ref to 8ffae1f43b31dc8136714fa612d22b6301773e27
This commit updates the EE repository reference after PR #434 was merged in windmill-ee-private.
Previous ee-repo-ref: da1f8bf8676f85cac2b6fa2705246e1819d4b6f0
New ee-repo-ref: 8ffae1f43b31dc8136714fa612d22b6301773e27
Automated by sync-ee-ref workflow.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* feat: add force branch param to git sync settings
* update hub path
* chore: update ee-repo-ref to a797dd4d619cdab737e133ce593f2f8582ba21de
This commit updates the EE repository reference after PR #430 was merged in windmill-ee-private.
Previous ee-repo-ref: 373c5870ae5499c8c8a22cb92b2fd3a64a651183
New ee-repo-ref: a797dd4d619cdab737e133ce593f2f8582ba21de
Automated by sync-ee-ref workflow.
* UI nits
* nit
---------
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
When a flow has a flow-level `skip_expr` (or `no_flow_overlap`) and the first
module has `skip_if` defined, the flow-level condition returns `UpdateFlow`
before any identity job is created. The `UpdateFlow` path passes `Uuid::nil()`
as `job_id_for_status`, causing `fetch_one` to fail with "no rows returned".
- Change `fetch_one` to `fetch_optional` so a missing row returns false
- Short-circuit the DB query with `stop_early && skip_if_stop_early` so both
skip mechanisms (identity job check and early-stop skip flag) are considered
- Also fixes the logical gap where a module with both `skip_if` and
`stop_after_if` would only check the identity job, ignoring the early-stop
skip signal
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
The num_columns i16 was written twice, causing the parser to read
the second copy as column data and misparse the column name.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: optimize slow list_assets query with covering index and v2_job join fix
Add a covering index on asset(workspace_id, path, kind, created_at DESC, id DESC)
with INCLUDE(usage_kind, usage_path) to enable index-only scans for the CTE aggregation.
Fix v2_job join to cast asset.usage_path::uuid instead of job.id::text, allowing
PostgreSQL to use the job_pkey primary key index instead of seq scanning the entire table.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: drop redundant asset indexes subsumed by new covering index
idx_asset_workspace_created_id and idx_asset_kind_path are fully covered
by the new idx_asset_ws_path_kind_recent + the primary key. Verified all
asset table queries still have optimal index coverage. Reduces write
amplification on inserts.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: use @-prefixed LIKE pattern for email domain matching in auto-invite
The SQL queries for auto-add and auto-invite used `LIKE CONCAT('%', domain)`
which could match emails from unrelated domains (e.g., `user@barfoo.com`
matching domain `foo.com`). Changed to `LIKE CONCAT('%@', domain)` so only
exact domain suffixes match.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: update ee-repo-ref.txt
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
