Compare commits

..

62 Commits

Author SHA1 Message Date
dependabot[bot]
9af8e4e626 chore(deps): bump actions/setup-node from 3 to 6
Bumps [actions/setup-node](https://github.com/actions/setup-node) from 3 to 6.
- [Release notes](https://github.com/actions/setup-node/releases)
- [Commits](https://github.com/actions/setup-node/compare/v3...v6)

---
updated-dependencies:
- dependency-name: actions/setup-node
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-03-13 12:45:23 +00:00
Ruben Fiszel
44dd3ee8cd fix(ci): remove provenance flag and use NPM_TOKEN for npm publish
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 12:43:45 +00:00
Ruben Fiszel
2a8e276b6d fix(ci): add NODE_AUTH_TOKEN for npm publish authentication
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 12:38:41 +00:00
Ruben Fiszel
bc35c94616 ci 2026-03-13 12:29:58 +00:00
Guilhem
b585dee64d fix(frontend): collapse flow topbar buttons to icon-only in narrow panes (#8322)
* feat: collapse flow topbar buttons to icon-only mode in narrow panes

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: show delete button on top-right of compact error handler

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: preserve bug icon and diff action bar in compact error handler

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: avoid duplicate delete buttons when diff action bar is active

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: use undefined instead of empty string for wrapperClasses

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-13 12:08:19 +00:00
hugocasa
96229575e6 chore: dev tooling — wm-ts-nav navigator, format hooks, review skill (#8337)
* chore: remove wm-cursor, add local-review skill, update PR skill for EE

- Remove the unused wm-cursor script and all references to it in
  README_WORKMUX_DEV.md and worktree-common.sh
- Add /local-review skill for code review (bugs + CLAUDE.md compliance)
- Add EE companion PR workflow to the /pr skill

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: add wm-ts-nav tree-sitter navigator and fix format hooks

- Add wm-ts-nav: standalone tree-sitter code navigator with SQLite index
  for fast symbol search, definition lookup, and file outlines across
  Rust, TypeScript, and Svelte files (~12ms warm, ~1s cold for 482 files)
- Fix format hooks: surface errors instead of swallowing with 2>/dev/null,
  use direct prettier path with svelte plugin, add success feedback
- Add wm-ts-nav commands to settings allow list
- Document wm-ts-nav usage in CLAUDE.md

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(wm-ts-nav): add refs command and --parent filter

- refs: find usages of a symbol in code, skipping comments and strings
  (tree-sitter AST walk, ~46ms for 482 files vs grep's 4ms but no noise)
- --parent filter on search: find all methods on a type across all files
  (e.g. search "%" --kind function --parent ServiceName)
- Update CLAUDE.md with clearer when-to-use guidance

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(wm-ts-nav): index refs in DB with import-path resolution

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(wm-ts-nav): add body, callers, callees commands and refs --file/--caller

- body: extract a symbol's source code from disk using indexed line ranges
- callers: cross-file call graph via SQL join of refs + symbols tables
- callees: list all identifiers referenced within a symbol's body
- refs --file: scope results to files matching a substring
- refs --caller: annotate each ref with the containing function name

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(wm-ts-nav): add auto-rebuilding wrapper script

The `wm-ts-nav/nav` wrapper checks if source files are newer than the
binary and rebuilds automatically. Invoked via `sh wm-ts-nav/nav` to
avoid needing executable permissions after clone.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: tighten CLAUDE.md nav section for actionable guidance

Remove redundant question→command mapping, latency numbers, and
excessive examples. Lead with "prefer wm-ts-nav over Read to save
context window" and keep only the patterns that change behavior.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: revert backend/Cargo.lock to main

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: promote wm-ts-nav in workflow, copy binary to worktrees

- CLAUDE.md: integrate wm-ts-nav into Workflow step 1 and Core
  Principles so agents use outline/body before full file reads
- workmux: copy built binary via files.copy
- worktree-common.sh: copy binary in wm_copy_dependencies for webmux

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(wm-ts-nav): fix double indexing, add TSX grammar, remove needless clone

- Reuse index stats from the pre-query update instead of indexing twice
  on the Index command
- Add Lang::Tsx variant so .tsx/.jsx files use LANGUAGE_TSX instead of
  LANGUAGE_TYPESCRIPT (Svelte stays on TS since script blocks are pure TS)
- Remove source.clone() for non-Svelte files — move directly instead

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(wm-ts-nav): fix svelte line numbers, add class methods, innermost caller

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 12:07:49 +00:00
Ruben Fiszel
2d5b72b3ce chore: update vite/vitest to stable v8/v4 and remove legacy-peer-deps (#8349)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-13 08:50:19 +00:00
Alexander Petric
2e430c4c0b feat: add GitHub Enterprise Server (GHES) support for GitHub App git sync (#8344)
* feat: add GitHub Enterprise Server (GHES) support for GitHub App git sync

Add a self-managed GitHub App mode alongside the existing managed
(stats.windmill.dev) mode, enabling git sync for GitHub Enterprise Server
and custom GitHub App installations.

Backend:
- Parameterize GitHub API URLs (no more hardcoded github.com)
- Add GITHUB_ENTERPRISE_APP_SETTING global setting
- Add OpenAPI specs for ghes_installation_callback and ghes_config endpoints

Frontend:
- Add instance settings UI for configuring self-managed GitHub Apps
  with setup instructions and validation
- GHES installation flow in gh_success page
- Dynamic installation URL based on GHES config
- Increase git sync test connection timeout to 10s
- Block "Review changes" save when settings are invalid

EE companion PR: windmill-labs/windmill-ee-private#<PR_NUMBER>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref to c74c86b78a66b976fd9968b21f77903723e668ec

This commit updates the EE repository reference after PR #459 was merged in windmill-ee-private.

Previous ee-repo-ref: 45e4550110799525b5502cf072c8af8132492638

New ee-repo-ref: c74c86b78a66b976fd9968b21f77903723e668ec

Automated by sync-ee-ref workflow.

* sqlx

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-03-13 08:21:15 +00:00
Pyra
4c2c165a5b feat(cli): add unified generate-metadata command (#8335)
* feat(cli): add unified generate-metadata command

- Add generate-metadata command that calls script, flow, and app handlers
- Export generateLocks from flow.ts and generateMetadata from script.ts
- Add deprecation warnings to individual metadata commands

* feat(cli): improve unified generate-metadata command

- Use internal handlers for single-pass collection of stale items
- Add --dry-run flag to show what would be updated
- Fix WASM parser init deprecation warning
- Add comprehensive tests for all flags
- Match original handler behavior for per-item messages

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(cli): add skip flags and comprehensive tests for generate-metadata

- Add --skip-scripts, --skip-flows, --skip-apps flags for granular control
- --schema-only now properly skips flows and apps (they only have locks)
- Dynamic "Checking X, Y, Z..." message based on what's being processed
- Show warning when all types are skipped
- Add comprehensive tests for all flags:
  - --dry-run shows stale items without updating
  - --schema-only only processes scripts
  - --skip-scripts, --skip-flows, --skip-apps work correctly
  - skipping all types shows warning
  - 'All metadata up-to-date' when nothing to update

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* improve output

Signed-off-by: pyranota <pyra@duck.com>

* refactor(cli): add shared test fixtures with cross-links

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(cli): add folder argument to generate-metadata command

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-13 06:10:28 +00:00
wendrul
2d7f325bb8 add a hint to commit message on git sync for renames (#8343)
* add a hint to commit message on git sync for renames

* chore: update ee-repo-ref to 344e2aa60d8cafe08b8c57445d2f9555bad7625a

This commit updates the EE repository reference after PR #458 was merged in windmill-ee-private.

Previous ee-repo-ref: ad4c90cc96ad1b4511cba6b6ee28351895882156

New ee-repo-ref: 344e2aa60d8cafe08b8c57445d2f9555bad7625a

Automated by sync-ee-ref workflow.

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-13 06:08:33 +00:00
Diego Imbert
0a838ca5dc separate storage tabs (#8341)
* Separate storage settings

* Separate VolumeStorageSettings

* nit
2026-03-13 06:05:27 +00:00
wendrul
8e3b8bdfd2 fix: Linked resources and vars not triggering both sync jobs on delete (#8342)
* fix: Linked resources and vars not triggering both sync jobs on delete

* prepare sqlx
2026-03-13 06:04:44 +00:00
Ruben Fiszel
d9d45cf2f9 fix: lower default indexer memory/batch settings to prevent OOM (#8347)
The windmill-indexer pod was crash-looping due to OOMKilled (exit 137)
with a 2Gi memory limit. Two concurrent tantivy IndexWriters each
allocating 300MB (600MB total), combined with large uncommitted batches
of 50K jobs, caused memory exhaustion during indexing.

- writer_memory_budget: 300MB → 150MB (2 writers = 300MB total)
- commit_job_max_batch_size: 50,000 → 10,000
- commit_log_max_batch_size: 10,000 → 5,000

These can still be overridden via env vars or global settings.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 06:03:45 +00:00
Ruben Fiszel
54202e4a96 nit 2026-03-13 06:03:31 +00:00
Ruben Fiszel
36b9db903b clean stale v8 build cache to prevent CI link failures (#8348)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 05:48:47 +00:00
Ruben Fiszel
aae77d6598 customUi sandbox 2026-03-13 05:48:20 +00:00
Ruben Fiszel
724d1350d0 fix: graceful shutdown instead of panic on job completion channel failure (#8345)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 05:14:02 +00:00
Ruben Fiszel
a0337e3b4a adjust exports 2026-03-13 04:30:54 +00:00
Ruben Fiszel
55755cb822 system_prompts update 2026-03-12 15:01:19 +00:00
Ruben Fiszel
749964e326 ci: add freshness check for auto-generated system prompts (#8338)
* ci: add freshness check for auto-generated system prompts

Add a CI workflow and script to verify system_prompts/auto-generated/
stays in sync with its source files (SDKs, schemas, CLI commands, etc).
Also remove the hardcoded CLI version from generated output to avoid
unnecessary churn on every release.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* imports

* imports

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* regenerate system prompts after rebase on main

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 14:44:27 +00:00
Ruben Fiszel
9f7f666af4 chore(main): release 1.655.0 (#8312)
* chore(main): release 1.655.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-03-12 14:05:09 +00:00
hugocasa
ec20d76216 feat: add auto_commit option to Kafka triggers with advanced UI badges (#8317)
* feat: add auto_commit option to Kafka triggers with manual commit API

Add ability to disable auto-commit on Kafka triggers so users can
manually commit offsets after processing messages. This prevents
message loss when processing fails.

Changes:
- Add `auto_commit` column to kafka_trigger table (default true)
- Add POST /kafka_triggers/commit_offsets/{path} endpoint using
  BaseConsumer with manual assign() to avoid rebalance
- Enrich trigger_info payload with partition and offset fields
- Conditionally commit based on auto_commit setting
- Add auto-commit toggle to frontend Kafka trigger config
- Add commitKafkaOffsets helpers to Python and TypeScript SDKs
- Add integration tests for auto_commit DB defaults

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: use DB-based pending commits for kafka manual offset commit

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: pass trigger_path to all v2 preprocessors, secure commit_offsets endpoint, fix commit semantics

- Add trigger_path to v2 preprocessor event for all trigger types (kafka, nats, sqs, mqtt, gcp, postgres, websocket, http, email)
- Secure commit_offsets endpoint: infer trigger from job token (OptJobAuthed) instead of requiring trigger path parameter
- Fix auto_commit: only commit offset after successful job push
- Fix pending commits: commit offset+1 (Kafka semantics) and use CommitMode::Sync
- Update TS/Python clients and frontend preprocessor templates

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add advanced section badges and reorganize kafka trigger settings

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove dead wm_trigger assertions from kafka e2e test

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* sqlx

* refactor: remove unused advancedCollapsed state from all trigger editors

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* update ref

* chore: update ee-repo-ref to ed2c9d360e6fab866b9744cc79f50038d1fc7152

This commit updates the EE repository reference after PR #452 was merged in windmill-ee-private.

Previous ee-repo-ref: 5b31116a1d5a042c6a780732901cfd89584d1773

New ee-repo-ref: ed2c9d360e6fab866b9744cc79f50038d1fc7152

Automated by sync-ee-ref workflow.

* fix: use path-based auth for kafka commit_offsets endpoint

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref to fcd3ea52b0cc94fbe1159baf662a38da947456de

This commit updates the EE repository reference after PR #457 was merged in windmill-ee-private.

Previous ee-repo-ref: b3a5c33c92cb1b2caf7a65986d71da291ff72a35

New ee-repo-ref: fcd3ea52b0cc94fbe1159baf662a38da947456de

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-03-12 14:00:30 +00:00
hugocasa
d2b9799ac4 test: git sync E2E tests + auto-manage git sync script version (#8253)
* test: add E2E git sync integration tests with Gitea

Add 7 end-to-end tests that verify the full git sync pipeline:
deploy objects in Windmill → DeploymentCallback job runs hub sync script →
correct files appear in a Gitea git repository.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: auto-manage git sync script version locked to Windmill release

- Add LATEST_GIT_SYNC_SCRIPT_PATH constant as single source of truth
- Backend auto-fills empty script_path with latest on save
- New repos use empty script_path (auto-managed by backend)
- Existing repos with pinned versions show warning with opt-in button
- cache_hub_scripts always caches the latest constant
- Rename hubPaths.json gitSync entries to deprecated_ prefix

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref.txt for git-sync-tests branch

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref.txt

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove upgrade_git_sync_script_paths from save path

Empty script_path is now resolved to latest at job dispatch time in EE,
not on save. Users opt in via the UI button.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: use Option<String> for git sync script_path

None means auto-managed (uses LATEST_GIT_SYNC_SCRIPT_PATH),
Some(path) means pinned to a specific script. Resolution happens
at job dispatch time via effective_script_path().

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: separate git sync tests into dedicated CI workflow

- Remove git_sync_test from default integration test suite
- Move gitea service to dedicated docker-compose.git-sync.yml
- Add run_git_sync.sh script
- New workflow triggers on changes to git sync crate, hub paths,
  ee-repo-ref, or the test files themselves

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add Rust integration tests for git sync filtering logic

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: run git sync E2E tests via cargo run instead of docker image

Build from source and run Windmill directly, start Gitea as a
standalone container. Tests run against localhost — no pre-built
Docker image needed, works on PRs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add relevance check before running git sync E2E tests

Only run the expensive build+test when actually relevant:
- Direct git sync file changes: always run
- ee-repo-ref.txt changed: check if EE diff touches windmill-git-sync/
- Unrelated changes to workspaces.rs or other files: skip

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove broad path triggers from git sync workflow

Remove workspaces.rs and wmill_integration_test_utils.py from path
triggers - they change too often for unrelated reasons. Keep only
git-sync-specific paths + ee-repo-ref.txt (filtered by check-relevance).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: rewrite git sync E2E tests with full coverage and fix test infra

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: remove accidentally committed gen files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: remove unit/integration tests for git sync filtering (covered by E2E)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: use correct build features and pass license key to test step in CI

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: add workflow_dispatch trigger to git sync test workflow

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Update commit reference in ee-repo-ref.txt

* fix: update stats_oss stubs to match EE telemetry signature changes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: disable -D warnings for git sync e2e build step

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: allow test connection button in auto-managed git sync mode

The test connection button was disabled and runTestJob() bailed out
when script_path was unset. The test job uses a separate hub script
(gitSyncTest), not the sync script, so the guard was wrong.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref to include auto-managed script_path fix

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: use full SHA in ee-repo-ref.txt

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review feedback

- Initialize current_count before loop in wait_for_sync_jobs
- Clean up temp directories in clone helpers with addCleanup
- Fail CI startup steps if Gitea/Windmill never become ready
- Assert exact job count in exclude_path test
- Remove docs/git-sync-tests-plan.md (stale planning doc)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: remove orphaned git_sync.sql fixture

No longer referenced after Rust integration tests were removed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: assert old file removal in rename test

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref to 612d96a66f9d0cfdae335ef3eb4881f3444ce7cd

This commit updates the EE repository reference after PR #442 was merged in windmill-ee-private.

Previous ee-repo-ref: a05004a7c82f3d1ee5f6863bb9f5a33827d30032

New ee-repo-ref: 612d96a66f9d0cfdae335ef3eb4881f3444ce7cd

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-12 12:48:38 +00:00
Ruben Fiszel
f3e9a29c13 add customUi props for History and Save to workspace editor bar buttons (#8336)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 12:28:17 +00:00
centdix
7fb729cc84 fix(cli): instruct agent to tell user about generate-metadata and sync push instead of running them (#8318)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-12 10:18:03 +00:00
wendrul
ca8a6274bc fix: use !inline ref for scripts inside flows (preproc, error, ai tool) (#8319)
* fix: use !inline ref for scripts inside flows (preproc, error, ai tool)

* add test

* path assign better
2026-03-12 10:15:00 +00:00
Ruben Fiszel
bf4340f40c fix: set min_connections(0) to prevent sqlx pool spin loop (#8334)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-11 23:46:24 +00:00
Clumsy
cbc7e78f8a fix: show diff editor content for resources without a language (#8331)
Resources like ansible_inventory have content but no language field,
causing the DiffEditor setupModel guard to skip initialization entirely.
Fall back to 'plaintext' when no language is provided but content exists.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 22:42:15 +00:00
hugocasa
d8b4132b9a fix: native mode now properly sets DB pool size and sleep queue (#8332)
Native mode (from DB config) was not accounted for when sizing the
connection pool or setting SLEEP_QUEUE, because both read NUM_WORKERS
from env which is never set when native mode is configured via the
worker group config in the database.

- Resolve native mode early (before connect_db) by querying the config
  table with the initial DB connection
- Pass num_workers directly to connect_db instead of re-reading env
- Replace SLEEP_QUEUE lazy_static with sleep_queue() function that
  checks NATIVE_MODE_RESOLVED at runtime (returns 300ms for native)
- Set NATIVE_MODE_RESOLVED immediately when is_native_mode_from_env()
- Allow native_mode in CE worker group config (was silently stripped)

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 22:39:15 +00:00
Ruben Fiszel
4306c9e4fe fix: skip python preinstall on native workers (#8329)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-11 18:22:07 +00:00
hugocasa
fe1519f128 feat: support minimal telemetry mode (#8243)
* feat: support minimal telemetry mode for EE

When EE customers disable telemetry, send a reduced payload with only
license-compliance data instead of ignoring the setting. Job usage data
is excluded in minimal mode. The telemetry settings UI now shows in EE
with context-appropriate descriptions for both CE and EE.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref for telemetry-minimal

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: make telemetry toggle label and description license-aware

Show "Minimal telemetry" with EE-specific description on EE, and
"Disable telemetry" with CE-specific description on CE.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Update commit hash in ee-repo-ref.txt

* Update reference hash in ee-repo-ref.txt

* chore: update ee-repo-ref to 2f52c015bc6c81391234fa87b27ee1d4cd3a48a3

This commit updates the EE repository reference after PR #440 was merged in windmill-ee-private.

Previous ee-repo-ref: 3628ed51426d8d29b3d5c62864ba256b7f9eab17

New ee-repo-ref: 2f52c015bc6c81391234fa87b27ee1d4cd3a48a3

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-11 19:12:47 +01:00
Ruben Fiszel
df1b1f9651 chore: fix 19 CVEs by upgrading php, docker, and node-tar (#8326)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 17:58:31 +00:00
Diego Imbert
ae019237d1 runs page nits (#8325) 2026-03-11 17:52:26 +00:00
Diego Imbert
577484d06a Separate asset parsers (#8321)
* Refactor asset_parser

* package update

* package lock
2026-03-11 16:54:05 +00:00
centdix
e7047761cf chore: webmux config (#8323) 2026-03-11 16:08:09 +00:00
hugocasa
8667329110 fix: skip token expiry notifications for debugger and mcp-oauth tokens (#8316)
* fix: skip token expiry notifications for debugger and mcp-oauth tokens

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: update frontend isUserToken to match backend filter

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: add cross-reference comments to token filter functions

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-11 10:47:21 +00:00
Ruben Fiszel
2aef01d18c feat: partition audit log table by day with configurable retention (#8292)
* feat: partition audit log table by day with configurable retention

Introduce daily range partitioning for audit logs to replace expensive
DELETE-based retention with instant DROP TABLE per partition.

- Create `audit_partitioned` table alongside existing `audit` table
- New inserts go to `audit_partitioned`, reads UNION ALL both tables
- Monitor creates future partitions and drops expired ones
- Add `audit_log_retention_days` instance setting (default 365 days)
- Old `audit` table empties naturally via existing DELETE cleanup

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add audit log retention setting to Core instance settings UI

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: bump audit partitioning migration timestamp to avoid collision

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref.txt for audit partitioning

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add RLS/grants to audit_partitioned, run partition mgmt hourly, CE default 14d

- Add grants for windmill_user/windmill_admin and all 5 RLS policies
- Move manage_audit_partitions to hourly via should_run(120)
- Default retention: 14 days CE, 365 days EE
- Download JSON button is now icon-only

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address code review — quote SQL identifiers, add workspace index, deduplicate retention logic

- Quote partition names in dynamic SQL for defense in depth
- Add idx_audit_partitioned_workspace(workspace_id, timestamp DESC) index
- Extract audit_log_retention_days() helper to deduplicate retention logic

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref for audit insert error handling

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref to cef4dfc45e6d6344c5d8d107bd2b4d1bf9bbdd64

This commit updates the EE repository reference after PR #450 was merged in windmill-ee-private.

Previous ee-repo-ref: f09284bb257d461bcbe3c50fe31eb6f1e7eafee5

New ee-repo-ref: cef4dfc45e6d6344c5d8d107bd2b4d1bf9bbdd64

Automated by sync-ee-ref workflow.

* fix: create audit partitions on startup in initial_load

Ensures partitions exist before any requests arrive, closing the gap
between server start and the first hourly monitor run.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-11 00:10:29 +00:00
Ruben Fiszel
48bc3e2445 fix: prevent zombie jobs from looping forever (#8313)
* fix: always increment zombie job counter to prevent infinite loop at restart limit

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: force-complete zombie jobs when handle_job_error fails

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: revert counter increment change, keep re-detection for robustness

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: disable schedule when permissioned_as user is not found

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 23:46:24 +00:00
Diego Imbert
425a75e030 fix: fix saved inputs popover infinite loop (#8311) 2026-03-10 21:12:46 +00:00
Ruben Fiszel
62c3294c35 chore(main): release 1.654.0 (#8291)
* chore(main): release 1.654.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-03-10 18:39:15 +00:00
hugocasa
dc0e59f432 feat: add preprocessor support for dedicated workers and bunnative scripts (#8284)
* feat: add preprocessor support for dedicated workers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref.txt

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: extract transform_and_run helper in python dedicated wrapper

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add preprocessor support for bunnative scripts

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: resolve unused postprocessor variable in python wrapper

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: add workflow_dispatch trigger to backend integration tests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: bunnative fixture lock format and PrewarmedIsolate::spawn callers

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: update generate_dedicated_worker_wrapper callers in bun_jobs test

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use non-dedicated workers in preprocessor integration tests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: prewarm preprocessor isolate for bunnative dedicated workers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: flatten bunnative dedicated worker preprocessing into single result path

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: use labeled block instead of async block for EE compatibility

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Update commit reference in ee-repo-ref.txt

* chore: update ee-repo-ref to e36945b987f7904fa984181baf3124e7b2722bd1

This commit updates the EE repository reference after PR #445 was merged in windmill-ee-private.

Previous ee-repo-ref: 8a2625833452aadb8907242bf502b24ca2dffd73

New ee-repo-ref: e36945b987f7904fa984181baf3124e7b2722bd1

Automated by sync-ee-ref workflow.

* Fix merge conflict in ee-repo-ref.txt

Resolve merge conflict in ee-repo-ref.txt

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-10 18:26:02 +00:00
Ruben Fiszel
fefc8c62a0 fix: teams selection not sticking in workspace settings (#8309)
Fix portal class mismatch in clickOutside that caused premature dropdown
closing, and simplify TeamSelector/ChannelSelector state sync to use
getter/setter bindings instead of bidirectional $effect chains.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 18:25:09 +00:00
centdix
cb349cb3d1 feat: add Vertex AI support for Google Gemini models (#8303)
* refactor: rename AnthropicPlatform to AIPlatform for generic vertex support

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add Vertex AI support for Google Gemini models

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update platform doc comments to be provider-generic

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update googleai default models to latest gemini 2.5 and 3

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: move AIPlatform to windmill_common to avoid duplication

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: hugocasa <hugo@casademont.ch>
2026-03-10 18:20:19 +00:00
Ruben Fiszel
dbfa271b89 fix: preserve teams oauth tenant on settings page reload (#8308)
* fix: preserve teams oauth tenant on settings page reload

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: don't set unused tenant field on microsoft oauth init

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 17:58:37 +00:00
Ruben Fiszel
83be59e0e8 fix: debounce webhook arg accumulation with max_count/max_time limits (#8307)
* fix: correct debounce max_total_debounces_amount semantics and complete previous job on limit exceeded

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: enable debounce arg accumulation for post-preprocessing flows

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add debounce accumulation tests for max_count and max_time limits

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add push-time max_count and max_time accumulation tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* sqlx

* sqlx

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 17:22:39 +00:00
wendrul
f291b1cc19 fix(cli): fail when passing an invalid --workspace arg (#8294) 2026-03-10 13:47:30 +00:00
wendrul
5baeb8c842 fix: explicilty fail when --base-url --token --workspace are invalid (#8302) 2026-03-10 13:44:45 +00:00
centdix
b40cf80fdd fix: optimize flow lock generation and add rt.d.ts guidance for TS resource types (#8295)
Instruct AI to pass specific flow folder path to `wmill flow generate-locks`
instead of running it on all flows. Also add guidance for TypeScript language
files to check `rt.d.ts` for available resource types before using them.
Re-ran generate.py to propagate changes to all auto-generated files.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 13:03:21 +00:00
Pyra
cbac81e3a1 fix ci test (#8301)
* feat: add git sync support for workspace dependencies

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: implement git sync for workspace dependencies

Signed-off-by: pyranota <pyra@duck.com>

* remove deno.lock

Signed-off-by: pyranota <pyra@duck.com>

* update ee

Signed-off-by: pyranota <pyra@duck.com>

* add tests to cli

Signed-off-by: pyranota <pyra@duck.com>

* sqlx

* chore: update ee-repo-ref to 09dfb247f6f59c61b7f2431932c4557fb26c22d8

This commit updates the EE repository reference after PR #446 was merged in windmill-ee-private.

Previous ee-repo-ref: 8a8832ae5d7efab85b3a57a740308ececa0e2aac

New ee-repo-ref: 09dfb247f6f59c61b7f2431932c4557fb26c22d8

Automated by sync-ee-ref workflow.

* fix test

* fix ci test

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-10 13:02:12 +00:00
Ruben Fiszel
438f609a78 fix: delete debounce_key on post-preprocessing limit exceeded (#8299)
* fix: delete debounce_key entry when post-preprocessing limits exceeded

For preprocessor flows, the runnable_settings_handle has
debounce_delay_s = None, so maybe_apply_debouncing at pull-time
won't clean up stale debounce_key entries. Previously we only
reset the entry (UPDATE), but since the flow executes immediately
without rescheduling, a stale entry would cause the next incoming
flow to incorrectly try to debounce against an already-executing job.

Change from UPDATE (reset) to DELETE so the entry is fully removed.
Update tests to expect deletion instead of reset.

Companion EE PR: https://github.com/windmill-labs/windmill-ee-private/pull/448

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: persist accumulated debounce args to v2_job for flows

The in-memory arg accumulation in maybe_apply_debouncing was not
persisted back to v2_job. For scripts this is fine (single execution),
but for flows, subsequent steps re-read args from the DB via
get_mini_pulled_job and would see the original (non-accumulated) value.

Also improve the job log message to show both original and accumulated
argument values for clarity.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref to d1c14355026151ecdd31adda8e2c60ecd1b5ad65

This commit updates the EE repository reference after PR #448 was merged in windmill-ee-private.

Previous ee-repo-ref: bff784002a3335af7c10982599c8f03e536d5abf

New ee-repo-ref: d1c14355026151ecdd31adda8e2c60ecd1b5ad65

Automated by sync-ee-ref workflow.

* test: assert accumulated debounce args are persisted to v2_job

Add DB persistence assertions to accumulation tests to prevent
regressions on the fix that writes accumulated args back to v2_job.
Without this, flow steps re-reading args from the DB would see
the original (non-accumulated) value.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* sqlx

* chore: update ee-repo-ref.txt to ee-private main

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-10 13:01:58 +00:00
hugocasa
b02f9e5c24 feat: add kafka trigger offset reset and auto.offset.reset config (#8283)
* fix: parallel branchall hang on bad stop_after_all_iters_if + results.x.length null

Two fixes:

1. When a parallel branchall/forloop has a `stop_after_all_iters_if` expression
   that fails (e.g. bad JS syntax), the error was propagated with `?`, causing
   the transaction to roll back the parallel index increment. Since all parallel
   jobs were already completed, nothing could ever increment the index again and
   the flow hung forever. Now the error is caught and converted to a stop-early
   failure so the transaction commits and the flow fails gracefully.

2. Expressions like `results.a.length` in step input transforms resolved to null
   because the `handle_full_regex` fast path intercepted them and used
   PostgreSQL's `#>` JSON path operator, which can't resolve JS runtime
   properties like `.length` on arrays. Now the fast path skips expressions
   ending with JS-only properties (like `length`), falling through to full
   QuickJS evaluation where they work correctly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add kafka trigger offset reset and auto.offset.reset configuration

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref for kafka offset reset

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref for subscribe+seek approach

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref for kafka offset reset fix

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: use ConfirmationModal instead of browser confirm() for kafka offset reset

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref for offset commit fix

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* sqlx update

* Update ee-repo-ref.txt

* update ee ref

* update sqlx

* update ee ref

* chore: update ee-repo-ref to a70d7db187aa78a7fbfd3bfaf92372160cff320a

This commit updates the EE repository reference after PR #444 was merged in windmill-ee-private.

Previous ee-repo-ref: 238c2c0a91f353126f349a5153173a6d16c9d652

New ee-repo-ref: a70d7db187aa78a7fbfd3bfaf92372160cff320a

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-10 12:58:00 +00:00
Diego Imbert
cda843922d fix: show meaningful error messages in database manager schema fetch (#8296)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 12:38:59 +00:00
Ruben Fiszel
b841e0a038 fix: handle missing schema in RunnableByPath during wmill.d.ts generation (#8300) 2026-03-10 12:38:43 +00:00
Ruben Fiszel
4f29e05e3a feat: add git sync support for workspace dependencies (#8144)
* feat: add git sync support for workspace dependencies

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: implement git sync for workspace dependencies

Signed-off-by: pyranota <pyra@duck.com>

* remove deno.lock

Signed-off-by: pyranota <pyra@duck.com>

* update ee

Signed-off-by: pyranota <pyra@duck.com>

* add tests to cli

Signed-off-by: pyranota <pyra@duck.com>

* sqlx

* chore: update ee-repo-ref to 09dfb247f6f59c61b7f2431932c4557fb26c22d8

This commit updates the EE repository reference after PR #446 was merged in windmill-ee-private.

Previous ee-repo-ref: 8a8832ae5d7efab85b3a57a740308ececa0e2aac

New ee-repo-ref: 09dfb247f6f59c61b7f2431932c4557fb26c22d8

Automated by sync-ee-ref workflow.

* fix test

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Pyra <92104930+pyranota@users.noreply.github.com>
Co-authored-by: pyranota <pyra@duck.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-10 11:29:11 +00:00
Diego Imbert
713ba009c4 nit only resync custom instance password on server (#8298) 2026-03-10 11:23:39 +00:00
Diego Imbert
53ac43f5ee fix: resync custom_instance_user password on startup (#8297)
On backend startup, verify the custom_instance_user can connect to the
database with the stored password. If the connection fails, automatically
refresh the password by calling refresh_custom_instance_user_pwd_inner().

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 11:18:15 +00:00
Guilhem
ac8c668cb9 fix: skip loading flow preview history for new flows (#8293)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 10:13:26 +00:00
Guilhem
cad44365ac feat(frontend): replace flat sugiyama with recursive compound layout for flow graph (#8204)
* feat(frontend): replace flat sugiyama with recursive compound layout for flow graph

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(frontend): double forloop wrapper padding and include wrappers in bbox

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(frontend): gate debug wrappers behind SHOW_DEBUG_WRAPPERS flag

Remove all debug console.log calls from compoundLayout and gate
WrapperInfo creation and wrapper node rendering behind an exported
SHOW_DEBUG_WRAPPERS constant. Replace wrapper-based bbox computation
with groupLayouts-based loop so no WrapperInfo is needed for correct
layout. Add contentMinX to LayoutResult for the top-level minX shift.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(frontend): remove debug wrapper nodes from flow graph

Remove WrapperInfo type, SHOW_DEBUG_WRAPPERS flag, buildDebugWrapperNodes
helper, DebugWrapperNode component, and all related plumbing in
FlowGraphV2. The bbox computation now uses groupLayouts directly,
keeping layout correctness without any debug wrapper overhead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* perf(frontend): optimize compoundLayout recursive algorithm

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(frontend): remove dead offset plumbing from flow graph

The old flat sugiyama layout used a CSS margin-left hack (offset) to
indent loop bodies. The new recursive compound layout handles indentation
natively via coordinates, making the entire offset pipeline dead code.

Removes offset from 11 node type definitions, NodeLayout, addNode helper,
processModules parameter, NodeWrapper prop, 9 node renderers, AssetNode
x-position calculations, AIToolNode x-position calculations, DragGhost
nodeOffset function, FlowGraphV2 layout pipeline, util.ts type signatures,
noteUtils NodeDep type, and noteEditor function signature.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(frontend): remove unused lastXCenter variable

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* perf(frontend): optimize compoundLayout hot paths

Replace O(N²) queue.shift() with index pointer in BFS, eliminate
redundant groupOwnedIds double-build, use Set for parent dedup,
track minY in existing bbox loop, and cache maxBranchHeight.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: remove debug artifacts from PR

Remove elk_viewer test page, console log dumps, and layout screenshots
that were used during development.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(frontend): guard data.module.value access in ModuleNode

When rapidly clicking expand/collapse on a subflow, the graph rebuilds
and data.module can be transiently undefined. Add optional chaining to
prevent "Cannot read properties of undefined (reading 'value')" errors.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(frontend): simplify CompoundGroup type to 'branch' | 'loop'

The layout never distinguishes branchall/branchone or forloop/whileloop,
so collapse to two variants that match the actual code paths.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(frontend): address PR review feedback on flow layout

- Add max recursion depth guard (50) to layoutLevel to prevent stack
  overflow with malformed flow data
- Log swallowed decrossOpt error as console.debug for debuggability
- Initialize maxY to -Infinity for correctness with negative positions
- Fix indentation artifacts in graphBuilder data objects

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* formatting

* fix: remove offset field from asset node data in FlowGraphV2

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 09:06:02 +00:00
Ruben Fiszel
f89da1c5ef chore(main): release 1.653.0 (#8288)
* chore(main): release 1.653.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-03-10 05:39:29 +00:00
Ruben Fiszel
0c4d72cfe3 feat: add indexer time window setting (default 7 days) (#8290)
* feat: add indexer time window setting (default 7 days)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add time window note to search UIs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: fetch indexer time window from API in search UIs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref to 9df755c57fbfc88f4a724e1ea51b1d5f5af4fe52

This commit updates the EE repository reference after PR #447 was merged in windmill-ee-private.

Previous ee-repo-ref: c17f16bf45091272974e3aa8009cdf5cc15669bf

New ee-repo-ref: 9df755c57fbfc88f4a724e1ea51b1d5f5af4fe52

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-10 05:22:12 +00:00
Ruben Fiszel
2d8335dc43 perf: optimize job_stats storage for timestamps and zero-memory jobs (#8289)
* perf: optimize job_stats storage for timestamps and zero-memory jobs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update sqlx offline cache nullable metadata

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: use centisecond offsets for job_stats timestamps (~248 day range)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: update SELECT to use offsets_cs column name

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 04:59:56 +00:00
Alexander Petric
39e77ecd00 feat: add slack connection fields to workspace settings export/import (#8287)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 21:39:03 +00:00
324 changed files with 19238 additions and 6393 deletions

View File

@@ -13,8 +13,10 @@ fi
# Check if the file is in the backend directory and is a Rust file
if [[ "$FILE_PATH" == *"/backend/"* ]] && [[ "$FILE_PATH" =~ \.rs$ ]]; then
cd "$CLAUDE_PROJECT_DIR/backend" || exit 0
# Run rustfmt with config from rustfmt.toml (edition=2021)
rustfmt --config-path rustfmt.toml "$FILE_PATH" 2>/dev/null || true
# Run rustfmt, surface errors as context but don't block Claude
if rustfmt --config-path rustfmt.toml "$FILE_PATH" 2>&1; then
echo "Formatted $(basename "$FILE_PATH")"
fi
fi
exit 0

View File

@@ -15,8 +15,10 @@ if [[ "$FILE_PATH" == *"/frontend/"* ]]; then
# Check if it's a formattable file type
if [[ "$FILE_PATH" =~ \.(ts|js|svelte|json|css|html|md)$ ]]; then
cd "$CLAUDE_PROJECT_DIR/frontend" || exit 0
# Run prettier silently, don't fail the hook if prettier fails
npx prettier --write "$FILE_PATH" 2>/dev/null || true
# Run prettier, surface errors as context but don't block Claude
if ./node_modules/.bin/prettier --plugin prettier-plugin-svelte --write "$FILE_PATH" 2>&1; then
echo "Formatted $(basename "$FILE_PATH")"
fi
fi
fi

View File

@@ -28,6 +28,12 @@
"Bash(git show:*)",
"Bash(git blame:*)",
"Bash(cargo check:*)",
"Bash(cargo build --release:*)",
"Bash(sh wm-ts-nav/nav:*)",
"Bash(wm-ts-nav/nav:*)",
"Bash(./wm-ts-nav/nav:*)",
"Bash(wm-ts-nav/target/release/wm-ts-nav:*)",
"Bash(./wm-ts-nav/target/release/wm-ts-nav:*)",
"mcp__ide__getDiagnostics",
"Bash(npm run generate-backend-client:*)",
"Bash(npm run check:*)",

View File

@@ -0,0 +1,98 @@
---
name: local-review
user_invocable: true
description: Code review a pull request for bugs and CLAUDE.md compliance. MUST use when asked to review code.
---
# Local Code Review Skill
Review a pull request for real bugs and CLAUDE.md compliance violations. This review targets HIGH SIGNAL issues only.
## Review Philosophy
- **Only flag issues you are certain about.** If you are not sure an issue is real, do not flag it. False positives erode trust and waste reviewer time.
- Think like a senior engineer doing a final review — flag things that would cause incidents, not things that are merely imperfect.
## What to Flag
- Code that won't compile or parse (syntax errors, type errors, missing imports)
- Code that will definitely produce wrong results regardless of inputs
- Clear, unambiguous CLAUDE.md violations (quote the exact rule being violated)
- Security issues in introduced code (injection, auth bypass, data exposure)
- Incorrect logic that will fail in production
## What NOT to Flag
- Code style or quality concerns
- Potential issues that depend on specific inputs or runtime state
- Subjective suggestions or improvements
- Pre-existing issues not introduced by this PR
- Pedantic nitpicks a senior engineer wouldn't flag
- Issues a linter or type checker will catch
- General quality concerns unless explicitly prohibited in CLAUDE.md
- Issues silenced via lint ignore comments
## Execution Steps
1. **Determine the PR scope**:
- If an argument is provided, use it as the PR number or branch
- Otherwise, detect from the current branch vs main
- Run `gh pr view` if a PR exists, or use `git diff main...HEAD`
2. **Find relevant CLAUDE.md files**:
- Read the root `CLAUDE.md`
- Check for CLAUDE.md files in directories containing changed files
3. **Get the diff and metadata**:
- `gh pr diff` or `git diff main...HEAD` for the full diff
- `gh pr view` or `git log main..HEAD --oneline` for context
4. **Read changed files** where the diff alone is insufficient to understand context
5. **Review for**:
- CLAUDE.md compliance — check each rule against the changed code
- Bugs and logic errors — will this code work correctly?
- Security issues — injection, auth, data exposure in new code
6. **Self-validate each finding**: Before reporting, ask yourself:
- "Is this definitely a real issue, not a false positive?"
- "Would a senior engineer flag this in review?"
- If the answer to either is no, discard the finding
7. **Output findings** to the terminal (default) or post as PR comments (with `--comment` flag)
## Output Format
```
## Code review
Found N issues:
1. <description> (<reason: CLAUDE.md adherence | bug | security>)
<file_path:line_number>
2. <description> (<reason>)
<file_path:line_number>
```
If no issues are found:
```
## Code review
No issues found. Checked for bugs and CLAUDE.md compliance.
```
## Posting Comments (--comment flag)
If the user passes `--comment`, post findings as inline PR comments using:
```bash
gh pr review --comment --body "<summary>"
```
Or for inline comments on specific lines:
```bash
gh api repos/{owner}/{repo}/pulls/{pr}/reviews -f body="<summary>" -f event="COMMENT" -f comments="[...]"
```

View File

@@ -33,6 +33,7 @@ Follow conventional commit format for the PR title:
- Keep under 70 characters
- Use lowercase, imperative mood
- No period at the end
- If `*_ee.rs` files were modified, prefix with `[ee]`: `[ee] <type>: <description>`
## PR Body Format
@@ -85,3 +86,25 @@ Generated with [Claude Code](https://claude.com/claude-code)
)"
```
7. Return the PR URL to the user
## EE Companion PR (when `*_ee.rs` files were modified)
The `*_ee.rs` files in the windmill repo are **symlinks** to `windmill-ee-private` — changes won't appear in `git diff` of the windmill repo. Instead, check the EE repo for uncommitted or unpushed changes.
Follow the full EE PR workflow in `docs/enterprise.md`. The key PR-specific details:
1. Find the EE repo/worktree: see "Finding the EE Repo" in `docs/enterprise.md`
2. Check for changes: `git -C <ee-path> status --short`
- If there are no changes in the EE repo, skip this entire section
3. Follow steps 15 from the "EE PR Workflow" in `docs/enterprise.md`
4. Create the companion PR (title does NOT get the `[ee]` prefix):
```bash
gh pr create --draft --repo windmill-labs/windmill-ee-private --title "<type>: <description>" --body "$(cat <<'EOF'
Companion PR for windmill-labs/windmill#<PR_NUMBER>
---
Generated with [Claude Code](https://claude.com/claude-code)
EOF
)"
```
5. Commit `ee-repo-ref.txt` and push the updated windmill branch

View File

@@ -119,6 +119,18 @@ jobs:
with:
cache-workspaces: backend
toolchain: 1.93.0
- name: Fix stale v8 build cache
working-directory: ./backend
run: |
# Cargo cache may preserve v8 build fingerprints without the actual
# librusty_v8.a library. Since fingerprints look valid, cargo skips
# build.rs re-run, causing "could not find native static library rusty_v8".
for profile in debug release; do
if [ -d "target/$profile/.fingerprint" ] && [ ! -f "target/$profile/gn_out/obj/librusty_v8.a" ]; then
echo "Cleaning stale v8 build artifacts in target/$profile"
rm -rf "target/$profile/build/v8-"* "target/$profile/.fingerprint/v8-"*
fi
done
- name: cargo check
timeout-minutes: 16
working-directory: ./backend

View File

@@ -66,7 +66,7 @@ jobs:
with:
bun-version: 1.3.10
- uses: actions/setup-node@v4
- uses: actions/setup-node@v6
with:
node-version: "20"

View File

@@ -1,6 +1,7 @@
name: Backend only integration tests
on:
workflow_dispatch:
push:
branches:
- "main"
@@ -56,7 +57,7 @@ jobs:
- uses: oven-sh/setup-bun@v2
with:
bun-version: 1.3.10
- uses: actions/setup-node@v4
- uses: actions/setup-node@v6
with:
node-version: "20"
- uses: astral-sh/setup-uv@v6.2.1
@@ -88,6 +89,18 @@ jobs:
with:
cache-workspaces: backend
toolchain: 1.93.0
- name: Fix stale v8 build cache
working-directory: ./backend
run: |
# Cargo cache may preserve v8 build fingerprints without the actual
# librusty_v8.a library. Since fingerprints look valid, cargo skips
# build.rs re-run, causing "could not find native static library rusty_v8".
for profile in debug release; do
if [ -d "target/$profile/.fingerprint" ] && [ ! -f "target/$profile/gn_out/obj/librusty_v8.a" ]; then
echo "Cleaning stale v8 build artifacts in target/$profile"
rm -rf "target/$profile/build/v8-"* "target/$profile/.fingerprint/v8-"*
fi
done
- name: Read EE repo commit hash
run: |
echo "ee_repo_ref=$(cat ./ee-repo-ref.txt)" >> "$GITHUB_ENV"

View File

@@ -0,0 +1,37 @@
name: Check system prompts freshness
on:
push:
paths:
- "system_prompts/**"
- "typescript-client/**"
- "python-client/wmill/wmill/client.py"
- "openflow.openapi.yaml"
- "backend/windmill-api/openapi.yaml"
- "cli/src/main.ts"
- "cli/src/commands/**"
pull_request:
paths:
- "system_prompts/**"
- "typescript-client/**"
- "python-client/wmill/wmill/client.py"
- "openflow.openapi.yaml"
- "backend/windmill-api/openapi.yaml"
- "cli/src/main.ts"
- "cli/src/commands/**"
jobs:
check-freshness:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install dependencies
run: pip install pyyaml
- name: Check auto-generated files are up-to-date
run: bash system_prompts/check-freshness.sh

View File

@@ -24,7 +24,7 @@ jobs:
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
uses: actions/setup-node@v6
with:
node-version: "20"
@@ -70,7 +70,7 @@ jobs:
cache-workspaces: backend
- name: Setup Node.js
uses: actions/setup-node@v4
uses: actions/setup-node@v6
with:
node-version: "20"
@@ -124,7 +124,7 @@ jobs:
cache-workspaces: backend
- name: Setup Node.js
uses: actions/setup-node@v4
uses: actions/setup-node@v6
with:
node-version: "20"

View File

@@ -16,7 +16,7 @@ jobs:
runs-on: ubicloud-standard-8
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v5
- uses: actions/setup-node@v6
with:
node-version: 24
cache: "npm"

209
.github/workflows/git-sync-test.yml vendored Normal file
View File

@@ -0,0 +1,209 @@
name: Git Sync Integration Tests
on:
workflow_dispatch:
push:
branches: [main]
paths:
- "backend/windmill-git-sync/**"
- "backend/windmill-api-integration-tests/tests/git_sync*"
- "backend/ee-repo-ref.txt"
- "integration_tests/test/git_sync_test.py"
- ".github/workflows/git-sync-test.yml"
pull_request:
types: [opened, synchronize, reopened]
paths:
- "backend/windmill-git-sync/**"
- "backend/windmill-api-integration-tests/tests/git_sync*"
- "backend/ee-repo-ref.txt"
- "integration_tests/test/git_sync_test.py"
- ".github/workflows/git-sync-test.yml"
concurrency:
group: git-sync-test-${{ github.ref }}
cancel-in-progress: true
jobs:
check-relevance:
runs-on: ubuntu-latest
outputs:
should_run: ${{ steps.check.outputs.should_run }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Check if git sync related files changed
id: check
env:
WINDMILL_EE_PRIVATE_ACCESS: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
run: |
if [ "${{ github.event_name }}" = "pull_request" ]; then
BASE=${{ github.event.pull_request.base.sha }}
else
BASE=${{ github.event.before }}
fi
CHANGED_FILES=$(git diff --name-only "$BASE"..HEAD 2>/dev/null || echo "")
echo "Changed files:"
echo "$CHANGED_FILES"
# Direct git sync file changes — always relevant
if echo "$CHANGED_FILES" | grep -qE '^(backend/windmill-git-sync/|backend/windmill-api-integration-tests/tests/git_sync|integration_tests/test/git_sync|\.github/workflows/git-sync-test\.yml)'; then
echo "should_run=true" >> "$GITHUB_OUTPUT"
echo "Relevant: direct git sync file changes"
exit 0
fi
# If ee-repo-ref.txt changed, check if the EE diff touches windmill-git-sync/
if echo "$CHANGED_FILES" | grep -q '^backend/ee-repo-ref.txt$'; then
NEW_REF=$(cat backend/ee-repo-ref.txt)
OLD_REF=$(git show "$BASE:backend/ee-repo-ref.txt" 2>/dev/null || echo "")
if [ -n "$OLD_REF" ] && [ "$OLD_REF" != "$NEW_REF" ]; then
# Clone EE repo and check diff
git clone --bare "https://x-access-token:${WINDMILL_EE_PRIVATE_ACCESS}@github.com/windmill-labs/windmill-ee-private.git" /tmp/ee-repo 2>/dev/null
EE_CHANGED=$(git -C /tmp/ee-repo diff --name-only "$OLD_REF".."$NEW_REF" 2>/dev/null || echo "")
echo "EE changed files:"
echo "$EE_CHANGED"
if echo "$EE_CHANGED" | grep -q '^windmill-git-sync/'; then
echo "should_run=true" >> "$GITHUB_OUTPUT"
echo "Relevant: EE git sync files changed"
exit 0
fi
fi
fi
echo "should_run=false" >> "$GITHUB_OUTPUT"
echo "No git sync relevant changes detected, skipping tests"
git_sync_e2e:
needs: [check-relevance]
if: needs.check-relevance.outputs.should_run == 'true'
runs-on: ubicloud-standard-16
services:
postgres:
image: postgres:14
ports:
- 5432:5432
env:
POSTGRES_DB: windmill
POSTGRES_PASSWORD: changeme
options: >-
--health-cmd pg_isready --health-interval 10s --health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.ref }}
fetch-depth: 0
- name: Read EE repo commit hash
run: |
echo "ee_repo_ref=$(cat ./backend/ee-repo-ref.txt)" >> "$GITHUB_ENV"
- uses: actions/checkout@v4
with:
repository: windmill-labs/windmill-ee-private
path: ./windmill-ee-private
ref: ${{ env.ee_repo_ref }}
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
fetch-depth: 0
- name: Substitute EE code
run: |
cd backend && ./substitute_ee_code.sh --copy --dir ./windmill-ee-private
- uses: actions-rust-lang/setup-rust-toolchain@v1
with:
cache-workspaces: backend
toolchain: 1.93.0
- uses: oven-sh/setup-bun@v2
with:
bun-version: 1.3.10
- uses: denoland/setup-deno@v2
with:
deno-version: v2.x
- uses: actions/setup-node@v6
with:
node-version: "20"
- name: Install wmill CLI
run: |
cd cli && bash gen_wm_client.sh && bun install
mkdir -p "$HOME/.local/bin"
printf '#!/bin/sh\nexec bun run "%s/cli/src/main.ts" "$@"\n' "$GITHUB_WORKSPACE" > "$HOME/.local/bin/wmill"
chmod +x "$HOME/.local/bin/wmill"
echo "$HOME/.local/bin" >> $GITHUB_PATH
- name: Build Windmill
working-directory: ./backend
env:
SQLX_OFFLINE: true
CARGO_BUILD_JOBS: 12
RUSTFLAGS: ""
run: |
cargo build --features enterprise,private,license,zip
- name: Start Gitea
run: |
docker run -d --name gitea \
-e GITEA__database__DB_TYPE=sqlite3 \
-e GITEA__security__INSTALL_LOCK=true \
-e GITEA__server__HTTP_PORT=3000 \
-e GITEA__server__ROOT_URL=http://localhost:3000 \
-e GITEA__service__DISABLE_REGISTRATION=false \
-p 3000:3000 \
gitea/gitea:1.22-rootless
echo "Waiting for Gitea to be ready..."
for i in $(seq 1 30); do
if curl -sf http://localhost:3000/api/v1/version > /dev/null 2>&1; then
echo "Gitea is ready"
break
fi
sleep 2
done
curl -sf http://localhost:3000/api/v1/version > /dev/null || { echo "Gitea failed to start"; exit 1; }
- name: Start Windmill
working-directory: ./backend
env:
DATABASE_URL: postgres://postgres:changeme@localhost:5432/windmill
LICENSE_KEY: ${{ secrets.WM_LICENSE_KEY_CI }}
DENO_PATH: deno
BUN_PATH: bun
NODE_BIN_PATH: node
run: |
./target/debug/windmill &
echo "Waiting for Windmill to be ready..."
for i in $(seq 1 60); do
if curl -sf http://localhost:8000/api/version > /dev/null 2>&1; then
echo "Windmill is ready"
break
fi
sleep 2
done
curl -sf http://localhost:8000/api/version > /dev/null || { echo "Windmill failed to start"; exit 1; }
- name: Run git sync E2E tests
timeout-minutes: 10
env:
GITEA_DOCKER_URL: http://localhost:3000
LICENSE_KEY: ${{ secrets.WM_LICENSE_KEY_CI }}
run: |
python3 -m venv .venv
.venv/bin/pip install -r integration_tests/requirements.txt
cd integration_tests && ../.venv/bin/python -m unittest -v test.git_sync_test
- name: Archive logs
uses: actions/upload-artifact@v4
if: always()
with:
name: Git Sync Integration Tests Logs
path: |
integration_tests/logs

View File

@@ -10,24 +10,24 @@ jobs:
runs-on: ubicloud-standard-8
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v3
- uses: actions/setup-node@v6
with:
node-version: "20.x"
registry-url: "https://registry.npmjs.org"
- run: cd typescript-client && ./publish.sh && cd ..
- run: cd typescript-client && ./publish.sh --access public && cd ..
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
publish_cli:
runs-on: ubicloud-standard-8
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v3
- uses: actions/setup-node@v6
with:
node-version: "20.x"
registry-url: "https://registry.npmjs.org"
- uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- run: cd cli && ./build.sh && cd npm && npm publish
- run: cd cli && ./build.sh && cd npm && npm publish --access public
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

View File

@@ -15,7 +15,7 @@ jobs:
uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@v4
uses: actions/setup-node@v6
with:
node-version: '20'

1
.gitignore vendored
View File

@@ -27,3 +27,4 @@ typescript-client/node_modules
frontend/.svelte-kit
backend/chrome_profiler.json
.fast-check/
__pycache__/

View File

@@ -16,7 +16,8 @@ lifecycleHooks:
preRemove: bash ./scripts/pre-remove.sh
auto_name:
model: gemini-2.5-flash-lite
provider: claude
model: haiku
# Each service defines a port env var that webmux injects into pane and agent
# process environments when creating a worktree. Ports are auto-assigned:
@@ -32,7 +33,7 @@ services:
portStep: 10
profiles:
default:
full:
runtime: host
yolo: true
envPassthrough: []
@@ -44,7 +45,9 @@ profiles:
- Pane 2: frontend (npm run dev)
To check logs, use: \`tmux capture-pane -t .1 -p -S -50\` (backend) or \`tmux capture-pane -t .2 -p -S -50\` (frontend).
When restarting backend or frontend, make sure to use ${BACKEND_PORT} and ${FRONTEND_PORT}.
To connect to the database, use this connection string: ${DATABASE_URL}
Because we are running backend with cargo watch, to verify your changes, just check the logs in the backend pane. No need for cargo check.
IMPORTANT: Read docs/autonomous-mode.md before starting any work.
panes:
- id: agent
kind: agent
@@ -52,14 +55,51 @@ profiles:
- id: backend
kind: command
split: right
command: ROOT="$(git rev-parse --show-toplevel)"; [ -f "$ROOT/.env.local" ] && source "$ROOT/.env.local"; cd "$ROOT/backend" && PORT=${BACKEND_PORT:-8000} cargo watch -x "run ${CARGO_FEATURES:+--features $CARGO_FEATURES}"
command: ROOT="$(git rev-parse --show-toplevel)"; cd "$ROOT/backend" && PORT=${BACKEND_PORT:-8000} cargo watch -x "run ${CARGO_FEATURES:+--features $CARGO_FEATURES}"
- id: frontend
kind: command
split: bottom
command: ROOT="$(git rev-parse --show-toplevel)"; [ -f "$ROOT/.env.local" ] && source "$ROOT/.env.local"; cd "$ROOT/frontend" && npm run generate-backend-client && REMOTE=${REMOTE:-http://localhost:${BACKEND_PORT:-8000}} npm run dev -- --port ${FRONTEND_PORT:-3000} --host 0.0.0.0
command: ROOT="$(git rev-parse --show-toplevel)"; cd "$ROOT/frontend" && npm run generate-backend-client && REMOTE=${REMOTE:-http://localhost:${BACKEND_PORT:-8000}} npm run dev -- --port ${FRONTEND_PORT:-3000} --host 0.0.0.0
frontendOnly:
runtime: host
yolo: true
envPassthrough: []
systemPrompt: >
You are running inside a tmux session with other panes running services.
Pane layout (current window):
- Pane 0: this pane (claude agent)
- Pane 1: frontend (npm run dev)
To check logs, use: \`tmux capture-pane -t .1 -p -S -50\` (frontend).
When restarting frontend, make sure to use ${FRONTEND_PORT}.
To connect to the database, use this connection string: ${DATABASE_URL}
Because we are running frontend with npm run dev, to verify your changes, just check the logs in the frontend pane. No need for npm run build.
IMPORTANT: Read docs/autonomous-mode.md before starting any work.
panes:
- id: agent
kind: agent
focus: true
- id: frontend
kind: command
split: right
command: ROOT="$(git rev-parse --show-toplevel)"; cd "$ROOT/frontend" && npm run generate-backend-client && npm run dev -- --port ${FRONTEND_PORT:-3000} --host 0.0.0.0
agentOnly:
runtime: host
yolo: true
envPassthrough: []
systemPrompt: >
IMPORTANT: Read docs/autonomous-mode.md before starting any work.
panes:
- id: agent
kind: agent
focus: true
integrations:
github:
linkedRepos: []
linkedRepos:
- repo: windmill-labs/windmill-ee-private
alias: ee-private
dir: ../windmill-ee-private__worktrees
linear:
enabled: true

View File

@@ -67,6 +67,7 @@ files:
copy:
- backend/.env
- scripts/
- wm-ts-nav/target/release/wm-ts-nav
sandbox:
enabled: false

View File

@@ -1,5 +1,66 @@
# Changelog
## [1.655.0](https://github.com/windmill-labs/windmill/compare/v1.654.0...v1.655.0) (2026-03-12)
### Features
* add auto_commit option to Kafka triggers with advanced UI badges ([#8317](https://github.com/windmill-labs/windmill/issues/8317)) ([ec20d76](https://github.com/windmill-labs/windmill/commit/ec20d76216492086842c4f5e4e3b36727a5631e9))
* partition audit log table by day with configurable retention ([#8292](https://github.com/windmill-labs/windmill/issues/8292)) ([2aef01d](https://github.com/windmill-labs/windmill/commit/2aef01d18c0723aedcc626f4f3991195620774ab))
* support minimal telemetry mode ([#8243](https://github.com/windmill-labs/windmill/issues/8243)) ([fe1519f](https://github.com/windmill-labs/windmill/commit/fe1519f1284aadd67d5dce46cf0cb52ab351f789))
### Bug Fixes
* **cli:** instruct agent to tell user about generate-metadata and sync push instead of running them ([#8318](https://github.com/windmill-labs/windmill/issues/8318)) ([7fb729c](https://github.com/windmill-labs/windmill/commit/7fb729cc8483a2e6966a8e8995678929f4d451a0))
* fix saved inputs popover infinite loop ([#8311](https://github.com/windmill-labs/windmill/issues/8311)) ([425a75e](https://github.com/windmill-labs/windmill/commit/425a75e030b15fe65676169f9069fbb7da19828e))
* native mode now properly sets DB pool size and sleep queue ([#8332](https://github.com/windmill-labs/windmill/issues/8332)) ([d8b4132](https://github.com/windmill-labs/windmill/commit/d8b4132b9ae90af759c6655f4f69479f6738e60a))
* prevent zombie jobs from looping forever ([#8313](https://github.com/windmill-labs/windmill/issues/8313)) ([48bc3e2](https://github.com/windmill-labs/windmill/commit/48bc3e244558dccb1f08f455b299600861788b0d))
* set min_connections(0) to prevent sqlx pool spin loop ([#8334](https://github.com/windmill-labs/windmill/issues/8334)) ([bf4340f](https://github.com/windmill-labs/windmill/commit/bf4340f40c1eb9cacee4c32e07ba44f2c92bf7c4))
* show diff editor content for resources without a language ([#8331](https://github.com/windmill-labs/windmill/issues/8331)) ([cbc7e78](https://github.com/windmill-labs/windmill/commit/cbc7e78f8a60bff1d8730a6183cdbc9125d8e2b1))
* skip python preinstall on native workers ([#8329](https://github.com/windmill-labs/windmill/issues/8329)) ([4306c9e](https://github.com/windmill-labs/windmill/commit/4306c9e4fef317e298a76924edb4f20aa7ced105))
* skip token expiry notifications for debugger and mcp-oauth tokens ([#8316](https://github.com/windmill-labs/windmill/issues/8316)) ([8667329](https://github.com/windmill-labs/windmill/commit/86673291100fd16aaf216ed33ca9b648b8a2b7a5))
* use !inline ref for scripts inside flows (preproc, error, ai tool) ([#8319](https://github.com/windmill-labs/windmill/issues/8319)) ([ca8a627](https://github.com/windmill-labs/windmill/commit/ca8a6274bc81ad49fa0c6166694ae4d65a4048cb))
## [1.654.0](https://github.com/windmill-labs/windmill/compare/v1.653.0...v1.654.0) (2026-03-10)
### Features
* add git sync support for workspace dependencies ([#8144](https://github.com/windmill-labs/windmill/issues/8144)) ([4f29e05](https://github.com/windmill-labs/windmill/commit/4f29e05e3ae725e0be7ab797f8fa2186d8c5c0a5))
* add kafka trigger offset reset and auto.offset.reset config ([#8283](https://github.com/windmill-labs/windmill/issues/8283)) ([b02f9e5](https://github.com/windmill-labs/windmill/commit/b02f9e5c2426bff2356e1aaaa18e05b18c5efc6b))
* add preprocessor support for dedicated workers and bunnative scripts ([#8284](https://github.com/windmill-labs/windmill/issues/8284)) ([dc0e59f](https://github.com/windmill-labs/windmill/commit/dc0e59f432a0e3a53606adb8ac76d2dd2d365ace))
* add Vertex AI support for Google Gemini models ([#8303](https://github.com/windmill-labs/windmill/issues/8303)) ([cb349cb](https://github.com/windmill-labs/windmill/commit/cb349cb3d1b7561fb70a8c23fa83dc1c9441821c))
* **frontend:** replace flat sugiyama with recursive compound layout for flow graph ([#8204](https://github.com/windmill-labs/windmill/issues/8204)) ([cad4436](https://github.com/windmill-labs/windmill/commit/cad44365ac17029a2257f12cef061219b0265570))
### Bug Fixes
* **cli:** fail when passing an invalid --workspace arg ([#8294](https://github.com/windmill-labs/windmill/issues/8294)) ([f291b1c](https://github.com/windmill-labs/windmill/commit/f291b1cc19689e69e7aa008c19ce747e9c56240e))
* debounce webhook arg accumulation with max_count/max_time limits ([#8307](https://github.com/windmill-labs/windmill/issues/8307)) ([83be59e](https://github.com/windmill-labs/windmill/commit/83be59e0e866ebd091f1e27c0571710a989fd2e4))
* delete debounce_key on post-preprocessing limit exceeded ([#8299](https://github.com/windmill-labs/windmill/issues/8299)) ([438f609](https://github.com/windmill-labs/windmill/commit/438f609a78325ee5c2493079ca27bf587fa0d5ff))
* explicilty fail when --base-url --token --workspace are invalid ([#8302](https://github.com/windmill-labs/windmill/issues/8302)) ([5baeb8c](https://github.com/windmill-labs/windmill/commit/5baeb8c842a392c21457b7561e30b385e02a6a48))
* handle missing schema in RunnableByPath during wmill.d.ts generation ([#8300](https://github.com/windmill-labs/windmill/issues/8300)) ([b841e0a](https://github.com/windmill-labs/windmill/commit/b841e0a0384941079f37374f8fbbe2dd7fb51897))
* optimize flow lock generation and add rt.d.ts guidance for TS resource types ([#8295](https://github.com/windmill-labs/windmill/issues/8295)) ([b40cf80](https://github.com/windmill-labs/windmill/commit/b40cf80fdd62cbc31db0872ada551ce213b9dac8))
* preserve teams oauth tenant on settings page reload ([#8308](https://github.com/windmill-labs/windmill/issues/8308)) ([dbfa271](https://github.com/windmill-labs/windmill/commit/dbfa271b8962fe7b3d2aa8bf494e9557047fc8b3))
* resync custom_instance_user password on startup ([#8297](https://github.com/windmill-labs/windmill/issues/8297)) ([53ac43f](https://github.com/windmill-labs/windmill/commit/53ac43f5ee34570a9bb7b3441c73095e23690300))
* show meaningful error messages in database manager schema fetch ([#8296](https://github.com/windmill-labs/windmill/issues/8296)) ([cda8439](https://github.com/windmill-labs/windmill/commit/cda843922dcfd9a02ef9926751cbf8f544d2d4b6))
* skip loading flow preview history for new flows ([#8293](https://github.com/windmill-labs/windmill/issues/8293)) ([ac8c668](https://github.com/windmill-labs/windmill/commit/ac8c668cb93e56bc2a247bbdbbec14e5608125d2))
* teams selection not sticking in workspace settings ([#8309](https://github.com/windmill-labs/windmill/issues/8309)) ([fefc8c6](https://github.com/windmill-labs/windmill/commit/fefc8c62a00fe7a39f3104091e08087cd7c37afb))
## [1.653.0](https://github.com/windmill-labs/windmill/compare/v1.652.0...v1.653.0) (2026-03-10)
### Features
* add indexer time window setting (default 7 days) ([#8290](https://github.com/windmill-labs/windmill/issues/8290)) ([0c4d72c](https://github.com/windmill-labs/windmill/commit/0c4d72cfe38d61cf3f6e9bc31056005f1adb494d))
* add slack connection fields to workspace settings export/import ([#8287](https://github.com/windmill-labs/windmill/issues/8287)) ([39e77ec](https://github.com/windmill-labs/windmill/commit/39e77ecd002b41630fa8d146ee0f15369656acda))
### Performance Improvements
* optimize job_stats storage for timestamps and zero-memory jobs ([#8289](https://github.com/windmill-labs/windmill/issues/8289)) ([2d8335d](https://github.com/windmill-labs/windmill/commit/2d8335dc43a7cb182eb5a058119d8b0be067cdfd))
## [1.652.0](https://github.com/windmill-labs/windmill/compare/v1.651.1...v1.652.0) (2026-03-09)

View File

@@ -4,7 +4,7 @@ Open-source platform for internal tools, workflows, API integrations, background
## Workflow
1. **Understand**: Before coding, read relevant docs from `docs/` to understand the area you're changing
1. **Understand**: Before coding, use `wm-ts-nav` to explore (see Code Navigation below). Use `outline` to understand file structure, `body` to read specific symbols, `def`/`callers`/`callees` to trace code. Read `docs/` for domain context.
2. **Plan**: For non-trivial changes, use plan mode. For large features, break into reviewable stages
3. **Execute**: Follow coding patterns from skills (`rust-backend`, `svelte-frontend`)
4. **Validate**: After every change, run the appropriate checks per `docs/validation.md`
@@ -14,7 +14,8 @@ Open-source platform for internal tools, workflows, API integrations, background
- **Validation**: `docs/validation.md` — what checks to run based on what you changed
- **Enterprise**: `docs/enterprise.md` — EE file conventions and PR workflow
- **Backend patterns**: use the `rust-backend` skill when writing Rust code
- **Frontend patterns**: use the `svelte-frontend` skill when writing Svelte code
- **Frontend patterns**: use the `svelte-frontend` skill when writing Svelte code. Do NOT edit svelte files unless you have read that skill.
- **Code review**: use `/local-review` to review a PR for bugs and CLAUDE.md compliance
- **Domain guides**: `.claude/skills/native-trigger/` and `frontend/tutorial-system-guide.mdc`
- **Brand/UI guidelines**: `frontend/brand-guidelines.md`
@@ -33,6 +34,7 @@ Open-source platform for internal tools, workflows, API integrations, background
Using `$bindable(default_value)` on props that can be `undefined` is **banned**. This pattern causes subtle bugs because the default value masks the `undefined` state.
**Bad:**
```svelte
let { my_prop = $bindable(default_value) }: { my_prop?: string } = $props()
```
@@ -40,6 +42,7 @@ let { my_prop = $bindable(default_value) }: { my_prop?: string } = $props()
**Correct alternatives:**
1. **Use `$derived` with nullish coalescing** — handle the potential `undefined` at the usage site:
```svelte
let { my_prop = $bindable() }: { my_prop?: string } = $props()
let effective_value = $derived(my_prop ?? default_value)
@@ -47,8 +50,43 @@ let { my_prop = $bindable(default_value) }: { my_prop?: string } = $props()
2. **Create a `useMyPropState()` helper** — encapsulate the undefined-handling logic in a reusable function and call it higher in the component tree, so the child component always receives a defined value.
## Code Navigation
`wm-ts-nav` is an AST-aware code navigator. Use **Grep** for regex/pattern search. Use **wm-ts-nav** for structural queries — it skips comments/strings and understands symbol boundaries.
**Prefer wm-ts-nav over Read** to save context window:
- `outline <file>` instead of reading a full file — understand structure first, then `body` or Read for specifics
- `body "X"` instead of reading a full file to see one function/struct
- `refs "X" --caller` instead of reading files to find which function contains each reference
- `callers "X"` / `callees "X"` for call-graph questions
```bash
NAV="sh wm-ts-nav/nav"
# Use --root backend for Rust, --root frontend/src for TS/Svelte
$NAV --root backend outline backend/path/to/file.rs # file structure
$NAV --root backend def "ServiceName" # find definition
$NAV --root backend body "decrypt_oauth_data" # extract source code
$NAV --root backend search "%" --parent ServiceName # methods on a type
$NAV --root backend search "Trigger" --kind struct # find by kind
$NAV --root backend refs "X" --file handler.rs --caller # scoped refs with caller
$NAV --root backend callers "X" # who calls X?
$NAV --root backend callees "X" # what does X call?
```
**Limitations** — syntax-level analysis, no type inference:
- Import paths are stored literally — `crate::X` and `super::X` pointing to the same type won't be linked
- Re-export chains (`pub use`) aren't followed — refs through different re-export paths won't connect
- Trait methods can't be resolved to their trait definition
- Nested `use` trees (`use foo::{bar::{A, B}, baz::C}`) aren't parsed correctly
- Glob imports (`use foo::*`) — refs won't show import origin
- Macro-generated symbols (e.g. `sqlx::FromRow`) — invisible to tree-sitter
- Single-char identifiers — intentionally filtered out of refs
- `callees` shows all identifiers in a function body, not just actual calls
- `import * as ns` namespace imports — member accesses through `ns.X` aren't resolved
## Core Principles
- **Use `outline`/`body` to explore, then `Read` with offset/limit from the results before editing** — avoid reading full files
- Search for existing code to reuse before writing new code
- Follow established patterns in the codebase
- Keep changes focused — don't refactor beyond what's asked

View File

@@ -268,11 +268,11 @@ RUN bun install -g windmill-cli \
RUN curl -fsSL https://claude.ai/install.sh | bash \
&& cp /root/.local/share/claude/versions/* /usr/bin/claude
COPY --from=php:8.3.7-cli /usr/local/bin/php /usr/bin/php
COPY --from=composer:2.7.6 /usr/bin/composer /usr/bin/composer
COPY --from=php:8.3.30-cli /usr/local/bin/php /usr/bin/php
COPY --from=composer:2.9.5 /usr/bin/composer /usr/bin/composer
# add the docker client to call docker from a worker if enabled
COPY --from=docker:dind /usr/local/bin/docker /usr/local/bin/
COPY --from=docker:29-dind /usr/local/bin/docker /usr/local/bin/
ENV RUSTUP_HOME="/tmp/windmill/cache/rustup"
ENV CARGO_HOME="/tmp/windmill/cache/cargo"

View File

@@ -192,70 +192,6 @@ sandbox:
This mounts both the main EE repo (used by the main worktree) and the EE worktrees directory (used by feature worktrees) into every sandbox container.
## Cursor SSH Integration (`wmc`)
`wm-cursor` (aliased as `wmc`) gives each worktree its own Cursor SSH remote window with an independently-focused tmux session. All windows are visible in the status bar across all Cursor terminals, but each one is focused on its own worktree.
This uses **grouped tmux sessions** — multiple sessions that share the same window list but track focus independently:
```
tmux session: main <-- your main Cursor terminal
tmux session: cursor-feat-a <-- Cursor window for feat-a (focused on wm-feat-a)
tmux session: cursor-feat-b <-- Cursor window for feat-b (focused on wm-feat-b)
\__ all three share the same windows in the status bar
```
### Setup
Run once from inside tmux on the remote:
```bash
./scripts/wm-cursor setup /home/hugo/projects/windmill
```
This:
1. **Merges `.vscode/settings.json`** — adds the `wm-tmux` terminal profile (auto-attaches to the `main` tmux session), disables auto port forwarding, configures forwarding for ports 8000/3000/5432, and stops rust-analyzer from auto-starting. Existing settings are preserved.
2. **Creates `.vscode/tasks.json`** — auto-starts the dev database (`start-dev-db.sh`) when the folder opens.
3. **Adds `wmc` alias to `~/.zshrc`** — so you can use `wmc` from any tmux window.
4. **Adds `eval "$(wmc completions)"`** to `~/.zshrc` — provides tab-completion for subcommands and worktree names (for `open`, `open-ee`, and `close`).
After setup, reopen Cursor's terminal to pick up the new profile.
### Usage
All commands run from inside a tmux session (i.e., from Cursor's integrated terminal after setup).
**Create a new worktree + open Cursor:**
```bash
wmc add -A -p "implement feature X"
```
This runs `workmux add`, creates a grouped tmux session, writes `.vscode/settings.json` in the worktree (with port forwarding matching the worktree's assigned ports), and opens a new Cursor window.
**Open Cursor for an existing worktree:**
```bash
wmc open my-feature
```
**Open the EE worktree in Cursor (no tmux session):**
```bash
wmc open-ee my-feature
```
This finds the matching `windmill-ee-private__worktrees/<name>` directory and opens it in a new Cursor window.
**Close a worktree's Cursor window and tmux window (keeps the worktree):**
```bash
wmc close my-feature
```
This kills the grouped tmux session and calls `workmux close` to close the tmux window. The worktree and branch are preserved. Grouped sessions are also automatically cleaned up when you `workmux rm` a worktree (via `scripts/worktree-cleanup`).
## Cargo Features
To build the backend with specific Cargo features (e.g., `enterprise`, `parquet`), pass them via `CARGO_FEATURES`. The backend pane reads this from `.env.local` and appends `--features <value>` to the `cargo watch` command.
@@ -270,20 +206,6 @@ CARGO_FEATURES="enterprise,parquet" wm add my-feature
This gets written to `.env.local` by the `post_create` hook (`scripts/worktree-env`), and the backend pane picks it up automatically.
**With `wmc` (wm-cursor):**
Use the `--features` flag:
```bash
# Create a new worktree with features
wmc add --features "enterprise,parquet" -A -p "implement feature X"
# Open an existing worktree with different features
wmc open my-feature --features "enterprise,parquet"
```
The `--features` flag exports `CARGO_FEATURES` so the `post_create` hook writes it to `.env.local`. When using `wmc open`, it updates the existing `.env.local` with the new features.
## Login
Default credentials: `admin@windmill.dev` / `changeme`

View File

@@ -0,0 +1,41 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id, topic, partition, \"offset\" FROM kafka_pending_commits\n WHERE workspace_id = $1 AND kafka_trigger_path = $2\n ORDER BY id",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Int8"
},
{
"ordinal": 1,
"name": "topic",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "partition",
"type_info": "Int4"
},
{
"ordinal": 3,
"name": "offset",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
false,
false,
false
]
},
"hash": "038d2fde90fa9e99e30d15161777fa3ab402e33edfca46daa95b52e525424586"
}

View File

@@ -46,11 +46,11 @@
]
},
"nullable": [
false,
false,
false,
false,
false,
true,
true,
true,
true,
true,
true,
true
]

View File

@@ -0,0 +1,29 @@
{
"db_name": "PostgreSQL",
"query": "\n UPDATE kafka_trigger\n SET\n kafka_resource_path = $1,\n group_id = $2,\n topics = $3,\n filters = $4,\n auto_offset_reset = $5,\n auto_commit = $6,\n script_path = $7,\n path = $8,\n is_flow = $9,\n edited_by = $10,\n email = $11,\n edited_at = now(),\n server_id = NULL,\n error = NULL,\n error_handler_path = $14,\n error_handler_args = $15,\n retry = $16\n WHERE\n workspace_id = $12 AND path = $13\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"VarcharArray",
"JsonbArray",
"Varchar",
"Bool",
"Varchar",
"Varchar",
"Bool",
"Varchar",
"Varchar",
"Text",
"Text",
"Varchar",
"Jsonb",
"Jsonb"
]
},
"nullable": []
},
"hash": "072e5ab78f929c6b7264f98c1588cb24cc635836276ee6faa2438f494bfbce04"
}

View File

@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE job_stats SET offsets_cs = array_append(offsets_cs, (EXTRACT(EPOCH FROM (now() - timeseries_start)) * 100)::int), timeseries_int = array_append(timeseries_int, $4) WHERE workspace_id = $1 AND job_id = $2 AND metric_id = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Uuid",
"Text",
"Int4"
]
},
"nullable": []
},
"hash": "0af0e0a1dddeee2021ba060e390e1b60caa3752669636e9fb0817a68121a9451"
}

View File

@@ -0,0 +1,40 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT j.id, j.runnable_path, j.args, j.kind::text AS \"kind!\"\n FROM v2_job j\n JOIN v2_job_queue q ON j.id = q.id\n WHERE j.runnable_path = $1\n AND j.kind = 'deploymentcallback'\n ORDER BY j.created_at DESC\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Uuid"
},
{
"ordinal": 1,
"name": "runnable_path",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "args",
"type_info": "Jsonb"
},
{
"ordinal": 3,
"name": "kind!",
"type_info": "Text"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false,
true,
true,
null
]
},
"hash": "0d4f28ca0c5697c96711ca7225a9a4013e6ccabb495c371471c9d1287defda8f"
}

View File

@@ -1,14 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n WITH _ AS (\n UPDATE debounce_key\n SET debounced_times = 0, -- reset debounced_times\n first_started_at = now(), -- rest\n previous_job_id = NULL\n WHERE job_id = $1\n )\n UPDATE v2_job_debounce_batch \n SET debounce_batch = nextval('debounce_batch_seq') -- move to new batch\n WHERE id = $1\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid"
]
},
"nullable": []
},
"hash": "16c96166ffa6b9aec65c6072b204b52b87e3c2f3d76e47eb173fc78721355066"
}

View File

@@ -1,17 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE job_stats SET timestamps = array_append(timestamps, now()), timeseries_int = array_append(timeseries_int, $4) WHERE workspace_id = $1 AND job_id = $2 AND metric_id = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Uuid",
"Text",
"Int4"
]
},
"nullable": []
},
"hash": "1db82007445ff5f644bb607aa28f5747cb50d193475fff5fcfdde37d1bc74636"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT reset_offset FROM kafka_trigger WHERE workspace_id = $1 AND path = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "reset_offset",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false
]
},
"hash": "1df610a583e86edb70c374fd66c68554a6a4291426c09dd5b04fd832f9d31208"
}

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT EXISTS(SELECT 1 FROM v2_job_queue WHERE id = $1)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "exists",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Uuid"
]
},
"nullable": [
null
]
},
"hash": "2c8b8ed14647332491846ae3fa8b0ab8113d52ae8ae613a810c2b452e0972d05"
}

View File

@@ -15,7 +15,7 @@
]
},
"nullable": [
true
false
]
},
"hash": "2d6607b3c38fe72b5663c32de58dacbabed4c5ae28101e3ae2694f96fd055a91"

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM variable WHERE path = $1 AND workspace_id = $2 RETURNING path",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false
]
},
"hash": "3317484a9c09c07c2c9db9debaecc4a4d518093ab48e79365dbb808068e0b8ff"
}

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT (config->>'native_mode')::boolean FROM config WHERE name = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "bool",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
null
]
},
"hash": "36b95bc7956eb7bba7cd6fa9cd829980a0bf4970b919cabad1daab16627404fc"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO kafka_trigger (\n path, kafka_resource_path, topics, group_id, script_path,\n is_flow, workspace_id, edited_by, email, auto_commit\n )\n VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"VarcharArray",
"Varchar",
"Varchar",
"Bool",
"Varchar",
"Varchar",
"Varchar",
"Bool"
]
},
"nullable": []
},
"hash": "45fc21026fa76e5d69f00a68a7be81abb3ec627578f2d14f0ce33896dc6ab4cf"
}

View File

@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "\n UPDATE workspace_settings\n SET git_app_installations = (\n SELECT jsonb_agg(\n CASE\n WHEN (elem->>'installation_id')::bigint = $2 THEN $1::jsonb\n ELSE elem\n END\n )\n FROM jsonb_array_elements(git_app_installations) AS elem\n )\n WHERE workspace_id = $3\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Jsonb",
"Int8",
"Text"
]
},
"nullable": []
},
"hash": "48b394bd9ca63d33a7ea97113b0096bd0777da52c05e23262572089e0c3c6c46"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT auto_commit FROM kafka_trigger WHERE workspace_id = $1 AND path = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "auto_commit",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false
]
},
"hash": "4b2a29b3ef7ec4802d81ec4b706623b991c938e40d0db25290b03dc0577c2740"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT kafka_resource_path, topics, group_id, mode AS \"mode: String\"\n FROM kafka_trigger\n WHERE workspace_id = $1 AND path = $2\n ",
"query": "\n SELECT kafka_resource_path, topics, group_id, mode AS \"mode: String\",\n auto_offset_reset, auto_commit, reset_offset\n FROM kafka_trigger\n WHERE workspace_id = $1 AND path = $2\n ",
"describe": {
"columns": [
{
@@ -33,6 +33,21 @@
}
}
}
},
{
"ordinal": 4,
"name": "auto_offset_reset",
"type_info": "Varchar"
},
{
"ordinal": 5,
"name": "auto_commit",
"type_info": "Bool"
},
{
"ordinal": 6,
"name": "reset_offset",
"type_info": "Bool"
}
],
"parameters": {
@@ -42,11 +57,14 @@
]
},
"nullable": [
false,
false,
false,
false,
false,
false,
false
]
},
"hash": "7e3bfb33fb771aec39b43a7550091ce7c9b1261b52d10f4a7f3273fed3c916df"
"hash": "4cf4be7a981173d3f242887d9313c7e60d23e9827f23c0de5b546ed56697d54a"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT auto_commit\n FROM kafka_trigger\n WHERE workspace_id = $1 AND path = $2\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "auto_commit",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false
]
},
"hash": "50807b807bb901a380926798be655c13a18dfd26e237a8218d3006e2898b5aa3"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO kafka_trigger (\n workspace_id,\n path,\n kafka_resource_path,\n group_id,\n topics,\n filters,\n script_path,\n is_flow,\n mode,\n edited_by,\n email,\n edited_at,\n error_handler_path,\n error_handler_args,\n retry\n ) VALUES (\n $1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, now(), $12, $13, $14\n )\n ",
"query": "\n INSERT INTO kafka_trigger (\n workspace_id,\n path,\n kafka_resource_path,\n group_id,\n topics,\n filters,\n auto_offset_reset,\n auto_commit,\n script_path,\n is_flow,\n mode,\n edited_by,\n email,\n edited_at,\n error_handler_path,\n error_handler_args,\n retry\n ) VALUES (\n $1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, now(), $14, $15, $16\n )\n ",
"describe": {
"columns": [],
"parameters": {
@@ -13,6 +13,8 @@
"JsonbArray",
"Varchar",
"Bool",
"Varchar",
"Bool",
{
"Custom": {
"name": "trigger_mode",
@@ -34,5 +36,5 @@
},
"nullable": []
},
"hash": "aed5439aa6dad950e505f9f8f6914fa5ca21319c501b2822c9bc751ddfc9a0a4"
"hash": "5dd6315ec270c268e905262e4b0a920837354d91a0ae16b1236c1267da71765f"
}

View File

@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM kafka_pending_commits WHERE id = ANY($1)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Int8Array"
]
},
"nullable": []
},
"hash": "80bad96cbec6b5eca57a6380e7515565490a271050dcc4b5aac2b730ae3a55b9"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "WITH active_users AS (SELECT distinct username as email FROM audit WHERE timestamp > NOW() - INTERVAL '1 month' AND (operation = 'users.login' OR operation = 'oauth.login' OR operation = 'users.token.refresh')),\n authors as (SELECT distinct email FROM usr WHERE usr.operator IS false)\n SELECT email, email NOT IN (SELECT email FROM authors) as operator_only, login_type::text, verified, super_admin, devops, name, company, username, first_time_user\n FROM password\n WHERE email IN (SELECT email FROM active_users)\n ORDER BY super_admin DESC, devops DESC\n LIMIT $1 OFFSET $2",
"query": "WITH active_users AS (SELECT distinct username as email FROM (SELECT username, timestamp, operation FROM audit_partitioned UNION ALL SELECT username, timestamp, operation FROM audit) AS a WHERE timestamp > NOW() - INTERVAL '1 month' AND (operation = 'users.login' OR operation = 'oauth.login' OR operation = 'users.token.refresh')),\n authors as (SELECT distinct email FROM usr WHERE usr.operator IS false)\n SELECT email, email NOT IN (SELECT email FROM authors) as operator_only, login_type::text, verified, super_admin, devops, name, company, username, first_time_user\n FROM password\n WHERE email IN (SELECT email FROM active_users)\n ORDER BY super_admin DESC, devops DESC\n LIMIT $1 OFFSET $2",
"describe": {
"columns": [
{
@@ -73,5 +73,5 @@
false
]
},
"hash": "72d3ebb05ac1ffeb0e8d0a3146d95bb5b90e7c4d1dc2c8a6ef06eddf6678f230"
"hash": "9229d9a9ff389cf26e480b604b83900e2d362ee934ef27284ef39f4eed440e59"
}

View File

@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE job_stats SET offsets_cs = array_append(offsets_cs, (EXTRACT(EPOCH FROM (now() - timeseries_start)) * 100)::int), timeseries_float = array_append(timeseries_float, $4) WHERE workspace_id = $1 AND job_id = $2 AND metric_id = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Uuid",
"Text",
"Float4"
]
},
"nullable": []
},
"hash": "a837494a58ab58bfa18c0385350a861076a5fc4f7eefceb1c0de5cf55293f327"
}

View File

@@ -0,0 +1,28 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT\n (elem->>'installation_id')::bigint as installation_id,\n elem->>'github_base_url' as github_base_url\n FROM workspace_settings,\n LATERAL jsonb_array_elements(git_app_installations) AS elem\n WHERE workspace_id = $1\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "installation_id",
"type_info": "Int8"
},
{
"ordinal": 1,
"name": "github_base_url",
"type_info": "Text"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
null,
null
]
},
"hash": "ad5fc9212a123a8328397496ef3b5eea1780226698e419ac59ba55012296913d"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT\n (elem->>'installation_id')::bigint as installation_id,\n elem->>'account_id' as account_id\n FROM workspace_settings,\n LATERAL jsonb_array_elements(git_app_installations) AS elem\n WHERE workspace_id = $1\n ",
"query": "\n SELECT\n (elem->>'installation_id')::bigint as installation_id,\n elem->>'account_id' as account_id,\n elem->>'github_base_url' as github_base_url\n FROM workspace_settings,\n LATERAL jsonb_array_elements(git_app_installations) AS elem\n WHERE workspace_id = $1\n ",
"describe": {
"columns": [
{
@@ -12,6 +12,11 @@
"ordinal": 1,
"name": "account_id",
"type_info": "Text"
},
{
"ordinal": 2,
"name": "github_base_url",
"type_info": "Text"
}
],
"parameters": {
@@ -20,9 +25,10 @@
]
},
"nullable": [
null,
null,
null
]
},
"hash": "0ee14619dd81df460b2b8cc6df2b89646279f77469c35deffca8e17a11d7f6c8"
"hash": "ae7adc583cdd3f876164ed60569ed531b05eaa17fccc599306eb1a96a65ee761"
}

View File

@@ -1,16 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n UPDATE workspace_settings\n SET git_app_installations = (\n SELECT jsonb_agg(\n CASE\n WHEN (elem->>'installation_id')::bigint = $2 THEN $1::jsonb\n ELSE elem\n END\n )\n FROM jsonb_array_elements(git_app_installations) AS elem\n )\n WHERE workspace_id = $3\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Jsonb",
"Int8",
"Text"
]
},
"nullable": []
},
"hash": "b4d48c820bf41619bffa8f62367e98369e1d93514e1618723a34bf96080d4ebc"
}

View File

@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT\n (elem->>'installation_id')::bigint as installation_id\n FROM workspace_settings,\n LATERAL jsonb_array_elements(git_app_installations) AS elem\n WHERE workspace_id = $1\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "installation_id",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
null
]
},
"hash": "be00ac55e8668a0ed3befda7d8595c7cda0cba0b119d4fdb8a0dea1b28a1d560"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO v2_job_completed\n (workspace_id, id, started_at, duration_ms, result, memory_peak, status, worker)\n SELECT q.workspace_id, q.id, q.started_at,\n COALESCE((EXTRACT('epoch' FROM now()) - EXTRACT('epoch' FROM COALESCE(q.started_at, now()))) * 1000, 0)::bigint,\n $2::jsonb, r.memory_peak, 'failure'::job_status, q.worker\n FROM v2_job_queue q\n LEFT JOIN v2_job_runtime r ON r.id = q.id\n WHERE q.id = $1\n ON CONFLICT (id) DO UPDATE SET status = 'failure', result = $2::jsonb",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid",
"Jsonb"
]
},
"nullable": []
},
"hash": "beecb176df512e4a94771d0d73c4c597e07e53d499131b57e4d6441fd0af09cb"
}

View File

@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM resource WHERE path = $1 AND workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": []
},
"hash": "bf2aeb9a1e649106d2a084c1d628690a44573c1869a206474811215714ba97c2"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE kafka_trigger SET reset_offset = true, server_id = NULL WHERE workspace_id = $1 AND path = $2 RETURNING true",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "?column?",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
null
]
},
"hash": "c2f38c9e09aac73d10e8f327715927c07832badb2c9145d5996b829163bdf7d9"
}

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT args FROM v2_job WHERE id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "args",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Uuid"
]
},
"nullable": [
true
]
},
"hash": "c944e384c4b4c6455b431978adc54d176f294b661675392cf92561c0e6e02e6e"
}

View File

@@ -1,14 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n WITH _ AS (\n UPDATE debounce_key\n SET debounced_times = 0,\n first_started_at = now(),\n previous_job_id = NULL\n WHERE job_id = $1\n )\n UPDATE v2_job_debounce_batch\n SET debounce_batch = nextval('debounce_batch_seq')\n WHERE id = $1\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid"
]
},
"nullable": []
},
"hash": "c9530931f670eab1208c4a284a55afdc3fcbb0eb5f98fd63e2ec89442becbfaa"
}

View File

@@ -1,17 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE job_stats SET timestamps = array_append(timestamps, now()), timeseries_float = array_append(timeseries_float, $4) WHERE workspace_id = $1 AND job_id = $2 AND metric_id = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Uuid",
"Text",
"Float4"
]
},
"nullable": []
},
"hash": "d44c37882150532383d1058639f4d3af288a346c50f29809dbc55a34088a0abc"
}

View File

@@ -1,27 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n UPDATE kafka_trigger\n SET\n kafka_resource_path = $1,\n group_id = $2,\n topics = $3,\n filters = $4,\n script_path = $5,\n path = $6,\n is_flow = $7,\n edited_by = $8,\n email = $9,\n edited_at = now(),\n server_id = NULL,\n error = NULL,\n error_handler_path = $12,\n error_handler_args = $13,\n retry = $14\n WHERE\n workspace_id = $10 AND path = $11\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"VarcharArray",
"JsonbArray",
"Varchar",
"Varchar",
"Bool",
"Varchar",
"Varchar",
"Text",
"Text",
"Varchar",
"Jsonb",
"Jsonb"
]
},
"nullable": []
},
"hash": "e2921e44c70cf6c76c55177f2b56985e84c59ecb3e1a13fcf27d5f7ae5f8d84c"
}

View File

@@ -15,7 +15,7 @@
]
},
"nullable": [
true
false
]
},
"hash": "eba16eb819e2644284fb073c891706d78a6f24cb0e614d7d81ba1b643805bf06"

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE kafka_trigger SET reset_offset = false WHERE workspace_id = $1 AND path = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": []
},
"hash": "ef15599f532fab2cbb487542ffec047cf3b7ce22ce868db1b1a63e6c10d0d12b"
}

View File

@@ -0,0 +1,18 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO kafka_pending_commits (workspace_id, kafka_trigger_path, topic, partition, \"offset\")\n VALUES ($1, $2, $3, $4, $5)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar",
"Int4",
"Int8"
]
},
"nullable": []
},
"hash": "f67e5c96eb9cb35953d4c3e83e0fcbb5b647737e0366529a2f418218b1a74679"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO audit\n (workspace_id, username, operation, action_kind, resource, parameters, email, span)\n VALUES ($1, $2, $3, $4, $5, $6, $7, $8)",
"query": "INSERT INTO audit_partitioned\n (workspace_id, username, operation, action_kind, resource, parameters, email, span)\n VALUES ($1, $2, $3, $4, $5, $6, $7, $8)",
"describe": {
"columns": [],
"parameters": {
@@ -29,5 +29,5 @@
},
"nullable": []
},
"hash": "ad8487a797713b3a6c10fb399c9fb8dcd940bb92e998145e250f28ccfe1c7033"
"hash": "fbccafe6d34093a723b9d5a6ee8d618a80ceba6de2d39202e6293ef5207c31f6"
}

214
backend/Cargo.lock generated
View File

@@ -8102,9 +8102,9 @@ dependencies = [
[[package]]
name = "libz-sys"
version = "1.1.24"
version = "1.1.25"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4735e9cbde5aac84a5ce588f6b23a90b9b0b528f6c5a8db8a4aff300463a0839"
checksum = "d52f4c29e2a68ac30c9087e1b772dc9f44a2b66ed44edf2266cf2be9b03dafc1"
dependencies = [
"cc",
"libc",
@@ -9383,9 +9383,9 @@ checksum = "80adb31078122c880307e9cdfd4e3361e6545c319f9b9dcafcb03acd3b51a575"
[[package]]
name = "once_cell"
version = "1.21.3"
version = "1.21.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "42f5e15c9953c5e4ccceeb2e7382a716482c34515315f7b03532b8b4e8393d2d"
checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50"
[[package]]
name = "once_cell_polyfill"
@@ -9460,9 +9460,9 @@ dependencies = [
[[package]]
name = "openssl"
version = "0.10.75"
version = "0.10.76"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08838db121398ad17ab8531ce9de97b244589089e290a384c900cb9ff7434328"
checksum = "951c002c75e16ea2c65b8c7e4d3d51d5530d8dfa7d060b4776828c88cfb18ecf"
dependencies = [
"bitflags 2.9.4",
"cfg-if",
@@ -9507,9 +9507,9 @@ dependencies = [
[[package]]
name = "openssl-sys"
version = "0.9.111"
version = "0.9.112"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "82cab2d520aa75e3c58898289429321eb788c3106963d0dc886ec7a5f4adc321"
checksum = "57d55af3b3e226502be1526dfdba67ab0e9c96fc293004e79576b2b9edb0dbdb"
dependencies = [
"cc",
"libc",
@@ -10641,9 +10641,9 @@ dependencies = [
[[package]]
name = "quick_cache"
version = "0.6.18"
version = "0.6.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7ada44a88ef953a3294f6eb55d2007ba44646015e18613d2f213016379203ef3"
checksum = "530e84778a55de0f52645a51d4e3b9554978acd6a1e7cd50b6a6784692b3029e"
dependencies = [
"ahash 0.8.12",
"equivalent",
@@ -11987,9 +11987,9 @@ checksum = "ece8e78b2f38ec51c51f5d475df0a7187ba5111b2a28bdc761ee05b075d40a71"
[[package]]
name = "schannel"
version = "0.1.28"
version = "0.1.29"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "891d81b926048e76efe18581bf793546b4c0eaf8448d72be8de2bbee5fd166e1"
checksum = "91c1b7e4904c873ef0710c1f407dde2e6287de2bebc1bbbf7d430bb7cbffd939"
dependencies = [
"windows-sys 0.61.2",
]
@@ -13854,9 +13854,9 @@ dependencies = [
[[package]]
name = "tempfile"
version = "3.26.0"
version = "3.27.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "82a72c767771b47409d2345987fda8628641887d5466101319899796367354a0"
checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd"
dependencies = [
"fastrand",
"getrandom 0.4.2",
@@ -15741,7 +15741,7 @@ dependencies = [
[[package]]
name = "windmill"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"async-nats",
@@ -15808,7 +15808,7 @@ dependencies = [
[[package]]
name = "windmill-alerting"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -15821,7 +15821,7 @@ dependencies = [
[[package]]
name = "windmill-api"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"argon2",
@@ -15941,6 +15941,7 @@ dependencies = [
"windmill-parser-py",
"windmill-parser-py-imports",
"windmill-parser-sql",
"windmill-parser-sql-asset",
"windmill-parser-ts",
"windmill-queue",
"windmill-store",
@@ -15961,7 +15962,7 @@ dependencies = [
[[package]]
name = "windmill-api-agent-workers"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -15984,7 +15985,7 @@ dependencies = [
[[package]]
name = "windmill-api-assets"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -15997,7 +15998,7 @@ dependencies = [
[[package]]
name = "windmill-api-auth"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -16023,7 +16024,7 @@ dependencies = [
[[package]]
name = "windmill-api-client"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"reqwest 0.12.28",
"serde",
@@ -16033,7 +16034,7 @@ dependencies = [
[[package]]
name = "windmill-api-configs"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16050,7 +16051,7 @@ dependencies = [
[[package]]
name = "windmill-api-debug"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"axum 0.7.9",
"base64 0.22.1",
@@ -16073,7 +16074,7 @@ dependencies = [
[[package]]
name = "windmill-api-embeddings"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -16096,7 +16097,7 @@ dependencies = [
[[package]]
name = "windmill-api-flow-conversations"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16112,7 +16113,7 @@ dependencies = [
[[package]]
name = "windmill-api-flows"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16132,7 +16133,7 @@ dependencies = [
[[package]]
name = "windmill-api-groups"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16152,7 +16153,7 @@ dependencies = [
[[package]]
name = "windmill-api-inputs"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16166,7 +16167,7 @@ dependencies = [
[[package]]
name = "windmill-api-integration-tests"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"async-nats",
@@ -16186,6 +16187,7 @@ dependencies = [
"windmill-api-auth",
"windmill-api-client",
"windmill-common",
"windmill-git-sync",
"windmill-native-triggers",
"windmill-test-utils",
"windmill-worker",
@@ -16193,7 +16195,7 @@ dependencies = [
[[package]]
name = "windmill-api-jobs"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -16218,7 +16220,7 @@ dependencies = [
[[package]]
name = "windmill-api-npm-proxy"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"axum 0.7.9",
"flate2",
@@ -16236,7 +16238,7 @@ dependencies = [
[[package]]
name = "windmill-api-openapi"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -16257,7 +16259,7 @@ dependencies = [
[[package]]
name = "windmill-api-schedule"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16277,7 +16279,7 @@ dependencies = [
[[package]]
name = "windmill-api-scripts"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16307,7 +16309,7 @@ dependencies = [
[[package]]
name = "windmill-api-settings"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -16334,7 +16336,7 @@ dependencies = [
[[package]]
name = "windmill-api-sse"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"lazy_static",
"serde",
@@ -16346,7 +16348,7 @@ dependencies = [
[[package]]
name = "windmill-api-users"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"argon2",
"axum 0.7.9",
@@ -16369,7 +16371,7 @@ dependencies = [
[[package]]
name = "windmill-api-workers"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16383,7 +16385,7 @@ dependencies = [
[[package]]
name = "windmill-api-workspaces"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16414,7 +16416,7 @@ dependencies = [
[[package]]
name = "windmill-audit"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"chrono",
"lazy_static",
@@ -16428,7 +16430,7 @@ dependencies = [
[[package]]
name = "windmill-autoscaling"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -16447,7 +16449,7 @@ dependencies = [
[[package]]
name = "windmill-common"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"aes-gcm",
"anyhow",
@@ -16546,7 +16548,7 @@ dependencies = [
[[package]]
name = "windmill-dep-map"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"chrono",
"itertools 0.14.0",
@@ -16565,7 +16567,7 @@ dependencies = [
[[package]]
name = "windmill-git-sync"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"regex",
"serde",
@@ -16580,7 +16582,7 @@ dependencies = [
[[package]]
name = "windmill-indexer"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"astral-tokio-tar",
@@ -16604,7 +16606,7 @@ dependencies = [
[[package]]
name = "windmill-jseval"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"futures",
@@ -16621,7 +16623,7 @@ dependencies = [
[[package]]
name = "windmill-macros"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"itertools 0.14.0",
"lazy_static",
@@ -16637,7 +16639,7 @@ dependencies = [
[[package]]
name = "windmill-mcp"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"async-trait",
@@ -16658,7 +16660,7 @@ dependencies = [
[[package]]
name = "windmill-native-triggers"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"async-trait",
@@ -16689,7 +16691,7 @@ dependencies = [
[[package]]
name = "windmill-oauth"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"async-oauth2",
@@ -16713,7 +16715,7 @@ dependencies = [
[[package]]
name = "windmill-object-store"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"async-stream",
@@ -16747,7 +16749,7 @@ dependencies = [
[[package]]
name = "windmill-operator"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"futures",
@@ -16765,7 +16767,7 @@ dependencies = [
[[package]]
name = "windmill-parser"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"convert_case 0.6.0",
"serde",
@@ -16774,7 +16776,7 @@ dependencies = [
[[package]]
name = "windmill-parser-bash"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -16786,7 +16788,7 @@ dependencies = [
[[package]]
name = "windmill-parser-csharp"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"serde_json",
@@ -16798,7 +16800,7 @@ dependencies = [
[[package]]
name = "windmill-parser-go"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"gosyn",
@@ -16810,7 +16812,7 @@ dependencies = [
[[package]]
name = "windmill-parser-graphql"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -16822,7 +16824,7 @@ dependencies = [
[[package]]
name = "windmill-parser-java"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"serde_json",
@@ -16834,7 +16836,7 @@ dependencies = [
[[package]]
name = "windmill-parser-nu"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"nu-parser",
@@ -16845,7 +16847,7 @@ dependencies = [
[[package]]
name = "windmill-parser-php"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"itertools 0.14.0",
@@ -16856,7 +16858,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"itertools 0.14.0",
@@ -16864,12 +16866,22 @@ dependencies = [
"rustpython-parser",
"serde_json",
"windmill-parser",
"windmill-parser-sql",
]
[[package]]
name = "windmill-parser-py-asset"
version = "1.655.0"
dependencies = [
"anyhow",
"rustpython-ast",
"rustpython-parser",
"windmill-parser",
"windmill-parser-sql-asset",
]
[[package]]
name = "windmill-parser-py-imports"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"async-recursion",
@@ -16893,7 +16905,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ruby"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -16907,7 +16919,7 @@ dependencies = [
[[package]]
name = "windmill-parser-rust"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"convert_case 0.6.0",
@@ -16924,7 +16936,7 @@ dependencies = [
[[package]]
name = "windmill-parser-sql"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -16932,6 +16944,17 @@ dependencies = [
"regex-lite",
"serde",
"serde_json",
"windmill-parser",
"windmill-types",
]
[[package]]
name = "windmill-parser-sql-asset"
version = "1.655.0"
dependencies = [
"anyhow",
"serde",
"serde_json",
"sqlparser 0.59.0",
"windmill-parser",
"windmill-types",
@@ -16939,7 +16962,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ts"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -16953,12 +16976,27 @@ dependencies = [
"triomphe",
"wasm-bindgen",
"windmill-parser",
"windmill-parser-sql",
]
[[package]]
name = "windmill-parser-ts-asset"
version = "1.655.0"
dependencies = [
"anyhow",
"serde-wasm-bindgen",
"swc_common",
"swc_ecma_ast",
"swc_ecma_parser",
"swc_ecma_visit",
"triomphe",
"wasm-bindgen",
"windmill-parser",
"windmill-parser-sql-asset",
]
[[package]]
name = "windmill-parser-wac"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"rustpython-ast",
@@ -16974,7 +17012,7 @@ dependencies = [
[[package]]
name = "windmill-parser-yaml"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"serde",
@@ -16985,7 +17023,7 @@ dependencies = [
[[package]]
name = "windmill-queue"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"async-recursion",
@@ -17022,7 +17060,7 @@ dependencies = [
[[package]]
name = "windmill-runtime-nativets"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"const_format",
@@ -17060,7 +17098,7 @@ dependencies = [
[[package]]
name = "windmill-sql-datatype-parser-wasm"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"getrandom 0.3.4",
"wasm-bindgen",
@@ -17071,7 +17109,7 @@ dependencies = [
[[package]]
name = "windmill-store"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"async-recursion",
@@ -17100,7 +17138,7 @@ dependencies = [
[[package]]
name = "windmill-test-utils"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -17123,7 +17161,7 @@ dependencies = [
[[package]]
name = "windmill-trigger"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"async-trait",
@@ -17156,7 +17194,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-email"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"async-trait",
@@ -17176,7 +17214,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-gcp"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"async-trait",
@@ -17210,7 +17248,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-http"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"async-trait",
@@ -17245,7 +17283,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-kafka"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"async-trait",
@@ -17268,7 +17306,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-mqtt"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"async-trait",
@@ -17292,7 +17330,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-nats"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"async-nats",
@@ -17316,7 +17354,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-postgres"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"async-trait",
@@ -17351,7 +17389,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-sqs"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"async-trait",
@@ -17379,7 +17417,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-websocket"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"async-trait",
@@ -17402,7 +17440,7 @@ dependencies = [
[[package]]
name = "windmill-types"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"bitflags 2.9.4",
@@ -17420,7 +17458,7 @@ dependencies = [
[[package]]
name = "windmill-worker"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"anyhow",
"async-once-cell",
@@ -17526,7 +17564,7 @@ dependencies = [
[[package]]
name = "windmill-worker-volumes"
version = "1.652.0"
version = "1.655.0"
dependencies = [
"bytes",
"futures",

View File

@@ -1,6 +1,6 @@
[package]
name = "windmill"
version = "1.652.0"
version = "1.655.0"
authors.workspace = true
edition.workspace = true
@@ -59,6 +59,7 @@ members = [
"./windmill-oauth",
"./parsers/windmill-parser",
"./parsers/windmill-parser-ts",
"./parsers/windmill-parser-ts-asset",
"./parsers/windmill-parser-go",
"./parsers/windmill-parser-rust",
"./parsers/windmill-parser-csharp",
@@ -67,8 +68,11 @@ members = [
"./parsers/windmill-parser-ruby",
"./parsers/windmill-parser-bash",
"./parsers/windmill-parser-py",
"./parsers/windmill-parser-py-asset",
"./parsers/windmill-parser-py-imports",
"./parsers/windmill-parser-wac",
"./parsers/windmill-parser-sql",
"./parsers/windmill-parser-sql-asset",
"./parsers/windmill-sql-datatype-parser-wasm",
"./parsers/windmill-parser-yaml", "windmill-macros", "parsers/windmill-parser-nu",
"./windmill-worker-volumes",
@@ -78,7 +82,7 @@ members = [
exclude = ["./windmill-duckdb-ffi-internal"]
[workspace.package]
version = "1.652.0"
version = "1.655.0"
authors = ["Ruben Fiszel <ruben@windmill.dev>"]
edition = "2021"
@@ -320,7 +324,9 @@ windmill-api-workers = { path = "./windmill-api-workers" }
windmill-store = { path = "./windmill-store" }
windmill-parser = { path = "./parsers/windmill-parser" }
windmill-parser-ts = { path = "./parsers/windmill-parser-ts" }
windmill-parser-ts-asset = { path = "./parsers/windmill-parser-ts-asset" }
windmill-parser-py = { path = "./parsers/windmill-parser-py" }
windmill-parser-py-asset = { path = "./parsers/windmill-parser-py-asset" }
windmill-parser-py-imports = { path = "./parsers/windmill-parser-py-imports" }
windmill-parser-go = { path = "./parsers/windmill-parser-go" }
windmill-parser-rust = { path = "./parsers/windmill-parser-rust" }
@@ -331,6 +337,7 @@ windmill-parser-ruby = { path = "./parsers/windmill-parser-ruby" }
windmill-parser-nu = { path = "./parsers/windmill-parser-nu" }
windmill-parser-bash = { path = "./parsers/windmill-parser-bash" }
windmill-parser-sql = { path = "./parsers/windmill-parser-sql" }
windmill-parser-sql-asset = { path = "./parsers/windmill-parser-sql-asset" }
windmill-parser-graphql = { path = "./parsers/windmill-parser-graphql" }
windmill-parser-php = { path = "./parsers/windmill-parser-php" }
windmill-parser-wac = { path = "./parsers/windmill-parser-wac" }

View File

@@ -1 +1 @@
716b350bce1730b302c66ea69df618fa40f2f16b
c74c86b78a66b976fd9968b21f77903723e668ec

View File

@@ -0,0 +1,2 @@
ALTER TABLE kafka_trigger DROP COLUMN auto_offset_reset;
ALTER TABLE kafka_trigger DROP COLUMN reset_offset;

View File

@@ -0,0 +1,2 @@
ALTER TABLE kafka_trigger ADD COLUMN auto_offset_reset VARCHAR(10) NOT NULL DEFAULT 'latest';
ALTER TABLE kafka_trigger ADD COLUMN reset_offset BOOLEAN NOT NULL DEFAULT FALSE;

View File

@@ -0,0 +1,2 @@
ALTER TABLE job_stats DROP COLUMN IF EXISTS timeseries_start;
ALTER TABLE job_stats DROP COLUMN IF EXISTS offsets_cs;

View File

@@ -0,0 +1,5 @@
-- Store timeseries timestamps as a start time + integer centisecond offsets
-- instead of full TIMESTAMPTZ[] arrays. Saves ~4 bytes per data point.
-- i32 centiseconds gives ~248 days of range with 10ms precision.
ALTER TABLE job_stats ADD COLUMN IF NOT EXISTS timeseries_start TIMESTAMPTZ;
ALTER TABLE job_stats ADD COLUMN IF NOT EXISTS offsets_cs INTEGER[];

View File

@@ -0,0 +1 @@
DROP TABLE IF EXISTS audit_partitioned CASCADE;

View File

@@ -0,0 +1,58 @@
-- Create a new daily-partitioned audit table alongside the existing one.
-- New inserts go to audit_partitioned; reads UNION ALL both tables.
-- The old audit table empties out naturally via retention cleanup.
CREATE TABLE audit_partitioned (
workspace_id VARCHAR(50) NOT NULL,
id BIGINT NOT NULL DEFAULT nextval('audit_id_seq'),
timestamp TIMESTAMPTZ NOT NULL DEFAULT now(),
username VARCHAR(255) NOT NULL,
operation VARCHAR(50) NOT NULL,
action_kind ACTION_KIND NOT NULL,
resource VARCHAR(255),
parameters JSONB,
email VARCHAR(255),
span VARCHAR(255),
PRIMARY KEY (id, timestamp)
) PARTITION BY RANGE (timestamp);
-- Create daily partitions for today + 3 days
DO $$
DECLARE
curr_date DATE := CURRENT_DATE;
end_date DATE := CURRENT_DATE + INTERVAL '3 days';
BEGIN
WHILE curr_date <= end_date LOOP
EXECUTE format(
'CREATE TABLE %I PARTITION OF audit_partitioned FOR VALUES FROM (%L) TO (%L)',
'audit_' || to_char(curr_date, 'YYYYMMDD'),
curr_date,
curr_date + INTERVAL '1 day'
);
curr_date := curr_date + INTERVAL '1 day';
END LOOP;
END $$;
-- Indexes (auto-propagated to all current and future partitions)
CREATE INDEX ix_audit_partitioned_timestamps ON audit_partitioned (timestamp DESC);
CREATE INDEX idx_audit_partitioned_workspace ON audit_partitioned (workspace_id, timestamp DESC);
CREATE INDEX idx_audit_partitioned_recent_login_activities
ON audit_partitioned (timestamp, username)
WHERE operation IN ('users.login', 'oauth.login', 'users.token.refresh');
-- Grants (match the old audit table)
GRANT ALL ON audit_partitioned TO windmill_user;
GRANT ALL ON audit_partitioned TO windmill_admin;
-- RLS (match the old audit table)
ALTER TABLE audit_partitioned ENABLE ROW LEVEL SECURITY;
CREATE POLICY admin_policy ON audit_partitioned FOR ALL TO windmill_admin USING (true);
CREATE POLICY see_own ON audit_partitioned FOR ALL TO windmill_user
USING ((username)::text = current_setting('session.user'::text));
CREATE POLICY schedule ON audit_partitioned FOR INSERT TO windmill_user
WITH CHECK ((username)::text ~~ 'schedule-%'::text);
CREATE POLICY schedule_audit ON audit_partitioned FOR INSERT TO windmill_user
WITH CHECK ((parameters ->> 'end_user'::text) ~~ 'schedule-%'::text);
CREATE POLICY webhook ON audit_partitioned FOR INSERT TO windmill_user
WITH CHECK ((username)::text ~~ 'webhook-%'::text);

View File

@@ -0,0 +1,2 @@
DROP TABLE IF EXISTS kafka_pending_commits;
ALTER TABLE kafka_trigger DROP COLUMN auto_commit;

View File

@@ -0,0 +1,14 @@
ALTER TABLE kafka_trigger ADD COLUMN auto_commit BOOLEAN NOT NULL DEFAULT TRUE;
CREATE TABLE kafka_pending_commits (
id BIGSERIAL PRIMARY KEY,
workspace_id VARCHAR(50) NOT NULL,
kafka_trigger_path VARCHAR(255) NOT NULL,
topic VARCHAR(255) NOT NULL,
partition INTEGER NOT NULL,
"offset" BIGINT NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
FOREIGN KEY (workspace_id, kafka_trigger_path) REFERENCES kafka_trigger(workspace_id, path) ON DELETE CASCADE
);
CREATE INDEX idx_kafka_pending_commits_trigger ON kafka_pending_commits (workspace_id, kafka_trigger_path);

View File

@@ -0,0 +1,16 @@
[package]
name = "windmill-parser-py-asset"
version.workspace = true
edition.workspace = true
authors.workspace = true
[lib]
name = "windmill_parser_py_asset"
path = "./src/lib.rs"
[dependencies]
windmill-parser.workspace = true
windmill-parser-sql-asset.workspace = true
rustpython-parser.workspace = true
rustpython-ast = { version = "0.4.0", features = ["visitor"] }
anyhow.workspace = true

View File

@@ -215,7 +215,7 @@ impl AssetsFinder {
_ => return Err(()),
};
// We use the SQL parser to detect RW, specific tables, etc.
let sql_assets = windmill_parser_sql::parse_wmill_sdk_sql_assets(
let sql_assets = windmill_parser_sql_asset::parse_wmill_sdk_sql_assets(
*kind,
path,
schema.as_deref(),

View File

@@ -10,7 +10,6 @@ path = "./src/lib.rs"
[dependencies]
windmill-parser.workspace = true
windmill-parser-sql.workspace = true
rustpython-parser.workspace = true
itertools.workspace = true
serde_json.workspace = true

View File

@@ -21,11 +21,8 @@ use rustpython_parser::{
Parse,
};
pub mod asset_parser;
pub mod pydantic_parser;
pub use asset_parser::parse_assets;
const FUNCTION_CALL: &str = "<function call>";
/// Cheap string-based check to see if code might contain Pydantic models or dataclasses.

View File

@@ -0,0 +1,17 @@
[package]
name = "windmill-parser-sql-asset"
version.workspace = true
edition.workspace = true
authors.workspace = true
[lib]
name = "windmill_parser_sql_asset"
path = "./src/lib.rs"
[dependencies]
windmill-parser.workspace = true
windmill-types.workspace = true
anyhow.workspace = true
serde_json.workspace = true
serde.workspace = true
sqlparser = { version = "0.59.0", features = ["visitor"] }

View File

@@ -0,0 +1,4 @@
mod asset_parser;
mod asset_parser_utils;
pub use asset_parser::parse_assets;
pub use asset_parser_utils::parse_wmill_sdk_sql_assets;

View File

@@ -20,5 +20,4 @@ windmill-types.workspace = true
anyhow.workspace = true
lazy_static.workspace = true
serde_json.workspace = true
serde.workspace = true
sqlparser = { version = "0.59.0", features = ["visitor"] }
serde.workspace = true

View File

@@ -20,11 +20,6 @@ pub use windmill_parser::{Arg, MainArgSignature, ObjectType, Typ};
pub const SANITIZED_ENUM_STR: &str = "__sanitized_enum__";
pub const SANITIZED_RAW_STRING_STR: &str = "__sanitized_raw_string__";
mod asset_parser;
mod asset_parser_utils;
pub use asset_parser::parse_assets;
pub use asset_parser_utils::parse_wmill_sdk_sql_assets;
pub fn parse_mysql_sig(code: &str) -> anyhow::Result<MainArgSignature> {
let parsed = parse_mysql_file(&code)?;
if let Some(x) = parsed {

View File

@@ -0,0 +1,23 @@
[package]
name = "windmill-parser-ts-asset"
version.workspace = true
edition.workspace = true
authors.workspace = true
[lib]
name = "windmill_parser_ts_asset"
path = "./src/lib.rs"
[target.'cfg(target_arch = "wasm32")'.dependencies]
wasm-bindgen.workspace = true
serde-wasm-bindgen.workspace = true
[dependencies]
windmill-parser.workspace = true
windmill-parser-sql-asset.workspace = true
swc_common.workspace = true
triomphe.workspace = true
swc_ecma_parser.workspace = true
swc_ecma_ast.workspace = true
swc_ecma_visit.workspace = true
anyhow.workspace = true

View File

@@ -259,7 +259,7 @@ impl Visit for AssetsFinder {
});
// We use the SQL parser to detect RW, specific tables, etc.
let sql_assets = windmill_parser_sql::parse_wmill_sdk_sql_assets(
let sql_assets = windmill_parser_sql_asset::parse_wmill_sdk_sql_assets(
*kind,
asset_name,
schema.as_deref(),

View File

@@ -15,7 +15,6 @@ serde-wasm-bindgen.workspace = true
[dependencies]
windmill-parser.workspace = true
windmill-parser-sql.workspace = true
swc_common.workspace = true
triomphe.workspace = true
swc_ecma_parser.workspace = true

View File

@@ -211,8 +211,6 @@ pub enum TypeDecl {
Interface(TsInterfaceDecl),
Alias(TsTypeAliasDecl),
}
pub mod asset_parser;
pub use asset_parser::parse_assets;
/// skip_params is a micro optimization for when we just want to find the main
/// function without parsing all the params.

View File

@@ -39,6 +39,7 @@ nu-parser = [ "dep:windmill-parser-nu"]
java-parser = [ "dep:windmill-parser-java"]
ruby-parser = [ "dep:windmill-parser-ruby"]
wac-parser = [ "dep:windmill-parser-wac"]
asset-parser = [ "dep:windmill-parser-ts-asset", "dep:windmill-parser-py-asset", "dep:windmill-parser-sql-asset"]
[dependencies]
anyhow.workspace = true
@@ -57,6 +58,9 @@ windmill-parser-nu = { workspace = true, optional = true }
windmill-parser-java = { workspace = true, optional = true }
windmill-parser-ruby = { workspace = true, optional = true }
windmill-parser-wac = { workspace = true, optional = true }
windmill-parser-ts-asset = { workspace = true, optional = true }
windmill-parser-py-asset = { workspace = true, optional = true }
windmill-parser-sql-asset = { workspace = true, optional = true }
wasm-bindgen.workspace = true
serde_json.workspace = true

View File

@@ -61,6 +61,11 @@ const targets = [
desc: "Workflow-as-Code",
features: "wac-parser",
env: "default",
}, {
ident: "asset",
desc: "Asset parsers (TS, Python, SQL) with SQL AST",
features: "asset-parser",
env: "default",
},
# ^^^ Add new entry here ^^^
];

View File

@@ -33,3 +33,6 @@ popd
pushd "pkg-java" && npm publish ${args}
popd
pushd "pkg-asset" && npm publish ${args}
popd

View File

@@ -187,28 +187,28 @@ pub fn parse_ruby(code: &str) -> String {
wrap_sig(windmill_parser_ruby::parse_ruby_signature(code))
}
#[cfg(feature = "sql-parser")]
#[cfg(feature = "asset-parser")]
#[wasm_bindgen]
pub fn parse_assets_sql(code: &str) -> String {
match windmill_parser_sql::parse_assets(code) {
match windmill_parser_sql_asset::parse_assets(code) {
Ok(r) => serde_json::to_string(&r).unwrap(),
Err(err) => format!("err: {:?}", err),
}
}
#[cfg(feature = "ts-parser")]
#[cfg(feature = "asset-parser")]
#[wasm_bindgen]
pub fn parse_assets_ts(code: &str) -> String {
match windmill_parser_ts::parse_assets(code) {
match windmill_parser_ts_asset::parse_assets(code) {
Ok(r) => serde_json::to_string(&r).unwrap(),
Err(err) => format!("err: {:?}", err),
}
}
#[cfg(feature = "py-parser")]
#[cfg(feature = "asset-parser")]
#[wasm_bindgen]
pub fn parse_assets_py(code: &str) -> String {
match windmill_parser_py::parse_assets(code) {
match windmill_parser_py_asset::parse_assets(code) {
Ok(r) => serde_json::to_string(&r).unwrap(),
Err(err) => format!("err: {:?}", err),
}

View File

@@ -20,6 +20,7 @@ pub async fn connect_db(
server_mode: bool,
indexer_mode: bool,
worker_mode: bool,
num_workers: i32,
#[cfg(feature = "private")] mut killpill_rx: tokio::sync::broadcast::Receiver<()>,
) -> anyhow::Result<sqlx::Pool<sqlx::Postgres>> {
use anyhow::Context;
@@ -34,13 +35,7 @@ pub async fn connect_db(
} else if indexer_mode {
DEFAULT_MAX_CONNECTIONS_INDEXER
} else {
DEFAULT_MAX_CONNECTIONS_WORKER
+ std::env::var("NUM_WORKERS")
.ok()
.map(|x| x.parse().ok())
.flatten()
.unwrap_or(1)
- 1
DEFAULT_MAX_CONNECTIONS_WORKER + (num_workers.max(1) as u32) - 1
}
}
};
@@ -103,7 +98,7 @@ pub async fn connect(
use sqlx::Executor;
use std::time::Duration;
let mut pool_options = sqlx::postgres::PgPoolOptions::new()
.min_connections((max_connections / 5).clamp(1, max_connections))
.min_connections(0)
.max_connections(max_connections)
.max_lifetime(Duration::from_secs(30 * 60)); // 30 mins
if worker_mode {

View File

@@ -37,12 +37,13 @@ use windmill_common::ee_oss::{
use windmill_common::{
agent_workers::AgentConfig,
global_settings::{
APP_WORKSPACED_ROUTE_SETTING, BASE_URL_SETTING, BUNFIG_INSTALL_SCOPES_SETTING,
CRITICAL_ALERTS_ON_DB_OVERSIZE_SETTING, CRITICAL_ALERTS_ON_TOKEN_EXPIRY_SETTING,
CRITICAL_ALERT_MUTE_UI_SETTING, CRITICAL_ERROR_CHANNELS_SETTING, CUSTOM_TAGS_SETTING,
DEFAULT_TAGS_PER_WORKSPACE_SETTING, DEFAULT_TAGS_WORKSPACES_SETTING, EMAIL_DOMAIN_SETTING,
ENV_SETTINGS, EXPOSE_DEBUG_METRICS_SETTING, EXPOSE_METRICS_SETTING,
EXTRA_PIP_INDEX_URL_SETTING, HUB_API_SECRET_SETTING, HUB_BASE_URL_SETTING, INDEXER_SETTING,
APP_WORKSPACED_ROUTE_SETTING, AUDIT_LOG_RETENTION_DAYS_SETTING, BASE_URL_SETTING,
BUNFIG_INSTALL_SCOPES_SETTING, CRITICAL_ALERTS_ON_DB_OVERSIZE_SETTING,
CRITICAL_ALERTS_ON_TOKEN_EXPIRY_SETTING, CRITICAL_ALERT_MUTE_UI_SETTING,
CRITICAL_ERROR_CHANNELS_SETTING, CUSTOM_TAGS_SETTING, DEFAULT_TAGS_PER_WORKSPACE_SETTING,
DEFAULT_TAGS_WORKSPACES_SETTING, EMAIL_DOMAIN_SETTING, ENV_SETTINGS,
EXPOSE_DEBUG_METRICS_SETTING, EXPOSE_METRICS_SETTING, EXTRA_PIP_INDEX_URL_SETTING,
HUB_API_SECRET_SETTING, HUB_BASE_URL_SETTING, INDEXER_SETTING,
INSTANCE_PYTHON_VERSION_SETTING, JOB_DEFAULT_TIMEOUT_SECS_SETTING, JOB_ISOLATION_SETTING,
JWT_SECRET_SETTING, KEEP_JOB_DIR_SETTING, LICENSE_KEY_SETTING, MAVEN_REPOS_SETTING,
MAVEN_SETTINGS_XML_SETTING, MONITOR_LOGS_ON_OBJECT_STORE_SETTING, NO_DEFAULT_MAVEN_SETTING,
@@ -97,12 +98,13 @@ use windmill_worker::{
use crate::monitor::{
initial_load, load_keep_job_dir, load_metrics_debug_enabled, load_require_preexisting_user,
load_tag_per_workspace_enabled, load_tag_per_workspace_workspaces, monitor_db,
reload_app_workspaced_route_setting, reload_base_url_setting,
reload_bunfig_install_scopes_setting, reload_critical_alert_mute_ui_setting,
reload_critical_alerts_on_token_expiry_setting, reload_critical_error_channels_setting,
reload_extra_pip_index_url_setting, reload_hub_api_secret_setting, reload_hub_base_url_setting,
reload_job_default_timeout_setting, reload_job_isolation_setting, reload_jwt_secret_setting,
reload_license_key, reload_npm_config_registry_setting, reload_otel_tracing_proxy_setting,
reload_app_workspaced_route_setting, reload_audit_log_retention_days_setting,
reload_base_url_setting, reload_bunfig_install_scopes_setting,
reload_critical_alert_mute_ui_setting, reload_critical_alerts_on_token_expiry_setting,
reload_critical_error_channels_setting, reload_extra_pip_index_url_setting,
reload_hub_api_secret_setting, reload_hub_base_url_setting, reload_job_default_timeout_setting,
reload_job_isolation_setting, reload_jwt_secret_setting, reload_license_key,
reload_npm_config_registry_setting, reload_otel_tracing_proxy_setting,
reload_pip_index_url_setting, reload_retention_period_setting, reload_scim_token_setting,
reload_smtp_config, reload_uv_index_strategy_setting, reload_worker_config, MonitorIteration,
};
@@ -241,7 +243,14 @@ async fn cache_hub_scripts(file_path: Option<String>) -> anyhow::Result<()> {
create_dir_all(&*HUB_CACHE_DIR)?;
create_dir_all(&*BUN_BUNDLE_CACHE_DIR)?;
for path in paths.values() {
// Ensure the latest git sync script is always cached, regardless of hubPaths.json contents
let mut all_paths: Vec<String> = paths.into_values().collect();
let latest_git_sync = windmill_common::workspaces::LATEST_GIT_SYNC_SCRIPT_PATH.to_string();
if !all_paths.contains(&latest_git_sync) {
all_paths.push(latest_git_sync);
}
for path in &all_paths {
tracing::info!("Caching hub script at {path}");
let res = get_hub_script_content_and_requirements(Some(path), None).await?;
if res
@@ -517,6 +526,51 @@ fn print_help() {
println!("- At startup, Windmill logs currently set configuration keys for visibility.");
}
async fn resync_custom_instance_user_pwd_if_needed(db: &Pool<Postgres>) {
use windmill_common::utils::get_custom_pg_instance_password;
use windmill_common::{get_database_url, PgDatabase};
let user_pwd = match get_custom_pg_instance_password(db).await {
Ok(pwd) => pwd,
Err(_) => {
// Setting doesn't exist yet (fresh install or pre-migration), skip check
return;
}
};
let mut pg_creds = match get_database_url().await {
Ok(url) => match PgDatabase::parse_uri(&url.as_str().await) {
Ok(creds) => creds,
Err(e) => {
tracing::warn!("Failed to parse database URL for custom_instance_user check: {e}");
return;
}
},
Err(e) => {
tracing::warn!("Failed to get database URL for custom_instance_user check: {e}");
return;
}
};
pg_creds.user = Some("custom_instance_user".to_string());
pg_creds.password = Some(user_pwd);
match pg_creds.connect().await {
Ok(_) => {
tracing::info!("custom_instance_user password is in sync");
}
Err(e) => {
tracing::warn!("custom_instance_user password is out of sync ({e}), refreshing...");
if let Err(e) = windmill_api_settings::refresh_custom_instance_user_pwd_inner(db).await
{
tracing::error!("Failed to refresh custom_instance_user password: {e}");
} else {
tracing::info!("Successfully refreshed custom_instance_user password");
}
}
}
}
async fn windmill_main() -> anyhow::Result<()> {
let (killpill_tx, mut killpill_rx) = KillpillSender::new(2);
let mut monitor_killpill_rx = killpill_tx.subscribe();
@@ -647,6 +701,7 @@ async fn windmill_main() -> anyhow::Result<()> {
let mut num_workers = if mode == Mode::Server || mode == Mode::Indexer || mode == Mode::MCP {
0
} else if is_native_mode_from_env() {
NATIVE_MODE_RESOLVED.store(true, std::sync::atomic::Ordering::Relaxed);
println!("Native mode enabled: forcing NUM_WORKERS=8");
8
} else {
@@ -819,6 +874,30 @@ async fn windmill_main() -> anyhow::Result<()> {
}
}
// Resolve native mode early (before connect_db) so connection pool size accounts for it.
// native_mode can come from env OR from the DB worker group config.
if worker_mode && !is_native_mode_from_env() {
if let Some(db) = conn.as_sql() {
let native_from_db: bool = sqlx::query_scalar!(
"SELECT (config->>'native_mode')::boolean FROM config WHERE name = $1",
format!("worker__{}", *windmill_common::worker::WORKER_GROUP)
)
.fetch_optional(db)
.await
.ok()
.flatten()
.flatten()
.unwrap_or(false);
if native_from_db {
NATIVE_MODE_RESOLVED.store(true, std::sync::atomic::Ordering::Relaxed);
num_workers = 8;
tracing::info!(
"Native mode detected from worker config (early): forcing NUM_WORKERS=8"
);
}
}
}
let conn = if mode == Mode::Agent {
conn
} else {
@@ -831,6 +910,7 @@ async fn windmill_main() -> anyhow::Result<()> {
server_mode,
indexer_mode,
worker_mode,
num_workers,
#[cfg(feature = "private")]
killpill_rx.resubscribe(),
)
@@ -838,6 +918,11 @@ async fn windmill_main() -> anyhow::Result<()> {
// NOTE: Variable/resource cache initialization moved to API server in windmill-api
// Check if custom_instance_user password is in sync
if server_mode {
resync_custom_instance_user_pwd_if_needed(&db).await;
}
Connection::Sql(db)
};
@@ -930,16 +1015,6 @@ Windmill Community Edition {GIT_VERSION}
)
.await;
// native_mode may also be set via DB worker group config (not just env).
// NATIVE_MODE_RESOLVED is updated by load_worker_config during initial_load.
if worker_mode
&& !is_native_mode_from_env()
&& NATIVE_MODE_RESOLVED.load(std::sync::atomic::Ordering::Relaxed)
{
num_workers = 8;
tracing::info!("Native mode detected from worker config: forcing NUM_WORKERS=8");
}
monitor_db(
&conn,
&base_internal_url,
@@ -1614,6 +1689,9 @@ async fn process_notify_event(
}
TIMEOUT_WAIT_RESULT_SETTING => reload_timeout_wait_result_setting(conn).await,
RETENTION_PERIOD_SECS_SETTING => reload_retention_period_setting(conn).await,
AUDIT_LOG_RETENTION_DAYS_SETTING => {
reload_audit_log_retention_days_setting(conn).await
}
MONITOR_LOGS_ON_OBJECT_STORE_SETTING => {
reload_delete_logs_periodically_setting(conn).await
}
@@ -1829,7 +1907,7 @@ pub async fn run_workers(
tracing::info!(
"Starting {num_workers} workers and SLEEP_QUEUE={}ms",
*windmill_worker::SLEEP_QUEUE
windmill_worker::sleep_queue()
);
for i in 1..(num_workers + 1) {

View File

@@ -48,16 +48,17 @@ use windmill_common::{
error,
flow_status::{FlowStatus, FlowStatusModule},
global_settings::{
BASE_URL_SETTING, BUNFIG_INSTALL_SCOPES_SETTING, CRITICAL_ALERTS_ON_DB_OVERSIZE_SETTING,
CRITICAL_ALERTS_ON_TOKEN_EXPIRY_SETTING, CRITICAL_ALERT_MUTE_UI_SETTING,
CRITICAL_ERROR_CHANNELS_SETTING, DEFAULT_TAGS_PER_WORKSPACE_SETTING,
DEFAULT_TAGS_WORKSPACES_SETTING, EXPOSE_DEBUG_METRICS_SETTING, EXPOSE_METRICS_SETTING,
EXTRA_PIP_INDEX_URL_SETTING, HUB_API_SECRET_SETTING, HUB_BASE_URL_SETTING,
INSTANCE_PYTHON_VERSION_SETTING, JOB_DEFAULT_TIMEOUT_SECS_SETTING, JOB_ISOLATION_SETTING,
JWT_SECRET_SETTING, KEEP_JOB_DIR_SETTING, LICENSE_KEY_SETTING,
MONITOR_LOGS_ON_OBJECT_STORE_SETTING, NPMRC_SETTING, NPM_CONFIG_REGISTRY_SETTING,
NUGET_CONFIG_SETTING, OTEL_SETTING, OTEL_TRACING_PROXY_SETTING, PIP_INDEX_URL_SETTING,
POWERSHELL_REPO_PAT_SETTING, POWERSHELL_REPO_URL_SETTING, REQUEST_SIZE_LIMIT_SETTING,
AUDIT_LOG_RETENTION_DAYS_SETTING, BASE_URL_SETTING, BUNFIG_INSTALL_SCOPES_SETTING,
CRITICAL_ALERTS_ON_DB_OVERSIZE_SETTING, CRITICAL_ALERTS_ON_TOKEN_EXPIRY_SETTING,
CRITICAL_ALERT_MUTE_UI_SETTING, CRITICAL_ERROR_CHANNELS_SETTING,
DEFAULT_TAGS_PER_WORKSPACE_SETTING, DEFAULT_TAGS_WORKSPACES_SETTING,
EXPOSE_DEBUG_METRICS_SETTING, EXPOSE_METRICS_SETTING, EXTRA_PIP_INDEX_URL_SETTING,
HUB_API_SECRET_SETTING, HUB_BASE_URL_SETTING, INSTANCE_PYTHON_VERSION_SETTING,
JOB_DEFAULT_TIMEOUT_SECS_SETTING, JOB_ISOLATION_SETTING, JWT_SECRET_SETTING,
KEEP_JOB_DIR_SETTING, LICENSE_KEY_SETTING, MONITOR_LOGS_ON_OBJECT_STORE_SETTING,
NPMRC_SETTING, NPM_CONFIG_REGISTRY_SETTING, NUGET_CONFIG_SETTING, OTEL_SETTING,
OTEL_TRACING_PROXY_SETTING, PIP_INDEX_URL_SETTING, POWERSHELL_REPO_PAT_SETTING,
POWERSHELL_REPO_URL_SETTING, REQUEST_SIZE_LIMIT_SETTING,
REQUIRE_PREEXISTING_USER_FOR_OAUTH_SETTING, RETENTION_PERIOD_SECS_SETTING,
SAML_METADATA_SETTING, SCIM_TOKEN_SETTING, TIMEOUT_WAIT_RESULT_SETTING,
UV_INDEX_STRATEGY_SETTING,
@@ -77,9 +78,9 @@ use windmill_common::{
DEFAULT_TAGS_WORKSPACES, INDEXER_CONFIG, SCRIPT_TOKEN_EXPIRY, SMTP_CONFIG, WINDMILL_DIR,
WORKER_CONFIG, WORKER_GROUP,
},
KillpillSender, BASE_URL, CRITICAL_ALERTS_ON_DB_OVERSIZE, CRITICAL_ALERTS_ON_TOKEN_EXPIRY,
CRITICAL_ALERT_MUTE_UI_ENABLED, CRITICAL_ERROR_CHANNELS, DB, DEFAULT_HUB_BASE_URL,
HUB_BASE_URL, JOB_RETENTION_SECS, METRICS_DEBUG_ENABLED, METRICS_ENABLED,
KillpillSender, AUDIT_LOG_RETENTION_DAYS, BASE_URL, CRITICAL_ALERTS_ON_DB_OVERSIZE,
CRITICAL_ALERTS_ON_TOKEN_EXPIRY, CRITICAL_ALERT_MUTE_UI_ENABLED, CRITICAL_ERROR_CHANNELS, DB,
DEFAULT_HUB_BASE_URL, HUB_BASE_URL, JOB_RETENTION_SECS, METRICS_DEBUG_ENABLED, METRICS_ENABLED,
MONITOR_LOGS_ON_OBJECT_STORE, OTEL_LOGS_ENABLED, OTEL_METRICS_ENABLED, OTEL_TRACING_ENABLED,
SERVICE_LOG_RETENTION_SECS,
};
@@ -250,9 +251,8 @@ pub async fn initial_load(
.map(|x| x.tags.clone())
.unwrap_or_default();
// we only check from env as native_mode is not stored in the token
// NATIVE_MODE_RESOLVED is already set in main.rs during startup
let native_mode = windmill_common::worker::is_native_mode_from_env();
windmill_common::worker::NATIVE_MODE_RESOLVED
.store(native_mode, std::sync::atomic::Ordering::Relaxed);
*config = WorkerConfig {
worker_tags,
env_vars: load_env_vars(
@@ -323,9 +323,15 @@ pub async fn initial_load(
if server_mode {
reload_retention_period_setting(&conn).await;
reload_audit_log_retention_days_setting(&conn).await;
reload_request_size(&conn).await;
reload_saml_metadata_setting(&conn).await;
reload_scim_token_setting(&conn).await;
// Ensure audit partitions exist before any requests arrive
if let Some(db) = conn.as_sql() {
manage_audit_partitions(&db, audit_log_retention_days().await).await;
}
}
if worker_mode {
@@ -873,10 +879,19 @@ struct TokenRow {
workspace_id: Option<String>,
}
/// When updating this filter, also update:
/// - `register_token_expiry_notification` in windmill-api-auth/src/lib.rs
/// - `isUserToken` in frontend/src/lib/components/settings/TokensTable.svelte
fn is_user_token(label: Option<&str>) -> bool {
match label {
None => true,
Some(l) => l != "session" && !l.starts_with("ephemeral") && !l.starts_with("Ephemeral"),
Some(l) => {
l != "session"
&& !l.starts_with("ephemeral")
&& !l.starts_with("Ephemeral")
&& l != "debugger-token"
&& !l.starts_with("mcp-oauth-")
}
}
}
@@ -1027,12 +1042,10 @@ pub async fn delete_expired_items(db: &DB) -> () {
Err(e) => tracing::error!("Error deleting log file: {:?}", e),
}
#[cfg(not(feature = "enterprise"))]
let audit_retention_secs = 1 * 60 * 60 * 24 * 14;
#[cfg(feature = "enterprise")]
let audit_retention_secs = 1 * 60 * 60 * 24 * 365;
let audit_retention_days = audit_log_retention_days().await;
let audit_retention_secs: i64 = audit_retention_days * 60 * 60 * 24;
// Clean up old (non-partitioned) audit table — will eventually be empty and dropped
if let Err(e) = sqlx::query_scalar!(
"DELETE FROM audit WHERE timestamp <= now() - ($1::bigint::text || ' s')::interval",
audit_retention_secs,
@@ -1040,7 +1053,7 @@ pub async fn delete_expired_items(db: &DB) -> () {
.fetch_all(db)
.await
{
tracing::error!("Error deleting audit log on CE: {:?}", e);
tracing::error!("Error deleting audit log: {:?}", e);
}
if let Err(e) = sqlx::query_scalar!(
@@ -1565,6 +1578,22 @@ pub async fn reload_retention_period_setting(conn: &Connection) {
tracing::error!("Error reloading retention period: {:?}", e)
}
}
pub async fn reload_audit_log_retention_days_setting(conn: &Connection) {
if let Err(e) = reload_setting(
conn,
AUDIT_LOG_RETENTION_DAYS_SETTING,
"AUDIT_LOG_RETENTION_DAYS",
0, // 0 means use default: 365 for EE, 14 for CE
AUDIT_LOG_RETENTION_DAYS.clone(),
|x| x,
)
.await
{
tracing::error!("Error reloading audit log retention days: {:?}", e)
}
}
pub async fn reload_delete_logs_periodically_setting(conn: &Connection) {
if let Err(e) = reload_setting(
conn,
@@ -2182,6 +2211,15 @@ pub async fn monitor_db(
}
};
// run every hour (120 iterations * 30s = 3600s)
let manage_audit_partitions_f = async {
if server_mode && iteration.is_some() && iteration.as_ref().unwrap().should_run(120) {
if let Some(db) = conn.as_sql() {
manage_audit_partitions(&db, audit_log_retention_days().await).await;
}
}
};
join!(
expired_items_f,
zombie_jobs_f,
@@ -2204,6 +2242,7 @@ pub async fn monitor_db(
native_triggers_sync_f,
cleanup_notify_events_f,
check_expiring_tokens_f,
manage_audit_partitions_f,
);
}
@@ -2825,7 +2864,7 @@ async fn handle_zombie_jobs(db: &Pool<Postgres>, base_internal_url: &str, node_n
&windmill_queue::MiniCompletedJob::from(job),
memory_peak,
None,
error::Error::ExecutionErr(error_message),
error::Error::ExecutionErr(error_message.clone()),
matches!(error_kind, ErrorMessage::SameWorker), // unrecoverable if the job is a same worker zombie
Some(&same_worker_tx_never_used),
"",
@@ -2836,10 +2875,74 @@ async fn handle_zombie_jobs(db: &Pool<Postgres>, base_internal_url: &str, node_n
&mut windmill_common::bench::BenchmarkIter::new(),
)
.await;
// If handle_job_error failed (e.g. schedule push failure rolled back the tx),
// the job is still in the queue. Force-complete it to prevent infinite zombie loops.
if let Err(e) = force_complete_zombie_job(db, &job_id, &error_message).await {
tracing::error!("Failed to force-complete zombie job {}: {e:#}", job_id);
}
}
}
}
/// Force-complete a zombie job that handle_job_error failed to complete.
/// This is a minimal fallback: it inserts a failed completed job and deletes
/// from the queue in a single transaction, without schedule pushing or
/// error handler logic that could cause the completion to fail.
async fn force_complete_zombie_job(
db: &Pool<Postgres>,
job_id: &Uuid,
error_message: &str,
) -> error::Result<()> {
let still_queued = sqlx::query_scalar!(
"SELECT EXISTS(SELECT 1 FROM v2_job_queue WHERE id = $1)",
job_id
)
.fetch_one(db)
.await?
.unwrap_or(false);
if !still_queued {
return Ok(());
}
tracing::error!(
"Zombie job {job_id} was not completed by handle_job_error, force-completing it"
);
let error_value = serde_json::json!({
"message": error_message,
"name": "ExecutionErr",
});
let mut tx = db.begin().await?;
sqlx::query!(
"INSERT INTO v2_job_completed
(workspace_id, id, started_at, duration_ms, result, memory_peak, status, worker)
SELECT q.workspace_id, q.id, q.started_at,
COALESCE((EXTRACT('epoch' FROM now()) - EXTRACT('epoch' FROM COALESCE(q.started_at, now()))) * 1000, 0)::bigint,
$2::jsonb, r.memory_peak, 'failure'::job_status, q.worker
FROM v2_job_queue q
LEFT JOIN v2_job_runtime r ON r.id = q.id
WHERE q.id = $1
ON CONFLICT (id) DO UPDATE SET status = 'failure', result = $2::jsonb",
job_id,
error_value,
)
.execute(&mut *tx)
.await?;
sqlx::query!("DELETE FROM v2_job_queue WHERE id = $1", job_id)
.execute(&mut *tx)
.await?;
tx.commit().await?;
tracing::info!("Force-completed zombie job {job_id}");
Ok(())
}
async fn cleanup_concurrency_counters_orphaned_keys(db: &DB) -> error::Result<()> {
let result = sqlx::query!(
"
@@ -3368,3 +3471,72 @@ RETURNING job_id
}
Ok(())
}
async fn audit_log_retention_days() -> i64 {
let v = *AUDIT_LOG_RETENTION_DAYS.read().await;
if v > 0 {
v
} else if cfg!(feature = "enterprise") {
365
} else {
14
}
}
async fn manage_audit_partitions(db: &DB, retention_days: i64) {
let today = chrono::Utc::now().date_naive();
// Create partitions for today and the next 3 days
for days_ahead in 0..=3i64 {
let date = today + chrono::Duration::days(days_ahead);
let next_date = date + chrono::Duration::days(1);
let partition_name = format!("audit_{}", date.format("%Y%m%d"));
let quoted_name = format!("\"{}\"", partition_name.replace('"', "\"\""));
let sql = format!(
"CREATE TABLE IF NOT EXISTS {quoted_name} PARTITION OF audit_partitioned \
FOR VALUES FROM ('{date}') TO ('{next_date}')"
);
if let Err(e) = sqlx::query(&sql).execute(db).await {
if !e.to_string().contains("already exists") {
tracing::error!("Error creating audit partition {partition_name}: {e:?}");
}
}
}
// Drop expired partitions
let cutoff_date = today - chrono::Duration::days(retention_days);
let partitions = sqlx::query_scalar::<_, String>(
"SELECT c.relname::text \
FROM pg_inherits i \
JOIN pg_class c ON c.oid = i.inhrelid \
WHERE i.inhparent = 'audit_partitioned'::regclass",
)
.fetch_all(db)
.await;
match partitions {
Ok(partitions) => {
for partition_name in partitions {
if let Some(date_str) = partition_name.strip_prefix("audit_") {
if let Ok(date) = chrono::NaiveDate::parse_from_str(date_str, "%Y%m%d") {
if date < cutoff_date {
let quoted_name =
format!("\"{}\"", partition_name.replace('"', "\"\""));
let sql = format!("DROP TABLE IF EXISTS {quoted_name}");
match sqlx::query(&sql).execute(db).await {
Ok(_) => tracing::info!(
"Dropped expired audit partition {partition_name}"
),
Err(e) => tracing::error!(
"Error dropping audit partition {partition_name}: {e:?}"
),
}
}
}
}
}
}
Err(e) => tracing::error!("Error listing audit partitions: {e:?}"),
}
}

View File

@@ -109,7 +109,9 @@ job_result_stream_v2: job_id(uuid), workspace_id(text), stream(text), idx(int)
job_settings: job_id(uuid), runnable_settings(bigint)
job_stats: workspace_id(char), job_id(uuid), metric_id(char), metric_name(char), metric_kind(metric_kind), scalar_int(int), scalar_float(float), timestamps(ts), timeseries_int(int[]), timeseries_float(float[])
FK: (workspace_id) -> workspace(id)
kafka_trigger: path(char), kafka_resource_path(char), topics(char), group_id(char), script_path(char), is_flow(bool), workspace_id(char), edited_by(char), email(char), edited_at(ts), extra_perms(jsonb), server_id(char), last_server_ping(ts), error(text), error_handler_path(char), error_handler_args(jsonb), retry(jsonb), mode(trigger_mode), filters(jsonb[])
kafka_pending_commits: id(bigint), workspace_id(char), kafka_trigger_path(char), topic(char), partition(int), offset(bigint), created_at(ts)
FK: (workspace_id, kafka_trigger_path) -> kafka_trigger(workspace_id, path)
kafka_trigger: path(char), kafka_resource_path(char), topics(char), group_id(char), script_path(char), is_flow(bool), workspace_id(char), edited_by(char), email(char), edited_at(ts), extra_perms(jsonb), server_id(char), last_server_ping(ts), error(text), error_handler_path(char), error_handler_args(jsonb), retry(jsonb), mode(trigger_mode), filters(jsonb[]), auto_commit(bool)
log_file: hostname(char), log_ts(ts), ok_lines(bigint), err_lines(bigint), mode(log_mode), worker_group(char), file_path(char), json_fmt(bool)
magic_link: email(char), token(char), expiration(ts)
mcp_oauth_client: mcp_server_url(text), client_id(text), client_secret(text), client_secret_expires_at(ts), token_endpoint(text), created_at(ts)

View File

@@ -888,7 +888,7 @@ mod dedicated_worker_protocol {
if bundle_for_node {
// For Node.js: bundle to JavaScript first (like production's build_loader with LoaderMode::Node)
let wrapper = generate_dedicated_worker_wrapper(arg_names, "./main.js", None);
let wrapper = generate_dedicated_worker_wrapper(arg_names, "./main.js", None, None);
std::fs::write(dir.join("wrapper.mjs"), wrapper).unwrap();
// Use the exact same build_loader function as production
@@ -925,7 +925,7 @@ mod dedicated_worker_protocol {
output_path
} else {
// For Bun: use TypeScript directly (like production)
let wrapper = generate_dedicated_worker_wrapper(arg_names, "./main.ts", None);
let wrapper = generate_dedicated_worker_wrapper(arg_names, "./main.ts", None, None);
let wrapper_path = dir.join("wrapper.mjs");
std::fs::write(&wrapper_path, wrapper).unwrap();
wrapper_path

View File

@@ -30,6 +30,89 @@ export async function main(foo: string, bar: string) {
'',
'f/system/hello_with_preprocessor', 123413, 'deno', '');
INSERT INTO public.script(workspace_id, created_by, content, schema, summary, description, path, hash, language, lock) VALUES (
'test-workspace',
'system',
'
export async function preprocessor(foo: string, bar: string) {
return { foo: foo + "_preprocessed", bar: bar + "_preprocessed" };
}
export async function main(foo: string, bar: string) {
return "Hello " + foo + " " + bar;
}
',
'{"$schema":"https://json-schema.org/draft/2020-12/schema","properties":{"foo":{"default":null,"description":"","originalType":"string","type":"string"},"bar":{"default":null,"description":"","originalType":"string","type":"string"}},"required":["foo","bar"],"type":"object"}',
'',
'',
'f/system/hello_preprocessor_dedicated_bun', 123414, 'bun', E'{}\n//bun.lock\n{}');
INSERT INTO public.script(workspace_id, created_by, content, schema, summary, description, path, hash, language, lock) VALUES (
'test-workspace',
'system',
'
def preprocessor(foo: str, bar: str):
return {"foo": foo + "_preprocessed", "bar": bar + "_preprocessed"}
def main(foo: str, bar: str):
return "Hello " + foo + " " + bar
',
'{"$schema":"https://json-schema.org/draft/2020-12/schema","properties":{"foo":{"default":null,"description":"","originalType":"string","type":"string"},"bar":{"default":null,"description":"","originalType":"string","type":"string"}},"required":["foo","bar"],"type":"object"}',
'',
'',
'f/system/hello_preprocessor_dedicated_python', 123415, 'python3', '');
INSERT INTO public.script(workspace_id, created_by, content, schema, summary, description, path, hash, language, lock) VALUES (
'test-workspace',
'system',
'
export async function preprocessor(foo: string, bar: string) {
return { foo: foo + "_preprocessed", bar: bar + "_preprocessed" };
}
export async function main(foo: string, bar: string) {
return "Hello " + foo + " " + bar;
}
',
'{"$schema":"https://json-schema.org/draft/2020-12/schema","properties":{"foo":{"default":null,"description":"","originalType":"string","type":"string"},"bar":{"default":null,"description":"","originalType":"string","type":"string"}},"required":["foo","bar"],"type":"object"}',
'',
'',
'f/system/hello_preprocessor_dedicated_deno', 123416, 'deno', '');
INSERT INTO public.script(workspace_id, created_by, content, schema, summary, description, path, hash, language, lock) VALUES (
'test-workspace',
'system',
'//native
export async function preprocessor(foo: string, bar: string) {
return { foo: foo + "_preprocessed", bar: bar + "_preprocessed" };
}
export async function main(foo: string, bar: string) {
return "Hello " + foo + " " + bar;
}
',
'{"$schema":"https://json-schema.org/draft/2020-12/schema","properties":{"foo":{"default":null,"description":"","originalType":"string","type":"string"},"bar":{"default":null,"description":"","originalType":"string","type":"string"}},"required":["foo","bar"],"type":"object"}',
'',
'',
'f/system/hello_preprocessor_bunnative', 123417, 'bunnative', E'{}\n//bun.lock\n{}');
INSERT INTO public.script(workspace_id, created_by, content, schema, summary, description, path, hash, language, lock) VALUES (
'test-workspace',
'system',
'//native
export async function preprocessor(foo: string, bar: string) {
return { foo: foo + "_preprocessed", bar: bar + "_preprocessed" };
}
export async function main(foo: string, bar: string) {
return "Hello " + foo + " " + bar;
}
',
'{"$schema":"https://json-schema.org/draft/2020-12/schema","properties":{"foo":{"default":null,"description":"","originalType":"string","type":"string"},"bar":{"default":null,"description":"","originalType":"string","type":"string"}},"required":["foo","bar"],"type":"object"}',
'',
'',
'f/system/hello_preprocessor_dedicated_bunnative', 123418, 'bunnative', E'{}\n//bun.lock\n{}');
INSERT INTO public.flow(workspace_id, summary, description, path, versions, schema, value, edited_by) VALUES (
'test-workspace',
'',

View File

@@ -1,16 +1,16 @@
mod job_payload {
use serde_json::json;
use sqlx::{Pool, Postgres};
use windmill_common::flow_status::RestartedFrom;
use windmill_common::flows::{FlowModule, FlowModuleValue, FlowValue};
use windmill_common::jobs::JobPayload;
use windmill_common::scripts::{ScriptHash, ScriptLang};
use windmill_common::flow_status::RestartedFrom;
use windmill_test_utils::*;
use windmill_common::min_version::{
MIN_VERSION, MIN_VERSION_IS_AT_LEAST_1_427, MIN_VERSION_IS_AT_LEAST_1_432,
MIN_VERSION_IS_AT_LEAST_1_440,
};
use windmill_test_utils::*;
pub async fn initialize_tracing() {
use std::sync::Once;
@@ -305,7 +305,7 @@ mod job_payload {
path: "f/system/hello_with_nodes_flow".to_string(),
dedicated_worker: None,
version: 1443253234253454,
debouncing_settings: Default::default(),
debouncing_settings: Default::default(),
})
.run_until_complete(&db, false, port)
.await
@@ -768,4 +768,256 @@ mod job_payload {
.await;
Ok(())
}
#[sqlx::test(fixtures("base", "hello"))]
async fn test_dedicated_worker_preprocessor_bun(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let test = || async {
let db = &db;
let job = RunJob::from(JobPayload::ScriptHash {
hash: ScriptHash(123414),
path: "f/system/hello_preprocessor_dedicated_bun".to_string(),
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
language: ScriptLang::Bun,
priority: None,
apply_preprocessor: true,
concurrency_settings:
windmill_common::runnable_settings::ConcurrencySettings::default(),
debouncing_settings:
windmill_common::runnable_settings::DebouncingSettings::default(),
})
.arg("foo", json!("hello"))
.arg("bar", json!("world"))
.run_until_complete_with(db, false, port, |id| async move {
let job = sqlx::query!("SELECT preprocessed FROM v2_job WHERE id = $1", id)
.fetch_one(db)
.await
.unwrap();
assert_eq!(job.preprocessed, Some(false));
})
.await;
let args = job.args.as_ref().unwrap();
assert_eq!(args.get("foo"), Some(&json!("hello_preprocessed")));
assert_eq!(args.get("bar"), Some(&json!("world_preprocessed")));
assert_eq!(
job.json_result().unwrap(),
json!("Hello hello_preprocessed world_preprocessed")
);
let job = sqlx::query!("SELECT preprocessed FROM v2_job WHERE id = $1", job.id)
.fetch_one(db)
.await
.unwrap();
assert_eq!(job.preprocessed, Some(true));
};
test_for_versions(VERSION_FLAGS.iter().copied(), test).await;
Ok(())
}
#[sqlx::test(fixtures("base", "hello"))]
async fn test_dedicated_worker_preprocessor_python(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let test = || async {
let db = &db;
let job = RunJob::from(JobPayload::ScriptHash {
hash: ScriptHash(123415),
path: "f/system/hello_preprocessor_dedicated_python".to_string(),
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
language: ScriptLang::Python3,
priority: None,
apply_preprocessor: true,
concurrency_settings:
windmill_common::runnable_settings::ConcurrencySettings::default(),
debouncing_settings:
windmill_common::runnable_settings::DebouncingSettings::default(),
})
.arg("foo", json!("hello"))
.arg("bar", json!("world"))
.run_until_complete_with(db, false, port, |id| async move {
let job = sqlx::query!("SELECT preprocessed FROM v2_job WHERE id = $1", id)
.fetch_one(db)
.await
.unwrap();
assert_eq!(job.preprocessed, Some(false));
})
.await;
let args = job.args.as_ref().unwrap();
assert_eq!(args.get("foo"), Some(&json!("hello_preprocessed")));
assert_eq!(args.get("bar"), Some(&json!("world_preprocessed")));
assert_eq!(
job.json_result().unwrap(),
json!("Hello hello_preprocessed world_preprocessed")
);
let job = sqlx::query!("SELECT preprocessed FROM v2_job WHERE id = $1", job.id)
.fetch_one(db)
.await
.unwrap();
assert_eq!(job.preprocessed, Some(true));
};
test_for_versions(VERSION_FLAGS.iter().copied(), test).await;
Ok(())
}
#[sqlx::test(fixtures("base", "hello"))]
async fn test_dedicated_worker_preprocessor_deno(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let test = || async {
let db = &db;
let job = RunJob::from(JobPayload::ScriptHash {
hash: ScriptHash(123416),
path: "f/system/hello_preprocessor_dedicated_deno".to_string(),
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
language: ScriptLang::Deno,
priority: None,
apply_preprocessor: true,
concurrency_settings:
windmill_common::runnable_settings::ConcurrencySettings::default(),
debouncing_settings:
windmill_common::runnable_settings::DebouncingSettings::default(),
})
.arg("foo", json!("hello"))
.arg("bar", json!("world"))
.run_until_complete_with(db, false, port, |id| async move {
let job = sqlx::query!("SELECT preprocessed FROM v2_job WHERE id = $1", id)
.fetch_one(db)
.await
.unwrap();
assert_eq!(job.preprocessed, Some(false));
})
.await;
let args = job.args.as_ref().unwrap();
assert_eq!(args.get("foo"), Some(&json!("hello_preprocessed")));
assert_eq!(args.get("bar"), Some(&json!("world_preprocessed")));
assert_eq!(
job.json_result().unwrap(),
json!("Hello hello_preprocessed world_preprocessed")
);
let job = sqlx::query!("SELECT preprocessed FROM v2_job WHERE id = $1", job.id)
.fetch_one(db)
.await
.unwrap();
assert_eq!(job.preprocessed, Some(true));
};
test_for_versions(VERSION_FLAGS.iter().copied(), test).await;
Ok(())
}
#[sqlx::test(fixtures("base", "hello"))]
async fn test_bunnative_preprocessor(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let test = || async {
let db = &db;
let job = RunJob::from(JobPayload::ScriptHash {
hash: ScriptHash(123417),
path: "f/system/hello_preprocessor_bunnative".to_string(),
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
language: ScriptLang::Bunnative,
priority: None,
apply_preprocessor: true,
concurrency_settings:
windmill_common::runnable_settings::ConcurrencySettings::default(),
debouncing_settings:
windmill_common::runnable_settings::DebouncingSettings::default(),
})
.arg("foo", json!("hello"))
.arg("bar", json!("world"))
.run_until_complete_with(db, false, port, |id| async move {
let job = sqlx::query!("SELECT preprocessed FROM v2_job WHERE id = $1", id)
.fetch_one(db)
.await
.unwrap();
assert_eq!(job.preprocessed, Some(false));
})
.await;
let args = job.args.as_ref().unwrap();
assert_eq!(args.get("foo"), Some(&json!("hello_preprocessed")));
assert_eq!(args.get("bar"), Some(&json!("world_preprocessed")));
assert_eq!(
job.json_result().unwrap(),
json!("Hello hello_preprocessed world_preprocessed")
);
let job = sqlx::query!("SELECT preprocessed FROM v2_job WHERE id = $1", job.id)
.fetch_one(db)
.await
.unwrap();
assert_eq!(job.preprocessed, Some(true));
};
test_for_versions(VERSION_FLAGS.iter().copied(), test).await;
Ok(())
}
#[sqlx::test(fixtures("base", "hello"))]
async fn test_dedicated_worker_preprocessor_bunnative(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let test = || async {
let db = &db;
let job = RunJob::from(JobPayload::ScriptHash {
hash: ScriptHash(123418),
path: "f/system/hello_preprocessor_dedicated_bunnative".to_string(),
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
language: ScriptLang::Bunnative,
priority: None,
apply_preprocessor: true,
concurrency_settings:
windmill_common::runnable_settings::ConcurrencySettings::default(),
debouncing_settings:
windmill_common::runnable_settings::DebouncingSettings::default(),
})
.arg("foo", json!("hello"))
.arg("bar", json!("world"))
.run_until_complete_with(db, false, port, |id| async move {
let job = sqlx::query!("SELECT preprocessed FROM v2_job WHERE id = $1", id)
.fetch_one(db)
.await
.unwrap();
assert_eq!(job.preprocessed, Some(false));
})
.await;
let args = job.args.as_ref().unwrap();
assert_eq!(args.get("foo"), Some(&json!("hello_preprocessed")));
assert_eq!(args.get("bar"), Some(&json!("world_preprocessed")));
assert_eq!(
job.json_result().unwrap(),
json!("Hello hello_preprocessed world_preprocessed")
);
let job = sqlx::query!("SELECT preprocessed FROM v2_job WHERE id = $1", job.id)
.fetch_one(db)
.await
.unwrap();
assert_eq!(job.preprocessed, Some(true));
};
test_for_versions(VERSION_FLAGS.iter().copied(), test).await;
Ok(())
}
}

View File

@@ -80,8 +80,13 @@ mod prewarmed_isolate_tests {
let mut results = Vec::new();
for job_args in &jobs {
let mut isolate =
PrewarmedIsolate::spawn("".to_string(), js.clone(), ann.clone(), arg_names.clone());
let mut isolate = PrewarmedIsolate::spawn(
"".to_string(),
js.clone(),
ann.clone(),
arg_names.clone(),
None,
);
isolate.wait_ready().await.expect("isolate failed to warm");
let args = serde_json::to_string(job_args).unwrap();
@@ -180,8 +185,13 @@ export function main(n: number): number {
let ann = default_annotation();
// Pre-warm first isolate
let mut warm =
PrewarmedIsolate::spawn("".to_string(), js.clone(), ann.clone(), arg_names.clone());
let mut warm = PrewarmedIsolate::spawn(
"".to_string(),
js.clone(),
ann.clone(),
arg_names.clone(),
None,
);
warm.wait_ready()
.await
.expect("first isolate failed to warm");
@@ -193,8 +203,13 @@ export function main(n: number): number {
let executing = warm.start_execution(args);
// Pipeline: start pre-warming next isolate while current one runs
warm =
PrewarmedIsolate::spawn("".to_string(), js.clone(), ann.clone(), arg_names.clone());
warm = PrewarmedIsolate::spawn(
"".to_string(),
js.clone(),
ann.clone(),
arg_names.clone(),
None,
);
let prewarmed_result = executing.wait().await.expect("isolate execution failed");
match prewarmed_result.result {

View File

@@ -581,6 +581,9 @@ pub async fn create_token_internal(
}
/// Insert a pending expiry notification row for user tokens that have an expiration.
/// When updating this filter, also update:
/// - `is_user_token` in src/monitor.rs
/// - `isUserToken` in frontend/src/lib/components/settings/TokensTable.svelte
pub async fn register_token_expiry_notification(
tx: &mut sqlx::PgConnection,
token: &str,
@@ -589,7 +592,12 @@ pub async fn register_token_expiry_notification(
) {
let Some(expiration) = expiration else { return };
if label == Some("session")
|| label.is_some_and(|l| l.starts_with("ephemeral") || l.starts_with("Ephemeral"))
|| label.is_some_and(|l| {
l.starts_with("ephemeral")
|| l.starts_with("Ephemeral")
|| l == "debugger-token"
|| l.starts_with("mcp-oauth-")
})
{
return;
}

View File

@@ -129,11 +129,12 @@ async fn update_config(
#[cfg(not(feature = "enterprise"))]
let config = if name.starts_with("worker__") {
// In CE, only allow setting worker_tags, cache_clear, and init_bash
// In CE, only allow setting worker_tags, cache_clear, init_bash, and native_mode
serde_json::json!({
"worker_tags": config.get("worker_tags"),
"cache_clear": config.get("cache_clear"),
"init_bash": config.get("init_bash")
"init_bash": config.get("init_bash"),
"native_mode": config.get("native_mode")
})
} else {
config

View File

@@ -10,8 +10,8 @@ path = "src/lib.rs"
[features]
default = []
private = ["windmill-test-utils/private", "dep:aws-config", "dep:aws-credential-types", "dep:aws-sdk-sqs"]
enterprise = ["windmill-test-utils/enterprise", "dep:base64"]
private = ["windmill-test-utils/private", "dep:aws-config", "dep:aws-credential-types", "dep:aws-sdk-sqs", "windmill-git-sync/private"]
enterprise = ["windmill-test-utils/enterprise", "dep:base64", "windmill-git-sync/enterprise"]
deno_core = ["windmill-test-utils/deno_core"]
mcp = []
run_inline = ["dep:windmill-worker", "windmill-test-utils/run_inline", "windmill-test-utils/duckdb"]
@@ -22,6 +22,7 @@ windmill-api-client.workspace = true
windmill-common = { workspace = true, default-features = false }
windmill-native-triggers = { workspace = true, features = ["native_trigger"] }
windmill-api-auth.workspace = true
windmill-git-sync.workspace = true
windmill-worker = { workspace = true, optional = true }
sqlx.workspace = true
serde_json.workspace = true

View File

@@ -251,7 +251,8 @@ async fn test_websocket_e2e(db: Pool<Postgres>) -> anyhow::Result<()> {
"test-workspace",
"test-user",
"test@windmill.dev",
&[json!({"type": "RawMessage", "content": "hello from e2e test"})] as &[serde_json::Value],
&[json!({"type": "RawMessage", "content": "hello from e2e test"})]
as &[serde_json::Value],
)
.execute(&db)
.await?;
@@ -302,9 +303,11 @@ async fn test_postgres_e2e(db: Pool<Postgres>) -> anyhow::Result<()> {
sqlx::query("CREATE TABLE test_trigger_table (id serial PRIMARY KEY, data text)")
.execute(&db)
.await?;
sqlx::query(&format!("CREATE PUBLICATION {pub_name} FOR TABLE test_trigger_table"))
.execute(&db)
.await?;
sqlx::query(&format!(
"CREATE PUBLICATION {pub_name} FOR TABLE test_trigger_table"
))
.execute(&db)
.await?;
sqlx::query(&format!(
"SELECT pg_create_logical_replication_slot('{slot_name}', 'pgoutput')"
))
@@ -313,10 +316,9 @@ async fn test_postgres_e2e(db: Pool<Postgres>) -> anyhow::Result<()> {
// Extract the test DB name from the pool so the resource points here,
// not at the main windmill database.
let test_db_name: String =
sqlx::query_scalar("SELECT current_database()")
.fetch_one(&db)
.await?;
let test_db_name: String = sqlx::query_scalar("SELECT current_database()")
.fetch_one(&db)
.await?;
insert_resource(
&db,

View File

@@ -214,12 +214,9 @@ async fn test_capture_delete(db: Pool<Postgres>) -> anyhow::Result<()> {
.execute(&db)
.await?;
let count = sqlx::query_scalar!(
"SELECT COUNT(*) FROM capture WHERE id = $1",
id,
)
.fetch_one(&db)
.await?;
let count = sqlx::query_scalar!("SELECT COUNT(*) FROM capture WHERE id = $1", id,)
.fetch_one(&db)
.await?;
assert_eq!(count, Some(0));
@@ -385,7 +382,10 @@ async fn test_capture_api_list_captures(db: Pool<Postgres>) -> anyhow::Result<()
.send()
.await?;
assert!(response.status().is_success(), "list captures should succeed");
assert!(
response.status().is_success(),
"list captures should succeed"
);
let captures: Vec<CaptureResponse> = response.json().await?;
assert_eq!(captures.len(), 3);
@@ -480,12 +480,9 @@ async fn test_capture_api_delete(db: Pool<Postgres>) -> anyhow::Result<()> {
assert!(response.status().is_success(), "delete should succeed");
let count = sqlx::query_scalar!(
"SELECT COUNT(*) FROM capture WHERE id = $1",
id,
)
.fetch_one(&db)
.await?;
let count = sqlx::query_scalar!("SELECT COUNT(*) FROM capture WHERE id = $1", id,)
.fetch_one(&db)
.await?;
assert_eq!(count, Some(0));
@@ -933,7 +930,8 @@ async fn test_kafka_trigger_insert(db: Pool<Postgres>) -> anyhow::Result<()> {
let trigger = sqlx::query!(
r#"
SELECT kafka_resource_path, topics, group_id, mode AS "mode: String"
SELECT kafka_resource_path, topics, group_id, mode AS "mode: String",
auto_offset_reset, auto_commit, reset_offset
FROM kafka_trigger
WHERE workspace_id = $1 AND path = $2
"#,
@@ -947,6 +945,50 @@ async fn test_kafka_trigger_insert(db: Pool<Postgres>) -> anyhow::Result<()> {
assert_eq!(trigger.topics, vec!["topic-a", "topic-b"]);
assert_eq!(trigger.group_id, "my-consumer-group");
assert_eq!(trigger.mode, "enabled");
assert_eq!(trigger.auto_offset_reset, "latest");
assert_eq!(trigger.auto_commit, true);
assert_eq!(trigger.reset_offset, false);
Ok(())
}
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_kafka_trigger_insert_auto_commit_disabled(db: Pool<Postgres>) -> anyhow::Result<()> {
sqlx::query!(
r#"
INSERT INTO kafka_trigger (
path, kafka_resource_path, topics, group_id, script_path,
is_flow, workspace_id, edited_by, email, auto_commit
)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)
"#,
"f/test/kafka_trigger_no_commit",
"u/admin/kafka_resource",
&["topic-c"] as &[&str],
"my-consumer-group-2",
"f/test/kafka_handler",
false,
"test-workspace",
"test-user",
"test@windmill.dev",
false,
)
.execute(&db)
.await?;
let trigger = sqlx::query!(
r#"
SELECT auto_commit
FROM kafka_trigger
WHERE workspace_id = $1 AND path = $2
"#,
"test-workspace",
"f/test/kafka_trigger_no_commit",
)
.fetch_one(&db)
.await?;
assert_eq!(trigger.auto_commit, false);
Ok(())
}

View File

@@ -0,0 +1,452 @@
/*!
* Integration tests for workspace dependencies git sync.
*
* These tests verify that creating, archiving, and deleting workspace dependencies
* triggers deployment callback jobs with the correct arguments for git sync.
*
* Run with enterprise features:
* ```bash
* cargo test --test workspace_dependencies_git_sync --features enterprise,private
* ```
*/
use serde_json::json;
use sqlx::{Pool, Postgres};
use std::time::Duration;
use windmill_test_utils::*;
/// Row shape for querying deployment callback jobs from v2_job_queue
#[derive(Debug)]
#[allow(dead_code)]
struct DeploymentCallbackJob {
id: uuid::Uuid,
runnable_path: Option<String>,
args: Option<serde_json::Value>,
kind: String,
}
/// Poll for deployment callback jobs in the queue for a given script path
async fn get_deployment_callback_jobs(
db: &Pool<Postgres>,
script_path: &str,
timeout: Duration,
) -> anyhow::Result<Vec<DeploymentCallbackJob>> {
let deadline = tokio::time::Instant::now() + timeout;
loop {
let rows = sqlx::query_as!(
DeploymentCallbackJob,
r#"
SELECT j.id, j.runnable_path, j.args, j.kind::text AS "kind!"
FROM v2_job j
JOIN v2_job_queue q ON j.id = q.id
WHERE j.runnable_path = $1
AND j.kind = 'deploymentcallback'
ORDER BY j.created_at DESC
"#,
script_path,
)
.fetch_all(db)
.await?;
if !rows.is_empty() {
return Ok(rows);
}
if tokio::time::Instant::now() >= deadline {
// Return empty if timeout - caller will handle assertion
return Ok(vec![]);
}
tokio::time::sleep(Duration::from_millis(100)).await;
}
}
/// Configure git sync for the test workspace with workspace dependencies enabled
async fn setup_git_sync_config(db: &Pool<Postgres>, sync_script_path: &str) -> anyhow::Result<()> {
let git_sync_config = json!({
"include_type": ["workspacedependencies"],
"include_path": ["**"],
"repositories": [{
"script_path": sync_script_path,
"git_repo_resource_path": "$res:u/test-user/test_git_repo",
"use_individual_branch": false,
"group_by_folder": false
}]
});
sqlx::query!(
"UPDATE workspace_settings SET git_sync = $1 WHERE workspace_id = $2",
git_sync_config,
"test-workspace"
)
.execute(db)
.await?;
Ok(())
}
/// Create a git repository resource for testing
async fn create_git_repo_resource(db: &Pool<Postgres>) -> anyhow::Result<()> {
sqlx::query(
r#"
INSERT INTO resource (workspace_id, path, value, resource_type, extra_perms, created_by)
VALUES ('test-workspace', 'u/test-user/test_git_repo', $1::jsonb, 'git_repository', '{}'::jsonb, 'test-user')
ON CONFLICT (workspace_id, path) DO NOTHING
"#,
)
.bind(json!({
"url": "https://github.com/test/test.git",
"branch": "main",
"token": "test-token"
}))
.execute(db)
.await?;
Ok(())
}
/// Create a dummy sync script for testing (with version >= 28103 for debouncing support)
async fn create_sync_script(db: &Pool<Postgres>, path: &str) -> anyhow::Result<i64> {
let hash: i64 = rand::random::<i64>().unsigned_abs() as i64;
sqlx::query(
r#"
INSERT INTO script (workspace_id, hash, path, summary, description, content,
created_by, language, kind, lock)
VALUES ('test-workspace', $1, $2, 'sync script', '',
'export function main(items: any[]) { return { synced: items.length }; }',
'test-user', 'bun', 'script', '')
"#,
)
.bind(hash)
.bind(path)
.execute(db)
.await?;
Ok(hash)
}
/// Create a folder for the versioned script path
async fn create_folder(db: &Pool<Postgres>, name: &str) -> anyhow::Result<()> {
sqlx::query(
r#"
INSERT INTO folder (workspace_id, name, display_name, owners, extra_perms, created_by)
VALUES ('test-workspace', $1, $1, ARRAY['u/test-user'], '{}'::jsonb, 'test-user')
ON CONFLICT (workspace_id, name) DO NOTHING
"#,
)
.bind(name)
.execute(db)
.await?;
Ok(())
}
// ============================================================================
// Tests
// ============================================================================
/// Test that creating a workspace dependency triggers a git sync deployment callback
/// with the correct path_type and path arguments.
#[cfg(all(feature = "enterprise", feature = "private"))]
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_create_workspace_dependencies_triggers_git_sync(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
initialize_tracing().await;
// Setup: Create folder, git repo resource, sync script, and configure git sync
create_folder(&db, "28103").await?;
create_git_repo_resource(&db).await?;
let sync_script_path = "f/28103/test_sync_script";
create_sync_script(&db, sync_script_path).await?;
setup_git_sync_config(&db, sync_script_path).await?;
// Start API server
let (client, _port, _server) = init_client(db.clone()).await;
// Create workspace dependency via API
let response = client
.client()
.post(format!(
"{}/w/test-workspace/workspace_dependencies/create",
client.baseurl()
))
.json(&json!({
"workspace_id": "test-workspace",
"language": "python3",
"name": "test-deps",
"content": "requests==2.28.0\nnumpy==1.24.0"
}))
.send()
.await?;
assert!(
response.status().is_success(),
"Failed to create workspace dependency: {:?}",
response.text().await
);
// Wait for deployment callback job to be created
tokio::time::sleep(Duration::from_millis(500)).await;
// Query for deployment callback jobs
let jobs = get_deployment_callback_jobs(&db, sync_script_path, Duration::from_secs(5)).await?;
assert!(
!jobs.is_empty(),
"Expected at least one deployment callback job to be created"
);
// Verify the job arguments
let job = &jobs[0];
let args = job.args.as_ref().expect("Job should have args");
// Check that path_type is "workspace_dependencies" (or check items array)
// The exact structure depends on whether debouncing is enabled
if let Some(items) = args.get("items") {
// Debounced format: items is an array
let items_arr = items.as_array().expect("items should be an array");
assert!(!items_arr.is_empty(), "items array should not be empty");
let item = &items_arr[0];
assert_eq!(
item.get("path_type").and_then(|v| v.as_str()),
Some("workspace_dependencies"),
"path_type should be 'workspace_dependencies'"
);
// Path should be "dependencies/test-deps.requirements.in" or similar
let path = item.get("path").and_then(|v| v.as_str()).unwrap_or("");
assert!(
path.contains("dependencies") || path.contains("requirements"),
"path should contain dependencies or requirements: got {}",
path
);
} else if let Some(path_type) = args.get("path_type") {
// Non-debounced format: path_type is a direct field
assert_eq!(
path_type.as_str(),
Some("workspace_dependencies"),
"path_type should be 'workspace_dependencies'"
);
} else {
panic!(
"Job args should contain either 'items' array or 'path_type' field: {:?}",
args
);
}
Ok(())
}
/// Test that archiving a workspace dependency triggers a git sync deployment callback
#[cfg(all(feature = "enterprise", feature = "private"))]
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_archive_workspace_dependencies_triggers_git_sync(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
initialize_tracing().await;
// Setup
create_folder(&db, "28103").await?;
create_git_repo_resource(&db).await?;
let sync_script_path = "f/28103/test_sync_script_archive";
create_sync_script(&db, sync_script_path).await?;
setup_git_sync_config(&db, sync_script_path).await?;
let (client, _port, _server) = init_client(db.clone()).await;
// First create a workspace dependency
let create_response = client
.client()
.post(format!(
"{}/w/test-workspace/workspace_dependencies/create",
client.baseurl()
))
.json(&json!({
"workspace_id": "test-workspace",
"language": "python3",
"name": "archive-test-deps",
"content": "flask==2.0.0"
}))
.send()
.await?;
assert!(create_response.status().is_success());
// Wait a bit for the create job to be processed
tokio::time::sleep(Duration::from_millis(300)).await;
// Now archive it
let archive_response = client
.client()
.post(format!(
"{}/w/test-workspace/workspace_dependencies/archive/python3?name=archive-test-deps",
client.baseurl()
))
.send()
.await?;
assert!(
archive_response.status().is_success(),
"Failed to archive workspace dependency: {:?}",
archive_response.text().await
);
// Wait for deployment callback job
tokio::time::sleep(Duration::from_millis(500)).await;
// Verify at least one deployment callback job exists
// (create test already validates create triggers git sync; this test validates archive does too)
let jobs = get_deployment_callback_jobs(&db, sync_script_path, Duration::from_secs(5)).await?;
assert!(
!jobs.is_empty(),
"Expected at least one deployment callback job after archive"
);
Ok(())
}
/// Test that workspace dependencies are NOT synced when workspacedependencies type is excluded
#[cfg(all(feature = "enterprise", feature = "private"))]
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_workspace_dependencies_respects_include_type_filter(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
initialize_tracing().await;
// Setup with git sync that EXCLUDES workspacedependencies
create_folder(&db, "28103").await?;
create_git_repo_resource(&db).await?;
let sync_script_path = "f/28103/test_sync_script_filter";
create_sync_script(&db, sync_script_path).await?;
// Configure git sync to only include scripts (not workspace dependencies)
let git_sync_config = json!({
"include_type": ["script"], // Note: workspacedependencies is NOT included
"include_path": ["**"],
"repositories": [{
"script_path": sync_script_path,
"git_repo_resource_path": "$res:u/test-user/test_git_repo",
"use_individual_branch": false,
"group_by_folder": false
}]
});
sqlx::query!(
"UPDATE workspace_settings SET git_sync = $1 WHERE workspace_id = $2",
git_sync_config,
"test-workspace"
)
.execute(&db)
.await?;
let (client, _port, _server) = init_client(db.clone()).await;
// Create workspace dependency
let response = client
.client()
.post(format!(
"{}/w/test-workspace/workspace_dependencies/create",
client.baseurl()
))
.json(&json!({
"workspace_id": "test-workspace",
"language": "python3",
"name": "filtered-deps",
"content": "django==4.0.0"
}))
.send()
.await?;
assert!(response.status().is_success());
// Wait a bit
tokio::time::sleep(Duration::from_secs(1)).await;
// Should NOT have any deployment callback jobs because workspacedependencies is filtered out
let jobs =
get_deployment_callback_jobs(&db, sync_script_path, Duration::from_millis(500)).await?;
assert!(
jobs.is_empty(),
"Expected NO deployment callback jobs when workspacedependencies is not in include_type, got {}",
jobs.len()
);
Ok(())
}
/// Test that the commit message is correctly generated for workspace dependencies
#[cfg(all(feature = "enterprise", feature = "private"))]
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_workspace_dependencies_commit_message(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
// Setup
create_folder(&db, "28103").await?;
create_git_repo_resource(&db).await?;
let sync_script_path = "f/28103/test_sync_script_msg";
create_sync_script(&db, sync_script_path).await?;
setup_git_sync_config(&db, sync_script_path).await?;
let (client, _port, _server) = init_client(db.clone()).await;
// Create workspace dependency
let response = client
.client()
.post(format!(
"{}/w/test-workspace/workspace_dependencies/create",
client.baseurl()
))
.json(&json!({
"workspace_id": "test-workspace",
"language": "bun",
"name": null, // unnamed/default dependency
"content": "lodash: ^4.17.21"
}))
.send()
.await?;
assert!(response.status().is_success());
tokio::time::sleep(Duration::from_millis(500)).await;
let jobs = get_deployment_callback_jobs(&db, sync_script_path, Duration::from_secs(5)).await?;
assert!(!jobs.is_empty());
let job = &jobs[0];
let args = job.args.as_ref().expect("Job should have args");
// Check commit message format
if let Some(items) = args.get("items") {
let items_arr = items.as_array().expect("items should be an array");
if !items_arr.is_empty() {
let commit_msg = items_arr[0]
.get("commit_msg")
.and_then(|v| v.as_str())
.unwrap_or("");
assert!(
commit_msg.contains("[WM]"),
"Commit message should contain '[WM]' prefix: {}",
commit_msg
);
assert!(
commit_msg.to_lowercase().contains("workspace")
|| commit_msg.to_lowercase().contains("dependency")
|| commit_msg.to_lowercase().contains("deployed"),
"Commit message should mention workspace dependency or deployed: {}",
commit_msg
);
}
} else if let Some(commit_msg) = args.get("commit_msg").and_then(|v| v.as_str()) {
assert!(
commit_msg.contains("[WM]"),
"Commit message should contain '[WM]' prefix: {}",
commit_msg
);
}
Ok(())
}

Some files were not shown because too many files have changed in this diff Show More