Compare commits

..

55 Commits

Author SHA1 Message Date
Ruben Fiszel
7fb394ad65 feat: feature-gate heavy deps for faster default builds
Reduce default build from 761 to 609 crates (20% reduction) by making
heavy dependencies optional behind their appropriate feature flags.

Feature-gated deps:
- windmill-worker: hudsucker, rcgen, prost, opentelemetry-proto (EE otel proxy)
- windmill-common: aws-config, aws-credential-types, aws-smithy-types,
  systemstat, globset
- windmill-api: aws-sigv4, aws-sdk-config, windmill-parser-py-imports,
  windmill-autoscaling
- windmill-autoscaling: kube, k8s-openapi (EE only)
- windmill-parser-py-imports: removed unused malachite deps
- Root: removed 12 unused direct dependencies

Incremental compilation with additional local optimizations (mold linker,
split-debuginfo, line-tables-only) brings windmill-api rebuilds from
~5.6s to ~4.7s (16% faster).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 14:32:02 +00:00
Ruben Fiszel
727bd21640 fix: make private registries settings password in the instance settings 2026-02-04 11:51:10 +00:00
hugocasa
d16df49f86 add Claude Code hooks for formatting and notifications (#7767)
* feat: add Claude Code hooks for formatting and notifications

- Add PostToolUse hooks to auto-format files after Edit/Write:
  - format-frontend.sh: runs prettier on frontend files
  - format-backend.sh: runs rustfmt on backend Rust files
- Add Notification hook to alert user when Claude needs input
- Add edition=2021 to rustfmt.toml for proper parsing
- Update .gitignore for symlinked cache directories
- Add additional bash permissions for cargo check and npm scripts

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* remove echo

* notification when in ssh as well

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 11:39:16 +00:00
hugocasa
ef89a51f3a feat: upgrade bun to v1.3.8 with regression tests (#7761)
* test: add bun executor tests with minimal production code changes

- Add comprehensive bun job tests (bun_jobs.rs) covering:
  - Basic execution, error handling, annotation modes
  - Relative imports, deeply nested imports
  - Dedicated worker protocol for both Node.js and Bun runtimes
  - Builder tests for lockfile generation (import scanning)

- Minimize changes to bun_executor.rs by exposing:
  - RELATIVE_BUN_LOADER and RELATIVE_BUN_BUILDER constants
  - build_loader() function and LoaderMode enum
  - BUN_DEDICATED_WORKER_ARGS constant
  - generate_dedicated_worker_wrapper() function

- Tests call production code directly (build_loader) instead of
  duplicating script generation logic

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit

* fix: reuse BUN_PATH/NODE_BIN_PATH from windmill-worker, add node to CI

- Tests now use exported BUN_PATH and NODE_BIN_PATH constants instead
  of duplicating env var logic
- Update backend-test.yml:
  - Upgrade bun to v1.3.8
  - Add setup-node action
  - Add NODE_BIN_PATH to cargo test command

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* add private repo test

* fix private repo test

* try fix again

* fix

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 11:38:15 +00:00
Ruben Fiszel
74aeeeddec fix quickjs ser impl 2026-02-04 11:02:23 +00:00
Diego Imbert
05084254a3 feat: Assets page exploration UI (#7784)
* Disable secondary storage button if no primary

* stash

* Asset cards

* nits

* Nit page header uniform height

* ui nits

* fix manage buttons wrong path

* sqlx prepare

* nit
2026-02-04 10:53:41 +00:00
Ruben Fiszel
60240bb54f fix e2e 2026-02-04 10:25:03 +00:00
Guilhem
011fefd2a1 display (not started) for scheduled jobs (#7783) 2026-02-04 09:28:01 +00:00
Ruben Fiszel
f151fdcf7f fix: handle Date serialization in quickjs flow eval via toJSON
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 07:54:41 +00:00
Ruben Fiszel
e860847073 feat: replace LISTEN/NOTIFY with polling-based event system (#7778)
* feat: replace LISTEN/NOTIFY with polling-based event system

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: add exhaustive tests for polling-based notify events

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: add missing triggers and fix tests for polling-based events

- Add variable/resource cache invalidation triggers to migration
- Fix flow test to UPDATE flow table instead of INSERT into flow_version
- Improve test isolation with unique channel names per test
- All 26 tests now pass

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: add multi-server polling tests for cross-server event propagation

Add 4 tests simulating independent server instances with separate DB
connections and polling state:
- test_two_servers_both_receive_trigger_event: both servers see same event
- test_two_servers_cross_trigger_visibility: each triggers a change, both see both
- test_server_catches_up_after_being_offline: server catches up on missed events
- test_two_servers_incremental_polling: multi-round polling with cursor advancement

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add LISTEN_NEW_EVENTS_INTERVAL_SEC env var and e2e server test

- Make poll interval configurable via LISTEN_NEW_EVENTS_INTERVAL_SEC
  (defaults to 30s)
- Make migration idempotent with IF NOT EXISTS
- Replace mock multi-server tests with actual e2e test that starts two
  windmill server processes on ports 19100/19200 with 1s poll interval,
  triggers a DB change, and verifies both servers log the event

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: ignore notify_events tests in CI

These tests require a running database, like other integration tests
in the codebase. Run with --ignored flag locally.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: only ignore slow e2e test, not fast DB tests

Only test_two_server_processes_both_receive_event is slow (~10s,
starts two server processes). The other 26 tests run in <0.2s.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: ignore all notify_events tests in CI

All tests depend on the notify_event table from the polling-based
events migration, which is not applied in CI.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use sqlx::test for notify_events tests so they work in CI

Convert all 26 fast tests from #[tokio::test] + manual get_db() to
#[sqlx::test(fixtures("base"))], which creates temporary databases
with all migrations applied. This ensures the notify_event table
exists in CI without manual setup. Only the slow e2e multi-server
test retains #[tokio::test] + #[ignore].

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: reduce default polling interval from 30s to 10s

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: address review feedback on polling-based events

- Remove redundant notify_event_id_idx index (id is already PRIMARY KEY)
- Add LIMIT 1000 to poll_notify_events to bound memory per poll cycle
- Fix potential UTF-8 panic in token log truncation using str::get
- Remove var/resource cache triggers that were re-enabled by mistake
  (they were intentionally dropped in migration 20250902085504)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 07:35:29 +00:00
Ruben Fiszel
b807e0f5fc chore(main): release 1.624.0 (#7757)
* chore(main): release 1.624.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-02-03 19:28:19 +01:00
Diego Imbert
df51f96905 fix: Prettier and less invasive toasts (#7758)
* Toast update

* ToastType

* nit adjustements

* nit smaller toast
2026-02-03 18:18:21 +00:00
hugocasa
1d51dc97e9 simplify loading of hubPaths.json in frontend (#7775) 2026-02-03 18:17:37 +00:00
Ruben Fiszel
701eb4bae4 fix(cli): prevent branch-specific items from being marked for deletion on pull (#7781)
The elementsToMap function was incorrectly skipping remote base files that
were configured as branch-specific, causing pull to mark them for deletion.

Root cause: PR #7643 added a check to skip base files when configured as
branch-specific, but this was applied to both local AND remote sources.
Remote workspace files only have base paths (e.g., TestVar.variable.yaml),
not branch-specific paths (e.g., TestVar.staging.variable.yaml).

Fix: Add isRemote parameter to elementsToMap to distinguish remote vs local
processing. Only skip base files for local sources where we expect the
branch-specific version to exist.

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 18:15:38 +00:00
centdix
d4a1b4abed add aiagent module support to inline script extraction/replacement (#7773)
* dual build for utils-internal

* bump version

* feat(cli): add aiagent module support to inline script extraction/replacement

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* add missing field in openapi

* bump yaml validator version

* cleaning

* cleaning

* cleaning

* nit

* cleaning

* cleaning

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 18:00:34 +00:00
Henri Courdent
4cfedd26b0 Remove first video (#7780)
Removed a link to an asset from the README.
2026-02-03 17:59:02 +00:00
Henri Courdent
9ea7094f76 Refresh readme (#7779)
* Refresh readme

* Video demo product

Updated asset links in README.md.

* Local dev video

Updated the image link for the CLI Screencast in the README.
2026-02-03 17:50:56 +00:00
centdix
2e470816ed prevent sql migration modal from closing when next migration arrives (#7764)
* fix: prevent sql migration modal from closing when next migration arrives

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 17:40:41 +00:00
Guilhem
3b5c1657c7 fix(frontend): improve runs detail page (#7694)
* separate flow and status with splitpane

* fix autoscroll behavior

* Flow log viewer nit

* not graph viewer

* Improve flow job result

* Create job detail header to replace metadata

* Remove FlowPreviewResult

* Create compact job header

* Use job header in runs page

* Improve runs page run preview

* Use flow header in detail section

* Show logs for script steps

* Clean old schedule status

* Limit result height

* Script run detail improvement

* Script run preview improvement

* Fix csv table overflow

* surface tertiary as background for Inputs

* nit

* Improve runs detail skeleton

* fix check

* nit

* Improve node definition

* fix flow module component overflow

* Use component DataTable for flow schema viewer

* Add language icon to step detail

* improve run header

* Improve Job detail header

* nit

* restore isOwner logic

* Handle resume flows

* restore execution status in run preview

* restore flow execution status in the preview

* flow preview, add status bar

* nit module status

* nit

* Remove flor preview result

* nit

* nit

* fix flow result card

* nit

* nit

* nit

* improve field selection on runs detail based on job type

* Improve column layout

* create JobStatusIcon component

* remove job status badge icons

* improve compact version

* use shared job field display

* improve job detail field display

* fix badge alignment

* increase padding

* nit

* nit

* improve compact display

* make background darker for metadata

* use auto layout

* fix auto layout

* improve display

* fix truncate logic

* fix compact

* improve flex adaptibility

* improve responsive layout

* improve extra compact header

* nit

* remove unused icons

* nit

* Improve flow result display

* nit

* merge progressbar and execution status

* handle canceled flow better
2026-02-03 17:20:35 +00:00
Guilhem
d47c1d31db update resource drawer to design guidelines (#7770)
* fix(frontend): update resource drawer to design guidelines

* nit
2026-02-03 13:06:18 +00:00
Diego Imbert
56c88361b8 nit frontend undefined fixes (#7772) 2026-02-03 13:05:59 +00:00
Guilhem
861b167a14 replace hard hat icon with server cog (#7771) 2026-02-03 12:26:05 +00:00
Diego Imbert
3a719cea6b fix: Fix app multiselect not refreshing result when creating element (#7766) 2026-02-03 10:02:18 +00:00
centdix
18d85f1412 fix: remove $schema field from Google AI output schema requests (#7765)
* fix: remove $schema field from Google AI output schema requests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: add $schema field to all output schema integration tests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: remove $schema field from Google AI tool parameter schemas

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: add workspace script tool test for AI agents

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 01:04:36 +00:00
Diego Imbert
635a24f82c feat: runtime assets (#7656)
* Runtime assets

* Nits

* Revert "Nits"

This reverts commit 3031a2ddd1.

* detection_kinds

* don't delete runtime assets

* Show latest executions

* conditional unique idx

* nit status

* refactor

* nit refactor

* prepare sql

* Detect assets in complex JSON input objects

* false positive prevent

* nit

* redundant idx

* Update frontend/src/lib/components/assets/AssetsUsageDrawer.svelte

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* Update backend/migrations/20260122134517_runtime_assets.up.sql

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* runtime assets are inserted in a loop

* nit

* nit fix

* Don't use lazy static

* fix compilation

* nits

* missing on conflict do nothing

* add index

* Fix max n logic

* created at

* nits

* remove pagination

* sqlx prepare

* Only detect resource assets in input

* get_runtime_asset_sender()

* use global get_runtime_asset_sender to avoid prop drilling

* nit refactor : register_runtime_asset

* get job_id from token

* job as a usage kind

* fixes

* ee

* nit refactor

* merge access types when same job uses same asset multiple times

* Refactor to support wmill s3 API

* nit

* parse_wmill_sdk_sql_assets refactor

* Detect datatable and ducklake usage

* nit order by

* Join with v2_job

* better UI

* add sequential id for cursor pagination

* useInfiniteQuery

* useScrollToBottom

* sql index

* claude code stash

* migration fixes

* Infinite scroll UI

* nit

* style nit

* runtime asset created at

* Asset filters

* fix usage kind filter

* also check runnable_path for jobs when filtering

* better filters

* avoid flickering

* debounced filters

* nit

* tooltips

* fix: update AssetUsage type to match new ListAssetsResponse structure

The ListAssetsResponse changed from an array to an object with an 'assets' property.
Updated the type extraction accordingly.

Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com>

* sqlx prepare

* Delete .claude/hooks/.symlink-manifest

* unnecessary dep

* nit refactor

* nit comment

* nit naming

* CI fix attempt 1

* ee ref

* nit remove alerts

* nit

* chore: update ee-repo-ref to 138a4f5f868f3bded5bb7cb77b222b532c07e4af

This commit updates the EE repository reference after PR #395 was merged in windmill-ee-private.

Previous ee-repo-ref: 7d3a21d53066726e97dfea9f117373299bc9318c

New ee-repo-ref: 138a4f5f868f3bded5bb7cb77b222b532c07e4af

Automated by sync-ee-ref workflow.

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-02-02 10:09:40 +00:00
Ruben Fiszel
bdf9447e82 feat: default to quickjs on ce for flow eval (#7756)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 08:04:37 +00:00
Ruben Fiszel
790ead082c chore(main): release 1.623.1 (#7754)
* chore(main): release 1.623.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-02-01 21:02:32 +00:00
Ruben Fiszel
50b6c199e7 Merge branch 'fix/retention-active-flows' 2026-02-01 20:53:50 +00:00
Ruben Fiszel
799db94683 fix: prevent retention cleanup from deleting jobs of active flows (#7755)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 20:52:25 +00:00
Ruben Fiszel
4226ec8260 fix: prevent retention cleanup from deleting jobs of active flows
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 20:46:14 +00:00
hugocasa
a8523f552c fix: resolve infinite effect loop in PocketIdSetting component (#7753) 2026-02-01 16:12:10 +00:00
Ruben Fiszel
5c9b95e786 chore(main): release 1.623.0 (#7748)
* chore(main): release 1.623.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-02-01 08:05:05 +00:00
Ruben Fiszel
6e824a6289 nit frontend fix 2026-01-31 22:38:47 +00:00
Ruben Fiszel
f405dff2e2 fix: preserve script envs field during sync push
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-31 22:36:00 +00:00
Devdatta Talele
720e3c5436 feat: add PocketID OAuth provider support (#7318)
* feat(oauth): add Pocket-ID OAuth provider component

- Implements PocketIdSetting.svelte following Keycloak pattern
- Configures OIDC endpoints for Pocket-ID (/authorize, /api/oidc/*)
- Supports standard OIDC scopes (openid, profile, email)
- Uses passkey-only authentication via Pocket-ID

Refs #5678

* feat(oauth): register Pocket-ID in SSO provider list

- Import PocketIdSetting component
- Add Pocket-ID to provider list in SSO tab
- Update exclusion filter to prevent duplicate custom entries

Refs #5678

* fix(oauth): add missing PocketID icon and fix component integration

- Create PocketIdIcon.svelte component with user profile icon
- Register pocket-id in APP_TO_ICON_COMPONENT mapping
- Fix PocketIdSetting to use IconedResourceType pattern matching other OAuth providers

This resolves the issue where PocketID toggle was not appearing in SSO settings.

Refs #5678

* refactor: migrate PocketIdSetting to Svelte 5 runes syntax

- Use $props() with $bindable() for reactive prop binding
- Use $state() for local reactive state
- Use $derived() for computed values
- Use $effect() for reactive side effects
- Replace on:change with onchange event handler
- Pre-populate base URL from existing config when editing
- Clean up bracket notation to dot notation for value properties

Addresses reviewer feedback

* fix: rename pocket-id to pocketid for naming convention compliance

Change identifier from 'pocket-id' to 'pocketid' to match Windmill's naming convention.
No OAuth provider uses hyphens - all custom SSO providers (keycloak, authentik, authelia,
kanidm, zitadel) use no separator.

Changes:
- AuthSettings.svelte: oauths['pocket-id'] → oauths['pocketid'] (2 locations)
- PocketIdSetting.svelte: name={'pocket-id'} → name={'pocketid'}
- icons/index.ts: 'pocket-id': PocketIdIcon → pocketid: PocketIdIcon

Note: PocketID does not need oauth_connect.json entry as it's a custom SSO provider
with user-configured endpoints, similar to Keycloak/Authentik.

Addresses reviewer feedback

* fix: use TextInput component for consistency

---------

Co-authored-by: hugocasa <hugo@casademont.ch>
2026-01-30 17:27:44 +00:00
hugocasa
1f1ef9ee94 nit ui nextcloud triggers (#7749) 2026-01-30 14:06:53 +00:00
centdix
297aa23ed4 fix: add schema compatibility layer for MCP clients like n8n (#7747)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 13:57:59 +00:00
Ruben Fiszel
9d2785bece fix npm check 2026-01-30 08:17:58 +00:00
Ruben Fiszel
45aa9ab746 chore(main): release 1.622.0 (#7742)
* chore(main): release 1.622.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-01-29 23:49:09 +00:00
centdix
ce23f21c0e feat: add token usage tracking to AI agent output (#7738)
* feat: add token usage tracking to AI agent output

Add TokenUsage struct to track input/output/cache tokens from AI providers.
Currently implemented for Bedrock provider, with infrastructure in place
for other providers. Usage is included in the AI agent result alongside
output and messages when available.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add token usage extraction for Anthropic provider

Extract usage from message_delta SSE event and convert to TokenUsage.
Includes input_tokens, output_tokens, cache_read_input_tokens, and
cache_creation_input_tokens (mapped to cache_write_input_tokens).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add token usage extraction for Google AI/Gemini provider

Extract usage from usageMetadata in Gemini SSE events and convert to TokenUsage.
Maps promptTokenCount -> input_tokens, candidatesTokenCount -> output_tokens,
totalTokenCount -> total_tokens.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add token usage extraction for OpenAI Responses API provider

Extract usage from response.completed SSE event and convert to TokenUsage.
Maps input_tokens, output_tokens, and total_tokens directly.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add token usage extraction for Azure OpenAI / Chat Completions API

Add stream_options.include_usage to request and parse usage from final
SSE chunk for providers using the standard OpenAI Chat Completions API
(Azure OpenAI, Mistral, DeepSeek, Groq, TogetherAI, CustomAI).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: cleanup token usage tracking - remove unused Image usage field and accumulate across iterations

- Remove unused `usage` field from ParsedResponse::Image variant
- Add TokenUsage::accumulate() method to sum usage across agent iterations
- Accumulate input/output/total/cache tokens instead of replacing with last iteration

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: remove verbose debug logging from AI providers

Remove tracing::info!("[debug] ...") statements that were too verbose
for production. These logged raw events on every streaming event.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: add retry mechanism for OpenAI-compatible providers without stream_options support

Some OpenAI-compatible providers don't support the stream_options parameter
for usage tracking. This adds a retry mechanism that:
- First attempts the request with stream_options.include_usage
- If it fails with 400 and error mentions stream_options/include_usage,
  automatically retries without the parameter

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: remove unnecessary text parsing overhead in image response handlers

Revert debugging changes that read response as text before parsing JSON.
Using response.json() directly is more efficient.

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: centralize TokenUsage conversion with constructor methods

Add new(), from_input_output(), and with_cache() constructors to TokenUsage
to eliminate duplicate conversion logic across providers. Also fixes potential
truncation in Bedrock cache token conversion by using i32::try_from with
fallback to i32::MAX.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: simplify Anthropic usage extraction and add Default derive

- Use idiomatic `if let` pattern instead of `is_some()` check for usage extraction
- Add Default derive to OpenAIChatUsage for consistency with other usage structs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use saturating_add to prevent overflow in token accumulation

In long-running agents with many iterations, token counts could
potentially overflow. Using saturating_add ensures values cap at
i32::MAX instead of wrapping around.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* better claude

* nit

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 23:45:46 +00:00
Alexander Petric
ca8dbc0676 fix: forward teams error to client (#7746)
* fix: forward teams error to client

* chore: update ee-repo-ref to 9a3d71f2c6a41ed4d17111a8c05d8e1d4933898d

This commit updates the EE repository reference after PR #400 was merged in windmill-ee-private.

Previous ee-repo-ref: 25d35a8de1cd70e281dc876e51cd30402580b5c0

New ee-repo-ref: 9a3d71f2c6a41ed4d17111a8c05d8e1d4933898d

Automated by sync-ee-ref workflow.

* fix

* fix

* fix

* al

* sqlx

* sqlx

* all

* all

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-01-29 23:44:42 +00:00
Ruben Fiszel
6c84a89053 fix: require AGENT_TOKEN and BASE_INTERNAL_URL for agent mode
- Add AgentConfig struct to validate required env vars on startup
- Change build_agent_http_client to require explicit token and URL
- Remove DEFAULT_BASE_INTERNAL_URL fallback (no more silent localhost:8000)
- Exit immediately if agent cannot connect to server on initial load
- Update integration tests to use dynamic port for BASE_INTERNAL_URL

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 23:12:18 +00:00
wendrul
998f11a10d fix: visibility bug on deployment UI (issue when renaming items) + add tracking of folders and resource types (#7739)
* fix: Raw apps deployment UI (and merge UI)

* Add folders and resource tpyes to merge UI

* claude first pass on adding the new arg for h_deploy_metadata

* Add missing argument to handle_deployment_metadata in all its calls

* Add support for folders and resource types in merge UI

* Update eereporef for CI

* Update ee repo

* Add migration to reset cached diff with potential artifacts

* fix type in frontend

* Preapare sqlx

* Remove unused import and logs

* update ee-repo

* Update eerepo

* chore: update ee-repo-ref to aca38475afd2cafaf63f4bbffc65be9437d57d86

This commit updates the EE repository reference after PR #397 was merged in windmill-ee-private.

Previous ee-repo-ref: 19c64cf8c61d83f45047b37660054b29658cd403

New ee-repo-ref: aca38475afd2cafaf63f4bbffc65be9437d57d86

Automated by sync-ee-ref workflow.

* Make integration  test for workspace comparisons

* Update SQLx metadata

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-01-29 22:31:49 +00:00
centdix
6a37af09bb refactor: remove seed parameter from AI chat completions (#7745)
* better claude

* refactor: remove seed parameter from AI chat completions

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 20:35:15 +00:00
wendrul
6679ecb9a2 fix: indexer build error (#7744)
* fix: indexer build error

* prepare sqlx

* Remove changes from Cargo.toml
2026-01-29 18:03:40 +00:00
Ruben Fiszel
ad5293c0ed fix: remove uuid-ossp extension requirement for RDS compatibility
The uuid-ossp extension was created in the first migration but never
actually used - the codebase uses gen_random_uuid() which is built-in
to PostgreSQL 13+. This allows Windmill to run on AWS RDS where
application users may not have CREATE SCHEMA privileges.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 17:58:42 +00:00
hugocasa
60858d1e20 feat: workspace dedicated workers (#7741)
* feat: workspace dedicated workers

* ref

* chore: update ee-repo-ref to a18ac31062ac092cb9a5fc87629e217d97f4911d

This commit updates the EE repository reference after PR #398 was merged in windmill-ee-private.

Previous ee-repo-ref: 98cfe3fef764d9d815d326d5056c734a03689d33

New ee-repo-ref: a18ac31062ac092cb9a5fc87629e217d97f4911d

Automated by sync-ee-ref workflow.

* fix(frontend): workspace script in flow steps

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-01-29 17:32:48 +00:00
Ruben Fiszel
f45d9adf6a chore(main): release 1.621.2 (#7735)
* chore(main): release 1.621.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-01-29 17:19:39 +00:00
Ruben Fiszel
20357f41f5 fix(cli): revert findCodebase change that broke ../shared codebases (#7740)
* fix(cli): revert findCodebase relative_path check that broke ../shared codebases

The previous change added a check to ensure script paths start with the
codebase's relative_path. However, this broke cases where relative_path
uses parent directory references (e.g., "../shared") because:

1. path.join normalizes paths, so "/project/../shared/f/script.ts" becomes
   "/shared/f/script.ts"
2. FSFSElement strips the cwd prefix, resulting in "f/script.ts"
3. The check "f/script.ts".startsWith("../shared/") failed

The original behavior was correct - relative_path indicates where to find
codebase files, while includes/excludes patterns match against the normalized
paths that get passed during sync.

Fixes regression reported in #7729 comments.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test(cli): add preview test for codebase with imports

Tests that codebase bundling correctly includes imported modules,
which is the key functionality needed for ../shared codebases.
The test creates a helper module and a main script that imports
from it, then verifies the bundled script executes correctly.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 17:09:02 +00:00
centdix
fe4a230833 docs: add nuanced mutex selection guidance to Rust backend skill (#7737)
Add "Mutex Selection in Async Code" section explaining when to use
std::sync::Mutex vs tokio::sync::Mutex based on official Tokio docs.
std::sync::Mutex is preferred for data protection as it's faster;
tokio::sync::Mutex only needed when holding locks across .await points.

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 15:08:41 +00:00
centdix
d004aa8ec1 internal: add Rust backend coding skill and consolidate guidelines (#7731)
* feat: add Rust backend coding skill and consolidate guidelines

Create a dedicated skill at .claude/skills/rust-backend/SKILL.md that
provides comprehensive Rust coding guidelines adapted to Windmill patterns:
- Iterator chains, error handling, early returns, variable shadowing
- JSON handling with Box<RawValue>, Serde optimizations
- SQLx patterns (no SELECT *, batch operations, avoid N+1)
- Async/Tokio patterns (spawn_blocking, bounded channels)

Consolidate project context into backend/CLAUDE.md and remove the
redundant rust-best-practices.mdc file.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* add command plugins

* nit

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 12:02:27 +00:00
centdix
1aad20b7eb hide cred check (#7736) 2026-01-29 12:01:29 +00:00
wendrul
8bb6b6331b fix: do not quit indexer when receiving handoff during pull (#7659)
* fix: do not quit indexer when receiving handoff during pull

* update

* Add correct return type

* update ee-repo-ref [CI only]

* chore: update ee-repo-ref to c05572e93739e2697ab310d87efe2744cd0e1aaf

This commit updates the EE repository reference after PR #394 was merged in windmill-ee-private.

Previous ee-repo-ref: 4358aa9c5b3b38ba74d7ea52cafd49899d338a07

New ee-repo-ref: c05572e93739e2697ab310d87efe2744cd0e1aaf

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-01-29 11:43:26 +00:00
centdix
0089ebd4fb chore: handle empty strings in AI resource fields via serde deserializer (#7723)
* fix: handle empty strings in AI resource fields via serde deserializer

Add `empty_string_as_none` deserializer that converts empty strings to None
during deserialization. Applied to base_url, api_key, region, and AWS
credential fields in AIStandardResource and ProviderResource.

This fixes the "relative URL without a base" error when creating Anthropic
resources with empty base_url fields.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit

* nit

* nit

* cleaning

* cleaning

* cleaning

* cleaning

* fix: apply empty_string_as_none deserializer to api_key field

Consistent with other fields in ProviderResource, empty strings are now
deserialized as None for the api_key field.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 11:43:02 +00:00
centdix
f856f672d8 Allow wmill app dev to accept folder argument (#7718)
Enable running the dev command from any directory by specifying the
target .raw_app folder as an argument. Workspace resolution and
authentication still happen from the original cwd to find wmill.yaml.

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 11:42:51 +00:00
312 changed files with 16100 additions and 17067 deletions

20
.claude/hooks/format-backend.sh Executable file
View File

@@ -0,0 +1,20 @@
#!/bin/bash
# Format backend Rust files with rustfmt after Claude edits them
# Get the file path from the tool result (passed via stdin as JSON)
INPUT=$(cat)
FILE_PATH=$(echo "$INPUT" | jq -r '.tool_input.file_path // empty')
# Exit if no file path
if [ -z "$FILE_PATH" ]; then
exit 0
fi
# Check if the file is in the backend directory and is a Rust file
if [[ "$FILE_PATH" == *"/backend/"* ]] && [[ "$FILE_PATH" =~ \.rs$ ]]; then
cd "$CLAUDE_PROJECT_DIR/backend" || exit 0
# Run rustfmt with config from rustfmt.toml (edition=2021)
rustfmt --config-path rustfmt.toml "$FILE_PATH" 2>/dev/null || true
fi
exit 0

View File

@@ -0,0 +1,23 @@
#!/bin/bash
# Format frontend files with prettier after Claude edits them
# Get the file path from the tool result (passed via stdin as JSON)
INPUT=$(cat)
FILE_PATH=$(echo "$INPUT" | jq -r '.tool_input.file_path // empty')
# Exit if no file path
if [ -z "$FILE_PATH" ]; then
exit 0
fi
# Check if the file is in the frontend directory
if [[ "$FILE_PATH" == *"/frontend/"* ]]; then
# Check if it's a formattable file type
if [[ "$FILE_PATH" =~ \.(ts|js|svelte|json|css|html|md)$ ]]; then
cd "$CLAUDE_PROJECT_DIR/frontend" || exit 0
# Run prettier silently, don't fail the hook if prettier fails
npx prettier --write "$FILE_PATH" 2>/dev/null || true
fi
fi
exit 0

25
.claude/hooks/notify-user.sh Executable file
View File

@@ -0,0 +1,25 @@
#!/bin/bash
# Notify user when Claude requires input (works on macOS and Linux)
# Check if we're in an SSH session
if [[ -n "$SSH_CLIENT" || -n "$SSH_TTY" || -n "$SSH_CONNECTION" ]]; then
# SSH session - use terminal bell
# If using VSCode, enable audible terminal bell for SSH sessions:
# Add the following to .vscode/settings.json:
# "accessibility.signals.terminalBell": {
# "sound": "on"
# },
# "terminal.integrated.enableVisualBell": true
printf '\a'
else
# Local session - use native notifications
if [[ "$OSTYPE" == "darwin"* ]]; then
osascript -e 'display notification "Claude is waiting for your input" with title "Claude Code" sound name "Glass"' 2>/dev/null || printf '\a'
elif [[ "$OSTYPE" == "linux-gnu"* ]]; then
notify-send "Claude Code" "Claude is waiting for your input" 2>/dev/null || printf '\a'
else
printf '\a'
fi
fi
exit 0

View File

@@ -1,39 +1,4 @@
{
"hooks": {
"UserPromptSubmit": [
{
"hooks": [
{
"type": "command",
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/resolve-symlinks.sh",
"timeout": 30
}
]
}
],
"Stop": [
{
"hooks": [
{
"type": "command",
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/restore-symlinks.sh",
"timeout": 30
}
]
}
],
"SessionEnd": [
{
"hooks": [
{
"type": "command",
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/restore-symlinks.sh",
"timeout": 30
}
]
}
]
},
"permissions": {
"allow": [
"Bash(ls:*)",
@@ -58,7 +23,11 @@
"Bash(git log:*)",
"Bash(git branch:*)",
"Bash(git show:*)",
"Bash(git blame:*)"
"Bash(git blame:*)",
"Bash(cargo check:*)",
"mcp__ide__getDiagnostics",
"Bash(npm run generate-backend-client:*)",
"Bash(npm run check:*)"
],
"deny": [
"Read(.env)",
@@ -93,8 +62,72 @@
]
},
"enableAllProjectMcpServers": true,
"hooks": {
"UserPromptSubmit": [
{
"hooks": [
{
"type": "command",
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/resolve-symlinks.sh",
"timeout": 30
}
]
}
],
"Stop": [
{
"hooks": [
{
"type": "command",
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/restore-symlinks.sh",
"timeout": 30
}
]
}
],
"SessionEnd": [
{
"hooks": [
{
"type": "command",
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/restore-symlinks.sh",
"timeout": 30
}
]
}
],
"PostToolUse": [
{
"matcher": "Edit|Write",
"hooks": [
{
"type": "command",
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/format-frontend.sh",
"timeout": 30
},
{
"type": "command",
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/format-backend.sh",
"timeout": 30
}
]
}
],
"Notification": [
{
"hooks": [
{
"type": "command",
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/notify-user.sh",
"timeout": 10
}
]
}
]
},
"enabledPlugins": {
"rust-analyzer-lsp@claude-plugins-official": true,
"typescript-lsp@claude-plugins-official": true
"typescript-lsp@claude-plugins-official": true,
"code-review@claude-plugins-official": true
}
}

View File

@@ -0,0 +1,60 @@
---
name: commit
user_invocable: true
description: Create a git commit with conventional commit format. MUST use anytime you want to commit changes.
---
# Git Commit Skill
Create a focused, single-line commit following conventional commit conventions.
## Instructions
1. **Analyze changes**: Run `git status` and `git diff` to understand what was modified
2. **Stage only modified files**: Add files individually by name. NEVER use `git add -A` or `git add .`
3. **Write commit message**: Follow the conventional commit format as a single line
## Conventional Commit Format
```
<type>: <description>
```
### Types
- `feat`: New feature or capability
- `fix`: Bug fix
- `refactor`: Code change that neither fixes a bug nor adds a feature
- `docs`: Documentation only changes
- `style`: Formatting, missing semicolons, etc (no code change)
- `test`: Adding or correcting tests
- `chore`: Maintenance tasks, dependency updates, etc
- `perf`: Performance improvement
### Rules
- Message MUST be a single line (no multi-line messages)
- Description should be lowercase, imperative mood ("add" not "added")
- No period at the end
- Keep under 72 characters total
### Examples
```
feat: add token usage tracking for AI providers
fix: resolve null pointer in job executor
refactor: extract common validation logic
docs: update API endpoint documentation
chore: upgrade sqlx to 0.7
```
## Execution Steps
1. Run `git status` to see all changes
2. Run `git diff` to understand the changes in detail
3. Run `git log --oneline -5` to see recent commit style
4. Stage ONLY the modified/relevant files: `git add <file1> <file2> ...`
5. Create the commit with conventional format:
```bash
git commit -m "<type>: <description>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>"
```
6. Run `git status` to verify the commit succeeded

View File

@@ -0,0 +1,87 @@
---
name: pr
user_invocable: true
description: Open a draft pull request on GitHub. MUST use when you want to create/open a PR.
---
# Pull Request Skill
Create a draft pull request with a clear title and explicit description of changes.
## Instructions
1. **Analyze branch changes**: Understand all commits since diverging from main
2. **Push to remote**: Ensure all commits are pushed
3. **Create draft PR**: Always open as draft for review before merging
## PR Title Format
Follow conventional commit format for the PR title:
```
<type>: <description>
```
### Types
- `feat`: New feature or capability
- `fix`: Bug fix
- `refactor`: Code restructuring
- `docs`: Documentation changes
- `chore`: Maintenance tasks
- `perf`: Performance improvements
### Title Rules
- Keep under 70 characters
- Use lowercase, imperative mood
- No period at the end
## PR Body Format
The body MUST be explicit about what changed. Structure:
```markdown
## Summary
<Clear description of what this PR does and why>
## Changes
- <Specific change 1>
- <Specific change 2>
- <Specific change 3>
## Test plan
- [ ] <How to verify change 1>
- [ ] <How to verify change 2>
---
Generated with [Claude Code](https://claude.com/claude-code)
```
## Execution Steps
1. Run `git status` to check for uncommitted changes
2. Run `git log main..HEAD --oneline` to see all commits in this branch
3. Run `git diff main...HEAD` to see the full diff against main
4. Check if remote branch exists and is up to date:
```bash
git rev-parse --abbrev-ref --symbolic-full-name @{u} 2>/dev/null || echo "no upstream"
```
5. Push to remote if needed: `git push -u origin HEAD`
6. Create draft PR using gh CLI:
```bash
gh pr create --draft --title "<type>: <description>" --body "$(cat <<'EOF'
## Summary
<description>
## Changes
- <change 1>
- <change 2>
## Test plan
- [ ] <test 1>
- [ ] <test 2>
---
Generated with [Claude Code](https://claude.com/claude-code)
EOF
)"
```
7. Return the PR URL to the user

View File

@@ -0,0 +1,495 @@
---
name: rust-backend
description: Rust coding guidelines for the Windmill backend. MUST use when writing or modifying Rust code in the backend directory.
---
# Rust Backend Coding Guidelines
Apply these patterns when writing or modifying Rust code in the `backend/` directory.
## Data Structure Design
Choose between `struct`, `enum`, or `newtype` based on domain needs:
- Use `enum` for state machines instead of boolean flags or loosely related fields
- Model invariants explicitly using types (e.g., `NonZeroU32`, `Duration`, custom enums)
- Consider ownership of each field:
- Use `&str` vs `String`, slices vs vectors
- Use `Arc<T>` when sharing across threads
- Use `Cow<'a, T>` for flexible ownership
```rust
// State machine with enum
enum JobState {
Pending { scheduled_for: DateTime<Utc> },
Running { started_at: DateTime<Utc>, worker: String },
Completed { result: JobResult, duration_ms: i64 },
Failed { error: String, retries: u32 },
}
// Avoid multiple booleans
struct Job {
is_pending: bool, // Don't do this
is_running: bool,
is_completed: bool,
}
```
## Impl Block Organization
Place `impl` blocks immediately below the struct/enum they modify. Group methods logically:
```rust
struct JobQueue {
jobs: Vec<Job>,
capacity: usize,
}
impl JobQueue {
// Constructors first
pub fn new(capacity: usize) -> Self { ... }
pub fn with_jobs(jobs: Vec<Job>) -> Self { ... }
// Getters
pub fn len(&self) -> usize { ... }
pub fn is_empty(&self) -> bool { ... }
// Mutation methods
pub fn push(&mut self, job: Job) -> Result<()> { ... }
pub fn pop(&mut self) -> Option<Job> { ... }
// Domain logic
pub fn next_scheduled(&self) -> Option<&Job> { ... }
}
```
## Iterator Chains Over For-Loops
Prefer functional iterator chains (`.filter().map().collect()`) over imperative for-loops:
```rust
// Preferred
let results: Vec<_> = items
.iter()
.filter(|item| item.is_valid())
.map(|item| item.transform())
.collect();
// Avoid
let mut results = Vec::new();
for item in items.iter() {
if item.is_valid() {
results.push(item.transform());
}
}
```
## Error Handling
Use the `Error` type from `windmill_common::error`. Return `Result<T, Error>` or `JsonResult<T>` for fallible functions:
```rust
use windmill_common::error::{Error, Result};
// Use ? operator for propagation
pub async fn get_job(db: &DB, id: Uuid) -> Result<Job> {
let job = sqlx::query_as!(Job, "SELECT ... WHERE id = $1", id)
.fetch_optional(db)
.await?
.ok_or_else(|| Error::NotFound("job not found".to_string()))?;
Ok(job)
}
```
Prefer `if let` for optional handling. Use `let...else` when early return makes code clearer:
```rust
let Some(config) = get_config() else {
return Err(Error::MissingConfig);
};
```
Never panic in library code. Reserve `.unwrap()` for cases with compile-time guarantees. Keep functions short to help lifetime inference and clarity.
## Early Returns
Return early to avoid deep nesting. Handle error cases and edge conditions first:
```rust
// Preferred - early returns
fn process_job(job: Option<Job>) -> Result<Output> {
let Some(job) = job else {
return Ok(Output::default());
};
if !job.is_valid() {
return Err(Error::InvalidJob);
}
if job.is_cached() {
return Ok(job.cached_result());
}
// Main logic at the end, not nested
execute_job(job)
}
// Avoid - deep nesting
fn process_job(job: Option<Job>) -> Result<Output> {
if let Some(job) = job {
if job.is_valid() {
if !job.is_cached() {
execute_job(job)
} else {
Ok(job.cached_result())
}
} else {
Err(Error::InvalidJob)
}
} else {
Ok(Output::default())
}
}
```
## Variable Shadowing
Shadow variables instead of creating new names with prefixes:
```rust
// Preferred
let data = fetch_raw_data();
let data = parse(data);
let data = validate(data)?;
// Avoid
let raw_data = fetch_raw_data();
let parsed_data = parse(raw_data);
let validated_data = validate(parsed_data)?;
```
## Minimal Comments
- No inline comments explaining obvious code
- No TODO/FIXME comments in committed code
- Doc comments (`///`) only on public items
- Let code be self-documenting through clear naming
## Type Safety
Use enums over boolean flags for clarity:
```rust
// Preferred
enum JobStatus {
Pending,
Running,
Completed,
}
// Avoid
struct Job {
is_running: bool,
is_completed: bool,
}
```
## Pattern Matching
Prefer explicit matching. Use wildcards strategically for fallback cases or ignored fields:
```rust
// Explicit matching preferred
match status {
JobStatus::Pending => handle_pending(),
JobStatus::Running => handle_running(),
JobStatus::Completed => handle_completed(),
}
// Wildcards OK for fallback
match result {
Ok(value) => process(value),
Err(_) => return default_value(),
}
// Wildcards OK for ignoring fields in destructuring
let Point { x, y, .. } = point;
```
## Destructuring in Function Signatures
Destructure structs directly in function parameters:
```rust
// Preferred
async fn process_job(
Extension(db): Extension<DB>,
Path((workspace, job_id)): Path<(String, Uuid)>,
Query(pagination): Query<Pagination>,
) -> Result<Json<Job>> {
// ...
}
// Avoid
async fn process_job(
db_ext: Extension<DB>,
path: Path<(String, Uuid)>,
query: Query<Pagination>,
) -> Result<Json<Job>> {
let Extension(db) = db_ext;
let Path((workspace, job_id)) = path;
// ...
}
```
## Trait Implementations
Use standard trait implementations to simplify conversions and reduce boilerplate:
```rust
// Implement From/Into for type conversions
impl From<DbJob> for ApiJob {
fn from(db: DbJob) -> Self {
ApiJob {
id: db.id,
status: db.status.into(),
}
}
}
// Use TryFrom for fallible conversions
impl TryFrom<String> for JobKind {
type Error = Error;
fn try_from(s: String) -> Result<Self, Self::Error> { ... }
}
```
Apply `derive` macros to reduce boilerplate:
```rust
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Job { ... }
```
## Module Structure
- Use `pub(crate)` instead of `pub` when possible; expose only what needs exposing
- Keep APIs small and expressive; avoid leaking internal types
- Organize code into modules reflecting ownership and domain boundaries
```rust
// Prefer restricted visibility
pub(crate) fn internal_helper() { ... }
// Only pub for external API
pub fn create_job(...) -> Result<Job> { ... }
```
## Code Navigation
Always use rust-analyzer LSP for:
- Go to definition
- Find references
- Type information
- Import resolution
Do not guess at module paths or type definitions.
## JSON Handling
Prefer `Box<serde_json::value::RawValue>` over `serde_json::Value` when:
- Storing JSON in the database (JSONB columns)
- Passing JSON through without modification
- The JSON structure doesn't need inspection
```rust
// Preferred - avoids parsing/serialization overhead
pub struct Job {
pub id: Uuid,
pub args: Option<Box<serde_json::value::RawValue>>,
}
// Only use Value when you need to inspect/modify JSON
let value: serde_json::Value = serde_json::from_str(&json)?;
if let Some(field) = value.get("field") {
// modify or inspect
}
```
## Serde Optimizations
Use serde attributes to optimize serialization:
```rust
#[derive(Serialize, Deserialize)]
pub struct Job {
#[serde(rename = "jobId")]
pub id: Uuid,
#[serde(default)]
pub priority: i32,
#[serde(skip_serializing_if = "Option::is_none")]
pub parent_job: Option<Uuid>,
#[serde(skip_serializing_if = "Vec::is_empty")]
pub tags: Vec<String>,
}
```
Prefer borrowing for zero-copy deserialization when lifetimes allow:
```rust
#[derive(Deserialize)]
pub struct JobInput<'a> {
#[serde(borrow)]
pub workspace_id: Cow<'a, str>,
#[serde(borrow)]
pub script_path: &'a str,
}
```
## SQLx Patterns
**Never use `SELECT *`** - always list columns explicitly. This is critical for backwards compatibility when workers run behind the API server version:
```rust
// Preferred - explicit columns
sqlx::query_as!(
Job,
"SELECT id, workspace_id, path, created_at FROM v2_job WHERE id = $1",
job_id
)
// Avoid - breaks when columns are added
sqlx::query_as!(Job, "SELECT * FROM v2_job WHERE id = $1", job_id)
```
Use batch operations to minimize round trips:
```rust
// Preferred - single query with multiple values
sqlx::query!(
"INSERT INTO job_logs (job_id, logs) VALUES ($1, $2), ($3, $4)",
id1, log1, id2, log2
)
// Avoid N+1 queries
for id in ids {
sqlx::query!("SELECT ... WHERE id = $1", id).fetch_one(db).await?;
}
// Preferred - single query with IN clause
sqlx::query!("SELECT ... WHERE id = ANY($1)", &ids[..]).fetch_all(db).await?
```
Use transactions for multi-step operations and parameterize all queries.
## Async & Tokio Patterns
Never block the async runtime. Use `spawn_blocking` for CPU-intensive or blocking I/O:
```rust
// Preferred - offload blocking work
let result = tokio::task::spawn_blocking(move || {
expensive_computation(&data)
}).await?;
// Avoid - blocks the runtime
let result = expensive_computation(&data); // Don't do this in async
```
Use tokio primitives for sleep and channels:
```rust
use tokio::sync::mpsc;
use tokio::time::sleep;
// Avoid in async contexts
use std::thread::sleep; // Blocks the runtime
```
Use bounded channels for backpressure:
```rust
// Preferred - bounded channel prevents overwhelming
let (tx, rx) = tokio::sync::mpsc::channel(100);
// Be careful with unbounded
let (tx, rx) = tokio::sync::mpsc::unbounded_channel();
```
## Mutex Selection in Async Code
**Prefer `std::sync::Mutex` (or `parking_lot::Mutex`) over `tokio::sync::Mutex`** for protecting data in async code. The async mutex is more expensive and only needed when holding locks across `.await` points.
```rust
// Preferred for data protection - std mutex is faster
use std::sync::Mutex;
struct Cache {
data: Mutex<HashMap<String, Value>>,
}
impl Cache {
fn get(&self, key: &str) -> Option<Value> {
self.data.lock().unwrap().get(key).cloned()
}
fn insert(&self, key: String, value: Value) {
self.data.lock().unwrap().insert(key, value);
}
}
```
**Use `tokio::sync::Mutex` only when you must hold the lock across `.await` points**, typically for IO resources like database connections:
```rust
use tokio::sync::Mutex;
use std::sync::Arc;
// Async mutex for IO resources held across await points
let conn = Arc::new(Mutex::new(db_connection));
async fn execute_query(conn: Arc<Mutex<DbConn>>, query: &str) {
let mut lock = conn.lock().await;
lock.execute(query).await; // Lock held across .await
}
```
**Common pattern**: Wrap `Arc<Mutex<...>>` in a struct with non-async methods that lock internally, keeping lock scope minimal:
```rust
struct SharedState {
inner: std::sync::Mutex<StateInner>,
}
impl SharedState {
fn update(&self, value: i32) {
self.inner.lock().unwrap().value = value;
}
fn get(&self) -> i32 {
self.inner.lock().unwrap().value
}
}
```
**Alternative for IO resources**: Spawn a dedicated task to manage the resource and communicate via message passing:
```rust
let (tx, mut rx) = tokio::sync::mpsc::channel(32);
tokio::spawn(async move {
while let Some(cmd) = rx.recv().await {
handle_io_command(&mut resource, cmd).await;
}
});
```
## Build & Tooling
Build speed tips:
- Use `cargo check` during rapid iteration over `cargo build`
- Minimize unnecessary dependencies and feature flags

View File

@@ -42,7 +42,7 @@ RUN wget https://www.python.org/ftp/python/${PYTHON_VERSION}/Python-${PYTHON_VER
RUN /usr/local/bin/python3 -m pip install pip-tools
# Bun
COPY --from=oven/bun:1.2.23 /usr/local/bin/bun /usr/bin/bun
COPY --from=oven/bun:1.3.8 /usr/local/bin/bun /usr/bin/bun
ARG TARGETPLATFORM

View File

@@ -44,7 +44,10 @@ jobs:
go-version: 1.21.5
- uses: oven-sh/setup-bun@v2
with:
bun-version: 1.1.43
bun-version: 1.3.8
- uses: actions/setup-node@v4
with:
node-version: '20'
- uses: astral-sh/setup-uv@v6.2.1
with:
version: "0.9.24"
@@ -67,6 +70,111 @@ jobs:
- name: Substitute EE code (EE logic is behind feature flag)
run: |
./substitute_ee_code.sh --copy --dir ./windmill-ee-private
- name: Setup private npm registry with test package
working-directory: /tmp
run: |
set -e
# Install Verdaccio globally
npm install -g verdaccio
# Create Verdaccio config that requires authentication for @windmill-test packages
mkdir -p /tmp/verdaccio/storage
cat > /tmp/verdaccio/config.yaml << 'VERDACCIO_CONFIG'
storage: /tmp/verdaccio/storage
auth:
htpasswd:
file: /tmp/verdaccio/htpasswd
max_users: 100
uplinks:
npmjs:
url: https://registry.npmjs.org/
packages:
'@windmill-test/*':
access: $authenticated
publish: $authenticated
'@*/*':
access: $all
publish: $authenticated
proxy: npmjs
'**':
access: $all
publish: $authenticated
proxy: npmjs
server:
keepAliveTimeout: 60
middlewares:
audit:
enabled: true
log: { type: stdout, format: pretty, level: warn }
VERDACCIO_CONFIG
# Create empty htpasswd file (users will be created via API)
touch /tmp/verdaccio/htpasswd
# Start Verdaccio in background
verdaccio --config /tmp/verdaccio/config.yaml &
VERDACCIO_PID=$!
# Wait for Verdaccio to be ready
echo "Waiting for Verdaccio to start..."
for i in {1..30}; do
if curl -s http://localhost:4873/-/ping > /dev/null 2>&1; then
echo "Verdaccio is ready"
break
fi
sleep 1
done
# Login to get a token
echo "Getting auth token..."
RESPONSE=$(curl -s -X PUT \
-H "Content-Type: application/json" \
-d '{"name":"testuser","password":"testpass123"}' \
http://localhost:4873/-/user/org.couchdb.user:testuser)
echo "Auth response: $RESPONSE"
NPM_TOKEN=$(echo "$RESPONSE" | jq -r '.token')
if [ -z "$NPM_TOKEN" ] || [ "$NPM_TOKEN" = "null" ]; then
echo "Failed to get NPM token from response"
exit 1
fi
echo "NPM_TOKEN=${NPM_TOKEN}" >> $GITHUB_ENV
echo "Got NPM token successfully: ${NPM_TOKEN:0:10}..."
# Configure npm globally with the auth token
echo "//localhost:4873/:_authToken=${NPM_TOKEN}" > ~/.npmrc
echo "Configured ~/.npmrc with auth token"
# Create a simple test package
mkdir -p /tmp/windmill-test-private-pkg
cat > /tmp/windmill-test-private-pkg/package.json << 'PKG_JSON'
{
"name": "@windmill-test/private-pkg",
"version": "1.0.0",
"main": "index.js"
}
PKG_JSON
cat > /tmp/windmill-test-private-pkg/index.js << 'PKG_JS'
module.exports.greet = (name) => `Hello from private package, ${name}!`;
PKG_JS
# Publish to Verdaccio with auth
cd /tmp/windmill-test-private-pkg
echo "Publishing package..."
npm publish --registry http://localhost:4873
echo "Package published successfully"
# Verify the package requires auth by trying anonymous access (should fail)
rm -f ~/.npmrc
echo "Testing anonymous access (should fail)..."
if npm view @windmill-test/private-pkg --registry http://localhost:4873 2>/dev/null; then
echo "ERROR: Package should require authentication but anonymous access worked"
exit 1
fi
echo "Verified: Package requires authentication for @windmill-test/private-pkg"
- name: Cache DuckDB FFI module build
uses: actions/cache@v3
with:
@@ -84,9 +192,10 @@ jobs:
RUST_LOG_STYLE: never
CARGO_NET_GIT_FETCH_WITH_CLI: true
WMDEBUG_FORCE_V0_WORKSPACE_DEPENDENCIES: 1
WMDEBUG_FORCE_RUNNABLE_SETTINGS_V0: 1
WMDEBUG_FORCE_RUNNABLE_SETTINGS_V0: 1
WMDEBUG_FORCE_NO_LEGACY_DEBOUNCING_COMPAT: 1
TEST_NPM_REGISTRY: "http://localhost:4873/:_authToken=${{ env.NPM_TOKEN }}"
run: |
deno --version && bun -v && go version && python3 --version
deno --version && bun -v && node --version && go version && python3 --version
cd windmill-duckdb-ffi-internal && ./build_dev.sh && cd ..
DENO_PATH=$(which deno) BUN_PATH=$(which bun) GO_PATH=$(which go) UV_PATH=$(which uv) cargo test --features enterprise,deno_core,duckdb,license,python,rust,scoped_cache,parquet,private --all -- --nocapture
DENO_PATH=$(which deno) BUN_PATH=$(which bun) NODE_BIN_PATH=$(which node) GO_PATH=$(which go) UV_PATH=$(which uv) cargo test --features enterprise,deno_core,duckdb,license,python,rust,scoped_cache,parquet,private,private_registry_test --all -- --nocapture

6
.gitignore vendored
View File

@@ -13,3 +13,9 @@ backend/.minio-data
.aider*
!.aiderignore
rust-client/Cargo.toml
# Symlinked cache directories (for git worktrees)
backend/target
frontend/node_modules
typescript-client/node_modules
frontend/.svelte-kit

View File

@@ -1,5 +1,69 @@
# Changelog
## [1.624.0](https://github.com/windmill-labs/windmill/compare/v1.623.1...v1.624.0) (2026-02-03)
### Features
* default to quickjs on ce for flow eval ([#7756](https://github.com/windmill-labs/windmill/issues/7756)) ([bdf9447](https://github.com/windmill-labs/windmill/commit/bdf9447e821c6d02198534198a5878849cac23e5))
* runtime assets ([#7656](https://github.com/windmill-labs/windmill/issues/7656)) ([635a24f](https://github.com/windmill-labs/windmill/commit/635a24f82cae8e85b584efca115968872723889f))
### Bug Fixes
* **cli:** prevent branch-specific items from being marked for deletion on pull ([#7781](https://github.com/windmill-labs/windmill/issues/7781)) ([701eb4b](https://github.com/windmill-labs/windmill/commit/701eb4bae47a809e6da34c62b8e250ac6379db53))
* Fix app multiselect not refreshing result when creating element ([#7766](https://github.com/windmill-labs/windmill/issues/7766)) ([3a719ce](https://github.com/windmill-labs/windmill/commit/3a719cea6b7b099f32054957eb04148c592786ad))
* **frontend:** improve runs detail page ([#7694](https://github.com/windmill-labs/windmill/issues/7694)) ([3b5c165](https://github.com/windmill-labs/windmill/commit/3b5c1657c7d41178283d02017914543461565a3a))
* Prettier and less invasive toasts ([#7758](https://github.com/windmill-labs/windmill/issues/7758)) ([df51f96](https://github.com/windmill-labs/windmill/commit/df51f9690520db80db2133e2e61002f399c0dfaf))
* remove $schema field from Google AI output schema requests ([#7765](https://github.com/windmill-labs/windmill/issues/7765)) ([18d85f1](https://github.com/windmill-labs/windmill/commit/18d85f14127e50673ccb460bfa9ebe80730df68e))
## [1.623.1](https://github.com/windmill-labs/windmill/compare/v1.623.0...v1.623.1) (2026-02-01)
### Bug Fixes
* prevent retention cleanup from deleting jobs of active flows ([4226ec8](https://github.com/windmill-labs/windmill/commit/4226ec826084eabbb9fff418ea6e67eb73e27cf0))
* prevent retention cleanup from deleting jobs of active flows ([#7755](https://github.com/windmill-labs/windmill/issues/7755)) ([799db94](https://github.com/windmill-labs/windmill/commit/799db9468395adafe43630d861dac367e5559791))
* resolve infinite effect loop in PocketIdSetting component ([#7753](https://github.com/windmill-labs/windmill/issues/7753)) ([a8523f5](https://github.com/windmill-labs/windmill/commit/a8523f552c39c4bbe3c585f97df5223903013bb2))
## [1.623.0](https://github.com/windmill-labs/windmill/compare/v1.622.0...v1.623.0) (2026-01-31)
### Features
* add PocketID OAuth provider support ([#7318](https://github.com/windmill-labs/windmill/issues/7318)) ([720e3c5](https://github.com/windmill-labs/windmill/commit/720e3c543623c2612b1af704c13d032c53368efb))
### Bug Fixes
* add schema compatibility layer for MCP clients like n8n ([#7747](https://github.com/windmill-labs/windmill/issues/7747)) ([297aa23](https://github.com/windmill-labs/windmill/commit/297aa23ed46315dfd4b034d44361a5bd8aaca884))
* preserve script envs field during sync push ([f405dff](https://github.com/windmill-labs/windmill/commit/f405dff2e22681dc8d4f3a9b7427e278c6cfb0cc))
## [1.622.0](https://github.com/windmill-labs/windmill/compare/v1.621.2...v1.622.0) (2026-01-29)
### Features
* add token usage tracking to AI agent output ([#7738](https://github.com/windmill-labs/windmill/issues/7738)) ([ce23f21](https://github.com/windmill-labs/windmill/commit/ce23f21c0e0bc6365f616ace4c45fa341741c555))
* workspace dedicated workers ([#7741](https://github.com/windmill-labs/windmill/issues/7741)) ([60858d1](https://github.com/windmill-labs/windmill/commit/60858d1e20e68b83fddcdbfc0ff34decaff5d1c5))
### Bug Fixes
* forward teams error to client ([#7746](https://github.com/windmill-labs/windmill/issues/7746)) ([ca8dbc0](https://github.com/windmill-labs/windmill/commit/ca8dbc0676dda619aff6fab7f6ff05ed773738e0))
* indexer build error ([#7744](https://github.com/windmill-labs/windmill/issues/7744)) ([6679ecb](https://github.com/windmill-labs/windmill/commit/6679ecb9a2ead08d2252a64f2a27a6d539fa23e9))
* remove uuid-ossp extension requirement for RDS compatibility ([ad5293c](https://github.com/windmill-labs/windmill/commit/ad5293c0edacfaf1431a3639ef5ea32d9bd761b0))
* require AGENT_TOKEN and BASE_INTERNAL_URL for agent mode ([6c84a89](https://github.com/windmill-labs/windmill/commit/6c84a8905382e29a4bbe0ae947eda794bc4dc566))
* visibility bug on deployment UI (issue when renaming items) + add tracking of folders and resource types ([#7739](https://github.com/windmill-labs/windmill/issues/7739)) ([998f11a](https://github.com/windmill-labs/windmill/commit/998f11a10da45c6d933d8b78ca24ed4f55a53f3b))
## [1.621.2](https://github.com/windmill-labs/windmill/compare/v1.621.1...v1.621.2) (2026-01-29)
### Bug Fixes
* **cli:** revert findCodebase change that broke ../shared codebases ([#7740](https://github.com/windmill-labs/windmill/issues/7740)) ([20357f4](https://github.com/windmill-labs/windmill/commit/20357f41f55ce246220ec56ef257ea7d6ac82e3a))
* do not quit indexer when receiving handoff during pull ([#7659](https://github.com/windmill-labs/windmill/issues/7659)) ([8bb6b63](https://github.com/windmill-labs/windmill/commit/8bb6b6331b74d43b1ecfa08d3393254f54a94f87))
## [1.621.1](https://github.com/windmill-labs/windmill/compare/v1.621.0...v1.621.1) (2026-01-29)

View File

@@ -234,7 +234,7 @@ COPY --from=windmill_duckdb_ffi_internal_builder /windmill-duckdb-ffi-internal/t
COPY --from=denoland/deno:2.2.1 --chmod=755 /usr/bin/deno /usr/bin/deno
COPY --from=oven/bun:1.2.23 /usr/local/bin/bun /usr/bin/bun
COPY --from=oven/bun:1.3.8 /usr/local/bin/bun /usr/bin/bun
COPY --from=php:8.3.7-cli /usr/local/bin/php /usr/bin/php
COPY --from=composer:2.7.6 /usr/bin/composer /usr/bin/composer

241
README.md
View File

@@ -3,10 +3,10 @@
</p>
<p align=center>
Open-source developer infrastructure for internal tools (APIs, background jobs, workflows and UIs). Self-hostable alternative to Retool, Pipedream, Superblocks and a simplified Temporal with autogenerated UIs and custom UIs to trigger workflows and scripts as internal apps.
Open-source developer platform for internal code: APIs, background jobs, workflows and UIs. Self-hostable alternative to Retool, Pipedream, Superblocks and a simplified Temporal with autogenerated UIs and custom UIs to trigger workflows and scripts as internal apps.
<p align=center>
Scripts are turned into sharable UIs automatically, and can be composed together into flows or used into richer apps built with low-code. Supported script languages supported are: Python, TypeScript, Go, Bash, SQL, and GraphQL.
Scripts are turned into sharable UIs automatically, and can be composed together into flows or used into richer apps built with low-code. Supported languages: Python, TypeScript, Go, Bash, SQL, GraphQL, PowerShell, Rust, and more.
</p>
<p align="center">
@@ -36,75 +36,58 @@ Scripts are turned into sharable UIs automatically, and can be composed together
# Windmill - Developer platform for APIs, background jobs, workflows and UIs
Windmill is <b>fully open-sourced (AGPLv3)</b> and Windmill Labs offers
dedicated instance and commercial support and licenses.
Windmill is fully open-sourced (AGPLv3) and Windmill Labs offers dedicated instances and commercial support and licenses.
![Windmill Diagram](./imgs/stacks.svg)
https://github.com/windmill-labs/windmill/assets/122811744/0b132cd1-ee67-4505-822f-0c7ee7104252
https://github.com/user-attachments/assets/d80de1d9-64de-4d89-aacd-6df23fa81fc4
- [Windmill - Developer platform for APIs, background jobs, workflows and UIs](#windmill---developer-platform-for-apis-background-jobs-workflows-and-uis)
- [Main Concepts](#main-concepts)
- [Show me some actual script code](#show-me-some-actual-script-code)
- [CLI](#cli)
- [Running scripts locally](#running-scripts-locally)
- [Local Development](#local-development)
- [Stack](#stack)
- [Fastest Self-Hostable Workflow Engine](#fastest-self-hostable-workflow-engine)
- [Security](#security)
- [Sandboxing](#sandboxing)
- [Secrets, credentials and sensitive values](#secrets-credentials-and-sensitive-values)
- [Performance](#performance)
- [Architecture](#architecture)
- [How to self-host](#how-to-self-host)
- [Docker compose](#docker-compose)
- [Kubernetes (k8s) and Helm charts](#kubernetes-k8s-and-helm-charts)
- [Run from binaries](#run-from-binaries)
- [Kubernetes (Helm charts)](#kubernetes-helm-charts)
- [Cloud providers](#cloud-providers)
- [OAuth, SSO \& SMTP](#oauth-sso--smtp)
- [Commercial license](#commercial-license)
- [License](#license)
- [Integrations](#integrations)
- [Environment Variables](#environment-variables)
- [Run a local dev setup](#run-a-local-dev-setup)
- [only Frontend](#only-frontend)
- [Frontend only](#frontend-only)
- [Backend + Frontend](#backend--frontend)
- [Contributors](#contributors)
- [Copyright](#copyright)
## Main Concepts
1. Define a minimal and generic script in Python, TypeScript, Go or Bash that
solves a specific task. The code can be defined in the
[provided Web IDE](https://www.windmill.dev/docs/code_editor) or
[synchronized with your own GitHub repo](https://www.windmill.dev/docs/advanced/cli/sync)
(e.g. through
[VS Code](https://www.windmill.dev/docs/cli_local_dev/vscode-extension)
extension):
1. Define a minimal and generic script in Python, TypeScript, Go or Bash that solves a specific task. The code can be defined in the provided Web IDE or synchronized with your own GitHub repo (e.g. through VS Code extension): [provided Web IDE](https://www.windmill.dev/docs/code_editor) or [synchronized with your own GitHub repo](https://www.windmill.dev/docs/advanced/cli/sync) (e.g. through [VS Code](https://www.windmill.dev/docs/cli_local_dev/vscode-extension) extension):
![Step 1](./imgs/windmill-editor.png)
![Step 1](./imgs/windmill-editor.png)
2. Your scripts parameters are automatically parsed and
[generate a frontend](https://www.windmill.dev/docs/core_concepts/auto_generated_uis).
2. Your scripts parameters are automatically parsed and [generate a frontend](https://www.windmill.dev/docs/core_concepts/auto_generated_uis).
![Step 2](./imgs/windmill-run.png)
![Step 3](./imgs/windmill-result.png)
3. Make it [flow](https://www.windmill.dev/docs/flows/flow_editor)! You can
chain your scripts or scripts made by the community shared on
[WindmillHub](https://hub.windmill.dev).
3. Make it [flow](https://www.windmill.dev/docs/flows/flow_editor)! You can chain your scripts or scripts made by the community shared on [WindmillHub](https://hub.windmill.dev).
![Step 3](./imgs/windmill-flow.png)
![Step 3](./imgs/windmill-flow.png)
4. Build [complex UIs](https://www.windmill.dev/docs/apps/app_editor) on top of
your scripts and flows.
4. Build [complex UIs](https://www.windmill.dev/docs/apps/app_editor) on top of your scripts and flows.
![Step 4](./imgs/windmill-builder.png)
![Step 4](./imgs/windmill-builder.png)
Scripts and flows can also be triggered by a
[cron schedule](https://www.windmill.dev/docs/core_concepts/scheduling) (e.g.
'_/5 _ \* \* \*') or through
[webhooks](https://www.windmill.dev/docs/core_concepts/webhooks).
Scripts and flows can be triggered by [schedules](https://www.windmill.dev/docs/core_concepts/scheduling), [webhooks](https://www.windmill.dev/docs/core_concepts/webhooks), [HTTP routes](https://www.windmill.dev/docs/core_concepts/http_routing), [Kafka](https://www.windmill.dev/docs/core_concepts/kafka_triggers), [WebSockets](https://www.windmill.dev/docs/core_concepts/websocket_triggers), [emails](https://www.windmill.dev/docs/core_concepts/email_triggers), and more.
You can build your entire infra on top of Windmill!
Build your entire infra on top of Windmill!
## Show me some actual script code
@@ -144,43 +127,31 @@ export async function main(
}
```
## CLI
## Local Development
We have a powerful CLI to interact with the windmill platform and sync your
scripts from local files, GitHub repos and to run scripts and flows on the
instance from local commands. See
[more details](https://www.windmill.dev/docs/advanced/cli).
Windmill supports multiple ways to develop locally and sync with your instance:
![CLI Screencast](./cli/vhs/output/setup.gif)
| Tool | Description |
|------|-------------|
| **[CLI](https://www.windmill.dev/docs/advanced/cli)** | Sync scripts from local files or GitHub, run scripts/flows from the command line |
| **[VS Code Extension](https://www.windmill.dev/docs/cli_local_dev/vscode-extension)** | Edit and test scripts & flows directly from VS Code / Cursor with full IDE support |
| **[Git Sync](https://www.windmill.dev/docs/advanced/git_sync)** | Two-way sync between Windmill and your Git repository |
| **[Claude Code](https://www.windmill.dev/docs/core_concepts/ai_generation)** | AI-assisted development with Claude for scripts, flows, and apps |
### Running scripts locally
https://github.com/user-attachments/assets/c541c326-e9ae-4602-a09a-1989aaded1e9
You can run your script locally easily, you simply need to pass the right
environment variables for the `wmill` client library to fetch resources and
variables from your instance if necessary. See more:
<https://www.windmill.dev/docs/advanced/local_development>.
To develop & test locally scripts & flows, we recommend using the Windmill VS
Code extension: <https://www.windmill.dev/docs/cli_local_dev/vscode-extension>.
You can run scripts locally by passing the right environment variables for the `wmill` client library to fetch resources and variables from your instance. See [local development docs](https://www.windmill.dev/docs/advanced/local_development).
## Stack
- Postgres as the database.
- Backend in Rust with the following highly-available and horizontally scalable.
Architecture:
- Stateless API backend.
- Workers that pull jobs from a queue in Postgres (and later, Kafka or Redis.
Upvote [#173](#https://github.com/windmill-labs/windmill/issues/173) if
interested).
- Frontend in Svelte.
- Scripts executions are sandboxed using Google's
[nsjail](https://github.com/google/nsjail).
- Javascript runtime is the
[deno_core rust library](https://denolib.gitbook.io/guide/) (which itself uses
the [rusty_v8](https://github.com/denoland/rusty_v8) and hence V8 underneath).
- TypeScript runtime is Bun and deno.
- Python runtime is python3.
- Golang runtime is 1.19.1.
- **Database**: Postgres (compatible with Aurora, Cloud SQL, Neon, Azure PostgreSQL)
- **Backend**: Rust - stateless API servers and workers pulling jobs from a Postgres queue
- **Frontend**: Svelte 5
- **Sandboxing**: [nsjail](https://github.com/google/nsjail) and PID namespace isolation
- **Runtimes**:
- TypeScript/JavaScript: Bun (default) and Deno
- Python: python3 with uv for dependency management
- Go, Bash, PowerShell, PHP, Rust, C#, Java, Ansible
## Fastest Self-Hostable Workflow Engine
@@ -197,19 +168,10 @@ page.
## Security
### Sandboxing
- **Sandboxing**: [nsjail](https://github.com/google/nsjail) for filesystem/resource isolation, and PID namespace isolation (enabled by default) to prevent jobs from accessing worker process memory
- **Secrets**: One encryption key per workspace for credentials stored in Windmill's K/V store. We recommend encrypting the Postgres database as well.
Windmill can use [nsjail](https://github.com/google/nsjail). It is production
multi-tenant grade secure. Do not take our word for it, take
[fly.io's one](https://fly.io/blog/sandboxing-and-workload-isolation/).
### Secrets, credentials and sensitive values
There is one encryption key per workspace to encrypt the credentials and secrets
stored in Windmill's K/V store.
In addition, we strongly recommend that you encrypt the whole Postgres database.
That is what we do at <https://app.windmill.dev>.
See [Security documentation](https://www.windmill.dev/docs/advanced/security_isolation) for details.
## Performance
@@ -229,19 +191,13 @@ back to the database is ~50ms. A typical lightweight deno job will take around
## How to self-host
We only provide docker-compose setup here. For more advanced setups, like
compiling from source or using without a postgres super user, see
[Self-Host documentation](https://www.windmill.dev/docs/advanced/self_host).
For detailed setup options, see [Self-Host documentation](https://www.windmill.dev/docs/advanced/self_host).
### Docker compose
Windmill can be deployed using 3 files:
([docker-compose.yml](./docker-compose.yml), [Caddyfile](./Caddyfile) and a
[.env](./.env)) in a single command.
Deploy Windmill with 3 files ([docker-compose.yml](./docker-compose.yml), [Caddyfile](./Caddyfile), [.env](./.env)):
Make sure Docker is started, and run:
```
```bash
curl https://raw.githubusercontent.com/windmill-labs/windmill/main/docker-compose.yml -o docker-compose.yml
curl https://raw.githubusercontent.com/windmill-labs/windmill/main/Caddyfile -o Caddyfile
curl https://raw.githubusercontent.com/windmill-labs/windmill/main/.env -o .env
@@ -249,86 +205,45 @@ curl https://raw.githubusercontent.com/windmill-labs/windmill/main/.env -o .env
docker compose up -d
```
Go to http://localhost et voilà :)
Go to http://localhost - default credentials: `admin@windmill.dev` / `changeme`
The default super-admin user is: admin@windmill.dev / changeme.
**Using an external database**: Set `DATABASE_URL` in `.env` to point to your managed Postgres (AWS RDS, GCP Cloud SQL, Azure, Neon, etc.) and set db replicas to 0.
From there, you can follow the setup app and create other users.
More details in
[Self-Host Documention](https://www.windmill.dev/docs/advanced/self_host#docker).
### Kubernetes (k8s) and Helm charts
We publish helm charts at:
<https://github.com/windmill-labs/windmill-helm-charts>.
### Run from binaries
Each release includes the corresponding binaries for x86_64. You can simply
download the latest `windmill` binary using the following set of bash commands.
### Kubernetes (Helm charts)
```bash
BINARY_NAME='windmill-amd64' # or windmill-ee-amd64 for the enterprise edition
LATEST_RELEASE=$(curl -L -s -H 'Accept: application/json' https://github.com/windmill-labs/windmill/releases/latest)
LATEST_VERSION=$(echo $LATEST_RELEASE | sed -e 's/.*"tag_name":"\([^"]*\)".*/\1/')
ARTIFACT_URL="https://github.com/windmill-labs/windmill/releases/download/$LATEST_VERSION/$BINARY_NAME"
wget "$ARTIFACT_URL" -O windmill
helm repo add windmill https://windmill-labs.github.io/windmill-helm-charts/
helm install windmill-chart windmill/windmill --namespace=windmill --create-namespace
```
See [windmill-helm-charts](https://github.com/windmill-labs/windmill-helm-charts) for configuration options.
### Cloud providers
Windmill works on AWS (EKS/ECS), GCP, Azure, Ubicloud, Fly.io, Render.com, Hetzner, Digital Ocean, and others. Rule of thumb: 1 worker per 1vCPU and 1-2 GB RAM.
### OAuth, SSO & SMTP
Windmill Community Edition allows to configure the OAuth, SSO (including Google
Workspace SSO, Microsoft/Azure and Okta) directly from the UI in the superadmin
settings. Do note that there is a limit of 10 SSO users on the community
edition.
Configure OAuth and SSO (Google Workspace, Microsoft/Azure, Okta) directly from the superadmin UI. [See documentation](https://www.windmill.dev/docs/misc/setup_oauth).
[See documentation](https://www.windmill.dev/docs/misc/setup_oauth).
### License
### Commercial license
The Community Edition is free to use internally. For commercial redistribution or managed services, contact <sales@windmill.dev>. See [LICENSE](./LICENSE) and [Pricing](https://www.windmill.dev/pricing) for details.
See the [LICENSE](https://github.com/windmill-labs/windmill/blob/main/LICENSE)
file for the full license text.
The "Community Edition" of Windmill available in the docker images hosted under ghcr.io/windmill-labs/windmill and the github binary releases contains the files under the AGPLv3 and Apache 2 sources but also includes proprietary and non-public code and features which are not open source and under the following terms: Windmill Labs, Inc. grants a right to use all the features of the "Community Edition" for free without restrictions other than the limits and quotas set in the software and a right to distribute the community edition as is but not to sell, resell, serve Windmill as a managed service, modify or wrap under any form without an explicit agreement.
The "Community Edition" of Windmill available in the docker images hosted under
ghcr.io/windmill-labs/windmill and the github binary releases contains the files
under the AGPLv3 and Apache 2 sources but also includes proprietary and
non-public code and features which are not open source and under the following
terms: Windmill Labs, Inc. grants a right to use all the features of the
"Community Edition" for free without restrictions other than the limits and
quotas set in the software and a right to distribute the community edition as is
but not to sell, resell, serve Windmill as a managed service, modify or wrap
under any form without an explicit agreement.
The binary compilable from source code in this repository without the "enterprise" feature flag is open-source under the [LICENSE-AGPLv3](https://github.com/windmill-labs/windmill/blob/main/LICENSE-AGPL) License terms and conditions.
The binary compilable from source code in this repository without the
"enterprise" feature flag is open-source under the
[LICENSE-AGPLv3](https://github.com/windmill-labs/windmill/blob/main/LICENSE-AGPL)
License terms and conditions.
To [re-expose directly any Windmill parts to your users](https://www.windmill.dev/docs/misc/white_labelling) as a feature of your product, with the exception of iframed public Windmill "apps", or to build a feature on top of "Windmill Community Edition" that you sell commercially or embed in a distributable product or binary, you must get a commercial license. Contact us at <sales@windmill.dev> if you have any questions. To do the same from the binary compiled from the source code in this repository without the "enterprise" feature flag, you must comply with the AGPLv3 license terms and conditions or get a commercial license from Windmill Labs, Inc.
To
[re-expose directly any Windmill parts to your users](https://www.windmill.dev/docs/misc/white_labelling)
as a feature of your product, with the exception of iframed public Windmill
"apps", or to build a feature on top of "Windmill Community Edition" that you
sell commercially or embed in a distributable product or binary, you must get a
commercial license. Contact us at <sales@windmill.dev> if you have any
questions. To do the same from the binary compiled from the source code in this
repository without the "enterprise" feature flag, you must comply with the
AGPLv3 license terms and conditions or get a commercial license from Windmill
Labs, Inc.
To use Windmill "Community Edition" as is internally in your organization, or to
use its APIs as is, you do NOT need a commercial license.
To use Windmill "Community Edition" as is internally in your organization, or to use its APIs as is, you do NOT need a commercial license.
### Integrations
In Windmill, integrations are referred to as
[resources and resource types](https://www.windmill.dev/docs/core_concepts/resources_and_types).
Each Resource has a Resource Type that defines the schema that the resource
In Windmill, integrations are referred to as [resources and resource types](https://www.windmill.dev/docs/core_concepts/resources_and_types). Each Resource has a Resource Type that defines the schema that the resource
needs to implement.
On self-hosted instances, you might want to import all the approved resource
types from [WindmillHub](https://hub.windmill.dev). A setup script will prompt
you to have it being synced automatically everyday.
On self-hosted instances, you might want to import all the approved resource types from [WindmillHub](https://hub.windmill.dev). A setup script will prompt you to have it being synced automatically everyday.
## Environment Variables
@@ -369,30 +284,20 @@ you to have it being synced automatically everyday.
## Run a local dev setup
Using [Nix](./frontend/README_DEV.md#nix) (Recommended).
We recommend using [Nix](./frontend/README_DEV.md#nix). See [./frontend/README_DEV.md](./frontend/README_DEV.md) for all options.
See the [./frontend/README_DEV.md](./frontend/README_DEV.md) file for all
running options.
### Frontend only
### only Frontend
Uses the backend of <https://app.windmill.dev> with local frontend (hot-reload):
This will use the backend of <https://app.windmill.dev> but your own frontend
with hot-code reloading. Note that you will need to use a username / password
login due to CSRF checks using a different auth provider.
In the `frontend/` directory:
1. install the dependencies with `npm install` (or `pnpm install` or `yarn`)
2. generate the windmill client:
```
npm run generate-backend-client
## on mac use
npm run generate-backend-client-mac
```bash
cd frontend
npm install
npm run generate-backend-client # or generate-backend-client-mac on Mac
npm run dev
```
3. Run your dev server with `npm run dev`
4. Et voilà, windmill should be available at `http://localhost/`
Windmill available at `http://localhost/`
### Backend + Frontend
@@ -419,7 +324,7 @@ running options.
6. Go to `backend/`:
1. `env DATABASE_URL=<YOUR_DATABASE_URL> RUST_LOG=info cargo run`
2. You can specify any feature flag you want to enable, for example `cargo run --features python` to enable the python executor.
7. Et voilà, windmill should be available at `http://localhost:3000`
7. Windmill should be available at `http://localhost:3000`
## Contributors
@@ -429,4 +334,4 @@ running options.
## Copyright
Windmill Labs, Inc 2023
© 2023-2026 Windmill Labs, Inc.

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT flow_version.value AS \"value!: sqlx::types::Json<Box<sqlx::types::JsonRawValue>>\" \n FROM flow \n LEFT JOIN flow_version \n ON flow_version.id = flow.versions[array_upper(flow.versions, 1)]\n WHERE flow.path = $1 AND flow.workspace_id = $2",
"query": "SELECT flow_version.value AS \"value!: sqlx::types::Json<Box<sqlx::types::JsonRawValue>>\"\n FROM flow\n LEFT JOIN flow_version\n ON flow_version.id = flow.versions[array_upper(flow.versions, 1)]\n WHERE flow.path = $1 AND flow.workspace_id = $2",
"describe": {
"columns": [
{
@@ -19,5 +19,5 @@
false
]
},
"hash": "bbce3e1eae78c48409d4204cd6cb3b9db088f6e51bea5e74a494c4e9f4c3b78e"
"hash": "02bf9763298f301d4fc75490c070a0663142d4d23a2df007361622b94d4783e1"
}

View File

@@ -0,0 +1,37 @@
{
"db_name": "PostgreSQL",
"query": "\n DELETE FROM asset\n WHERE (workspace_id, path, kind) IN (\n SELECT workspace_id, path, kind FROM (\n SELECT a.workspace_id, a.path, a.kind, a.usage_kind, ROW_NUMBER() OVER (\n PARTITION BY a.workspace_id, a.path, a.kind\n ORDER BY a.created_at DESC\n ) as rn,\n limits.max_n\n FROM asset a\n INNER JOIN (\n SELECT * FROM UNNEST(\n $1::varchar[], \n $2::varchar[], \n $3::asset_kind[],\n $4::int[]\n ) AS t(workspace_id, path, kind, max_n)\n ) limits\n ON a.workspace_id = limits.workspace_id \n AND a.path = limits.path \n AND a.kind = limits.kind\n WHERE a.usage_kind = 'job'\n ) ranked\n WHERE rn > max_n\n )",
"describe": {
"columns": [],
"parameters": {
"Left": [
"VarcharArray",
"VarcharArray",
{
"Custom": {
"name": "asset_kind[]",
"kind": {
"Array": {
"Custom": {
"name": "asset_kind",
"kind": {
"Enum": [
"s3object",
"resource",
"variable",
"ducklake",
"datatable"
]
}
}
}
}
}
},
"Int4Array"
]
},
"nullable": []
},
"hash": "02e526146f3584cd599dec708e1be48db3b0cd1c74adbfa2e4039377daa016f0"
}

View File

@@ -46,11 +46,11 @@
]
},
"nullable": [
true,
true,
true,
true,
true,
false,
false,
false,
false,
false,
true,
true
]

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_diff\n (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)\n VALUES\n ('test-workspace', 'wm-fork-test-workspace', 'f/shared/old_name', 'resource', 0, 1, NULL),\n ('test-workspace', 'wm-fork-test-workspace', 'f/shared/new_name', 'resource', 1, 0, NULL)",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "14110b9c1b68cf29a6cbdfa737707a44bda831246736e60d63696c3227491adb"
}

View File

@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE flow SET value = $1\n WHERE workspace_id = 'test-workspace' AND path = 'f/shared/original_flow'",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Jsonb"
]
},
"nullable": []
},
"hash": "18272dde939afcd87464d74c03bc3c8ee4395919fbfa50565d8d800e3886911e"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT \n tag\n FROM \n v2_job\n WHERE \n id = $1\n ",
"query": "\n SELECT\n tag\n FROM\n v2_job\n WHERE\n id = $1\n ",
"describe": {
"columns": [
{
@@ -18,5 +18,5 @@
false
]
},
"hash": "49e2430af74ec10857e5df7f7e1ad1b53ba70bb51b0259a1f765f76db9b733ad"
"hash": "1d32bd9309bf2066399b446e8c47502a0ec72ffc07b22593311915ab5e98f80a"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_diff\n (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)\n VALUES\n ('test-workspace', 'wm-fork-test-workspace', 'f/shared/new_in_parent', 'script', 1, 0, NULL),\n ('test-workspace', 'wm-fork-test-workspace', 'new_type', 'resource_type', 1, 0, NULL)",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "1ed8d27979cd903cfd12f52461aada26f807236364fb56bf826228983bdab3ab"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE script SET archived = true\n WHERE workspace_id = 'wm-fork-test-workspace' AND path = 'f/shared/to_delete'",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "21941635afc295e5a42aceffce7aad910477c4537e9fe40b227e4ecc7eae76d3"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_diff\n (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)\n VALUES ('test-workspace', 'wm-fork-test-workspace', 'f/shared/new_in_fork', 'script', 0, 1, NULL)",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "2248e2c5ca8e6fe58704e45066477a1257b15cde7ea44962ce8566d6d7396add"
}

View File

@@ -1,23 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM v2_job_completed\n WHERE id IN (\n SELECT id FROM v2_job_completed\n WHERE completed_at <= now() - ($1::bigint::text || ' s')::interval\n ORDER BY completed_at ASC\n LIMIT $2\n FOR UPDATE SKIP LOCKED\n )\n RETURNING id",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Uuid"
}
],
"parameters": {
"Left": [
"Int8",
"Int8"
]
},
"nullable": [
false
]
},
"hash": "306e0156ee1541710c1c6512ecb4f61baeb3ae6f31ba3fd57a3ec485108a7f49"
}

View File

@@ -0,0 +1,53 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO asset (workspace_id, path, kind, usage_access_type, usage_path, usage_kind, asset_detection_kind, job_id)\n VALUES ($1, $2, $3, $4, $5, $6, 'static', NULL) ON CONFLICT DO NOTHING",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
{
"Custom": {
"name": "asset_kind",
"kind": {
"Enum": [
"s3object",
"resource",
"variable",
"ducklake",
"datatable"
]
}
}
},
{
"Custom": {
"name": "asset_access_type",
"kind": {
"Enum": [
"r",
"w",
"rw"
]
}
}
},
"Varchar",
{
"Custom": {
"name": "asset_usage_kind",
"kind": {
"Enum": [
"script",
"flow"
]
}
}
}
]
},
"nullable": []
},
"hash": "31eada57708ac3347d1e0a4fb5fd412a4a0c6046dbe4cf91849def2eea2e4c62"
}

View File

@@ -0,0 +1,32 @@
{
"db_name": "PostgreSQL",
"query": "SELECT has_changes, exists_in_source, exists_in_fork FROM workspace_diff\n WHERE path = 'f/shared/new_in_parent' AND kind = 'script' AND source_workspace_id = 'test-workspace'",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "has_changes",
"type_info": "Bool"
},
{
"ordinal": 1,
"name": "exists_in_source",
"type_info": "Bool"
},
{
"ordinal": 2,
"name": "exists_in_fork",
"type_info": "Bool"
}
],
"parameters": {
"Left": []
},
"nullable": [
true,
true,
true
]
},
"hash": "338f8f878aba9d9361b17329ee448ee4582194cbbc853d26abd285c259642c60"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE script\n SET content = 'def main(): return \"fork_modified\"', summary = 'Modified in fork'\n WHERE workspace_id = 'wm-fork-test-workspace' AND path = 'f/shared/to_modify_fork'",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "33ad6eb46a13dd49605fdb863a03602878c9d1f40f61e349e06698140133153f"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO script (workspace_id, path, hash, content, summary, description, language, created_by, created_at, archived, schema_validation, ws_error_handler_muted, deleted)\n VALUES\n ('test-workspace', 'f/shared/original_script', 12345, 'def main(): pass', 'Original', '', 'python3', 'test@windmill.dev', NOW(), false, false, false, false),\n ('test-workspace', 'f/shared/to_modify_parent', 22222, 'def main(): return 1', 'To modify in parent', '', 'python3', 'test@windmill.dev', NOW(), false, false, false, false),\n ('test-workspace', 'f/shared/to_modify_fork', 33333, 'def main(): return 2', 'To modify in fork', '', 'python3', 'test@windmill.dev', NOW(), false, false, false, false),\n ('test-workspace', 'f/shared/to_conflict', 44444, 'def main(): return 3', 'To conflict', '', 'python3', 'test@windmill.dev', NOW(), false, false, false, false),\n ('test-workspace', 'f/shared/to_delete', 55555, 'def main(): return 4', 'To delete', '', 'python3', 'test@windmill.dev', NOW(), false, false, false, false)",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "3439a44ea40ea748346af50bc1529dc8fd135c8cb54b7e39d03b326c30a0ab25"
}

View File

@@ -0,0 +1,42 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO asset (workspace_id, path, kind, usage_access_type, usage_path, usage_kind, asset_detection_kind, job_id)\n VALUES ($1, $2, $3, NULL, $4, $5, 'runtime', $6) ON CONFLICT DO NOTHING",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
{
"Custom": {
"name": "asset_kind",
"kind": {
"Enum": [
"s3object",
"resource",
"variable",
"ducklake",
"datatable"
]
}
}
},
"Varchar",
{
"Custom": {
"name": "asset_usage_kind",
"kind": {
"Enum": [
"script",
"flow"
]
}
}
},
"Uuid"
]
},
"nullable": []
},
"hash": "3c6bed058591ad5438428b8c154ba6ee737e59aa033d833843b54e9f58eef3f2"
}

View File

@@ -0,0 +1,24 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM v2_job_completed\n WHERE id IN (\n SELECT jc.id FROM v2_job_completed jc\n LEFT JOIN v2_job j ON j.id = jc.id\n WHERE jc.completed_at <= now() - ($1::bigint::text || ' s')::interval\n AND COALESCE(j.root_job, j.flow_innermost_root_job, jc.id) != ALL($3)\n ORDER BY jc.completed_at ASC\n LIMIT $2\n FOR UPDATE OF jc SKIP LOCKED\n )\n RETURNING id",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Uuid"
}
],
"parameters": {
"Left": [
"Int8",
"Int8",
"UuidArray"
]
},
"nullable": [
false
]
},
"hash": "45997fcb4d9d62c7f7011966bf59bdb86e12ce1d0c8e925e738d2645121a5c1f"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM asset WHERE workspace_id = $1 AND usage_kind = 'script' AND usage_path = (SELECT path FROM script WHERE hash = $2 AND workspace_id = $1) AND asset_detection_kind = 'static'",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Int8"
]
},
"nullable": []
},
"hash": "46395d77f3393a058e25ff4efb0b61c743c58418ba2ae1104bcfc99fae7925e4"
}

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT (large_file_storage IS NOT NULL\n AND large_file_storage != 'null'::jsonb\n AND jsonb_typeof(large_file_storage) = 'object') AS \"has_primary!\"\n FROM workspace_settings WHERE workspace_id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "has_primary!",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
null
]
},
"hash": "46f6a3665d11ef573e27ab14be8a80c78d7eb735bc9615ebf2668a5715d83349"
}

View File

@@ -0,0 +1,20 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id FROM app WHERE path = 'f/shared/dashboard' AND workspace_id = 'test-workspace'",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Int8"
}
],
"parameters": {
"Left": []
},
"nullable": [
false
]
},
"hash": "49e4bef9d5e366c179d33794472201bfca768ee5ead967e0232fa87a59b9747c"
}

View File

@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO agent_token_blacklist (token, expires_at, blacklisted_by)\n VALUES ($1, $2, $3)\n ON CONFLICT (token) DO UPDATE SET\n expires_at = EXCLUDED.expires_at,\n blacklisted_at = NOW(),\n blacklisted_by = EXCLUDED.blacklisted_by",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Timestamp",
"Varchar"
]
},
"nullable": []
},
"hash": "4c1dac4ddeca8f053abb35c833acd6993448d0327404cf1ad089ec3df6f2db35"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE script\n SET content = 'def main(): return \"modified\"', summary = 'Modified in parent'\n WHERE workspace_id = 'test-workspace' AND path = 'f/shared/to_modify_parent'",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "4ef7fcfa9a8962497c1eace31e59138cb8b252b207122a823185b36c44c10045"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_diff\n (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)\n VALUES\n ('test-workspace', 'wm-fork-test-workspace', 'f/shared/to_modify_fork', 'script', 0, 1, NULL),\n ('test-workspace', 'wm-fork-test-workspace', 'f/shared/resource_to_modify', 'resource', 0, 1, NULL),\n ('test-workspace', 'wm-fork-test-workspace', 'shared', 'folder', 0, 1, NULL)",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "56e6be86747e953e4eb75b434906bea7f471c22923ac29894feae9c199d78788"
}

View File

@@ -15,7 +15,7 @@
]
},
"nullable": [
null
true
]
},
"hash": "5a219a2532517869578c4504ff3153c43903f929ae5d62fbba12610f89c36d55"

View File

@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_diff (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)\n VALUES ($1, $2, $3, $4, 1, 0, NULL)\n ON CONFLICT (source_workspace_id, fork_workspace_id, path, kind)\n DO UPDATE SET\n ahead = workspace_diff.ahead + 1,\n has_changes = NULL",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar",
"Varchar"
]
},
"nullable": []
},
"hash": "5a3cf6f0958a559c147a37e29458b48eb7488446eb69adeddf2007ae7ae897a7"
}

View File

@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_diff (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)\n SELECT $1, unnest($2::varchar[]), $3, $4, 0, 1, NULL\n ON CONFLICT (source_workspace_id, fork_workspace_id, path, kind)\n DO UPDATE SET\n behind = workspace_diff.behind + 1,\n has_changes = NULL",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"VarcharArray",
"Varchar",
"Varchar"
]
},
"nullable": []
},
"hash": "5b3b4119469b7b110657926e6f99093c8fa5980ad061804cf73dc29151beac4b"
}

View File

@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO resource_type (workspace_id, name, schema, description, created_by)\n VALUES ('test-workspace', 'custom_db', $1, 'Custom DB type', 'test@windmill.dev')",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Jsonb"
]
},
"nullable": []
},
"hash": "6974521dec19adbc4c4497cc50e11a3f3be2a76a68287b3da2b999a925f972b4"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM asset WHERE workspace_id = $1 AND usage_kind = 'script' AND usage_path = $2 AND asset_detection_kind = 'static'",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": []
},
"hash": "752fc50d3374ee5ccdb49f10c11e50bbffeec1088364a64f0849fb9b25e11ef4"
}

View File

@@ -0,0 +1,20 @@
{
"db_name": "PostgreSQL",
"query": "SELECT has_changes FROM workspace_diff\n WHERE path = 'f/shared/original_script' AND kind = 'script' AND source_workspace_id = 'test-workspace'",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "has_changes",
"type_info": "Bool"
}
],
"parameters": {
"Left": []
},
"nullable": [
true
]
},
"hash": "77929b1ab186a01113436b5739240285dc111218efb4df8dbb0c72280a4c6a2c"
}

View File

@@ -0,0 +1,35 @@
{
"db_name": "PostgreSQL",
"query": "SELECT schema, description, format_extension\n FROM resource_type\n WHERE workspace_id = $1 AND name = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "schema",
"type_info": "Jsonb"
},
{
"ordinal": 1,
"name": "description",
"type_info": "Text"
},
{
"ordinal": 2,
"name": "format_extension",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
true,
true,
true
]
},
"hash": "7bc9fc05dbd162866bef1fdd3e7faeb50429881ed1bc962903f06e4b3d5f8d44"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT token, expires_at, blacklisted_at, blacklisted_by \n FROM agent_token_blacklist \n ORDER BY blacklisted_at DESC",
"query": "SELECT token, expires_at, blacklisted_at, blacklisted_by\n FROM agent_token_blacklist\n ORDER BY blacklisted_at DESC",
"describe": {
"columns": [
{
@@ -34,5 +34,5 @@
false
]
},
"hash": "1c5d3556fc8436ddd294f39c5431e1f501a821d6143c5d8aece20814237a6b86"
"hash": "7cbfad812eeb80cff00336697052f266693cf838d62a8b1e581c7239ec42095b"
}

View File

@@ -0,0 +1,34 @@
{
"db_name": "PostgreSQL",
"query": "SELECT\n jsonb_build_object(\n 'path', path,\n 'kind', kind,\n 'access_type', usage_access_type\n ) as \"list!: _\"\n FROM asset\n WHERE workspace_id = $1 AND usage_path = $2 AND usage_kind = $3\n AND asset_detection_kind = 'static'\n ORDER BY path, kind",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "list!: _",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Text",
"Text",
{
"Custom": {
"name": "asset_usage_kind",
"kind": {
"Enum": [
"script",
"flow"
]
}
}
}
]
},
"nullable": [
null
]
},
"hash": "7d73a51d5a83b11c1c4088678dd22ef369e9e26fafb75b351abfdef2b8a7a3e4"
}

View File

@@ -0,0 +1,41 @@
{
"db_name": "PostgreSQL",
"query": "SELECT display_name, owners, extra_perms, summary\n FROM folder\n WHERE workspace_id = $1 AND name = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "display_name",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "owners",
"type_info": "VarcharArray"
},
{
"ordinal": 2,
"name": "extra_perms",
"type_info": "Jsonb"
},
{
"ordinal": 3,
"name": "summary",
"type_info": "Text"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
false,
false,
true
]
},
"hash": "8039b459914ceefe0c0a31b97473a5522e47b4b2fd3ddeff8f221560bc9c6f57"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_diff\n (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)\n VALUES ('test-workspace', 'wm-fork-test-workspace', 'f/shared/lazy_test', 'script', 1, 0, NULL)\n ON CONFLICT DO NOTHING",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "85711709ee4c9684cd46af85c2d21124bfaa6aa07eadeb8d55b4b9b9e3073f10"
}

View File

@@ -0,0 +1,20 @@
{
"db_name": "PostgreSQL",
"query": "SELECT has_changes FROM workspace_diff\n WHERE path = 'f/shared/lazy_test' AND kind = 'script' AND source_workspace_id = 'test-workspace'",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "has_changes",
"type_info": "Bool"
}
],
"parameters": {
"Left": []
},
"nullable": [
true
]
},
"hash": "8c0596e2a1c7cf9eef4e4ce3249a8bdd6cb4ead841591d873e5ec8c9c214b6b5"
}

View File

@@ -0,0 +1,20 @@
{
"db_name": "PostgreSQL",
"query": "SELECT EXISTS(SELECT 1 FROM workspace WHERE id = 'wm-fork-test-workspace')",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "exists",
"type_info": "Bool"
}
],
"parameters": {
"Left": []
},
"nullable": [
null
]
},
"hash": "8d1920926cbdce897305c775b1d286fed8fa2f4258bde54b9d771c91923751ff"
}

View File

@@ -0,0 +1,26 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM asset WHERE workspace_id = $1 AND usage_path = $2 AND usage_kind = $3 AND asset_detection_kind = 'static'",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
{
"Custom": {
"name": "asset_usage_kind",
"kind": {
"Enum": [
"script",
"flow"
]
}
}
}
]
},
"nullable": []
},
"hash": "8ff8b53147d857db7829889ccbb59d2ca0492704a0b4a3cc8d1f4f733e0c794b"
}

View File

@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO resource_type (workspace_id, name, schema, description, created_by)\n VALUES ('test-workspace', 'new_type', $1, 'New type in parent', 'test@windmill.dev')",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Jsonb"
]
},
"nullable": []
},
"hash": "907025d53448ea70760420f90cd95ad1d93ef6d4ebe0295cdc1e70108320f6a9"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_diff\n (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)\n VALUES ('test-workspace', 'wm-fork-test-workspace', 'f/shared/original_script', 'script', 0, 0, NULL)",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "93def20ecf1bcbffe57c54bb12f0a5da22d9d2845d69e998cad8b68954b7a289"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE folder SET display_name = 'Modified Shared Folder'\n WHERE workspace_id = 'wm-fork-test-workspace' AND name = 'shared'",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "96406f7d7a42ec5a614415ed2fbf6f24291e869aa028ef3190390aaeadd10a46"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM asset WHERE workspace_id = $1 AND usage_kind = 'flow' AND usage_path = $2 AND asset_detection_kind = 'static'",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": []
},
"hash": "97101fae34227628efbcd72457a83a683745a6e2fb120329deba5117cc523bf2"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO script (workspace_id, path, hash, content, summary, description, language, created_by, created_at, archived, schema_validation, ws_error_handler_muted, deleted)\n VALUES ('wm-fork-test-workspace', 'f/shared/new_in_fork', 99999, 'def main(): return \"fork\"', 'New in fork', '', 'python3', 'test@windmill.dev', NOW(), false, false, false, false)",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "9775cbe9f8592743575b06ae60b4afdca78d46a5934521314bb7a7d8688f9a35"
}

View File

@@ -0,0 +1,24 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO worker_ping (worker_instance, worker, ip, custom_tags, worker_group, dedicated_worker, dedicated_workers, wm_version, vcpus, memory, job_isolation) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11) ON CONFLICT (worker)\n DO UPDATE set ip = EXCLUDED.ip, custom_tags = EXCLUDED.custom_tags, worker_group = EXCLUDED.worker_group, dedicated_workers = EXCLUDED.dedicated_workers",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar",
"TextArray",
"Varchar",
"Varchar",
"TextArray",
"Varchar",
"Int8",
"Int8",
"Text"
]
},
"nullable": []
},
"hash": "97c61b6a9a5112ea484565236959a544511d5d501fb737da8110a8725b883465"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO app (workspace_id, path, summary, policy, versions, extra_perms, draft_only)\n VALUES ('test-workspace', 'f/shared/dashboard', 'Dashboard app', '{}', ARRAY[1::bigint], '{}', false)",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "9a7f4786fc29ed2b561d9eb96c274c66da29fdfb3e862e06c10c5deb4a2b5771"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE resource SET path = 'f/shared/new_name'\n WHERE workspace_id = 'test-workspace' AND path = 'f/shared/old_name'",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "9b7d0f980267e62fb50545570e83daae65e4458a88b53b65870523b64d183231"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE variable SET value = 'modified_value'\n WHERE workspace_id = 'test-workspace' AND path = 'f/shared/variable_to_modify'",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "9df747dec3f3c245b1cd3a5039f0a4a69d0667e328ad838c1a2f9bb9fee2d121"
}

View File

@@ -0,0 +1,37 @@
{
"db_name": "PostgreSQL",
"query": "\n DELETE FROM asset\n WHERE id IN (\n SELECT id FROM (\n SELECT a.id, ROW_NUMBER() OVER (\n PARTITION BY a.workspace_id, a.path, a.kind\n ORDER BY a.created_at DESC\n ) as rn,\n limits.max_n\n FROM asset a\n INNER JOIN (\n SELECT * FROM UNNEST(\n $1::varchar[], \n $2::varchar[], \n $3::asset_kind[],\n $4::int[]\n ) AS t(workspace_id, path, kind, max_n)\n ) limits\n ON a.workspace_id = limits.workspace_id \n AND a.path = limits.path \n AND a.kind = limits.kind\n WHERE a.asset_detection_kind = 'runtime'\n ) ranked\n WHERE rn > max_n\n )",
"describe": {
"columns": [],
"parameters": {
"Left": [
"VarcharArray",
"VarcharArray",
{
"Custom": {
"name": "asset_kind[]",
"kind": {
"Array": {
"Custom": {
"name": "asset_kind",
"kind": {
"Enum": [
"s3object",
"resource",
"variable",
"ducklake",
"datatable"
]
}
}
}
}
}
},
"Int4Array"
]
},
"nullable": []
},
"hash": "a14d43c1baebcdf7e8cebcb7c752f78a2212fe55d4247c2321b37d182d2a4386"
}

View File

@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO resource (workspace_id, path, value, resource_type, description, created_by)\n VALUES\n ('test-workspace', 'f/shared/db_config', $1, 'postgresql', '', 'test@windmill.dev'),\n ('test-workspace', 'f/shared/old_name', $2, 'generic', '', 'test@windmill.dev'),\n ('test-workspace', 'f/shared/resource_to_modify', $3, 'generic', '', 'test@windmill.dev')",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Jsonb",
"Jsonb",
"Jsonb"
]
},
"nullable": []
},
"hash": "a17539823e5f2b0ffa8d3f270801d6d41db9e2ef4c33f09565a858d6c55cc3a9"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_diff\n (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)\n VALUES\n ('test-workspace', 'wm-fork-test-workspace', 'f/shared/original_flow', 'flow', 1, 1, NULL),\n ('test-workspace', 'wm-fork-test-workspace', 'f/shared/to_conflict', 'script', 1, 1, NULL)",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "a1f8d5c97404166603de2fe214d965243ddf8a37685a7f62b593718d195dab06"
}

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT\n jsonb_strip_nulls(jsonb_build_object(\n 'path', asset.path,\n 'kind', asset.kind,\n 'usages', ARRAY_AGG(DISTINCT jsonb_build_object(\n 'path', asset.usage_path,\n 'kind', asset.usage_kind,\n 'access_type', asset.usage_access_type,\n 'detection_kinds', (\n SELECT ARRAY_AGG(DISTINCT a2.asset_detection_kind)\n FROM asset a2\n WHERE a2.workspace_id = asset.workspace_id\n AND a2.path = asset.path\n AND a2.kind = asset.kind\n AND a2.usage_path = asset.usage_path\n AND a2.usage_kind = asset.usage_kind\n )\n )),\n 'metadata', (CASE\n WHEN asset.kind = 'resource' THEN\n jsonb_build_object('resource_type', resource.resource_type)\n ELSE\n NULL\n END\n )\n )) as \"list!: _\"\n FROM asset\n LEFT JOIN resource ON asset.kind = 'resource'\n AND array_to_string((string_to_array(asset.path, '/'))[1:3], '/') = resource.path -- With specific table, asset path can be e.g u/diego/pg_db/table_name\n AND resource.workspace_id = $1\n WHERE asset.workspace_id = $1\n AND (asset.kind <> 'resource' OR resource.path IS NOT NULL)\n AND (asset.usage_kind <> 'flow' OR asset.usage_path = ANY(SELECT path FROM flow WHERE workspace_id = $1))\n AND (asset.usage_kind <> 'script' OR asset.usage_path = ANY(SELECT path FROM script WHERE workspace_id = $1))\n GROUP BY asset.path, asset.kind, resource.resource_type\n ORDER BY asset.path, asset.kind",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "list!: _",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
null
]
},
"hash": "a21337dac33ef7a1aff1c6797a3ca47fed231f542d5bd8725d6c86b8dd1a2535"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO script (workspace_id, path, hash, content, summary, description, language, created_by, created_at, archived, schema_validation, ws_error_handler_muted, deleted)\n VALUES ('test-workspace', 'f/shared/new_in_parent', 54321, 'def main(): return \"new\"', 'New in parent', '', 'python3', 'test@windmill.dev', NOW(), false, false, false, false)",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "a962567ecfcbf251a8fcc12db894580b53f209d5f9f771bfaf0453b8d704bf31"
}

View File

@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO flow (workspace_id, path, summary, description, value, schema, edited_by, edited_at, archived)\n VALUES ('test-workspace', 'f/shared/original_flow', 'Flow summary', '', $1, NULL, 'test@windmill.dev', NOW(), false)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Jsonb"
]
},
"nullable": []
},
"hash": "b32c7c24440137d6749e1fc3095512b0f5bc8b81e0b644b01256398664ecf9b1"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT name FROM resource_type\n WHERE workspace_id = $1 AND name = ANY($2)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "name",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"TextArray"
]
},
"nullable": [
false
]
},
"hash": "b95b6ee787c590dd21969965a687fef33fc9fbc80ca87f96aced906570e28c18"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO folder (workspace_id, name, display_name, owners, summary, created_by)\n VALUES ('test-workspace', 'shared', 'Shared Folder', ARRAY['test@windmill.dev']::varchar[], 'Test folder', 'test@windmill.dev')",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "ba3149ec0aa62dba41c459a1bf5be86aebccee68172abefe82f4bb76fc7a80ee"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO app_version (app_id, value, created_by, created_at)\n VALUES ($1, $2, 'test@windmill.dev', NOW())",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Int8",
"Json"
]
},
"nullable": []
},
"hash": "bda182047ae37e263b0011248d86c1b20d8b7908bb8a8ae4c5ed83b51db5446a"
}

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT q.id FROM v2_job_queue q\n JOIN v2_job j ON j.id = q.id\n WHERE j.parent_job IS NULL\n AND j.created_at <= now() - ($1::bigint::text || ' s')::interval",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Uuid"
}
],
"parameters": {
"Left": [
"Int8"
]
},
"nullable": [
false
]
},
"hash": "c01a94ec75990c8fb4068485c91211af295a1d2630861bf53a33966abc4562ec"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO deployment_metadata (workspace_id, path, script_hash, callback_job_ids, deployment_msg) VALUES ($1, $2, $3, $4, $5) \n ON CONFLICT (workspace_id, script_hash) WHERE script_hash IS NOT NULL DO UPDATE SET callback_job_ids = EXCLUDED.callback_job_ids, deployment_msg = EXCLUDED.deployment_msg",
"query": "INSERT INTO deployment_metadata (workspace_id, path, script_hash, callback_job_ids, deployment_msg) VALUES ($1, $2, $3, $4, $5)\n ON CONFLICT (workspace_id, script_hash) WHERE script_hash IS NOT NULL DO UPDATE SET callback_job_ids = EXCLUDED.callback_job_ids, deployment_msg = EXCLUDED.deployment_msg",
"describe": {
"columns": [],
"parameters": {
@@ -14,5 +14,5 @@
},
"nullable": []
},
"hash": "8e750d4b3af9b5844c11b1b92741f2ee9d2d3412d4a2c96c6ccc87ec1c382384"
"hash": "c64288c867ba944e834a44c5a6af7231efd899a148d3607316a5537f4e7b031c"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_diff\n (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)\n VALUES\n ('test-workspace', 'wm-fork-test-workspace', 'f/shared/to_modify_parent', 'script', 1, 0, NULL),\n ('test-workspace', 'wm-fork-test-workspace', 'f/shared/dashboard', 'app', 1, 0, NULL),\n ('test-workspace', 'wm-fork-test-workspace', 'f/shared/variable_to_modify', 'variable', 1, 0, NULL)",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "c925a52d90231f9e6f3af652ddfddd67bff8c3d306794e3583af86818b7054d0"
}

View File

@@ -1,16 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO agent_token_blacklist (token, expires_at, blacklisted_by) \n VALUES ($1, $2, $3) \n ON CONFLICT (token) DO UPDATE SET \n expires_at = EXCLUDED.expires_at,\n blacklisted_at = NOW(),\n blacklisted_by = EXCLUDED.blacklisted_by",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Timestamp",
"Varchar"
]
},
"nullable": []
},
"hash": "c9c040ec228a8fe4fda08439420141bee63339d4e3d5e2d68aabb12009f691c6"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT token, expires_at, blacklisted_at, blacklisted_by \n FROM agent_token_blacklist \n WHERE expires_at > $1 \n ORDER BY blacklisted_at DESC",
"query": "SELECT token, expires_at, blacklisted_at, blacklisted_by\n FROM agent_token_blacklist\n WHERE expires_at > $1\n ORDER BY blacklisted_at DESC",
"describe": {
"columns": [
{
@@ -36,5 +36,5 @@
false
]
},
"hash": "d56722c25877222af9affd5da5bb83b28fa8cbb528a2cfc90684cb10a69e4375"
"hash": "d20d6c43b16b762fb4cdb2cafe1fe9a2920124f398fca5bd54cb805b03b94763"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE app SET summary = 'Modified dashboard app'\n WHERE workspace_id = 'test-workspace' AND path = 'f/shared/dashboard'",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "d6ce9016308da0f20c2118c047349d8110d899121c9c00795c55217b9f1886c2"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO variable (workspace_id, path, value, is_secret, description)\n VALUES\n ('test-workspace', 'f/shared/api_key', 'secret123', false, 'Test key'),\n ('test-workspace', 'f/shared/variable_to_modify', 'original', false, 'To modify')",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "d8c5c64006cdd52570bbc56b0fa5df9b597d9a45353391411efa86959a7512bc"
}

View File

@@ -0,0 +1,37 @@
{
"db_name": "PostgreSQL",
"query": "SELECT COUNT(DISTINCT asset.job_id)::bigint as \"count!\"\n FROM asset\n WHERE asset.workspace_id = $1\n AND asset.path = $2\n AND asset.kind = $3\n AND asset.asset_detection_kind = 'runtime'\n AND asset.job_id IS NOT NULL",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "count!",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text",
"Text",
{
"Custom": {
"name": "asset_kind",
"kind": {
"Enum": [
"s3object",
"resource",
"variable",
"ducklake",
"datatable"
]
}
}
}
]
},
"nullable": [
null
]
},
"hash": "df290155c071e823a1d60573a3cba932d070ed730a37450cd30ba9b90205ed3b"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT name FROM folder\n WHERE workspace_id = $1 AND name = ANY($2)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "name",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"TextArray"
]
},
"nullable": [
false
]
},
"hash": "df804e2a8795af329aa84ccfeb90329383524232276908af181f6ecb3c8fc804"
}

View File

@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE flow SET value = $1\n WHERE workspace_id = 'wm-fork-test-workspace' AND path = 'f/shared/original_flow'",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Jsonb"
]
},
"nullable": []
},
"hash": "e42bb9155be93cbc750702cf7f9a3c54d1a236421861880fe3749052bd2ab43f"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_diff\n (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)\n VALUES ('test-workspace', 'wm-fork-test-workspace', 'f/shared/to_delete', 'script', 1, 0, NULL)",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "e881a777b8e67c76cf458828a02f7d5234ae76da567195f5da11cd11d7ebbc05"
}

View File

@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE resource SET value = $1\n WHERE workspace_id = 'wm-fork-test-workspace' AND path = 'f/shared/resource_to_modify'",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Jsonb"
]
},
"nullable": []
},
"hash": "e9e6c8ba034b4f80fada2e07e500a78983aeb1a7b1185b5267229c3197d6f714"
}

View File

@@ -1,23 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO worker_ping (worker_instance, worker, ip, custom_tags, worker_group, dedicated_worker, wm_version, vcpus, memory, job_isolation) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10) ON CONFLICT (worker)\n DO UPDATE set ip = EXCLUDED.ip, custom_tags = EXCLUDED.custom_tags, worker_group = EXCLUDED.worker_group",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar",
"TextArray",
"Varchar",
"Varchar",
"Varchar",
"Int8",
"Int8",
"Text"
]
},
"nullable": []
},
"hash": "ed90b9fcc57f530bab2d2d7426795bb358c91ebdd9dab50d7e3d2fdce63b947c"
}

View File

@@ -0,0 +1,63 @@
{
"db_name": "PostgreSQL",
"query": "SELECT DISTINCT\n v2_job.id,\n v2_job.created_at,\n v2_job.created_by,\n v2_job.runnable_path,\n CASE\n WHEN v2_job_completed.id IS NOT NULL THEN v2_job_completed.status::text\n ELSE NULL\n END as status\n FROM asset\n INNER JOIN v2_job ON asset.job_id = v2_job.id\n LEFT JOIN v2_job_completed ON v2_job.id = v2_job_completed.id\n WHERE asset.workspace_id = $1\n AND asset.path = $2\n AND asset.kind = $3\n AND asset.asset_detection_kind = 'runtime'\n AND asset.job_id IS NOT NULL\n ORDER BY v2_job.created_at DESC\n LIMIT $4 OFFSET $5",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Uuid"
},
{
"ordinal": 1,
"name": "created_at",
"type_info": "Timestamptz"
},
{
"ordinal": 2,
"name": "created_by",
"type_info": "Varchar"
},
{
"ordinal": 3,
"name": "runnable_path",
"type_info": "Varchar"
},
{
"ordinal": 4,
"name": "status",
"type_info": "Text"
}
],
"parameters": {
"Left": [
"Text",
"Text",
{
"Custom": {
"name": "asset_kind",
"kind": {
"Enum": [
"s3object",
"resource",
"variable",
"ducklake",
"datatable"
]
}
}
},
"Int8",
"Int8"
]
},
"nullable": [
false,
false,
false,
true,
null
]
},
"hash": "fb6656bc8fbd2e1b2c59f9ecbab55e5cdccdd7c99af0b61946694c6e03e7f636"
}

View File

@@ -1,288 +0,0 @@
▐▛███▜▌ Claude Code v2.1.23
▝▜█████▛▘ Opus 4.5 · Claude Max
▘▘ ▝▝ ~/windmill/backend
✻ Conversation compacted (ctrl+o for history)
● Let me read the current state of the test file to understand the compilation
error and fix it.
● Read 1 file (ctrl+o to expand)
● Let me check the common test module to understand the run_until_complete
implementation and the Send trait issue.
● Read 1 file (ctrl+o to expand)
● Now I understand the issue. The run_until_complete function uses non-Send
types internally (like PgListener), so it can't be spawned with tokio::spawn.
The correct approach for stress testing is to push all jobs to the queue
first, then let a single worker process them concurrently. This is actually
more realistic since it tests the actual concurrent V8 execution.
Let me fix the stress tests:
● Update(tests/nativets_tests.rs)
 Added 88 lines, removed 88 lines
743 // Stress Tests - Concurrent Execution
744 // ===========================================================
=================
745
746 +/// Push multiple jobs and process them with a single worker.
747 +/// This tests concurrent V8 isolate execution more realistica
+lly.
748 +async fn push_jobs_and_wait_for_completion(
749 + db: &Pool<Postgres>,
750 + payloads: Vec<JobPayload>,
751 + port: u16,
752 +) -> Vec<CompletedJob> {
753 + use windmill_common::worker::Connection;
754 +
755 + // Push all jobs first
756 + let mut job_ids = Vec::with_capacity(payloads.len());
757 + for payload in payloads {
758 + let uuid = RunJob::from(payload).push(db).await;
759 + job_ids.push(uuid);
760 + }
761 +
762 + // Listen for completed jobs
763 + let mut listener = listen_for_completed_jobs(db).await;
764 +
765 + // Start the worker and wait for all jobs to complete
766 + let job_count = job_ids.len();
767 + let db_clone = db.clone();
768 + let job_ids_clone = job_ids.clone();
769 +
770 + in_test_worker(
771 + Connection::Sql(db.clone()),
772 + async move {
773 + let mut completed_count = 0;
774 + while completed_count < job_count {
775 + if let Some(uuid) = listener.next().await {
776 + if job_ids_clone.contains(&uuid) {
777 + completed_count += 1;
778 + }
779 + }
780 + }
781 + },
782 + port,
783 + )
784 + .await;
785 +
786 + // Fetch all completed jobs
787 + let mut results = Vec::with_capacity(job_ids.len());
788 + for uuid in job_ids {
789 + results.push(completed_job(uuid, db).await);
790 + }
791 + results
792 +}
793 +
794 #[sqlx::test(fixtures("base"))]
795 async fn test_stress_concurrent_fetch(db: Pool<Postgres>) -> a
nyhow::Result<()> {
796 initialize_tracing().await;
...
806
807 const NUM_CONCURRENT: usize = 20;
808
761 - let barrier = Arc::new(Barrier::new(NUM_CONCURRENT));
762 - let mut handles = Vec::new();
763 -
809 + // Create all job payloads
810 + let mut payloads = Vec::with_capacity(NUM_CONCURRENT);
811 for i in 0..NUM_CONCURRENT {
765 - let db = db.clone();
766 - let mock_url = mock.url("/counter/increment");
767 - let barrier = barrier.clone();
768 -
769 - let handle = tokio::spawn(async move {
770 - // Wait for all tasks to be ready
771 - barrier.wait().await;
772 -
773 - let code = format!(r#"
812 + let code = format!(r#"
813 export async function main() {{
814 const response = await fetch("{}", {{ method: "POST" }});
776 - return await response.json();
815 + const data = await response.json();
816 + return {{ ...data, jobIndex: {} }};
817 }}
778 -"#, mock_url);
779 -
780 - let result = run_nativets(&db, &code, port).await;
781 - (i, result.success, result.json_result())
782 - });
783 - handles.push(handle);
818 +"#, mock.url("/counter/increment"), i);
819 + payloads.push(nativets_code(&code));
820 }
821
786 - // Wait for all jobs to complete
787 - let results: Vec<_> = futures::future::join_all(handles)
788 - .await
789 - .into_iter()
790 - .map(|r| r.unwrap())
791 - .collect();
822 + // Push all jobs and wait for completion
823 + let results = push_jobs_and_wait_for_completion(&db, paylo
+ads, port).await;
824
825 // Verify all succeeded
794 - for (i, success, _) in &results {
795 - assert!(success, "Job {} should succeed", i);
826 + for (i, result) in results.iter().enumerate() {
827 + assert!(result.success, "Job {} should succeed", i);
828 }
829
830 // Verify counter was incremented NUM_CONCURRENT times
...
849
850 const NUM_CONCURRENT: usize = 10;
851
820 - let barrier = Arc::new(Barrier::new(NUM_CONCURRENT));
821 - let mut handles = Vec::new();
822 -
852 + // Create all job payloads with CPU-bound computation
853 + let mut payloads = Vec::with_capacity(NUM_CONCURRENT);
854 for i in 0..NUM_CONCURRENT {
824 - let db = db.clone();
825 - let barrier = barrier.clone();
826 -
827 - let handle = tokio::spawn(async move {
828 - barrier.wait().await;
829 -
830 - // CPU-bound computation
831 - let code = r#"
832 -export async function main() {
855 + let code = format!(r#"
856 +export async function main() {{
857 // Compute fibonacci iteratively
834 - function fib(n: number): number {
858 + function fib(n: number): number {{
859 let [a, b] = [0, 1];
836 - for (let i = 0; i < n; i++) {
860 + for (let i = 0; i < n; i++) {{
861 [a, b] = [b, a + b];
838 - }
862 + }}
863 return a;
840 - }
864 + }}
865
866 // Do some work
867 const results = [];
844 - for (let i = 0; i < 100; i++) {
868 + for (let i = 0; i < 100; i++) {{
869 results.push(fib(20));
846 - }
870 + }}
871
848 - return { computed: results.length, sample: results[0] };
849 -}
850 -"#;
851 -
852 - let result = run_nativets(&db, code, port).await;
853 - (i, result.success, result.json_result())
854 - });
855 - handles.push(handle);
872 + return {{ computed: results.length, sample: results[0], jo
+bIndex: {} }};
873 +}}
874 +"#, i);
875 + payloads.push(nativets_code(&code));
876 }
877
858 - let results: Vec<_> = futures::future::join_all(handles)
859 - .await
860 - .into_iter()
861 - .map(|r| r.unwrap())
862 - .collect();
878 + // Push all jobs and wait for completion
879 + let results = push_jobs_and_wait_for_completion(&db, paylo
+ads, port).await;
880
864 - for (i, success, result) in &results {
865 - assert!(success, "Job {} should succeed", i);
866 - let json = result.as_ref().unwrap();
881 + for (i, result) in results.iter().enumerate() {
882 + assert!(result.success, "Job {} should succeed", i);
883 + let json = result.json_result().unwrap();
884 assert_eq!(json["computed"], 100);
885 assert_eq!(json["sample"], 6765); // fib(20)
886 }
...
897
898 const NUM_CONCURRENT: usize = 15;
899
883 - let barrier = Arc::new(Barrier::new(NUM_CONCURRENT));
884 - let mut handles = Vec::new();
885 -
900 + // Create all job payloads with mixed workload
901 + let mut payloads = Vec::with_capacity(NUM_CONCURRENT);
902 for i in 0..NUM_CONCURRENT {
887 - let db = db.clone();
888 - let mock_url = mock.url("");
889 - let barrier = barrier.clone();
890 -
891 - let handle = tokio::spawn(async move {
892 - barrier.wait().await;
893 -
894 - // Mixed workload: fetch + computation
895 - let code = format!(r#"
903 + let code = format!(r#"
904 export async function main() {{
905 // Fetch data
906 const response = await fetch("{}/array/50");
...
928 jobIndex: {}
929 }};
930 }}
923 -"#, mock_url, i);
924 -
925 - let result = run_nativets(&db, &code, port).await;
926 - (i, result.success, result.json_result())
927 - });
928 - handles.push(handle);
931 +"#, mock.url(""), i);
932 + payloads.push(nativets_code(&code));
933 }
934
931 - let results: Vec<_> = futures::future::join_all(handles)
932 - .await
933 - .into_iter()
934 - .map(|r| r.unwrap())
935 - .collect();
935 + // Push all jobs and wait for completion
936 + let results = push_jobs_and_wait_for_completion(&db, paylo
+ads, port).await;
937
937 - for (i, success, result) in &results {
938 - assert!(success, "Job {} should succeed", i);
939 - let json = result.as_ref().unwrap();
938 + for result in &results {
939 + assert!(result.success, "Job should succeed");
940 + let json = result.json_result().unwrap();
941 assert_eq!(json["primeCount"], 25); // Primes under 10
0
941 - assert_eq!(json["jobIndex"], *i);
942 }
943
944 mock.shutdown().await;
● Now let me update the imports to remove unused ones and add the required ones:
● Update(tests/nativets_tests.rs)
 Removed 3 lines
21 use sqlx::{Pool, Postgres};
22 use std::net::SocketAddr;
23 use std::sync::atomic::{AtomicUsize, Ordering};
24 -use std::sync::Arc;
25 -use tokio::sync::Barrier;
26 -use uuid::Uuid;
24 use windmill_common::jobs::{JobPayload, RawCode};
25 use windmill_common::scripts::ScriptLang;
26
● Let me compile the test to check for any remaining issues:

View File

@@ -1,22 +1,61 @@
# Backend Development (Rust)
## Core Principles
## Project Structure
- Follow @rust-best-practices.mdc for detailed guidelines
- Database schema reference: @summarized_schema.txt
- The API routes prefixes are all listed in windmill-api/src/lib.rs
- This repository is the open source side of the project. The enterprise files (\*\_ee.rs) are in the `windmill-ee-private` folder (a sibling directory). Those files are symlinked into their corresponding locations within each crate's `src/` directory.
Windmill uses a workspace-based architecture with multiple crates:
## JSON Handling
- **windmill-api**: API server functionality
- **windmill-worker**: Job execution
- **windmill-common**: Shared code used by all crates
- **windmill-queue**: Job & flow queuing
- **windmill-audit**: Audit logging
- Other specialized crates (git-sync, autoscaling, etc.)
- **Prefer `Box<serde_json::value::RawValue>` over `serde_json::Value`** when possible, especially:
- When storing JSON in the database (JSONB columns)
- When passing JSON through without modification
- When the JSON structure doesn't need to be inspected or manipulated
- This avoids unnecessary parsing/serialization overhead and preserves the original JSON format
- Use `serde_json::Value` only when you need to inspect, modify, or construct JSON programmatically
## Key References (MUST FOLLOW THESE)
## Adding New Features
- You MUST follow best-practices by using the `rust-backend` skill, everytime you write RUST code.
- When working with the database: read `summarized_schema.txt` before starting
- When working with the API routes: you can read `windmill-api/src/lib.rs` to get started
1. Update database schema with migration if necessary
2. Update backend/windmill-api/openapi.yaml after modifying API endpoints
## Adding New Code
### Module Organization
- Place new code in the appropriate crate based on functionality
- For API endpoints, create or modify files in `windmill-api/src/` organized by domain
- For shared functionality, use `windmill-common/src/`
- Follow existing patterns for file structure and organization
### API Endpoints
- Follow existing patterns in the `windmill-api` crate
- Use axum's routing system and extractors
- Update `backend/windmill-api/openapi.yaml` after modifying API endpoints
### Database Changes
- Update database schema with migration if necessary
- Use `sqlx` for database operations with prepared statements
- Use transactions for multi-step operations
## Enterprise Features
- Enterprise files use the `*_ee.rs` suffix
- Enterprise source is in `windmill-ee-private` folder (sibling directory), symlinked into each crate's `src/`
- Use feature flags: `#[cfg(feature = "enterprise")]`
- Isolate enterprise code in separate modules
## Testing
- Write unit tests for core functionality
- Use the `#[cfg(test)]` module for test code
- For database tests, use the existing test utilities
## Common Crates
- **tokio**: Async runtime
- **axum**: Web server and routing
- **sqlx**: Database operations
- **serde**: Serialization/deserialization
- **tracing**: Logging and diagnostics
- **reqwest**: HTTP client

View File

@@ -0,0 +1,97 @@
# Backend Compilation Optimization
## Summary
Feature-gated heavy dependencies that were compiled by default but only used behind enterprise/EE feature flags. This reduces the default build from **761 crates to 609 crates** (20% reduction).
## Changes
### Root `Cargo.toml`
- Removed 12 unused direct dependencies: `kube`, `k8s-openapi`, `aws-sigv4`, `aws-sdk-config`, `opentelemetry-proto`, `systemstat`, `globset`, `libloading`, `bitflags`, `memchr`, `quote`, `pep440_rs`
### `windmill-worker/Cargo.toml`
- Made optional (only needed for EE OTEL tracing proxy): `hudsucker`, `hyper-http-proxy`, `hyper-tls`, `hyper-util`, `rcgen`, `opentelemetry-proto`, `prost`
- Made optional (only needed for EE features): `aws-config`, `aws-credential-types`, `aws-smithy-types`
- Created `otel_proxy` feature to group the OTEL proxy deps
- Updated `private` feature to include `otel_proxy`
### `windmill-common/Cargo.toml`
- Made optional: `aws-config`, `aws-credential-types`, `aws-smithy-types`, `systemstat`, `globset`
- Added AWS deps to `private`, `parquet`, `aws_auth`, `bedrock` features
- Added `systemstat` to `private` feature
- Added `globset` to `parquet` feature
### `windmill-api/Cargo.toml`
- Made optional: `aws-sigv4`, `aws-sdk-config`, `aws-credential-types`, `aws-smithy-types`, `windmill-parser-py-imports`, `windmill-autoscaling`
- Added AWS deps to `parquet` and `bedrock` features
- Added `windmill-parser-py-imports` to `python` and `agent_worker_server` features
- Added `windmill-autoscaling` to `enterprise` feature
### `windmill-autoscaling/Cargo.toml`
- Made optional: `kube`, `k8s-openapi` (only used in EE code)
- Added to `private` feature
### `parsers/windmill-parser-py-imports/Cargo.toml`
- Removed unused direct dependencies: `malachite`, `malachite-bigint` (still available transitively via `rustpython-parser`)
## Benchmarks
### Default build (no features)
| Metric | Before | After |
|---|---|---|
| Crates compiled | 761 | 609 |
| Notable deps eliminated | - | aws-sdk-config (9.5s), k8s-openapi (7.3s), zstd-sys (7.7s), kube-client (1.9s) |
### Incremental compilation (stable-state, warm cache)
| Scenario | Before | After |
|---|---|---|
| Touch `windmill-api/src/users.rs` | ~5.6s | ~5.4s |
| Touch `windmill-worker/src/worker.rs` | ~6.7s | ~6.2s |
| Touch `windmill-common/src/worker.rs` (cascade) | ~8.5s | ~8.5s |
Incremental compilation improvement from feature-gating alone is modest because the bottleneck is the compilation of the windmill crates themselves (especially windmill-api at 90k LOC), not the dependencies.
## Developer-Local Speed Tips
These settings are **not committed** because they are developer-local preferences that depend on toolchain availability. Combined, they yield ~16% faster incremental compilation.
### mold linker (~6% improvement)
Install `mold` and add to `.cargo/config.toml`:
```toml
[target.x86_64-unknown-linux-gnu]
linker = "clang"
rustflags = ["-C", "link-arg=-fuse-ld=mold"]
```
### Reduced debug info (~10% improvement)
Add to `[profile.dev]` in `Cargo.toml`:
```toml
split-debuginfo = "unpacked"
debug = "line-tables-only"
```
### SQLX offline mode (~4% improvement)
If you're not modifying SQL queries:
```bash
export SQLX_OFFLINE=true
```
### Combined effect
| Scenario | Baseline | With all tips |
|---|---|---|
| Touch `windmill-api` file | 5.6s | **4.7s** |
| Touch `windmill-worker` file | 6.7s | **6.0s** |
| Touch `windmill-common` file (cascade) | 8.5s | **7.6s** |
## What would help more (future work)
The single biggest improvement would be **splitting `windmill-api`** (90k LOC) into smaller crates. Currently, any file change in the crate triggers re-analysis of all 90k lines. However, this requires significant refactoring due to tight coupling between the triggers subsystem, jobs, users, and the axum router initialization.

334
backend/Cargo.lock generated
View File

@@ -234,9 +234,9 @@ dependencies = [
[[package]]
name = "arc-swap"
version = "1.8.0"
version = "1.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "51d03449bb8ca2cc2ef70869af31463d1ae5ccc8fa3e334b307203fbf815207e"
checksum = "9ded5f9a03ac8f24d1b8a25101ee812cd32cdc8c50a4c50237de2c4915850e73"
dependencies = [
"rustversion",
]
@@ -490,7 +490,7 @@ dependencies = [
"memchr",
"num",
"regex",
"regex-syntax 0.8.8",
"regex-syntax 0.8.9",
]
[[package]]
@@ -828,7 +828,7 @@ dependencies = [
"aws-sdk-ssooidc",
"aws-sdk-sts",
"aws-smithy-async",
"aws-smithy-http",
"aws-smithy-http 0.62.6",
"aws-smithy-json",
"aws-smithy-runtime",
"aws-smithy-runtime-api",
@@ -890,7 +890,7 @@ dependencies = [
"aws-sigv4",
"aws-smithy-async",
"aws-smithy-eventstream",
"aws-smithy-http",
"aws-smithy-http 0.62.6",
"aws-smithy-runtime",
"aws-smithy-runtime-api",
"aws-smithy-types",
@@ -914,7 +914,7 @@ dependencies = [
"aws-credential-types",
"aws-runtime",
"aws-smithy-async",
"aws-smithy-http",
"aws-smithy-http 0.62.6",
"aws-smithy-json",
"aws-smithy-observability",
"aws-smithy-runtime",
@@ -939,7 +939,7 @@ dependencies = [
"aws-sigv4",
"aws-smithy-async",
"aws-smithy-eventstream",
"aws-smithy-http",
"aws-smithy-http 0.62.6",
"aws-smithy-json",
"aws-smithy-observability",
"aws-smithy-runtime",
@@ -963,7 +963,7 @@ dependencies = [
"aws-credential-types",
"aws-runtime",
"aws-smithy-async",
"aws-smithy-http",
"aws-smithy-http 0.62.6",
"aws-smithy-json",
"aws-smithy-runtime",
"aws-smithy-runtime-api",
@@ -987,7 +987,7 @@ dependencies = [
"aws-runtime",
"aws-sigv4",
"aws-smithy-async",
"aws-smithy-http",
"aws-smithy-http 0.62.6",
"aws-smithy-json",
"aws-smithy-observability",
"aws-smithy-query",
@@ -1012,7 +1012,7 @@ dependencies = [
"aws-credential-types",
"aws-runtime",
"aws-smithy-async",
"aws-smithy-http",
"aws-smithy-http 0.62.6",
"aws-smithy-json",
"aws-smithy-runtime",
"aws-smithy-runtime-api",
@@ -1034,7 +1034,7 @@ dependencies = [
"aws-credential-types",
"aws-runtime",
"aws-smithy-async",
"aws-smithy-http",
"aws-smithy-http 0.62.6",
"aws-smithy-json",
"aws-smithy-runtime",
"aws-smithy-runtime-api",
@@ -1056,7 +1056,7 @@ dependencies = [
"aws-credential-types",
"aws-runtime",
"aws-smithy-async",
"aws-smithy-http",
"aws-smithy-http 0.62.6",
"aws-smithy-json",
"aws-smithy-runtime",
"aws-smithy-runtime-api",
@@ -1078,7 +1078,7 @@ dependencies = [
"aws-credential-types",
"aws-runtime",
"aws-smithy-async",
"aws-smithy-http",
"aws-smithy-http 0.62.6",
"aws-smithy-json",
"aws-smithy-query",
"aws-smithy-runtime",
@@ -1100,7 +1100,7 @@ checksum = "69e523e1c4e8e7e8ff219d732988e22bfeae8a1cafdbe6d9eca1546fa080be7c"
dependencies = [
"aws-credential-types",
"aws-smithy-eventstream",
"aws-smithy-http",
"aws-smithy-http 0.62.6",
"aws-smithy-runtime-api",
"aws-smithy-types",
"bytes",
@@ -1117,9 +1117,9 @@ dependencies = [
[[package]]
name = "aws-smithy-async"
version = "1.2.8"
version = "1.2.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9330762ee48c6cecfad2cb37b1506c16c8e858c90638eda2b1a7272b56f88bd5"
checksum = "52eec3db979d18cb807fc1070961cc51d87d069abe9ab57917769687368a8c6c"
dependencies = [
"futures-util",
"pin-project-lite",
@@ -1128,9 +1128,9 @@ dependencies = [
[[package]]
name = "aws-smithy-eventstream"
version = "0.60.15"
version = "0.60.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0810b22ae554f5076c3eabe1fe89b01aee61c354c575789f67e248e83c5f472b"
checksum = "35b9c7354a3b13c66f60fe4616d6d1969c9fd36b1b5333a5dfb3ee716b33c588"
dependencies = [
"aws-smithy-types",
"bytes",
@@ -1160,10 +1160,31 @@ dependencies = [
]
[[package]]
name = "aws-smithy-http-client"
version = "1.1.6"
name = "aws-smithy-http"
version = "0.63.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ec918f18147cec121cb142a91b0038f66d99bbe903e585dccf871920e90b22ab"
checksum = "630e67f2a31094ffa51b210ae030855cb8f3b7ee1329bdd8d085aaf61e8b97fc"
dependencies = [
"aws-smithy-runtime-api",
"aws-smithy-types",
"bytes",
"bytes-utils",
"futures-core",
"futures-util",
"http 1.4.0",
"http-body 1.0.1",
"http-body-util",
"percent-encoding",
"pin-project-lite",
"pin-utils",
"tracing",
]
[[package]]
name = "aws-smithy-http-client"
version = "1.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "12fb0abf49ff0cab20fd31ac1215ed7ce0ea92286ba09e2854b42ba5cabe7525"
dependencies = [
"aws-smithy-async",
"aws-smithy-runtime-api",
@@ -1200,18 +1221,18 @@ dependencies = [
[[package]]
name = "aws-smithy-observability"
version = "0.2.1"
version = "0.2.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a700a7702874cd78b85fecdc9f64f3f72eb22fb713791cb445bcfd2a15bc1ecf"
checksum = "c0a46543fbc94621080b3cf553eb4cbbdc41dd9780a30c4756400f0139440a1d"
dependencies = [
"aws-smithy-runtime-api",
]
[[package]]
name = "aws-smithy-query"
version = "0.60.10"
version = "0.60.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "adc4a6cdc289a37be7fddb7f4365448187d62c603a40e6d46d13c68e5e81900f"
checksum = "0cebbddb6f3a5bd81553643e9c7daf3cc3dc5b0b5f398ac668630e8a84e6fff0"
dependencies = [
"aws-smithy-types",
"urlencoding",
@@ -1219,12 +1240,12 @@ dependencies = [
[[package]]
name = "aws-smithy-runtime"
version = "1.9.8"
version = "1.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bb5b6167fcdf47399024e81ac08e795180c576a20e4d4ce67949f9a88ae37dc1"
checksum = "f3df87c14f0127a0d77eb261c3bc45d5b4833e2a1f63583ebfb728e4852134ee"
dependencies = [
"aws-smithy-async",
"aws-smithy-http",
"aws-smithy-http 0.63.3",
"aws-smithy-http-client",
"aws-smithy-observability",
"aws-smithy-runtime-api",
@@ -1235,6 +1256,7 @@ dependencies = [
"http 1.4.0",
"http-body 0.4.6",
"http-body 1.0.1",
"http-body-util",
"pin-project-lite",
"pin-utils",
"tokio",
@@ -1243,9 +1265,9 @@ dependencies = [
[[package]]
name = "aws-smithy-runtime-api"
version = "1.11.0"
version = "1.11.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5c47b1e62accf759b01aba295e40479d1ba8fb77c2a54f0fed861c809ca49761"
checksum = "49952c52f7eebb72ce2a754d3866cc0f87b97d2a46146b79f80f3a93fb2b3716"
dependencies = [
"aws-smithy-async",
"aws-smithy-types",
@@ -1260,9 +1282,9 @@ dependencies = [
[[package]]
name = "aws-smithy-types"
version = "1.4.0"
version = "1.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c2d447863bdec38c899e5753a48c0abcf590f3ec629e257ad5a9ef8806ad7714"
checksum = "3b3a26048eeab0ddeba4b4f9d51654c79af8c3b32357dc5f336cee85ab331c33"
dependencies = [
"base64-simd 0.8.0",
"bytes",
@@ -1286,9 +1308,9 @@ dependencies = [
[[package]]
name = "aws-smithy-types-convert"
version = "0.60.11"
version = "0.60.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b70bc27e41d5ed80b376602ff4becdab6ea8489403fad3abbfea2c9c825c1e1e"
checksum = "059deaa8583331f9f610b44c7cbc005d0cccec6dec3a7b387de096dbe6c06b8a"
dependencies = [
"aws-smithy-types",
"chrono",
@@ -1947,7 +1969,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8c6d47a4e2961fb8721bcfc54feae6455f2f64e7054f9bc67e875f0e77f4c58d"
dependencies = [
"rust_decimal",
"schemars 1.2.0",
"schemars 1.2.1",
"serde",
"utf8-width",
]
@@ -1976,9 +1998,9 @@ dependencies = [
[[package]]
name = "bytemuck"
version = "1.24.0"
version = "1.25.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1fbdf580320f38b612e485521afda1ee26d10cc9884efaaa750d383e13e3c5f4"
checksum = "c8efb64bd706a16a1bdde310ae86b351e4d21550d98d056f22f8a7f7a2183fec"
dependencies = [
"bytemuck_derive",
]
@@ -2002,9 +2024,9 @@ checksum = "1fd0f2584146f6f2ef48085050886acf353beff7305ebd1ae69500e27c67f64b"
[[package]]
name = "bytes"
version = "1.11.0"
version = "1.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b35204fbdc0b3f4446b89fc1ac2cf84a8a68971995d0bf2e925ec7cd960f9cb3"
checksum = "1e748733b7cbc798e1434b6ac524f0c1ff2ab456fe201501e6497c8417a4fc33"
dependencies = [
"serde",
]
@@ -2150,9 +2172,9 @@ dependencies = [
[[package]]
name = "cc"
version = "1.2.54"
version = "1.2.55"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6354c81bbfd62d9cfa9cb3c773c2b7b2a3a482d569de977fd0e961f6e7c00583"
checksum = "47b26a0954ae34af09b50f0de26458fa95369a0d478d8236d3f93082b219bd29"
dependencies = [
"find-msvc-tools",
"jobserver",
@@ -2265,9 +2287,9 @@ dependencies = [
[[package]]
name = "clap"
version = "4.5.55"
version = "4.5.57"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3e34525d5bbbd55da2bb745d34b36121baac88d07619a9a09cfcf4a6c0832785"
checksum = "6899ea499e3fb9305a65d5ebf6e3d2248c5fab291f300ad0a704fbe142eae31a"
dependencies = [
"clap_builder",
"clap_derive",
@@ -2275,9 +2297,9 @@ dependencies = [
[[package]]
name = "clap_builder"
version = "4.5.55"
version = "4.5.57"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "59a20016a20a3da95bef50ec7238dbd09baeef4311dcdd38ec15aba69812fb61"
checksum = "7b12c8b680195a62a8364d16b8447b01b6c2c8f9aaf68bee653be34d4245e238"
dependencies = [
"anstream",
"anstyle",
@@ -3449,7 +3471,7 @@ dependencies = [
"log",
"recursive",
"regex",
"regex-syntax 0.8.8",
"regex-syntax 0.8.9",
]
[[package]]
@@ -5456,7 +5478,7 @@ checksum = "6e24cb5a94bcae1e5408b0effca5cd7172ea3c5755049c5f3af4cd283a165298"
dependencies = [
"bit-set 0.8.0",
"regex-automata",
"regex-syntax 0.8.8",
"regex-syntax 0.8.9",
]
[[package]]
@@ -5467,7 +5489,7 @@ checksum = "72cf461f865c862bb7dc573f643dd6a2b6842f7c30b07882b56bd148cc2761b8"
dependencies = [
"bit-set 0.8.0",
"regex-automata",
"regex-syntax 0.8.8",
"regex-syntax 0.8.9",
]
[[package]]
@@ -5566,9 +5588,9 @@ dependencies = [
[[package]]
name = "find-msvc-tools"
version = "0.1.8"
version = "0.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8591b0bcc8a98a64310a2fae1bb3e9b8564dd10e381e6e28010fde8e8e8568db"
checksum = "5baebc0774151f905a1a2cc41989300b1e6fbb29aff0ceffa1064fdd3088d582"
[[package]]
name = "fixedbitset"
@@ -5588,9 +5610,9 @@ dependencies = [
[[package]]
name = "flate2"
version = "1.1.8"
version = "1.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b375d6465b98090a5f25b1c7703f3859783755aa9a80433b36e0379a3ec2f369"
checksum = "843fba2746e448b37e26a819579957415c8cef339bf08564fe8b7ddbd959573c"
dependencies = [
"crc32fast",
"libz-sys",
@@ -5609,9 +5631,9 @@ dependencies = [
[[package]]
name = "float8"
version = "0.6.0"
version = "0.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f463a8a37ede13dac13316d1a1eeafa992300906a0c4c7fa1177f366d10bcbf"
checksum = "719a903cc23e4a89e87962c2a80fdb45cdaad0983a89bd150bb57b4c8571a7d5"
dependencies = [
"half",
"num-traits",
@@ -6161,7 +6183,7 @@ dependencies = [
"bstr",
"log",
"regex-automata",
"regex-syntax 0.8.8",
"regex-syntax 0.8.9",
]
[[package]]
@@ -6986,14 +7008,13 @@ dependencies = [
[[package]]
name = "hyper-util"
version = "0.1.19"
version = "0.1.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "727805d60e7938b76b826a6ef209eb70eaa1812794f9424d4a4e2d740662df5f"
checksum = "96547c2556ec9d12fb1578c4eaf448b04993e7fb79cbaad930a656880a6bdfa0"
dependencies = [
"base64 0.22.1",
"bytes",
"futures-channel",
"futures-core",
"futures-util",
"http 1.4.0",
"http-body 1.0.1",
@@ -10054,9 +10075,9 @@ dependencies = [
[[package]]
name = "portable-atomic"
version = "1.13.0"
version = "1.13.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f89776e4d69bb58bc6993e99ffa1d11f228b839984854c7daeb5d37f87cbe950"
checksum = "c33a9471896f1c69cecef8d20cbe2f7accd12527ce60845ff44c153bb2a21b49"
[[package]]
name = "postgres-native-tls"
@@ -10866,32 +10887,32 @@ dependencies = [
[[package]]
name = "regex"
version = "1.12.2"
version = "1.12.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "843bc0191f75f3e22651ae5f1e72939ab2f72a4bc30fa80a066bd66edefc24d4"
checksum = "e10754a14b9137dd7b1e3e5b0493cc9171fdd105e0ab477f51b72e7f3ac0e276"
dependencies = [
"aho-corasick",
"memchr",
"regex-automata",
"regex-syntax 0.8.8",
"regex-syntax 0.8.9",
]
[[package]]
name = "regex-automata"
version = "0.4.13"
version = "0.4.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5276caf25ac86c8d810222b3dbb938e512c55c6831a10f3e6ed1c93b84041f1c"
checksum = "6e1dd4122fc1595e8162618945476892eefca7b88c52820e74af6262213cae8f"
dependencies = [
"aho-corasick",
"memchr",
"regex-syntax 0.8.8",
"regex-syntax 0.8.9",
]
[[package]]
name = "regex-lite"
version = "0.1.8"
version = "0.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8d942b98df5e658f56f20d592c7f868833fe38115e65c33003d8cd224b0155da"
checksum = "cab834c73d247e67f4fae452806d17d3c7501756d98c8808d7c9c7aa7d18f973"
[[package]]
name = "regex-syntax"
@@ -10901,9 +10922,9 @@ checksum = "dbb5fb1acd8a1a18b3dd5be62d25485eb770e05afb408a9627d14d451bae12da"
[[package]]
name = "regex-syntax"
version = "0.8.8"
version = "0.8.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7a2d987857b319362043e95f5353c0535c1f58eec5336fdfcf626430af7def58"
checksum = "a96887878f22d7bad8a3b6dc5b7440e0ada9a245242924394987b21cf2210a4c"
[[package]]
name = "relative-path"
@@ -11162,7 +11183,7 @@ dependencies = [
"rand 0.9.0",
"reqwest 0.12.28",
"rmcp-macros",
"schemars 1.2.0",
"schemars 1.2.1",
"serde",
"serde_json",
"sse-stream",
@@ -11818,14 +11839,14 @@ dependencies = [
[[package]]
name = "schemars"
version = "1.2.0"
version = "1.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "54e910108742c57a770f492731f99be216a52fadd361b06c8fb59d74ccc267d2"
checksum = "a2b42f36aa1cd011945615b92222f6bf73c599a102a300334cd7f8dbeec726cc"
dependencies = [
"chrono",
"dyn-clone",
"ref-cast",
"schemars_derive 1.2.0",
"schemars_derive 1.2.1",
"serde",
"serde_json",
]
@@ -11844,9 +11865,9 @@ dependencies = [
[[package]]
name = "schemars_derive"
version = "1.2.0"
version = "1.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4908ad288c5035a8eb12cfdf0d49270def0a268ee162b75eeee0f85d155a7c45"
checksum = "7d115b50f4aaeea07e79c1912f645c7513d81715d0420f8bc77a18c6260b307f"
dependencies = [
"proc-macro2",
"quote",
@@ -12153,7 +12174,7 @@ dependencies = [
"indexmap 1.9.3",
"indexmap 2.11.1",
"schemars 0.9.0",
"schemars 1.2.0",
"schemars 1.2.1",
"serde",
"serde_derive",
"serde_json",
@@ -12399,9 +12420,9 @@ dependencies = [
[[package]]
name = "slab"
version = "0.4.11"
version = "0.4.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7a2ae44ef20feb57a68b23d846850f861394c2e02dc425a50098ae8c90267589"
checksum = "0c790de23124f9ab44544d7ac05d60440adc586479ce501c1d6d7da3cd8c9cf5"
[[package]]
name = "slotmap"
@@ -13446,9 +13467,9 @@ dependencies = [
[[package]]
name = "system-configuration"
version = "0.6.1"
version = "0.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3c879d448e9d986b661742763247d3693ed13609438cf3d006f51f5368a5ba6b"
checksum = "a13f3d0daba03132c0aa9767f98351b3488edc2c100cda2d2ec2b04f3d8d3c8b"
dependencies = [
"bitflags 2.9.4",
"core-foundation 0.9.4",
@@ -13578,7 +13599,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d60769b80ad7953d8a7b2c70cdfe722bbcdcac6bccc8ac934c40c034d866fc18"
dependencies = [
"byteorder",
"regex-syntax 0.8.8",
"regex-syntax 0.8.9",
"utf8-ranges",
]
@@ -14511,7 +14532,7 @@ checksum = "0203df02a3b6dd63575cc1d6e609edc2181c9a11867a271b25cfd2abff3ec5ca"
dependencies = [
"cc",
"regex",
"regex-syntax 0.8.8",
"regex-syntax 0.8.9",
"tree-sitter-language",
]
@@ -14537,9 +14558,9 @@ dependencies = [
[[package]]
name = "tree-sitter-language"
version = "0.1.6"
version = "0.1.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4ae62f7eae5eb549c71b76658648b72cc6111f2d87d24a1e31fa907f4943e3ce"
checksum = "009994f150cc0cd50ff54917d5bc8bffe8cad10ca10d81c34da2ec421ae61782"
[[package]]
name = "tree-sitter-ruby"
@@ -15466,14 +15487,11 @@ checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
[[package]]
name = "windmill"
version = "1.621.1"
version = "1.624.0"
dependencies = [
"anyhow",
"aws-sdk-config",
"aws-sigv4",
"axum 0.7.9",
"base64 0.22.1",
"bitflags 2.9.4",
"chrono",
"constant_time_eq 0.3.1",
"deno_core",
@@ -15481,18 +15499,10 @@ dependencies = [
"futures",
"gethostname",
"git-version",
"globset",
"k8s-openapi",
"kube",
"lazy_static",
"libloading 0.8.9",
"memchr",
"object_store",
"once_cell",
"opentelemetry-proto 0.29.0",
"pep440_rs",
"prometheus",
"quote",
"rand 0.9.0",
"reqwest 0.13.1",
"rustls 0.23.35",
@@ -15505,7 +15515,7 @@ dependencies = [
"sql-builder",
"sqlx",
"strum 0.27.2",
"systemstat",
"tempfile",
"tikv-jemalloc-ctl",
"tikv-jemalloc-sys",
"tikv-jemallocator",
@@ -15521,7 +15531,6 @@ dependencies = [
"windmill-common",
"windmill-git-sync",
"windmill-indexer",
"windmill-nativets",
"windmill-queue",
"windmill-worker",
"windows-service",
@@ -15530,7 +15539,7 @@ dependencies = [
[[package]]
name = "windmill-api"
version = "1.621.1"
version = "1.624.0"
dependencies = [
"anyhow",
"argon2",
@@ -15567,6 +15576,8 @@ dependencies = [
"cookie 0.17.0",
"cron",
"datafusion",
"deno_core",
"deno_error",
"ed25519-dalek",
"flate2",
"futures",
@@ -15603,7 +15614,6 @@ dependencies = [
"rdkafka-sys",
"regex",
"reqwest 0.13.1",
"rquickjs",
"rsa",
"rumqttc",
"rust-embed",
@@ -15652,6 +15662,7 @@ dependencies = [
"windmill-parser",
"windmill-parser-py",
"windmill-parser-py-imports",
"windmill-parser-sql",
"windmill-parser-ts",
"windmill-queue",
"windmill-worker",
@@ -15659,7 +15670,7 @@ dependencies = [
[[package]]
name = "windmill-api-client"
version = "1.621.1"
version = "1.624.0"
dependencies = [
"reqwest 0.12.28",
"serde",
@@ -15669,7 +15680,7 @@ dependencies = [
[[package]]
name = "windmill-audit"
version = "1.621.1"
version = "1.624.0"
dependencies = [
"chrono",
"lazy_static",
@@ -15683,7 +15694,7 @@ dependencies = [
[[package]]
name = "windmill-autoscaling"
version = "1.621.1"
version = "1.624.0"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -15702,7 +15713,7 @@ dependencies = [
[[package]]
name = "windmill-common"
version = "1.621.1"
version = "1.624.0"
dependencies = [
"anyhow",
"async-recursion",
@@ -15798,7 +15809,7 @@ dependencies = [
[[package]]
name = "windmill-git-sync"
version = "1.621.1"
version = "1.624.0"
dependencies = [
"regex",
"serde",
@@ -15813,7 +15824,7 @@ dependencies = [
[[package]]
name = "windmill-indexer"
version = "1.621.1"
version = "1.624.0"
dependencies = [
"anyhow",
"astral-tokio-tar",
@@ -15837,7 +15848,7 @@ dependencies = [
[[package]]
name = "windmill-macros"
version = "1.621.1"
version = "1.624.0"
dependencies = [
"itertools 0.14.0",
"lazy_static",
@@ -15853,7 +15864,7 @@ dependencies = [
[[package]]
name = "windmill-mcp"
version = "1.621.1"
version = "1.624.0"
dependencies = [
"anyhow",
"async-trait",
@@ -15871,46 +15882,9 @@ dependencies = [
"windmill-common",
]
[[package]]
name = "windmill-nativets"
version = "1.621.1"
dependencies = [
"anyhow",
"const_format",
"crossbeam-channel",
"deno_ast",
"deno_console",
"deno_core",
"deno_error",
"deno_fetch",
"deno_io",
"deno_net",
"deno_permissions",
"deno_runtime",
"deno_telemetry",
"deno_tls",
"deno_url",
"deno_web",
"deno_webidl",
"futures",
"itertools 0.14.0",
"lazy_static",
"regex",
"serde",
"serde_json",
"sqlx",
"tokio",
"tracing",
"uuid",
"winapi",
"windmill-common",
"windmill-parser-ts",
"windmill-queue",
]
[[package]]
name = "windmill-oauth"
version = "1.621.1"
version = "1.624.0"
dependencies = [
"anyhow",
"async-oauth2",
@@ -15934,7 +15908,7 @@ dependencies = [
[[package]]
name = "windmill-parser"
version = "1.621.1"
version = "1.624.0"
dependencies = [
"convert_case 0.6.0",
"serde",
@@ -15943,7 +15917,7 @@ dependencies = [
[[package]]
name = "windmill-parser-bash"
version = "1.621.1"
version = "1.624.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -15955,7 +15929,7 @@ dependencies = [
[[package]]
name = "windmill-parser-csharp"
version = "1.621.1"
version = "1.624.0"
dependencies = [
"anyhow",
"serde_json",
@@ -15967,7 +15941,7 @@ dependencies = [
[[package]]
name = "windmill-parser-go"
version = "1.621.1"
version = "1.624.0"
dependencies = [
"anyhow",
"gosyn",
@@ -15979,7 +15953,7 @@ dependencies = [
[[package]]
name = "windmill-parser-graphql"
version = "1.621.1"
version = "1.624.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -15991,7 +15965,7 @@ dependencies = [
[[package]]
name = "windmill-parser-java"
version = "1.621.1"
version = "1.624.0"
dependencies = [
"anyhow",
"serde_json",
@@ -16003,7 +15977,7 @@ dependencies = [
[[package]]
name = "windmill-parser-nu"
version = "1.621.1"
version = "1.624.0"
dependencies = [
"anyhow",
"nu-parser",
@@ -16014,7 +15988,7 @@ dependencies = [
[[package]]
name = "windmill-parser-php"
version = "1.621.1"
version = "1.624.0"
dependencies = [
"anyhow",
"itertools 0.14.0",
@@ -16025,7 +15999,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py"
version = "1.621.1"
version = "1.624.0"
dependencies = [
"anyhow",
"itertools 0.14.0",
@@ -16038,14 +16012,12 @@ dependencies = [
[[package]]
name = "windmill-parser-py-imports"
version = "1.621.1"
version = "1.624.0"
dependencies = [
"anyhow",
"async-recursion",
"itertools 0.14.0",
"lazy_static",
"malachite",
"malachite-bigint",
"pep440_rs",
"phf 0.11.3",
"regex",
@@ -16062,7 +16034,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ruby"
version = "1.621.1"
version = "1.624.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -16076,7 +16048,7 @@ dependencies = [
[[package]]
name = "windmill-parser-rust"
version = "1.621.1"
version = "1.624.0"
dependencies = [
"anyhow",
"convert_case 0.6.0",
@@ -16093,7 +16065,7 @@ dependencies = [
[[package]]
name = "windmill-parser-sql"
version = "1.621.1"
version = "1.624.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -16107,7 +16079,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ts"
version = "1.621.1"
version = "1.624.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -16126,7 +16098,7 @@ dependencies = [
[[package]]
name = "windmill-parser-yaml"
version = "1.621.1"
version = "1.624.0"
dependencies = [
"anyhow",
"serde",
@@ -16137,7 +16109,7 @@ dependencies = [
[[package]]
name = "windmill-queue"
version = "1.621.1"
version = "1.624.0"
dependencies = [
"anyhow",
"async-recursion",
@@ -16174,7 +16146,7 @@ dependencies = [
[[package]]
name = "windmill-sql-datatype-parser-wasm"
version = "1.621.1"
version = "1.624.0"
dependencies = [
"wasm-bindgen",
"wasm-bindgen-test",
@@ -16184,7 +16156,7 @@ dependencies = [
[[package]]
name = "windmill-worker"
version = "1.621.1"
version = "1.624.0"
dependencies = [
"anyhow",
"async-once-cell",
@@ -16204,6 +16176,20 @@ dependencies = [
"chrono",
"const_format",
"convert_case 0.6.0",
"deno_ast",
"deno_console",
"deno_core",
"deno_error",
"deno_fetch",
"deno_io",
"deno_net",
"deno_permissions",
"deno_runtime",
"deno_telemetry",
"deno_tls",
"deno_url",
"deno_web",
"deno_webidl",
"derive_more 1.0.0",
"dotenv",
"eventsource-stream",
@@ -16244,6 +16230,7 @@ dependencies = [
"reqwest-middleware",
"rquickjs",
"rust_decimal",
"rustls-pemfile 2.2.0",
"serde",
"serde_json",
"sha2 0.10.9",
@@ -16266,7 +16253,6 @@ dependencies = [
"windmill-git-sync",
"windmill-macros",
"windmill-mcp",
"windmill-nativets",
"windmill-parser",
"windmill-parser-bash",
"windmill-parser-csharp",
@@ -17071,18 +17057,18 @@ dependencies = [
[[package]]
name = "zerocopy"
version = "0.8.35"
version = "0.8.38"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fdea86ddd5568519879b8187e1cf04e24fce28f7fe046ceecbce472ff19a2572"
checksum = "57cf3aa6855b23711ee9852dfc97dfaa51c45feaba5b645d0c777414d494a961"
dependencies = [
"zerocopy-derive",
]
[[package]]
name = "zerocopy-derive"
version = "0.8.35"
version = "0.8.38"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0c15e1b46eff7c6c91195752e0eeed8ef040e391cdece7c25376957d5f15df22"
checksum = "8a616990af1a287837c4fe6596ad77ef57948f787e46ce28e166facc0cc1cb75"
dependencies = [
"proc-macro2",
"quote",
@@ -17177,9 +17163,9 @@ dependencies = [
[[package]]
name = "zlib-rs"
version = "0.5.5"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "40990edd51aae2c2b6907af74ffb635029d5788228222c4bb811e9351c0caad3"
checksum = "a7948af682ccbc3342b6e9420e8c51c1fe5d7bf7756002b4a3c6cabfe96a7e3c"
[[package]]
name = "zstd"

View File

@@ -1,6 +1,6 @@
[package]
name = "windmill"
version = "1.621.1"
version = "1.624.0"
authors.workspace = true
edition.workspace = true
@@ -18,7 +18,6 @@ members = [
"./windmill-indexer",
"./windmill-macros",
"./windmill-oauth",
"./windmill-nativets",
"./parsers/windmill-parser",
"./parsers/windmill-parser-ts",
"./parsers/windmill-parser-go",
@@ -36,7 +35,7 @@ members = [
exclude = ["./windmill-duckdb-ffi-internal"]
[workspace.package]
version = "1.621.1"
version = "1.624.0"
authors = ["Ruben Fiszel <ruben@windmill.dev>"]
edition = "2021"
@@ -68,8 +67,8 @@ cloud = ["windmill-queue/cloud", "windmill-worker/cloud", "windmill-common/cloud
jemalloc = ["windmill-common/jemalloc", "dep:tikv-jemallocator", "dep:tikv-jemalloc-sys", "dep:tikv-jemalloc-ctl"]
tantivy = ["dep:windmill-indexer", "windmill-api/tantivy", "windmill-indexer/enterprise", "windmill-indexer/parquet", "windmill-common/tantivy", "enterprise", "parquet"]
sqlx = ["windmill-worker/sqlx"]
# deno_core is only needed for NativeTS execution - flow expressions and batch rerun use QuickJS (always enabled)
deno_core = ["windmill-worker/deno_core", "dep:deno_core", "dep:v8", "dep:windmill-nativets"]
deno_core = ["windmill-worker/deno_core", "windmill-api/deno_core", "dep:deno_core", "dep:v8"]
quickjs = ["windmill-worker/quickjs"]
deno_core_mac = ["deno_core", "windmill-worker/libffi_mac"]
kafka = ["windmill-api/kafka"]
nats = ["windmill-api/nats"]
@@ -91,6 +90,7 @@ zip = ["windmill-api/zip"]
static_frontend = ["windmill-api/static_frontend"]
scoped_cache = ["windmill-common/scoped_cache"]
test_job_debouncing = []
private_registry_test = []
# Languages
python = ["windmill-worker/python", "windmill-api/python"]
rust = ["windmill-worker/rust"]
@@ -147,25 +147,16 @@ serde_derive.workspace = true
serde_yml.workspace = true
serde.workspace = true
deno_core = { workspace = true, optional = true }
windmill-nativets = { workspace = true, optional = true }
object_store = { workspace = true, optional = true }
sha1 = { workspace = true, optional = true }
constant_time_eq = { workspace = true, optional = true }
quote.workspace = true
memchr.workspace = true
v8 = { workspace = true, optional = true }
rustls.workspace = true
pep440_rs.workspace = true
strum.workspace = true
aws-sigv4.workspace = true
aws-sdk-config.workspace = true
kube.workspace = true
k8s-openapi.workspace = true
libloading.workspace = true
bitflags.workspace = true
globset.workspace = true
opentelemetry-proto.workspace = true
systemstat.workspace = true
[target.'cfg(windows)'.dependencies]
windows-service = "0.7"
@@ -184,6 +175,7 @@ axum.workspace = true
serde.workspace = true
windmill-api-client.workspace = true
deno_core = { workspace = true, features = ["include_js_files_for_snapshotting", "unsafe_use_unprotected_platform"] }
tempfile.workspace = true
[workspace.dependencies]
@@ -198,7 +190,6 @@ windmill-indexer = {path = "./windmill-indexer"}
windmill-mcp = {path = "./windmill-mcp"}
windmill-oauth = {path = "./windmill-oauth"}
windmill-macros = {path = "./windmill-macros"}
windmill-nativets = {path = "./windmill-nativets"}
windmill-parser = { path = "./parsers/windmill-parser" }
windmill-parser-ts = { path = "./parsers/windmill-parser-ts" }
windmill-parser-py = { path = "./parsers/windmill-parser-py" }
@@ -283,7 +274,7 @@ aws-sigv4 = "^1.3.4"
aws-sdk-config = "=1.68.0"
aws-sdk-rds = "^1"
async-trait = "0.1.88"
crossbeam-channel = "0.5"
v8 = "=130.0.7" # Exact version NOTE: Do not forget to update version and hash in flake.nix
deno_fetch = "0.214.0"

View File

@@ -1 +1 @@
5d841b358dd32130c9f34b54f59b96b5c322f213
138a4f5f868f3bded5bb7cb77b222b532c07e4af

View File

@@ -0,0 +1,5 @@
ALTER TABLE asset
DROP COLUMN IF EXISTS created_at,
DROP COLUMN IF EXISTS id;
DELETE FROM asset WHERE usage_kind = 'job';

View File

@@ -0,0 +1,11 @@
ALTER TABLE asset
ADD COLUMN created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
ADD COLUMN id BIGSERIAL UNIQUE;
DO
$do$
BEGIN
ALTER TYPE ASSET_USAGE_KIND ADD VALUE 'job';
EXCEPTION WHEN OTHERS THEN
RAISE NOTICE 'Couldn''t create ASSET_USAGE_KIND::job: %', SQLERRM;
END
$do$;

View File

@@ -0,0 +1,2 @@
DROP INDEX IF EXISTS idx_asset_job_pruning;
DROP INDEX IF EXISTS idx_asset_workspace_created_id;

View File

@@ -0,0 +1,9 @@
-- Postgres requires indexes to be created in a separate migration (transaction) after columns are added.
-- Index for pagination queries that use workspace_id, created_at, and id for cursor pagination
-- Supports: SELECT with GROUP BY path, kind ORDER BY MAX(created_at) DESC, MAX(id) DESC
CREATE INDEX idx_asset_workspace_created_id ON asset (workspace_id, created_at DESC, id DESC);
-- Filtered index for job pruning operations that delete old job assets
-- Supports: DELETE queries with WHERE usage_kind = 'job' and window functions on (workspace_id, path, kind) ORDER BY created_at DESC
CREATE INDEX idx_asset_job_pruning ON asset (workspace_id, path, kind, created_at DESC) WHERE usage_kind = 'job';

View File

@@ -0,0 +1 @@
ALTER TABLE worker_ping DROP COLUMN IF EXISTS dedicated_workers;

View File

@@ -0,0 +1 @@
ALTER TABLE worker_ping ADD COLUMN IF NOT EXISTS dedicated_workers TEXT[];

View File

@@ -0,0 +1 @@
-- Add down migration script here

View File

@@ -0,0 +1,2 @@
-- Add up migration script here
UPDATE workspace_diff SET has_changes = NULL;

View File

@@ -0,0 +1,121 @@
-- Revert to pg_notify based event system
-- Restore notify_config_change function
CREATE OR REPLACE FUNCTION notify_config_change()
RETURNS TRIGGER AS $$
BEGIN
PERFORM pg_notify('notify_config_change', NEW.name::text);
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
-- Restore notify_global_setting_change function
CREATE OR REPLACE FUNCTION notify_global_setting_change()
RETURNS TRIGGER AS $$
BEGIN
PERFORM pg_notify('notify_global_setting_change', NEW.name::text);
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
-- Restore notify_global_setting_delete function
CREATE OR REPLACE FUNCTION notify_global_setting_delete()
RETURNS TRIGGER AS $$
BEGIN
PERFORM pg_notify('notify_global_setting_change', OLD.name::text);
RETURN OLD;
END;
$$ LANGUAGE plpgsql;
-- Restore notify_webhook_change function
CREATE OR REPLACE FUNCTION notify_webhook_change()
RETURNS TRIGGER AS $$
BEGIN
PERFORM pg_notify('notify_webhook_change', NEW.workspace_id);
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
-- Restore notify_workspace_envs_change function
CREATE OR REPLACE FUNCTION notify_workspace_envs_change()
RETURNS TRIGGER AS $$
BEGIN
PERFORM pg_notify('notify_workspace_envs_change', NEW.workspace_id);
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
-- Restore notify_workspace_premium_change function
CREATE OR REPLACE FUNCTION notify_workspace_premium_change()
RETURNS TRIGGER AS $$
BEGIN
PERFORM pg_notify('notify_workspace_premium_change', NEW.id);
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
-- Restore notify_team_plan_status_change function
CREATE OR REPLACE FUNCTION notify_team_plan_status_change()
RETURNS TRIGGER AS $$
BEGIN
PERFORM pg_notify('notify_workspace_premium_change', NEW.workspace_id);
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
-- Restore notify_runnable_version_change function
CREATE OR REPLACE FUNCTION notify_runnable_version_change()
RETURNS TRIGGER AS $$
DECLARE
source_type TEXT;
kind TEXT;
BEGIN
source_type := TG_ARGV[0];
IF source_type = 'script' THEN
kind := NEW.kind;
ELSE
kind := 'flow';
END IF;
PERFORM pg_notify('notify_runnable_version_change', NEW.workspace_id || ':' || source_type || ':' || NEW.path || ':' || kind);
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
-- Restore notify_http_trigger_change function
CREATE OR REPLACE FUNCTION notify_http_trigger_change()
RETURNS TRIGGER AS $$
BEGIN
PERFORM pg_notify('notify_http_trigger_change', NEW.workspace_id || ':' || NEW.path);
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
-- Restore notify_token_invalidation function
CREATE OR REPLACE FUNCTION notify_token_invalidation()
RETURNS TRIGGER AS $$
BEGIN
IF OLD.label = 'session' AND OLD.email IS NOT NULL THEN
PERFORM pg_notify('notify_token_invalidation', OLD.token);
END IF;
RETURN OLD;
END;
$$ LANGUAGE plpgsql;
-- Restore notify_workspace_key_change function
CREATE OR REPLACE FUNCTION notify_workspace_key_change()
RETURNS TRIGGER AS $$
BEGIN
IF TG_OP = 'DELETE' THEN
PERFORM pg_notify('notify_workspace_key_change', OLD.workspace_id);
RETURN OLD;
ELSE
PERFORM pg_notify('notify_workspace_key_change', NEW.workspace_id);
RETURN NEW;
END IF;
END;
$$ LANGUAGE plpgsql;
-- Drop the notify_event table
DROP TABLE IF EXISTS notify_event;

View File

@@ -0,0 +1,135 @@
-- Create notify_event table for polling-based event system
-- This replaces PostgreSQL LISTEN/NOTIFY with a table-based approach
CREATE TABLE IF NOT EXISTS notify_event (
id BIGSERIAL PRIMARY KEY,
channel TEXT NOT NULL,
payload TEXT NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS notify_event_created_at_idx ON notify_event (created_at);
-- Drop redundant index if it exists (id is already the PRIMARY KEY)
DROP INDEX IF EXISTS notify_event_id_idx;
-- Update notify_config_change function
CREATE OR REPLACE FUNCTION notify_config_change()
RETURNS TRIGGER AS $$
BEGIN
INSERT INTO notify_event (channel, payload) VALUES ('notify_config_change', NEW.name::text);
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
-- Update notify_global_setting_change function
CREATE OR REPLACE FUNCTION notify_global_setting_change()
RETURNS TRIGGER AS $$
BEGIN
INSERT INTO notify_event (channel, payload) VALUES ('notify_global_setting_change', NEW.name::text);
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
-- Update notify_global_setting_delete function
CREATE OR REPLACE FUNCTION notify_global_setting_delete()
RETURNS TRIGGER AS $$
BEGIN
INSERT INTO notify_event (channel, payload) VALUES ('notify_global_setting_change', OLD.name::text);
RETURN OLD;
END;
$$ LANGUAGE plpgsql;
-- Update notify_webhook_change function
CREATE OR REPLACE FUNCTION notify_webhook_change()
RETURNS TRIGGER AS $$
BEGIN
INSERT INTO notify_event (channel, payload) VALUES ('notify_webhook_change', NEW.workspace_id);
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
-- Update notify_workspace_envs_change function
CREATE OR REPLACE FUNCTION notify_workspace_envs_change()
RETURNS TRIGGER AS $$
BEGIN
INSERT INTO notify_event (channel, payload) VALUES ('notify_workspace_envs_change', COALESCE(NEW.workspace_id, OLD.workspace_id));
RETURN COALESCE(NEW, OLD);
END;
$$ LANGUAGE plpgsql;
-- Update notify_workspace_premium_change function
CREATE OR REPLACE FUNCTION notify_workspace_premium_change()
RETURNS TRIGGER AS $$
BEGIN
INSERT INTO notify_event (channel, payload) VALUES ('notify_workspace_premium_change', NEW.id);
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
-- Update notify_team_plan_status_change function
CREATE OR REPLACE FUNCTION notify_team_plan_status_change()
RETURNS TRIGGER AS $$
BEGIN
INSERT INTO notify_event (channel, payload) VALUES ('notify_workspace_premium_change', NEW.workspace_id);
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
-- Update notify_runnable_version_change function
CREATE OR REPLACE FUNCTION notify_runnable_version_change()
RETURNS TRIGGER AS $$
DECLARE
source_type TEXT;
kind TEXT;
BEGIN
source_type := TG_ARGV[0];
IF source_type = 'script' THEN
kind := NEW.kind;
ELSE
kind := 'flow';
END IF;
INSERT INTO notify_event (channel, payload) VALUES ('notify_runnable_version_change', NEW.workspace_id || ':' || source_type || ':' || NEW.path || ':' || kind);
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
-- Update notify_http_trigger_change function
CREATE OR REPLACE FUNCTION notify_http_trigger_change()
RETURNS TRIGGER AS $$
BEGIN
INSERT INTO notify_event (channel, payload) VALUES ('notify_http_trigger_change', COALESCE(NEW.workspace_id, OLD.workspace_id) || ':' || COALESCE(NEW.path, OLD.path));
RETURN COALESCE(NEW, OLD);
END;
$$ LANGUAGE plpgsql;
-- Update notify_token_invalidation function
CREATE OR REPLACE FUNCTION notify_token_invalidation()
RETURNS TRIGGER AS $$
BEGIN
IF OLD.label = 'session' AND OLD.email IS NOT NULL THEN
INSERT INTO notify_event (channel, payload) VALUES ('notify_token_invalidation', OLD.token);
END IF;
RETURN OLD;
END;
$$ LANGUAGE plpgsql;
-- Update notify_workspace_key_change function
CREATE OR REPLACE FUNCTION notify_workspace_key_change()
RETURNS TRIGGER AS $$
BEGIN
IF TG_OP = 'DELETE' THEN
INSERT INTO notify_event (channel, payload) VALUES ('notify_workspace_key_change', OLD.workspace_id);
RETURN OLD;
ELSE
INSERT INTO notify_event (channel, payload) VALUES ('notify_workspace_key_change', NEW.workspace_id);
RETURN NEW;
END IF;
END;
$$ LANGUAGE plpgsql;
-- NOTE: var_cache_invalidation / resource_cache_invalidation triggers were
-- intentionally dropped in migration 20250902085504. We do NOT re-create them
-- here to keep this migration scoped to the LISTEN/NOTIFY → polling swap only.

View File

@@ -18,8 +18,6 @@ regex.workspace = true
windmill-parser.workspace = true
windmill-common.workspace = true
rustpython-parser.workspace = true
malachite.workspace = true
malachite-bigint.workspace = true
phf.workspace = true
itertools.workspace = true
serde_json.workspace = true

View File

@@ -205,30 +205,15 @@ impl AssetsFinder {
Some(Expr::Constant(ExprConstant { value: Constant::Str(sql), .. })) => sql,
_ => return Err(()),
};
let duckdb_conn_prefix = match kind {
AssetKind::DataTable => "datatable",
AssetKind::Ducklake => "ducklake",
_ => return Ok(()),
};
let sql = format!("ATTACH '{duckdb_conn_prefix}://{path}' AS dt; USE dt; {sql}");
// We use the SQL parser to detect if it's a read or write query
match windmill_parser_sql::parse_assets(&sql) {
Ok(mut sql_assets) => {
if let Some(schema_name) = schema {
for asset in &mut sql_assets.assets {
if asset.kind == *kind && asset.path.starts_with(path.as_str()) {
asset.path = format!(
"{}/{}.{}",
path,
schema_name,
&asset.path[path.len() + 1..]
);
}
}
}
self.assets.extend(sql_assets.assets);
}
// We use the SQL parser to detect RW, specific tables, etc.
let sql_assets = windmill_parser_sql::parse_wmill_sdk_sql_assets(
*kind,
path,
schema.as_deref(),
&sql,
);
match sql_assets {
Ok(Some(sql_assets)) => self.assets.extend(sql_assets),
_ => {}
}
return Ok(());

View File

@@ -0,0 +1,37 @@
use windmill_parser::asset_parser::{AssetKind, ParseAssetsResult};
// Parse assets from sql snippets inside e.g sql`SELECT * FROM my_table`
pub fn parse_wmill_sdk_sql_assets(
kind: AssetKind,
asset_name: &str,
schema: Option<&str>,
sql: &str,
) -> anyhow::Result<Option<Vec<ParseAssetsResult>>> {
let duckdb_conn_prefix = match kind {
AssetKind::DataTable => "datatable",
AssetKind::Ducklake => "ducklake",
_ => return Err(anyhow::anyhow!("Unsupported asset kind for SQL parsing")),
};
let sql_with_attach =
format!("ATTACH '{duckdb_conn_prefix}://{asset_name}' AS dt; USE dt; {sql}");
// We use the SQL parser to detect if it's a read or write query
match crate::parse_assets(&sql_with_attach) {
Ok(mut sql_assets) => {
if let Some(schema) = schema {
for asset in &mut sql_assets.assets {
if asset.kind == kind && asset.path.starts_with(asset_name) {
asset.path = format!(
"{}/{}.{}",
asset_name,
schema,
&asset.path[asset_name.len() + 1..]
);
}
}
}
return Ok(Some(sql_assets.assets));
}
_ => Ok(None),
}
}

Some files were not shown because too many files have changed in this diff Show More