Commit Graph

766 Commits

Author SHA1 Message Date
Ruben Fiszel
ad5044267d all 2026-02-06 15:40:01 +00:00
Ruben Fiszel
2b8af5ddd4 with cc 2026-02-06 13:56:30 +00:00
Ruben Fiszel
e655a06563 fix: prevent schedule pool connection exhaustion (#7821)
* fix: prevent schedule pool connection exhaustion

Schedules were disabling themselves after upgrading from v1.605.0 to
v1.614.0 due to pool connection deadlock. The root cause was
fetch_authed_from_permissioned_as acquiring a pool connection inside
push() while a transaction already held one, exhausting the pool under
pressure.

Fix: pre-compute Authed before db.begin() for the normal path, and
reuse the transaction connection via fetch_authed_from_permissioned_as_conn
for the on_behalf_of_email path. Peak pool usage drops from 2 to 1 for
all schedule push paths.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: replace pool exhaustion tests with comprehensive schedule push tests

Replace the 16 pool-specific tests with 19 tests covering all schedule
push code paths: script/flow scheduling, on_behalf_of_email (script and
flow), retry wrapping, duplicate detection, invalid timezone/cron/args,
script/flow not found, paused schedules, clock shift detection, disabled
schedule, path mismatch, push failure disabling schedule, and trigger
metadata.

Also simplify the obo_authed pattern in push_scheduled_job to use a
single match assignment instead of two bindings with .or() chaining.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-05 23:25:42 +00:00
Ruben Fiszel
309de58134 Revert "feat: restriction rulesets for workspaces (#7791)" (#7820)
This reverts commit a1cd02d7f8.
2026-02-05 18:16:40 +00:00
wendrul
a1cd02d7f8 feat: restriction rulesets for workspaces (#7791)
* Add frontend for the workspace proctection rules

* api to add and update workspace protection rules

* Fix bypasser selection

* Fix Select bug on refocus

* Change rulesets to use bitflags

* Messages for protection rules errors

* claude remove ui for rules

* Hide edit buttons when rule

* No edit refactor pt1

* Update edit buttons to be disabled when rule is active

* Merge deploy ui and rulsets in one tab

* Remove not cleaned line in migration

* multiple fixes

* Remove old protection rule logic

* Add prrotection rule for deploying through Merge UI

* Add Alert on legacy Deploy UI

* Add backend enforcing of workspace rules

* Finish backend blocking on rulsets

* Last changes to api ruleset blocks

* Prepare sqlx

* Remove unused import and argument

* Update SQLx metadata

* fix npm run check

* Re trigger CI

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-02-05 18:02:41 +00:00
hugocasa
63f9d85bf6 feat: public app rate limiting + fork hub raw apps + raw apps publish to hub button (#7789)
* feat: public app rate limiting + fork hub raw apps + raw apps publish to hub button

* sqlx

* missing sqlx file

* cache rate limiting

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-02-04 17:53:55 +00:00
Diego Imbert
0caa533fbd feat: column-level asset tracking for ducklake and datatables (#7774)
* Refactor 1

* claude tmp1

* fixes1

* support for insert and update

* Fix returning

* docs: add nuanced mutex selection guidance to Rust backend skill (#7737)

Add "Mutex Selection in Async Code" section explaining when to use
std::sync::Mutex vs tokio::sync::Mutex based on official Tokio docs.
std::sync::Mutex is preferred for data protection as it's faster;
tokio::sync::Mutex only needed when holding locks across .await points.

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): revert findCodebase change that broke ../shared codebases (#7740)

* fix(cli): revert findCodebase relative_path check that broke ../shared codebases

The previous change added a check to ensure script paths start with the
codebase's relative_path. However, this broke cases where relative_path
uses parent directory references (e.g., "../shared") because:

1. path.join normalizes paths, so "/project/../shared/f/script.ts" becomes
   "/shared/f/script.ts"
2. FSFSElement strips the cwd prefix, resulting in "f/script.ts"
3. The check "f/script.ts".startsWith("../shared/") failed

The original behavior was correct - relative_path indicates where to find
codebase files, while includes/excludes patterns match against the normalized
paths that get passed during sync.

Fixes regression reported in #7729 comments.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test(cli): add preview test for codebase with imports

Tests that codebase bundling correctly includes imported modules,
which is the key functionality needed for ../shared codebases.
The test creates a helper module and a main script that imports
from it, then verifies the bundled script executes correctly.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* chore(main): release 1.621.2 (#7735)

* chore(main): release 1.621.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* feat: workspace dedicated workers (#7741)

* feat: workspace dedicated workers

* ref

* chore: update ee-repo-ref to a18ac31062ac092cb9a5fc87629e217d97f4911d

This commit updates the EE repository reference after PR #398 was merged in windmill-ee-private.

Previous ee-repo-ref: 98cfe3fef764d9d815d326d5056c734a03689d33

New ee-repo-ref: a18ac31062ac092cb9a5fc87629e217d97f4911d

Automated by sync-ee-ref workflow.

* fix(frontend): workspace script in flow steps

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* fix: remove uuid-ossp extension requirement for RDS compatibility

The uuid-ossp extension was created in the first migration but never
actually used - the codebase uses gen_random_uuid() which is built-in
to PostgreSQL 13+. This allows Windmill to run on AWS RDS where
application users may not have CREATE SCHEMA privileges.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: indexer build error (#7744)

* fix: indexer build error

* prepare sqlx

* Remove changes from Cargo.toml

* refactor: remove seed parameter from AI chat completions (#7745)

* better claude

* refactor: remove seed parameter from AI chat completions

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* fix: visibility bug on deployment UI (issue when renaming items) + add tracking of folders and resource types (#7739)

* fix: Raw apps deployment UI (and merge UI)

* Add folders and resource tpyes to merge UI

* claude first pass on adding the new arg for h_deploy_metadata

* Add missing argument to handle_deployment_metadata in all its calls

* Add support for folders and resource types in merge UI

* Update eereporef for CI

* Update ee repo

* Add migration to reset cached diff with potential artifacts

* fix type in frontend

* Preapare sqlx

* Remove unused import and logs

* update ee-repo

* Update eerepo

* chore: update ee-repo-ref to aca38475afd2cafaf63f4bbffc65be9437d57d86

This commit updates the EE repository reference after PR #397 was merged in windmill-ee-private.

Previous ee-repo-ref: 19c64cf8c61d83f45047b37660054b29658cd403

New ee-repo-ref: aca38475afd2cafaf63f4bbffc65be9437d57d86

Automated by sync-ee-ref workflow.

* Make integration  test for workspace comparisons

* Update SQLx metadata

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* fix: require AGENT_TOKEN and BASE_INTERNAL_URL for agent mode

- Add AgentConfig struct to validate required env vars on startup
- Change build_agent_http_client to require explicit token and URL
- Remove DEFAULT_BASE_INTERNAL_URL fallback (no more silent localhost:8000)
- Exit immediately if agent cannot connect to server on initial load
- Update integration tests to use dynamic port for BASE_INTERNAL_URL

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: forward teams error to client (#7746)

* fix: forward teams error to client

* chore: update ee-repo-ref to 9a3d71f2c6a41ed4d17111a8c05d8e1d4933898d

This commit updates the EE repository reference after PR #400 was merged in windmill-ee-private.

Previous ee-repo-ref: 25d35a8de1cd70e281dc876e51cd30402580b5c0

New ee-repo-ref: 9a3d71f2c6a41ed4d17111a8c05d8e1d4933898d

Automated by sync-ee-ref workflow.

* fix

* fix

* fix

* al

* sqlx

* sqlx

* all

* all

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>

* feat: add token usage tracking to AI agent output (#7738)

* feat: add token usage tracking to AI agent output

Add TokenUsage struct to track input/output/cache tokens from AI providers.
Currently implemented for Bedrock provider, with infrastructure in place
for other providers. Usage is included in the AI agent result alongside
output and messages when available.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add token usage extraction for Anthropic provider

Extract usage from message_delta SSE event and convert to TokenUsage.
Includes input_tokens, output_tokens, cache_read_input_tokens, and
cache_creation_input_tokens (mapped to cache_write_input_tokens).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add token usage extraction for Google AI/Gemini provider

Extract usage from usageMetadata in Gemini SSE events and convert to TokenUsage.
Maps promptTokenCount -> input_tokens, candidatesTokenCount -> output_tokens,
totalTokenCount -> total_tokens.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add token usage extraction for OpenAI Responses API provider

Extract usage from response.completed SSE event and convert to TokenUsage.
Maps input_tokens, output_tokens, and total_tokens directly.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add token usage extraction for Azure OpenAI / Chat Completions API

Add stream_options.include_usage to request and parse usage from final
SSE chunk for providers using the standard OpenAI Chat Completions API
(Azure OpenAI, Mistral, DeepSeek, Groq, TogetherAI, CustomAI).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: cleanup token usage tracking - remove unused Image usage field and accumulate across iterations

- Remove unused `usage` field from ParsedResponse::Image variant
- Add TokenUsage::accumulate() method to sum usage across agent iterations
- Accumulate input/output/total/cache tokens instead of replacing with last iteration

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: remove verbose debug logging from AI providers

Remove tracing::info!("[debug] ...") statements that were too verbose
for production. These logged raw events on every streaming event.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: add retry mechanism for OpenAI-compatible providers without stream_options support

Some OpenAI-compatible providers don't support the stream_options parameter
for usage tracking. This adds a retry mechanism that:
- First attempts the request with stream_options.include_usage
- If it fails with 400 and error mentions stream_options/include_usage,
  automatically retries without the parameter

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: remove unnecessary text parsing overhead in image response handlers

Revert debugging changes that read response as text before parsing JSON.
Using response.json() directly is more efficient.

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: centralize TokenUsage conversion with constructor methods

Add new(), from_input_output(), and with_cache() constructors to TokenUsage
to eliminate duplicate conversion logic across providers. Also fixes potential
truncation in Bedrock cache token conversion by using i32::try_from with
fallback to i32::MAX.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: simplify Anthropic usage extraction and add Default derive

- Use idiomatic `if let` pattern instead of `is_some()` check for usage extraction
- Add Default derive to OpenAIChatUsage for consistency with other usage structs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use saturating_add to prevent overflow in token accumulation

In long-running agents with many iterations, token counts could
potentially overflow. Using saturating_add ensures values cap at
i32::MAX instead of wrapping around.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* better claude

* nit

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* chore(main): release 1.622.0 (#7742)

* chore(main): release 1.622.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* fix npm check

* fix: add schema compatibility layer for MCP clients like n8n (#7747)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* nit ui nextcloud triggers (#7749)

* feat: add PocketID OAuth provider support (#7318)

* feat(oauth): add Pocket-ID OAuth provider component

- Implements PocketIdSetting.svelte following Keycloak pattern
- Configures OIDC endpoints for Pocket-ID (/authorize, /api/oidc/*)
- Supports standard OIDC scopes (openid, profile, email)
- Uses passkey-only authentication via Pocket-ID

Refs #5678

* feat(oauth): register Pocket-ID in SSO provider list

- Import PocketIdSetting component
- Add Pocket-ID to provider list in SSO tab
- Update exclusion filter to prevent duplicate custom entries

Refs #5678

* fix(oauth): add missing PocketID icon and fix component integration

- Create PocketIdIcon.svelte component with user profile icon
- Register pocket-id in APP_TO_ICON_COMPONENT mapping
- Fix PocketIdSetting to use IconedResourceType pattern matching other OAuth providers

This resolves the issue where PocketID toggle was not appearing in SSO settings.

Refs #5678

* refactor: migrate PocketIdSetting to Svelte 5 runes syntax

- Use $props() with $bindable() for reactive prop binding
- Use $state() for local reactive state
- Use $derived() for computed values
- Use $effect() for reactive side effects
- Replace on:change with onchange event handler
- Pre-populate base URL from existing config when editing
- Clean up bracket notation to dot notation for value properties

Addresses reviewer feedback

* fix: rename pocket-id to pocketid for naming convention compliance

Change identifier from 'pocket-id' to 'pocketid' to match Windmill's naming convention.
No OAuth provider uses hyphens - all custom SSO providers (keycloak, authentik, authelia,
kanidm, zitadel) use no separator.

Changes:
- AuthSettings.svelte: oauths['pocket-id'] → oauths['pocketid'] (2 locations)
- PocketIdSetting.svelte: name={'pocket-id'} → name={'pocketid'}
- icons/index.ts: 'pocket-id': PocketIdIcon → pocketid: PocketIdIcon

Note: PocketID does not need oauth_connect.json entry as it's a custom SSO provider
with user-configured endpoints, similar to Keycloak/Authentik.

Addresses reviewer feedback

* fix: use TextInput component for consistency

---------

Co-authored-by: hugocasa <hugo@casademont.ch>

* fix: preserve script envs field during sync push

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit frontend fix

* chore(main): release 1.623.0 (#7748)

* chore(main): release 1.623.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* fix: resolve infinite effect loop in PocketIdSetting component (#7753)

* fix: prevent retention cleanup from deleting jobs of active flows (#7755)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* chore(main): release 1.623.1 (#7754)

* chore(main): release 1.623.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* feat: default to quickjs on ce for flow eval (#7756)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* feat: runtime assets (#7656)

* Runtime assets

* Nits

* Revert "Nits"

This reverts commit 3031a2ddd1.

* detection_kinds

* don't delete runtime assets

* Show latest executions

* conditional unique idx

* nit status

* refactor

* nit refactor

* prepare sql

* Detect assets in complex JSON input objects

* false positive prevent

* nit

* redundant idx

* Update frontend/src/lib/components/assets/AssetsUsageDrawer.svelte

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* Update backend/migrations/20260122134517_runtime_assets.up.sql

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* runtime assets are inserted in a loop

* nit

* nit fix

* Don't use lazy static

* fix compilation

* nits

* missing on conflict do nothing

* add index

* Fix max n logic

* created at

* nits

* remove pagination

* sqlx prepare

* Only detect resource assets in input

* get_runtime_asset_sender()

* use global get_runtime_asset_sender to avoid prop drilling

* nit refactor : register_runtime_asset

* get job_id from token

* job as a usage kind

* fixes

* ee

* nit refactor

* merge access types when same job uses same asset multiple times

* Refactor to support wmill s3 API

* nit

* parse_wmill_sdk_sql_assets refactor

* Detect datatable and ducklake usage

* nit order by

* Join with v2_job

* better UI

* add sequential id for cursor pagination

* useInfiniteQuery

* useScrollToBottom

* sql index

* claude code stash

* migration fixes

* Infinite scroll UI

* nit

* style nit

* runtime asset created at

* Asset filters

* fix usage kind filter

* also check runnable_path for jobs when filtering

* better filters

* avoid flickering

* debounced filters

* nit

* tooltips

* fix: update AssetUsage type to match new ListAssetsResponse structure

The ListAssetsResponse changed from an array to an object with an 'assets' property.
Updated the type extraction accordingly.

Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com>

* sqlx prepare

* Delete .claude/hooks/.symlink-manifest

* unnecessary dep

* nit refactor

* nit comment

* nit naming

* CI fix attempt 1

* ee ref

* nit remove alerts

* nit

* chore: update ee-repo-ref to 138a4f5f868f3bded5bb7cb77b222b532c07e4af

This commit updates the EE repository reference after PR #395 was merged in windmill-ee-private.

Previous ee-repo-ref: 7d3a21d53066726e97dfea9f117373299bc9318c

New ee-repo-ref: 138a4f5f868f3bded5bb7cb77b222b532c07e4af

Automated by sync-ee-ref workflow.

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* fix: remove $schema field from Google AI output schema requests (#7765)

* fix: remove $schema field from Google AI output schema requests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: add $schema field to all output schema integration tests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: remove $schema field from Google AI tool parameter schemas

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: add workspace script tool test for AI agents

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* fix: Fix app multiselect not refreshing result when creating element (#7766)

* Fix returning

* asset columns are saved and displayed in the assets page

* runtime assets column detectionz

* frontend nits

* update regex parsers

* UI nits

* Display asset columns in flow graph

* Column hint directly in asset node

* nit bg

* sqlx prepare

* ee repo ref

* chore: update ee-repo-ref to 66a68df97e8c65c498b28f302a365ab8687cad9e

This commit updates the EE repository reference after PR #402 was merged in windmill-ee-private.

Previous ee-repo-ref: 0a32bc104cbaec9664a4d7cb1565823722c875a1

New ee-repo-ref: 66a68df97e8c65c498b28f302a365ab8687cad9e

Automated by sync-ee-ref workflow.

---------

Co-authored-by: centdix <40307056+centdix@users.noreply.github.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
Co-authored-by: hugocasa <hugo@casademont.ch>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: wendrul <53628737+wendrul@users.noreply.github.com>
Co-authored-by: Alexander Petric <alpetric@users.noreply.github.com>
Co-authored-by: Devdatta Talele <50290838+devdattatalele@users.noreply.github.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com>
2026-02-04 16:27:53 +00:00
Ruben Fiszel
92cbeb7c6e more verbose s3 errors 2026-02-04 14:43:43 +00:00
Ruben Fiszel
e860847073 feat: replace LISTEN/NOTIFY with polling-based event system (#7778)
* feat: replace LISTEN/NOTIFY with polling-based event system

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: add exhaustive tests for polling-based notify events

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: add missing triggers and fix tests for polling-based events

- Add variable/resource cache invalidation triggers to migration
- Fix flow test to UPDATE flow table instead of INSERT into flow_version
- Improve test isolation with unique channel names per test
- All 26 tests now pass

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: add multi-server polling tests for cross-server event propagation

Add 4 tests simulating independent server instances with separate DB
connections and polling state:
- test_two_servers_both_receive_trigger_event: both servers see same event
- test_two_servers_cross_trigger_visibility: each triggers a change, both see both
- test_server_catches_up_after_being_offline: server catches up on missed events
- test_two_servers_incremental_polling: multi-round polling with cursor advancement

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add LISTEN_NEW_EVENTS_INTERVAL_SEC env var and e2e server test

- Make poll interval configurable via LISTEN_NEW_EVENTS_INTERVAL_SEC
  (defaults to 30s)
- Make migration idempotent with IF NOT EXISTS
- Replace mock multi-server tests with actual e2e test that starts two
  windmill server processes on ports 19100/19200 with 1s poll interval,
  triggers a DB change, and verifies both servers log the event

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: ignore notify_events tests in CI

These tests require a running database, like other integration tests
in the codebase. Run with --ignored flag locally.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: only ignore slow e2e test, not fast DB tests

Only test_two_server_processes_both_receive_event is slow (~10s,
starts two server processes). The other 26 tests run in <0.2s.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: ignore all notify_events tests in CI

All tests depend on the notify_event table from the polling-based
events migration, which is not applied in CI.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use sqlx::test for notify_events tests so they work in CI

Convert all 26 fast tests from #[tokio::test] + manual get_db() to
#[sqlx::test(fixtures("base"))], which creates temporary databases
with all migrations applied. This ensures the notify_event table
exists in CI without manual setup. Only the slow e2e multi-server
test retains #[tokio::test] + #[ignore].

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: reduce default polling interval from 30s to 10s

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: address review feedback on polling-based events

- Remove redundant notify_event_id_idx index (id is already PRIMARY KEY)
- Add LIMIT 1000 to poll_notify_events to bound memory per poll cycle
- Fix potential UTF-8 panic in token log truncation using str::get
- Remove var/resource cache triggers that were re-enabled by mistake
  (they were intentionally dropped in migration 20250902085504)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 07:35:29 +00:00
Diego Imbert
635a24f82c feat: runtime assets (#7656)
* Runtime assets

* Nits

* Revert "Nits"

This reverts commit 3031a2ddd1.

* detection_kinds

* don't delete runtime assets

* Show latest executions

* conditional unique idx

* nit status

* refactor

* nit refactor

* prepare sql

* Detect assets in complex JSON input objects

* false positive prevent

* nit

* redundant idx

* Update frontend/src/lib/components/assets/AssetsUsageDrawer.svelte

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* Update backend/migrations/20260122134517_runtime_assets.up.sql

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* runtime assets are inserted in a loop

* nit

* nit fix

* Don't use lazy static

* fix compilation

* nits

* missing on conflict do nothing

* add index

* Fix max n logic

* created at

* nits

* remove pagination

* sqlx prepare

* Only detect resource assets in input

* get_runtime_asset_sender()

* use global get_runtime_asset_sender to avoid prop drilling

* nit refactor : register_runtime_asset

* get job_id from token

* job as a usage kind

* fixes

* ee

* nit refactor

* merge access types when same job uses same asset multiple times

* Refactor to support wmill s3 API

* nit

* parse_wmill_sdk_sql_assets refactor

* Detect datatable and ducklake usage

* nit order by

* Join with v2_job

* better UI

* add sequential id for cursor pagination

* useInfiniteQuery

* useScrollToBottom

* sql index

* claude code stash

* migration fixes

* Infinite scroll UI

* nit

* style nit

* runtime asset created at

* Asset filters

* fix usage kind filter

* also check runnable_path for jobs when filtering

* better filters

* avoid flickering

* debounced filters

* nit

* tooltips

* fix: update AssetUsage type to match new ListAssetsResponse structure

The ListAssetsResponse changed from an array to an object with an 'assets' property.
Updated the type extraction accordingly.

Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com>

* sqlx prepare

* Delete .claude/hooks/.symlink-manifest

* unnecessary dep

* nit refactor

* nit comment

* nit naming

* CI fix attempt 1

* ee ref

* nit remove alerts

* nit

* chore: update ee-repo-ref to 138a4f5f868f3bded5bb7cb77b222b532c07e4af

This commit updates the EE repository reference after PR #395 was merged in windmill-ee-private.

Previous ee-repo-ref: 7d3a21d53066726e97dfea9f117373299bc9318c

New ee-repo-ref: 138a4f5f868f3bded5bb7cb77b222b532c07e4af

Automated by sync-ee-ref workflow.

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-02-02 10:09:40 +00:00
Ruben Fiszel
6c84a89053 fix: require AGENT_TOKEN and BASE_INTERNAL_URL for agent mode
- Add AgentConfig struct to validate required env vars on startup
- Change build_agent_http_client to require explicit token and URL
- Remove DEFAULT_BASE_INTERNAL_URL fallback (no more silent localhost:8000)
- Exit immediately if agent cannot connect to server on initial load
- Update integration tests to use dynamic port for BASE_INTERNAL_URL

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 23:12:18 +00:00
hugocasa
60858d1e20 feat: workspace dedicated workers (#7741)
* feat: workspace dedicated workers

* ref

* chore: update ee-repo-ref to a18ac31062ac092cb9a5fc87629e217d97f4911d

This commit updates the EE repository reference after PR #398 was merged in windmill-ee-private.

Previous ee-repo-ref: 98cfe3fef764d9d815d326d5056c734a03689d33

New ee-repo-ref: a18ac31062ac092cb9a5fc87629e217d97f4911d

Automated by sync-ee-ref workflow.

* fix(frontend): workspace script in flow steps

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-01-29 17:32:48 +00:00
centdix
0089ebd4fb chore: handle empty strings in AI resource fields via serde deserializer (#7723)
* fix: handle empty strings in AI resource fields via serde deserializer

Add `empty_string_as_none` deserializer that converts empty strings to None
during deserialization. Applied to base_url, api_key, region, and AWS
credential fields in AIStandardResource and ProviderResource.

This fixes the "relative URL without a base" error when creating Anthropic
resources with empty base_url fields.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit

* nit

* nit

* cleaning

* cleaning

* cleaning

* cleaning

* fix: apply empty_string_as_none deserializer to api_key field

Consistent with other fields in ProviderResource, empty strings are now
deserialized as None for the api_key field.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 11:43:02 +00:00
centdix
7cd51def2b fix: handle empty base_url and region strings in AI providers (#7719)
When creating an Anthropic resource with "standard platform", the resource
JSON may contain `"base_url": ""` rather than omitting the field. Serde
deserializes this as `Some("")`, which bypassed the fallback logic and
caused "relative URL without a base" errors.

Similarly, AWS Bedrock with an empty region string would produce an
invalid URL like `https://bedrock-runtime..amazonaws.com`.

Filter out empty strings when checking for custom base_url and region
values, allowing the default URLs to be used correctly.

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 15:08:30 +00:00
centdix
720a7e56d1 fix(mcp): use computed base_internal_url instead of static default (#7701)
* fix(mcp): use computed base_internal_url instead of static default

Pass the actual base_internal_url (computed from the runtime port) to
the MCP backend instead of using the static BASE_INTERNAL_URL which
defaults to http://localhost:8000. This fixes internal API calls when
the server runs on a non-default port.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix

* remove BASE_INTERNAL_URL

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 17:01:24 +00:00
Pyra
0785809a91 fix(workspace-dependencies): lock hash instead of seq (#7697)
Signed-off-by: pyranota <pyra@duck.com>
2026-01-27 11:53:30 +00:00
Ruben Fiszel
7892887f01 feat: add LOGIN_DOMAIN env var to normalize emails during external login
Add LOGIN_DOMAIN environment variable that appends a domain to emails
missing one during external login (OAuth/SAML/SCIM). When set, emails
without '@' will have '@{LOGIN_DOMAIN}' appended.

Example: LOGIN_DOMAIN=example.com transforms "john" to "john@example.com"

Also includes a migration to lowercase existing emails in critical tables:
- password (primary user identity)
- usr (workspace users)
- email_to_igroup (instance group memberships)
- token (active sessions)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 00:01:35 +00:00
dieriba
6418c4bcc6 feat: nextcloud native triggers (#6797) 2026-01-26 16:49:52 +00:00
Ruben Fiszel
95e5d7e469 fix: change min worker version to 1.420.0 2026-01-26 16:02:20 +00:00
Ruben Fiszel
1ac57589cf change min to 1,580,0 2026-01-26 15:53:50 +00:00
Pyra
1ae157dadd fix: mixed version error (#7686)
* fix mixed version error

Signed-off-by: pyranota <pyra@duck.com>

* fix comp error

Signed-off-by: pyranota <pyra@duck.com>

* fix

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
2026-01-26 15:49:31 +00:00
centdix
29b274a08a fix(aiagent): fix usage for gemini 3 models (#7682)
* fix for gemini3 models

* cleaning

* cleaning

* small opti
2026-01-26 13:15:18 +00:00
centdix
2553d987d9 feat(ai): native bedrock compatibility (#7668)
* check endpoint

* use flag for cred check on worker

* use sdk for auth chat side

* cleaning

* cleaning

* also handle GET

* Add session token support in BedrockClient::from_credentials

Users with temporary STS credentials need session tokens for AWS
authentication. This adds the optional session_token parameter to
support these use cases.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Cache AWS SDK config loading to avoid repeated I/O

aws_config::load_defaults() performs environment variable lookups and
potentially file I/O on each call. Cache the result in a static OnceCell
to improve performance on high-frequency Bedrock requests.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Add debouncing to Bedrock credential check button

Prevent rapid clicks from spawning multiple concurrent flow preview
jobs for the worker credential check.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Remove orphaned SigV4 signing comment

The manual SigV4 signing code was removed in favor of the AWS SDK,
but this documentation comment was left behind.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Remove verbose tracing info and change-related comments

- Remove per-request auth method info logs (too verbose for production)
- Simplify from_env log to single debug-level message
- Remove comments describing future changes rather than current behavior

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* cleaning

* Fix Bedrock feature flag compilation warnings

Gate AWS-specific imports, struct fields, and methods behind the bedrock
feature flag to eliminate dead code warnings when building without the
bedrock feature enabled.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* bedrock flag

* cleaning

* Move BedrockCredentialsCheck from edit drawer to add drawer

The component was incorrectly shown when editing resources instead of
when adding new ones. Moved it from ResourceEditor.svelte to
ApiConnectForm.svelte.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* cleaning

* cleaning

* make aws-config optional

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-01-26 11:04:14 +00:00
Ruben Fiszel
181e68d266 refactor: replace v2 live migration with regular SQL migration (#7678)
* refactor: replace v2 live migration with regular SQL migration

Remove the v2_finalize live migration that waited for workers to reach
v1.461+ and replace it with an idempotent SQL migration. Since enough
time has passed, all deployments are assumed to be on v1.461+.

Changes:
- Remove v2_finalize() function and spawn task from live_migrations.rs
- Remove MIN_VERSION_IS_AT_LEAST_1_461 constant from min_version.rs
- Add 20260125000000_v2_finalize.up.sql with all finalization steps
- All SQL operations use IF EXISTS/CASCADE for idempotency

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: remove migration-related delays and tighten CLI tests CI trigger

- Remove 5s backend initialization delay in cargo_backend.ts that was
  needed for the v2 live migration (now replaced with SQL migration)
- Restrict CLI tests workflow to only trigger on cli/** changes
  (removed backend/**, openapi.yaml, openflow.openapi.yaml paths)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: skip v2_finalize migration if live migration already ran

Wrap migration in DO block that checks for 'v2_finalize_job_completed'
in windmill_migrations table. If present (live migration already ran),
skip entirely to avoid unnecessary table locks on upgraded instances.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 22:32:21 +00:00
Ruben Fiszel
a22d179903 add GitHub Actions CI and raw app sync tests (#7677)
* feat(cli): add GitHub Actions CI and raw app sync tests

- Add CLI tests GitHub Action that runs on Linux and Windows
- Add build check job that runs on CLI and openapi.yaml changes
- Uses Rust cargo backend instead of Docker for better CI compatibility
- Add cargo_backend.ts and test_backend.ts for test infrastructure
- Fix Windows path separator bug in raw_apps.ts (use "/" for relative paths)
- Fix PostgreSQL URL parsing in cargo_backend.ts
- Update tests to use gitBranches format instead of deprecated overrides
- Add raw_app_sync.test.ts for raw app sync workflow testing (ignored for now - needs EE)
- Skip tests that require EE features (git sync settings, raw apps)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): Fix Windows path compatibility issues in tests

- Use fromFileUrl() in cargo_backend.ts for proper Windows path handling
- Normalize path separators to forward slashes in resource_folders.ts
- Fix readDirRecursive to return normalized paths in test helper
- Use forward slashes consistently in buildMetadataPath and detection functions

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): Use SEP in test assertions instead of modifying logic

- Revert resource_folders.ts to use SEP as intended
- Update test assertions to use SEP for platform-specific paths
- Keep readDirRecursive normalization for consistent test comparisons

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): Use SEP for all path separators in test assertions

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): Use resolve() for proper cross-platform path handling in cargo_backend

String concatenation with path separators creates malformed paths on Windows.
Use path.resolve() instead for proper cross-platform path resolution.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(backend): Add cfg attributes for Windows compatibility

- Add #[cfg(unix)] to anyhow::anyhow import (only used in unix cfg block)
- Add #[cfg(not(windows))] to parse_file function (uses cat, only for cgroups)
- Remove unused std::io import, use std::io::Result directly

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: Windows compilation + convert integration tests to withTestBackend

- Fix unused import SYSTEM_ROOT in csharp_executor.rs on Windows by
  requiring both windows and csharp feature
- Fix unused variable id in handle_child.rs on Windows by adding
  #[allow(unused_variables)] since id is only used in cfg(unix) code
- Convert all RUN_INTEGRATION_TESTS dependent tests in sync_pull_push.test.ts
  to use withTestBackend pattern for automatic backend setup

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: configurable test features with CI_MINIMAL_FEATURES env var

- Default: full features (zip, private, enterprise) for local development
- CI mode: minimal features (zip only) when CI_MINIMAL_FEATURES=true
- Add shouldSkipOnCI() helper for tests requiring EE features
- Update EE-dependent tests to use shouldSkipOnCI()
- Add test instructions to cli/README.md

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: enable raw app tests (not EE-dependent)

Raw apps work with minimal features. 2 tests pass, 2 have test logic
bugs to investigate separately:
- "delete file and push" - file deletion not syncing correctly
- "dry-run push shows expected changes" - JSON output parsing issue

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: gate cgroups module to Linux only

cgroups are Linux-specific, the module was causing dead_code warnings
on Windows compilation.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(ci): add CI_MINIMAL_FEATURES env var to CLI tests workflow

Set CI_MINIMAL_FEATURES=true in both Linux and Windows test jobs so
the backend compiles with minimal features (zip only) and EE-dependent
tests self-skip via shouldSkipOnCI().

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): raw app tests and backend startup timing

- Add 5s delay after backend ready for migrations to complete
- Fix dry-run JSON output parsing (handle pretty-printed JSON)
- Temporarily ignore "delete file" test (needs isSuperset fix)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): raw app file deletion sync

- Add deepEqual check for files in raw_apps.ts isSuperset comparison
- Handle raw_app file deletions in sync.ts by re-pushing the entire app
- Fix test to remove CSS import before deleting the file

When deleting a file from a raw app, the sync now properly updates
the backend with the new file list (excluding the deleted file).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): Windows path separators in tests

Normalize paths for cross-platform comparison by converting
backslashes to forward slashes before path assertions.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): normalize featurePaths in multi_instance_workspace test

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test(cli): add mixed case paths sync tests for Windows compatibility

Add comprehensive tests for sync pull/push with capitalized folder paths
to catch Windows case-insensitivity issues:
- Scripts in f/MyFolder/MyScript
- Flows in f/MyFlows/DataProcessor
- Apps in f/MyApps/Dashboard
- Variables in f/MyVars/ApiKey
- Deeply nested paths with mixed case
- Multiple resources in same capitalized folder
- CamelCase folder names with numbers

Each test verifies the full pull -> modify -> push -> verify cycle.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test(cli): add idempotency check to mixed case paths tests

After each push, pull again with --dry-run --json-output and verify
that no changes are detected. This ensures the sync is stable and
catches issues where pull/push cycles cause spurious diffs.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 20:38:33 +00:00
Ruben Fiszel
85e460d853 feat: add cache-rt command and SYNC_CACHED_RT env variable for resource types (#7666)
* feat: add cache-rt command and SYNC_CACHED_RT env variable for resource types

Add a new CLI command `cache-rt` that fetches resource types from the hub
and caches them locally. This command is run during Docker image build to
pre-populate the cache.

Add `SYNC_CACHED_RT` environment variable that, when set to true, syncs
cached resource types to the admins workspace's resource_type table on
server startup. The sync is incremental - it only updates resource types
that have changed.

Changes:
- Add HUB_RT_CACHE_DIR constant for resource type cache location
- Add cache-rt CLI command to fetch and cache resource types from hub
- Add sync_cached_resource_types() to sync cached types to database
- Update all Dockerfiles to run cache-rt during image build

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: parse schema string from hub into proper JSONB

The hub API returns schema as a JSON string that needs to be parsed
into a proper JSON value before inserting into the database.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-23 12:23:07 +00:00
Alexander Petric
08b483eaca fix: use pgoptions for iam rds connection (#7660)
* use pgoptions

* fix: use pgoptions for iam rds connection

* ee ref

* chore: update ee-repo-ref to 1549849fadc4e5634334a384bfe52343eb1e93f0

This commit updates the EE repository reference after PR #388 was merged in windmill-ee-private.

Previous ee-repo-ref: ffc1de1498a8018a9cbc2daba846e6c57d500a1c

New ee-repo-ref: 1549849fadc4e5634334a384bfe52343eb1e93f0

Automated by sync-ee-ref workflow.

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-01-23 12:09:40 +00:00
Pyra
7249b82dba feat: better mixed versions handling (#7628) 2026-01-22 23:16:57 +00:00
centdix
0797e89aa0 feat(ai): handle google vertex for claude models + base url overrides (#7654)
* fix hardcoded gemini url

* allow overriding any provider url

* handle vertex

* same for chat proxy

* fix for chat
2026-01-22 17:28:51 +00:00
Diego Imbert
1526d3ae2b fix: S3 advanced custom permissions (#7632)
* audit_author

* Fix S3 Permissions

* ee

* chore: update ee-repo-ref to e8605e72a6c93c9cf43737ebea74dd28e1f00e83

This commit updates the EE repository reference after PR #381 was merged in windmill-ee-private.

Previous ee-repo-ref: 0c8638d3895a1ead9422fc8e428e3e0405e3a060

New ee-repo-ref: e8605e72a6c93c9cf43737ebea74dd28e1f00e83

Automated by sync-ee-ref workflow.

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-01-20 20:44:10 +00:00
Diego Imbert
0e91a86458 [ee] S3 Buckets workspace restrictions (#7627)
* S3_BUCKETS_WORKSPACE_RESTRICTIONS

* ee repo ref

* nit frontend refactor
2026-01-20 16:50:32 +00:00
Pyra
95df7b9a6a feat: otel REST tracing (#7571) 2026-01-20 13:38:02 +00:00
Ruben Fiszel
f33b79936b fix: add HIDE_WORKERS_FOR_NON_ADMINS env var and workspace-scoped custom_tags endpoint (#7613) 2026-01-19 19:14:18 +00:00
Ruben Fiszel
1b9d1c56c7 feat: add HashiCorp Vault secret storage integration (#7599)
* feat: add HashiCorp Vault secret storage integration

- Create SecretBackend trait abstraction for secret storage
- Add VaultBackend implementation with CRUD operations
- Integrate secret backend into variable CRUD operations
- Add migration functions (DB → Vault and Vault → DB)
- Add frontend configuration UI for secret backend
- Add test connection and migration endpoints
2026-01-18 23:08:29 +00:00
Ruben Fiszel
8dd5e81a32 fix: improve job deletion performance and batching 2026-01-17 07:35:46 +00:00
Ruben Fiszel
57fd32b11d fix improve tags sensitivity behavior for certain backend apis 2026-01-09 17:32:44 +00:00
Ruben Fiszel
d432cdbec1 split out windmill-mcp + pin tokio-utils 2026-01-09 06:16:26 +00:00
centdix
a8e34de58d upgrade rmcp crate (#7518) 2026-01-07 16:02:51 +00:00
Pyra
b31d8dffc3 feat(git-sync): sync jobs debouncing for greatly improved perf (#7489)
* feat(git-sync): sync jobs debouncing [merge-ee-first]

* add compat

* ee repo

Signed-off-by: pyranota <pyra@duck.com>

* nit

Signed-off-by: pyranota <pyra@duck.com>

* change is_script_meets_min_version

Signed-off-by: pyranota <pyra@duck.com>

* update version

Signed-off-by: pyranota <pyra@duck.com>

* chore: update ee-repo-ref to bc904859dd66c55ebad002e8526103c73de841cd

This commit updates the EE repository reference after PR #371 was merged in windmill-ee-private.

Previous ee-repo-ref: 346d38e91776e987f88866543adbf78a9c721073

New ee-repo-ref: bc904859dd66c55ebad002e8526103c73de841cd

Automated by sync-ee-ref workflow.

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-01-07 04:07:26 +00:00
claude[bot]
62bb11e297 feat: implement quiet mode to filter verbose logs (#7478)
* feat: implement quiet mode to filter verbose logs

Add QUIET_MODE environment variable (set to "true" or "1" to enable) that:
- Filters logs with the "windmill_verbose" target via tracing_init
- Increases frequency of periodic logs by 10x when enabled
- Marks per-job worker logs as verbose so they are suppressed in quiet mode

Key changes:
- Add QUIET_MODE lazy_static and VERBOSE_TARGET constant to tracing_init.rs
- Create create_targets_filter() helper that filters windmill_verbose target
- Mark periodic "still running" and "memory snapshot" logs with VERBOSE_TARGET
- Mark per-job "fetched job", "started handling", "job finished" logs as verbose
- Increase memory snapshot interval from every 10 ticks to every 100 ticks in quiet mode
- Increase "still running" log interval from every tick to every 10 ticks in quiet mode
- Add quiet mode notification in job logs

Closes #7477

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* Update tracing_init.rs

* Update worker.rs

* Remove target from tracing info logs

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-01-06 18:38:58 +00:00
Diego Imbert
e6757994d5 fix: ducklake default connection extra_args (#7509)
* Ducklake default extra_args

* indicator nit
2026-01-06 17:55:05 +00:00
Pyra
cbcf0aa344 feat(python): set latest stable to 3.12 (#7405)
Signed-off-by: pyranota <pyra@duck.com>
2025-12-23 20:50:43 +00:00
Pyra
0f51f9702c fix: clone script by path instead of hash (#7439)
* fix: clone script by path instead of hash

Signed-off-by: pyranota <pyra@duck.com>

* Update backend/windmill-common/src/scripts.rs

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
2025-12-23 16:05:58 +00:00
Pyra
9d698dabb4 feat: v2 job debouncing (#7411)
* rework everything again

Signed-off-by: pyranota <pyra@duck.com>

* updcate sqlx

Signed-off-by: pyranota <pyra@duck.com>

* update ref

Signed-off-by: pyranota <pyra@duck.com>

* fix things

Signed-off-by: pyranota <pyra@duck.com>

* fix function

Signed-off-by: pyranota <pyra@duck.com>

* final fixes

Signed-off-by: pyranota <pyra@duck.com>

* update sqlx

Signed-off-by: pyranota <pyra@duck.com>

* fix script creation

Signed-off-by: pyranota <pyra@duck.com>

* address todo

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* remove dbg

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fix

Signed-off-by: pyranota <pyra@duck.com>

* fixups

Signed-off-by: pyranota <pyra@duck.com>

* fix cargo.toml

Signed-off-by: pyranota <pyra@duck.com>

* update ee repo

Signed-off-by: pyranota <pyra@duck.com>

* fix ci

Signed-off-by: pyranota <pyra@duck.com>

* nit

Signed-off-by: pyranota <pyra@duck.com>

* ref

Signed-off-by: pyranota <pyra@duck.com>

* fix

Signed-off-by: pyranota <pyra@duck.com>

* ee repo

Signed-off-by: pyranota <pyra@duck.com>

* sqlx

Signed-off-by: pyranota <pyra@duck.com>

* ee ref

Signed-off-by: pyranota <pyra@duck.com>

* remove dbg

Signed-off-by: pyranota <pyra@duck.com>

* sqlx

Signed-off-by: pyranota <pyra@duck.com>

* feat: v2 job debouncing

Signed-off-by: pyranota <pyra@duck.com>

* v2 debouncing

Signed-off-by: pyranota <pyra@duck.com>

* compat

Signed-off-by: pyranota <pyra@duck.com>

* fix all

Signed-off-by: pyranota <pyra@duck.com>

* remove legacy logic

Signed-off-by: pyranota <pyra@duck.com>

* nit

Signed-off-by: pyranota <pyra@duck.com>

* finish

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* stage import

Signed-off-by: pyranota <pyra@duck.com>

* remove unused import

Signed-off-by: pyranota <pyra@duck.com>

* remove comments

Signed-off-by: pyranota <pyra@duck.com>

* ee ref

Signed-off-by: pyranota <pyra@duck.com>

* ee ref

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* Update backend/windmill-queue/src/jobs.rs

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* Update backend/windmill-common/src/runnable_settings/settings.rs

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* Update frontend/src/lib/components/flows/DebounceLimit.svelte

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* fix nits

Signed-off-by: pyranota <pyra@duck.com>

* force no compat in ci

Signed-off-by: pyranota <pyra@duck.com>

* better default key handling

Signed-off-by: pyranota <pyra@duck.com>

* simplify

Signed-off-by: pyranota <pyra@duck.com>

* sqlx

Signed-off-by: pyranota <pyra@duck.com>

* optimize

Signed-off-by: pyranota <pyra@duck.com>

* sqlx

Signed-off-by: pyranota <pyra@duck.com>

* fix flow step

Signed-off-by: pyranota <pyra@duck.com>

* add openapi

Signed-off-by: pyranota <pyra@duck.com>

* fix ci

Signed-off-by: pyranota <pyra@duck.com>

* ee ref

Signed-off-by: pyranota <pyra@duck.com>

* rename WMDEBUG flag

Signed-off-by: pyranota <pyra@duck.com>

* ee ref

Signed-off-by: pyranota <pyra@duck.com>

* remove gate for is_flow_step on pull

Signed-off-by: pyranota <pyra@duck.com>

* remove dead code

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* chore: update ee-repo-ref to 71d872d27de44227b82f75a9c0879d8428684880

This commit updates the EE repository reference after PR #369 was merged in windmill-ee-private.

Previous ee-repo-ref: 4c7642dc9f7f4467fc055136bd56558b66ef2d54

New ee-repo-ref: 71d872d27de44227b82f75a9c0879d8428684880

Automated by sync-ee-ref workflow.

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-23 13:24:04 +00:00
claude[bot]
6be060bea8 feat(ai): add websearch tool for AI agents (#7399)
* refactor(ai): use responses API for OpenAI/Azure, create 'other' provider for completion endpoint

- Created new 'other.rs' provider using the OpenAI-compatible completion endpoint
- Refactored 'openai.rs' to use the Responses API for both text and image output
- Updated query_builder to route OpenAI/AzureOpenAI to OpenAIQueryBuilder
- All other providers (Mistral, DeepSeek, Groq, etc.) now use OtherQueryBuilder
- Updated OpenRouter to delegate to OtherQueryBuilder instead of OpenAIQueryBuilder

This prepares the codebase for adding websearch tool support using the Responses API.

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* feat(ai): add websearch as AI agent tool type

- Added WebsearchToolValue to ToolValue enum in flows.rs
- Updated all pattern matches to handle websearch tool type
- Added has_websearch parameter to run_agent function
- Websearch tools don't require additional configuration

This prepares the backend for provider-specific websearch implementations.

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* feat(frontend): add websearch as tool option in AI agent UI

- Added WebsearchTool type and utility functions to agentToolUtils.ts
- Added "Web Search" option in tool insertion menu (InsertModuleInner)
- Updated NewAIToolNode to handle pickWebsearchTool event
- Updated AIToolNode to render websearch tools properly
- Updated FlowModuleSchemaMap to create websearch tools

Users can now add websearch tools to AI agents through the UI.

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* feat(ai): implement websearch for OpenAI provider

- Added has_websearch parameter to BuildRequestArgs
- OpenAI provider now adds web_search tool to requests when enabled
- Uses OpenAI Responses API web_search tool type
- Websearch tool is added before other custom tools in the request

Implements websearch functionality for OpenAI and Azure OpenAI providers.

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* feat(ai): implement websearch for Anthropic provider

- Created anthropic.rs provider with native Anthropic API format
- Added web_search tool to Anthropic requests when enabled
- Anthropic uses /messages endpoint with x-api-key authentication
- Updated query_builder to route Anthropic to new provider

Implements websearch functionality for Anthropic Claude models.

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* feat(ai): implement websearch for Gemini/GoogleAI provider

- GoogleAI now uses completion endpoint (other.rs) for text instead of responses API
- Added Google Search grounding when websearch is enabled
- Uses google_search_retrieval tool in request when has_websearch is true
- Updated parse methods to use OtherQueryBuilder for completion endpoint

Implements websearch functionality for Google Gemini models.

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* fix frontend

* fix anthropic and openai

* better for gemini

* structured output

* cleaning

* fix validate tool

* fixes

* cleaning

* cleaning

* fix for openai

* no responses api for azure

* fixes

* fix

* add tests for ai agent

* avoid panic

* better tests

* test user images

* fix tool choice

* always use streaming backend side

* big cleaning

* show annotations plus agent action for open ai websearch use

* show annotations plus agent action for anthropic websearch use

* show annotations plus agent action for google websearch use

* nit forntend

* rm

* fix

* add test for image ouptut

* fix for azure

* add in openflow

* fix

* fix

* nit tests

* fixes

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: centdix <farhadg110@gmail.com>
2025-12-22 11:41:28 +00:00
Diego Imbert
afe74f74fa fix: improve MS SQL Numeric rounding (#7404)
* Fix MS SQL Numeric rounding

* Fix result collection to avoid JSON parsing

* don't reparse json through sqlx, pass raw string to postgres

* Revert "don't reparse json through sqlx, pass raw string to postgres"

This reverts commit 355691fa7d.

* mistake

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-12-20 09:18:46 +00:00
Diego Imbert
3affbb3321 feat: type-checked data tables v0 (#7381)
* data tables settings ui

* install runed

* zod 4 fixes

* use new toJSONSchema

* Migrate ducklake catalogs to more generic custom instance databases

* fix compilation

* Safety conversion for old duckdb ffi

* data tables settings

* ts client basis

* inline run works

* datatables work

* Revert "datatables work"

This reverts commit 6e1588d59e.

* datatables work (without leaking pg credentials)

* println

* separate sqlUtils.ts

* nit

* Separate custom instance db Select and Wizard components

* nit

* nit wording

* add tags to custom instance dbs

* error when trying to use ducklake as datatable or opposite

* show status in dropdown

* data table instance setup works

* sqk function for ducklake

* factorize logic

* fix temp reactivity

* Data table assetexplore

* Migrate S3 permissions to modal

* Revert "Migrate S3 permissions to modal"

This reverts commit 0631d03cb0.

* nit query -> fetch

* Custom instance setup new look

* run_language_executor separate fn

* run_inline param

* nit wording

* Better typed client

* Data tables display as assets in frontend

* asset db icon

* nit

* cleaner errors

* nit

* Fix sed calls in mac

* run_inline_script_preview in python client

* basic python datatable client

* datatable and datalake parser in python

* ducklake client python

* nit fix

* Fix migration producing NULL instead of {} when no custom databases

* merge conflict fail

* python ducklake client arg fix

* parse or infer sql types in ts client

* ts asset parser, detect datatable & ducklake R/W

* fix sql repl for other read ops than select

* export type SqlTemplateFunction

* rename list_custom_instance_pg_databases

* typecheck datatable and ducklake name in Typescript

* Fix typecheck datatable and ducklake in TS

* declare module overriding instead of extending

* infer_sql_type in python client

* SqlQuery object in python

* fix merge conflicts

* update const_format

* CI fix

* factor out to var_identifiers

* sqlx prepare

* unnecessary security (admin is required)

* clearer comment

* ee repo ref

* nit snake case

* claude step 1: detect var declarations

* move detect_sql_access_type to common mod

* claude step 2: detect when saved vars are queried

* Revert "claude step 2: detect when saved vars are queried"

This reverts commit 1e1f930568.

* Revert "claude step 1: detect var declarations"

This reverts commit f866f4819d.

* remove ducklake/datatable and default

* detect data table assigns in var_identifiers

* Python parser successfully infers R/W/RW from ducklake / datatable

* still register ducklake/datatable if not used as unknown R/W

* Go to settings button in Assets Dropdown on not found

* nit

* sqlx prepare fail

* manual fix, somehow sqlx prepare won't do it

* fix frontend ci

* ee repo ref

* ducklake_user doesnt exist in unit tests

* nit fix

* ui nit

* nit

* nit missing clone

* fork ducklakes and datatables

* fix surface hover bug

* stupid mistake

* better deeply reactive mutable derived

* Ducklake picker

* Editor bar data tables

* DuckDB supports datatables

* datatable in duckdb asset parser

* duckdb asset parser var_identifiers

* Revert "duckdb asset parser var_identifiers"

This reverts commit 88068b1a77.

* sqlx prepare

* Box pin in test_workflow_as_code to fix stack overflow

* stash

* sql asset parser parses most s3 literals

* nit

* Detect attach + handle returning RW

* detect assets used with dot notation

* detect implicit access with USE dl; syntax

* Add assets as unknown if var was never used

* Support default ducklake/datatable main in parser

* ignore asset parsing errors in frontend (avoid flow layout shift)

* super weird duplication (merge conflict ?)

* nits

* fix duckdb parser detecting too much as asset when RW ctx is unknown

* fix transparent assets btn

* missing arg

* nit styling

* asset parser specific table parsing

* fix resource specific table parsing

* More concise asset display in flows + better icons

* fix assets page filtering out resources with added table

* Fix frontend to support specific table assets

* Open DB Manager to specific table

* Specific table parser in Python and TS + unit tests

* Fix UPDATE setting access to None

* fix flow edge rendering on top of output picker

* python parser fix var override bug

* add ts test

* fix compilation

* sqlx prepare

* update parsers version

* fix missing schema key onDelete

* Grant permission to create schemas in custom instance databases

* Update pg query to return empty schemas

* Create schema

* Select nits

* support schemas in sql parser

* ts parser handle schema with sql parser result

* detect .schema() syntax

* detect schema syntax in python

* support .schema() in ts and py SDK

* open db manager to specific schema

* support reassignment in ts parser

* nit better unitest

* : syntax in ts

* datatable:schema syntax in python

* fix client py

* nit select dropdown darkmode

* object | null fetchOne

* ts client nits

* parse_sql_client_name fn

* getImportWmillTsStatement refactor in EditorBar

* text to json() in python client

* update parser versions

* pkg lock

* Sql query details in TS asset parser

* code transformation with type parameter in Editor

* Custom Language Worker, code substition works !

* Error marker mapping works

* hover info is correct

* completions work correctly

* other overrides

* type inference kinda works

* Position mapping tests

* refactor prepare_queries

* Refactor PgDatabase to share common code

* Pgdatabase in prepare_queries

* TokioPgConnection refactor

* refactor prepare_queries

* type parameter to sql function

* Fix deadlock

* nit fix

* Fix worker async call freezing because of svelte Proxy

* Force worker to recompute when we set queries

* nit refactor

* nits console logs

* wait that ts worker initialize

* monaco change file version

* update diagnostics

* Refactor for errors

* Show SQL errors in Monaco

* improve sdk

* cleaning refactor + MapResource + usePreparedAssetSqlQueries

* Fixes

* Fix error position mapping

* cache in typescript worker

* fix insert no values

* don't inject type if already present

* Support schema in prepare queries

* update parsers

* ChangeOnDeepInequality

* inferAsset ScriptEditor usage refactor

* sql query typecheck work in flow editor

* Assets and SQL Query check in Raw App Inline Editor

* pkg lock

* Fix DatatableSqlTemplateFunction nit

* prepare query schema nit

* duplicate diagnostics

* nit getScriptVersion mock

* Reprepare queries when switching workspaces

* nit fix

* nit fix

* fetch_one_scalar and execute in python client

* limit pg_connections

* -- prepare flag in postgres

* skip serializing

* fix destructuring undefined

* Prepare queries in workers instead of backend

* nit

* Execute search_path instructions normally

* nit fix

* Fix SET search_path issue in prepare

* only support preparing single-statement queries for now

* update parsers

* safety

* better remove_comments

* Fix getQueryStmtCountHeuristic

* getQueryStmtCountHeuristic tests

* comment out failing tests

* Fix getQueryStmtCountHeuristic impl

* only datatable
2025-12-20 08:51:07 +00:00
hugocasa
f89fb292da fix(backend): put for loop itered in a separate table (#7419)
* fix(backend): put for loop itered in a separate table

* Update SQLx metadata

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-19 19:46:46 +00:00
hugocasa
a6993823af feat: restart flow from step with different flow version (#7409)
* feat: restart flow from step with different flow version

* fix tests

* fix tests
2025-12-18 20:27:51 +00:00
Diego Imbert
ca45937166 Refactor PgDatabase to share common code (#7386)
* Refactor PgDatabase to share common code

* TokioPgConnection refactor
2025-12-17 11:34:39 +00:00