Commit Graph

12764 Commits

Author SHA1 Message Date
Ruben Fiszel
c41565b6b8 perf: allow BATCH_PULL_SIZE in any mode for benchmarking
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 12:06:33 +00:00
Ruben Fiszel
d1b8d5427b perf: batch commit completed jobs in single transaction
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 18:41:10 +00:00
Ruben Fiszel
fc5e479424 chore: update ee-repo-ref.txt for batch_pull endpoint
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 18:04:38 +00:00
Ruben Fiszel
da18a69808 perf: add agent-batch mode with batch pull from server
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 17:59:10 +00:00
Ruben Fiszel
1b6e2556b7 perf: add worker-side batch job pull from DB
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 15:37:53 +00:00
Diego Imbert
470b8aa5f1 feat: add status indicator dots to parallel loop iteration picker (#8761)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 13:26:21 +00:00
Diego Imbert
5713760b7a Fix TS typechecker for Ducklake emitting error for NULL params (#8760) 2026-04-08 12:43:41 +00:00
Ruben Fiszel
f5c9ff709b sqlx 2026-04-08 06:11:22 +00:00
Ruben Fiszel
f0bb270723 add missing delete_after_secs column to explicit SQL queries (#8759)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 06:00:37 +00:00
Ruben Fiszel
4c16877366 update sys prompts 2026-04-08 05:38:38 +00:00
Ruben Fiszel
4342c18541 feat: add CLI workspace merge command and enhance fork with datatable/color support (#8756)
* feat: add CLI workspace merge command and enhance fork with datatable/color support

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: abort fork on git branch failure, per-datatable error handling, guard resetDiffTally

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: add fork/merge integration tests covering full cycle

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: support deploying deletions during fork merge (archive/delete in target)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: share deploy logic between CLI and frontend via windmill-utils-internal

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: revert frontend to self-contained deploy, fix failure_module handling

The frontend imports windmill-utils-internal from npm (published v1.3.4)
which doesn't have the new deploy module yet. Revert frontend to its own
self-contained implementation with two improvements:
- Pass failure_module to getAllModules in flow deploy and getItemValue
- Add deleteItemInWorkspace for deploying deletions during merge

The shared deploy.ts in windmill-utils-internal remains for CLI use.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: share deploy logic via published windmill-utils-internal, add comprehensive integration tests

- Publish windmill-utils-internal v1.3.8 with DeployProvider interface
- Frontend now uses shared deploy module (deployItem, deleteItemInWorkspace,
  checkItemExists, getOnBehalfOf, getItemValue) via provider adapter
- Add 4 new integration test sub-tests: all item types, secret variables,
  special characters, partial deploy + resetDiffTally

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: remove unused folderName function from frontend utils_workspace_deploy

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-04-08 05:29:48 +00:00
Ruben Fiszel
01e6414ddb Squashed commit of the following:
commit a5400b92cc4d523589d7e3c98d866c56d950dd9f
Author: Ruben Fiszel <ruben@windmill.dev>
Date:   Wed Apr 8 04:24:25 2026 +0000

    fix
2026-04-08 04:25:26 +00:00
Ruben Fiszel
2d18a68099 feat: add scheduled job deletion with configurable retention period (#8753)
* feat: add scheduled job deletion with configurable retention period

Extends delete_after_use with delete_after_secs to enable configurable
retention periods for job args/result/logs. At completion, jobs can be
scheduled for future deletion via a new job_delete_schedule table,
processed by a monitor task. Supports per-script, per-flow, and
per-flow-step configuration. Backward compatible.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add integration tests, revert query! macros, fix review issues

- Add integration tests for resolve_delete_after_secs, schedule_job_deletion,
  flow-level and module-level delete_after_secs, backward compat
- Revert sqlx::query() back to sqlx::query!() macros for compile-time safety
- Regenerate sqlx offline cache
- Fix FlowModule/NewScript/FlowValue constructions in all test files
- Fix autoscaling_ee.rs for updated script_path_to_payload return type

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref.txt for autoscaling_ee fix

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: gate cleanup_scheduled_job_deletions behind enterprise feature

Prevents dead_code warning (which CI treats as error via -D warnings)
when compiling without enterprise feature.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: regenerate sqlx cache after merge with main

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address review feedback on scheduled deletion

- Monitor: roll back transaction on any cleanup error so schedule rows
  survive for retry on next cycle (instead of best-effort then discard)
- Migration: add FK with ON DELETE CASCADE to job_delete_schedule.job_id
  to prevent orphan rows when jobs are deleted through other means
- Simplify bool-to-Option conversion with .then_some(true)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: stop setting delete_after_use alongside delete_after_secs

No mixed-version deployment scenario exists, so delete_after_secs alone
is sufficient. The backend's resolve_delete_after_secs handles
(None, Some(secs)) correctly without needing delete_after_use set.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: remove delete_after_use from public API surface

Remove delete_after_use from OpenAPI spec, API client, runtime client,
and workspace export. Only delete_after_secs is exposed going forward.

The field remains in Rust backend types with #[serde(skip_serializing)]
for backward-compatible deserialization of existing scripts/flows that
were saved with delete_after_use: true.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 1d4b7a31fc115d6aba8640f7cd3fd5a01abe6806

This commit updates the EE repository reference after PR #519 was merged in windmill-ee-private.

Previous ee-repo-ref: 9eba09a13b778caafc6ae65098b90e53c91984d3

New ee-repo-ref: 1d4b7a31fc115d6aba8640f7cd3fd5a01abe6806

Automated by sync-ee-ref workflow.

* fix: regenerate system prompts, remove unused import

- Regenerate auto-generated system prompts after openflow schema change
- Remove unused serde_json::json import in test file (CI -D warnings)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: insert dummy v2_job row in schedule tests for FK constraint

The job_delete_schedule table has a FK to v2_job, so tests need a
real v2_job row before inserting into the schedule table.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: trigger CI re-run

* fix: remove heavy flow integration tests to avoid CI worker contention

The flow integration tests spawn workers that compete for CPU with
the existing relock_skip tests under --test-threads=10, causing
consistent 60s timeouts in CI. Keep only the lightweight unit tests
and DB integration tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: restore correct ee-repo-ref for our branch

The ref was overwritten to main's EE ref during a rebase. Restore to
our branch's EE commit that includes the autoscaling tuple fix.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: retrigger CI on fresh runner

* fix: remove FK constraint from job_delete_schedule to unblock CI

The FK with ON DELETE CASCADE to v2_job may have caused performance
overhead during test DB setup (each sqlx::test creates a fresh DB
with all migrations). Remove the FK — orphan schedule rows are
harmlessly cleaned by the monitor.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* ee-ref

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-04-08 04:15:28 +00:00
Ruben Fiszel
9e6427d150 chore(main): release 1.679.0 (#8755)
* chore(main): release 1.679.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
v1.679.0
2026-04-07 21:14:29 +00:00
Diego Imbert
3d4f4c6c38 feat: Fork datatables (#8339)
* export_datatable_schema

* Propose to fork the datatable on ws fork

* dump datatable

* Dockerfile

* Fix import_datatable_dump

* datatable schema fork works!

* Option to copy both schema and data

* Datatable fork behavior

* nit ui

* use psql instead

* remove fork_datatable route

* feat: add fork_pg_database and export_pg_schema routes with DB Manager UI

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style: pluralize "schema" to "schemas" in DB Manager export/import UI

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add import mode select (schema only vs schema + data) to DB Manager import

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Select schema or schema+data when important database

* fix: prepend $res: prefix to resource paths in DB Manager import/export

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style: dynamic import button label based on selected mode

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* nits

* feat: add warning alert when schema+data import mode is selected

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* nit hide on cloud hosted

* refactor: remove fork_behavior from datatable settings

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: split CreateWorkspace into layout wrapper and CreateWorkspaceInner

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: instantiate CreateWorkspaceInner in globalForkModal

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* nit icons

* Data table fork UI

* feat: pass per-datatable fork behaviors from UI to backend during workspace fork

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix fork overwriting all datatables

* UI nits

* custom instance db refactor

* custom instance db wizard btn for all in dropdown

* nit

* Delete custom instance database button

* Disable forking for resource datatables

* Big import buttons when db empty

* Revert "Disable forking for resource datatables"

This reverts commit 9561cc8fd4.

* feat: add non_diffable flag to resource table

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add resource-type datatable fork with CREATE DATABASE

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: tag forked datatables with nonDiffable and forkedFrom

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: diff datatable and ducklake settings individually on workspace merge

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: skip non_diffable resources and datatables in workspace diff

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: default datatable fork behavior to keep_original

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: make grant permissions non-fatal in instance datatable fork

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: make datatable and ducklake diffs visible in workspace comparison

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: remove datatable fork logic from workspace fork route

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: correct ahead/behind logic for datatable and ducklake diffs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Revert "fix: correct ahead/behind logic for datatable and ducklake diffs"

This reverts commit 6b50884dc6.

* revert: remove datatable and ducklake settings diffing logic

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add datatable clone UI with step-by-step confirmation modal

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: extract datatable fork UI into ForkDatatableSection component

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit

* fix: run datatable cloning before workspace fork creation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit disable fork admins

* nit fix switching workspace prematurely

* fix: use source workspace for forkPgDatabase calls during fork

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: update forked workspace datatable settings after fork creation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add forked_from field to DataTable and set it for instance forks

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit onFinish

* fix: add forked_from to DataTableSettings OpenAPI schema

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: track datatable table DDL changes in workspace_diff

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Revert "feat: track datatable table DDL changes in workspace_diff"

This reverts commit 7526dd68b9.

* feat: add get_datatable_full_schema endpoint and snapshot schema on fork

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix duplicate migration key

* fix: set forked_from on datatable config for both instance and resource types

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nits

* feat: drop forked databases on workspace deletion with confirmation UI

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: extract drop_forked_datatable_databases from delete_workspace

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: cast pg char columns to text in FK schema query

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: show dbname instead of resource type in fork deletion modal

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* ui nit

* refactor: extract drop_custom_instance_database into windmill-common

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add datatable schema diff section to merge UI

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* UI

* feat: add review drawer with YAML diff and SQL migration runner

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: use Monaco DiffEditor for YAML diff in review drawer

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit

* Revert "refactor: use Monaco DiffEditor for YAML diff in review drawer"

This reverts commit a86008ba4c.

* Revert "feat: add review drawer with YAML diff and SQL migration runner"

This reverts commit 0a0deb5ddb.

* feat: add review drawer with DiffEditor and SQL migration runner

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* ui nits

* fix: show diff between forked_from schema and changed side

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: re-fetch target live schema after migration for correct baseline

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* revert

* nit auto next

* feat: add confirmation modal before deploying migration to parent

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: handle missing columns/foreignKeys in schema conversion

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nits

* refactor: use temp file on disk for pg_dump instead of in-memory string

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Don't replace postgres dbname

* fix: add validation to drop_custom_instance_database and use source db for CREATE/DROP

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: type DataTable.forked_from as DataTableForkedFrom struct

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: simplify fork_pg_database to take source + target_dbname

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* dead code

* feat: enforce schema_and_data admin-only and extract create_custom_instance_database

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: rename fork_pg_database to import_pg_database with source/target/override params

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit

* refactor: remove original_dbname/original_resource from forked_from, resolve from parent

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit

* fix: resolve forked dbname from fork workspace when dropping resource databases

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nits

* fix: always clean up global_settings even if database doesn't exist

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: check datatable resource_type from config instead of URL prefix

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: wrap PG default value expressions in braces to prevent CAST quoting

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Revert "fix: wrap PG default value expressions in braces to prevent CAST quoting"

This reverts commit 77f5a2c4e8.

* refactor: reuse columnDefToTableEditorValuesColumn for default value handling

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: store raw API schema in forked_from to avoid double transformation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Revert "fix: store raw API schema in forked_from to avoid double transformation"

This reverts commit e326197a20.

* Revert "refactor: reuse columnDefToTableEditorValuesColumn for default value handling"

This reverts commit bd8f071d9f.

* fix: validate dbname with strict regex to prevent SQL injection

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix default value

* always validate dbname

* refactor: move get_datatable_full_schema structs and logic to query_builders.rs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: split import_pg_database into create_pg_database + import_pg_database

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: extract drop_forked_datatable_databases into its own route

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: require admin when using $res: resource paths in import_pg_database

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use UserDB for $res: resource access and restrict dbname creation

- resolve_pg_source_checked uses UserDB (row-level security) for $res: paths
- transform_json_unchecked is now pub(crate) to prevent misuse
- Non-superadmins can only create databases with wm_fork_ prefix
- datatable:// remains accessible to everyone

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: refuse to drop forked databases unless name starts with wm_fork_

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: remove resolve_pg_source, use resolve_pg_source_checked everywhere

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Fix permissions

* sqlx prepare

* compilation nits

* sqlx prepare

* sqlx prepare

* wrong route syntax

* fix: allow workspace owner to edit datatable config for fork setup

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Revert "fix: allow workspace owner to edit datatable config for fork setup"

This reverts commit ab683e637b.

* refactor: move datatable fork setup into create_workspace_fork backend

Instead of updating datatable settings from the frontend after fork
creation (which required admin/owner access), pass forked_datatables
info to create_workspace_fork and handle it atomically in the same
transaction. Removes applyPostForkDatatableUpdates from frontend.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: snapshot schema in backend during fork instead of frontend

The schema snapshot is now taken by the backend in apply_forked_datatable
via snapshot_datatable_schema, which connects to the parent workspace's
datatable and runs pg_get_full_schema. This removes the need for the
frontend to call getDatatableFullSchema and pass the schema through.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use get_resource_value_interpolated_internal for $res: to resolve $var: references

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit

* sqlx prepare

* fix: add permission check to drop_forked_datatable_databases, validate dbnames, restrict temp file perms

- drop_forked_datatable_databases: same permission as delete_workspace
  (fork owner or super admin)
- validate_dbname on target_dbname_override and ForkedDatatableInfo.new_dbname
- Enforce wm_fork_ prefix on forked datatable new_dbname
- DumpFile: set /tmp/windmill/ to 0700 and create files with 0600

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit CLI

* Rename to ws_specific

* sqlx prepare

* nit always validate dbname

* fix: include foreign keys in CREATE TABLE migration for added tables

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: detect nextval defaults and use SERIAL/BIGSERIAL types in CREATE TABLE

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Update frontend/src/lib/components/DBManagerDrawer.svelte

Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>

* Update backend/windmill-common/src/lib.rs

Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>

* Update backend/windmill-common/src/lib.rs

Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>

* fix: sort foreign keys by constraint name for deterministic schema output

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* sqlx prepare

* rename migration to update timestamp

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-04-07 21:03:06 +00:00
Ruben Fiszel
0bcbc8bd3c chore(main): release 1.678.0 (#8745)
* chore(main): release 1.678.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
v1.678.0
2026-04-07 20:25:49 +00:00
Diego Imbert
2413dbefe3 fix: Fix FlowTimeline duplicate key (#8754) 2026-04-07 20:13:30 +00:00
hugocasa
db55e8efb0 fix: remove span.enter() in dedicated worker to prevent tracing panic (#8749)
* [ee] fix: remove span.enter() in dedicated worker to prevent tracing panic

Update EE ref to include fix for dedicated worker tracing span panic that
caused benchmark failures after ~8000 jobs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 86158dde674238fd94f925bdcd5155759e823ed6

This commit updates the EE repository reference after PR #518 was merged in windmill-ee-private.

Previous ee-repo-ref: a0480130c241d32b7e02951bfb5a03fdfc5737c8

New ee-repo-ref: 86158dde674238fd94f925bdcd5155759e823ed6

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-04-07 17:59:46 +00:00
wendrul
5125263859 add tracings for long debounce key errors (#8747)
* Add messages about debug key

* update ee repo ref

* undo b

* update merged ee repo ref
2026-04-07 17:59:35 +00:00
hugocasa
d938625785 feat: add download all logs button for flow jobs (#8748)
* feat: add download all logs button for flow jobs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use recursive CTE to include all nested flow jobs in log download

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: start iteration index at 1 and interleave children with parents

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: distinguish branch vs loop iteration in log section headers

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: include flownode and singlestepflow kinds in branch/iteration labels

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: improve branch labels (branchone: default/1/2, branchall: 1/2)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: resolve module types from flow_node table for nested structures

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use full path in iteration/branch labels and show step kind name

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: show iteration index for simple module forloop optimized jobs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: handle aiagent jobs as intermediate flow jobs with tool call children

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: reuse existing get_logs_from_store/disk instead of duplicating

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* sqlx

* sqlx

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 17:59:16 +00:00
wendrul
eb32206940 cloud debounce keys potential 'value too long' error (#8750) 2026-04-07 17:57:48 +00:00
Ruben Fiszel
8b9523e03c fix: delete raw_script_temp rows before workspace deletion to avoid FK violation (#8752)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 17:34:18 +00:00
centdix
2f7ba9edac fix: restore ai agent tool deletion (#8744)
* fix: restore ai agent tool deletion

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: reduce ai tool delete tree walks

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-04-07 12:32:57 +00:00
hugocasa
208a597d59 feat: accept any content type on webhooks/http triggers with fallback (#8743)
* Revert "feat: restore bun for dedicated workers, fix dispatch & serialization, cross-workspace deps (#8645)"

This reverts commit 619ebb65ce.

* feat: accept any content type on webhooks/http triggers with fallback

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Reapply "feat: restore bun for dedicated workers, fix dispatch & serialization, cross-workspace deps (#8645)"

This reverts commit ee5420e401.

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 12:14:24 +00:00
Ruben Fiszel
c4be833bc0 chore(main): release 1.677.0 (#8737)
* chore(main): release 1.677.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
v1.677.0
2026-04-06 19:54:41 +00:00
Ruben Fiszel
b5c1eb3137 test: verify esbuild bundle output in app dev server for React and Svelte (#8741)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 18:29:07 +00:00
Ruben Fiszel
edfe074e98 fix: use runnable key for file naming in generate-metadata to prevent duplicate scripts in raw apps (#8740)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 18:08:07 +00:00
Ruben Fiszel
c09a4311fd fix: remove stale KMS openapi/description, restore stripped doc comments
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 15:19:03 +00:00
Ruben Fiszel
09bbc18bb7 feat: add AWS Secrets Manager as secret storage backend (Beta) (#8734)
* feat: add AWS KMS as secret backend (EE)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: switch from AWS KMS to AWS Secrets Manager as secret backend

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: add AWS Secrets Manager integration tests (requires LocalStack)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: mark AWS Secrets Manager as beta

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove leftover KMS handler functions from api-settings

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to include AWS Secrets Manager EE impl

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use full commit hash in ee-repo-ref.txt

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* sqlx

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 15:17:15 +00:00
Ruben Fiszel
a78eb6e93d test: add integration test for public_app_by_custom_path endpoint (#8735)
Regression test for the missing labels column bug. Creates an app with
a custom path and anonymous execution mode, then fetches it via the
public custom path endpoint.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 13:17:00 +00:00
Ruben Fiszel
af1f6506d2 chore(main): release 1.676.0 (#8730)
* chore(main): release 1.676.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
v1.676.0
2026-04-06 12:45:08 +00:00
Ruben Fiszel
d2abc0d430 fix: fix custom urls not found 2026-04-06 12:33:21 +00:00
Ruben Fiszel
e32662169a feat: add path name autocomplete with ghost text and folder cycling (#8731)
* feat: add path name autocomplete with ghost text and folder cycling

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: filter out archived/deleted/draft paths from autocomplete

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: show subfolders immediately after Tab-navigating into a folder

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: remove 2-char minimum for suggestions, hide placeholder when suggestions show

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: show LCP ghost text for multiple matches, Enter accepts it for Tab cycling

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: suppress Path.svelte Enter dispatch when ghost text is accepted

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: compute LCP inline in Enter handler to avoid reactive timing issues

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: Enter picks the first folder and navigates into it

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: Enter picks the currently Tab-highlighted folder, not always the first

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: remove stray blank lines in applyCycleOrComplete

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: review fixes — $bindable default, openapi cache description, non-null assertion

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: add sqlx query cache for path_autocomplete

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 03:44:21 +00:00
Ruben Fiszel
c721fac466 perf: add partial index for expired cache resource cleanup (#8728)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-04-05 18:33:03 +00:00
Ruben Fiszel
7eaaf3021a chore(main): release 1.675.1 (#8727)
* chore(main): release 1.675.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
v1.675.1
2026-04-05 17:01:17 +00:00
Ruben Fiszel
f703fba1ef fix: log cleanup scans S3 orphans and works cross-server (#8729)
* fix: log cleanup scans S3 orphans and works cross-server

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: don't skip service log orphan scan when job retention is disabled

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: time-based heartbeat + flag partial folder sizes on list errors

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: move background_task module from common to api-settings

Only log_cleanup and storage_usage use it today, both in windmill-api-settings.
Keeping it in the consumer crate narrows the blast radius; if workers or
indexer later need cross-server lease+progress coordination they can move it
back to common then.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-04-05 16:56:46 +00:00
Ruben Fiszel
eae46a21a9 perf: add indexes for cleanup deletes on concurrency_key and autoscaling_event (#8726)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 11:10:30 -04:00
Ruben Fiszel
31df861ee2 chore(main): release 1.675.0 (#8716)
* chore(main): release 1.675.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
v1.675.0
2026-04-05 14:18:34 +00:00
Ruben Fiszel
e605bc4e07 sqlx 2026-04-05 14:15:24 +00:00
Yoaquim Cintrón
7bf6ac2b69 fix: enrich OTEL spans with job_kind, trigger_kind, trigger, created_by, and script_hash (#8718)
Add five new attributes to the `job` and `job_postprocessing` tracing spans
so that OTEL-consuming backends (Sentry, Honeycomb, Datadog, etc.) can
filter and group telemetry by how a job was triggered and what type it is.

New span attributes:
- `job_kind`     — Script, Flow, AppScript, AIAgent, Preview, etc.
- `created_by`   — the user or system identity that queued the job
- `trigger_kind` — schedule, webhook, kafka, http, sqs, etc.
- `trigger`      — the schedule/trigger path (when applicable)
- `runnable_id`  — the id of the runnable that ran

Also adds `JobKind::as_str()` for a consistent lowercase string
representation, following the same pattern as `ScriptLang::as_str()`.

Existing attributes (job_id, workspace_id, script_path, language, tag,
flow_step_id, parent_job, root_job) are unchanged.

Note: the EE `full_job` span in `otel_ee.rs` and the log records emitted
by `job_logger_ee.rs` would also benefit from these attributes. This PR
covers only the public-repo spans; a follow-up EE change would propagate
the same fields to logs and the full_job span.
2026-04-05 14:11:22 +00:00
Ruben Fiszel
01e39d9cd1 fix: split DB health endpoint and add slow query controls (#8725)
Split the DB health page into independent panes so fast pg_catalog-based
diagnostics render without waiting for the slower job table scans, and
enrich the slow queries view with server-side sort, reset, and better
setup guidance.

Backend:
- Split /api/db_health into two endpoints: fast panes (database_size,
  connection_pool, table_maintenance, slow_queries, datatables) and
  /jobs (job_retention, large_results with scan_limit).
- Add GET /api/db_health/slow_queries?sort=total|mean|calls for
  server-side sorting of pg_stat_statements queries (sort whitelisted
  via enum, SQL-injection safe).
- Add POST /api/db_health/slow_queries/reset to call
  pg_stat_statements_reset().
- Return stats_reset timestamp from pg_stat_statements_info (PG 14+).
- Bump slow queries to top 50 sorted by total_exec_time (was top 10 by
  mean_exec_time, which misses high-cumulative-load queries).
- Truncate slow queries to 500 chars (was 200).
- Filter table_maintenance to tables with >= 1000 total tuples.

Frontend (DbHealth.svelte):
- Two tabs (Overview / Jobs) with auto-refresh on selection.
- Refresh buttons right-aligned in both tabs; Jobs tab keeps the
  scan_limit selector on the left.
- Job Retention & Large Results always render, with "Click Refresh to
  load" placeholders when no data yet.
- Slow queries table: clickable column headers for server-side sort,
  click a row to toggle the full query text.
- Reset stats button with confirmation dialog, displays "Stats since"
  timestamp for before/after comparison workflow.
- When pg_stat_statements is not installed, show numbered setup
  instructions with copyable SQL snippets.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 14:00:44 +00:00
Ruben Fiszel
02d0ee9198 feat: add object storage usage view and manual log cleanup (#8724) 2026-04-05 13:10:48 +00:00
Ruben Fiszel
dd39c110a8 fix: add admin check to count_completed_jobs_detail and document query builder SQL safety (#8722)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 12:06:14 +00:00
Ruben Fiszel
342defecd2 block adding/inviting members to admins workspace (#8721)
* fix: block adding/inviting members to admins workspace on CE

The admins workspace is reserved for superadmins only. On CE (non-enterprise),
prevent adding or inviting users to it via both API and UI.

Backend: add #[cfg(not(feature = "enterprise"))] guards to invite_user and
add_user endpoints that reject requests targeting the admins workspace.

Frontend: show an info alert on the admins workspace members page and hide
the add/invite/auto-add buttons.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use derived variable for admins workspace alert consistency

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 11:06:14 +00:00
Ruben Fiszel
2b865c0694 fix: allow private AI base URLs in ai_proxy integration test (#8715)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-04-03 21:26:24 -04:00
Ruben Fiszel
7c3bd67639 chore(main): release 1.674.2 (#8714)
* chore(main): release 1.674.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
v1.674.2
2026-04-04 01:00:33 +00:00
Ruben Fiszel
ff8e39c69b fix: enforce RLS on $var: resolution in AI proxy (GHSA-jwg4-v3cj-rvfm) (#8713)
* fix: enforce RLS on $var: resolution in AI proxy to prevent secret exfiltration

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update sqlx prepared queries

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 00:55:57 +00:00
Ruben Fiszel
f394e674f2 fix: SSRF via X-Resource-Path header in AI proxy endpoint (#8712)
* fix: validate AI provider base URLs to prevent SSRF via X-Resource-Path header

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: improve SSRF error message to mention ALLOW_PRIVATE_AI_BASE_URLS env var

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-04-04 00:43:08 +00:00
Ruben Fiszel
6bb5cac0c4 chore(main): release 1.674.1 (#8711)
* chore(main): release 1.674.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
v1.674.1
2026-04-04 00:05:05 +00:00
Ruben Fiszel
3e9a9f44bb nit windows tests 2026-04-04 00:04:49 +00:00