Commit Graph

2998 Commits

Author SHA1 Message Date
Diego Imbert
3affbb3321 feat: type-checked data tables v0 (#7381)
* data tables settings ui

* install runed

* zod 4 fixes

* use new toJSONSchema

* Migrate ducklake catalogs to more generic custom instance databases

* fix compilation

* Safety conversion for old duckdb ffi

* data tables settings

* ts client basis

* inline run works

* datatables work

* Revert "datatables work"

This reverts commit 6e1588d59e.

* datatables work (without leaking pg credentials)

* println

* separate sqlUtils.ts

* nit

* Separate custom instance db Select and Wizard components

* nit

* nit wording

* add tags to custom instance dbs

* error when trying to use ducklake as datatable or opposite

* show status in dropdown

* data table instance setup works

* sqk function for ducklake

* factorize logic

* fix temp reactivity

* Data table assetexplore

* Migrate S3 permissions to modal

* Revert "Migrate S3 permissions to modal"

This reverts commit 0631d03cb0.

* nit query -> fetch

* Custom instance setup new look

* run_language_executor separate fn

* run_inline param

* nit wording

* Better typed client

* Data tables display as assets in frontend

* asset db icon

* nit

* cleaner errors

* nit

* Fix sed calls in mac

* run_inline_script_preview in python client

* basic python datatable client

* datatable and datalake parser in python

* ducklake client python

* nit fix

* Fix migration producing NULL instead of {} when no custom databases

* merge conflict fail

* python ducklake client arg fix

* parse or infer sql types in ts client

* ts asset parser, detect datatable & ducklake R/W

* fix sql repl for other read ops than select

* export type SqlTemplateFunction

* rename list_custom_instance_pg_databases

* typecheck datatable and ducklake name in Typescript

* Fix typecheck datatable and ducklake in TS

* declare module overriding instead of extending

* infer_sql_type in python client

* SqlQuery object in python

* fix merge conflicts

* update const_format

* CI fix

* factor out to var_identifiers

* sqlx prepare

* unnecessary security (admin is required)

* clearer comment

* ee repo ref

* nit snake case

* claude step 1: detect var declarations

* move detect_sql_access_type to common mod

* claude step 2: detect when saved vars are queried

* Revert "claude step 2: detect when saved vars are queried"

This reverts commit 1e1f930568.

* Revert "claude step 1: detect var declarations"

This reverts commit f866f4819d.

* remove ducklake/datatable and default

* detect data table assigns in var_identifiers

* Python parser successfully infers R/W/RW from ducklake / datatable

* still register ducklake/datatable if not used as unknown R/W

* Go to settings button in Assets Dropdown on not found

* nit

* sqlx prepare fail

* manual fix, somehow sqlx prepare won't do it

* fix frontend ci

* ee repo ref

* ducklake_user doesnt exist in unit tests

* nit fix

* ui nit

* nit

* nit missing clone

* fork ducklakes and datatables

* fix surface hover bug

* stupid mistake

* better deeply reactive mutable derived

* Ducklake picker

* Editor bar data tables

* DuckDB supports datatables

* datatable in duckdb asset parser

* duckdb asset parser var_identifiers

* Revert "duckdb asset parser var_identifiers"

This reverts commit 88068b1a77.

* sqlx prepare

* Box pin in test_workflow_as_code to fix stack overflow

* stash

* sql asset parser parses most s3 literals

* nit

* Detect attach + handle returning RW

* detect assets used with dot notation

* detect implicit access with USE dl; syntax

* Add assets as unknown if var was never used

* Support default ducklake/datatable main in parser

* ignore asset parsing errors in frontend (avoid flow layout shift)

* super weird duplication (merge conflict ?)

* nits

* fix duckdb parser detecting too much as asset when RW ctx is unknown

* fix transparent assets btn

* missing arg

* nit styling

* asset parser specific table parsing

* fix resource specific table parsing

* More concise asset display in flows + better icons

* fix assets page filtering out resources with added table

* Fix frontend to support specific table assets

* Open DB Manager to specific table

* Specific table parser in Python and TS + unit tests

* Fix UPDATE setting access to None

* fix flow edge rendering on top of output picker

* python parser fix var override bug

* add ts test

* fix compilation

* sqlx prepare

* update parsers version

* fix missing schema key onDelete

* Grant permission to create schemas in custom instance databases

* Update pg query to return empty schemas

* Create schema

* Select nits

* support schemas in sql parser

* ts parser handle schema with sql parser result

* detect .schema() syntax

* detect schema syntax in python

* support .schema() in ts and py SDK

* open db manager to specific schema

* support reassignment in ts parser

* nit better unitest

* : syntax in ts

* datatable:schema syntax in python

* fix client py

* nit select dropdown darkmode

* object | null fetchOne

* ts client nits

* parse_sql_client_name fn

* getImportWmillTsStatement refactor in EditorBar

* text to json() in python client

* update parser versions

* pkg lock

* Sql query details in TS asset parser

* code transformation with type parameter in Editor

* Custom Language Worker, code substition works !

* Error marker mapping works

* hover info is correct

* completions work correctly

* other overrides

* type inference kinda works

* Position mapping tests

* refactor prepare_queries

* Refactor PgDatabase to share common code

* Pgdatabase in prepare_queries

* TokioPgConnection refactor

* refactor prepare_queries

* type parameter to sql function

* Fix deadlock

* nit fix

* Fix worker async call freezing because of svelte Proxy

* Force worker to recompute when we set queries

* nit refactor

* nits console logs

* wait that ts worker initialize

* monaco change file version

* update diagnostics

* Refactor for errors

* Show SQL errors in Monaco

* improve sdk

* cleaning refactor + MapResource + usePreparedAssetSqlQueries

* Fixes

* Fix error position mapping

* cache in typescript worker

* fix insert no values

* don't inject type if already present

* Support schema in prepare queries

* update parsers

* ChangeOnDeepInequality

* inferAsset ScriptEditor usage refactor

* sql query typecheck work in flow editor

* Assets and SQL Query check in Raw App Inline Editor

* pkg lock

* Fix DatatableSqlTemplateFunction nit

* prepare query schema nit

* duplicate diagnostics

* nit getScriptVersion mock

* Reprepare queries when switching workspaces

* nit fix

* nit fix

* fetch_one_scalar and execute in python client

* limit pg_connections

* -- prepare flag in postgres

* skip serializing

* fix destructuring undefined

* Prepare queries in workers instead of backend

* nit

* Execute search_path instructions normally

* nit fix

* Fix SET search_path issue in prepare

* only support preparing single-statement queries for now

* update parsers

* safety

* better remove_comments

* Fix getQueryStmtCountHeuristic

* getQueryStmtCountHeuristic tests

* comment out failing tests

* Fix getQueryStmtCountHeuristic impl

* only datatable
2025-12-20 08:51:07 +00:00
Ruben Fiszel
d86d233843 chore(main): release 1.595.0 (#7416)
* chore(main): release 1.595.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-19 21:10:51 +00:00
Alexander Petric
8268354889 fix: teams, need both guid and thread id format (#7420)
* fix: teams, need both guid and thread id format

* chore: update ee-repo-ref to 576abf6519d1aa12a2b989a58a123501206284fb

This commit updates the EE repository reference after PR #368 was merged in windmill-ee-private.

Previous ee-repo-ref: ae41589212f16ff0cec8516c66227f30e17c5564

New ee-repo-ref: 576abf6519d1aa12a2b989a58a123501206284fb

Automated by sync-ee-ref workflow.

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-19 20:07:23 +00:00
Ruben Fiszel
330c321fe9 chore(main): release 1.594.0 (#7413)
* chore(main): release 1.594.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-19 13:08:32 +00:00
Alexander Petric
96aaceef95 fix: improve teams search ux (#7407)
* feat: improve teams search ux

* ee ref

* claude review

* chore: update ee-repo-ref to e218dfce97dcea56c6ef6032592dab812a3f5047

This commit updates the EE repository reference after PR #363 was merged in windmill-ee-private.

Previous ee-repo-ref: 1b95a24ab25d96e59d2f22588901e9d3ce6c72b3

New ee-repo-ref: e218dfce97dcea56c6ef6032592dab812a3f5047

Automated by sync-ee-ref workflow.

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-19 13:03:15 +00:00
hugocasa
a6993823af feat: restart flow from step with different flow version (#7409)
* feat: restart flow from step with different flow version

* fix tests

* fix tests
2025-12-18 20:27:51 +00:00
Ruben Fiszel
2c6dad2f06 chore(main): release 1.593.1 (#7408)
* chore(main): release 1.593.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-18 16:30:13 +00:00
Ruben Fiszel
81b88fcd24 chore(main): release 1.593.0 (#7361)
* chore(main): release 1.593.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-17 13:52:45 +00:00
Diego Imbert
ca45937166 Refactor PgDatabase to share common code (#7386)
* Refactor PgDatabase to share common code

* TokioPgConnection refactor
2025-12-17 11:34:39 +00:00
centdix
8c55f61bba feat(ai): support IAM auth for bedrock provider (#7379)
* support iam for bedrock ai

* lock

* cleaning
2025-12-16 22:00:14 +00:00
Pyra
fe56191422 feat(internal): runnable settings (#7298)
* rework everything again

Signed-off-by: pyranota <pyra@duck.com>

* updcate sqlx

Signed-off-by: pyranota <pyra@duck.com>

* update ref

Signed-off-by: pyranota <pyra@duck.com>

* fix things

Signed-off-by: pyranota <pyra@duck.com>

* fix function

Signed-off-by: pyranota <pyra@duck.com>

* final fixes

Signed-off-by: pyranota <pyra@duck.com>

* update sqlx

Signed-off-by: pyranota <pyra@duck.com>

* fix script creation

Signed-off-by: pyranota <pyra@duck.com>

* address todo

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* remove dbg

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fix

Signed-off-by: pyranota <pyra@duck.com>

* fixups

Signed-off-by: pyranota <pyra@duck.com>

* fix cargo.toml

Signed-off-by: pyranota <pyra@duck.com>

* update ee repo

Signed-off-by: pyranota <pyra@duck.com>

* fix ci

Signed-off-by: pyranota <pyra@duck.com>

* nit

Signed-off-by: pyranota <pyra@duck.com>

* ref

Signed-off-by: pyranota <pyra@duck.com>

* fix

Signed-off-by: pyranota <pyra@duck.com>

* ee repo

Signed-off-by: pyranota <pyra@duck.com>

* sqlx

Signed-off-by: pyranota <pyra@duck.com>

* ee ref

Signed-off-by: pyranota <pyra@duck.com>

* remove dbg

Signed-off-by: pyranota <pyra@duck.com>

* sqlx

Signed-off-by: pyranota <pyra@duck.com>

* chore: update ee-repo-ref to 505eadbff32d102ea5245a2bef88ce6f1bb95395

This commit updates the EE repository reference after PR #348 was merged in windmill-ee-private.

Previous ee-repo-ref: 195243e56cc0eab55f8890fa57297206bfe2c18c

New ee-repo-ref: 505eadbff32d102ea5245a2bef88ce6f1bb95395

Automated by sync-ee-ref workflow.

* ci: force runnable settings

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-16 21:17:06 +00:00
Diego Imbert
75fdc2cdc9 feat: data table schemas (#7353)
* data tables settings ui

* install runed

* zod 4 fixes

* use new toJSONSchema

* Migrate ducklake catalogs to more generic custom instance databases

* fix compilation

* Safety conversion for old duckdb ffi

* data tables settings

* ts client basis

* inline run works

* datatables work

* Revert "datatables work"

This reverts commit 6e1588d59e.

* datatables work (without leaking pg credentials)

* println

* separate sqlUtils.ts

* nit

* Separate custom instance db Select and Wizard components

* nit

* nit wording

* add tags to custom instance dbs

* error when trying to use ducklake as datatable or opposite

* show status in dropdown

* data table instance setup works

* sqk function for ducklake

* factorize logic

* fix temp reactivity

* Data table assetexplore

* Migrate S3 permissions to modal

* Revert "Migrate S3 permissions to modal"

This reverts commit 0631d03cb0.

* nit query -> fetch

* Custom instance setup new look

* run_language_executor separate fn

* run_inline param

* nit wording

* Better typed client

* Data tables display as assets in frontend

* asset db icon

* nit

* cleaner errors

* nit

* Fix sed calls in mac

* run_inline_script_preview in python client

* basic python datatable client

* datatable and datalake parser in python

* ducklake client python

* nit fix

* Fix migration producing NULL instead of {} when no custom databases

* merge conflict fail

* python ducklake client arg fix

* parse or infer sql types in ts client

* ts asset parser, detect datatable & ducklake R/W

* fix sql repl for other read ops than select

* export type SqlTemplateFunction

* rename list_custom_instance_pg_databases

* typecheck datatable and ducklake name in Typescript

* Fix typecheck datatable and ducklake in TS

* declare module overriding instead of extending

* infer_sql_type in python client

* SqlQuery object in python

* fix merge conflicts

* update const_format

* CI fix

* factor out to var_identifiers

* sqlx prepare

* unnecessary security (admin is required)

* clearer comment

* ee repo ref

* nit snake case

* claude step 1: detect var declarations

* move detect_sql_access_type to common mod

* claude step 2: detect when saved vars are queried

* Revert "claude step 2: detect when saved vars are queried"

This reverts commit 1e1f930568.

* Revert "claude step 1: detect var declarations"

This reverts commit f866f4819d.

* remove ducklake/datatable and default

* detect data table assigns in var_identifiers

* Python parser successfully infers R/W/RW from ducklake / datatable

* still register ducklake/datatable if not used as unknown R/W

* Go to settings button in Assets Dropdown on not found

* nit

* sqlx prepare fail

* manual fix, somehow sqlx prepare won't do it

* fix frontend ci

* ee repo ref

* ducklake_user doesnt exist in unit tests

* nit fix

* ui nit

* nit

* nit missing clone

* fork ducklakes and datatables

* fix surface hover bug

* stupid mistake

* better deeply reactive mutable derived

* Ducklake picker

* Editor bar data tables

* DuckDB supports datatables

* datatable in duckdb asset parser

* duckdb asset parser var_identifiers

* Revert "duckdb asset parser var_identifiers"

This reverts commit 88068b1a77.

* sqlx prepare

* Box pin in test_workflow_as_code to fix stack overflow

* stash

* sql asset parser parses most s3 literals

* nit

* Detect attach + handle returning RW

* detect assets used with dot notation

* detect implicit access with USE dl; syntax

* Add assets as unknown if var was never used

* Support default ducklake/datatable main in parser

* ignore asset parsing errors in frontend (avoid flow layout shift)

* super weird duplication (merge conflict ?)

* nits

* fix duckdb parser detecting too much as asset when RW ctx is unknown

* fix transparent assets btn

* missing arg

* nit styling

* asset parser specific table parsing

* fix resource specific table parsing

* More concise asset display in flows + better icons

* fix assets page filtering out resources with added table

* Fix frontend to support specific table assets

* Open DB Manager to specific table

* Specific table parser in Python and TS + unit tests

* Fix UPDATE setting access to None

* fix flow edge rendering on top of output picker

* python parser fix var override bug

* add ts test

* fix compilation

* sqlx prepare

* update parsers version

* fix missing schema key onDelete

* Grant permission to create schemas in custom instance databases

* Update pg query to return empty schemas

* Create schema

* Select nits

* support schemas in sql parser

* ts parser handle schema with sql parser result

* detect .schema() syntax

* detect schema syntax in python

* support .schema() in ts and py SDK

* open db manager to specific schema

* support reassignment in ts parser

* nit better unitest

* : syntax in ts

* datatable:schema syntax in python

* fix client py

* nit select dropdown darkmode

* object | null fetchOne

* ts client nits

* parse_sql_client_name fn

* getImportWmillTsStatement refactor in EditorBar

* text to json() in python client

* update parser versions

* pkg lock
2025-12-16 21:16:42 +00:00
hugocasa
b4eb7c6ac0 feat: http triggers scopes (#7385) 2025-12-16 19:46:45 +00:00
hugocasa
3ba361ad1a fix(backend): better trigger listening logs (#7392)
* fix(backend): better trigger listening logs

* chore: update ee-repo-ref to d347295041426d03039b747a148a71e3583c3a6b

This commit updates the EE repository reference after PR #362 was merged in windmill-ee-private.

Previous ee-repo-ref: 37b533704e1b40e616ac144bebeff574a5d048e1

New ee-repo-ref: d347295041426d03039b747a148a71e3583c3a6b

Automated by sync-ee-ref workflow.

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-16 16:23:21 +00:00
centdix
1b86a39051 fix(mcp): fix unresovled schema (#7383) 2025-12-15 22:52:49 +00:00
Ruben Fiszel
d249d0f860 nit 2025-12-15 20:35:51 +00:00
hugocasa
ebc82dbe58 feat(backend): stop schedules and cancel jobs when archiving a workspace (#7377) 2025-12-15 18:03:28 +00:00
Devdatta Talele
b3603d8720 fix: add history directly viewable in folder/group viewer (#7365) 2025-12-15 18:01:07 +00:00
Alexander Petric
c2a9ce46be improve github app error handling (#7357)
* improve github app error handling

* npm check

* sqlx

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-12-12 22:53:21 +00:00
wendrul
9d06c152ee feat: workspace forks merge UI (#7333)
* feat: Add workspace diff viewer and deployment UI for forked workspaces

- Add backend endpoint for comparing two workspaces
- Implement comparison logic for scripts, flows, apps, resources, variables
- Create ForkWorkspaceBanner component to detect and display fork status
- Build WorkspaceComparisonDrawer for detailed diff viewing and deployment
- Add DiffViewer component for line-by-line comparisons
- Support bidirectional deployment (fork to parent or parent to fork)
- Add conflict detection for items that are both ahead and behind
- Include delete fork option when no changes remain

Note: Backend implementation requires sqlx prepare to be run for full functionality

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* Fix banner and use wworkspace_diff table

* satisfactory UI WIP

* UI complete

* Deploy button

* Comaprison and reset tally

* compare all types of items

* Show summaries

* Disable buttons during deployment

* Auto select all on entering page

* Change migration to have 'exists_in' cols

* Show new and deleted items

* frontend fixes

* Block delpoyment if changes don't match (new chagnes detected)

* Message to block whe changes are behind

* Skip workspaces pre-migration

* Remove unused code

* Fix apps comparison

* Only return changes where user has visibility

* No deploy button if no access to all changes

* Prepare sqlx

* Remove redundant message

* CI: update ee repo ref

* eereporef bis

* Small tweaks

* Remove unused struct

* Remove unused refactor component

* Fix npm run check

* Remove unused component

* chore: update ee-repo-ref to bbf406edc222199ca2e6076da12c376fb4ff28c5

This commit updates the EE repository reference after PR #359 was merged in windmill-ee-private.

Previous ee-repo-ref: 6aae845c5629ae32da43dbfbdc4566e5bf90fb1e

New ee-repo-ref: bbf406edc222199ca2e6076da12c376fb4ff28c5

Automated by sync-ee-ref workflow.

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-12-12 22:52:26 +00:00
Ruben Fiszel
734b6c0587 chore(main): release 1.592.1 (#7360)
* chore(main): release 1.592.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-12 18:20:06 +00:00
Ruben Fiszel
af1b85f7d0 chore(main): release 1.592.0 (#7356)
* chore(main): release 1.592.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-12 17:47:18 +00:00
Ruben Fiszel
e2953862af chore(main): release 1.591.4 (#7354)
* chore(main): release 1.591.4

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-12 12:27:45 +00:00
Ruben Fiszel
aeea39abd6 chore(main): release 1.591.3 (#7344)
* chore(main): release 1.591.3

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-12 00:36:10 +01:00
Diego Imbert
c6c7f3415a Specific asset tables (#7323)
* data tables settings ui

* install runed

* zod 4 fixes

* use new toJSONSchema

* Migrate ducklake catalogs to more generic custom instance databases

* fix compilation

* Safety conversion for old duckdb ffi

* data tables settings

* ts client basis

* inline run works

* datatables work

* Revert "datatables work"

This reverts commit 6e1588d59e.

* datatables work (without leaking pg credentials)

* println

* separate sqlUtils.ts

* nit

* Separate custom instance db Select and Wizard components

* nit

* nit wording

* add tags to custom instance dbs

* error when trying to use ducklake as datatable or opposite

* show status in dropdown

* data table instance setup works

* sqk function for ducklake

* factorize logic

* fix temp reactivity

* Data table assetexplore

* Migrate S3 permissions to modal

* Revert "Migrate S3 permissions to modal"

This reverts commit 0631d03cb0.

* nit query -> fetch

* Custom instance setup new look

* run_language_executor separate fn

* run_inline param

* nit wording

* Better typed client

* Data tables display as assets in frontend

* asset db icon

* nit

* cleaner errors

* nit

* Fix sed calls in mac

* run_inline_script_preview in python client

* basic python datatable client

* datatable and datalake parser in python

* ducklake client python

* nit fix

* Fix migration producing NULL instead of {} when no custom databases

* merge conflict fail

* python ducklake client arg fix

* parse or infer sql types in ts client

* ts asset parser, detect datatable & ducklake R/W

* fix sql repl for other read ops than select

* export type SqlTemplateFunction

* rename list_custom_instance_pg_databases

* typecheck datatable and ducklake name in Typescript

* Fix typecheck datatable and ducklake in TS

* declare module overriding instead of extending

* infer_sql_type in python client

* SqlQuery object in python

* fix merge conflicts

* update const_format

* CI fix

* factor out to var_identifiers

* sqlx prepare

* unnecessary security (admin is required)

* clearer comment

* ee repo ref

* nit snake case

* claude step 1: detect var declarations

* move detect_sql_access_type to common mod

* claude step 2: detect when saved vars are queried

* Revert "claude step 2: detect when saved vars are queried"

This reverts commit 1e1f930568.

* Revert "claude step 1: detect var declarations"

This reverts commit f866f4819d.

* remove ducklake/datatable and default

* detect data table assigns in var_identifiers

* Python parser successfully infers R/W/RW from ducklake / datatable

* still register ducklake/datatable if not used as unknown R/W

* Go to settings button in Assets Dropdown on not found

* nit

* sqlx prepare fail

* manual fix, somehow sqlx prepare won't do it

* fix frontend ci

* ee repo ref

* ducklake_user doesnt exist in unit tests

* nit fix

* ui nit

* nit

* nit missing clone

* fork ducklakes and datatables

* fix surface hover bug

* stupid mistake

* better deeply reactive mutable derived

* Ducklake picker

* Editor bar data tables

* DuckDB supports datatables

* datatable in duckdb asset parser

* duckdb asset parser var_identifiers

* Revert "duckdb asset parser var_identifiers"

This reverts commit 88068b1a77.

* sqlx prepare

* Box pin in test_workflow_as_code to fix stack overflow

* stash

* sql asset parser parses most s3 literals

* nit

* Detect attach + handle returning RW

* detect assets used with dot notation

* detect implicit access with USE dl; syntax

* Add assets as unknown if var was never used

* Support default ducklake/datatable main in parser

* ignore asset parsing errors in frontend (avoid flow layout shift)

* super weird duplication (merge conflict ?)

* nits

* fix duckdb parser detecting too much as asset when RW ctx is unknown

* fix transparent assets btn

* missing arg

* nit styling

* asset parser specific table parsing

* fix resource specific table parsing

* More concise asset display in flows + better icons

* fix assets page filtering out resources with added table

* Fix frontend to support specific table assets

* Open DB Manager to specific table

* Specific table parser in Python and TS + unit tests

* Fix UPDATE setting access to None

* fix flow edge rendering on top of output picker

* python parser fix var override bug

* add ts test

* fix compilation

* sqlx prepare

* update parsers version

* fix missing schema key onDelete
2025-12-11 13:34:17 +00:00
Ruben Fiszel
355ef401a4 chore(main): release 1.591.2 (#7337)
* chore(main): release 1.591.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-11 07:44:02 +00:00
Ruben Fiszel
babfbe735c chore(main): release 1.591.1 (#7335)
* chore(main): release 1.591.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-10 22:01:59 +00:00
Ruben Fiszel
bf614df262 chore(main): release 1.591.0 (#7327)
* chore(main): release 1.591.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-10 21:27:34 +00:00
centdix
bd3271f967 feat(mcp): add documentation endpoint (#7331)
* simplify inkeep endpoint + add to openapi

* adapt frontend

* add to mcp endpoints

* nit

* udpate ref
2025-12-10 17:28:56 +00:00
Ruben Fiszel
651681b7ef fix(backend): add presigned url support for object storage (#7328)
* presigned

* all

* all

* all

* all

* all

* all

* all

* nit

* nit

* ee-ref

* presigned

* presigned
2025-12-10 15:35:26 +00:00
Alexander Petric
8159b8e017 fix: alining group name normalization between scim and igroup api (#7325)
* fix: aling group name normalization between scim and igroup api

* ee-ref
2025-12-09 21:23:13 +00:00
Ruben Fiszel
fb387dbd98 add accel buffering no 2025-12-08 22:55:47 +00:00
Ruben Fiszel
0bf0fbdbdc chore(main): release 1.590.0 (#7309)
* chore(main): release 1.590.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-08 18:16:22 +00:00
hugocasa
e6adf16fc9 feat: add query arg for oidc expiration (#7312)
* feat: add query arg for oidc expiration

* Update ee-repo-ref.txt

* Update ee-repo-ref.txt

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-08 18:00:51 +00:00
Tristan TR
3699ce7a8f feat: new live onboarding for flows (#7194)
* Start workspace onboarding

* Add pictures to tutorial steps

* Remove unecessary step

* Continue tutorial by creating a flow together

* Add image into the Create Flow tutorial pop up

* Generate flow from frontend

* Set pause between each node

* Add automatic scripts overview

* Simplify tutorial, and add step to show the code

* Add input step

* Autoremove last step after 5 seconds

* Add flow typing when opening code editor

* Remove lock field from json file

* Add Guides tab on left menu

* Add /guides page

* Add tutorial card in Guides tab

* Add step to show data connector

* Add second text input to show 2 types of inputs and fill them dynamically

* Improve tutorial chronology

* Add flow input connexion with first sctript

* Improve overlay

* Improve wording

* Add new tutorial step to show node b

* Add test step

* Add cursor to pick typescript

* Improve end of tutorial

* Refactor

* Highlight bottom right corner for 5 and 6

* Fix last step overlay

* change home tutorial button

* guidelines nits

* Automate onNext() trigger on step 3

* Improve fakr cursor for Test this step button

* Improve overlay transitions

* Merge data connectors and test step steps

* Improve live code writing in step 3

* Add a step to complete the flow

* Improve the step where we generate remaining scripts

* Refactor

* Add blocking behavior on step 3

* nit about delay

* Prevent clicking on Next while code not generated

* Sharpen wordings

* Remove Svelte 4 and migrate to Svelte 5

* Remove unecesary helper function

* Add toast if the user clicks on Next button before code finished generating

* Add toasts to each step

* Improve tutorial trigger timing

* Improve delays

* Add cursor movement to Test Flow button

* Block previous on certain steps to prevent bug

* Fix for github npm check

* Fix for github npm check

* Unlike workspace onboarding and flow tutorial

* Rename flow tutorial with better name

* Remove the automatic trigger for flow previous and broken tutorial

* Push tutorials to Help sectionof the sidebar

* Fix redirection t /tutorials page

* Add tutorials page and update workspace onboarding flow

- Rename guides to tutorials page (/tutorials)
- Add workspace onboarding tutorial to tutorials page
- Remove Tutorial button from homepage
- Add welcome cards for empty workspace with 3 tutorial options
- Update workspace onboarding to redirect to homepage before starting
- Clean up URL parameter after tutorial completion
- Move Tutorials to Help menu in sidebar
- Remove automatic "action" tutorial trigger for new flows
- Add flow-live-tutorial (renamed from workspace-onboarding-continue)
- Add Previous button blocking with toast notifications in flow tutorial

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add tutorials to workspace homepage

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Start tutorials for Run/logs section

* Fix data connector

* Add flow execution graph from Run drawer

* Add tabs highlighting in drawer

* Improve tutorial on run drawer

* Add mouse cursor moving from graph tab

* Add cursor click on script in Drawer Graph tabs

* Add troubleshooting flow in tutorial

* Add step to show logs of failed step

* add step 7 to invite the user to fix by himself and se the new results

* Improve wording

* Nit improvements

* Nits

* Refactor

* Refactor

* Rename the tutorial

* Remove deleted file

* Improve wording

* Improve first step of troubleshooting flow tutorial

* Add tutorials to /tutorials page and create component

* Remove previous Flow tutorials

* Fixes, and improve tutorial button design

* Improve status in Tutorial button

* Align tutorial button to brand guidelines

* Add skip all to onboarding workspace tutorial

* Add skipped_all to tutorial_progress

* Connect backend and frontend for tutorial progress

* Add store and helper to display or not Tutorials from left menu

* Add reminder at the end of each tutorial

* Add tutorial banner

* Remove tutorials from elpty workspace

* Improve Tutorials page

* Align banner to guidelines

* Add reset tutorials buttons

* Refactor

* Refactor to make it easy to add new tutorials and tabs

* Improve tutorial config to make it easy to add new tutorials

* Refactor and remove hardcoded indexes

* Add getTutorialIndex in tutorial config file

* Nit

* Add Mark all as complete button in tutorial page

* Add skip tutorial button in banner toast

* Replace if else in tutorials router by map to make it easier to maintain and scale

* Delete broken simple app tutorial

* Add Guide flow guide buttons inside the Create Flow page

* Add flow editor tutorials into flow builder page

* Update existing app tutorials with new tutorial system

* Create a dedicated tutorial category for app editor

* Add global progress bar

* Add Reset & Skip at tutorial category level

* Add progress to tab title

* Nits on design

* Make progress bar a props and design nits

* Add active props for Tutorial Category

* Display tutorials according to the user role

* Adapt progress bar to the user role

* Add roles array for each tutorial

* Add Tutorials tab in Operator menu

* Edge case if no Category and no Tutorial available for my role

* Allow the user to reset a single tutorial

* Allow a user to mark as completed a single tutorial

* Nit on hoovering tutorial status

* Allow admins to see which tutorials are available per role

* Create utils that allow admins to see which tutorials can access other roles of their organization

* Refactor resetSingleTutorial and completeSingleTutorial into one function

* Improve role system

* Remove hardcoded MAX_TUTORIAL_ID

* Fix type assertion

* Remove console log

* Reduce recalculations when unrelated state changes

* Add console.error

* Remove unused function

* Add tutorial wrapper and better router

* Nits to pass npm checks

* Fix typescripts and lint errors

* Add SQLx query cache for tutorial_progress queries

* Improve wording for workspace tutorial

---------

Co-authored-by: Diego Imbert <diego@windmill.dev>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-12-08 16:42:35 +00:00
Diego Imbert
df884e6135 Custom instance db password safety (#7313)
* update parsers version

* safety custom instance db migration

* Refresh custom instance pwd button

* nit indent

* nit button accent

* Prevent leave without saving
2025-12-08 16:35:39 +00:00
hugocasa
eb284dfabd feat: triggers suspended mode (#7297)
* first commit

* base

* ok

* remove print

* rm packe json

* fix discard

* fix

* update .sqlx

* update

* update ref

* jobs update

* nits

* update ref

* big

* update sqlx

* fix

* update

* nits and fix

* nits

* ok

* sqlx + fix import

* better

* nit

* suspended mode draft

* UI and http triggers

* nits

* done for http triggers

* feat: better retry/error handler check

* generalize to all triggers

* fix merge conflicts

* fix listener

* nits

* nit

* fix merge conflict

* update ee ref

---------

Co-authored-by: dieriba <dieriba.pro@gmail.com>
2025-12-08 16:20:49 +00:00
Ruben Fiszel
0e63dd301f fix: raw app v0.5 (#7310) 2025-12-06 16:06:30 +00:00
Diego Imbert
9bbab3321e feat: Data tables (#7226)
* data tables settings ui

* install runed

* zod 4 fixes

* use new toJSONSchema

* Migrate ducklake catalogs to more generic custom instance databases

* fix compilation

* Safety conversion for old duckdb ffi

* data tables settings

* ts client basis

* inline run works

* datatables work

* Revert "datatables work"

This reverts commit 6e1588d59e.

* datatables work (without leaking pg credentials)

* println

* separate sqlUtils.ts

* nit

* Separate custom instance db Select and Wizard components

* nit

* nit wording

* add tags to custom instance dbs

* error when trying to use ducklake as datatable or opposite

* show status in dropdown

* data table instance setup works

* sqk function for ducklake

* factorize logic

* fix temp reactivity

* Data table assetexplore

* Migrate S3 permissions to modal

* Revert "Migrate S3 permissions to modal"

This reverts commit 0631d03cb0.

* nit query -> fetch

* Custom instance setup new look

* run_language_executor separate fn

* run_inline param

* nit wording

* Better typed client

* Data tables display as assets in frontend

* asset db icon

* nit

* cleaner errors

* nit

* Fix sed calls in mac

* run_inline_script_preview in python client

* basic python datatable client

* datatable and datalake parser in python

* ducklake client python

* nit fix

* Fix migration producing NULL instead of {} when no custom databases

* merge conflict fail

* python ducklake client arg fix

* parse or infer sql types in ts client

* ts asset parser, detect datatable & ducklake R/W

* fix sql repl for other read ops than select

* export type SqlTemplateFunction

* rename list_custom_instance_pg_databases

* typecheck datatable and ducklake name in Typescript

* Fix typecheck datatable and ducklake in TS

* declare module overriding instead of extending

* infer_sql_type in python client

* SqlQuery object in python

* fix merge conflicts

* update const_format

* CI fix

* factor out to var_identifiers

* sqlx prepare

* unnecessary security (admin is required)

* clearer comment

* ee repo ref

* nit snake case

* claude step 1: detect var declarations

* move detect_sql_access_type to common mod

* claude step 2: detect when saved vars are queried

* Revert "claude step 2: detect when saved vars are queried"

This reverts commit 1e1f930568.

* Revert "claude step 1: detect var declarations"

This reverts commit f866f4819d.

* remove ducklake/datatable and default

* detect data table assigns in var_identifiers

* Python parser successfully infers R/W/RW from ducklake / datatable

* still register ducklake/datatable if not used as unknown R/W

* Go to settings button in Assets Dropdown on not found

* nit

* sqlx prepare fail

* manual fix, somehow sqlx prepare won't do it

* fix frontend ci

* ee repo ref

* ducklake_user doesnt exist in unit tests

* nit fix

* ui nit

* nit

* nit missing clone

* fork ducklakes and datatables

* fix surface hover bug

* stupid mistake

* better deeply reactive mutable derived

* Ducklake picker

* Editor bar data tables

* DuckDB supports datatables

* datatable in duckdb asset parser

* duckdb asset parser var_identifiers

* Revert "duckdb asset parser var_identifiers"

This reverts commit 88068b1a77.

* sqlx prepare

* Box pin in test_workflow_as_code to fix stack overflow

* go to settings button

* ee repo ref

* fix compilation

* wording nit
2025-12-05 23:08:58 +00:00
Diego Imbert
7b1a8b0dce fix: improve S3 etag caching (#7301) 2025-12-05 18:01:27 +00:00
Ruben Fiszel
cb44d94bc6 improve folder error 2025-12-05 17:12:31 +00:00
Ruben Fiszel
3ccdd51b9a chore(main): release 1.589.3 (#7305)
* chore(main): release 1.589.3

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-05 13:29:56 +00:00
Ruben Fiszel
d6d093d342 chore(main): release 1.589.2 (#7295)
* chore(main): release 1.589.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-05 07:11:46 +00:00
Pyra
1dfb0f0078 refactor: move concurrency and debouncing settings to their own structs (#7286)
* move concurrency and debouncing settings to it's own structs

Signed-off-by: pyranota <pyra@duck.com>

* cleanup queue::jobs::pull

Signed-off-by: pyranota <pyra@duck.com>

* done

Signed-off-by: pyranota <pyra@duck.com>

* migrated the rest

Signed-off-by: pyranota <pyra@duck.com>

* implement todo!

* impmenet todo! v2

* more cleanup

* fix all

Signed-off-by: pyranota <pyra@duck.com>

* remove unrelated code

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* do not serialize rawcode

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fixes

Signed-off-by: pyranota <pyra@duck.com>

* update ee repo ref

Signed-off-by: pyranota <pyra@duck.com>

* update ee repo

Signed-off-by: pyranota <pyra@duck.com>

* fix tests

Signed-off-by: pyranota <pyra@duck.com>

* Update ee-repo-ref.txt

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-04 10:11:54 +00:00
claude[bot]
e9f13065bf fix: show related job when deployment is in progress (#7294)
* feat: show related job when deployment is in progress

- Added job_id column to deployment_metadata table to track current deployment jobs
- Updated backend to store job_id when creating dependency jobs for scripts and flows
- Modified deployment status API endpoints to include job_id in responses
- Updated frontend to display clickable job link in "Deployment in progress" badge
- Added OpenAPI schema updates for new job_id field

Resolves #7293

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* update

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-12-04 09:50:07 +00:00
Ruben Fiszel
0c74253319 chore(main): release 1.589.1 (#7287)
* chore(main): release 1.589.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-03 19:22:40 +00:00
Ruben Fiszel
752e43e653 fix(backend): fix broken cancel selection/all 2025-12-03 18:05:31 +00:00
Ruben Fiszel
d4c1be5b8b chore(main): release 1.589.0 (#7277)
* chore(main): release 1.589.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-03 17:51:21 +00:00
Pyra
e15af78d93 feat: agent workers min version support (#7284)
* feat: agent workers min version support

Signed-off-by: pyranota <pyra@duck.com>

* update oapi

Signed-off-by: pyranota <pyra@duck.com>

* fix what claude wrote

Signed-off-by: pyranota <pyra@duck.com>

* final cleanup

Signed-off-by: pyranota <pyra@duck.com>

* change a bit

Signed-off-by: pyranota <pyra@duck.com>

* remove unsed import

Signed-off-by: pyranota <pyra@duck.com>

* run update_min_version periodically

Signed-off-by: pyranota <pyra@duck.com>

* Update ee-repo-ref.txt

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-03 13:48:07 +00:00
Ruben Fiszel
60d5ea8579 fix: get flow by id doesn't require path (#7278) 2025-12-02 19:45:30 +00:00