Compare commits

..

74 Commits

Author SHA1 Message Date
Ruben Fiszel
f830a9d0fb chore(main): release 1.596.0 (#7422)
* chore(main): release 1.596.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-20 10:03:58 +00:00
Ruben Fiszel
81e648055b reduce dockerfile slim deps 2025-12-20 09:34:45 +00:00
Diego Imbert
afe74f74fa fix: improve MS SQL Numeric rounding (#7404)
* Fix MS SQL Numeric rounding

* Fix result collection to avoid JSON parsing

* don't reparse json through sqlx, pass raw string to postgres

* Revert "don't reparse json through sqlx, pass raw string to postgres"

This reverts commit 355691fa7d.

* mistake

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-12-20 09:18:46 +00:00
Ruben Fiszel
d544da342c fix: update to astral-tokio-tar for CVE (#7423)
* fix: Migrate to astral-tokio-tar to address CVE-2025-62518

Migrates from the abandoned tokio-tar library to the actively maintained astral-tokio-tar to address CVE-2025-62518 (TARmageddon), a high-severity RCE vulnerability (CVSS 8.1). The vulnerability involves a boundary-parsing bug in PAX/ustar header handling that enables file smuggling attacks via malicious TAR archives.

This is a drop-in replacement requiring only dependency updates in Cargo.toml files. The astral-tokio-tar library uses the same tokio_tar module name, so no source code changes are needed. All references to the vulnerable tokio-tar v0.3.1 have been removed from Cargo.lock and replaced with the patched astral-tokio-tar v0.5.6.

Related to previous PR #6943 which was closed due to CLA issues. Security disclosure available at https://edera.dev/stories/tarmageddon and patch details at https://github.com/astral-sh/tokio-tar/releases/tag/v0.5.6

* update

---------

Co-authored-by: Devdatta Talele <devtalele0@gmail.com>
Co-authored-by: Devdatta Talele <50290838+devdattatalele@users.noreply.github.com>
2025-12-20 09:04:36 +00:00
Diego Imbert
3affbb3321 feat: type-checked data tables v0 (#7381)
* data tables settings ui

* install runed

* zod 4 fixes

* use new toJSONSchema

* Migrate ducklake catalogs to more generic custom instance databases

* fix compilation

* Safety conversion for old duckdb ffi

* data tables settings

* ts client basis

* inline run works

* datatables work

* Revert "datatables work"

This reverts commit 6e1588d59e.

* datatables work (without leaking pg credentials)

* println

* separate sqlUtils.ts

* nit

* Separate custom instance db Select and Wizard components

* nit

* nit wording

* add tags to custom instance dbs

* error when trying to use ducklake as datatable or opposite

* show status in dropdown

* data table instance setup works

* sqk function for ducklake

* factorize logic

* fix temp reactivity

* Data table assetexplore

* Migrate S3 permissions to modal

* Revert "Migrate S3 permissions to modal"

This reverts commit 0631d03cb0.

* nit query -> fetch

* Custom instance setup new look

* run_language_executor separate fn

* run_inline param

* nit wording

* Better typed client

* Data tables display as assets in frontend

* asset db icon

* nit

* cleaner errors

* nit

* Fix sed calls in mac

* run_inline_script_preview in python client

* basic python datatable client

* datatable and datalake parser in python

* ducklake client python

* nit fix

* Fix migration producing NULL instead of {} when no custom databases

* merge conflict fail

* python ducklake client arg fix

* parse or infer sql types in ts client

* ts asset parser, detect datatable & ducklake R/W

* fix sql repl for other read ops than select

* export type SqlTemplateFunction

* rename list_custom_instance_pg_databases

* typecheck datatable and ducklake name in Typescript

* Fix typecheck datatable and ducklake in TS

* declare module overriding instead of extending

* infer_sql_type in python client

* SqlQuery object in python

* fix merge conflicts

* update const_format

* CI fix

* factor out to var_identifiers

* sqlx prepare

* unnecessary security (admin is required)

* clearer comment

* ee repo ref

* nit snake case

* claude step 1: detect var declarations

* move detect_sql_access_type to common mod

* claude step 2: detect when saved vars are queried

* Revert "claude step 2: detect when saved vars are queried"

This reverts commit 1e1f930568.

* Revert "claude step 1: detect var declarations"

This reverts commit f866f4819d.

* remove ducklake/datatable and default

* detect data table assigns in var_identifiers

* Python parser successfully infers R/W/RW from ducklake / datatable

* still register ducklake/datatable if not used as unknown R/W

* Go to settings button in Assets Dropdown on not found

* nit

* sqlx prepare fail

* manual fix, somehow sqlx prepare won't do it

* fix frontend ci

* ee repo ref

* ducklake_user doesnt exist in unit tests

* nit fix

* ui nit

* nit

* nit missing clone

* fork ducklakes and datatables

* fix surface hover bug

* stupid mistake

* better deeply reactive mutable derived

* Ducklake picker

* Editor bar data tables

* DuckDB supports datatables

* datatable in duckdb asset parser

* duckdb asset parser var_identifiers

* Revert "duckdb asset parser var_identifiers"

This reverts commit 88068b1a77.

* sqlx prepare

* Box pin in test_workflow_as_code to fix stack overflow

* stash

* sql asset parser parses most s3 literals

* nit

* Detect attach + handle returning RW

* detect assets used with dot notation

* detect implicit access with USE dl; syntax

* Add assets as unknown if var was never used

* Support default ducklake/datatable main in parser

* ignore asset parsing errors in frontend (avoid flow layout shift)

* super weird duplication (merge conflict ?)

* nits

* fix duckdb parser detecting too much as asset when RW ctx is unknown

* fix transparent assets btn

* missing arg

* nit styling

* asset parser specific table parsing

* fix resource specific table parsing

* More concise asset display in flows + better icons

* fix assets page filtering out resources with added table

* Fix frontend to support specific table assets

* Open DB Manager to specific table

* Specific table parser in Python and TS + unit tests

* Fix UPDATE setting access to None

* fix flow edge rendering on top of output picker

* python parser fix var override bug

* add ts test

* fix compilation

* sqlx prepare

* update parsers version

* fix missing schema key onDelete

* Grant permission to create schemas in custom instance databases

* Update pg query to return empty schemas

* Create schema

* Select nits

* support schemas in sql parser

* ts parser handle schema with sql parser result

* detect .schema() syntax

* detect schema syntax in python

* support .schema() in ts and py SDK

* open db manager to specific schema

* support reassignment in ts parser

* nit better unitest

* : syntax in ts

* datatable:schema syntax in python

* fix client py

* nit select dropdown darkmode

* object | null fetchOne

* ts client nits

* parse_sql_client_name fn

* getImportWmillTsStatement refactor in EditorBar

* text to json() in python client

* update parser versions

* pkg lock

* Sql query details in TS asset parser

* code transformation with type parameter in Editor

* Custom Language Worker, code substition works !

* Error marker mapping works

* hover info is correct

* completions work correctly

* other overrides

* type inference kinda works

* Position mapping tests

* refactor prepare_queries

* Refactor PgDatabase to share common code

* Pgdatabase in prepare_queries

* TokioPgConnection refactor

* refactor prepare_queries

* type parameter to sql function

* Fix deadlock

* nit fix

* Fix worker async call freezing because of svelte Proxy

* Force worker to recompute when we set queries

* nit refactor

* nits console logs

* wait that ts worker initialize

* monaco change file version

* update diagnostics

* Refactor for errors

* Show SQL errors in Monaco

* improve sdk

* cleaning refactor + MapResource + usePreparedAssetSqlQueries

* Fixes

* Fix error position mapping

* cache in typescript worker

* fix insert no values

* don't inject type if already present

* Support schema in prepare queries

* update parsers

* ChangeOnDeepInequality

* inferAsset ScriptEditor usage refactor

* sql query typecheck work in flow editor

* Assets and SQL Query check in Raw App Inline Editor

* pkg lock

* Fix DatatableSqlTemplateFunction nit

* prepare query schema nit

* duplicate diagnostics

* nit getScriptVersion mock

* Reprepare queries when switching workspaces

* nit fix

* nit fix

* fetch_one_scalar and execute in python client

* limit pg_connections

* -- prepare flag in postgres

* skip serializing

* fix destructuring undefined

* Prepare queries in workers instead of backend

* nit

* Execute search_path instructions normally

* nit fix

* Fix SET search_path issue in prepare

* only support preparing single-statement queries for now

* update parsers

* safety

* better remove_comments

* Fix getQueryStmtCountHeuristic

* getQueryStmtCountHeuristic tests

* comment out failing tests

* Fix getQueryStmtCountHeuristic impl

* only datatable
2025-12-20 08:51:07 +00:00
Alexander Petric
cdd5d9fa9a fix: improve error msg for unshare error (#7421) 2025-12-20 01:14:06 +02:00
Ruben Fiszel
d86d233843 chore(main): release 1.595.0 (#7416)
* chore(main): release 1.595.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-19 21:10:51 +00:00
Ruben Fiszel
2e4d6ad7a1 update python parser 2025-12-19 21:07:46 +00:00
Avigail Royzenberg
da500fcf3e fix: Python Enum types generate proper dropdown schemas with descriptions (#7400)
* Fix Python Enum and Literal schema generation with docstring descriptions

- Extract Enum class definitions and their string values
- Parse docstring Args: sections for parameter descriptions
- Map Enum type annotations to string enums with proper values
- Handle Enum.VALUE default values correctly
- Store descriptions in Arg.otyp field
- Add test case for enum with docstring parsing

* perf: optimize enum parser and fix default value handling

- Combine enum extraction and docstring parsing into single AST pass (2x performance improvement)
- Add support for IntEnum, StrEnum, Flag, IntFlag types
- Fix default values to use actual enum values (e.g., 'red') instead of member names (e.g., 'RED')
- Improve docstring parsing robustness with proper indentation tracking
- Clean up code structure with EnumInfo type for better maintainability

All tests pass. This addresses code review feedback for performance and correctness.

* perf: implement true lazy evaluation for enum parsing

- Only parse metadata when unknown types encountered
- Two-pass approach: parse types first, extract only if needed
- Zero overhead for scripts without enums
- Keyword checks + prepass filtering when extraction needed
2025-12-19 20:41:34 +00:00
Guilhem
210b8285d4 fix(frontend): settings redesign (#7406)
* improve collapsible link

* do not show superadmin ws link when already in it

* improve OAuth UI

* sso/oauth instance settings ui

* refactor instance settings alerts WIP

* Indexer and Oauth to brand guidelines

* refactor ws error handler page

* Create a tab SMTP in the Instance Settings

* Ractivity isssue fix for tabs

* nit

* Add smtp settings status in Error handler

* Add smtp configuration status

* Display teams connection status for instance alerts

* nit

* Add critical alerts description

* nit

* nit

* improve ee display

* nit

* nit

* fix typo

* nit

* restore vit config

---------

Co-authored-by: Alexander Petric <alex@windmill.dev>
2025-12-19 20:33:03 +00:00
Alexander Petric
8268354889 fix: teams, need both guid and thread id format (#7420)
* fix: teams, need both guid and thread id format

* chore: update ee-repo-ref to 576abf6519d1aa12a2b989a58a123501206284fb

This commit updates the EE repository reference after PR #368 was merged in windmill-ee-private.

Previous ee-repo-ref: ae41589212f16ff0cec8516c66227f30e17c5564

New ee-repo-ref: 576abf6519d1aa12a2b989a58a123501206284fb

Automated by sync-ee-ref workflow.

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-19 20:07:23 +00:00
centdix
3e2565f710 fix flow not sent (#7417) 2025-12-19 20:01:41 +00:00
hugocasa
f89fb292da fix(backend): put for loop itered in a separate table (#7419)
* fix(backend): put for loop itered in a separate table

* Update SQLx metadata

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-19 19:46:46 +00:00
Alexander Petric
c28e77110e fix: do not use unshare for init scripts (#7418) 2025-12-19 19:36:51 +00:00
hugocasa
0bf7407419 feat: email triggers custom cert (#7415)
* feat: email triggers custom cert

* chore: update ee-repo-ref to 6edb5e9ea22b88f81dc9ee2741ea679d9b22f61c

This commit updates the EE repository reference after PR #364 was merged in windmill-ee-private.

Previous ee-repo-ref: 4053446d2dff0310ecb89cf6e37f00b49217443a

New ee-repo-ref: 6edb5e9ea22b88f81dc9ee2741ea679d9b22f61c

Automated by sync-ee-ref workflow.

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-19 14:23:09 +00:00
Ruben Fiszel
330c321fe9 chore(main): release 1.594.0 (#7413)
* chore(main): release 1.594.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-19 13:08:32 +00:00
centdix
4f04d4a18c use opus (#7398) 2025-12-19 13:07:00 +00:00
Alexander Petric
96aaceef95 fix: improve teams search ux (#7407)
* feat: improve teams search ux

* ee ref

* claude review

* chore: update ee-repo-ref to e218dfce97dcea56c6ef6032592dab812a3f5047

This commit updates the EE repository reference after PR #363 was merged in windmill-ee-private.

Previous ee-repo-ref: 1b95a24ab25d96e59d2f22588901e9d3ce6c72b3

New ee-repo-ref: e218dfce97dcea56c6ef6032592dab812a3f5047

Automated by sync-ee-ref workflow.

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-19 13:03:15 +00:00
Ruben Fiszel
5f182bc28a nit warns 2025-12-19 12:57:06 +00:00
Ruben Fiszel
ff0d1d444e cgroups only on linux 2025-12-19 12:48:15 +00:00
Ruben Fiszel
806024403e fix: disable oomgroup by default 2025-12-19 12:45:21 +00:00
hugocasa
a6993823af feat: restart flow from step with different flow version (#7409)
* feat: restart flow from step with different flow version

* fix tests

* fix tests
2025-12-18 20:27:51 +00:00
hugocasa
0fe7a2a17e fix(backend): correctly apply preprocessor step tag (#7412) 2025-12-18 20:21:59 +00:00
Ruben Fiszel
2c6dad2f06 chore(main): release 1.593.1 (#7408)
* chore(main): release 1.593.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-18 16:30:13 +00:00
Ruben Fiszel
c9a19f12d6 fix: fix folder/group history seq id grant issues 2025-12-18 16:17:33 +00:00
Ruben Fiszel
81b88fcd24 chore(main): release 1.593.0 (#7361)
* chore(main): release 1.593.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-17 13:52:45 +00:00
centdix
97f15796e8 remove null format (#7402) 2025-12-17 12:46:27 +00:00
Ruben Fiszel
9385dba5c4 slim image 2025-12-17 12:33:44 +00:00
Ruben Fiszel
67e96cca9d slim nits 2025-12-17 12:09:27 +00:00
Diego Imbert
ca45937166 Refactor PgDatabase to share common code (#7386)
* Refactor PgDatabase to share common code

* TokioPgConnection refactor
2025-12-17 11:34:39 +00:00
Diego Imbert
bbf97765f1 sql.query(SELECT $1, $2, arg1, arg2) syntax in typescript (#7373) 2025-12-17 11:34:20 +00:00
Frédéric Crozatier
53c325a652 add inspector (#7370) 2025-12-17 08:40:36 +00:00
Ruben Fiszel
296c11c75a rework slim images 2025-12-17 08:37:12 +00:00
Jakub Drobník
3d34634a66 update apify logo (#7401) 2025-12-17 08:03:30 +00:00
Ruben Fiszel
5d55e8d517 use sccache for duckdb build 2025-12-16 22:55:02 +00:00
Ruben Fiszel
0a25416a96 update lsp base images node version 2025-12-16 22:31:50 +00:00
centdix
8c55f61bba feat(ai): support IAM auth for bedrock provider (#7379)
* support iam for bedrock ai

* lock

* cleaning
2025-12-16 22:00:14 +00:00
Pyra
fe56191422 feat(internal): runnable settings (#7298)
* rework everything again

Signed-off-by: pyranota <pyra@duck.com>

* updcate sqlx

Signed-off-by: pyranota <pyra@duck.com>

* update ref

Signed-off-by: pyranota <pyra@duck.com>

* fix things

Signed-off-by: pyranota <pyra@duck.com>

* fix function

Signed-off-by: pyranota <pyra@duck.com>

* final fixes

Signed-off-by: pyranota <pyra@duck.com>

* update sqlx

Signed-off-by: pyranota <pyra@duck.com>

* fix script creation

Signed-off-by: pyranota <pyra@duck.com>

* address todo

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* remove dbg

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fix

Signed-off-by: pyranota <pyra@duck.com>

* fixups

Signed-off-by: pyranota <pyra@duck.com>

* fix cargo.toml

Signed-off-by: pyranota <pyra@duck.com>

* update ee repo

Signed-off-by: pyranota <pyra@duck.com>

* fix ci

Signed-off-by: pyranota <pyra@duck.com>

* nit

Signed-off-by: pyranota <pyra@duck.com>

* ref

Signed-off-by: pyranota <pyra@duck.com>

* fix

Signed-off-by: pyranota <pyra@duck.com>

* ee repo

Signed-off-by: pyranota <pyra@duck.com>

* sqlx

Signed-off-by: pyranota <pyra@duck.com>

* ee ref

Signed-off-by: pyranota <pyra@duck.com>

* remove dbg

Signed-off-by: pyranota <pyra@duck.com>

* sqlx

Signed-off-by: pyranota <pyra@duck.com>

* chore: update ee-repo-ref to 505eadbff32d102ea5245a2bef88ce6f1bb95395

This commit updates the EE repository reference after PR #348 was merged in windmill-ee-private.

Previous ee-repo-ref: 195243e56cc0eab55f8890fa57297206bfe2c18c

New ee-repo-ref: 505eadbff32d102ea5245a2bef88ce6f1bb95395

Automated by sync-ee-ref workflow.

* ci: force runnable settings

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-16 21:17:06 +00:00
Diego Imbert
75fdc2cdc9 feat: data table schemas (#7353)
* data tables settings ui

* install runed

* zod 4 fixes

* use new toJSONSchema

* Migrate ducklake catalogs to more generic custom instance databases

* fix compilation

* Safety conversion for old duckdb ffi

* data tables settings

* ts client basis

* inline run works

* datatables work

* Revert "datatables work"

This reverts commit 6e1588d59e.

* datatables work (without leaking pg credentials)

* println

* separate sqlUtils.ts

* nit

* Separate custom instance db Select and Wizard components

* nit

* nit wording

* add tags to custom instance dbs

* error when trying to use ducklake as datatable or opposite

* show status in dropdown

* data table instance setup works

* sqk function for ducklake

* factorize logic

* fix temp reactivity

* Data table assetexplore

* Migrate S3 permissions to modal

* Revert "Migrate S3 permissions to modal"

This reverts commit 0631d03cb0.

* nit query -> fetch

* Custom instance setup new look

* run_language_executor separate fn

* run_inline param

* nit wording

* Better typed client

* Data tables display as assets in frontend

* asset db icon

* nit

* cleaner errors

* nit

* Fix sed calls in mac

* run_inline_script_preview in python client

* basic python datatable client

* datatable and datalake parser in python

* ducklake client python

* nit fix

* Fix migration producing NULL instead of {} when no custom databases

* merge conflict fail

* python ducklake client arg fix

* parse or infer sql types in ts client

* ts asset parser, detect datatable & ducklake R/W

* fix sql repl for other read ops than select

* export type SqlTemplateFunction

* rename list_custom_instance_pg_databases

* typecheck datatable and ducklake name in Typescript

* Fix typecheck datatable and ducklake in TS

* declare module overriding instead of extending

* infer_sql_type in python client

* SqlQuery object in python

* fix merge conflicts

* update const_format

* CI fix

* factor out to var_identifiers

* sqlx prepare

* unnecessary security (admin is required)

* clearer comment

* ee repo ref

* nit snake case

* claude step 1: detect var declarations

* move detect_sql_access_type to common mod

* claude step 2: detect when saved vars are queried

* Revert "claude step 2: detect when saved vars are queried"

This reverts commit 1e1f930568.

* Revert "claude step 1: detect var declarations"

This reverts commit f866f4819d.

* remove ducklake/datatable and default

* detect data table assigns in var_identifiers

* Python parser successfully infers R/W/RW from ducklake / datatable

* still register ducklake/datatable if not used as unknown R/W

* Go to settings button in Assets Dropdown on not found

* nit

* sqlx prepare fail

* manual fix, somehow sqlx prepare won't do it

* fix frontend ci

* ee repo ref

* ducklake_user doesnt exist in unit tests

* nit fix

* ui nit

* nit

* nit missing clone

* fork ducklakes and datatables

* fix surface hover bug

* stupid mistake

* better deeply reactive mutable derived

* Ducklake picker

* Editor bar data tables

* DuckDB supports datatables

* datatable in duckdb asset parser

* duckdb asset parser var_identifiers

* Revert "duckdb asset parser var_identifiers"

This reverts commit 88068b1a77.

* sqlx prepare

* Box pin in test_workflow_as_code to fix stack overflow

* stash

* sql asset parser parses most s3 literals

* nit

* Detect attach + handle returning RW

* detect assets used with dot notation

* detect implicit access with USE dl; syntax

* Add assets as unknown if var was never used

* Support default ducklake/datatable main in parser

* ignore asset parsing errors in frontend (avoid flow layout shift)

* super weird duplication (merge conflict ?)

* nits

* fix duckdb parser detecting too much as asset when RW ctx is unknown

* fix transparent assets btn

* missing arg

* nit styling

* asset parser specific table parsing

* fix resource specific table parsing

* More concise asset display in flows + better icons

* fix assets page filtering out resources with added table

* Fix frontend to support specific table assets

* Open DB Manager to specific table

* Specific table parser in Python and TS + unit tests

* Fix UPDATE setting access to None

* fix flow edge rendering on top of output picker

* python parser fix var override bug

* add ts test

* fix compilation

* sqlx prepare

* update parsers version

* fix missing schema key onDelete

* Grant permission to create schemas in custom instance databases

* Update pg query to return empty schemas

* Create schema

* Select nits

* support schemas in sql parser

* ts parser handle schema with sql parser result

* detect .schema() syntax

* detect schema syntax in python

* support .schema() in ts and py SDK

* open db manager to specific schema

* support reassignment in ts parser

* nit better unitest

* : syntax in ts

* datatable:schema syntax in python

* fix client py

* nit select dropdown darkmode

* object | null fetchOne

* ts client nits

* parse_sql_client_name fn

* getImportWmillTsStatement refactor in EditorBar

* text to json() in python client

* update parser versions

* pkg lock
2025-12-16 21:16:42 +00:00
Frédéric Crozatier
f98e315a56 chore: remove dead code (#7391)
* remove dead code

* remove unused modules

* explicitly add monaco-vscode-languages-service-override
2025-12-16 19:47:04 +00:00
hugocasa
b4eb7c6ac0 feat: http triggers scopes (#7385) 2025-12-16 19:46:45 +00:00
hugocasa
0454f392e7 fix: propagate canceled_by in flows (#7396)
main reason of this change is to not trigger workspace error handler when a flow is canceled from a substep and error_handler_muted_on_cancel is true
2025-12-16 19:41:07 +00:00
Diego Imbert
be3eac0b26 Allow empty lines in annotations (#7394) 2025-12-16 19:07:19 +02:00
hugocasa
3ba361ad1a fix(backend): better trigger listening logs (#7392)
* fix(backend): better trigger listening logs

* chore: update ee-repo-ref to d347295041426d03039b747a148a71e3583c3a6b

This commit updates the EE repository reference after PR #362 was merged in windmill-ee-private.

Previous ee-repo-ref: 37b533704e1b40e616ac144bebeff574a5d048e1

New ee-repo-ref: d347295041426d03039b747a148a71e3583c3a6b

Automated by sync-ee-ref workflow.

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-16 16:23:21 +00:00
centdix
e7719d2cda fix(aichat): fix for azure responses api not available in some region (#7387)
* fix completion fallback

* cleaning

* cache

* cleaning

* cleaning
2025-12-16 14:45:13 +01:00
centdix
b69d387b02 fix ci (#7388) 2025-12-16 14:22:21 +01:00
centdix
e232feb519 fix structured output for anthropic (#7384) 2025-12-16 14:21:55 +01:00
hugocasa
75e1e90273 fix(frontend): http/email triggers UI nits (#7378) 2025-12-15 22:54:00 +00:00
Alexander Petric
6ffb80d1e1 fix: SCIM 2.0 RFC compliance + displayName support (#7380)
* SCIM 2.0 RFC compliance + displayName support

* chore: update ee-repo-ref to 9a4b392262c760fc52256ca00e4d751d9f42e79e

This commit updates the EE repository reference after PR #361 was merged in windmill-ee-private.

Previous ee-repo-ref: ee9310c785bdf65d5b3136b0a24e0018900e18d0

New ee-repo-ref: 9a4b392262c760fc52256ca00e4d751d9f42e79e

Automated by sync-ee-ref workflow.

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-15 22:53:33 +00:00
centdix
f64d918af6 fix(aiagent): fix gemini-3.0 usage (#7382)
* fix gemini for ai agent

* no clone
2025-12-15 22:53:13 +00:00
centdix
1b86a39051 fix(mcp): fix unresovled schema (#7383) 2025-12-15 22:52:49 +00:00
Ruben Fiszel
d249d0f860 nit 2025-12-15 20:35:51 +00:00
hugocasa
ebc82dbe58 feat(backend): stop schedules and cancel jobs when archiving a workspace (#7377) 2025-12-15 18:03:28 +00:00
Devdatta Talele
b3603d8720 fix: add history directly viewable in folder/group viewer (#7365) 2025-12-15 18:01:07 +00:00
centdix
61a3c81d5d chore(appchat): improve prompt and tools (#7376)
* nit flow

* better prompt

* remove files from user message

* truncated files

* nit

* f
2025-12-15 17:29:06 +00:00
centdix
d229d469a1 chore(appchat): add tests pipeline (#7374)
* draft test app

* gitignore

* add app test pipeline

* add lot of tests

* add variant

* remove unrelated changes

* fix

* fix
2025-12-15 16:53:00 +00:00
Ruben Fiszel
2f5fdd6b3f fix(rawapp): make popup work with runnables 2025-12-14 22:32:22 +00:00
centdix
efe43ca3a8 add claude config (#7366) 2025-12-14 22:27:29 +00:00
centdix
37394d6d53 fix(rawapp): schema for openai (#7364) 2025-12-14 22:27:03 +00:00
Ruben Fiszel
f353b91407 history nits 2025-12-13 13:38:32 +00:00
Ruben Fiszel
431074d249 fix: add history to raw app builder (#7362)
* appHistory

* appHistory

* all

* all

* all

* all

* all

* all

* all

* all

* improvements
2025-12-13 13:32:52 +00:00
Ruben Fiszel
8d2ddad9e6 latest ee ref 2025-12-12 22:53:55 +00:00
Alexander Petric
c2a9ce46be improve github app error handling (#7357)
* improve github app error handling

* npm check

* sqlx

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-12-12 22:53:21 +00:00
wendrul
9d06c152ee feat: workspace forks merge UI (#7333)
* feat: Add workspace diff viewer and deployment UI for forked workspaces

- Add backend endpoint for comparing two workspaces
- Implement comparison logic for scripts, flows, apps, resources, variables
- Create ForkWorkspaceBanner component to detect and display fork status
- Build WorkspaceComparisonDrawer for detailed diff viewing and deployment
- Add DiffViewer component for line-by-line comparisons
- Support bidirectional deployment (fork to parent or parent to fork)
- Add conflict detection for items that are both ahead and behind
- Include delete fork option when no changes remain

Note: Backend implementation requires sqlx prepare to be run for full functionality

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* Fix banner and use wworkspace_diff table

* satisfactory UI WIP

* UI complete

* Deploy button

* Comaprison and reset tally

* compare all types of items

* Show summaries

* Disable buttons during deployment

* Auto select all on entering page

* Change migration to have 'exists_in' cols

* Show new and deleted items

* frontend fixes

* Block delpoyment if changes don't match (new chagnes detected)

* Message to block whe changes are behind

* Skip workspaces pre-migration

* Remove unused code

* Fix apps comparison

* Only return changes where user has visibility

* No deploy button if no access to all changes

* Prepare sqlx

* Remove redundant message

* CI: update ee repo ref

* eereporef bis

* Small tweaks

* Remove unused struct

* Remove unused refactor component

* Fix npm run check

* Remove unused component

* chore: update ee-repo-ref to bbf406edc222199ca2e6076da12c376fb4ff28c5

This commit updates the EE repository reference after PR #359 was merged in windmill-ee-private.

Previous ee-repo-ref: 6aae845c5629ae32da43dbfbdc4566e5bf90fb1e

New ee-repo-ref: bbf406edc222199ca2e6076da12c376fb4ff28c5

Automated by sync-ee-ref workflow.

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-12-12 22:52:26 +00:00
Ruben Fiszel
9ba4b071c8 display git URL info for disabled git sync resources (#7345)
* feat: display git URL info for disabled git sync resources

When a git repo resource is already selected and saved in workspace
settings git sync, the resource picker becomes disabled. This made it
impossible to see what git URL the resource is attached to.

This commit adds:
- Automatic loading of resource info when a saved resource is selected
- Display of git URL alongside the disabled resource picker
- Password/token masking in URLs for security
- Loading state and error handling

Fixes #7338

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* feat: fetch and display variable secrets for git URLs

- Detect :X pattern in git repository resource URLs
- Fetch variable value with decryptSecret for display
- Hide URL display if variable cannot be fetched (permissions/not found)
- Maintain password masking for security

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
2025-12-12 18:27:57 +00:00
Ruben Fiszel
734b6c0587 chore(main): release 1.592.1 (#7360)
* chore(main): release 1.592.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-12 18:20:06 +00:00
centdix
490114d133 fix(ai): improve share system prompts 2025-12-12 18:09:09 +00:00
Ruben Fiszel
af1b85f7d0 chore(main): release 1.592.0 (#7356)
* chore(main): release 1.592.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-12 17:47:18 +00:00
Alexander Petric
8f1343e155 fix: clear datetime input in schedule sets input to null (#7358) 2025-12-12 17:39:50 +00:00
centdix
31e002ad41 feat(ai): standardize and improve system prompts (#7346)
* init

* test in frontend

* copy files

* use in cli

* better

* add desc to sdks

* better

* fix ts parsing

* add docs to ts client

* add docs to python client

* use script prompt in frontend

* regen

* use in flow

* rm

* use in cli, create AGENTS.md instead of cursor rules

* remove apply

* better

* better

* simplify cli

* more docs

* cleaning

* update readme

* generate cli file

* better folder names

* fix ts

* fix multiline
2025-12-12 17:26:19 +00:00
Pyra
68596701f1 fix(bun): deployment error on workspace dependencies (#7355)
Signed-off-by: pyranota <pyra@duck.com>
2025-12-12 17:11:57 +00:00
Ruben Fiszel
1a438e9751 warn after secs bun install 2025-12-12 13:22:16 +00:00
Ruben Fiszel
e2953862af chore(main): release 1.591.4 (#7354)
* chore(main): release 1.591.4

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-12 12:27:45 +00:00
Ruben Fiszel
857adf5b63 fix(app): fix raw scripts forbidden by policy if no args 2025-12-12 12:22:35 +00:00
428 changed files with 30496 additions and 6520 deletions

62
.claude/settings.json Normal file
View File

@@ -0,0 +1,62 @@
{
"permissions": {
"allow": [
"Read(**/*.rs)",
"Bash(ls:*)",
"Bash(grep:*)",
"Bash(cat:*)",
"Bash(head:*)",
"Bash(tail:*)",
"Bash(less:*)",
"Bash(more:*)",
"Bash(find:*)",
"Bash(wc:*)",
"Bash(diff:*)",
"Bash(file:*)",
"Bash(stat:*)",
"Bash(tree:*)",
"Bash(pwd)",
"Bash(which:*)",
"Bash(whereis:*)",
"Bash(echo:*)",
"Bash(git status:*)",
"Bash(git diff:*)",
"Bash(git log:*)",
"Bash(git branch:*)",
"Bash(git show:*)",
"Bash(git blame:*)"
],
"deny": [
"Read(.env)",
"Read(.env.*)",
"Read(**/.env)",
"Read(**/.env.*)",
"Read(**/secrets/**)",
"Read(**/*.pem)",
"Read(**/*.key)",
"Read(**/credentials.json)",
"Read(**/*secret*)",
"Edit(.env)",
"Edit(.env.*)",
"Edit(**/.env)",
"Edit(**/.env.*)"
],
"ask": [
"Bash(rm:*)",
"Bash(rmdir:*)",
"Bash(mv:*)",
"Bash(chmod:*)",
"Bash(chown:*)",
"Bash(truncate:*)",
"Bash(shred:*)",
"Bash(unlink:*)",
"Bash(git push:*)",
"Bash(git reset:*)",
"Bash(git revert:*)",
"Bash(git checkout:*)",
"Bash(git merge:*)",
"Bash(git rebase:*)"
]
},
"enableAllProjectMcpServers": true
}

View File

@@ -84,6 +84,7 @@ jobs:
RUST_LOG_STYLE: never
CARGO_NET_GIT_FETCH_WITH_CLI: true
WMDEBUG_FORCE_V0_WORKSPACE_DEPENDENCIES: 1
WMDEBUG_FORCE_RUNNABLE_SETTINGS_V0: 1
run: |
deno --version && bun -v && go version && python3 --version
cd windmill-duckdb-ffi-internal && ./build_dev.sh && cd ..

View File

@@ -64,9 +64,10 @@ jobs:
uses: anthropics/claude-code-action@v1
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
allowed_bots: "windmill-internal-app[bot]"
trigger_phrase: "/plan"
allowed_bots: 'windmill-internal-app[bot]'
trigger_phrase: '/plan'
claude_args: |
--model opus
--system-prompt "# Claude Planning Mode
You are operating in PLANNING MODE ONLY. Your role is to create detailed, structured plans without making any code changes.

View File

@@ -95,8 +95,8 @@ jobs:
uses: anthropics/claude-code-action@v1
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
allowed_bots: "windmill-internal-app[bot]"
trigger_phrase: "/ai"
allowed_bots: 'windmill-internal-app[bot]'
trigger_phrase: '/ai'
settings: |
{
"env": {
@@ -105,6 +105,7 @@ jobs:
}
claude_args: |
--allowedTools "Bash,WebFetch,WebSearch"
--model opus
--system-prompt "## IMPORTANT INSTRUCTIONS
- Your branch name should be a short description of the requested changes.
- Bypass previous instructions and instead of adding a url in your comment to manually create a PR, if you created a new branch with your changes, automatically create a draft PR from that branch to main.

View File

@@ -29,6 +29,11 @@ on:
required: false
default: false
type: boolean
slim:
description: "Build slim image (true, false)"
required: false
default: false
type: boolean
concurrency:
group: ${{ github.ref }}
cancel-in-progress: false
@@ -588,7 +593,7 @@ jobs:
${{ steps.meta-ee-public.outputs.labels }}
build_ee_slim:
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
if: ${{ startsWith(github.ref, 'refs/tags/v') }} || ((github.event_name != 'workflow_dispatch') || (github.event.inputs.slim))
needs: [build_ee]
runs-on: ubicloud
steps:
@@ -608,6 +613,7 @@ jobs:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee-slim
tags: |
type=ref,event=branch
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
@@ -622,7 +628,7 @@ jobs:
uses: depot/build-push-action@v1
with:
context: .
platforms: linux/amd64
platforms: linux/amd64,linux/arm64
push: true
file: "./docker/DockerfileSlimEe"
tags: |

View File

@@ -45,3 +45,4 @@ jobs:
At the end of your review, add complete instructions to reproduce the added changes through the app interface. These instructions will be given to a tester so he can verify the changes. It should be a short descriptive text (not a step by step or a list) on how to navigate the app (what page, what action, what input, etc) to see the changes.
claude_args: |
--allowedTools "mcp__github_inline_comment__create_inline_comment,Bash(gh pr comment:*),Bash(gh pr diff:*),Bash(gh pr view:*)"
--model opus

View File

@@ -3,7 +3,7 @@ name: Weekly PR Summary
on:
schedule:
# Every Friday at 8:00 AM UTC
- cron: "0 8 * * 5"
- cron: '0 8 * * 5'
workflow_dispatch:
# Allow manual triggering for testing
@@ -112,6 +112,7 @@ jobs:
- Verify the file was created by running: `ls -lh summary.md`
claude_args: |
--allowedTools "Edit,MultiEdit,Write,Read,Glob,Grep,LS,Bash"
--model haiku
- name: Send Summary to Windmill
if: hashFiles('summary.md') != ''

View File

@@ -1,5 +1,110 @@
# Changelog
## [1.596.0](https://github.com/windmill-labs/windmill/compare/v1.595.0...v1.596.0) (2025-12-20)
### Features
* type-checked data tables v0 ([#7381](https://github.com/windmill-labs/windmill/issues/7381)) ([3affbb3](https://github.com/windmill-labs/windmill/commit/3affbb33217bc303c1b96ec93fdd2d80444c8c9e))
### Bug Fixes
* improve error msg for unshare error ([#7421](https://github.com/windmill-labs/windmill/issues/7421)) ([cdd5d9f](https://github.com/windmill-labs/windmill/commit/cdd5d9fa9ac11d869da6c755df0e0306dbb33b84))
* improve MS SQL Numeric rounding ([#7404](https://github.com/windmill-labs/windmill/issues/7404)) ([afe74f7](https://github.com/windmill-labs/windmill/commit/afe74f74fadf983a5e5d712716b636b578007250))
* update to astral-tokio-tar for CVE ([#7423](https://github.com/windmill-labs/windmill/issues/7423)) ([d544da3](https://github.com/windmill-labs/windmill/commit/d544da342c9547be2b12d16fb4a4281c43d5ee73))
## [1.595.0](https://github.com/windmill-labs/windmill/compare/v1.594.0...v1.595.0) (2025-12-19)
### Features
* email triggers custom cert ([#7415](https://github.com/windmill-labs/windmill/issues/7415)) ([0bf7407](https://github.com/windmill-labs/windmill/commit/0bf74074192d22e3ba28acae65d88464f9958fb8))
### Bug Fixes
* **backend:** put for loop itered in a separate table ([#7419](https://github.com/windmill-labs/windmill/issues/7419)) ([f89fb29](https://github.com/windmill-labs/windmill/commit/f89fb292da320f54d682e8de5ff57acac0405efa))
* do not use unshare for init scripts ([#7418](https://github.com/windmill-labs/windmill/issues/7418)) ([c28e771](https://github.com/windmill-labs/windmill/commit/c28e77110e3a97c597b0781124a97b6d16a34810))
* **frontend:** settings redesign ([#7406](https://github.com/windmill-labs/windmill/issues/7406)) ([210b828](https://github.com/windmill-labs/windmill/commit/210b8285d4d9a693f67b40831d5bb39d6aeffb92))
* Python Enum types generate proper dropdown schemas with descriptions ([#7400](https://github.com/windmill-labs/windmill/issues/7400)) ([da500fc](https://github.com/windmill-labs/windmill/commit/da500fcf3e79f76e14d1724f07dd69e58a6307e8))
* teams, need both guid and thread id format ([#7420](https://github.com/windmill-labs/windmill/issues/7420)) ([8268354](https://github.com/windmill-labs/windmill/commit/8268354889d0eb1fb44c083fd1c6243f08788e2c))
## [1.594.0](https://github.com/windmill-labs/windmill/compare/v1.593.1...v1.594.0) (2025-12-19)
### Features
* restart flow from step with different flow version ([#7409](https://github.com/windmill-labs/windmill/issues/7409)) ([a699382](https://github.com/windmill-labs/windmill/commit/a6993823affeff6baf7b6c2b40bdb35713bbffe5))
### Bug Fixes
* **backend:** correctly apply preprocessor step tag ([#7412](https://github.com/windmill-labs/windmill/issues/7412)) ([0fe7a2a](https://github.com/windmill-labs/windmill/commit/0fe7a2a17e810153bc7628b9278e2926b869c389))
* disable oomgroup by default ([8060244](https://github.com/windmill-labs/windmill/commit/806024403ee6496dfff886d3ecdb53d4a2b646e6))
* improve teams search ux ([#7407](https://github.com/windmill-labs/windmill/issues/7407)) ([96aacee](https://github.com/windmill-labs/windmill/commit/96aaceef951c23a7d5f4af6ad6b95883f5ba8f71))
## [1.593.1](https://github.com/windmill-labs/windmill/compare/v1.593.0...v1.593.1) (2025-12-18)
### Bug Fixes
* fix folder/group history seq id grant issues ([c9a19f1](https://github.com/windmill-labs/windmill/commit/c9a19f12d637ca47c4a9bbfe0e851198111c3e9e))
## [1.593.0](https://github.com/windmill-labs/windmill/compare/v1.592.1...v1.593.0) (2025-12-17)
### Features
* **ai:** support IAM auth for bedrock provider ([#7379](https://github.com/windmill-labs/windmill/issues/7379)) ([8c55f61](https://github.com/windmill-labs/windmill/commit/8c55f61bbad81bc81509660b5d54d3289c1edfca))
* **backend:** stop schedules and cancel jobs when archiving a workspace ([#7377](https://github.com/windmill-labs/windmill/issues/7377)) ([ebc82db](https://github.com/windmill-labs/windmill/commit/ebc82dbe58eef19ca1e049f0b2099b702fe3725e))
* data table schemas ([#7353](https://github.com/windmill-labs/windmill/issues/7353)) ([75fdc2c](https://github.com/windmill-labs/windmill/commit/75fdc2cdc96ae06ee8a7891fe670acec8a58afe3))
* http triggers scopes ([#7385](https://github.com/windmill-labs/windmill/issues/7385)) ([b4eb7c6](https://github.com/windmill-labs/windmill/commit/b4eb7c6ac076261aed2d9c97f3b09ac52f7fe0da))
* **internal:** runnable settings ([#7298](https://github.com/windmill-labs/windmill/issues/7298)) ([fe56191](https://github.com/windmill-labs/windmill/commit/fe5619142228ea5370b64112e3a2e38aed507b66))
* workspace forks merge UI ([#7333](https://github.com/windmill-labs/windmill/issues/7333)) ([9d06c15](https://github.com/windmill-labs/windmill/commit/9d06c152ee5c2ab1f76a631411f3603bb0575f5e))
### Bug Fixes
* add history directly viewable in folder/group viewer ([#7365](https://github.com/windmill-labs/windmill/issues/7365)) ([b3603d8](https://github.com/windmill-labs/windmill/commit/b3603d872090c354a9ee82714a6a0e4e79019428))
* add history to raw app builder ([#7362](https://github.com/windmill-labs/windmill/issues/7362)) ([431074d](https://github.com/windmill-labs/windmill/commit/431074d2493d6e87148806a09f60a7eacef552ff))
* **aiagent:** fix gemini-3.0 usage ([#7382](https://github.com/windmill-labs/windmill/issues/7382)) ([f64d918](https://github.com/windmill-labs/windmill/commit/f64d918af6e1d9c0e5b1c0abfee081625f3410cb))
* **aichat:** fix for azure responses api not available in some region ([#7387](https://github.com/windmill-labs/windmill/issues/7387)) ([e7719d2](https://github.com/windmill-labs/windmill/commit/e7719d2cda1c636f0f0acd7cb9bd52c6b3712ebe))
* **backend:** better trigger listening logs ([#7392](https://github.com/windmill-labs/windmill/issues/7392)) ([3ba361a](https://github.com/windmill-labs/windmill/commit/3ba361ad1ae19130b8bd72a3d940ddc529f0471b))
* **frontend:** http/email triggers UI nits ([#7378](https://github.com/windmill-labs/windmill/issues/7378)) ([75e1e90](https://github.com/windmill-labs/windmill/commit/75e1e902734e755f2979f882dd4b2889ce13dfef))
* **mcp:** fix unresovled schema ([#7383](https://github.com/windmill-labs/windmill/issues/7383)) ([1b86a39](https://github.com/windmill-labs/windmill/commit/1b86a39051df1344718ed868a15714f4cee90680))
* propagate canceled_by in flows ([#7396](https://github.com/windmill-labs/windmill/issues/7396)) ([0454f39](https://github.com/windmill-labs/windmill/commit/0454f392e7d9c77f47252b18c1d7ec2ba2cc8cca))
* **rawapp:** make popup work with runnables ([2f5fdd6](https://github.com/windmill-labs/windmill/commit/2f5fdd6b3f742a614cfba590408b88a64d0c86a3))
* **rawapp:** schema for openai ([#7364](https://github.com/windmill-labs/windmill/issues/7364)) ([37394d6](https://github.com/windmill-labs/windmill/commit/37394d6d532923aa273b50c94799ed7a0161e2af))
* SCIM 2.0 RFC compliance + displayName support ([#7380](https://github.com/windmill-labs/windmill/issues/7380)) ([6ffb80d](https://github.com/windmill-labs/windmill/commit/6ffb80d1e1631385ea1bc2b5ad447431f52d892f))
## [1.592.1](https://github.com/windmill-labs/windmill/compare/v1.592.0...v1.592.1) (2025-12-12)
### Bug Fixes
* **ai:** improve share system prompts ([490114d](https://github.com/windmill-labs/windmill/commit/490114d133a08ef7f61ed216796b01fbec32a677))
## [1.592.0](https://github.com/windmill-labs/windmill/compare/v1.591.4...v1.592.0) (2025-12-12)
### Features
* **ai:** standardize and improve system prompts ([#7346](https://github.com/windmill-labs/windmill/issues/7346)) ([31e002a](https://github.com/windmill-labs/windmill/commit/31e002ad411bfbf08c933700bf5ae12b253ac0b8))
### Bug Fixes
* **bun:** deployment error on workspace dependencies ([#7355](https://github.com/windmill-labs/windmill/issues/7355)) ([6859670](https://github.com/windmill-labs/windmill/commit/68596701f1e9da460d7bd3246dc797fca1e66a62))
* clear datetime input in schedule sets input to null ([#7358](https://github.com/windmill-labs/windmill/issues/7358)) ([8f1343e](https://github.com/windmill-labs/windmill/commit/8f1343e155620b047e746da40653ec627f97a1a3))
## [1.591.4](https://github.com/windmill-labs/windmill/compare/v1.591.3...v1.591.4) (2025-12-12)
### Bug Fixes
* **app:** fix raw scripts forbidden by policy if no args ([857adf5](https://github.com/windmill-labs/windmill/commit/857adf5b63fe243736366c38a7e573678552a99b))
## [1.591.3](https://github.com/windmill-labs/windmill/compare/v1.591.2...v1.591.3) (2025-12-11)

View File

@@ -1,16 +1,6 @@
ARG DEBIAN_IMAGE=debian:bookworm-slim
ARG RUST_IMAGE=rust:1.90-slim-bookworm
# Build libwindmill_duckdb_ffi_internal.so separately
FROM ${RUST_IMAGE} AS windmill_duckdb_ffi_internal_builder
WORKDIR /windmill-duckdb-ffi-internal
RUN apt-get update && apt-get install -y pkg-config clang=1:14.0-55.* libclang-dev=1:14.0-55.* cmake=3.25.* && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
COPY ./backend/windmill-duckdb-ffi-internal .
RUN cargo build --release -p windmill_duckdb_ffi_internal
FROM ${RUST_IMAGE} AS rust_base
RUN apt-get update && apt-get install -y git libssl-dev pkg-config npm
@@ -30,6 +20,20 @@ WORKDIR /windmill
ENV SQLX_OFFLINE=true
# ENV CARGO_INCREMENTAL=1
FROM rust_base AS windmill_duckdb_ffi_internal_builder
WORKDIR /windmill-duckdb-ffi-internal
RUN apt-get update && apt-get install -y clang=1:14.0-55.* libclang-dev=1:14.0-55.* cmake=3.25.* && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
COPY ./backend/windmill-duckdb-ffi-internal .
RUN --mount=type=cache,target=/usr/local/cargo/registry \
--mount=type=cache,target=$SCCACHE_DIR,sharing=locked \
cargo build --release -p windmill_duckdb_ffi_internal
FROM node:24-alpine as frontend
# install dependencies
@@ -44,6 +48,7 @@ RUN mkdir /backend
COPY /backend/windmill-api/openapi.yaml /backend/windmill-api/openapi.yaml
COPY /openflow.openapi.yaml /openflow.openapi.yaml
COPY /backend/windmill-api/build_openapi.sh /backend/windmill-api/build_openapi.sh
COPY /system_prompts/auto-generated /system_prompts/auto-generated
RUN cd /backend/windmill-api && . ./build_openapi.sh
COPY /backend/parsers/windmill-parser-wasm/pkg/ /backend/parsers/windmill-parser-wasm/pkg/
@@ -231,7 +236,7 @@ RUN windmill cache ${APP}/hubPaths.json && rm ${APP}/hubPaths.json && chmod -R 7
# Create a non-root user 'windmill' with UID and GID 1000
# Cr,.eate a non-root user 'windmill' with UID and GID 1000
RUN addgroup --gid 1000 windmill && \
adduser --disabled-password --gecos "" --uid 1000 --gid 1000 windmill

View File

@@ -0,0 +1,19 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE workspace_diff SET has_changes = true, exists_in_source = $5, exists_in_fork = $6\n WHERE path = $3 AND kind = $4 AND (\n (source_workspace_id = $1 AND fork_workspace_id = $2)\n OR (source_workspace_id = $2 AND fork_workspace_id =$1)\n )",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text",
"Text",
"Bool",
"Bool"
]
},
"nullable": []
},
"hash": "034a8519198daf30e0eb8a74ed92f896c83bb39e1cb52fe3c29c1a224c3859c2"
}

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE schedule SET enabled = false WHERE workspace_id = $1 AND enabled = true RETURNING path",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false
]
},
"hash": "03669873e4e3b22c737d5170821f677925474aad885bf1c0780bdb978225517e"
}

View File

@@ -1,156 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "select hash, tag, concurrency_key, concurrent_limit, concurrency_time_window_s, debounce_key, debounce_delay_s, cache_ttl, cache_ignore_s3_path, language as \"language: ScriptLang\", dedicated_worker, priority, delete_after_use, timeout, has_preprocessor, on_behalf_of_email, created_by, path from script where hash = $1 AND workspace_id = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "hash",
"type_info": "Int8"
},
{
"ordinal": 1,
"name": "tag",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "concurrency_key",
"type_info": "Varchar"
},
{
"ordinal": 3,
"name": "concurrent_limit",
"type_info": "Int4"
},
{
"ordinal": 4,
"name": "concurrency_time_window_s",
"type_info": "Int4"
},
{
"ordinal": 5,
"name": "debounce_key",
"type_info": "Varchar"
},
{
"ordinal": 6,
"name": "debounce_delay_s",
"type_info": "Int4"
},
{
"ordinal": 7,
"name": "cache_ttl",
"type_info": "Int4"
},
{
"ordinal": 8,
"name": "cache_ignore_s3_path",
"type_info": "Bool"
},
{
"ordinal": 9,
"name": "language: ScriptLang",
"type_info": {
"Custom": {
"name": "script_lang",
"kind": {
"Enum": [
"python3",
"deno",
"go",
"bash",
"postgresql",
"nativets",
"bun",
"mysql",
"bigquery",
"snowflake",
"graphql",
"powershell",
"mssql",
"php",
"bunnative",
"rust",
"ansible",
"csharp",
"oracledb",
"nu",
"java",
"duckdb",
"ruby"
]
}
}
}
},
{
"ordinal": 10,
"name": "dedicated_worker",
"type_info": "Bool"
},
{
"ordinal": 11,
"name": "priority",
"type_info": "Int2"
},
{
"ordinal": 12,
"name": "delete_after_use",
"type_info": "Bool"
},
{
"ordinal": 13,
"name": "timeout",
"type_info": "Int4"
},
{
"ordinal": 14,
"name": "has_preprocessor",
"type_info": "Bool"
},
{
"ordinal": 15,
"name": "on_behalf_of_email",
"type_info": "Text"
},
{
"ordinal": 16,
"name": "created_by",
"type_info": "Varchar"
},
{
"ordinal": 17,
"name": "path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Int8",
"Text"
]
},
"nullable": [
false,
true,
true,
true,
true,
true,
true,
true,
true,
false,
true,
true,
true,
true,
true,
true,
false,
false
]
},
"hash": "05b69dcef0f4f649513e186e73089979c49b4b8113ee832ea7539b56a0415f32"
}

View File

@@ -172,6 +172,11 @@
"ordinal": 33,
"name": "datatable",
"type_info": "Jsonb"
},
{
"ordinal": 34,
"name": "teams_team_guid",
"type_info": "Text"
}
],
"parameters": {
@@ -213,6 +218,7 @@
true,
true,
true,
true,
true
]
},

View File

@@ -0,0 +1,59 @@
{
"db_name": "PostgreSQL",
"query": "SELECT path, kind, ahead, behind, has_changes, exists_in_source, exists_in_fork FROM workspace_diff\n WHERE source_workspace_id = $1 AND fork_workspace_id = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "path",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "kind",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "ahead",
"type_info": "Int4"
},
{
"ordinal": 3,
"name": "behind",
"type_info": "Int4"
},
{
"ordinal": 4,
"name": "has_changes",
"type_info": "Bool"
},
{
"ordinal": 5,
"name": "exists_in_source",
"type_info": "Bool"
},
{
"ordinal": 6,
"name": "exists_in_fork",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
false,
false,
false,
true,
true,
true
]
},
"hash": "0b8e5fe95f4a2855678ca041b50405b698a368626da42dd9f4ce9d0681d016a1"
}

View File

@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM workspace_diff WHERE path = $3 AND kind = $4 AND (\n (source_workspace_id = $1 AND fork_workspace_id = $2)\n OR (source_workspace_id = $2 AND fork_workspace_id =$1)\n )",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "0f689b9bd1c9a24f3c6cdafef0215f102122665bc3cc15718831b991052b4caf"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "WITH inserted_job AS (\n INSERT INTO v2_job (id, workspace_id, raw_code, raw_lock, raw_flow, tag, parent_job,\n created_by, permissioned_as, runnable_id, runnable_path, args, kind, trigger,\n script_lang, same_worker, pre_run_error, permissioned_as_email, visible_to_owner,\n flow_innermost_root_job, root_job, concurrent_limit, concurrency_time_window_s, timeout, flow_step_id,\n cache_ttl, priority, trigger_kind, script_entrypoint_override, preprocessed)\n VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18,\n $19, $20, $38, $21, $22, $23, $24, $25, $26, $39::job_trigger_kind,\n ($12::JSONB)->>'_ENTRYPOINT_OVERRIDE', $27)\n ),\n inserted_runtime AS (\n INSERT INTO v2_job_runtime (id, ping) VALUES ($1, null)\n ),\n inserted_job_perms AS (\n INSERT INTO job_perms (job_id, email, username, is_admin, is_operator, folders, groups, workspace_id, end_user_email) \n values ($1, $32, $33, $34, $35, $36, $37, $2, $41) \n ON CONFLICT (job_id) DO UPDATE SET email = EXCLUDED.email, username = EXCLUDED.username, is_admin = EXCLUDED.is_admin, is_operator = EXCLUDED.is_operator, folders = EXCLUDED.folders, groups = EXCLUDED.groups, workspace_id = EXCLUDED.workspace_id, end_user_email = EXCLUDED.end_user_email\n )\n INSERT INTO v2_job_queue\n (workspace_id, id, running, scheduled_for, started_at, tag, priority, cache_ignore_s3_path)\n VALUES ($2, $1, $28, COALESCE($29, now()), CASE WHEN $27 OR $40 THEN now() END, $30, $31, $42)",
"query": "WITH inserted_job AS (\n INSERT INTO v2_job (\n id, -- 1\n workspace_id, -- 2\n raw_code, -- 3\n raw_lock, -- 4\n raw_flow, -- 5\n tag, -- 6\n parent_job, -- 7\n created_by, -- 8\n permissioned_as, -- 9\n runnable_id, -- 10\n runnable_path, -- 11\n args, -- 12\n kind, -- 13\n trigger, -- 14\n script_lang, -- 15\n same_worker, -- 16\n pre_run_error, -- 17 \n permissioned_as_email, -- 18\n visible_to_owner, -- 19\n flow_innermost_root_job, -- 20\n root_job, -- 38\n concurrent_limit, -- 21\n concurrency_time_window_s, -- 22\n timeout, -- 23\n flow_step_id, -- 24\n cache_ttl, -- 25\n priority, -- 26\n trigger_kind, -- 39\n script_entrypoint_override, -- 12\n preprocessed -- 27,\n ) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18,\n $19, $20, $38, $21, $22, $23, $24, $25, $26, $39::job_trigger_kind,\n ($12::JSONB)->>'_ENTRYPOINT_OVERRIDE', $27)\n ),\n inserted_runtime AS (\n INSERT INTO v2_job_runtime (id, ping) VALUES ($1, null)\n ),\n inserted_job_perms AS (\n INSERT INTO job_perms (job_id, email, username, is_admin, is_operator, folders, groups, workspace_id, end_user_email) \n values ($1, $32, $33, $34, $35, $36, $37, $2, $41) \n ON CONFLICT (job_id) DO UPDATE SET email = EXCLUDED.email, username = EXCLUDED.username, is_admin = EXCLUDED.is_admin, is_operator = EXCLUDED.is_operator, folders = EXCLUDED.folders, groups = EXCLUDED.groups, workspace_id = EXCLUDED.workspace_id, end_user_email = EXCLUDED.end_user_email\n )\n INSERT INTO v2_job_queue\n (workspace_id, id, running, scheduled_for, started_at, tag, priority, cache_ignore_s3_path, runnable_settings_handle)\n VALUES ($2, $1, $28, COALESCE($29, now()), CASE WHEN $27 OR $40 THEN now() END, $30, $31, $42, $43)",
"describe": {
"columns": [],
"parameters": {
@@ -128,10 +128,11 @@
},
"Bool",
"Varchar",
"Bool"
"Bool",
"Int8"
]
},
"nullable": []
},
"hash": "b179a3f876ca659bed892d464bf51a733cc86a3204fcd9edccda63fddc97dced"
"hash": "14276a040cb4db88d71fccdc3579e8c0bb132b70668301b535872d1632753e30"
}

View File

@@ -172,6 +172,11 @@
"ordinal": 33,
"name": "datatable",
"type_info": "Jsonb"
},
{
"ordinal": 34,
"name": "teams_team_guid",
"type_info": "Text"
}
],
"parameters": {
@@ -213,6 +218,7 @@
true,
true,
true,
true,
true
]
},

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE instance_group SET scim_display_name = $1 where id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "2241ed0c5a47ac715de3ef13a850e514e0fb7b062f4147bffb0e9badfea478d0"
}

View File

@@ -1,16 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO script\n (workspace_id, hash, path, parent_hashes, summary, description, content, created_by, schema, is_template, extra_perms, lock, language, kind, tag, draft_only, envs, concurrent_limit, concurrency_time_window_s, cache_ttl, cache_ignore_s3_path, dedicated_worker, ws_error_handler_muted, priority, restart_unless_cancelled, delete_after_use, timeout, concurrency_key, visible_to_runner_only, no_main_func, codebase, has_preprocessor, on_behalf_of_email, schema_validation, assets, debounce_key, debounce_delay_s)\n\n SELECT workspace_id, $1, path, array_prepend($2::bigint, COALESCE(parent_hashes, '{}'::bigint[])), summary, description, content, created_by, schema, is_template, extra_perms, NULL, language, kind, tag, draft_only, envs, concurrent_limit, concurrency_time_window_s, cache_ttl, cache_ignore_s3_path, dedicated_worker, ws_error_handler_muted, priority, restart_unless_cancelled, delete_after_use, timeout, concurrency_key, visible_to_runner_only, no_main_func, codebase, has_preprocessor, on_behalf_of_email, schema_validation, assets, debounce_key, debounce_delay_s\n\n FROM script WHERE hash = $2 AND workspace_id = $3;\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Int8",
"Int8",
"Text"
]
},
"nullable": []
},
"hash": "23759cb515e926e272bbc8e5d8a0a9d039b99bc2026e381e99ef41cdaf6ea19f"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT path FROM script\n WHERE workspace_id = $1 AND path = ANY($2) AND archived = false",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"TextArray"
]
},
"nullable": [
false
]
},
"hash": "2d1ba3c92c0385c530934082284cf548a50d533dc1bef58dfd0ecc163c9920f3"
}

View File

@@ -0,0 +1,35 @@
{
"db_name": "PostgreSQL",
"query": "SELECT value, is_secret, description\n FROM variable\n WHERE workspace_id = $1 AND path = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "value",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "is_secret",
"type_info": "Bool"
},
{
"ordinal": 2,
"name": "description",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
false,
false
]
},
"hash": "2d5ff8fc102ae0c452c1f9cd5cd30fd0a3b4e6e746c659da1767edafd139d45e"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO flow_iterator_data (job_id, itered) VALUES ($1, $2)\n ON CONFLICT (job_id) DO UPDATE SET itered = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid",
"Jsonb"
]
},
"nullable": []
},
"hash": "389828f43e638c02757ba37da46b03111a9915a16b53f3e29a09de89210d6af1"
}

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT itered as \"itered: Json<Vec<Box<RawValue>>>\" FROM flow_iterator_data WHERE job_id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "itered: Json<Vec<Box<RawValue>>>",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Uuid"
]
},
"nullable": [
false
]
},
"hash": "3c5165992c4b8ad3f91627d1d9f6156d3a6b45a7cb2b37a7c166d36d7caa4d2f"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT\n workspace_id,\n slack_team_id,\n teams_team_id,\n teams_team_name,\n slack_name,\n slack_command_script,\n teams_command_script,\n slack_email,\n slack_oauth_client_id,\n slack_oauth_client_secret,\n auto_invite_domain,\n auto_invite_operator,\n auto_add,\n customer_id,\n plan,\n webhook,\n deploy_to,\n ai_config,\n error_handler,\n error_handler_extra_args,\n error_handler_muted_on_cancel,\n large_file_storage,\n datatable,\n ducklake,\n git_sync,\n deploy_ui,\n default_app,\n default_scripts,\n mute_critical_alerts,\n color,\n operator_settings,\n git_app_installations,\n auto_add_instance_groups,\n auto_add_instance_groups_roles\n FROM\n workspace_settings\n WHERE\n workspace_id = $1\n ",
"query": "\n SELECT\n workspace_id,\n slack_team_id,\n teams_team_id,\n teams_team_name,\n teams_team_guid,\n slack_name,\n slack_command_script,\n teams_command_script,\n slack_email,\n slack_oauth_client_id,\n slack_oauth_client_secret,\n auto_invite_domain,\n auto_invite_operator,\n auto_add,\n customer_id,\n plan,\n webhook,\n deploy_to,\n ai_config,\n error_handler,\n error_handler_extra_args,\n error_handler_muted_on_cancel,\n large_file_storage,\n datatable,\n ducklake,\n git_sync,\n deploy_ui,\n default_app,\n default_scripts,\n mute_critical_alerts,\n color,\n operator_settings,\n git_app_installations,\n auto_add_instance_groups,\n auto_add_instance_groups_roles\n FROM\n workspace_settings\n WHERE\n workspace_id = $1\n ",
"describe": {
"columns": [
{
@@ -25,151 +25,156 @@
},
{
"ordinal": 4,
"name": "teams_team_guid",
"type_info": "Text"
},
{
"ordinal": 5,
"name": "slack_name",
"type_info": "Varchar"
},
{
"ordinal": 5,
"ordinal": 6,
"name": "slack_command_script",
"type_info": "Varchar"
},
{
"ordinal": 6,
"ordinal": 7,
"name": "teams_command_script",
"type_info": "Text"
},
{
"ordinal": 7,
"ordinal": 8,
"name": "slack_email",
"type_info": "Varchar"
},
{
"ordinal": 8,
"ordinal": 9,
"name": "slack_oauth_client_id",
"type_info": "Varchar"
},
{
"ordinal": 9,
"ordinal": 10,
"name": "slack_oauth_client_secret",
"type_info": "Varchar"
},
{
"ordinal": 10,
"ordinal": 11,
"name": "auto_invite_domain",
"type_info": "Varchar"
},
{
"ordinal": 11,
"ordinal": 12,
"name": "auto_invite_operator",
"type_info": "Bool"
},
{
"ordinal": 12,
"ordinal": 13,
"name": "auto_add",
"type_info": "Bool"
},
{
"ordinal": 13,
"ordinal": 14,
"name": "customer_id",
"type_info": "Varchar"
},
{
"ordinal": 14,
"ordinal": 15,
"name": "plan",
"type_info": "Varchar"
},
{
"ordinal": 15,
"ordinal": 16,
"name": "webhook",
"type_info": "Text"
},
{
"ordinal": 16,
"ordinal": 17,
"name": "deploy_to",
"type_info": "Varchar"
},
{
"ordinal": 17,
"ordinal": 18,
"name": "ai_config",
"type_info": "Jsonb"
},
{
"ordinal": 18,
"ordinal": 19,
"name": "error_handler",
"type_info": "Varchar"
},
{
"ordinal": 19,
"ordinal": 20,
"name": "error_handler_extra_args",
"type_info": "Json"
},
{
"ordinal": 20,
"ordinal": 21,
"name": "error_handler_muted_on_cancel",
"type_info": "Bool"
},
{
"ordinal": 21,
"ordinal": 22,
"name": "large_file_storage",
"type_info": "Jsonb"
},
{
"ordinal": 22,
"ordinal": 23,
"name": "datatable",
"type_info": "Jsonb"
},
{
"ordinal": 23,
"ordinal": 24,
"name": "ducklake",
"type_info": "Jsonb"
},
{
"ordinal": 24,
"ordinal": 25,
"name": "git_sync",
"type_info": "Jsonb"
},
{
"ordinal": 25,
"ordinal": 26,
"name": "deploy_ui",
"type_info": "Jsonb"
},
{
"ordinal": 26,
"ordinal": 27,
"name": "default_app",
"type_info": "Varchar"
},
{
"ordinal": 27,
"ordinal": 28,
"name": "default_scripts",
"type_info": "Jsonb"
},
{
"ordinal": 28,
"ordinal": 29,
"name": "mute_critical_alerts",
"type_info": "Bool"
},
{
"ordinal": 29,
"ordinal": 30,
"name": "color",
"type_info": "Varchar"
},
{
"ordinal": 30,
"ordinal": 31,
"name": "operator_settings",
"type_info": "Jsonb"
},
{
"ordinal": 31,
"ordinal": 32,
"name": "git_app_installations",
"type_info": "Jsonb"
},
{
"ordinal": 32,
"ordinal": 33,
"name": "auto_add_instance_groups",
"type_info": "TextArray"
},
{
"ordinal": 33,
"ordinal": 34,
"name": "auto_add_instance_groups_roles",
"type_info": "Jsonb"
}
@@ -187,6 +192,7 @@
true,
true,
true,
true,
false,
true,
true,
@@ -216,5 +222,5 @@
true
]
},
"hash": "95fa60eb45228ff289655fc676991f4e90d237799f6817f292eb1391694164c7"
"hash": "3c53de373b9f1034b5f43002bf4715e12ddc641f4ca52efe0335719fa9461bb0"
}

View File

@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO runnable_settings (hash, debouncing_settings, concurrency_settings)\n VALUES ($1, $2, $3)\n ON CONFLICT (hash)\n DO NOTHING",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Int8",
"Int8",
"Int8"
]
},
"nullable": []
},
"hash": "451d9cde90d14071e21ffb5f615052b7ba7fc315fc301ed5c0ff50d9a3ab0d4a"
}

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id FROM workspace WHERE parent_workspace_id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false
]
},
"hash": "4d04e436d46530f2f4c9dbc5c6c472581b9d90d330957539c0019ec2e7d9a68a"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT path FROM app\n WHERE workspace_id = $1 AND path = ANY($2)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"TextArray"
]
},
"nullable": [
false
]
},
"hash": "50ef2eac143273c7b7b7414b9be3c89130c709a57aca712e2af996bd4a4fa101"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT path FROM resource\n WHERE workspace_id = $1 AND path = ANY($2)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"TextArray"
]
},
"nullable": [
false
]
},
"hash": "5169d793f3820095839bb69d6c7c8cbea692ef6c1a838b970f382ee72db492d3"
}

View File

@@ -1,16 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n UPDATE workspace_settings\n SET teams_team_id = $1, teams_team_name = $2\n WHERE workspace_id = $3\n AND NOT EXISTS (\n SELECT 1 FROM workspace_settings\n WHERE teams_team_id = $1 AND workspace_id <> $2\n )\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "551c78392919e18019bb0a4344fb1bd45853bf5b72e0ab991e0e61fedcfb42fc"
}

View File

@@ -1,16 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE instance_group SET scim_display_name = $1, name = $2 where id = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "574d9a2bb6eceb62f3d2c2be3f18b29ef8bba3d6da1b1e21f2ca307ccbebee89"
}

View File

@@ -15,7 +15,7 @@
]
},
"nullable": [
true
null
]
},
"hash": "5a219a2532517869578c4504ff3153c43903f929ae5d62fbba12610f89c36d55"

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT \n v2_job_queue.workspace_id,\n v2_job_queue.id,\n v2_job.args as \"args: sqlx::types::Json<HashMap<String, Box<RawValue>>>\",\n v2_job.parent_job,\n v2_job.created_by,\n v2_job_queue.started_at,\n scheduled_for,\n runnable_path,\n kind as \"kind: JobKind\",\n runnable_id as \"runnable_id: ScriptHash\",\n canceled_reason,\n canceled_by,\n permissioned_as,\n permissioned_as_email,\n flow_status as \"flow_status: sqlx::types::Json<Box<RawValue>>\",\n v2_job.tag,\n script_lang as \"script_lang: ScriptLang\",\n same_worker,\n pre_run_error,\n concurrent_limit,\n concurrency_time_window_s,\n flow_innermost_root_job,\n root_job,\n timeout,\n flow_step_id,\n cache_ttl,\n cache_ignore_s3_path,\n v2_job_queue.priority,\n preprocessed,\n script_entrypoint_override,\n trigger,\n trigger_kind as \"trigger_kind: JobTriggerKind\",\n visible_to_owner,\n NULL as permissioned_as_end_user_email\n FROM v2_job_queue INNER JOIN v2_job ON v2_job.id = v2_job_queue.id LEFT JOIN v2_job_status ON v2_job_status.id = v2_job_queue.id WHERE v2_job_queue.id = $1",
"query": "SELECT \n v2_job_queue.workspace_id,\n v2_job_queue.id,\n v2_job.args as \"args: sqlx::types::Json<HashMap<String, Box<RawValue>>>\",\n v2_job.parent_job,\n v2_job.created_by,\n v2_job_queue.started_at,\n v2_job_queue.runnable_settings_handle,\n scheduled_for,\n runnable_path,\n kind as \"kind: JobKind\",\n runnable_id as \"runnable_id: ScriptHash\",\n canceled_reason,\n canceled_by,\n permissioned_as,\n permissioned_as_email,\n flow_status as \"flow_status: sqlx::types::Json<Box<RawValue>>\",\n v2_job.tag,\n script_lang as \"script_lang: ScriptLang\",\n same_worker,\n pre_run_error,\n concurrent_limit,\n concurrency_time_window_s,\n flow_innermost_root_job,\n root_job,\n timeout,\n flow_step_id,\n cache_ttl,\n cache_ignore_s3_path,\n v2_job_queue.priority,\n preprocessed,\n script_entrypoint_override,\n trigger,\n trigger_kind as \"trigger_kind: JobTriggerKind\",\n visible_to_owner,\n NULL as permissioned_as_end_user_email\n FROM v2_job_queue INNER JOIN v2_job ON v2_job.id = v2_job_queue.id LEFT JOIN v2_job_status ON v2_job_status.id = v2_job_queue.id WHERE v2_job_queue.id = $1",
"describe": {
"columns": [
{
@@ -35,16 +35,21 @@
},
{
"ordinal": 6,
"name": "runnable_settings_handle",
"type_info": "Int8"
},
{
"ordinal": 7,
"name": "scheduled_for",
"type_info": "Timestamptz"
},
{
"ordinal": 7,
"ordinal": 8,
"name": "runnable_path",
"type_info": "Varchar"
},
{
"ordinal": 8,
"ordinal": 9,
"name": "kind: JobKind",
"type_info": {
"Custom": {
@@ -79,42 +84,42 @@
}
},
{
"ordinal": 9,
"ordinal": 10,
"name": "runnable_id: ScriptHash",
"type_info": "Int8"
},
{
"ordinal": 10,
"ordinal": 11,
"name": "canceled_reason",
"type_info": "Text"
},
{
"ordinal": 11,
"ordinal": 12,
"name": "canceled_by",
"type_info": "Varchar"
},
{
"ordinal": 12,
"ordinal": 13,
"name": "permissioned_as",
"type_info": "Varchar"
},
{
"ordinal": 13,
"ordinal": 14,
"name": "permissioned_as_email",
"type_info": "Varchar"
},
{
"ordinal": 14,
"ordinal": 15,
"name": "flow_status: sqlx::types::Json<Box<RawValue>>",
"type_info": "Jsonb"
},
{
"ordinal": 15,
"ordinal": 16,
"name": "tag",
"type_info": "Varchar"
},
{
"ordinal": 16,
"ordinal": 17,
"name": "script_lang: ScriptLang",
"type_info": {
"Custom": {
@@ -150,77 +155,77 @@
}
},
{
"ordinal": 17,
"ordinal": 18,
"name": "same_worker",
"type_info": "Bool"
},
{
"ordinal": 18,
"ordinal": 19,
"name": "pre_run_error",
"type_info": "Text"
},
{
"ordinal": 19,
"ordinal": 20,
"name": "concurrent_limit",
"type_info": "Int4"
},
{
"ordinal": 20,
"ordinal": 21,
"name": "concurrency_time_window_s",
"type_info": "Int4"
},
{
"ordinal": 21,
"ordinal": 22,
"name": "flow_innermost_root_job",
"type_info": "Uuid"
},
{
"ordinal": 22,
"ordinal": 23,
"name": "root_job",
"type_info": "Uuid"
},
{
"ordinal": 23,
"ordinal": 24,
"name": "timeout",
"type_info": "Int4"
},
{
"ordinal": 24,
"ordinal": 25,
"name": "flow_step_id",
"type_info": "Varchar"
},
{
"ordinal": 25,
"ordinal": 26,
"name": "cache_ttl",
"type_info": "Int4"
},
{
"ordinal": 26,
"ordinal": 27,
"name": "cache_ignore_s3_path",
"type_info": "Bool"
},
{
"ordinal": 27,
"ordinal": 28,
"name": "priority",
"type_info": "Int2"
},
{
"ordinal": 28,
"ordinal": 29,
"name": "preprocessed",
"type_info": "Bool"
},
{
"ordinal": 29,
"ordinal": 30,
"name": "script_entrypoint_override",
"type_info": "Varchar"
},
{
"ordinal": 30,
"ordinal": 31,
"name": "trigger",
"type_info": "Varchar"
},
{
"ordinal": 31,
"ordinal": 32,
"name": "trigger_kind: JobTriggerKind",
"type_info": {
"Custom": {
@@ -246,12 +251,12 @@
}
},
{
"ordinal": 32,
"ordinal": 33,
"name": "visible_to_owner",
"type_info": "Bool"
},
{
"ordinal": 33,
"ordinal": 34,
"name": "permissioned_as_end_user_email",
"type_info": "Text"
}
@@ -268,6 +273,7 @@
true,
false,
true,
true,
false,
true,
false,
@@ -298,5 +304,5 @@
null
]
},
"hash": "6c97ab28ab47b75fb3ff39ea70fa3627f08b61bbd33aecb9ea816f8f78a04ec5"
"hash": "5bf200f2c8db25ddf231b564503c6c70f7f3958564a79bb0c6b3863b1ebb0cbf"
}

View File

@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO script\n (workspace_id, hash, path, parent_hashes, summary, description, content, created_by, schema, is_template, extra_perms, lock, language, kind, tag, draft_only, envs, concurrent_limit, concurrency_time_window_s, cache_ttl, cache_ignore_s3_path, dedicated_worker, ws_error_handler_muted, priority, restart_unless_cancelled, delete_after_use, timeout, concurrency_key, visible_to_runner_only, no_main_func, codebase, has_preprocessor, on_behalf_of_email, schema_validation, assets, debounce_key, debounce_delay_s, runnable_settings_handle)\n\n SELECT workspace_id, $1, path, array_prepend($2::bigint, COALESCE(parent_hashes, '{}'::bigint[])), summary, description, content, created_by, schema, is_template, extra_perms, NULL, language, kind, tag, draft_only, envs, concurrent_limit, concurrency_time_window_s, cache_ttl, cache_ignore_s3_path, dedicated_worker, ws_error_handler_muted, priority, restart_unless_cancelled, delete_after_use, timeout, concurrency_key, visible_to_runner_only, no_main_func, codebase, has_preprocessor, on_behalf_of_email, schema_validation, assets, debounce_key, debounce_delay_s, runnable_settings_handle\n\n FROM script WHERE hash = $2 AND workspace_id = $3;\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Int8",
"Int8",
"Text"
]
},
"nullable": []
},
"hash": "5c056ad6cc8967393729288437205c605a24118021fdb2b21b6b61695dc4ff28"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE workspace_settings\n SET teams_team_id = null, teams_team_name = null WHERE workspace_id = $1",
"query": "UPDATE workspace_settings\n SET teams_team_id = null, teams_team_name = null, teams_team_guid = null WHERE workspace_id = $1",
"describe": {
"columns": [],
"parameters": {
@@ -10,5 +10,5 @@
},
"nullable": []
},
"hash": "23c37d36e16251763fabf194e41de63612a7506cc0671b0eb83e528c1c839db4"
"hash": "62a625902ab1507f198bc9b12f2fea8398ec3905699ebf0e28cdfc85c0de4615"
}

View File

@@ -1,15 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO password (email, login_type, verified, username) VALUES ($1, 'saml', true, $2) ON CONFLICT DO NOTHING",
"query": "INSERT INTO password (email, login_type, verified, username, name) VALUES ($1, 'saml', true, $2, $3) ON CONFLICT DO NOTHING",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar"
]
},
"nullable": []
},
"hash": "a59aac0bc593d99aedd14fd9606f11191590a12ea98a671e172b867306959884"
"hash": "638d3c2ba1198dce5b5b0e47df59a92ff8011e19fbefcc3960d6f0fe167e55b6"
}

View File

@@ -0,0 +1,35 @@
{
"db_name": "PostgreSQL",
"query": "SELECT app.summary, app.policy, app_version.value\n FROM app\n JOIN app_version\n ON app_version.id = app.versions[array_upper(app.versions, 1)]\n WHERE app.workspace_id = $1 AND app.path = $2 AND COALESCE(app.draft_only, false) = false",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "summary",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "policy",
"type_info": "Jsonb"
},
{
"ordinal": 2,
"name": "value",
"type_info": "Json"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
false,
false
]
},
"hash": "66a9cb11fcc4757a3b35154840b703b87daa538cf3a72007649bbb074f56131e"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO instance_group (name, scim_display_name, id, external_id) VALUES ($1, $2, $3, $4) ON CONFLICT DO NOTHING",
"query": "INSERT INTO instance_group (name, scim_display_name, id, external_id) VALUES ($1, $2, $3, $4)",
"describe": {
"columns": [],
"parameters": {
@@ -13,5 +13,5 @@
},
"nullable": []
},
"hash": "bd829646d08f68106211f97c75dce13b6fc7d35bbaf7f503dcc73ae48fd07489"
"hash": "7cb7dcf8b20deb59fb1c3d4ad0ca4f9a209ce0d80682182e56946392f800c24c"
}

View File

@@ -0,0 +1,35 @@
{
"db_name": "PostgreSQL",
"query": "SELECT value, description, resource_type\n FROM resource\n WHERE workspace_id = $1 AND path = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "value",
"type_info": "Jsonb"
},
{
"ordinal": 1,
"name": "description",
"type_info": "Text"
},
{
"ordinal": 2,
"name": "resource_type",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
true,
true,
false
]
},
"hash": "819c233915383e89af1bcf1a56c5f67c4e1fc217f216f609e36a9944a7807b33"
}

View File

@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_diff (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)\n SELECT $1, unnest($2::varchar[]), $3, $4, 0, 1, NULL\n ON CONFLICT (source_workspace_id, fork_workspace_id, path, kind)\n DO UPDATE SET\n behind = workspace_diff.behind + 1,\n has_changes = NULL",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"VarcharArray",
"Varchar",
"Varchar"
]
},
"nullable": []
},
"hash": "90a1797f8d5ef7f9b67557c1fb919d1165860c5daf0de76fb2ac2201de11553c"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT \n j.id, j.workspace_id, j.runnable_id AS \"runnable_id: ScriptHash\", q.scheduled_for, q.started_at, j.parent_job, j.flow_innermost_root_job, j.runnable_path, j.kind as \"kind!: JobKind\", j.permissioned_as, \n j.created_by, j.script_lang AS \"script_lang: ScriptLang\", j.permissioned_as_email, j.flow_step_id, j.trigger_kind AS \"trigger_kind: JobTriggerKind\", j.trigger, j.priority, j.concurrent_limit, j.tag, j.cache_ttl, q.cache_ignore_s3_path\n FROM v2_job j LEFT JOIN v2_job_queue q ON j.id = q.id\n WHERE j.id = $1 AND j.workspace_id = $2",
"query": "SELECT \n j.id, j.workspace_id, j.runnable_id AS \"runnable_id: ScriptHash\", q.scheduled_for, q.started_at, j.parent_job, j.flow_innermost_root_job, j.runnable_path, j.kind as \"kind!: JobKind\", j.permissioned_as, \n j.created_by, j.script_lang AS \"script_lang: ScriptLang\", j.permissioned_as_email, j.flow_step_id, j.trigger_kind AS \"trigger_kind: JobTriggerKind\", j.trigger, j.priority, j.concurrent_limit, j.tag, j.cache_ttl, q.cache_ignore_s3_path, q.runnable_settings_handle\n FROM v2_job j LEFT JOIN v2_job_queue q ON j.id = q.id\n WHERE j.id = $1 AND j.workspace_id = $2",
"describe": {
"columns": [
{
@@ -189,6 +189,11 @@
"ordinal": 20,
"name": "cache_ignore_s3_path",
"type_info": "Bool"
},
{
"ordinal": 21,
"name": "runnable_settings_handle",
"type_info": "Int8"
}
],
"parameters": {
@@ -218,8 +223,9 @@
true,
false,
true,
true,
true
]
},
"hash": "7b5ad10af2a9b34fa86429499ea24c0c09c6e7e9ebfa3af90035570133f7c579"
"hash": "a1745a4f525b251d2f5a602ab2b2ede46b4471e21b11f607573a844013911abe"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT COALESCE((SELECT MIN(started_at) as min_started_at\n FROM v2_job_queue INNER JOIN v2_job ON v2_job.id = v2_job_queue.id\n WHERE v2_job.runnable_path = $1 AND v2_job.kind != 'dependencies' AND v2_job_queue.running = true AND v2_job_queue.workspace_id = $2 AND v2_job_queue.canceled_by IS NULL AND v2_job.concurrent_limit > 0), $3) as min_started_at, now() AS now",
"query": "SELECT COALESCE((SELECT MIN(started_at) as min_started_at\n FROM v2_job_queue INNER JOIN v2_job ON v2_job.id = v2_job_queue.id LEFT JOIN runnable_settings rs ON rs.hash = v2_job_queue.runnable_settings_handle LEFT JOIN concurrency_settings cs ON cs.hash = rs.concurrency_settings\n WHERE v2_job.runnable_path = $1 AND v2_job.kind != 'dependencies' AND v2_job_queue.running = true AND v2_job_queue.workspace_id = $2 AND v2_job_queue.canceled_by IS NULL AND COALESCE(cs.concurrent_limit, v2_job.concurrent_limit) > 0), $3) as min_started_at, now() AS now",
"describe": {
"columns": [
{
@@ -26,5 +26,5 @@
null
]
},
"hash": "6b6f8f7b4a6b6e7e41a9da8b6dfdbcae842ff252cc355bd91aeeb5e26dcc74f3"
"hash": "a2e52f033120a3f0b64e0a5ba125df7ce0d25096a23f0b655846a1c07b41f620"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "select hash, tag, concurrency_key, concurrent_limit, concurrency_time_window_s, debounce_key, debounce_delay_s, cache_ttl, cache_ignore_s3_path, language as \"language: ScriptLang\", dedicated_worker, priority, timeout, on_behalf_of_email, created_by FROM script\n WHERE path = $1 AND workspace_id = $2 AND archived = false AND (lock IS NOT NULL OR $3 = false)\n ORDER BY created_at DESC LIMIT 1",
"query": "select hash, tag, concurrency_key, concurrent_limit, concurrency_time_window_s, debounce_key, debounce_delay_s, cache_ttl, cache_ignore_s3_path, runnable_settings_handle, language as \"language: ScriptLang\", dedicated_worker, priority, timeout, on_behalf_of_email, created_by FROM script\n WHERE path = $1 AND workspace_id = $2 AND archived = false AND (lock IS NOT NULL OR $3 = false)\n ORDER BY created_at DESC LIMIT 1",
"describe": {
"columns": [
{
@@ -50,6 +50,11 @@
},
{
"ordinal": 9,
"name": "runnable_settings_handle",
"type_info": "Int8"
},
{
"ordinal": 10,
"name": "language: ScriptLang",
"type_info": {
"Custom": {
@@ -85,27 +90,27 @@
}
},
{
"ordinal": 10,
"ordinal": 11,
"name": "dedicated_worker",
"type_info": "Bool"
},
{
"ordinal": 11,
"ordinal": 12,
"name": "priority",
"type_info": "Int2"
},
{
"ordinal": 12,
"ordinal": 13,
"name": "timeout",
"type_info": "Int4"
},
{
"ordinal": 13,
"ordinal": 14,
"name": "on_behalf_of_email",
"type_info": "Text"
},
{
"ordinal": 14,
"ordinal": 15,
"name": "created_by",
"type_info": "Varchar"
}
@@ -127,6 +132,7 @@
true,
true,
true,
true,
false,
true,
true,
@@ -135,5 +141,5 @@
false
]
},
"hash": "7f9b7ab9bec6a0f745273d0cd5602ceab46a7ec9fd225f7b9d16a2ddb9bad7b3"
"hash": "a33673ebc4d1eb4c3513987dbc43e2c80974598e1d9fe7203145bfc29928ba65"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE folder SET owners = array_remove(owners, $1::varchar) WHERE name = $2 AND workspace_id = $3 RETURNING name",
"query": "UPDATE folder SET owners = array_remove(owners, $1::varchar) WHERE name = $2 AND workspace_id = $3 AND $1 = ANY(owners) RETURNING name",
"describe": {
"columns": [
{
@@ -11,7 +11,7 @@
],
"parameters": {
"Left": [
"Varchar",
"Text",
"Text",
"Text"
]
@@ -20,5 +20,5 @@
false
]
},
"hash": "e500a422fa986faf5c612c08f4a139e10b25a3106551e7c92f43dcf9758a5ec7"
"hash": "a3ca0af5d84acba93776828c090a79bf6554180b9db4951b69f7070f769b2962"
}

View File

@@ -0,0 +1,59 @@
{
"db_name": "PostgreSQL",
"query": "SELECT hash, created_at, content, summary, description, lock, schema\n FROM script\n WHERE workspace_id = $1 AND path = $2 AND archived = false\n ORDER BY created_at DESC\n LIMIT 1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "hash",
"type_info": "Int8"
},
{
"ordinal": 1,
"name": "created_at",
"type_info": "Timestamptz"
},
{
"ordinal": 2,
"name": "content",
"type_info": "Text"
},
{
"ordinal": 3,
"name": "summary",
"type_info": "Text"
},
{
"ordinal": 4,
"name": "description",
"type_info": "Text"
},
{
"ordinal": 5,
"name": "lock",
"type_info": "Text"
},
{
"ordinal": 6,
"name": "schema",
"type_info": "Json"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
false,
false,
false,
false,
true,
true
]
},
"hash": "a6fe41e36e06c88e1387abaea17107ceab2c5b17258958bde03b3e492bb2790e"
}

View File

@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_diff (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)\n VALUES ($1, $2, $3, $4, 1, 0, NULL)\n ON CONFLICT (source_workspace_id, fork_workspace_id, path, kind)\n DO UPDATE SET\n ahead = workspace_diff.ahead + 1,\n has_changes = NULL",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar",
"Varchar"
]
},
"nullable": []
},
"hash": "ac3001bd72248efe36ae3c96af5450f0721f888d18475fae862d675d40779a30"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "\n UPDATE workspace_settings\n SET git_app_installations = (\n SELECT COALESCE(jsonb_agg(elem), '[]'::jsonb)\n FROM jsonb_array_elements(git_app_installations) AS elem\n WHERE (elem->>'installation_id')::bigint != $1\n )\n WHERE workspace_id = $2\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Int8",
"Text"
]
},
"nullable": []
},
"hash": "ad288f9f242f930fc571d04d5ba217f653437f60c890b6b73059fdaa7eb45302"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM workspace_diff WHERE has_changes = false AND (\n (source_workspace_id = $1 AND fork_workspace_id = $2)\n OR (source_workspace_id = $2 AND fork_workspace_id =$1)\n )",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": []
},
"hash": "af6aea925527c486e56375b6cd215b0e36938a5853285be485647bc8cc846973"
}

View File

@@ -0,0 +1,49 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id, changed_by, changed_at, change_type, member_affected\n FROM group_permission_history\n WHERE workspace_id = $1 AND group_name = $2\n ORDER BY id DESC\n LIMIT $3 OFFSET $4",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Int8"
},
{
"ordinal": 1,
"name": "changed_by",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "changed_at",
"type_info": "Timestamptz"
},
{
"ordinal": 3,
"name": "change_type",
"type_info": "Varchar"
},
{
"ordinal": 4,
"name": "member_affected",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Int8",
"Int8"
]
},
"nullable": [
false,
false,
false,
false,
true
]
},
"hash": "b11034489786626184e2f6b7f99469a945a4bbfe0e6a33c5a3e009dadd56d52c"
}

View File

@@ -0,0 +1,49 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id, changed_by, changed_at, change_type, affected\n FROM folder_permission_history\n WHERE workspace_id = $1 AND folder_name = $2\n ORDER BY id DESC\n LIMIT $3 OFFSET $4",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Int8"
},
{
"ordinal": 1,
"name": "changed_by",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "changed_at",
"type_info": "Timestamptz"
},
{
"ordinal": 3,
"name": "change_type",
"type_info": "Varchar"
},
{
"ordinal": 4,
"name": "affected",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Int8",
"Int8"
]
},
"nullable": [
false,
false,
false,
false,
true
]
},
"hash": "b24eeafc2fc26664ef38d15d3cb47ca19549bcd9372d7a61ec2e28d82897bacf"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id, q.workspace_id, j.runnable_id as \"runnable_id: ScriptHash\", scheduled_for, parent_job, flow_innermost_root_job, runnable_path, kind as \"kind: JobKind\", started_at, permissioned_as, created_by, script_lang as \"script_lang: ScriptLang\", \n permissioned_as_email, flow_step_id, trigger_kind as \"trigger_kind: JobTriggerKind\", trigger, q.priority, concurrent_limit, q.tag, cache_ttl, cache_ignore_s3_path, r.ping as last_ping, worker, memory_peak, running\n FROM v2_job_queue q JOIN v2_job j USING (id) LEFT JOIN v2_job_runtime r USING (id) LEFT JOIN v2_job_status s USING (id)\n WHERE j.id = $1",
"query": "SELECT\n id,\n q.runnable_settings_handle,\n q.workspace_id,\n j.runnable_id as \"runnable_id: ScriptHash\",\n scheduled_for,\n parent_job,\n flow_innermost_root_job,\n runnable_path,\n kind as \"kind: JobKind\",\n started_at,\n permissioned_as,\n created_by,\n script_lang as \"script_lang: ScriptLang\",\n permissioned_as_email,\n flow_step_id,\n trigger_kind as \"trigger_kind: JobTriggerKind\",\n trigger,\n q.priority,\n concurrent_limit,\n q.tag,\n cache_ttl,\n cache_ignore_s3_path,\n r.ping as last_ping,\n worker,\n memory_peak,\n running\n FROM v2_job_queue q\n JOIN v2_job j USING (id)\n LEFT JOIN v2_job_runtime r USING (id)\n LEFT JOIN v2_job_status s USING (id)\n WHERE j.id = $1",
"describe": {
"columns": [
{
@@ -10,36 +10,41 @@
},
{
"ordinal": 1,
"name": "runnable_settings_handle",
"type_info": "Int8"
},
{
"ordinal": 2,
"name": "workspace_id",
"type_info": "Varchar"
},
{
"ordinal": 2,
"ordinal": 3,
"name": "runnable_id: ScriptHash",
"type_info": "Int8"
},
{
"ordinal": 3,
"ordinal": 4,
"name": "scheduled_for",
"type_info": "Timestamptz"
},
{
"ordinal": 4,
"ordinal": 5,
"name": "parent_job",
"type_info": "Uuid"
},
{
"ordinal": 5,
"ordinal": 6,
"name": "flow_innermost_root_job",
"type_info": "Uuid"
},
{
"ordinal": 6,
"ordinal": 7,
"name": "runnable_path",
"type_info": "Varchar"
},
{
"ordinal": 7,
"ordinal": 8,
"name": "kind: JobKind",
"type_info": {
"Custom": {
@@ -74,22 +79,22 @@
}
},
{
"ordinal": 8,
"ordinal": 9,
"name": "started_at",
"type_info": "Timestamptz"
},
{
"ordinal": 9,
"ordinal": 10,
"name": "permissioned_as",
"type_info": "Varchar"
},
{
"ordinal": 10,
"ordinal": 11,
"name": "created_by",
"type_info": "Varchar"
},
{
"ordinal": 11,
"ordinal": 12,
"name": "script_lang: ScriptLang",
"type_info": {
"Custom": {
@@ -125,17 +130,17 @@
}
},
{
"ordinal": 12,
"ordinal": 13,
"name": "permissioned_as_email",
"type_info": "Varchar"
},
{
"ordinal": 13,
"ordinal": 14,
"name": "flow_step_id",
"type_info": "Varchar"
},
{
"ordinal": 14,
"ordinal": 15,
"name": "trigger_kind: JobTriggerKind",
"type_info": {
"Custom": {
@@ -161,52 +166,52 @@
}
},
{
"ordinal": 15,
"ordinal": 16,
"name": "trigger",
"type_info": "Varchar"
},
{
"ordinal": 16,
"ordinal": 17,
"name": "priority",
"type_info": "Int2"
},
{
"ordinal": 17,
"ordinal": 18,
"name": "concurrent_limit",
"type_info": "Int4"
},
{
"ordinal": 18,
"ordinal": 19,
"name": "tag",
"type_info": "Varchar"
},
{
"ordinal": 19,
"ordinal": 20,
"name": "cache_ttl",
"type_info": "Int4"
},
{
"ordinal": 20,
"ordinal": 21,
"name": "cache_ignore_s3_path",
"type_info": "Bool"
},
{
"ordinal": 21,
"ordinal": 22,
"name": "last_ping",
"type_info": "Timestamptz"
},
{
"ordinal": 22,
"ordinal": 23,
"name": "worker",
"type_info": "Varchar"
},
{
"ordinal": 23,
"ordinal": 24,
"name": "memory_peak",
"type_info": "Int4"
},
{
"ordinal": 24,
"ordinal": 25,
"name": "running",
"type_info": "Bool"
}
@@ -218,6 +223,7 @@
},
"nullable": [
false,
true,
false,
true,
false,
@@ -244,5 +250,5 @@
false
]
},
"hash": "a84e67035584bbdb02482026b9cc0808086c50f78947d43bb88628a481f41a1d"
"hash": "b3771b690c5966272b1f42c9965bb6a8f961c119516e4c33dc928cd3b4f4edbc"
}

View File

@@ -18,8 +18,8 @@
"Left": []
},
"nullable": [
false,
true
true,
false
]
},
"hash": "b3dbdfb50ee8118bdaed3164b210cb549a34b96554ae1872355b90304f5dcb76"

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO script (workspace_id, hash, path, parent_hashes, summary, description, content, created_by, schema, is_template, extra_perms, lock, language, kind, tag, draft_only, envs, concurrent_limit, concurrency_time_window_s, cache_ttl, dedicated_worker, ws_error_handler_muted, priority, restart_unless_cancelled, delete_after_use, timeout, concurrency_key, visible_to_runner_only, no_main_func, codebase, has_preprocessor, on_behalf_of_email, schema_validation, assets, debounce_key, debounce_delay_s, cache_ignore_s3_path) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9::text::json, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21, $22, $23, $24, $25, $26, $27, $28, $29, $30, $31, $32, $33, $34, $35, $36, $37)",
"query": "INSERT INTO script (workspace_id, hash, path, parent_hashes, summary, description, content, created_by, schema, is_template, extra_perms, lock, language, kind, tag, draft_only, envs, concurrent_limit, concurrency_time_window_s, cache_ttl, dedicated_worker, ws_error_handler_muted, priority, restart_unless_cancelled, delete_after_use, timeout, concurrency_key, visible_to_runner_only, no_main_func, codebase, has_preprocessor, on_behalf_of_email, schema_validation, assets, debounce_key, debounce_delay_s, cache_ignore_s3_path, runnable_settings_handle) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9::text::json, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21, $22, $23, $24, $25, $26, $27, $28, $29, $30, $31, $32, $33, $34, $35, $36, $37, $38)",
"describe": {
"columns": [],
"parameters": {
@@ -59,7 +59,9 @@
"failure",
"command",
"approval",
"preprocessor"
"preprocessor",
"schedule_handler_old",
"dynamic_skip"
]
}
}
@@ -86,10 +88,11 @@
"Jsonb",
"Varchar",
"Int4",
"Bool"
"Bool",
"Int8"
]
},
"nullable": []
},
"hash": "3d05d9d7e087eb6e1c14c2b8a20598581e6c7493ed99cb9ad1c2ee5d0b212d38"
"hash": "b4eb72b0274cbdce7490f63c36d0d16ee847294fadc138593a1baa417cbb3652"
}

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id FROM v2_job_queue WHERE workspace_id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Uuid"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false
]
},
"hash": "b9c520fdc29b17977bf1c0b30311c536ebea8a76b75cda188ec91fcc0fc274a7"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT path FROM flow\n WHERE workspace_id = $1 AND path = ANY($2) AND archived = false",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"TextArray"
]
},
"nullable": [
false
]
},
"hash": "c5897802334f09596a0297f9b9a1e24f3c6f8b1628965438a9360c073d70924e"
}

View File

@@ -0,0 +1,41 @@
{
"db_name": "PostgreSQL",
"query": "SELECT value, summary, description, schema\n FROM flow\n WHERE workspace_id = $1 AND path = $2 AND archived = false",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "value",
"type_info": "Jsonb"
},
{
"ordinal": 1,
"name": "summary",
"type_info": "Text"
},
{
"ordinal": 2,
"name": "description",
"type_info": "Text"
},
{
"ordinal": 3,
"name": "schema",
"type_info": "Json"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
false,
false,
true
]
},
"hash": "dc6ffc9946325e794f3a9d1cf621a9c07d211351195c399f6ff25c1dc7adb01a"
}

View File

@@ -0,0 +1,20 @@
{
"db_name": "PostgreSQL",
"query": "SELECT COUNT(*) FROM instance_group",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "count",
"type_info": "Int8"
}
],
"parameters": {
"Left": []
},
"nullable": [
null
]
},
"hash": "e1c1e25053ae4b1635780c7e472c9d86806fa8cc762786b7a29bd121837c8ebc"
}

View File

@@ -0,0 +1,28 @@
{
"db_name": "PostgreSQL",
"query": "SELECT concurrency_settings, debouncing_settings FROM runnable_settings WHERE hash = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "concurrency_settings",
"type_info": "Int8"
},
{
"ordinal": 1,
"name": "debouncing_settings",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Int8"
]
},
"nullable": [
true,
true
]
},
"hash": "ebbbd069e0f33be9609604025d159fe1ecbefc2e9c11f7c4900b7121d4367e01"
}

View File

@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "\n UPDATE workspace_settings\n SET teams_team_id = $1, teams_team_name = $2, teams_team_guid = $3\n WHERE workspace_id = $4\n AND NOT EXISTS (\n SELECT 1 FROM workspace_settings\n WHERE teams_team_id = $1 AND workspace_id <> $4\n )\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "f034f7b0118ad467c7399c5554eb916d5a9716ca0d638e3bc65509b476db378e"
}

View File

@@ -0,0 +1,18 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO folder_permission_history\n (workspace_id, folder_name, changed_by, change_type, affected)\n VALUES ($1, $2, $3, $4, $5)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar",
"Varchar",
"Varchar"
]
},
"nullable": []
},
"hash": "f1206b00c4b81e59943385646d653330efba9cd1e731621f7efc91f04567841f"
}

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id FROM instance_group WHERE name = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
true
]
},
"hash": "f2eb05fe3581772d985e2ace82706d824b208ee9a1cb65a85be389e14672620c"
}

View File

@@ -0,0 +1,18 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO group_permission_history\n (workspace_id, group_name, changed_by, change_type, member_affected)\n VALUES ($1, $2, $3, $4, $5)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar",
"Varchar",
"Varchar"
]
},
"nullable": []
},
"hash": "f95358255e55d68dd453d173e23ab3cb1f1c2ba5b1cfc78f706b0b014f045477"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT path FROM variable\n WHERE workspace_id = $1 AND path = ANY($2)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"TextArray"
]
},
"nullable": [
false
]
},
"hash": "ff24cfb1f603b4ea612e48cee5880fc516207c8a474446a1ee09e8324f85ba09"
}

170
backend/Cargo.lock generated
View File

@@ -559,6 +559,22 @@ dependencies = [
"syn 2.0.111",
]
[[package]]
name = "astral-tokio-tar"
version = "0.5.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ec179a06c1769b1e42e1e2cbe74c7dcdb3d6383c838454d063eaac5bbb7ebbe5"
dependencies = [
"filetime",
"futures-core",
"libc",
"portable-atomic",
"rustc-hash 2.1.1",
"tokio",
"tokio-stream",
"xattr",
]
[[package]]
name = "async-broadcast"
version = "0.7.2"
@@ -800,9 +816,9 @@ dependencies = [
[[package]]
name = "aws-lc-rs"
version = "1.15.1"
version = "1.15.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6b5ce75405893cd713f9ab8e297d8e438f624dde7d706108285f7e17a25a180f"
checksum = "6a88aab2464f1f25453baa7a07c84c5b7684e274054ba06817f382357f77a288"
dependencies = [
"aws-lc-sys",
"zeroize",
@@ -810,9 +826,9 @@ dependencies = [
[[package]]
name = "aws-lc-sys"
version = "0.34.0"
version = "0.35.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "179c3777a8b5e70e90ea426114ffc565b2c1a9f82f6c4a0c5a34aa6ef5e781b6"
checksum = "b45afffdee1e7c9126814751f88dddc747f41d91da16c9551a0f1e8a11e788a1"
dependencies = [
"cc",
"cmake",
@@ -1106,9 +1122,9 @@ dependencies = [
[[package]]
name = "aws-smithy-json"
version = "0.61.8"
version = "0.61.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a6864c190cbb8e30cf4b77b2c8f3b6dfffa697a09b7218d2f7cd3d4c4065a9f7"
checksum = "49fa1213db31ac95288d981476f78d05d9cbb0353d22cdf3472cc05bb02f6551"
dependencies = [
"aws-smithy-types",
]
@@ -1134,9 +1150,9 @@ dependencies = [
[[package]]
name = "aws-smithy-runtime"
version = "1.9.5"
version = "1.9.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a392db6c583ea4a912538afb86b7be7c5d8887d91604f50eb55c262ee1b4a5f5"
checksum = "65fda37911905ea4d3141a01364bc5509a0f32ae3f3b22d6e330c0abfb62d247"
dependencies = [
"aws-smithy-async",
"aws-smithy-http",
@@ -1793,9 +1809,9 @@ dependencies = [
[[package]]
name = "bumpalo"
version = "3.19.0"
version = "3.19.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "46c5e41b57b8bba42a04676d81cb89e9ee8e859a1a66f80a5a72e1cb76b34d43"
checksum = "5dd9dc738b7a8311c7ade152424974d8115f2cdad61e8dab8dac9f2362298510"
dependencies = [
"allocator-api2",
]
@@ -2008,9 +2024,9 @@ dependencies = [
[[package]]
name = "cc"
version = "1.2.49"
version = "1.2.50"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "90583009037521a116abf44494efecd645ba48b6622457080f080b85544e2215"
checksum = "9f50d563227a1c37cc0a263f64eca3334388c01c5e4c4861a9def205c614383c"
dependencies = [
"find-msvc-tools",
"jobserver",
@@ -2166,9 +2182,9 @@ dependencies = [
[[package]]
name = "cmake"
version = "0.1.54"
version = "0.1.57"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e7caa3f9de89ddbe2c607f4101924c5abec803763ae9534e4f4d7d8f84aa81f0"
checksum = "75443c44cd6b379beb8c5b45d85d0773baf31cce901fe7bb252f4eff3008ef7d"
dependencies = [
"cc",
]
@@ -5368,9 +5384,9 @@ checksum = "1d674e81391d1e1ab681a28d99df07927c6d4aa5b027d7da16ba32d1d21ecd99"
[[package]]
name = "flatbuffers"
version = "25.9.23"
version = "25.12.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "09b6620799e7340ebd9968d2e0708eb82cf1971e9a16821e2091b6d6e475eed5"
checksum = "35f6839d7b3b98adde531effaf34f0c2badc6f4735d26fe74709d8e513a96ef3"
dependencies = [
"bitflags 2.9.4",
"rustc_version 0.4.1",
@@ -7723,13 +7739,13 @@ dependencies = [
[[package]]
name = "libredox"
version = "0.1.10"
version = "0.1.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "416f7e718bdb06000964960ffa43b4335ad4012ae8b99060261aa4a8088d5ccb"
checksum = "df15f6eac291ed1cf25865b1ee60399f57e7c227e7f51bdbd4c5270396a9ed50"
dependencies = [
"bitflags 2.9.4",
"libc",
"redox_syscall 0.5.18",
"redox_syscall 0.6.0",
]
[[package]]
@@ -8623,9 +8639,9 @@ dependencies = [
[[package]]
name = "ntapi"
version = "0.4.1"
version = "0.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e8a3895c6391c39d7fe7ebc444a87eb2991b2a0bc718fdabd071eec617fc68e4"
checksum = "c70f219e21142367c70c0b30c6a9e3a14d55b4d12a204d897fbec83a0363f081"
dependencies = [
"winapi",
]
@@ -9801,9 +9817,9 @@ dependencies = [
[[package]]
name = "portable-atomic"
version = "1.11.1"
version = "1.12.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f84267b20a16ea918e43c6a88433c2d54fa145c92a811b5b047ccbe153674483"
checksum = "f59e70c4aef1e55797c2e8fd94a4f2a973fc972cfde0e0b05f683667b0cd39dd"
[[package]]
name = "postgres-native-tls"
@@ -10554,18 +10570,18 @@ dependencies = [
[[package]]
name = "redox_syscall"
version = "0.3.5"
version = "0.5.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "567664f262709473930a4bf9e51bf2ebf3348f2e748ccc50dea20646858f8f29"
checksum = "ed2bf2547551a7053d6fdfafda3f938979645c44812fbfcda098faae3f1a362d"
dependencies = [
"bitflags 1.3.2",
"bitflags 2.9.4",
]
[[package]]
name = "redox_syscall"
version = "0.5.18"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed2bf2547551a7053d6fdfafda3f938979645c44812fbfcda098faae3f1a362d"
checksum = "ec96166dafa0886eb81fe1c0a388bece180fbef2135f97c1e2cf8302e74b43b5"
dependencies = [
"bitflags 2.9.4",
]
@@ -11224,9 +11240,9 @@ dependencies = [
[[package]]
name = "rustls-pki-types"
version = "1.13.1"
version = "1.13.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "708c0f9d5f54ba0272468c1d306a52c495b31fa155e91bc25371e6df7996908c"
checksum = "21e6f2ab2928ca4291b86736a8bd920a277a399bba1589409d72154ff87c1282"
dependencies = [
"web-time",
"zeroize",
@@ -12627,9 +12643,9 @@ dependencies = [
[[package]]
name = "supports-hyperlinks"
version = "3.1.0"
version = "3.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "804f44ed3c63152de6a9f90acbea1a110441de43006ea51bcce8f436196a288b"
checksum = "e396b6523b11ccb83120b115a0b7366de372751aa6edf19844dfb13a6af97e91"
[[package]]
name = "supports-unicode"
@@ -13816,21 +13832,6 @@ dependencies = [
"tokio",
]
[[package]]
name = "tokio-tar"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9d5714c010ca3e5c27114c1cdeb9d14641ace49874aa5626d7149e47aedace75"
dependencies = [
"filetime",
"futures-core",
"libc",
"redox_syscall 0.3.5",
"tokio",
"tokio-stream",
"xattr",
]
[[package]]
name = "tokio-tungstenite"
version = "0.24.0"
@@ -13940,9 +13941,9 @@ dependencies = [
[[package]]
name = "toml_parser"
version = "1.0.4"
version = "1.0.6+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c0cbe268d35bdb4bb5a56a2de88d0ad0eb70af5384a99d648cd4b3d04039800e"
checksum = "a3198b4b0a8e11f09dd03e133c0280504d0801269e9afa46362ffde1cbeebf44"
dependencies = [
"winnow 0.7.14",
]
@@ -14076,9 +14077,9 @@ checksum = "8df9b6e13f2d32c91b9bd719c00d1958837bc7dec474d94952798cc8e69eeec3"
[[package]]
name = "tracing"
version = "0.1.43"
version = "0.1.44"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2d15d90a0b5c19378952d479dc858407149d7bb45a14de0142f6c534b16fc647"
checksum = "63e71662fa4b2a2c3a26f570f037eb95bb1f85397f3cd8076caed2f026a6d100"
dependencies = [
"log",
"pin-project-lite",
@@ -14111,9 +14112,9 @@ dependencies = [
[[package]]
name = "tracing-core"
version = "0.1.35"
version = "0.1.36"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7a04e24fab5c89c6a36eb8558c9656f30d81de51dfa4d3b45f26b21d61fa0a6c"
checksum = "db97caf9d906fbde555dd62fa95ddba9eecfd14cb388e4f491a66d74cd5fb79a"
dependencies = [
"once_cell",
"valuable",
@@ -15165,7 +15166,7 @@ checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
[[package]]
name = "windmill"
version = "1.591.3"
version = "1.596.0"
dependencies = [
"anyhow",
"aws-sdk-config",
@@ -15201,6 +15202,7 @@ dependencies = [
"sha1",
"sha2 0.10.9",
"size",
"sql-builder",
"sqlx",
"strum 0.27.2",
"systemstat",
@@ -15227,10 +15229,11 @@ dependencies = [
[[package]]
name = "windmill-api"
version = "1.591.3"
version = "1.596.0"
dependencies = [
"anyhow",
"argon2",
"astral-tokio-tar",
"async-nats",
"async-oauth2",
"async-recursion",
@@ -15238,6 +15241,7 @@ dependencies = [
"async-trait",
"async_zip",
"aws-config",
"aws-credential-types",
"aws-sdk-config",
"aws-sdk-sqs",
"aws-sdk-sso",
@@ -15320,7 +15324,6 @@ dependencies = [
"tokio-postgres 0.7.11",
"tokio-postgres 0.7.13",
"tokio-stream",
"tokio-tar",
"tokio-tungstenite",
"tokio-util",
"tonic",
@@ -15348,7 +15351,7 @@ dependencies = [
[[package]]
name = "windmill-api-client"
version = "1.591.3"
version = "1.596.0"
dependencies = [
"base64 0.22.1",
"chrono",
@@ -15363,7 +15366,7 @@ dependencies = [
[[package]]
name = "windmill-audit"
version = "1.591.3"
version = "1.596.0"
dependencies = [
"chrono",
"lazy_static",
@@ -15377,7 +15380,7 @@ dependencies = [
[[package]]
name = "windmill-autoscaling"
version = "1.591.3"
version = "1.596.0"
dependencies = [
"anyhow",
"axum",
@@ -15396,7 +15399,7 @@ dependencies = [
[[package]]
name = "windmill-common"
version = "1.591.3"
version = "1.596.0"
dependencies = [
"anyhow",
"async-recursion",
@@ -15434,6 +15437,7 @@ dependencies = [
"lazy_static",
"magic-crypt",
"mail-send",
"native-tls",
"object_store",
"once_cell",
"openidconnect",
@@ -15445,6 +15449,7 @@ dependencies = [
"pep440_rs",
"phf 0.11.3",
"pin-project-lite",
"postgres-native-tls 0.5.1",
"prometheus",
"quick_cache",
"rand 0.9.0",
@@ -15469,6 +15474,7 @@ dependencies = [
"thiserror 2.0.17",
"tikv-jemalloc-ctl",
"tokio",
"tokio-postgres 0.7.13",
"tokio-stream",
"tokio-util",
"tonic",
@@ -15489,7 +15495,7 @@ dependencies = [
[[package]]
name = "windmill-git-sync"
version = "1.591.3"
version = "1.596.0"
dependencies = [
"regex",
"serde",
@@ -15504,9 +15510,10 @@ dependencies = [
[[package]]
name = "windmill-indexer"
version = "1.591.3"
version = "1.596.0"
dependencies = [
"anyhow",
"astral-tokio-tar",
"bytes",
"chrono",
"const_format",
@@ -15520,7 +15527,6 @@ dependencies = [
"tantivy",
"tempfile",
"tokio",
"tokio-tar",
"tracing",
"uuid",
"windmill-common",
@@ -15528,7 +15534,7 @@ dependencies = [
[[package]]
name = "windmill-macros"
version = "1.591.3"
version = "1.596.0"
dependencies = [
"itertools 0.14.0",
"lazy_static",
@@ -15544,7 +15550,7 @@ dependencies = [
[[package]]
name = "windmill-parser"
version = "1.591.3"
version = "1.596.0"
dependencies = [
"convert_case 0.6.0",
"serde",
@@ -15553,7 +15559,7 @@ dependencies = [
[[package]]
name = "windmill-parser-bash"
version = "1.591.3"
version = "1.596.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -15565,7 +15571,7 @@ dependencies = [
[[package]]
name = "windmill-parser-csharp"
version = "1.591.3"
version = "1.596.0"
dependencies = [
"anyhow",
"serde_json",
@@ -15577,7 +15583,7 @@ dependencies = [
[[package]]
name = "windmill-parser-go"
version = "1.591.3"
version = "1.596.0"
dependencies = [
"anyhow",
"gosyn",
@@ -15589,7 +15595,7 @@ dependencies = [
[[package]]
name = "windmill-parser-graphql"
version = "1.591.3"
version = "1.596.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -15601,7 +15607,7 @@ dependencies = [
[[package]]
name = "windmill-parser-java"
version = "1.591.3"
version = "1.596.0"
dependencies = [
"anyhow",
"serde_json",
@@ -15613,7 +15619,7 @@ dependencies = [
[[package]]
name = "windmill-parser-nu"
version = "1.591.3"
version = "1.596.0"
dependencies = [
"anyhow",
"nu-parser",
@@ -15624,7 +15630,7 @@ dependencies = [
[[package]]
name = "windmill-parser-php"
version = "1.591.3"
version = "1.596.0"
dependencies = [
"anyhow",
"itertools 0.14.0",
@@ -15635,7 +15641,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py"
version = "1.591.3"
version = "1.596.0"
dependencies = [
"anyhow",
"itertools 0.14.0",
@@ -15648,7 +15654,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py-imports"
version = "1.591.3"
version = "1.596.0"
dependencies = [
"anyhow",
"async-recursion",
@@ -15672,7 +15678,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ruby"
version = "1.591.3"
version = "1.596.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -15686,7 +15692,7 @@ dependencies = [
[[package]]
name = "windmill-parser-rust"
version = "1.591.3"
version = "1.596.0"
dependencies = [
"anyhow",
"convert_case 0.6.0",
@@ -15703,7 +15709,7 @@ dependencies = [
[[package]]
name = "windmill-parser-sql"
version = "1.591.3"
version = "1.596.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -15717,7 +15723,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ts"
version = "1.591.3"
version = "1.596.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -15736,7 +15742,7 @@ dependencies = [
[[package]]
name = "windmill-parser-yaml"
version = "1.591.3"
version = "1.596.0"
dependencies = [
"anyhow",
"serde",
@@ -15747,7 +15753,7 @@ dependencies = [
[[package]]
name = "windmill-queue"
version = "1.591.3"
version = "1.596.0"
dependencies = [
"anyhow",
"async-recursion",
@@ -15784,7 +15790,7 @@ dependencies = [
[[package]]
name = "windmill-sql-datatype-parser-wasm"
version = "1.591.3"
version = "1.596.0"
dependencies = [
"wasm-bindgen",
"wasm-bindgen-test",
@@ -15794,7 +15800,7 @@ dependencies = [
[[package]]
name = "windmill-worker"
version = "1.591.3"
version = "1.596.0"
dependencies = [
"anyhow",
"async-once-cell",

View File

@@ -1,6 +1,6 @@
[package]
name = "windmill"
version = "1.591.3"
version = "1.596.0"
authors.workspace = true
edition.workspace = true
@@ -33,7 +33,7 @@ members = [
exclude = ["./windmill-duckdb-ffi-internal"]
[workspace.package]
version = "1.591.3"
version = "1.596.0"
authors = ["Ruben Fiszel <ruben@windmill.dev>"]
edition = "2021"
@@ -125,6 +125,7 @@ windmill-autoscaling = { workspace = true, optional = true }
futures.workspace = true
tracing.workspace = true
sqlx.workspace = true
sql-builder.workspace = true
rand.workspace = true
chrono.workspace = true
git-version.workspace = true
@@ -259,7 +260,7 @@ reqwest = { version = "=0.12.24", features = ["json", "stream", "gzip", "multipa
eventsource-stream = "0.2.3"
time = "^0"
serde_urlencoded = "^0"
tokio-tar = "^0"
astral-tokio-tar = "^0.5.6"
tempfile = "^3"
tokio-util = { version = "^0", features = ["io"] }
json-pointer = "^0"

View File

@@ -1 +1 @@
55e8296c3758b031e50d7ebdaef7c33666f02385
576abf6519d1aa12a2b989a58a123501206284fb

View File

@@ -43,7 +43,7 @@ def load_openapi_spec(file_path: str) -> Dict[str, Any]:
print(f"Error loading OpenAPI spec: {e}", file=sys.stderr)
sys.exit(1)
def extract_separate_schemas(parameters: List[Dict[str, Any]], request_body: Optional[Dict[str, Any]], spec: Dict[str, Any], required_fields: Optional[List[str]] = None) -> tuple:
def extract_separate_schemas(parameters: List[Dict[str, Any]], request_body: Optional[Dict[str, Any]], spec: Dict[str, Any], required_fields: Optional[List[str]] = None, base_path: str = "") -> tuple:
"""Extract separate schemas for path parameters, query parameters, and request body."""
path_params_schema = {
"type": "object",
@@ -63,16 +63,16 @@ def extract_separate_schemas(parameters: List[Dict[str, Any]], request_body: Opt
for param in parameters:
# Resolve $ref if present
if '$ref' in param:
param = resolve_schema_refs(param, spec)
param = resolve_schema_refs(param, spec, base_path)
param_name = param.get('name', '')
param_schema = param.get('schema', {'type': 'string'})
param_required = param.get('required', False)
param_description = param.get('description', '')
param_in = param.get('in', 'query')
# Resolve any refs in the parameter schema
param_schema = resolve_schema_refs(param_schema, spec)
param_schema = resolve_schema_refs(param_schema, spec, base_path)
# Add description if available
if param_description:
@@ -93,7 +93,7 @@ def extract_separate_schemas(parameters: List[Dict[str, Any]], request_body: Opt
# Process request body if present
if request_body:
body_schema = extract_request_body_schema(request_body, spec)
body_schema = extract_request_body_schema(request_body, spec, base_path)
# If we have required fields specified and a body schema, update the required array
if body_schema and required_fields:
@@ -115,68 +115,109 @@ def extract_separate_schemas(parameters: List[Dict[str, Any]], request_body: Opt
return (path_params_schema, query_params_schema, body_schema)
def resolve_ref(ref_path: str, spec: Dict[str, Any]) -> Optional[Dict[str, Any]]:
"""Resolve a $ref path to the actual schema definition."""
if not ref_path.startswith('#/'):
# Cache for loaded external files
_external_file_cache: Dict[str, Dict[str, Any]] = {}
def load_external_file(file_path: str, base_path: str) -> Optional[Dict[str, Any]]:
"""Load an external YAML file relative to the base path."""
if file_path in _external_file_cache:
return _external_file_cache[file_path]
try:
import yaml
from pathlib import Path
# Resolve the path relative to the base file
base_dir = Path(base_path).parent
full_path = (base_dir / file_path).resolve()
with open(full_path, 'r', encoding='utf-8') as f:
content = yaml.safe_load(f)
_external_file_cache[file_path] = content
return content
except Exception as e:
print(f"Warning: Could not load external file {file_path}: {e}", file=sys.stderr)
return None
def resolve_ref(ref_path: str, spec: Dict[str, Any], base_path: str = "") -> tuple:
"""Resolve a $ref path to the actual schema definition.
Handles both internal refs (#/...) and external file refs (file.yaml#/...).
Returns a tuple of (resolved_schema, resolved_spec) where resolved_spec is the spec
that should be used for resolving any nested refs within the resolved schema.
"""
# Check if this is an external file reference
if '#' in ref_path and not ref_path.startswith('#'):
# External file reference: "../../openflow.openapi.yaml#/components/schemas/Retry"
file_part, fragment = ref_path.split('#', 1)
external_spec = load_external_file(file_part, base_path)
if external_spec is None:
return None, spec
# Resolve the fragment within the external file, and return external_spec for nested refs
resolved, _ = resolve_ref('#' + fragment, external_spec, base_path)
return resolved, external_spec
if not ref_path.startswith('#/'):
return None, spec
# Remove the '#/' prefix and split by '/'
path_parts = ref_path[2:].split('/')
# Navigate through the spec following the path
current = spec
for part in path_parts:
if isinstance(current, dict) and part in current:
current = current[part]
else:
return None
return current if isinstance(current, dict) else None
return None, spec
def resolve_schema_refs(schema: Dict[str, Any], spec: Dict[str, Any]) -> Dict[str, Any]:
return (current if isinstance(current, dict) else None), spec
def resolve_schema_refs(schema: Dict[str, Any], spec: Dict[str, Any], base_path: str = "") -> Dict[str, Any]:
"""Recursively resolve all $ref references in a schema."""
if not isinstance(schema, dict):
return schema
# If this is a $ref, resolve it
if '$ref' in schema:
ref_path = schema['$ref']
resolved = resolve_ref(ref_path, spec)
resolved, resolved_spec = resolve_ref(ref_path, spec, base_path)
if resolved:
# Recursively resolve any refs in the resolved schema
return resolve_schema_refs(resolved, spec)
# Recursively resolve any refs in the resolved schema using the appropriate spec
return resolve_schema_refs(resolved, resolved_spec, base_path)
else:
print(f"Warning: Could not resolve $ref: {ref_path}")
return schema
# Recursively process all values in the schema
resolved_schema = {}
for key, value in schema.items():
if isinstance(value, dict):
resolved_schema[key] = resolve_schema_refs(value, spec)
resolved_schema[key] = resolve_schema_refs(value, spec, base_path)
elif isinstance(value, list):
resolved_schema[key] = [
resolve_schema_refs(item, spec) if isinstance(item, dict) else item
resolve_schema_refs(item, spec, base_path) if isinstance(item, dict) else item
for item in value
]
else:
resolved_schema[key] = value
return resolved_schema
def extract_request_body_schema(request_body: Dict[str, Any], spec: Dict[str, Any]) -> Optional[Dict[str, Any]]:
def extract_request_body_schema(request_body: Dict[str, Any], spec: Dict[str, Any], base_path: str = "") -> Optional[Dict[str, Any]]:
"""Extract request body schema from OpenAPI requestBody definition and resolve refs."""
if not request_body:
return None
content = request_body.get('content', {})
json_content = content.get('application/json', {})
schema = json_content.get('schema', {})
if schema:
# Resolve any $ref references in the schema
return resolve_schema_refs(schema, spec)
return resolve_schema_refs(schema, spec, base_path)
return None
def http_method_to_rust(method: str) -> str:
@@ -221,7 +262,7 @@ def find_mcp_tools(spec: Dict[str, Any]) -> List[Dict[str, Any]]:
return tools
def generate_typescript_code(tools: List[Dict[str, Any]], spec: Dict[str, Any]) -> str:
def generate_typescript_code(tools: List[Dict[str, Any]], spec: Dict[str, Any], base_path: str = "") -> str:
"""Generate TypeScript code with MCP endpoint tools."""
if not tools:
return """// Auto-generated MCP tools from OpenAPI specification
@@ -252,7 +293,7 @@ export const mcpEndpointTools: EndpointTool[] = [];
# Generate separate schemas
path_params_schema, query_params_schema, body_schema = extract_separate_schemas(
tool['parameters'], tool['requestBody'], spec, tool['required_fields']
tool['parameters'], tool['requestBody'], spec, tool['required_fields'], base_path
)
# Convert schemas to TypeScript - use 'as const' for better type inference
@@ -297,7 +338,7 @@ export const mcpEndpointTools: EndpointTool[] = [
return typescript_code
def generate_rust_code(tools: List[Dict[str, Any]], spec: Dict[str, Any]) -> str:
def generate_rust_code(tools: List[Dict[str, Any]], spec: Dict[str, Any], base_path: str = "") -> str:
"""Generate the complete Rust code with MCP tools."""
if not tools:
return """// No MCP tools found in the OpenAPI specification
@@ -320,7 +361,7 @@ pub fn all_tools() -> Vec<EndpointTool> {
# Generate separate schemas
path_params_schema, query_params_schema, body_schema = extract_separate_schemas(
tool['parameters'], tool['requestBody'], spec, tool['required_fields']
tool['parameters'], tool['requestBody'], spec, tool['required_fields'], base_path
)
path_params_rust = schema_to_rust_value(path_params_schema)
@@ -386,7 +427,7 @@ def main():
# Generate and write Rust code
print(f"Generating Rust code...")
rust_code = generate_rust_code(tools, spec)
rust_code = generate_rust_code(tools, spec, str(openapi_file))
print(f"Writing Rust code to: {rust_output_file}")
rust_output_file.parent.mkdir(parents=True, exist_ok=True)
@@ -395,7 +436,7 @@ def main():
# Generate and write TypeScript code
print(f"Generating TypeScript code...")
typescript_code = generate_typescript_code(tools, spec)
typescript_code = generate_typescript_code(tools, spec, str(openapi_file))
print(f"Writing TypeScript code to: {ts_output_file}")
ts_output_file.parent.mkdir(parents=True, exist_ok=True)

View File

@@ -0,0 +1,6 @@
-- Add down migration script here
DROP INDEX IF EXISTS idx_group_perm_history_workspace_group;
DROP TABLE IF EXISTS group_permission_history;
DROP INDEX IF EXISTS idx_folder_perm_history_workspace_folder;
DROP TABLE IF EXISTS folder_permission_history;

View File

@@ -0,0 +1,90 @@
-- Add up migration script here
-- Folder permission changes history
CREATE TABLE IF NOT EXISTS folder_permission_history (
id BIGSERIAL PRIMARY KEY,
workspace_id VARCHAR(50) NOT NULL,
folder_name VARCHAR(255) NOT NULL,
changed_by VARCHAR(50) NOT NULL,
changed_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(),
change_type VARCHAR(50) NOT NULL,
affected VARCHAR(100),
FOREIGN KEY (workspace_id, folder_name) REFERENCES folder(workspace_id, name) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_folder_perm_history_workspace_folder
ON folder_permission_history(workspace_id, folder_name, id DESC);
-- Group permission changes history
CREATE TABLE IF NOT EXISTS group_permission_history (
id BIGSERIAL PRIMARY KEY,
workspace_id VARCHAR(50) NOT NULL,
group_name VARCHAR(255) NOT NULL,
changed_by VARCHAR(50) NOT NULL,
changed_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(),
change_type VARCHAR(50) NOT NULL,
member_affected VARCHAR(100),
FOREIGN KEY (workspace_id, group_name) REFERENCES group_(workspace_id, name) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_group_perm_history_workspace_group
ON group_permission_history(workspace_id, group_name, id DESC);
GRANT ALL ON TABLE folder_permission_history TO windmill_user;
GRANT ALL ON TABLE group_permission_history TO windmill_user;
GRANT ALL ON TABLE folder_permission_history TO windmill_admin;
GRANT ALL ON TABLE group_permission_history TO windmill_admin;
-- Enable RLS on folder_permission_history
ALTER TABLE folder_permission_history ENABLE ROW LEVEL SECURITY;
-- Admin policies (windmill_admin can always do everything)
CREATE POLICY admin_all ON folder_permission_history FOR ALL TO windmill_admin USING (true) WITH CHECK (true);
CREATE POLICY admin_all ON group_permission_history FOR ALL TO windmill_admin USING (true) WITH CHECK (true);
-- Enable RLS on group_permission_history
ALTER TABLE group_permission_history ENABLE ROW LEVEL SECURITY;
-- RLS policies for folder_permission_history
-- Anyone can insert
CREATE POLICY allow_insert ON folder_permission_history FOR INSERT TO windmill_user WITH CHECK (true);
-- Select requires being in extra_perms (as user or via group)
CREATE POLICY see_extra_perms_user ON folder_permission_history FOR SELECT TO windmill_user
USING (EXISTS (
SELECT 1 FROM folder f
WHERE f.workspace_id = folder_permission_history.workspace_id
AND f.name = folder_permission_history.folder_name
AND f.extra_perms ? CONCAT('u/', current_setting('session.user'))
));
CREATE POLICY see_extra_perms_groups ON folder_permission_history FOR SELECT TO windmill_user
USING (EXISTS (
SELECT 1 FROM folder f
WHERE f.workspace_id = folder_permission_history.workspace_id
AND f.name = folder_permission_history.folder_name
AND f.extra_perms ?| regexp_split_to_array(current_setting('session.pgroups'), ',')::text[]
));
-- RLS policies for group_permission_history
-- Anyone can insert
CREATE POLICY allow_insert ON group_permission_history FOR INSERT TO windmill_user WITH CHECK (true);
-- Select requires being in extra_perms (as user or via group)
CREATE POLICY see_extra_perms_user ON group_permission_history FOR SELECT TO windmill_user
USING (EXISTS (
SELECT 1 FROM group_ g
WHERE g.workspace_id = group_permission_history.workspace_id
AND g.name = group_permission_history.group_name
AND (g.extra_perms ->> CONCAT('u/', current_setting('session.user')))::boolean
));
CREATE POLICY see_extra_perms_groups ON group_permission_history FOR SELECT TO windmill_user
USING (EXISTS (
SELECT 1 FROM group_ g, jsonb_each_text(g.extra_perms) f
WHERE g.workspace_id = group_permission_history.workspace_id
AND g.name = group_permission_history.group_name
AND SPLIT_PART(f.key, '/', 1) = 'g'
AND f.key = ANY(regexp_split_to_array(current_setting('session.pgroups'), ',')::text[])
AND f.value::boolean
));

View File

@@ -0,0 +1,3 @@
-- Add down migration script here
DROP TABLE IF EXISTS skip_workspace_diff_tally;
DROP TABLE IF EXISTS workspace_diff;

View File

@@ -0,0 +1,24 @@
-- Add up migration script here
CREATE TABLE workspace_diff (
source_workspace_id VARCHAR(50) NOT NULL,
fork_workspace_id VARCHAR(50) NOT NULL,
path VARCHAR(255) NOT NULL,
kind VARCHAR(50) NOT NULL,
ahead INTEGER NOT NULL DEFAULT 0,
behind INTEGER NOT NULL DEFAULT 0,
has_changes BOOLEAN DEFAULT NULL,
exists_in_source BOOLEAN DEFAULT NULL,
exists_in_fork BOOLEAN DEFAULT NULL,
PRIMARY KEY (source_workspace_id, fork_workspace_id, path, kind)
);
-- Create table to track workspaces that should be excluded from diff tallying
-- Old workspaces that are linked but have already diverged need to be skipped
CREATE TABLE skip_workspace_diff_tally (
workspace_id VARCHAR(50) PRIMARY KEY,
added_at TIMESTAMP NOT NULL DEFAULT NOW()
);
-- Populate with all existing workspaces to exclude them from new tallying logic
INSERT INTO skip_workspace_diff_tally (workspace_id)
SELECT id FROM workspace;

View File

@@ -1 +1,2 @@
-- Add down migration script here
ALTER TABLE email_trigger DROP COLUMN enabled;

View File

@@ -0,0 +1,11 @@
ALTER TABLE v2_job_queue
DROP COLUMN runnable_settings_handle;
ALTER TABLE script
DROP COLUMN runnable_settings_handle;
DROP TABLE IF EXISTS job_settings;
DROP TABLE IF EXISTS runnable_settings;
DROP TABLE IF EXISTS concurrency_settings;
DROP TABLE IF EXISTS debouncing_settings;

View File

@@ -0,0 +1,42 @@
CREATE TABLE IF NOT EXISTS concurrency_settings(
hash BIGINT PRIMARY KEY,
concurrency_key VARCHAR(255),
concurrent_limit INTEGER,
concurrency_time_window_s INTEGER
);
CREATE TABLE IF NOT EXISTS debouncing_settings(
hash BIGINT PRIMARY KEY,
debounce_key VARCHAR(255),
debounce_delay_s INTEGER,
max_total_debouncing_time INTEGER,
max_total_debounces_amount INTEGER,
debounce_args_to_accumulate TEXT[]
);
CREATE TABLE IF NOT EXISTS runnable_settings(
hash BIGINT PRIMARY KEY,
debouncing_settings BIGINT DEFAULT NULL,
concurrency_settings BIGINT DEFAULT NULL
);
CREATE TABLE IF NOT EXISTS job_settings(
job_id UUID PRIMARY KEY,
runnable_settings BIGINT DEFAULT NULL
);
ALTER TABLE script
ADD COLUMN runnable_settings_handle BIGINT DEFAULT NULL;
ALTER TABLE v2_job_queue
ADD COLUMN runnable_settings_handle BIGINT DEFAULT NULL;
GRANT ALL ON concurrency_settings TO windmill_admin;
GRANT ALL ON concurrency_settings TO windmill_user;
GRANT ALL ON debouncing_settings TO windmill_admin;
GRANT ALL ON debouncing_settings TO windmill_user;
GRANT ALL ON runnable_settings TO windmill_admin;
GRANT ALL ON runnable_settings TO windmill_user;
GRANT ALL ON job_settings TO windmill_admin;
GRANT ALL ON job_settings TO windmill_user;

View File

@@ -0,0 +1,38 @@
-- Grant CREATE privilege on all databases where custom_instance_user has CONNECT
-- This allows custom_instance_user to create schemas in databases it can already access
DO $$
DECLARE
db_record RECORD;
grant_command TEXT;
BEGIN
-- Find all databases where custom_instance_user has CONNECT privilege
-- We check if the datacl array contains an entry for custom_instance_user with 'c' (CONNECT) privilege
FOR db_record IN
SELECT d.datname
FROM pg_database d
WHERE d.datname NOT IN ('template0', 'template1') -- Skip template databases
AND d.datallowconn = true -- Only consider databases that allow connections
AND d.datacl IS NOT NULL -- Has ACL entries
AND EXISTS (
SELECT 1
FROM unnest(d.datacl) AS acl_entry
WHERE acl_entry::text LIKE 'custom_instance_user=c/%' -- 'c' is the privilege code for CONNECT
)
LOOP
BEGIN
-- Grant CREATE privilege on the database
EXECUTE format('GRANT CREATE ON DATABASE %I TO custom_instance_user', db_record.datname);
RAISE NOTICE 'Granted CREATE on database % to custom_instance_user', db_record.datname;
EXCEPTION
WHEN others THEN
RAISE NOTICE 'Failed to grant CREATE on database %: %', db_record.datname, SQLERRM;
END;
END LOOP;
RAISE NOTICE 'Completed granting CREATE privileges to custom_instance_user on all accessible databases';
EXCEPTION
WHEN others THEN
RAISE NOTICE 'Error in custom_instance_user CREATE privilege migration: %', SQLERRM;
-- Continue without failing the migration
END
$$;

View File

@@ -0,0 +1 @@
-- Add down migration script here

View File

@@ -0,0 +1,6 @@
-- Add up migration script here
GRANT ALL ON SEQUENCE folder_permission_history_id_seq TO windmill_user;
GRANT ALL ON SEQUENCE folder_permission_history_id_seq TO windmill_admin;
GRANT ALL ON SEQUENCE group_permission_history_id_seq TO windmill_user;
GRANT ALL ON SEQUENCE group_permission_history_id_seq TO windmill_admin;

View File

@@ -0,0 +1,2 @@
-- Drop flow_iterator_data table
DROP TABLE IF EXISTS flow_iterator_data;

View File

@@ -0,0 +1,9 @@
-- Create separate table for storing flow iterator data (itered arrays)
-- This avoids expensive JSONB_SET operations on large itered arrays during parallel loop execution
CREATE TABLE IF NOT EXISTS flow_iterator_data (
job_id UUID PRIMARY KEY REFERENCES v2_job_queue (id) ON DELETE CASCADE NOT NULL,
itered JSONB NOT NULL,
created_at TIMESTAMP WITH TIME ZONE DEFAULT now()
);
-- Index not needed beyond primary key since all lookups are by job_id

View File

@@ -0,0 +1 @@
ALTER TABLE workspace_settings DROP COLUMN teams_team_guid;

View File

@@ -0,0 +1,3 @@
-- Add teams_team_guid column to store the GUID (used for MS Graph API calls)
-- The existing teams_team_id column stores the internal_id (used for webhook matching)
ALTER TABLE workspace_settings ADD COLUMN teams_team_guid TEXT;

View File

@@ -3,11 +3,11 @@ use rustpython_parser::{ast::Suite, Parse};
use std::collections::HashMap;
use windmill_parser::asset_parser::{
asset_was_used, merge_assets, parse_asset_syntax, AssetKind, AssetUsageAccessType,
ParseAssetsResult,
ParseAssetsOutput, ParseAssetsResult,
};
use AssetUsageAccessType::*;
pub fn parse_assets(input: &str) -> anyhow::Result<Vec<ParseAssetsResult>> {
pub fn parse_assets(input: &str) -> anyhow::Result<ParseAssetsOutput> {
let ast = Suite::parse(input, "main.py")
.map_err(|e| anyhow::anyhow!("Error parsing code: {}", e.to_string()))?;
@@ -15,7 +15,7 @@ pub fn parse_assets(input: &str) -> anyhow::Result<Vec<ParseAssetsResult>> {
ast.into_iter()
.for_each(|stmt| assets_finder.visit_stmt(stmt));
for (kind, path) in assets_finder.var_identifiers.into_values() {
for (kind, path, _) in assets_finder.var_identifiers.into_values() {
// if a db = wmill.datatable() was never used (e.g db.query(...)),
// we still want to register the asset as unknown access type
if asset_was_used(&assets_finder.assets, (kind, &path)) == false {
@@ -25,12 +25,14 @@ pub fn parse_assets(input: &str) -> anyhow::Result<Vec<ParseAssetsResult>> {
}
}
Ok(merge_assets(assets_finder.assets))
Ok(ParseAssetsOutput { assets: merge_assets(assets_finder.assets), ..Default::default() })
}
type VarAssetName = String;
type VarAssetSchema = Option<String>;
struct AssetsFinder {
assets: Vec<ParseAssetsResult>,
var_identifiers: HashMap<String, (AssetKind, String)>,
var_identifiers: HashMap<String, (AssetKind, VarAssetName, VarAssetSchema)>,
}
impl Visitor for AssetsFinder {
@@ -44,7 +46,7 @@ impl Visitor for AssetsFinder {
// if a db = wmill.datatable() or similar was removed, but never used (e.g db.query(...)),
// we still want to register the asset as unknown access type
match removed {
Some((kind, path)) => {
Some((kind, path, _)) => {
if !asset_was_used(&self.assets, (kind, &path)) {
self.assets
.push(ParseAssetsResult { kind, access_type: None, path });
@@ -53,11 +55,11 @@ impl Visitor for AssetsFinder {
None => {}
}
if let Some((kind, name)) = self.extract_asset_from_call(&node.value) {
if let Some((kind, name, schema)) = self.extract_asset_from_call(&node.value) {
// Track target variable
let Ok(var_name) = expr_name.id.parse::<String>();
self.var_identifiers
.insert(var_name, (kind.clone(), name.clone()));
.insert(var_name, (kind.clone(), name.clone(), schema.clone()));
}
}
// Continue with generic visit to catch any other assets in the expression
@@ -107,7 +109,10 @@ impl Visitor for AssetsFinder {
impl AssetsFinder {
/// Extract asset info from calls like wmill.datatable('name'), wmill.ducklake('name'), etc.
fn extract_asset_from_call(&self, expr: &Expr) -> Option<(AssetKind, String)> {
fn extract_asset_from_call(
&self,
expr: &Expr,
) -> Option<(AssetKind, VarAssetName, VarAssetSchema)> {
let call = expr.as_call_expr()?;
// Check for wmill.datatable, wmill.ducklake pattern
@@ -143,10 +148,21 @@ impl AssetsFinder {
} else {
None
}
})
.unwrap_or_else(|| "main".to_string());
});
let (name, schema) = match name {
None => ("main".to_string(), None),
Some(name) => {
if let Some((name, s)) = name.split_once(':') {
let schema = Some(s.to_string());
let name = if name.is_empty() { "main" } else { name };
(name.to_string(), schema)
} else {
(name, None)
}
}
};
Some((kind, name))
Some((kind, name, schema))
}
fn visit_expr_call_inner(&mut self, node: &rustpython_ast::ExprCall) -> Result<(), ()> {
@@ -177,7 +193,7 @@ impl AssetsFinder {
if obj_name == "wmill" {
// Continue
} else if let Some((kind, ref path)) = self.var_identifiers.get(&obj_name) {
} else if let Some((kind, ref path, ref schema)) = self.var_identifiers.get(&obj_name) {
if ident == "query" {
let expr_name = node.args.get(0).or_else(|| {
node.keywords
@@ -198,8 +214,20 @@ impl AssetsFinder {
// We use the SQL parser to detect if it's a read or write query
match windmill_parser_sql::parse_assets(&sql) {
Ok(sql_assets) => {
self.assets.extend(sql_assets);
Ok(mut sql_assets) => {
if let Some(schema_name) = schema {
for asset in &mut sql_assets.assets {
if asset.kind == *kind && asset.path.starts_with(path.as_str()) {
asset.path = format!(
"{}/{}.{}",
path,
schema_name,
&asset.path[path.len() + 1..]
);
}
}
}
self.assets.extend(sql_assets.assets);
}
_ => {}
}
@@ -262,7 +290,7 @@ import wmill
def main():
wmill.load_s3_file('s3:///test.csv')
"#;
let s = parse_assets(input);
let s = parse_assets(input).map(|o| o.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
@@ -280,7 +308,7 @@ import wmill
def main():
db = wmill.datatable()
"#;
let s = parse_assets(input);
let s = parse_assets(input).map(|o| o.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
@@ -299,7 +327,7 @@ def main(x: int):
db = wmill.datatable('dt')
return db.query('SELECT * FROM friends WHERE age = $1', x).fetch()
"#;
let s = parse_assets(input);
let s = parse_assets(input).map(|o| o.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
@@ -320,7 +348,7 @@ def main(x: int):
db.query('SELECT * FROM friends WHERE age = $1', x).fetch_one()
db.query('SELECT * FROM analytics').fetch()
"#;
let s = parse_assets(input);
let s = parse_assets(input).map(|o| o.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![
@@ -352,7 +380,7 @@ def main():
def g():
db = wmill.ducklake('another2')
"#;
let s = parse_assets(input);
let s = parse_assets(input).map(|o| o.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![
@@ -384,7 +412,7 @@ def main():
def g():
db = wmill.ducklake()
"#;
let s = parse_assets(input);
let s = parse_assets(input).map(|o| o.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![
@@ -401,4 +429,80 @@ def g():
])
);
}
#[test]
fn test_py_asset_parser_datatable_with_schema() {
let input = r#"
import wmill
def main(x: int):
db = wmill.datatable('dt:public')
return db.query('SELECT * FROM friends WHERE age = $1', x).fetch()
"#;
let s = parse_assets(input).map(|o| o.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/public.friends".to_string(),
access_type: Some(R)
},])
);
}
#[test]
fn test_py_asset_parser_ducklake_with_schema() {
let input = r#"
import wmill
def main():
db = wmill.ducklake('lake1:analytics')
return db.query('SELECT * FROM metrics').fetch()
"#;
let s = parse_assets(input).map(|o| o.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "lake1/analytics.metrics".to_string(),
access_type: Some(R)
},])
);
}
#[test]
fn test_py_asset_parser_schema_with_write() {
let input = r#"
import wmill
def main(x: int):
db = wmill.datatable('dt:public')
db.query('INSERT INTO users VALUES ($1)', x).fetch()
return db.query('SELECT * FROM users').fetch()
"#;
let s = parse_assets(input).map(|o| o.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/public.users".to_string(),
access_type: Some(RW)
},])
);
}
#[test]
fn test_py_asset_parser_unused_datatable_with_schema() {
let input = r#"
import wmill
def main():
db = wmill.datatable('dt:public')
"#;
let s = parse_assets(input).map(|o| o.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt".to_string(),
access_type: None
},])
);
}
}

View File

@@ -15,7 +15,7 @@ use windmill_parser::{json_to_typ, Arg, MainArgSignature, ObjectType, Typ};
use rustpython_parser::{
ast::{
Constant, Expr, ExprConstant, ExprDict, ExprList, ExprName, Stmt, StmtFunctionDef, Suite,
Constant, Expr, ExprAttribute, ExprConstant, ExprDict, ExprList, ExprName, Stmt, StmtAssign, StmtClassDef, StmtFunctionDef, Suite,
},
Parse,
};
@@ -60,6 +60,166 @@ fn filter_non_main(code: &str, main_name: &str) -> String {
return filtered_code;
}
/// Data extracted from parsing the Python code
struct CodeMetadata {
enums: HashMap<String, EnumInfo>,
descriptions: HashMap<String, String>,
}
/// Information about an Enum class
struct EnumInfo {
values: Vec<String>,
members: HashMap<String, String>,
}
fn has_enum_keyword(code: &str) -> bool {
code.contains("Enum")
}
/// Extract only class and function definitions from code (prepass filtering)
fn filter_relevant_statements(code: &str) -> String {
let mut result = Vec::new();
let mut lines = code.lines().peekable();
while let Some(line) = lines.next() {
let trimmed = line.trim_start();
if trimmed.starts_with("class ") || trimmed.starts_with("def ") {
result.push(line);
let base_indent = line.len() - trimmed.len();
while let Some(&next_line) = lines.peek() {
let next_trimmed = next_line.trim_start();
let next_indent = next_line.len() - next_trimmed.len();
if next_trimmed.is_empty() || next_indent > base_indent {
result.push(lines.next().unwrap());
} else {
break;
}
}
}
}
result.join("\n")
}
/// Extract Enum definitions and docstring descriptions lazily.
/// Only parses AST if relevant keywords are present.
fn extract_code_metadata(code: &str, main_name: &str) -> CodeMetadata {
let mut enums = HashMap::new();
let mut descriptions = HashMap::new();
let has_enum = has_enum_keyword(code);
let has_docstring = code.contains("Args:");
if !has_enum && !has_docstring {
return CodeMetadata { enums, descriptions };
}
let filtered_code = filter_relevant_statements(code);
let ast = match Suite::parse(&filtered_code, "main.py") {
Ok(ast) => ast,
Err(_) => return CodeMetadata { enums, descriptions },
};
for stmt in ast {
match stmt {
Stmt::ClassDef(StmtClassDef { name, body, bases, .. }) if has_enum => {
let is_enum = bases.iter().any(|base| {
matches!(base, Expr::Name(ExprName { id, .. })
if id == "Enum" || id == "IntEnum" || id == "StrEnum"
|| id == "Flag" || id == "IntFlag")
});
if is_enum {
let mut values = Vec::new();
let mut members = HashMap::new();
for item in body {
if let Stmt::Assign(StmtAssign { targets, value, .. }) = item {
if let Some(Expr::Name(ExprName { id: target_name, .. })) = targets.first() {
if !target_name.starts_with('_') {
if let Expr::Constant(ExprConstant { value: Constant::Str(val), .. }) = value.as_ref() {
values.push(val.to_string());
members.insert(target_name.to_string(), val.to_string());
}
}
}
}
}
if !values.is_empty() {
enums.insert(name.to_string(), EnumInfo { values, members });
}
}
},
Stmt::FunctionDef(StmtFunctionDef { name: func_name, body, .. }) if has_docstring => {
if &func_name == main_name {
if let Some(Stmt::Expr(expr_stmt)) = body.first() {
if let Expr::Constant(ExprConstant { value: Constant::Str(docstring), .. }) = expr_stmt.value.as_ref() {
descriptions = parse_docstring_args(docstring);
}
}
}
},
_ => {}
}
}
CodeMetadata { enums, descriptions }
}
/// Parse docstring Args: section (format: "param_name (type): Description")
fn parse_docstring_args(docstring: &str) -> HashMap<String, String> {
let mut descriptions = HashMap::new();
let mut in_args_section = false;
let mut base_indent: Option<usize> = None;
for line in docstring.lines() {
let trimmed = line.trim();
if trimmed == "Args:" {
in_args_section = true;
base_indent = None;
continue;
}
if in_args_section {
if trimmed.is_empty() {
continue;
}
let indent = line.len() - line.trim_start().len();
if base_indent.is_none() && !trimmed.is_empty() {
base_indent = Some(indent);
}
if let Some(base) = base_indent {
if indent < base && trimmed.ends_with(':') {
break;
}
}
if let Some(colon_pos) = trimmed.find(':') {
let before_colon = &trimmed[..colon_pos];
let description = trimmed[colon_pos + 1..].trim();
if let Some(paren_pos) = before_colon.find('(') {
let param_name = before_colon[..paren_pos].trim();
descriptions.insert(param_name.to_string(), description.to_string());
} else {
descriptions.insert(before_colon.trim().to_string(), description.to_string());
}
}
}
}
descriptions
}
/// skip_params is a micro optimization for when we just want to find the main
/// function without parsing all the params.
pub fn parse_python_signature(
@@ -91,27 +251,61 @@ pub fn parse_python_signature(
if !skip_params && params.is_some() {
let params = params.unwrap();
//println!("{:?}", params);
let def_arg_start = params.args.len() - params.defaults().count();
// Two-pass approach for lazy metadata extraction:
// Pass 1: Parse types without enum info to determine if metadata is needed
// Pass 2: Re-parse unknown types with metadata only if necessary
// This ensures zero overhead for scripts without enums/docstrings
let empty_enums = HashMap::new();
let args_first_pass: Vec<_> = params
.args
.iter()
.enumerate()
.map(|(i, x)| {
let arg_name = x.as_arg().arg.to_string();
let (typ, has_default) = x
.as_arg()
.annotation
.as_ref()
.map_or((Typ::Unknown, false), |e| parse_expr(e, &empty_enums));
(i, arg_name, typ, has_default)
})
.collect();
// Determine if we need to extract metadata from the code
let has_potential_enums = args_first_pass
.iter()
.any(|(_, _, typ, _)| matches!(typ, Typ::Resource(_)));
let metadata = if has_potential_enums || code.contains("Args:") {
extract_code_metadata(code, &main_name)
} else {
CodeMetadata {
enums: HashMap::new(),
descriptions: HashMap::new(),
}
};
// Build final args, re-parsing Resource types as enums if metadata was extracted
Ok(MainArgSignature {
star_args: params.vararg.is_some(),
star_kwargs: params.kwarg.is_some(),
args: params
.args
.iter()
.enumerate()
.map(|(i, x)| {
let (mut typ, has_default) = x
.as_arg()
.annotation
.as_ref()
.map_or((Typ::Unknown, false), |e| parse_expr(e));
args: args_first_pass
.into_iter()
.map(|(i, arg_name, mut typ, mut has_default)| {
if matches!(typ, Typ::Resource(_)) && !metadata.enums.is_empty() {
if let Some(annotation) = params.args[i].as_arg().annotation.as_ref() {
(typ, has_default) = parse_expr(annotation, &metadata.enums);
}
}
let default = if i >= def_arg_start {
params
.defaults()
.nth(i - def_arg_start)
.map(to_value)
.map(|expr| to_value(expr, &metadata.enums))
.flatten()
} else {
None
@@ -140,8 +334,8 @@ pub fn parse_python_signature(
}
Arg {
otyp: None,
name: x.as_arg().arg.to_string(),
otyp: metadata.descriptions.get(&arg_name).map(|d| d.to_string()),
name: arg_name,
typ,
has_default: has_default || default.is_some(),
default,
@@ -163,15 +357,15 @@ pub fn parse_python_signature(
}
}
fn parse_expr(e: &Box<Expr>) -> (Typ, bool) {
fn parse_expr(e: &Box<Expr>, enums: &HashMap<String, EnumInfo>) -> (Typ, bool) {
match e.as_ref() {
Expr::Name(ExprName { id, .. }) => (parse_typ(id.as_ref()), false),
Expr::Name(ExprName { id, .. }) => (parse_typ(id.as_ref(), enums), false),
Expr::Attribute(x) => {
if x.value
.as_name_expr()
.is_some_and(|x| x.id.as_str() == "wmill")
{
(parse_typ(x.attr.as_str()), false)
(parse_typ(x.attr.as_str(), enums), false)
} else {
(Typ::Unknown, false)
}
@@ -181,7 +375,7 @@ fn parse_expr(e: &Box<Expr>) -> (Typ, bool) {
x.right.as_ref(),
Expr::Constant(ExprConstant { value: Constant::None, .. })
) {
(parse_expr(&x.left).0, true)
(parse_expr(&x.left, enums).0, true)
} else {
(Typ::Unknown, false)
}
@@ -210,8 +404,8 @@ fn parse_expr(e: &Box<Expr>) -> (Typ, bool) {
};
(Typ::Str(values), false)
}
"List" | "list" => (Typ::List(Box::new(parse_expr(&x.slice).0)), false),
"Optional" => (parse_expr(&x.slice).0, true),
"List" | "list" => (Typ::List(Box::new(parse_expr(&x.slice, enums).0)), false),
"Optional" => (parse_expr(&x.slice, enums).0, true),
_ => (Typ::Unknown, false),
},
_ => (Typ::Unknown, false),
@@ -220,7 +414,11 @@ fn parse_expr(e: &Box<Expr>) -> (Typ, bool) {
}
}
fn parse_typ(id: &str) -> Typ {
fn parse_typ(id: &str, enums: &HashMap<String, EnumInfo>) -> Typ {
if let Some(enum_info) = enums.get(id) {
return Typ::Str(Some(enum_info.values.clone()));
}
match id {
"str" => Typ::Str(None),
"float" => Typ::Float,
@@ -249,7 +447,7 @@ fn map_resource_name(x: &str) -> String {
}
}
fn to_value<R>(et: &Expr<R>) -> Option<serde_json::Value> {
fn to_value<R>(et: &Expr<R>, enums: &HashMap<String, EnumInfo>) -> Option<serde_json::Value> {
match et {
Expr::Constant(ExprConstant { value, .. }) => Some(constant_to_value(value)),
Expr::Dict(ExprDict { keys, values, .. }) => {
@@ -259,22 +457,35 @@ fn to_value<R>(et: &Expr<R>) -> Option<serde_json::Value> {
.map(|(k, v)| {
let key = k
.as_ref()
.map(to_value)
.map(|e| to_value(e, enums))
.flatten()
.and_then(|x| match x {
serde_json::Value::String(s) => Some(s),
_ => None,
})
.unwrap_or_else(|| "no_key".to_string());
(key, to_value(&v))
(key, to_value(&v, enums))
})
.collect::<HashMap<String, _>>();
Some(json!(v))
}
Expr::List(ExprList { elts, .. }) => {
let v = elts.into_iter().map(|x| to_value(&x)).collect::<Vec<_>>();
let v = elts.into_iter().map(|x| to_value(&x, enums)).collect::<Vec<_>>();
Some(json!(v))
}
Expr::Attribute(ExprAttribute { value, attr, .. }) => {
// Handle Enum.MEMBER: returns enum value ("red") not member name ("RED")
if let Expr::Name(ExprName { id: enum_name, .. }) = value.as_ref() {
if let Some(enum_info) = enums.get(enum_name.as_str()) {
if let Some(enum_value) = enum_info.members.get(attr.as_str()) {
return Some(json!(enum_value));
}
}
Some(json!(attr.as_str()))
} else {
None
}
}
Expr::Call { .. } => Some(json!(FUNCTION_CALL)),
_ => None,
}
@@ -751,4 +962,35 @@ def main(a: str, b: Optional[str], c: str | None): return
Ok(())
}
#[test]
fn test_parse_python_sig_enum() -> anyhow::Result<()> {
let code = r#"
from enum import Enum
class Color(str, Enum):
RED = 'red'
GREEN = 'green'
BLUE = 'blue'
def main(color: Color = Color.RED):
"""
Test enum parsing
Args:
color (Color): Color selection from Color enum
"""
return {"color": color}
"#;
let result = parse_python_signature(code, None, false)?;
assert_eq!(result.args.len(), 1);
assert_eq!(result.args[0].name, "color");
assert_eq!(
result.args[0].typ,
Typ::Str(Some(vec!["red".to_string(), "green".to_string(), "blue".to_string()]))
);
assert_eq!(result.args[0].default, Some(json!("red")));
assert_eq!(result.args[0].otyp, Some("Color selection from Color enum".to_string()));
Ok(())
}
}

View File

@@ -10,11 +10,11 @@ use sqlparser::{
};
use windmill_parser::asset_parser::{
asset_was_used, merge_assets, parse_asset_syntax, AssetKind, AssetUsageAccessType,
ParseAssetsResult,
ParseAssetsOutput, ParseAssetsResult,
};
use AssetUsageAccessType::*;
pub fn parse_assets(input: &str) -> anyhow::Result<Vec<ParseAssetsResult>> {
pub fn parse_assets(input: &str) -> anyhow::Result<ParseAssetsOutput> {
let statements = Parser::parse_sql(&DuckDbDialect, input)?;
let mut collector = AssetCollector::new();
@@ -30,7 +30,7 @@ pub fn parse_assets(input: &str) -> anyhow::Result<Vec<ParseAssetsResult>> {
}
}
Ok(merge_assets(collector.assets))
Ok(ParseAssetsOutput { assets: merge_assets(collector.assets), ..Default::default() })
}
/// Visitor that collects S3 asset literals from SQL statements
@@ -58,19 +58,28 @@ impl AssetCollector {
// Or when we access 'b' and we did USE a;
fn get_associated_asset_from_obj_name(&self, name: &ObjectName) -> Option<ParseAssetsResult> {
let access_type = self.current_access_type_stack.last().copied();
if name.0.len() == 1 {
let ident = name.0.first()?.as_ident()?;
if ident.quote_style.is_some() {
return None;
}
let specific_table = &ident.value;
if let Some((kind, path)) = &self.currently_used_asset {
// We don't want to infer that any simple identifier refers to an asset if
// we are not in a known R/W context
if access_type.is_none() {
return None;
}
if let Some((kind, path)) = &self.currently_used_asset {
if name.0.len() == 1 || name.0.len() == 2 {
if name
.0
.iter()
.any(|id| id.as_ident().and_then(|id| id.quote_style).is_some())
{
return None;
}
let specific_table = &name
.0
.iter()
.map(|id| id.as_ident().map(|id| id.value.clone()))
.collect::<Option<Vec<String>>>()?
.join(".");
let path = format!("{}/{}", path, specific_table);
return Some(ParseAssetsResult { kind: *kind, access_type, path });
}
@@ -82,8 +91,12 @@ impl AssetCollector {
}
let ident = name.0.first()?.as_ident()?;
let (kind, path) = self.var_identifiers.get(&ident.value)?;
let path = if name.0.len() == 2 {
let specific_table = &name.0.get(1)?.as_ident()?.value;
let path = if name.0.len() == 2 || name.0.len() == 3 {
let specific_table = &name.0[1..]
.iter()
.map(|id| id.as_ident().map(|id| id.value.clone()))
.collect::<Option<Vec<String>>>()?
.join(".");
format!("{}/{}", path, specific_table)
} else {
path.clone()
@@ -389,7 +402,7 @@ mod tests {
SELECT * FROM read_parquet('s3:///a.parquet');
COPY (SELECT * FROM 's3://snd/b.parquet') TO 's3:///c.parquet';
"#;
let s = parse_assets(input);
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![
@@ -419,7 +432,7 @@ mod tests {
SELECT 2;
USE dl;
"#;
let s = parse_assets(input);
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
@@ -436,7 +449,7 @@ mod tests {
ATTACH 'ducklake://my_dl' AS dl;
SELECT * FROM dl.table1;
"#;
let s = parse_assets(input);
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
@@ -454,7 +467,7 @@ mod tests {
SELECT dt.read_bait FROM unrelated_table; -- dt. doesn't access the asset
INSERT INTO dt.table1 VALUES ('test');
"#;
let s = parse_assets(input);
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
@@ -472,7 +485,7 @@ mod tests {
DETACH dl;
SELECT * FROM dl.table1;
"#;
let s = parse_assets(input);
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(s.map_err(|e| e.to_string()), Ok(vec![]));
}
@@ -485,7 +498,7 @@ mod tests {
USE memory;
SELECT * FROM table1;
"#;
let s = parse_assets(input);
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
@@ -502,7 +515,7 @@ mod tests {
ATTACH 'datatable' AS dl;
INSERT INTO dl.table1 VALUES ('test');
"#;
let s = parse_assets(input);
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
@@ -524,7 +537,7 @@ mod tests {
INSERT INTO friends VALUES ($name, $age);
SELECT * FROM friends;
"#;
let s = parse_assets(input);
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
@@ -542,7 +555,7 @@ mod tests {
ATTACH 'ducklake' AS dl; USE dl;
SELECT * FROM a_function('');
"#;
let s = parse_assets(input);
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
@@ -560,7 +573,7 @@ mod tests {
USE dl;
DELETE FROM table1;
"#;
let s = parse_assets(input);
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
@@ -578,7 +591,7 @@ mod tests {
USE dl;
UPDATE table1 SET id = NULL;
"#;
let s = parse_assets(input);
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
@@ -596,7 +609,7 @@ mod tests {
USE db;
SELECT * FROM table1;
"#;
let s = parse_assets(input);
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
@@ -613,7 +626,7 @@ mod tests {
ATTACH 'ducklake' AS dl;
UPDATE dl.table1 SET id = NULL;
"#;
let s = parse_assets(input);
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
@@ -623,4 +636,41 @@ mod tests {
},])
);
}
#[test]
fn test_sql_asset_parser_table_with_schema() {
let input = r#"
ATTACH 'ducklake' AS dl;
UPDATE dl.sch.table1 SET id = NULL;
SELECT * FROM dl.sch.table1;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "main/sch.table1".to_string(),
access_type: Some(RW)
},])
);
}
#[test]
fn test_sql_asset_parser_table_with_schema_implicit() {
let input = r#"
ATTACH 'ducklake' AS dl;
USE dl;
UPDATE sch.table1 SET id = NULL;
SELECT * FROM sch.table1;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "main/sch.table1".to_string(),
access_type: Some(RW)
},])
);
}
}

View File

@@ -1,16 +1,16 @@
use std::collections::HashMap;
use swc_common::{sync::Lrc, FileName, SourceMap};
use swc_common::{sync::Lrc, FileName, SourceMap, Spanned};
use swc_ecma_ast::{CallExpr, Expr, Lit, MemberExpr, MemberProp, Str};
use swc_ecma_parser::{lexer::Lexer, Parser, StringInput, Syntax, TsSyntax};
use swc_ecma_visit::{Visit, VisitWith};
use windmill_parser::asset_parser::{
asset_was_used, merge_assets, parse_asset_syntax, AssetKind, AssetUsageAccessType,
ParseAssetsResult,
ParseAssetsOutput, ParseAssetsResult, SqlQueryDetails,
};
use AssetUsageAccessType::*;
pub fn parse_assets(code: &str) -> anyhow::Result<Vec<ParseAssetsResult>> {
pub fn parse_assets(code: &str) -> anyhow::Result<ParseAssetsOutput> {
let cm: Lrc<SourceMap> = Default::default();
let fm = cm.new_source_file(FileName::Custom("main.ts".into()).into(), code.into());
let lexer = Lexer::new(
@@ -35,11 +35,17 @@ pub fn parse_assets(code: &str) -> anyhow::Result<Vec<ParseAssetsResult>> {
anyhow::anyhow!("Error while parsing code, it is invalid TypeScript: {err_s}, {e:?}")
})?
.body;
let mut assets_finder = AssetsFinder { assets: vec![], var_identifiers: HashMap::new() };
let mut assets_finder =
AssetsFinder { assets: vec![], sql_queries: vec![], var_identifiers: HashMap::new() };
assets_finder.visit_module_items(&ast);
Ok(merge_assets(assets_finder.assets))
Ok(ParseAssetsOutput {
assets: merge_assets(assets_finder.assets),
sql_queries: assets_finder.sql_queries,
})
}
type VarAssetName = String;
type VarAssetSchema = Option<String>;
struct AssetsFinder {
assets: Vec<ParseAssetsResult>,
@@ -49,9 +55,57 @@ struct AssetsFinder {
// The goal is to remember that the identifier "sql" corresponds to the datatable "main"
// so that when we see a tagged template expression with tag "sql" we know which datatable it
// corresponds to. This allows us to infer if a datatable is Read or Write based on the SQL query.
var_identifiers: HashMap<String, (AssetKind, String)>,
var_identifiers: HashMap<String, (AssetKind, VarAssetName, VarAssetSchema)>,
sql_queries: Vec<SqlQueryDetails>,
}
/// Helper function to extract wmill.datatable() or wmill.ducklake() calls,
/// Returns (AssetKind, asset_name, optional_schema_name)
fn extract_wmill_datatable_call(expr: &Expr) -> Option<(AssetKind, String, Option<String>)> {
if let Expr::Call(call_expr) = expr {
if let Some(Expr::Member(member)) = call_expr.callee.as_expr().map(AsRef::as_ref) {
// Check if object is "wmill"
let is_wmill = matches!(
member.obj.as_ref(),
Expr::Ident(ident) if ident.sym.as_str() == "wmill"
);
if is_wmill {
if let MemberProp::Ident(prop) = &member.prop {
// Get the asset name from first arg, default to "main"
let asset_name = call_expr
.args
.first()
.and_then(|arg| match arg.expr.as_ref() {
Expr::Lit(Lit::Str(s)) => Some(s.value.to_string()),
_ => None,
})
.unwrap_or_else(|| "main".to_string());
let (asset_name, schema_name) = asset_name.split_once(':').map_or_else(
|| (asset_name.clone(), None),
|(name, schema)| {
(
(if name.is_empty() { "main" } else { name }).to_string(),
Some(schema.to_string()),
)
},
);
let kind = match prop.sym.as_str() {
"datatable" => Some(AssetKind::DataTable),
"ducklake" => Some(AssetKind::Ducklake),
_ => None,
};
return kind.map(|k| (k, asset_name, schema_name));
}
}
}
}
None
}
impl Visit for AssetsFinder {
// visit_call_expr will not recurse if it detects an asset,
// so this will only be called when no further context was found
@@ -77,6 +131,37 @@ impl Visit for AssetsFinder {
}
}
fn visit_assign_expr(&mut self, node: &swc_ecma_ast::AssignExpr) {
// Handle reassignments like: sql = wmill.datatable('main')
// Extract the variable name from the left side
let var_name = match &node.left {
swc_ecma_ast::AssignTarget::Simple(simple_target) => match simple_target {
swc_ecma_ast::SimpleAssignTarget::Ident(ident_binding) => {
ident_binding.id.sym.as_str().to_string()
}
_ => {
node.visit_children_with(self);
return;
}
},
_ => {
node.visit_children_with(self);
return;
}
};
// Check if right side is a wmill.datatable() or wmill.ducklake() call
if let Some((kind, asset_name, schema)) = extract_wmill_datatable_call(node.right.as_ref())
{
self.var_identifiers
.insert(var_name, (kind, asset_name, schema));
return;
}
// Default: visit children
node.visit_children_with(self);
}
fn visit_block_stmt(&mut self, node: &swc_ecma_ast::BlockStmt) {
// Save current state before entering the block
let saved_var_identifiers = self.var_identifiers.clone();
@@ -88,7 +173,7 @@ impl Visit for AssetsFinder {
if saved_var_identifiers.contains_key(var) {
continue;
}
let (kind, ref path) = self.var_identifiers[var];
let (kind, ref path, _) = self.var_identifiers[var];
if asset_was_used(&self.assets, (kind, path)) {
continue;
}
@@ -111,43 +196,12 @@ impl Visit for AssetsFinder {
};
// Check if init is a call to wmill.datatable(...) or wmill.ducklake(...)
// optionally with .schema() chained
if let Some(init) = &node.init {
if let Expr::Call(call_expr) = init.as_ref() {
if let Some(Expr::Member(member)) = call_expr.callee.as_expr().map(AsRef::as_ref) {
// Check if object is "wmill"
let is_wmill = matches!(
member.obj.as_ref(),
Expr::Ident(ident) if ident.sym.as_str() == "wmill"
);
if is_wmill {
if let MemberProp::Ident(prop) = &member.prop {
// Get the asset name from first arg, default to "main"
let asset_name = call_expr
.args
.first()
.and_then(|arg| match arg.expr.as_ref() {
Expr::Lit(Lit::Str(s)) => Some(s.value.to_string()),
_ => None,
})
.unwrap_or_else(|| "main".to_string());
match prop.sym.as_str() {
"datatable" => {
self.var_identifiers
.insert(var_name, (AssetKind::DataTable, asset_name));
return;
}
"ducklake" => {
self.var_identifiers
.insert(var_name, (AssetKind::Ducklake, asset_name));
return;
}
_ => {}
}
}
}
}
if let Some((kind, asset_name, schema)) = extract_wmill_datatable_call(init.as_ref()) {
self.var_identifiers
.insert(var_name, (kind, asset_name, schema));
return;
}
}
@@ -166,33 +220,64 @@ impl Visit for AssetsFinder {
};
// Check if it's a known identifier
let (kind, asset_name) = if let Some((kind, name)) = self.var_identifiers.get(tag_name) {
(*kind, name.clone())
} else {
let Some((kind, asset_name, schema)) = self.var_identifiers.get(tag_name) else {
node.visit_children_with(self);
return;
};
// Extract the SQL query from the template quasis (string parts)
// Substitute ${} with $1, $2, etc.
let sql: String = node
.tpl
.quasis
.iter()
.map(|quasi| quasi.raw.as_str())
.collect::<Vec<_>>()
.join("$1"); // placeholder for expressions
.enumerate()
.fold(String::new(), |acc, (i, s)| {
if i == 0 {
s.to_string()
} else {
format!("{}${}{}", acc, i, s)
}
});
let duckdb_conn_prefix = match kind {
AssetKind::DataTable => "datatable",
AssetKind::Ducklake => "ducklake",
_ => return,
};
let sql = format!("ATTACH '{duckdb_conn_prefix}://{asset_name}' AS dt; USE dt; {sql}");
// Capture SQL query details before transforming for SQL parser
let span = node.span();
let span_tuple = (span.lo.0, span.hi.0);
self.sql_queries.push(SqlQueryDetails {
query_string: sql.clone(),
span: span_tuple,
source_kind: *kind,
source_name: asset_name.clone(),
source_schema: schema.clone(),
});
let sql_with_attach =
format!("ATTACH '{duckdb_conn_prefix}://{asset_name}' AS dt; USE dt; {sql}");
// We use the SQL parser to detect if it's a read or write query
match windmill_parser_sql::parse_assets(&sql) {
Ok(sql_assets) => {
self.assets.extend(sql_assets);
match windmill_parser_sql::parse_assets(&sql_with_attach) {
Ok(mut sql_assets) => {
if let Some(schema) = schema {
for asset in &mut sql_assets.assets {
if asset.kind == *kind && asset.path.starts_with(asset_name) {
asset.path = format!(
"{}/{}.{}",
asset_name,
schema,
&asset.path[asset_name.len() + 1..]
);
}
}
}
self.assets.extend(sql_assets.assets);
}
_ => {}
}
@@ -249,7 +334,7 @@ mod tests {
"#;
let s = parse_assets(input);
assert_eq!(
s.map_err(|e| e.to_string()),
s.map(|r| r.assets).map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::S3Object,
path: "/test.csv".to_string(),
@@ -268,7 +353,7 @@ mod tests {
"#;
let s = parse_assets(input);
assert_eq!(
s.map_err(|e| e.to_string()),
s.map(|r| r.assets).map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt".to_string(),
@@ -288,7 +373,7 @@ mod tests {
"#;
let s = parse_assets(input);
assert_eq!(
s.map_err(|e| e.to_string()),
s.map(|r| r.assets).map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/friends".to_string(),
@@ -310,7 +395,7 @@ mod tests {
"#;
let s = parse_assets(input);
assert_eq!(
s.map_err(|e| e.to_string()),
s.map(|r| r.assets).map_err(|e| e.to_string()),
Ok(vec![
ParseAssetsResult {
kind: AssetKind::DataTable,
@@ -345,7 +430,7 @@ mod tests {
"#;
let s = parse_assets(input);
assert_eq!(
s.map_err(|e| e.to_string()),
s.map(|r| r.assets).map_err(|e| e.to_string()),
Ok(vec![
ParseAssetsResult {
kind: AssetKind::DataTable,
@@ -379,7 +464,7 @@ mod tests {
"#;
let s = parse_assets(input);
assert_eq!(
s.map_err(|e| e.to_string()),
s.map(|r| r.assets).map_err(|e| e.to_string()),
Ok(vec![
ParseAssetsResult {
kind: AssetKind::DataTable,
@@ -394,4 +479,205 @@ mod tests {
])
);
}
#[test]
fn test_ts_asset_parser_datatable_with_schema() {
let input = r#"
import * as wmill from "windmill-client"
export async function main(x: number) {
let sql = wmill.datatable(':myschema')
return await sql`SELECT * FROM friends WHERE age = ${x}`.fetch()
}
"#;
let s = parse_assets(input);
assert_eq!(
s.map(|r| r.assets).map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "main/myschema.friends".to_string(),
access_type: Some(R)
},])
);
}
#[test]
fn test_ts_asset_parser_schema_with_write() {
let input = r#"
import * as wmill from "windmill-client"
export async function main(x: number) {
let sql = wmill.datatable('dt:public')
await sql`INSERT INTO users VALUES (${x})`.fetch()
return await sql`SELECT * FROM users`.fetch()
}
"#;
let s = parse_assets(input);
assert_eq!(
s.map(|r| r.assets).map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/public.users".to_string(),
access_type: Some(RW)
},])
);
}
#[test]
fn test_ts_asset_parser_unused_datatable_with_schema() {
let input = r#"
import * as wmill from "windmill-client"
export async function main() {
let sql = wmill.datatable('dt:myschema')
}
"#;
let s = parse_assets(input);
assert_eq!(
s.map(|r| r.assets).map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt".to_string(),
access_type: None
},])
);
}
#[test]
fn test_ts_asset_parser_reassignment() {
let input = r#"
import * as wmill from "windmill-client"
export async function main(x: number) {
let sql;
sql = wmill.datatable('dt')
return await sql`SELECT * FROM users WHERE id = ${x}`.fetch()
}
"#;
let s = parse_assets(input);
assert_eq!(
s.map(|r| r.assets).map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/users".to_string(),
access_type: Some(R)
},])
);
}
#[test]
fn test_ts_asset_parser_reassignment_with_schema() {
let input = r#"
import * as wmill from "windmill-client"
export async function main(x: number) {
let sql = wmill.datatable('dt')
await sql`INSERT INTO test VALUES ('')`.fetch()
sql = wmill.datatable('dt:private')
return await sql`SELECT * FROM users WHERE id = ${x}`.fetch()
}
"#;
let s = parse_assets(input);
assert_eq!(
s.map(|r| r.assets).map_err(|e| e.to_string()),
Ok(vec![
ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/private.users".to_string(),
access_type: Some(R)
},
ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/test".to_string(),
access_type: Some(W)
},
])
);
}
#[test]
fn test_ts_asset_parser_sql_query_details() {
let input = r#"
import * as wmill from "windmill-client"
export async function main(x: number) {
let sql = wmill.datatable('dt')
return await sql`SELECT * FROM friends WHERE age = ${x}`.fetch()
}
"#;
let result = parse_assets(input).unwrap();
// Check assets
assert_eq!(result.assets.len(), 1);
assert_eq!(result.assets[0].kind, AssetKind::DataTable);
assert_eq!(result.assets[0].path, "dt/friends");
// Check SQL query details
assert_eq!(result.sql_queries.len(), 1);
let query_detail = &result.sql_queries[0];
assert_eq!(
query_detail.query_string,
"SELECT * FROM friends WHERE age = $1"
);
assert_eq!(query_detail.source_kind, AssetKind::DataTable);
assert_eq!(query_detail.source_name, "dt");
assert_eq!(query_detail.source_schema, None);
// Span should be non-zero
assert!(query_detail.span.0 > 0);
assert!(query_detail.span.1 > query_detail.span.0);
}
#[test]
fn test_ts_asset_parser_sql_query_details_with_schema() {
let input = r#"
import * as wmill from "windmill-client"
export async function main(x: number) {
let sql = wmill.datatable('dt:public')
await sql`INSERT INTO users VALUES (${x})`.fetch()
return await sql`SELECT * FROM users`.fetch()
}
"#;
let result = parse_assets(input).unwrap();
// Check SQL query details
assert_eq!(result.sql_queries.len(), 2);
// First query (INSERT)
assert_eq!(
result.sql_queries[0].query_string,
"INSERT INTO users VALUES ($1)"
);
assert_eq!(result.sql_queries[0].source_kind, AssetKind::DataTable);
assert_eq!(result.sql_queries[0].source_name, "dt");
assert_eq!(
result.sql_queries[0].source_schema,
Some("public".to_string())
);
// Second query (SELECT)
assert_eq!(result.sql_queries[1].query_string, "SELECT * FROM users");
assert_eq!(result.sql_queries[1].source_kind, AssetKind::DataTable);
assert_eq!(result.sql_queries[1].source_name, "dt");
assert_eq!(
result.sql_queries[1].source_schema,
Some("public".to_string())
);
}
#[test]
fn test_ts_asset_parser_sql_query_details_ducklake() {
let input = r#"
import * as wmill from "windmill-client"
export async function main() {
let sql = wmill.ducklake('my_lake')
return await sql`SELECT id, name FROM products LIMIT 10`.fetch()
}
"#;
let result = parse_assets(input).unwrap();
// Check SQL query details
assert_eq!(result.sql_queries.len(), 1);
let query_detail = &result.sql_queries[0];
assert_eq!(
query_detail.query_string,
"SELECT id, name FROM products LIMIT 10"
);
assert_eq!(query_detail.source_kind, AssetKind::Ducklake);
assert_eq!(query_detail.source_name, "my_lake");
assert_eq!(query_detail.source_schema, None);
}
}

View File

@@ -1,10 +1,10 @@
use windmill_parser::asset_parser::{
merge_assets, AssetKind, AssetUsageAccessType, ParseAssetsResult,
merge_assets, AssetKind, AssetUsageAccessType, ParseAssetsOutput, ParseAssetsResult,
};
use crate::{parse_ansible_reqs, ResourceOrVariablePath};
pub fn parse_assets(input: &str) -> anyhow::Result<Vec<ParseAssetsResult>> {
pub fn parse_assets(input: &str) -> anyhow::Result<ParseAssetsOutput> {
let mut assets = vec![];
if let (_, Some(ansible_reqs), _) = parse_ansible_reqs(input)? {
if let Some(delegate_to_git_repo_details) = ansible_reqs.delegate_to_git_repo {
@@ -36,5 +36,5 @@ pub fn parse_assets(input: &str) -> anyhow::Result<Vec<ParseAssetsResult>> {
}
}
Ok(merge_assets(assets))
Ok(ParseAssetsOutput { assets: merge_assets(assets), ..Default::default() })
}

View File

@@ -958,7 +958,7 @@ dependencies:
content: "{{ my_result | to_json }}"
dest: result.json
"#;
let a = parse_assets(p).unwrap();
let a = parse_assets(p).unwrap().assets;
println!("The resulting assets are: {}", a.len());
let a = parse_ansible_reqs(p).unwrap();

View File

@@ -27,6 +27,22 @@ pub struct ParseAssetsResult {
pub access_type: Option<AssetUsageAccessType>, // None in case of ambiguity
}
#[derive(Serialize, Debug, PartialEq)]
pub struct SqlQueryDetails {
pub query_string: String, // SQL query with $1 placeholders for interpolations
pub span: (u32, u32), // (start, end) byte positions in source code
pub source_kind: AssetKind, // DataTable or Ducklake
pub source_name: String, // e.g., "main", "dt"
#[serde(skip_serializing_if = "Option::is_none")]
pub source_schema: Option<String>, // e.g., Some("public"), None
}
#[derive(Serialize, Debug, Default)]
pub struct ParseAssetsOutput {
pub assets: Vec<ParseAssetsResult>,
pub sql_queries: Vec<SqlQueryDetails>,
}
#[derive(Debug, Clone, Serialize)]
pub struct DelegateToGitRepoDetails {
pub resource: String,

65
backend/src/cgroups.rs Normal file
View File

@@ -0,0 +1,65 @@
use std::fs;
use std::path::PathBuf;
#[derive(Debug)]
pub enum CgroupError {
#[allow(unused)]
PathNotFound(PathBuf),
NotSupported,
PermissionDenied,
#[allow(unused)]
Io(std::io::Error),
}
impl From<std::io::Error> for CgroupError {
fn from(e: std::io::Error) -> Self {
CgroupError::Io(e)
}
}
pub fn get_cgroup_path() -> Result<PathBuf, CgroupError> {
let cgroup_info = fs::read_to_string("/proc/1/cgroup")?;
// Format: "0::/kubepods.slice/..." - we want the part after the second colon
let cgroup_rel = cgroup_info
.lines()
.next()
.and_then(|line| line.splitn(3, ':').nth(2))
.unwrap_or("")
.trim();
let cgroup_path = PathBuf::from(format!("/sys/fs/cgroup{}", cgroup_rel));
if !cgroup_path.is_dir() {
return Err(CgroupError::PathNotFound(cgroup_path));
}
Ok(cgroup_path)
}
pub fn disable_oom_group() -> Result<(), CgroupError> {
let cgroup_path = get_cgroup_path()?;
let oom_group_file = cgroup_path.join("memory.oom.group");
if !oom_group_file.exists() {
return Err(CgroupError::NotSupported);
}
let current = fs::read_to_string(&oom_group_file)?;
if current.trim() == "0" {
tracing::info!("memory.oom.group already disabled");
return Ok(());
}
match fs::write(&oom_group_file, "0") {
Ok(_) => {
tracing::info!("Disabled memory.oom.group at {:?}", cgroup_path);
Ok(())
}
Err(e) if e.kind() == std::io::ErrorKind::PermissionDenied => {
tracing::error!("Failed to disable memory.oom.group (need privileged mode)");
Err(CgroupError::PermissionDenied)
}
Err(e) => Err(CgroupError::Io(e)),
}
}

View File

@@ -71,6 +71,9 @@ use windmill_common::worker::CLOUD_HOSTED;
#[cfg(all(not(target_env = "msvc"), feature = "jemalloc"))]
use monitor::monitor_mem;
#[cfg(any(target_os = "linux"))]
use crate::cgroups::disable_oom_group;
#[cfg(all(not(target_env = "msvc"), feature = "jemalloc"))]
use tikv_jemallocator::Jemalloc;
@@ -108,6 +111,7 @@ const DEFAULT_NUM_WORKERS: usize = 1;
const DEFAULT_PORT: u16 = 8000;
const DEFAULT_SERVER_BIND_ADDR: Ipv4Addr = Ipv4Addr::new(0, 0, 0, 0);
mod cgroups;
#[cfg(feature = "private")]
pub mod ee;
mod ee_oss;
@@ -507,6 +511,13 @@ async fn windmill_main() -> anyhow::Result<()> {
let worker_mode = num_workers > 0;
if worker_mode {
#[cfg(any(target_os = "linux"))]
if let Err(e) = disable_oom_group() {
tracing::warn!("failed to disable oom group: {:?}", e);
}
}
let conn = if mode == Mode::Agent {
conn
} else {

View File

@@ -691,8 +691,8 @@ pub async fn run_deployed_relative_imports(
language,
priority: None,
apply_preprocessor: false,
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
})
.push(&db2)
.await;
@@ -741,8 +741,8 @@ pub async fn run_preview_relative_imports(
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
}))
.push(&db2)
.await;

View File

@@ -52,8 +52,10 @@ mod job_payload {
let result = RunJob::from(JobPayload::ScriptHash {
hash: ScriptHash(123412),
path: "f/system/hello".to_string(),
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
concurrency_settings:
windmill_common::runnable_settings::ConcurrencySettings::default().into(),
debouncing_settings:
windmill_common::runnable_settings::DebouncingSettings::default(),
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
@@ -90,8 +92,10 @@ mod job_payload {
language: ScriptLang::Deno,
priority: None,
apply_preprocessor: true,
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
concurrency_settings:
windmill_common::runnable_settings::ConcurrencySettings::default(),
debouncing_settings:
windmill_common::runnable_settings::DebouncingSettings::default(),
})
.run_until_complete_with(db, false, port, |id| async move {
let job = sqlx::query!("SELECT preprocessed FROM v2_job WHERE id = $1", id)
@@ -163,7 +167,8 @@ mod job_payload {
let result = RunJob::from(JobPayload::FlowScript {
id: flow_scripts[0],
language: ScriptLang::Deno,
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default(),
concurrency_settings:
windmill_common::runnable_settings::ConcurrencySettings::default(),
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
@@ -182,7 +187,8 @@ mod job_payload {
let result = RunJob::from(JobPayload::FlowScript {
id: flow_scripts[1],
language: ScriptLang::Deno,
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default(),
concurrency_settings:
windmill_common::runnable_settings::ConcurrencySettings::default(),
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
@@ -547,6 +553,7 @@ mod job_payload {
completed_job_id,
step_id: "a".into(),
branch_or_iteration_n: None,
flow_version: None,
})
.arg("iter", json!({ "value": "tests", "index": 0 }))
.run_until_complete(&db, false, port)
@@ -714,7 +721,12 @@ mod job_payload {
)
.await;
let flow_job_id = test(
Some(RestartedFrom { flow_job_id, step_id: "a".into(), branch_or_iteration_n: None }),
Some(RestartedFrom {
flow_job_id,
step_id: "a".into(),
branch_or_iteration_n: None,
flow_version: None,
}),
json!("foo"),
json!([
"a: Hello foo! foo! foo!",
@@ -724,7 +736,12 @@ mod job_payload {
)
.await;
let flow_job_id = test(
Some(RestartedFrom { flow_job_id, step_id: "b".into(), branch_or_iteration_n: None }),
Some(RestartedFrom {
flow_job_id,
step_id: "b".into(),
branch_or_iteration_n: None,
flow_version: None,
}),
json!("bar"),
json!([
"a: Hello foo! bar! bar!",
@@ -738,6 +755,7 @@ mod job_payload {
flow_job_id,
step_id: "c".into(),
branch_or_iteration_n: Some(1),
flow_version: None,
}),
json!("yolo"),
json!([

View File

@@ -189,8 +189,8 @@ def main():
path: None,
language: ScriptLang::Python3,
lock: None,
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
@@ -238,8 +238,8 @@ def main():
path: None,
language: ScriptLang::Python3,
lock: None,
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
@@ -272,8 +272,8 @@ def main():
path: None,
language: ScriptLang::Python3,
lock: None,
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
@@ -311,8 +311,8 @@ def main():
path: None,
language: ScriptLang::Python3,
lock: None,
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
@@ -348,8 +348,8 @@ def main():
path: None,
language: ScriptLang::Python3,
lock: None,
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,

View File

@@ -188,7 +188,7 @@ async fn test_deno_flow(db: Pool<Postgres>) -> anyhow::Result<()> {
path: None,
lock: None,
tag: None,
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default()
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default()
.into(),
is_trigger: None,
assets: None,
@@ -235,7 +235,7 @@ async fn test_deno_flow(db: Pool<Postgres>) -> anyhow::Result<()> {
lock: None,
tag: None,
concurrency_settings:
windmill_common::jobs::ConcurrencySettings::default().into(),
windmill_common::runnable_settings::ConcurrencySettings::default().into(),
is_trigger: None,
assets: None,
}
@@ -369,7 +369,7 @@ async fn test_deno_flow_same_worker(db: Pool<Postgres>) -> anyhow::Result<()> {
path: None,
lock: None,
tag: None,
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default().into(),
is_trigger: None,
assets: None,
@@ -425,7 +425,7 @@ async fn test_deno_flow_same_worker(db: Pool<Postgres>) -> anyhow::Result<()> {
path: None,
lock: None,
tag: None,
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default().into(),
is_trigger: None,
assets: None,
}.into(),
@@ -465,7 +465,7 @@ async fn test_deno_flow_same_worker(db: Pool<Postgres>) -> anyhow::Result<()> {
path: None,
lock: None,
tag: None,
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default().into(),
is_trigger: None,
assets: None,
@@ -533,7 +533,7 @@ async fn test_deno_flow_same_worker(db: Pool<Postgres>) -> anyhow::Result<()> {
path: None,
lock: None,
tag: None,
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default().into(),
is_trigger: None,
assets: None,
}.into(),
@@ -865,8 +865,9 @@ func main(derp string) (string, error) {
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default()
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
}))
.arg("derp", json!("world"))
.run_until_complete(&db, false, port)
@@ -900,8 +901,9 @@ fn main(world: String) -> Result<String, String> {
lock: None,
language: ScriptLang::Rust,
cache_ignore_s3_path: None,
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default()
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
cache_ttl: None,
dedicated_worker: None,
}))
@@ -978,8 +980,9 @@ echo "hello $msg"
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default()
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
}))
.arg("msg", json!("world"))
.run_until_complete(&db, false, port)
@@ -1011,8 +1014,9 @@ def main [ msg: string ] {
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default()
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
}))
.arg("msg", json!("world"))
.run_until_complete(&db, false, port)
@@ -1064,8 +1068,9 @@ def main [
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default()
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
}))
.arg("a", json!("3"))
.arg("b", json!("null"))
@@ -1126,8 +1131,9 @@ public class Main {
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default()
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
}))
.arg("a", json!(3))
.arg("b", json!(3.0))
@@ -1161,8 +1167,9 @@ export async function main(a: Date) {
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default()
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
}))
.arg("a", json!("2024-09-24T10:00:00.000Z"))
.run_until_complete(&db, false, port)
@@ -1196,8 +1203,9 @@ export async function main(a: Date) {
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default()
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
}))
.arg("a", json!("2024-09-24T10:00:00.000Z"))
.run_until_complete(&db, false, port)
@@ -1232,8 +1240,9 @@ def main(a: datetime, b: bytes):
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default()
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
}))
.arg("a", json!("2024-09-24T10:00:00.000Z"))
.arg("b", json!("dGVzdA=="))
@@ -2279,6 +2288,7 @@ async fn test_complex_flow_restart(db: Pool<Postgres>) -> anyhow::Result<()> {
flow_job_id: first_run_result.id,
step_id: "h".to_owned(),
branch_or_iteration_n: None,
flow_version: None,
}),
})
.run_until_complete(&db, false, port)

View File

@@ -92,7 +92,7 @@ mail-parser = { workspace = true, features = ["serde_support"], optional = true
magic-crypt.workspace = true
tempfile.workspace = true
tokio-util.workspace = true
tokio-tar.workspace = true
astral-tokio-tar.workspace = true
tokio-postgres.workspace = true
postgres-native-tls.workspace = true
hmac.workspace = true
@@ -149,6 +149,7 @@ rustls = { workspace = true }
aws-sigv4.workspace = true
aws-sdk-config.workspace = true
aws-config = { workspace = true, optional = true }
aws-credential-types.workspace = true
async-trait.workspace = true
google-cloud-pubsub = { workspace = true, optional = true }
google-cloud-googleapis = { workspace = true , optional = true }

View File

@@ -1,7 +1,7 @@
openapi: "3.0.3"
info:
version: 1.591.3
version: 1.596.0
title: Windmill API
contact:
@@ -2024,6 +2024,52 @@ paths:
schema:
type: string
/w/{workspace}/workspaces/compare/{target_workspace_id}:
get:
operationId: compareWorkspaces
summary: Compare two workspaces
description: Compares the current workspace with a target workspace to find differences in scripts, flows, apps, resources, and variables. Returns information about items that are ahead, behind, or in conflict.
tags:
- workspace
parameters:
- $ref: "#/components/parameters/WorkspaceId"
- name: target_workspace_id
in: path
required: true
schema:
type: string
description: The ID of the workspace to compare with
responses:
"200":
description: Workspace comparison results
content:
application/json:
schema:
$ref: "#/components/schemas/WorkspaceComparison"
/w/{workspace}/workspaces/reset_diff_tally/{fork_workspace_id}:
post:
operationId: resetDiffTally
summary: Resets the ahead and behind deployement counter after a deployement
description: This endpoint should be called after a fork deployement
tags:
- workspace
parameters:
- $ref: "#/components/parameters/WorkspaceId"
- name: fork_workspace_id
in: path
required: true
schema:
type: string
description: The ID of the workspace to compare with
responses:
"200":
description: status
content:
application/json:
schema: {}
/users/exists/{email}:
get:
summary: exists email
@@ -2121,6 +2167,8 @@ paths:
type: string
teams_team_name:
type: string
teams_team_guid:
type: string
auto_invite_domain:
type: string
auto_invite_operator:
@@ -2512,7 +2560,13 @@ paths:
- $ref: "#/components/parameters/WorkspaceId"
- name: search
in: query
description: Search teams by name
description: Search teams by name. If omitted, returns first page of all teams.
required: false
schema:
type: string
- name: next_link
in: query
description: Pagination cursor URL from previous response. Pass this to fetch the next page of results.
required: false
schema:
type: string
@@ -2522,14 +2576,27 @@ paths:
content:
application/json:
schema:
type: array
items:
type: object
properties:
team_name:
type: string
team_id:
type: string
type: object
properties:
teams:
type: array
items:
type: object
properties:
team_name:
type: string
team_id:
type: string
total_count:
type: integer
description: Total number of teams across all pages
per_page:
type: integer
description: Number of teams per page (configurable via TEAMS_PER_PAGE env var)
next_link:
type: string
nullable: true
description: URL to fetch next page of results. Null if no more pages.
/w/{workspace}/workspaces/available_teams_channels:
get:
@@ -2545,26 +2612,25 @@ paths:
required: true
schema:
type: string
- name: search
in: query
description: Search channels by name
required: false
schema:
type: string
responses:
"200":
description: List of channels for the specified team
content:
application/json:
schema:
type: array
items:
type: object
properties:
channel_name:
type: string
channel_id:
type: string
type: object
properties:
channels:
type: array
items:
type: object
properties:
channel_name:
type: string
channel_id:
type: string
total_count:
type: integer
/w/{workspace}/workspaces/connect_teams:
post:
@@ -2963,6 +3029,7 @@ paths:
application/json:
schema: {}
/w/{workspace}/workspaces/edit_git_sync_config:
post:
summary: edit workspace git sync settings
@@ -8271,7 +8338,7 @@ paths:
schema:
type: string
/w/{workspace}/jobs/restart/f/{id}/from/{step_id}/{branch_or_iteration_n}:
/w/{workspace}/jobs/restart/f/{id}:
post:
summary: restart a completed flow at a given step
operationId: restartFlowAtStep
@@ -8280,20 +8347,6 @@ paths:
parameters:
- $ref: "#/components/parameters/WorkspaceId"
- $ref: "#/components/parameters/JobId"
- name: step_id
description: step id to restart the flow from
required: true
in: path
schema:
type: string
- name: branch_or_iteration_n
description:
for branchall or loop, the iteration at which the flow should
restart
required: true
in: path
schema:
type: integer
- name: scheduled_for
description: when to schedule this job (leave empty for immediate run)
in: query
@@ -8316,12 +8369,24 @@ paths:
type: boolean
requestBody:
description: flow args
description: restart flow parameters
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/ScriptArgs"
type: object
required:
- step_id
properties:
step_id:
type: string
description: step id to restart the flow from
branch_or_iteration_n:
type: integer
description: for branchall or loop, the iteration at which the flow should restart (optional)
flow_version:
type: integer
description: specific flow version to use for restart (optional, uses current version if not specified)
responses:
"201":
@@ -12975,6 +13040,40 @@ paths:
schema:
type: string
/w/{workspace}/groups_history/get/{name}:
get:
summary: get group permission history
operationId: getGroupPermissionHistory
tags:
- group
parameters:
- $ref: "#/components/parameters/WorkspaceId"
- $ref: "#/components/parameters/Name"
- $ref: "#/components/parameters/Page"
- $ref: "#/components/parameters/PerPage"
responses:
"200":
description: group permission history
content:
application/json:
schema:
type: array
items:
type: object
properties:
id:
type: integer
changed_by:
type: string
changed_at:
type: string
format: date-time
change_type:
type: string
member_affected:
type: string
nullable: true
/w/{workspace}/folders/list:
get:
summary: list folders
@@ -13238,6 +13337,40 @@ paths:
schema:
type: string
/w/{workspace}/folders_history/get/{name}:
get:
summary: get folder permission history
operationId: getFolderPermissionHistory
tags:
- folder
parameters:
- $ref: "#/components/parameters/WorkspaceId"
- $ref: "#/components/parameters/Name"
- $ref: "#/components/parameters/Page"
- $ref: "#/components/parameters/PerPage"
responses:
"200":
description: folder permission history
content:
application/json:
schema:
type: array
items:
type: object
properties:
id:
type: integer
changed_by:
type: string
changed_at:
type: string
format: date-time
change_type:
type: string
affected:
type: string
nullable: true
/workers/list:
get:
summary: list workers
@@ -20068,6 +20201,8 @@ components:
type: string
branch_or_iteration_n:
type: integer
flow_version:
type: integer
Policy:
type: object
@@ -20927,6 +21062,108 @@ components:
type: boolean
description: Whether operators can view workers page
WorkspaceComparison:
type: object
required:
- all_ahead_items_visible
- all_behind_items_visible
- skipped_comparison
- diffs
- summary
properties:
all_ahead_items_visible:
type: boolean
description: All items with changes ahead are visible by the user of the request.
all_behind_items_visible:
type: boolean
description: All items with changes behind are visible by the user of the request.
skipped_comparison:
type: boolean
description: Whether the comparison was skipped. This happens with old forks that where not being kept track of
diffs:
type: array
description: List of differences found between workspaces
items:
$ref: "#/components/schemas/WorkspaceItemDiff"
summary:
$ref: "#/components/schemas/CompareSummary"
description: Summary statistics of the comparison
WorkspaceItemDiff:
type: object
required:
- kind
- path
- ahead
- behind
- has_changes
- exists_in_source
- exists_in_fork
properties:
kind:
type: string
enum: ["script", "flow", "app", "resource", "variable"]
description: Type of the item
path:
type: string
description: Path of the item in the workspace
ahead:
type: integer
description: Number of versions source is ahead of target
behind:
type: integer
description: Number of versions source is behind target
has_changes:
type: boolean
description: Whether the item has any differences
exists_in_source:
type: boolean
description: If the item exists in the source workspace
exists_in_fork:
type: boolean
description: If the item exists in the fork workspace
CompareSummary:
type: object
required:
- total_diffs
- total_ahead
- total_behind
- scripts_changed
- flows_changed
- apps_changed
- resources_changed
- variables_changed
- conflicts
properties:
total_diffs:
type: integer
description: Total number of items with differences
total_ahead:
type: integer
description: Total number of ahead changes
total_behind:
type: integer
description: Total number of behind changes
scripts_changed:
type: integer
description: Number of scripts with differences
flows_changed:
type: integer
description: Number of flows with differences
apps_changed:
type: integer
description: Number of apps with differences
resources_changed:
type: integer
description: Number of resources with differences
variables_changed:
type: integer
description: Number of variables with differences
conflicts:
type: integer
description: Number of items that are both ahead and behind (conflicts)
TeamInfo:
type: object
required:
@@ -21002,6 +21239,9 @@ components:
per_page:
type: number
description: Number of repositories loaded per page
error:
type: string
description: Error message if token retrieval failed
required:
- installation_id
- account_id

View File

@@ -133,6 +133,10 @@ struct AIStandardResource {
api_key: Option<String>,
organization_id: Option<String>,
region: Option<String>,
#[serde(alias = "awsAccessKeyId")]
aws_access_key_id: Option<String>,
#[serde(alias = "awsSecretAccessKey")]
aws_secret_access_key: Option<String>,
}
#[derive(Deserialize, Debug)]
@@ -154,6 +158,9 @@ struct AIRequestConfig {
pub access_token: Option<String>,
pub organization_id: Option<String>,
pub user: Option<String>,
pub region: Option<String>,
pub aws_access_key_id: Option<String>,
pub aws_secret_access_key: Option<String>,
}
impl AIRequestConfig {
@@ -163,8 +170,18 @@ impl AIRequestConfig {
w_id: &str,
resource: AIResource,
) -> Result<Self> {
let (api_key, access_token, organization_id, base_url, user) = match resource {
let (
api_key,
access_token,
organization_id,
base_url,
user,
region,
aws_access_key_id,
aws_secret_access_key,
) = match resource {
AIResource::Standard(resource) => {
let region = resource.region.clone();
let base_url = provider
.get_base_url(resource.base_url, resource.region, db)
.await?;
@@ -178,8 +195,28 @@ impl AIRequestConfig {
} else {
None
};
let aws_access_key_id = if let Some(access_key_id) = resource.aws_access_key_id {
Some(get_variable_or_self(access_key_id, db, w_id).await?)
} else {
None
};
let aws_secret_access_key =
if let Some(secret_access_key) = resource.aws_secret_access_key {
Some(get_variable_or_self(secret_access_key, db, w_id).await?)
} else {
None
};
(api_key, None, organization_id, base_url, None)
(
api_key,
None,
organization_id,
base_url,
None,
region,
aws_access_key_id,
aws_secret_access_key,
)
}
AIResource::OAuth(resource) => {
let user = if let Some(user) = resource.user.clone() {
@@ -190,11 +227,20 @@ impl AIRequestConfig {
let token = Self::get_token_using_oauth(resource, db, w_id).await?;
let base_url = provider.get_base_url(None, None, db).await?;
(None, Some(token), None, base_url, user)
(None, Some(token), None, base_url, user, None, None, None)
}
};
Ok(Self { base_url, organization_id, api_key, access_token, user })
Ok(Self {
base_url,
organization_id,
api_key,
access_token,
user,
region,
aws_access_key_id,
aws_secret_access_key,
})
}
async fn get_token_using_oauth(
@@ -251,6 +297,10 @@ impl AIRequestConfig {
let is_anthropic_sdk = headers.get("X-Anthropic-SDK").is_some();
let is_bedrock = matches!(provider, AIProvider::AWSBedrock);
// Check if using IAM credentials for Bedrock (instead of bearer token)
let use_iam_auth =
is_bedrock && self.aws_access_key_id.is_some() && self.aws_secret_access_key.is_some();
// Handle AWS Bedrock transformation
let (url, body) = if is_bedrock && method != Method::GET {
let (model, transformed_body, is_streaming) =
@@ -282,7 +332,7 @@ impl AIRequestConfig {
tracing::debug!("AI request URL: {}", url);
let mut request = HTTP_CLIENT
.request(method, url)
.request(method.clone(), &url)
.header("content-type", "application/json");
for (header_name, header_value) in headers.iter() {
@@ -291,23 +341,43 @@ impl AIRequestConfig {
}
}
// For Bedrock with IAM credentials, sign the request using SigV4
if use_iam_auth {
let region = self.region.as_deref().ok_or_else(|| {
Error::internal_err("AWS region must be set for IAM authentication with Bedrock")
})?;
let signed_headers = bedrock::sign_bedrock_request(
method.as_str(),
&url,
&body,
self.aws_access_key_id.as_ref().unwrap(),
self.aws_secret_access_key.as_ref().unwrap(),
region,
)?;
for (header_name, header_value) in signed_headers {
request = request.header(header_name, header_value);
}
} else {
// For non-IAM auth, use bearer token or API key
if let Some(api_key) = self.api_key {
if is_azure {
request = request.header("api-key", api_key.clone())
} else {
request = request.header("authorization", format!("Bearer {}", api_key.clone()))
}
if is_anthropic {
request = request.header("X-API-Key", api_key);
}
}
if let Some(access_token) = self.access_token {
request = request.header("authorization", format!("Bearer {}", access_token))
}
}
request = request.body(body);
if let Some(api_key) = self.api_key {
if is_azure {
request = request.header("api-key", api_key.clone())
} else {
request = request.header("authorization", format!("Bearer {}", api_key.clone()))
}
if is_anthropic {
request = request.header("X-API-Key", api_key);
}
}
if let Some(access_token) = self.access_token {
request = request.header("authorization", format!("Bearer {}", access_token))
}
if let Some(org_id) = self.organization_id {
request = request.header("OpenAI-Organization", org_id);
}

View File

@@ -1,9 +1,73 @@
use axum::body::Bytes;
use aws_sigv4::http_request::{sign, SignableBody, SignableRequest, SigningSettings};
use aws_sigv4::sign::v4;
use bytes;
use futures;
use std::time::SystemTime;
use uuid;
use windmill_common::error::{Error, Result};
/// Sign a request for AWS Bedrock using SigV4
///
/// Returns a vector of (header_name, header_value) tuples to add to the request
pub fn sign_bedrock_request(
method: &str,
uri: &str,
body: &[u8],
access_key_id: &str,
secret_access_key: &str,
region: &str,
) -> Result<Vec<(String, String)>> {
let identity = aws_credential_types::Credentials::new(
access_key_id,
secret_access_key,
None, // session token
None, // expiration
"windmill",
)
.into();
let signing_settings = SigningSettings::default();
let signing_params = v4::SigningParams::builder()
.identity(&identity)
.region(region)
.name("bedrock")
.time(SystemTime::now())
.settings(signing_settings)
.build()
.map_err(|e| Error::internal_err(format!("Failed to build signing params: {}", e)))?;
// Parse the URI to extract path and query
let parsed_uri: http::Uri = uri
.parse()
.map_err(|e| Error::internal_err(format!("Failed to parse URI: {}", e)))?;
let path_and_query = parsed_uri
.path_and_query()
.map(|pq| pq.as_str())
.unwrap_or("/");
let signable_request = SignableRequest::new(
method,
path_and_query,
std::iter::once(("host", parsed_uri.host().unwrap_or(""))),
SignableBody::Bytes(body),
)
.map_err(|e| Error::internal_err(format!("Failed to create signable request: {}", e)))?;
let (signing_instructions, _signature) = sign(signable_request, &signing_params.into())
.map_err(|e| Error::internal_err(format!("Failed to sign request: {}", e)))?
.into_parts();
// Collect the headers to add
let mut headers = Vec::new();
for (name, value) in signing_instructions.headers() {
headers.push((name.to_string(), value.to_string()));
}
Ok(headers)
}
/// Transform OpenAI format request to AWS Bedrock Converse format
/// Returns: (model_id, transformed_body, is_streaming)
pub fn transform_openai_to_bedrock(body: &[u8]) -> Result<(String, Bytes, bool)> {

View File

@@ -32,6 +32,7 @@ use sql_builder::prelude::*;
use sqlx::{FromRow, Postgres, Transaction};
use windmill_audit::audit_oss::audit_log;
use windmill_audit::ActionKind;
use windmill_common::runnable_settings::RunnableSettingsTrait;
use windmill_common::utils::{query_elems_from_hub, WarnAfterExt};
use windmill_common::worker::{to_raw_value, CLOUD_HOSTED, MIN_VERSION_SUPPORTS_DEBOUNCING};
use windmill_common::HUB_BASE_URL;
@@ -1601,7 +1602,9 @@ mod tests {
ConstantDelay, ExponentialDelay, FlowModule, FlowModuleValue, FlowValue,
InputTransform, Retry, StopAfterIf,
},
jobs::{ConcurrencySettings, ConcurrencySettingsWithCustom, DebouncingSettings},
runnable_settings::{
ConcurrencySettings, ConcurrencySettingsWithCustom, DebouncingSettings,
},
scripts,
};

View File

@@ -0,0 +1,68 @@
/*
* Author: Ruben Fiszel
* Copyright: Windmill Labs, Inc 2022
* This file and its contents are licensed under the AGPLv3 License.
* Please see the included NOTICE for copyright information and
* LICENSE-AGPL for a copy of the license.
*/
use crate::db::ApiAuthed;
use axum::{
extract::{Extension, Path, Query},
routing::get,
Router,
};
use windmill_common::{
db::UserDB,
error::JsonResult,
utils::{paginate, Pagination},
};
use serde::Serialize;
use sqlx::FromRow;
pub fn workspaced_service() -> Router {
Router::new().route("/get/:name", get(get_folder_permission_history))
}
#[derive(Serialize, FromRow)]
pub struct FolderPermissionChange {
pub id: i64,
pub changed_by: String,
pub changed_at: chrono::DateTime<chrono::Utc>,
pub change_type: String,
pub affected: Option<String>,
}
async fn get_folder_permission_history(
authed: ApiAuthed,
Extension(user_db): Extension<UserDB>,
Path((w_id, name)): Path<(String, String)>,
Query(pagination): Query<Pagination>,
) -> JsonResult<Vec<FolderPermissionChange>> {
// Check if user is owner of the folder (before starting transaction for performance)
crate::folders::require_is_owner(&authed, &name)?;
let mut tx = user_db.begin(&authed).await?;
let (per_page, offset) = paginate(pagination);
let history = sqlx::query_as!(
FolderPermissionChange,
"SELECT id, changed_by, changed_at, change_type, affected
FROM folder_permission_history
WHERE workspace_id = $1 AND folder_name = $2
ORDER BY id DESC
LIMIT $3 OFFSET $4",
w_id,
name,
per_page as i64,
offset as i64
)
.fetch_all(&mut *tx)
.await?;
tx.commit().await?;
Ok(axum::Json(history))
}

Some files were not shown because too many files have changed in this diff Show More