Compare commits

...

105 Commits

Author SHA1 Message Date
pyranota
059e8db67f add pg_activity to flake.nix
Signed-off-by: pyranota <pyra@duck.com>
2025-12-22 21:25:55 +01:00
centdix
68b4dc3971 add agents (#7429) 2025-12-22 15:05:40 +02:00
HugoCasa
ed4aa0a6ce chore: remove outdated llm folder 2025-12-22 12:43:43 +01:00
claude[bot]
6be060bea8 feat(ai): add websearch tool for AI agents (#7399)
* refactor(ai): use responses API for OpenAI/Azure, create 'other' provider for completion endpoint

- Created new 'other.rs' provider using the OpenAI-compatible completion endpoint
- Refactored 'openai.rs' to use the Responses API for both text and image output
- Updated query_builder to route OpenAI/AzureOpenAI to OpenAIQueryBuilder
- All other providers (Mistral, DeepSeek, Groq, etc.) now use OtherQueryBuilder
- Updated OpenRouter to delegate to OtherQueryBuilder instead of OpenAIQueryBuilder

This prepares the codebase for adding websearch tool support using the Responses API.

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* feat(ai): add websearch as AI agent tool type

- Added WebsearchToolValue to ToolValue enum in flows.rs
- Updated all pattern matches to handle websearch tool type
- Added has_websearch parameter to run_agent function
- Websearch tools don't require additional configuration

This prepares the backend for provider-specific websearch implementations.

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* feat(frontend): add websearch as tool option in AI agent UI

- Added WebsearchTool type and utility functions to agentToolUtils.ts
- Added "Web Search" option in tool insertion menu (InsertModuleInner)
- Updated NewAIToolNode to handle pickWebsearchTool event
- Updated AIToolNode to render websearch tools properly
- Updated FlowModuleSchemaMap to create websearch tools

Users can now add websearch tools to AI agents through the UI.

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* feat(ai): implement websearch for OpenAI provider

- Added has_websearch parameter to BuildRequestArgs
- OpenAI provider now adds web_search tool to requests when enabled
- Uses OpenAI Responses API web_search tool type
- Websearch tool is added before other custom tools in the request

Implements websearch functionality for OpenAI and Azure OpenAI providers.

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* feat(ai): implement websearch for Anthropic provider

- Created anthropic.rs provider with native Anthropic API format
- Added web_search tool to Anthropic requests when enabled
- Anthropic uses /messages endpoint with x-api-key authentication
- Updated query_builder to route Anthropic to new provider

Implements websearch functionality for Anthropic Claude models.

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* feat(ai): implement websearch for Gemini/GoogleAI provider

- GoogleAI now uses completion endpoint (other.rs) for text instead of responses API
- Added Google Search grounding when websearch is enabled
- Uses google_search_retrieval tool in request when has_websearch is true
- Updated parse methods to use OtherQueryBuilder for completion endpoint

Implements websearch functionality for Google Gemini models.

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* fix frontend

* fix anthropic and openai

* better for gemini

* structured output

* cleaning

* fix validate tool

* fixes

* cleaning

* cleaning

* fix for openai

* no responses api for azure

* fixes

* fix

* add tests for ai agent

* avoid panic

* better tests

* test user images

* fix tool choice

* always use streaming backend side

* big cleaning

* show annotations plus agent action for open ai websearch use

* show annotations plus agent action for anthropic websearch use

* show annotations plus agent action for google websearch use

* nit forntend

* rm

* fix

* add test for image ouptut

* fix for azure

* add in openflow

* fix

* fix

* nit tests

* fixes

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: centdix <farhadg110@gmail.com>
2025-12-22 11:41:28 +00:00
Diego Imbert
980dfcc366 fix: clear app form on submit option (#7428) 2025-12-22 10:40:11 +00:00
centdix
5f2101a32b feat(aiagent): allow giving messages history (#7395)
* handle messages array for ai agent

* better

* nit

* make tool_calls and tool_call_id nullable

* fix empty json behavior

* nits

* cleaning

* feat(backend): replace messages/messages_context_length with history oneOf field

Replace the separate 'messages' array and 'messages_context_length' fields with
a single 'history' field that uses a oneOf discriminator.

The 'history' field can be either:
- 'auto' mode: automatically manages conversation history with memory, takes a
  'context_length' number parameter
- 'manual' mode: bypasses memory and uses explicitly provided messages array

Backward compatibility is maintained: if 'messages_context_length' is provided
in the old schema format, it is automatically converted to 'auto' mode with the
specified context_length.

Co-authored-by: centdix <centdix@users.noreply.github.com>

* feat(frontend): replace messages/messages_context_length with history oneOf field

Replace the separate 'messages' array and 'messages_context_length' fields with
a single 'history' field in the AI agent schema.

The 'history' field uses a oneOf discriminator with two modes:
- 'auto': { mode: 'auto', context_length: number } - automatically manages
  conversation history with memory
- 'manual': { mode: 'manual', messages: array } - bypasses memory and uses
  explicitly provided messages

The schema includes comprehensive descriptions for each mode explaining the
behavior. The order array has been updated to include 'history' in place of
the old 'messages_context_length' and 'messages' fields.

Co-authored-by: centdix <centdix@users.noreply.github.com>

* fix(frontend): add support for 'mode' discriminator in oneOf rendering

Update ArgInput.svelte to properly handle oneOf schemas that use 'mode' as the
discriminator field, in addition to the existing 'kind' and 'label' support.

Changes:
- Updated tagKey derivation to check for 'mode' first, then 'kind', then 'label'
- Added 'mode' to the onOneOfChange function to track mode changes
- Added 'mode' to the list of keys excluded from enum validation
- Added 'mode' to hiddenArgs to prevent it from being shown in the form
- Added title fields to the history oneOf variants in flowInfers.ts

This allows the AI agent's history field to properly render with toggle buttons
for 'auto' and 'manual' modes.

Co-authored-by: centdix <centdix@users.noreply.github.com>

* fixes

* frontend fix

* nit

* cleaning

* cleaning

* better

* reword

* reword

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: centdix <centdix@users.noreply.github.com>
2025-12-20 15:43:44 +02:00
Ruben Fiszel
f830a9d0fb chore(main): release 1.596.0 (#7422)
* chore(main): release 1.596.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-20 10:03:58 +00:00
Ruben Fiszel
81e648055b reduce dockerfile slim deps 2025-12-20 09:34:45 +00:00
Diego Imbert
afe74f74fa fix: improve MS SQL Numeric rounding (#7404)
* Fix MS SQL Numeric rounding

* Fix result collection to avoid JSON parsing

* don't reparse json through sqlx, pass raw string to postgres

* Revert "don't reparse json through sqlx, pass raw string to postgres"

This reverts commit 355691fa7d.

* mistake

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-12-20 09:18:46 +00:00
Ruben Fiszel
d544da342c fix: update to astral-tokio-tar for CVE (#7423)
* fix: Migrate to astral-tokio-tar to address CVE-2025-62518

Migrates from the abandoned tokio-tar library to the actively maintained astral-tokio-tar to address CVE-2025-62518 (TARmageddon), a high-severity RCE vulnerability (CVSS 8.1). The vulnerability involves a boundary-parsing bug in PAX/ustar header handling that enables file smuggling attacks via malicious TAR archives.

This is a drop-in replacement requiring only dependency updates in Cargo.toml files. The astral-tokio-tar library uses the same tokio_tar module name, so no source code changes are needed. All references to the vulnerable tokio-tar v0.3.1 have been removed from Cargo.lock and replaced with the patched astral-tokio-tar v0.5.6.

Related to previous PR #6943 which was closed due to CLA issues. Security disclosure available at https://edera.dev/stories/tarmageddon and patch details at https://github.com/astral-sh/tokio-tar/releases/tag/v0.5.6

* update

---------

Co-authored-by: Devdatta Talele <devtalele0@gmail.com>
Co-authored-by: Devdatta Talele <50290838+devdattatalele@users.noreply.github.com>
2025-12-20 09:04:36 +00:00
Diego Imbert
3affbb3321 feat: type-checked data tables v0 (#7381)
* data tables settings ui

* install runed

* zod 4 fixes

* use new toJSONSchema

* Migrate ducklake catalogs to more generic custom instance databases

* fix compilation

* Safety conversion for old duckdb ffi

* data tables settings

* ts client basis

* inline run works

* datatables work

* Revert "datatables work"

This reverts commit 6e1588d59e.

* datatables work (without leaking pg credentials)

* println

* separate sqlUtils.ts

* nit

* Separate custom instance db Select and Wizard components

* nit

* nit wording

* add tags to custom instance dbs

* error when trying to use ducklake as datatable or opposite

* show status in dropdown

* data table instance setup works

* sqk function for ducklake

* factorize logic

* fix temp reactivity

* Data table assetexplore

* Migrate S3 permissions to modal

* Revert "Migrate S3 permissions to modal"

This reverts commit 0631d03cb0.

* nit query -> fetch

* Custom instance setup new look

* run_language_executor separate fn

* run_inline param

* nit wording

* Better typed client

* Data tables display as assets in frontend

* asset db icon

* nit

* cleaner errors

* nit

* Fix sed calls in mac

* run_inline_script_preview in python client

* basic python datatable client

* datatable and datalake parser in python

* ducklake client python

* nit fix

* Fix migration producing NULL instead of {} when no custom databases

* merge conflict fail

* python ducklake client arg fix

* parse or infer sql types in ts client

* ts asset parser, detect datatable & ducklake R/W

* fix sql repl for other read ops than select

* export type SqlTemplateFunction

* rename list_custom_instance_pg_databases

* typecheck datatable and ducklake name in Typescript

* Fix typecheck datatable and ducklake in TS

* declare module overriding instead of extending

* infer_sql_type in python client

* SqlQuery object in python

* fix merge conflicts

* update const_format

* CI fix

* factor out to var_identifiers

* sqlx prepare

* unnecessary security (admin is required)

* clearer comment

* ee repo ref

* nit snake case

* claude step 1: detect var declarations

* move detect_sql_access_type to common mod

* claude step 2: detect when saved vars are queried

* Revert "claude step 2: detect when saved vars are queried"

This reverts commit 1e1f930568.

* Revert "claude step 1: detect var declarations"

This reverts commit f866f4819d.

* remove ducklake/datatable and default

* detect data table assigns in var_identifiers

* Python parser successfully infers R/W/RW from ducklake / datatable

* still register ducklake/datatable if not used as unknown R/W

* Go to settings button in Assets Dropdown on not found

* nit

* sqlx prepare fail

* manual fix, somehow sqlx prepare won't do it

* fix frontend ci

* ee repo ref

* ducklake_user doesnt exist in unit tests

* nit fix

* ui nit

* nit

* nit missing clone

* fork ducklakes and datatables

* fix surface hover bug

* stupid mistake

* better deeply reactive mutable derived

* Ducklake picker

* Editor bar data tables

* DuckDB supports datatables

* datatable in duckdb asset parser

* duckdb asset parser var_identifiers

* Revert "duckdb asset parser var_identifiers"

This reverts commit 88068b1a77.

* sqlx prepare

* Box pin in test_workflow_as_code to fix stack overflow

* stash

* sql asset parser parses most s3 literals

* nit

* Detect attach + handle returning RW

* detect assets used with dot notation

* detect implicit access with USE dl; syntax

* Add assets as unknown if var was never used

* Support default ducklake/datatable main in parser

* ignore asset parsing errors in frontend (avoid flow layout shift)

* super weird duplication (merge conflict ?)

* nits

* fix duckdb parser detecting too much as asset when RW ctx is unknown

* fix transparent assets btn

* missing arg

* nit styling

* asset parser specific table parsing

* fix resource specific table parsing

* More concise asset display in flows + better icons

* fix assets page filtering out resources with added table

* Fix frontend to support specific table assets

* Open DB Manager to specific table

* Specific table parser in Python and TS + unit tests

* Fix UPDATE setting access to None

* fix flow edge rendering on top of output picker

* python parser fix var override bug

* add ts test

* fix compilation

* sqlx prepare

* update parsers version

* fix missing schema key onDelete

* Grant permission to create schemas in custom instance databases

* Update pg query to return empty schemas

* Create schema

* Select nits

* support schemas in sql parser

* ts parser handle schema with sql parser result

* detect .schema() syntax

* detect schema syntax in python

* support .schema() in ts and py SDK

* open db manager to specific schema

* support reassignment in ts parser

* nit better unitest

* : syntax in ts

* datatable:schema syntax in python

* fix client py

* nit select dropdown darkmode

* object | null fetchOne

* ts client nits

* parse_sql_client_name fn

* getImportWmillTsStatement refactor in EditorBar

* text to json() in python client

* update parser versions

* pkg lock

* Sql query details in TS asset parser

* code transformation with type parameter in Editor

* Custom Language Worker, code substition works !

* Error marker mapping works

* hover info is correct

* completions work correctly

* other overrides

* type inference kinda works

* Position mapping tests

* refactor prepare_queries

* Refactor PgDatabase to share common code

* Pgdatabase in prepare_queries

* TokioPgConnection refactor

* refactor prepare_queries

* type parameter to sql function

* Fix deadlock

* nit fix

* Fix worker async call freezing because of svelte Proxy

* Force worker to recompute when we set queries

* nit refactor

* nits console logs

* wait that ts worker initialize

* monaco change file version

* update diagnostics

* Refactor for errors

* Show SQL errors in Monaco

* improve sdk

* cleaning refactor + MapResource + usePreparedAssetSqlQueries

* Fixes

* Fix error position mapping

* cache in typescript worker

* fix insert no values

* don't inject type if already present

* Support schema in prepare queries

* update parsers

* ChangeOnDeepInequality

* inferAsset ScriptEditor usage refactor

* sql query typecheck work in flow editor

* Assets and SQL Query check in Raw App Inline Editor

* pkg lock

* Fix DatatableSqlTemplateFunction nit

* prepare query schema nit

* duplicate diagnostics

* nit getScriptVersion mock

* Reprepare queries when switching workspaces

* nit fix

* nit fix

* fetch_one_scalar and execute in python client

* limit pg_connections

* -- prepare flag in postgres

* skip serializing

* fix destructuring undefined

* Prepare queries in workers instead of backend

* nit

* Execute search_path instructions normally

* nit fix

* Fix SET search_path issue in prepare

* only support preparing single-statement queries for now

* update parsers

* safety

* better remove_comments

* Fix getQueryStmtCountHeuristic

* getQueryStmtCountHeuristic tests

* comment out failing tests

* Fix getQueryStmtCountHeuristic impl

* only datatable
2025-12-20 08:51:07 +00:00
Alexander Petric
cdd5d9fa9a fix: improve error msg for unshare error (#7421) 2025-12-20 01:14:06 +02:00
Ruben Fiszel
d86d233843 chore(main): release 1.595.0 (#7416)
* chore(main): release 1.595.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-19 21:10:51 +00:00
Ruben Fiszel
2e4d6ad7a1 update python parser 2025-12-19 21:07:46 +00:00
Avigail Royzenberg
da500fcf3e fix: Python Enum types generate proper dropdown schemas with descriptions (#7400)
* Fix Python Enum and Literal schema generation with docstring descriptions

- Extract Enum class definitions and their string values
- Parse docstring Args: sections for parameter descriptions
- Map Enum type annotations to string enums with proper values
- Handle Enum.VALUE default values correctly
- Store descriptions in Arg.otyp field
- Add test case for enum with docstring parsing

* perf: optimize enum parser and fix default value handling

- Combine enum extraction and docstring parsing into single AST pass (2x performance improvement)
- Add support for IntEnum, StrEnum, Flag, IntFlag types
- Fix default values to use actual enum values (e.g., 'red') instead of member names (e.g., 'RED')
- Improve docstring parsing robustness with proper indentation tracking
- Clean up code structure with EnumInfo type for better maintainability

All tests pass. This addresses code review feedback for performance and correctness.

* perf: implement true lazy evaluation for enum parsing

- Only parse metadata when unknown types encountered
- Two-pass approach: parse types first, extract only if needed
- Zero overhead for scripts without enums
- Keyword checks + prepass filtering when extraction needed
2025-12-19 20:41:34 +00:00
Guilhem
210b8285d4 fix(frontend): settings redesign (#7406)
* improve collapsible link

* do not show superadmin ws link when already in it

* improve OAuth UI

* sso/oauth instance settings ui

* refactor instance settings alerts WIP

* Indexer and Oauth to brand guidelines

* refactor ws error handler page

* Create a tab SMTP in the Instance Settings

* Ractivity isssue fix for tabs

* nit

* Add smtp settings status in Error handler

* Add smtp configuration status

* Display teams connection status for instance alerts

* nit

* Add critical alerts description

* nit

* nit

* improve ee display

* nit

* nit

* fix typo

* nit

* restore vit config

---------

Co-authored-by: Alexander Petric <alex@windmill.dev>
2025-12-19 20:33:03 +00:00
Alexander Petric
8268354889 fix: teams, need both guid and thread id format (#7420)
* fix: teams, need both guid and thread id format

* chore: update ee-repo-ref to 576abf6519d1aa12a2b989a58a123501206284fb

This commit updates the EE repository reference after PR #368 was merged in windmill-ee-private.

Previous ee-repo-ref: ae41589212f16ff0cec8516c66227f30e17c5564

New ee-repo-ref: 576abf6519d1aa12a2b989a58a123501206284fb

Automated by sync-ee-ref workflow.

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-19 20:07:23 +00:00
centdix
3e2565f710 fix flow not sent (#7417) 2025-12-19 20:01:41 +00:00
hugocasa
f89fb292da fix(backend): put for loop itered in a separate table (#7419)
* fix(backend): put for loop itered in a separate table

* Update SQLx metadata

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-19 19:46:46 +00:00
Alexander Petric
c28e77110e fix: do not use unshare for init scripts (#7418) 2025-12-19 19:36:51 +00:00
hugocasa
0bf7407419 feat: email triggers custom cert (#7415)
* feat: email triggers custom cert

* chore: update ee-repo-ref to 6edb5e9ea22b88f81dc9ee2741ea679d9b22f61c

This commit updates the EE repository reference after PR #364 was merged in windmill-ee-private.

Previous ee-repo-ref: 4053446d2dff0310ecb89cf6e37f00b49217443a

New ee-repo-ref: 6edb5e9ea22b88f81dc9ee2741ea679d9b22f61c

Automated by sync-ee-ref workflow.

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-19 14:23:09 +00:00
Ruben Fiszel
330c321fe9 chore(main): release 1.594.0 (#7413)
* chore(main): release 1.594.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-19 13:08:32 +00:00
centdix
4f04d4a18c use opus (#7398) 2025-12-19 13:07:00 +00:00
Alexander Petric
96aaceef95 fix: improve teams search ux (#7407)
* feat: improve teams search ux

* ee ref

* claude review

* chore: update ee-repo-ref to e218dfce97dcea56c6ef6032592dab812a3f5047

This commit updates the EE repository reference after PR #363 was merged in windmill-ee-private.

Previous ee-repo-ref: 1b95a24ab25d96e59d2f22588901e9d3ce6c72b3

New ee-repo-ref: e218dfce97dcea56c6ef6032592dab812a3f5047

Automated by sync-ee-ref workflow.

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-19 13:03:15 +00:00
Ruben Fiszel
5f182bc28a nit warns 2025-12-19 12:57:06 +00:00
Ruben Fiszel
ff0d1d444e cgroups only on linux 2025-12-19 12:48:15 +00:00
Ruben Fiszel
806024403e fix: disable oomgroup by default 2025-12-19 12:45:21 +00:00
hugocasa
a6993823af feat: restart flow from step with different flow version (#7409)
* feat: restart flow from step with different flow version

* fix tests

* fix tests
2025-12-18 20:27:51 +00:00
hugocasa
0fe7a2a17e fix(backend): correctly apply preprocessor step tag (#7412) 2025-12-18 20:21:59 +00:00
Ruben Fiszel
2c6dad2f06 chore(main): release 1.593.1 (#7408)
* chore(main): release 1.593.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-18 16:30:13 +00:00
Ruben Fiszel
c9a19f12d6 fix: fix folder/group history seq id grant issues 2025-12-18 16:17:33 +00:00
Ruben Fiszel
81b88fcd24 chore(main): release 1.593.0 (#7361)
* chore(main): release 1.593.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-17 13:52:45 +00:00
centdix
97f15796e8 remove null format (#7402) 2025-12-17 12:46:27 +00:00
Ruben Fiszel
9385dba5c4 slim image 2025-12-17 12:33:44 +00:00
Ruben Fiszel
67e96cca9d slim nits 2025-12-17 12:09:27 +00:00
Diego Imbert
ca45937166 Refactor PgDatabase to share common code (#7386)
* Refactor PgDatabase to share common code

* TokioPgConnection refactor
2025-12-17 11:34:39 +00:00
Diego Imbert
bbf97765f1 sql.query(SELECT $1, $2, arg1, arg2) syntax in typescript (#7373) 2025-12-17 11:34:20 +00:00
Frédéric Crozatier
53c325a652 add inspector (#7370) 2025-12-17 08:40:36 +00:00
Ruben Fiszel
296c11c75a rework slim images 2025-12-17 08:37:12 +00:00
Jakub Drobník
3d34634a66 update apify logo (#7401) 2025-12-17 08:03:30 +00:00
Ruben Fiszel
5d55e8d517 use sccache for duckdb build 2025-12-16 22:55:02 +00:00
Ruben Fiszel
0a25416a96 update lsp base images node version 2025-12-16 22:31:50 +00:00
centdix
8c55f61bba feat(ai): support IAM auth for bedrock provider (#7379)
* support iam for bedrock ai

* lock

* cleaning
2025-12-16 22:00:14 +00:00
Pyra
fe56191422 feat(internal): runnable settings (#7298)
* rework everything again

Signed-off-by: pyranota <pyra@duck.com>

* updcate sqlx

Signed-off-by: pyranota <pyra@duck.com>

* update ref

Signed-off-by: pyranota <pyra@duck.com>

* fix things

Signed-off-by: pyranota <pyra@duck.com>

* fix function

Signed-off-by: pyranota <pyra@duck.com>

* final fixes

Signed-off-by: pyranota <pyra@duck.com>

* update sqlx

Signed-off-by: pyranota <pyra@duck.com>

* fix script creation

Signed-off-by: pyranota <pyra@duck.com>

* address todo

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* remove dbg

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fix

Signed-off-by: pyranota <pyra@duck.com>

* fixups

Signed-off-by: pyranota <pyra@duck.com>

* fix cargo.toml

Signed-off-by: pyranota <pyra@duck.com>

* update ee repo

Signed-off-by: pyranota <pyra@duck.com>

* fix ci

Signed-off-by: pyranota <pyra@duck.com>

* nit

Signed-off-by: pyranota <pyra@duck.com>

* ref

Signed-off-by: pyranota <pyra@duck.com>

* fix

Signed-off-by: pyranota <pyra@duck.com>

* ee repo

Signed-off-by: pyranota <pyra@duck.com>

* sqlx

Signed-off-by: pyranota <pyra@duck.com>

* ee ref

Signed-off-by: pyranota <pyra@duck.com>

* remove dbg

Signed-off-by: pyranota <pyra@duck.com>

* sqlx

Signed-off-by: pyranota <pyra@duck.com>

* chore: update ee-repo-ref to 505eadbff32d102ea5245a2bef88ce6f1bb95395

This commit updates the EE repository reference after PR #348 was merged in windmill-ee-private.

Previous ee-repo-ref: 195243e56cc0eab55f8890fa57297206bfe2c18c

New ee-repo-ref: 505eadbff32d102ea5245a2bef88ce6f1bb95395

Automated by sync-ee-ref workflow.

* ci: force runnable settings

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-16 21:17:06 +00:00
Diego Imbert
75fdc2cdc9 feat: data table schemas (#7353)
* data tables settings ui

* install runed

* zod 4 fixes

* use new toJSONSchema

* Migrate ducklake catalogs to more generic custom instance databases

* fix compilation

* Safety conversion for old duckdb ffi

* data tables settings

* ts client basis

* inline run works

* datatables work

* Revert "datatables work"

This reverts commit 6e1588d59e.

* datatables work (without leaking pg credentials)

* println

* separate sqlUtils.ts

* nit

* Separate custom instance db Select and Wizard components

* nit

* nit wording

* add tags to custom instance dbs

* error when trying to use ducklake as datatable or opposite

* show status in dropdown

* data table instance setup works

* sqk function for ducklake

* factorize logic

* fix temp reactivity

* Data table assetexplore

* Migrate S3 permissions to modal

* Revert "Migrate S3 permissions to modal"

This reverts commit 0631d03cb0.

* nit query -> fetch

* Custom instance setup new look

* run_language_executor separate fn

* run_inline param

* nit wording

* Better typed client

* Data tables display as assets in frontend

* asset db icon

* nit

* cleaner errors

* nit

* Fix sed calls in mac

* run_inline_script_preview in python client

* basic python datatable client

* datatable and datalake parser in python

* ducklake client python

* nit fix

* Fix migration producing NULL instead of {} when no custom databases

* merge conflict fail

* python ducklake client arg fix

* parse or infer sql types in ts client

* ts asset parser, detect datatable & ducklake R/W

* fix sql repl for other read ops than select

* export type SqlTemplateFunction

* rename list_custom_instance_pg_databases

* typecheck datatable and ducklake name in Typescript

* Fix typecheck datatable and ducklake in TS

* declare module overriding instead of extending

* infer_sql_type in python client

* SqlQuery object in python

* fix merge conflicts

* update const_format

* CI fix

* factor out to var_identifiers

* sqlx prepare

* unnecessary security (admin is required)

* clearer comment

* ee repo ref

* nit snake case

* claude step 1: detect var declarations

* move detect_sql_access_type to common mod

* claude step 2: detect when saved vars are queried

* Revert "claude step 2: detect when saved vars are queried"

This reverts commit 1e1f930568.

* Revert "claude step 1: detect var declarations"

This reverts commit f866f4819d.

* remove ducklake/datatable and default

* detect data table assigns in var_identifiers

* Python parser successfully infers R/W/RW from ducklake / datatable

* still register ducklake/datatable if not used as unknown R/W

* Go to settings button in Assets Dropdown on not found

* nit

* sqlx prepare fail

* manual fix, somehow sqlx prepare won't do it

* fix frontend ci

* ee repo ref

* ducklake_user doesnt exist in unit tests

* nit fix

* ui nit

* nit

* nit missing clone

* fork ducklakes and datatables

* fix surface hover bug

* stupid mistake

* better deeply reactive mutable derived

* Ducklake picker

* Editor bar data tables

* DuckDB supports datatables

* datatable in duckdb asset parser

* duckdb asset parser var_identifiers

* Revert "duckdb asset parser var_identifiers"

This reverts commit 88068b1a77.

* sqlx prepare

* Box pin in test_workflow_as_code to fix stack overflow

* stash

* sql asset parser parses most s3 literals

* nit

* Detect attach + handle returning RW

* detect assets used with dot notation

* detect implicit access with USE dl; syntax

* Add assets as unknown if var was never used

* Support default ducklake/datatable main in parser

* ignore asset parsing errors in frontend (avoid flow layout shift)

* super weird duplication (merge conflict ?)

* nits

* fix duckdb parser detecting too much as asset when RW ctx is unknown

* fix transparent assets btn

* missing arg

* nit styling

* asset parser specific table parsing

* fix resource specific table parsing

* More concise asset display in flows + better icons

* fix assets page filtering out resources with added table

* Fix frontend to support specific table assets

* Open DB Manager to specific table

* Specific table parser in Python and TS + unit tests

* Fix UPDATE setting access to None

* fix flow edge rendering on top of output picker

* python parser fix var override bug

* add ts test

* fix compilation

* sqlx prepare

* update parsers version

* fix missing schema key onDelete

* Grant permission to create schemas in custom instance databases

* Update pg query to return empty schemas

* Create schema

* Select nits

* support schemas in sql parser

* ts parser handle schema with sql parser result

* detect .schema() syntax

* detect schema syntax in python

* support .schema() in ts and py SDK

* open db manager to specific schema

* support reassignment in ts parser

* nit better unitest

* : syntax in ts

* datatable:schema syntax in python

* fix client py

* nit select dropdown darkmode

* object | null fetchOne

* ts client nits

* parse_sql_client_name fn

* getImportWmillTsStatement refactor in EditorBar

* text to json() in python client

* update parser versions

* pkg lock
2025-12-16 21:16:42 +00:00
Frédéric Crozatier
f98e315a56 chore: remove dead code (#7391)
* remove dead code

* remove unused modules

* explicitly add monaco-vscode-languages-service-override
2025-12-16 19:47:04 +00:00
hugocasa
b4eb7c6ac0 feat: http triggers scopes (#7385) 2025-12-16 19:46:45 +00:00
hugocasa
0454f392e7 fix: propagate canceled_by in flows (#7396)
main reason of this change is to not trigger workspace error handler when a flow is canceled from a substep and error_handler_muted_on_cancel is true
2025-12-16 19:41:07 +00:00
Diego Imbert
be3eac0b26 Allow empty lines in annotations (#7394) 2025-12-16 19:07:19 +02:00
hugocasa
3ba361ad1a fix(backend): better trigger listening logs (#7392)
* fix(backend): better trigger listening logs

* chore: update ee-repo-ref to d347295041426d03039b747a148a71e3583c3a6b

This commit updates the EE repository reference after PR #362 was merged in windmill-ee-private.

Previous ee-repo-ref: 37b533704e1b40e616ac144bebeff574a5d048e1

New ee-repo-ref: d347295041426d03039b747a148a71e3583c3a6b

Automated by sync-ee-ref workflow.

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-16 16:23:21 +00:00
centdix
e7719d2cda fix(aichat): fix for azure responses api not available in some region (#7387)
* fix completion fallback

* cleaning

* cache

* cleaning

* cleaning
2025-12-16 14:45:13 +01:00
centdix
b69d387b02 fix ci (#7388) 2025-12-16 14:22:21 +01:00
centdix
e232feb519 fix structured output for anthropic (#7384) 2025-12-16 14:21:55 +01:00
hugocasa
75e1e90273 fix(frontend): http/email triggers UI nits (#7378) 2025-12-15 22:54:00 +00:00
Alexander Petric
6ffb80d1e1 fix: SCIM 2.0 RFC compliance + displayName support (#7380)
* SCIM 2.0 RFC compliance + displayName support

* chore: update ee-repo-ref to 9a4b392262c760fc52256ca00e4d751d9f42e79e

This commit updates the EE repository reference after PR #361 was merged in windmill-ee-private.

Previous ee-repo-ref: ee9310c785bdf65d5b3136b0a24e0018900e18d0

New ee-repo-ref: 9a4b392262c760fc52256ca00e4d751d9f42e79e

Automated by sync-ee-ref workflow.

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-15 22:53:33 +00:00
centdix
f64d918af6 fix(aiagent): fix gemini-3.0 usage (#7382)
* fix gemini for ai agent

* no clone
2025-12-15 22:53:13 +00:00
centdix
1b86a39051 fix(mcp): fix unresovled schema (#7383) 2025-12-15 22:52:49 +00:00
Ruben Fiszel
d249d0f860 nit 2025-12-15 20:35:51 +00:00
hugocasa
ebc82dbe58 feat(backend): stop schedules and cancel jobs when archiving a workspace (#7377) 2025-12-15 18:03:28 +00:00
Devdatta Talele
b3603d8720 fix: add history directly viewable in folder/group viewer (#7365) 2025-12-15 18:01:07 +00:00
centdix
61a3c81d5d chore(appchat): improve prompt and tools (#7376)
* nit flow

* better prompt

* remove files from user message

* truncated files

* nit

* f
2025-12-15 17:29:06 +00:00
centdix
d229d469a1 chore(appchat): add tests pipeline (#7374)
* draft test app

* gitignore

* add app test pipeline

* add lot of tests

* add variant

* remove unrelated changes

* fix

* fix
2025-12-15 16:53:00 +00:00
Ruben Fiszel
2f5fdd6b3f fix(rawapp): make popup work with runnables 2025-12-14 22:32:22 +00:00
centdix
efe43ca3a8 add claude config (#7366) 2025-12-14 22:27:29 +00:00
centdix
37394d6d53 fix(rawapp): schema for openai (#7364) 2025-12-14 22:27:03 +00:00
Ruben Fiszel
f353b91407 history nits 2025-12-13 13:38:32 +00:00
Ruben Fiszel
431074d249 fix: add history to raw app builder (#7362)
* appHistory

* appHistory

* all

* all

* all

* all

* all

* all

* all

* all

* improvements
2025-12-13 13:32:52 +00:00
Ruben Fiszel
8d2ddad9e6 latest ee ref 2025-12-12 22:53:55 +00:00
Alexander Petric
c2a9ce46be improve github app error handling (#7357)
* improve github app error handling

* npm check

* sqlx

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-12-12 22:53:21 +00:00
wendrul
9d06c152ee feat: workspace forks merge UI (#7333)
* feat: Add workspace diff viewer and deployment UI for forked workspaces

- Add backend endpoint for comparing two workspaces
- Implement comparison logic for scripts, flows, apps, resources, variables
- Create ForkWorkspaceBanner component to detect and display fork status
- Build WorkspaceComparisonDrawer for detailed diff viewing and deployment
- Add DiffViewer component for line-by-line comparisons
- Support bidirectional deployment (fork to parent or parent to fork)
- Add conflict detection for items that are both ahead and behind
- Include delete fork option when no changes remain

Note: Backend implementation requires sqlx prepare to be run for full functionality

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* Fix banner and use wworkspace_diff table

* satisfactory UI WIP

* UI complete

* Deploy button

* Comaprison and reset tally

* compare all types of items

* Show summaries

* Disable buttons during deployment

* Auto select all on entering page

* Change migration to have 'exists_in' cols

* Show new and deleted items

* frontend fixes

* Block delpoyment if changes don't match (new chagnes detected)

* Message to block whe changes are behind

* Skip workspaces pre-migration

* Remove unused code

* Fix apps comparison

* Only return changes where user has visibility

* No deploy button if no access to all changes

* Prepare sqlx

* Remove redundant message

* CI: update ee repo ref

* eereporef bis

* Small tweaks

* Remove unused struct

* Remove unused refactor component

* Fix npm run check

* Remove unused component

* chore: update ee-repo-ref to bbf406edc222199ca2e6076da12c376fb4ff28c5

This commit updates the EE repository reference after PR #359 was merged in windmill-ee-private.

Previous ee-repo-ref: 6aae845c5629ae32da43dbfbdc4566e5bf90fb1e

New ee-repo-ref: bbf406edc222199ca2e6076da12c376fb4ff28c5

Automated by sync-ee-ref workflow.

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-12-12 22:52:26 +00:00
Ruben Fiszel
9ba4b071c8 display git URL info for disabled git sync resources (#7345)
* feat: display git URL info for disabled git sync resources

When a git repo resource is already selected and saved in workspace
settings git sync, the resource picker becomes disabled. This made it
impossible to see what git URL the resource is attached to.

This commit adds:
- Automatic loading of resource info when a saved resource is selected
- Display of git URL alongside the disabled resource picker
- Password/token masking in URLs for security
- Loading state and error handling

Fixes #7338

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* feat: fetch and display variable secrets for git URLs

- Detect :X pattern in git repository resource URLs
- Fetch variable value with decryptSecret for display
- Hide URL display if variable cannot be fetched (permissions/not found)
- Maintain password masking for security

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
2025-12-12 18:27:57 +00:00
Ruben Fiszel
734b6c0587 chore(main): release 1.592.1 (#7360)
* chore(main): release 1.592.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-12 18:20:06 +00:00
centdix
490114d133 fix(ai): improve share system prompts 2025-12-12 18:09:09 +00:00
Ruben Fiszel
af1b85f7d0 chore(main): release 1.592.0 (#7356)
* chore(main): release 1.592.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-12 17:47:18 +00:00
Alexander Petric
8f1343e155 fix: clear datetime input in schedule sets input to null (#7358) 2025-12-12 17:39:50 +00:00
centdix
31e002ad41 feat(ai): standardize and improve system prompts (#7346)
* init

* test in frontend

* copy files

* use in cli

* better

* add desc to sdks

* better

* fix ts parsing

* add docs to ts client

* add docs to python client

* use script prompt in frontend

* regen

* use in flow

* rm

* use in cli, create AGENTS.md instead of cursor rules

* remove apply

* better

* better

* simplify cli

* more docs

* cleaning

* update readme

* generate cli file

* better folder names

* fix ts

* fix multiline
2025-12-12 17:26:19 +00:00
Pyra
68596701f1 fix(bun): deployment error on workspace dependencies (#7355)
Signed-off-by: pyranota <pyra@duck.com>
2025-12-12 17:11:57 +00:00
Ruben Fiszel
1a438e9751 warn after secs bun install 2025-12-12 13:22:16 +00:00
Ruben Fiszel
e2953862af chore(main): release 1.591.4 (#7354)
* chore(main): release 1.591.4

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-12 12:27:45 +00:00
Ruben Fiszel
857adf5b63 fix(app): fix raw scripts forbidden by policy if no args 2025-12-12 12:22:35 +00:00
Ruben Fiszel
aeea39abd6 chore(main): release 1.591.3 (#7344)
* chore(main): release 1.591.3

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-12 00:36:10 +01:00
centdix
4337a8810e fix: fix test up to step (#7348) 2025-12-11 15:34:26 +00:00
hugocasa
c57c5547db include instance uid in ee usage (#7347)
* include instance uid in ee usage

* Update ee-repo-ref.txt

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-11 15:24:51 +00:00
Diego Imbert
c6c7f3415a Specific asset tables (#7323)
* data tables settings ui

* install runed

* zod 4 fixes

* use new toJSONSchema

* Migrate ducklake catalogs to more generic custom instance databases

* fix compilation

* Safety conversion for old duckdb ffi

* data tables settings

* ts client basis

* inline run works

* datatables work

* Revert "datatables work"

This reverts commit 6e1588d59e.

* datatables work (without leaking pg credentials)

* println

* separate sqlUtils.ts

* nit

* Separate custom instance db Select and Wizard components

* nit

* nit wording

* add tags to custom instance dbs

* error when trying to use ducklake as datatable or opposite

* show status in dropdown

* data table instance setup works

* sqk function for ducklake

* factorize logic

* fix temp reactivity

* Data table assetexplore

* Migrate S3 permissions to modal

* Revert "Migrate S3 permissions to modal"

This reverts commit 0631d03cb0.

* nit query -> fetch

* Custom instance setup new look

* run_language_executor separate fn

* run_inline param

* nit wording

* Better typed client

* Data tables display as assets in frontend

* asset db icon

* nit

* cleaner errors

* nit

* Fix sed calls in mac

* run_inline_script_preview in python client

* basic python datatable client

* datatable and datalake parser in python

* ducklake client python

* nit fix

* Fix migration producing NULL instead of {} when no custom databases

* merge conflict fail

* python ducklake client arg fix

* parse or infer sql types in ts client

* ts asset parser, detect datatable & ducklake R/W

* fix sql repl for other read ops than select

* export type SqlTemplateFunction

* rename list_custom_instance_pg_databases

* typecheck datatable and ducklake name in Typescript

* Fix typecheck datatable and ducklake in TS

* declare module overriding instead of extending

* infer_sql_type in python client

* SqlQuery object in python

* fix merge conflicts

* update const_format

* CI fix

* factor out to var_identifiers

* sqlx prepare

* unnecessary security (admin is required)

* clearer comment

* ee repo ref

* nit snake case

* claude step 1: detect var declarations

* move detect_sql_access_type to common mod

* claude step 2: detect when saved vars are queried

* Revert "claude step 2: detect when saved vars are queried"

This reverts commit 1e1f930568.

* Revert "claude step 1: detect var declarations"

This reverts commit f866f4819d.

* remove ducklake/datatable and default

* detect data table assigns in var_identifiers

* Python parser successfully infers R/W/RW from ducklake / datatable

* still register ducklake/datatable if not used as unknown R/W

* Go to settings button in Assets Dropdown on not found

* nit

* sqlx prepare fail

* manual fix, somehow sqlx prepare won't do it

* fix frontend ci

* ee repo ref

* ducklake_user doesnt exist in unit tests

* nit fix

* ui nit

* nit

* nit missing clone

* fork ducklakes and datatables

* fix surface hover bug

* stupid mistake

* better deeply reactive mutable derived

* Ducklake picker

* Editor bar data tables

* DuckDB supports datatables

* datatable in duckdb asset parser

* duckdb asset parser var_identifiers

* Revert "duckdb asset parser var_identifiers"

This reverts commit 88068b1a77.

* sqlx prepare

* Box pin in test_workflow_as_code to fix stack overflow

* stash

* sql asset parser parses most s3 literals

* nit

* Detect attach + handle returning RW

* detect assets used with dot notation

* detect implicit access with USE dl; syntax

* Add assets as unknown if var was never used

* Support default ducklake/datatable main in parser

* ignore asset parsing errors in frontend (avoid flow layout shift)

* super weird duplication (merge conflict ?)

* nits

* fix duckdb parser detecting too much as asset when RW ctx is unknown

* fix transparent assets btn

* missing arg

* nit styling

* asset parser specific table parsing

* fix resource specific table parsing

* More concise asset display in flows + better icons

* fix assets page filtering out resources with added table

* Fix frontend to support specific table assets

* Open DB Manager to specific table

* Specific table parser in Python and TS + unit tests

* Fix UPDATE setting access to None

* fix flow edge rendering on top of output picker

* python parser fix var override bug

* add ts test

* fix compilation

* sqlx prepare

* update parsers version

* fix missing schema key onDelete
2025-12-11 13:34:17 +00:00
centdix
9adde111c4 chore(app): stream app ai chat tools (#7341) 2025-12-11 13:26:59 +00:00
Diego Imbert
2ee00b3c7b fix: S3 SDK nits + Presigned S3 Public URL function (#7342)
* export S3Object + URI / Record in TS SDK

* stash getS3SignedPublicUrls

* getPresignedS3PublicUrls in TS client

* update python client for get_presigned_s3_public_urls
2025-12-11 13:26:30 +00:00
claude[bot]
8fcb9c4292 add hover edit button for resource pickers (#7340)
* feat: add hover edit button for disabled resource pickers

When a resource picker is disabled but has a value selected, show an edit
button on hover that allows editing the selected resource. This provides
a way to modify the resource even when the picker itself is disabled.

Changes:
- Added hover state tracking to ResourcePicker component
- Edit button (Pen icon) appears on hover when picker is disabled and has a value
- Button opens ResourceEditorDrawer for the selected resource
- Added ARIA role="group" for accessibility compliance

Fixes #7339

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* Update ResourcePicker.svelte

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-12-11 13:06:21 +00:00
Ruben Fiszel
355ef401a4 chore(main): release 1.591.2 (#7337)
* chore(main): release 1.591.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-11 07:44:02 +00:00
Ruben Fiszel
99bf866f90 fix(backend): pin reqwest to 0.12.24 for better handling of redirects (#7336) 2025-12-11 07:33:44 +00:00
Ruben Fiszel
babfbe735c chore(main): release 1.591.1 (#7335)
* chore(main): release 1.591.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-10 22:01:59 +00:00
Ruben Fiszel
da65ddd8f8 fix: update git sync to latest cli 2025-12-10 21:57:48 +00:00
Ruben Fiszel
bf614df262 chore(main): release 1.591.0 (#7327)
* chore(main): release 1.591.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-12-10 21:27:34 +00:00
Ruben Fiszel
3470c927ae fix(cli): do not remove schema for normal apps 2025-12-10 21:21:38 +00:00
Diego Imbert
9aadbdc98f Refactor DB Table schema-level operations into a single factory (#7330)
* Refactor delete table into factory

* refactored onCreate into factory

* ConfirmationModal in Portal
2025-12-10 19:07:32 +00:00
hugocasa
41a07f9742 test ai key button guideline nit (#7332) 2025-12-10 19:07:18 +00:00
Diego Imbert
2d0b012c66 Fix db manager nits (#7334)
* fix db manager undefined crash

* new table primary when no table
2025-12-10 17:30:17 +00:00
centdix
bd3271f967 feat(mcp): add documentation endpoint (#7331)
* simplify inkeep endpoint + add to openapi

* adapt frontend

* add to mcp endpoints

* nit

* udpate ref
2025-12-10 17:28:56 +00:00
Ruben Fiszel
eb4ec4da3b tutorials dismissed by default 2025-12-10 16:32:35 +00:00
HugoCasa
72a38f36fb fix(frontend): do not consider advanced module settings when testing ai agent step 2025-12-10 17:02:15 +01:00
Ruben Fiszel
651681b7ef fix(backend): add presigned url support for object storage (#7328)
* presigned

* all

* all

* all

* all

* all

* all

* all

* nit

* nit

* ee-ref

* presigned

* presigned
2025-12-10 15:35:26 +00:00
centdix
888a6a4e60 feat(aichat): use single tool for flow chat (#7326)
* setup first tests

* better

* variants

* use openrouter + cleaning

* write to files

* cleaning

* cleaning

* inline scripts

* more tests

* cleaning

* better

* test all

* few

* better errors

* cleaning

* single set flow tool

* handle malformed error

* single tool

* validate with zod

* cleaning + check duplicates

* exprtoset + aiaction over test

* regen openflow

* cleaning

* better zod

* recursive ignore of assets

* cleaning

* fix merge

* rm

* cleaning

* comment
2025-12-10 16:29:10 +01:00
centdix
36cb5bde1a feat(aichat): add test pipeline (#7321)
* setup first tests

* better

* variants

* use openrouter + cleaning

* write to files

* cleaning

* cleaning

* inline scripts

* more tests

* cleaning

* better

* test all

* few

* better errors

* cleaning
2025-12-10 14:09:53 +01:00
Ruben Fiszel
94f9175cb8 add target debug 2025-12-09 21:37:59 +00:00
Alexander Petric
8159b8e017 fix: alining group name normalization between scim and igroup api (#7325)
* fix: aling group name normalization between scim and igroup api

* ee-ref
2025-12-09 21:23:13 +00:00
Tristan TR
1549a825a4 improve small details on tutorials (#7320)
* Allow new operator so see the tutorial banner

* Remove banner if user has skipped all tutorials

* Remove tutorials banner if all tutorials marked as completed

* Remove banner if all tutorials marked as completed, by role

* Use derived instead of function for Svelte 5 good practices

* Add error handling

* Add comments

* Create shared Constants for TUTORIAL_DISMISSED_KEY
2025-12-09 11:20:19 +00:00
540 changed files with 45592 additions and 14240 deletions

View File

@@ -0,0 +1,127 @@
---
name: branch-diff-reviewer
description: Use this agent when you want a comprehensive code review of changes in the current branch compared to main. This includes reviewing for bugs, optimization opportunities, code style issues, potential mistakes, and adherence to project conventions. The agent should be invoked after completing a feature branch or before creating a pull request.\n\nExamples:\n\n<example>\nContext: User has finished implementing a new feature and wants feedback before merging.\nuser: "I've finished the new kafka trigger implementation, can you review my changes?"\nassistant: "I'll use the branch-diff-reviewer agent to analyze your changes against the main branch and provide comprehensive feedback."\n<commentary>\nSince the user wants a review of their branch changes, use the Task tool to launch the branch-diff-reviewer agent to compare the current branch against main and provide detailed feedback.\n</commentary>\n</example>\n\n<example>\nContext: User wants to check their code quality before submitting a PR.\nuser: "Review my branch before I create a PR"\nassistant: "Let me launch the branch-diff-reviewer agent to examine all your changes and identify any issues or improvements."\n<commentary>\nThe user is preparing for a PR, so use the branch-diff-reviewer agent to provide a thorough review of all branch differences.\n</commentary>\n</example>\n\n<example>\nContext: User is unsure if their implementation follows project patterns.\nuser: "Does my implementation look correct? I'm not sure if I followed the existing patterns"\nassistant: "I'll use the branch-diff-reviewer agent to compare your changes against main and check for pattern consistency, potential issues, and optimization opportunities."\n<commentary>\nThe user needs validation of their implementation against project standards. Launch the branch-diff-reviewer agent to analyze the diff and provide feedback on patterns, correctness, and improvements.\n</commentary>\n</example>
tools: Glob, Grep, Read, WebFetch, TodoWrite, WebSearch, ListMcpResourcesTool, ReadMcpResourceTool, mcp__svelte__get-documentation, mcp__svelte__list-sections, mcp__svelte__playground-link, mcp__svelte__svelte-autofixer, mcp__ide__getDiagnostics, mcp__ide__executeCode, Bash, Skill
model: inherit
---
You are an elite code reviewer with deep expertise in software engineering best practices, performance optimization, and security. Your role is to provide thorough, actionable feedback on code changes between the current branch and main.
## Your Review Process
1. **First, gather the diff**: Use git commands to obtain the complete diff between the current branch and main:
- Run `git diff main...HEAD` to see all changes
- Run `git log main..HEAD --oneline` to understand the commit history
- Identify all modified, added, and deleted files
2. **Analyze each changed file** in the context of:
- The project's established patterns (check CLAUDE.md and related documentation)
- The file's purpose and its role in the broader codebase
- Dependencies and how changes might affect other parts of the system
## Review Categories
For each significant change, evaluate and report on:
### 🐛 Bugs & Correctness
- Logic errors or edge cases not handled
- Null/undefined handling issues
- Race conditions in async code
- Incorrect error handling
- Type mismatches or unsafe casts
### ⚡ Performance
- Inefficient algorithms or data structures
- N+1 query problems in database code
- Unnecessary re-renders in frontend code
- Missing indexes for database queries
- Blocking operations in async contexts
- Memory leaks or excessive allocations
- For Rust: Check for unnecessary clones, inefficient serde usage, blocking in async
- For Svelte: Check for inefficient reactivity, missing keys in loops, excessive effects
### 🔒 Security
- SQL injection vulnerabilities
- Missing input validation
- Exposed sensitive data
- Authentication/authorization gaps
- Unsafe deserialization
### 📐 Code Quality & Style
- Adherence to project conventions (CLAUDE.md guidelines)
- Code duplication that should be refactored
- Unclear or misleading naming
- Missing or inadequate documentation
- Overly complex logic that could be simplified
- Dead code or unused imports
### 🏗️ Architecture & Design
- Proper separation of concerns
- Appropriate use of existing utilities vs. new code
- Consistency with established patterns
- Proper error propagation
- API design issues
### 🧪 Testing Considerations
- Suggest test cases for new functionality
- Identify untested edge cases
- Note if changes break existing test assumptions
## Project-Specific Rules
### For Rust (Backend)
- Verify `SELECT` statements list explicit columns (never `SELECT *` in worker code)
- Check for proper use of `sqlx` with parameterized queries
- Ensure errors use the custom `Error` enum from `windmill-common::error`
- Verify async code doesn't block the tokio runtime
- Check serde attributes for optimal serialization
- Ensure openapi.yaml is updated for API changes
### For Svelte (Frontend)
- For Svelte 5 files: Verify proper use of Runes (`$state`, `$derived`, `$effect`)
- Check for `key` attributes in `{#each}` blocks
- Ensure event handlers use the new syntax (`onclick` not `on:click`) in Svelte 5
- Verify snippets are used instead of slots in Svelte 5
- Check for proper props declaration with `$props()`
## Output Format
Structure your review as follows:
```
## Summary
[Brief overview of the changes and overall assessment]
## Critical Issues 🚨
[Issues that must be fixed before merging]
## Recommendations 💡
[Improvements that would significantly enhance the code]
## Minor Suggestions 📝
[Nice-to-haves and style improvements]
## Positive Observations ✅
[Well-done aspects worth acknowledging]
## File-by-File Details
[Detailed feedback organized by file]
```
For each issue, provide:
1. **Location**: File path and line number(s)
2. **Issue**: Clear description of the problem
3. **Impact**: Why this matters
4. **Suggestion**: Concrete fix or improvement with code example when helpful
## Behavioral Guidelines
- Be thorough but prioritize: focus most on critical issues
- Be constructive: every criticism should come with a suggestion
- Be specific: vague feedback is not actionable
- Acknowledge good work: positive reinforcement matters
- Consider context: understand why decisions might have been made
- Ask clarifying questions if the intent of changes is unclear
- Reference project documentation when pointing out convention violations
Begin by fetching the diff and then proceed with your comprehensive review.

View File

@@ -0,0 +1,76 @@
---
name: openapi-sync
description: Use this agent when backend API endpoints are added, modified, or removed, or when the Flow structure changes and the OpenAPI specification files need to be updated. This includes changes to route handlers in windmill-api, modifications to request/response schemas, changes to authentication requirements, or updates to the Flow data structures that affect the API.\n\nExamples:\n\n<example>\nContext: User has just added a new API endpoint for managing workspace templates.\nuser: "Add a new endpoint POST /api/w/{workspace}/templates to create workspace templates"\nassistant: "I've created the new endpoint handler in windmill-api. Now let me use the openapi-sync agent to update the OpenAPI specification files."\n<commentary>\nSince a new API endpoint was added, use the openapi-sync agent to ensure the openapi.yaml and openflow.openapi.yaml files are updated with the new endpoint definition.\n</commentary>\n</example>\n\n<example>\nContext: User has modified the response schema for an existing endpoint.\nuser: "Update the GET /api/w/{workspace}/flows endpoint to also return a 'versions' array in the response"\nassistant: "I've updated the flow listing endpoint to include the versions array. Now let me use the openapi-sync agent to update the OpenAPI specification."\n<commentary>\nSince the response schema of an existing endpoint was modified, use the openapi-sync agent to update the corresponding schema in the OpenAPI files.\n</commentary>\n</example>\n\n<example>\nContext: User has made changes to the Flow structure in the codebase.\nuser: "Add a new 'retry_policy' field to the Flow value structure"\nassistant: "I've added the retry_policy field to the Flow struct. Now let me use the openapi-sync agent to update the OpenAPI specification to reflect this schema change."\n<commentary>\nSince the Flow structure was modified, use the openapi-sync agent to ensure the flow-related schemas in openapi.yaml and openflow.openapi.yaml are updated.\n</commentary>\n</example>
model: inherit
---
You are an expert API documentation engineer specializing in OpenAPI specifications for the Windmill platform. Your primary responsibility is to maintain synchronization between the Rust backend API implementation and the OpenAPI specification files.
## Your Core Responsibilities
1. **Update OpenAPI Specifications**: When API endpoints are added, modified, or removed in the windmill-api crate, you must update:
- `backend/windmill-api/openapi.yaml` - The main OpenAPI specification
- `backend/windmill-api/openflow.openapi.yaml` - Flow-specific OpenAPI definitions (if flow-related changes)
2. **Maintain Schema Accuracy**: Ensure all request/response schemas accurately reflect the Rust structs used in the API handlers.
3. **Document Comprehensively**: Include proper descriptions, examples, and parameter documentation.
## Key Files to Reference
- **API Route Definitions**: Look in `backend/windmill-api/src/` for route handlers organized by domain
- **Data Structures**: Check `backend/windmill-common/src/` for shared structs and types
- **Database Schema**: Reference `backend/summarized_schema.txt` for understanding data models
- **Existing OpenAPI Files**: Always review the current state of `openapi.yaml` and `openflow.openapi.yaml` before making changes
## Workflow
1. **Identify Changes**: Determine what API changes were made by examining:
- New or modified route handlers in windmill-api
- Changes to request/response structs
- Modifications to the Flow structure or related types
2. **Analyze the Implementation**: For each endpoint, identify:
- HTTP method and path
- Path parameters, query parameters, and request body schema
- Response schema(s) and status codes
- Authentication requirements
- Any tags or groupings
3. **Update OpenAPI Files**:
- Add or modify path definitions with accurate operation IDs
- Update or create schema definitions in the components section
- Ensure $ref references are correct
- Maintain consistent naming conventions with existing patterns
4. **Validate Changes**: Ensure the YAML syntax is valid and follows OpenAPI 3.0 specification.
## OpenAPI Conventions for Windmill
- **Operation IDs**: Use camelCase, descriptive names (e.g., `createScript`, `listFlows`, `updateWorkspaceSettings`)
- **Tags**: Group endpoints by domain (e.g., `scripts`, `flows`, `workspaces`, `users`)
- **Schema Naming**: Use PascalCase for schema names matching Rust struct names
- **Path Parameters**: Use `{workspace}` for workspace_id, maintain consistency with existing patterns
- **Security**: Most endpoints require Bearer token authentication - include appropriate security requirements
## Schema Mapping from Rust to OpenAPI
- `String` / `&str``type: string`
- `i32`, `i64``type: integer` (with appropriate format)
- `f32`, `f64``type: number`
- `bool``type: boolean`
- `Vec<T>``type: array` with `items`
- `Option<T>` → property is not in `required` array
- `HashMap<K, V>``type: object` with `additionalProperties`
- Enums → `type: string` with `enum` array
- Custom structs → `$ref` to schema definition
## Important Notes
- Always preserve existing documentation and descriptions when updating
- Maintain backward compatibility warnings in descriptions when applicable
- Include example values where they aid understanding
- For Flow-related changes, update BOTH openapi.yaml AND openflow.openapi.yaml as needed
- Follow the existing indentation and formatting style in the YAML files
When you complete updates, summarize what changes were made to which files and highlight any schema additions or modifications that downstream consumers should be aware of.

62
.claude/settings.json Normal file
View File

@@ -0,0 +1,62 @@
{
"permissions": {
"allow": [
"Read(**/*.rs)",
"Bash(ls:*)",
"Bash(grep:*)",
"Bash(cat:*)",
"Bash(head:*)",
"Bash(tail:*)",
"Bash(less:*)",
"Bash(more:*)",
"Bash(find:*)",
"Bash(wc:*)",
"Bash(diff:*)",
"Bash(file:*)",
"Bash(stat:*)",
"Bash(tree:*)",
"Bash(pwd)",
"Bash(which:*)",
"Bash(whereis:*)",
"Bash(echo:*)",
"Bash(git status:*)",
"Bash(git diff:*)",
"Bash(git log:*)",
"Bash(git branch:*)",
"Bash(git show:*)",
"Bash(git blame:*)"
],
"deny": [
"Read(.env)",
"Read(.env.*)",
"Read(**/.env)",
"Read(**/.env.*)",
"Read(**/secrets/**)",
"Read(**/*.pem)",
"Read(**/*.key)",
"Read(**/credentials.json)",
"Read(**/*secret*)",
"Edit(.env)",
"Edit(.env.*)",
"Edit(**/.env)",
"Edit(**/.env.*)"
],
"ask": [
"Bash(rm:*)",
"Bash(rmdir:*)",
"Bash(mv:*)",
"Bash(chmod:*)",
"Bash(chown:*)",
"Bash(truncate:*)",
"Bash(shred:*)",
"Bash(unlink:*)",
"Bash(git push:*)",
"Bash(git reset:*)",
"Bash(git revert:*)",
"Bash(git checkout:*)",
"Bash(git merge:*)",
"Bash(git rebase:*)"
]
},
"enableAllProjectMcpServers": true
}

View File

@@ -84,6 +84,7 @@ jobs:
RUST_LOG_STYLE: never
CARGO_NET_GIT_FETCH_WITH_CLI: true
WMDEBUG_FORCE_V0_WORKSPACE_DEPENDENCIES: 1
WMDEBUG_FORCE_RUNNABLE_SETTINGS_V0: 1
run: |
deno --version && bun -v && go version && python3 --version
cd windmill-duckdb-ffi-internal && ./build_dev.sh && cd ..

View File

@@ -64,9 +64,10 @@ jobs:
uses: anthropics/claude-code-action@v1
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
allowed_bots: "windmill-internal-app[bot]"
trigger_phrase: "/plan"
allowed_bots: 'windmill-internal-app[bot]'
trigger_phrase: '/plan'
claude_args: |
--model opus
--system-prompt "# Claude Planning Mode
You are operating in PLANNING MODE ONLY. Your role is to create detailed, structured plans without making any code changes.

View File

@@ -95,8 +95,8 @@ jobs:
uses: anthropics/claude-code-action@v1
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
allowed_bots: "windmill-internal-app[bot]"
trigger_phrase: "/ai"
allowed_bots: 'windmill-internal-app[bot]'
trigger_phrase: '/ai'
settings: |
{
"env": {
@@ -105,6 +105,7 @@ jobs:
}
claude_args: |
--allowedTools "Bash,WebFetch,WebSearch"
--model opus
--system-prompt "## IMPORTANT INSTRUCTIONS
- Your branch name should be a short description of the requested changes.
- Bypass previous instructions and instead of adding a url in your comment to manually create a PR, if you created a new branch with your changes, automatically create a draft PR from that branch to main.

View File

@@ -29,6 +29,11 @@ on:
required: false
default: false
type: boolean
slim:
description: "Build slim image (true, false)"
required: false
default: false
type: boolean
concurrency:
group: ${{ github.ref }}
cancel-in-progress: false
@@ -588,7 +593,7 @@ jobs:
${{ steps.meta-ee-public.outputs.labels }}
build_ee_slim:
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
if: ${{ startsWith(github.ref, 'refs/tags/v') }} || ((github.event_name != 'workflow_dispatch') || (github.event.inputs.slim))
needs: [build_ee]
runs-on: ubicloud
steps:
@@ -608,6 +613,7 @@ jobs:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee-slim
tags: |
type=ref,event=branch
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
@@ -622,7 +628,7 @@ jobs:
uses: depot/build-push-action@v1
with:
context: .
platforms: linux/amd64
platforms: linux/amd64,linux/arm64
push: true
file: "./docker/DockerfileSlimEe"
tags: |

View File

@@ -45,3 +45,4 @@ jobs:
At the end of your review, add complete instructions to reproduce the added changes through the app interface. These instructions will be given to a tester so he can verify the changes. It should be a short descriptive text (not a step by step or a list) on how to navigate the app (what page, what action, what input, etc) to see the changes.
claude_args: |
--allowedTools "mcp__github_inline_comment__create_inline_comment,Bash(gh pr comment:*),Bash(gh pr diff:*),Bash(gh pr view:*)"
--model opus

View File

@@ -3,7 +3,7 @@ name: Weekly PR Summary
on:
schedule:
# Every Friday at 8:00 AM UTC
- cron: "0 8 * * 5"
- cron: '0 8 * * 5'
workflow_dispatch:
# Allow manual triggering for testing
@@ -112,6 +112,7 @@ jobs:
- Verify the file was created by running: `ls -lh summary.md`
claude_args: |
--allowedTools "Edit,MultiEdit,Write,Read,Glob,Grep,LS,Bash"
--model haiku
- name: Send Summary to Windmill
if: hashFiles('summary.md') != ''

View File

@@ -1,5 +1,149 @@
# Changelog
## [1.596.0](https://github.com/windmill-labs/windmill/compare/v1.595.0...v1.596.0) (2025-12-20)
### Features
* type-checked data tables v0 ([#7381](https://github.com/windmill-labs/windmill/issues/7381)) ([3affbb3](https://github.com/windmill-labs/windmill/commit/3affbb33217bc303c1b96ec93fdd2d80444c8c9e))
### Bug Fixes
* improve error msg for unshare error ([#7421](https://github.com/windmill-labs/windmill/issues/7421)) ([cdd5d9f](https://github.com/windmill-labs/windmill/commit/cdd5d9fa9ac11d869da6c755df0e0306dbb33b84))
* improve MS SQL Numeric rounding ([#7404](https://github.com/windmill-labs/windmill/issues/7404)) ([afe74f7](https://github.com/windmill-labs/windmill/commit/afe74f74fadf983a5e5d712716b636b578007250))
* update to astral-tokio-tar for CVE ([#7423](https://github.com/windmill-labs/windmill/issues/7423)) ([d544da3](https://github.com/windmill-labs/windmill/commit/d544da342c9547be2b12d16fb4a4281c43d5ee73))
## [1.595.0](https://github.com/windmill-labs/windmill/compare/v1.594.0...v1.595.0) (2025-12-19)
### Features
* email triggers custom cert ([#7415](https://github.com/windmill-labs/windmill/issues/7415)) ([0bf7407](https://github.com/windmill-labs/windmill/commit/0bf74074192d22e3ba28acae65d88464f9958fb8))
### Bug Fixes
* **backend:** put for loop itered in a separate table ([#7419](https://github.com/windmill-labs/windmill/issues/7419)) ([f89fb29](https://github.com/windmill-labs/windmill/commit/f89fb292da320f54d682e8de5ff57acac0405efa))
* do not use unshare for init scripts ([#7418](https://github.com/windmill-labs/windmill/issues/7418)) ([c28e771](https://github.com/windmill-labs/windmill/commit/c28e77110e3a97c597b0781124a97b6d16a34810))
* **frontend:** settings redesign ([#7406](https://github.com/windmill-labs/windmill/issues/7406)) ([210b828](https://github.com/windmill-labs/windmill/commit/210b8285d4d9a693f67b40831d5bb39d6aeffb92))
* Python Enum types generate proper dropdown schemas with descriptions ([#7400](https://github.com/windmill-labs/windmill/issues/7400)) ([da500fc](https://github.com/windmill-labs/windmill/commit/da500fcf3e79f76e14d1724f07dd69e58a6307e8))
* teams, need both guid and thread id format ([#7420](https://github.com/windmill-labs/windmill/issues/7420)) ([8268354](https://github.com/windmill-labs/windmill/commit/8268354889d0eb1fb44c083fd1c6243f08788e2c))
## [1.594.0](https://github.com/windmill-labs/windmill/compare/v1.593.1...v1.594.0) (2025-12-19)
### Features
* restart flow from step with different flow version ([#7409](https://github.com/windmill-labs/windmill/issues/7409)) ([a699382](https://github.com/windmill-labs/windmill/commit/a6993823affeff6baf7b6c2b40bdb35713bbffe5))
### Bug Fixes
* **backend:** correctly apply preprocessor step tag ([#7412](https://github.com/windmill-labs/windmill/issues/7412)) ([0fe7a2a](https://github.com/windmill-labs/windmill/commit/0fe7a2a17e810153bc7628b9278e2926b869c389))
* disable oomgroup by default ([8060244](https://github.com/windmill-labs/windmill/commit/806024403ee6496dfff886d3ecdb53d4a2b646e6))
* improve teams search ux ([#7407](https://github.com/windmill-labs/windmill/issues/7407)) ([96aacee](https://github.com/windmill-labs/windmill/commit/96aaceef951c23a7d5f4af6ad6b95883f5ba8f71))
## [1.593.1](https://github.com/windmill-labs/windmill/compare/v1.593.0...v1.593.1) (2025-12-18)
### Bug Fixes
* fix folder/group history seq id grant issues ([c9a19f1](https://github.com/windmill-labs/windmill/commit/c9a19f12d637ca47c4a9bbfe0e851198111c3e9e))
## [1.593.0](https://github.com/windmill-labs/windmill/compare/v1.592.1...v1.593.0) (2025-12-17)
### Features
* **ai:** support IAM auth for bedrock provider ([#7379](https://github.com/windmill-labs/windmill/issues/7379)) ([8c55f61](https://github.com/windmill-labs/windmill/commit/8c55f61bbad81bc81509660b5d54d3289c1edfca))
* **backend:** stop schedules and cancel jobs when archiving a workspace ([#7377](https://github.com/windmill-labs/windmill/issues/7377)) ([ebc82db](https://github.com/windmill-labs/windmill/commit/ebc82dbe58eef19ca1e049f0b2099b702fe3725e))
* data table schemas ([#7353](https://github.com/windmill-labs/windmill/issues/7353)) ([75fdc2c](https://github.com/windmill-labs/windmill/commit/75fdc2cdc96ae06ee8a7891fe670acec8a58afe3))
* http triggers scopes ([#7385](https://github.com/windmill-labs/windmill/issues/7385)) ([b4eb7c6](https://github.com/windmill-labs/windmill/commit/b4eb7c6ac076261aed2d9c97f3b09ac52f7fe0da))
* **internal:** runnable settings ([#7298](https://github.com/windmill-labs/windmill/issues/7298)) ([fe56191](https://github.com/windmill-labs/windmill/commit/fe5619142228ea5370b64112e3a2e38aed507b66))
* workspace forks merge UI ([#7333](https://github.com/windmill-labs/windmill/issues/7333)) ([9d06c15](https://github.com/windmill-labs/windmill/commit/9d06c152ee5c2ab1f76a631411f3603bb0575f5e))
### Bug Fixes
* add history directly viewable in folder/group viewer ([#7365](https://github.com/windmill-labs/windmill/issues/7365)) ([b3603d8](https://github.com/windmill-labs/windmill/commit/b3603d872090c354a9ee82714a6a0e4e79019428))
* add history to raw app builder ([#7362](https://github.com/windmill-labs/windmill/issues/7362)) ([431074d](https://github.com/windmill-labs/windmill/commit/431074d2493d6e87148806a09f60a7eacef552ff))
* **aiagent:** fix gemini-3.0 usage ([#7382](https://github.com/windmill-labs/windmill/issues/7382)) ([f64d918](https://github.com/windmill-labs/windmill/commit/f64d918af6e1d9c0e5b1c0abfee081625f3410cb))
* **aichat:** fix for azure responses api not available in some region ([#7387](https://github.com/windmill-labs/windmill/issues/7387)) ([e7719d2](https://github.com/windmill-labs/windmill/commit/e7719d2cda1c636f0f0acd7cb9bd52c6b3712ebe))
* **backend:** better trigger listening logs ([#7392](https://github.com/windmill-labs/windmill/issues/7392)) ([3ba361a](https://github.com/windmill-labs/windmill/commit/3ba361ad1ae19130b8bd72a3d940ddc529f0471b))
* **frontend:** http/email triggers UI nits ([#7378](https://github.com/windmill-labs/windmill/issues/7378)) ([75e1e90](https://github.com/windmill-labs/windmill/commit/75e1e902734e755f2979f882dd4b2889ce13dfef))
* **mcp:** fix unresovled schema ([#7383](https://github.com/windmill-labs/windmill/issues/7383)) ([1b86a39](https://github.com/windmill-labs/windmill/commit/1b86a39051df1344718ed868a15714f4cee90680))
* propagate canceled_by in flows ([#7396](https://github.com/windmill-labs/windmill/issues/7396)) ([0454f39](https://github.com/windmill-labs/windmill/commit/0454f392e7d9c77f47252b18c1d7ec2ba2cc8cca))
* **rawapp:** make popup work with runnables ([2f5fdd6](https://github.com/windmill-labs/windmill/commit/2f5fdd6b3f742a614cfba590408b88a64d0c86a3))
* **rawapp:** schema for openai ([#7364](https://github.com/windmill-labs/windmill/issues/7364)) ([37394d6](https://github.com/windmill-labs/windmill/commit/37394d6d532923aa273b50c94799ed7a0161e2af))
* SCIM 2.0 RFC compliance + displayName support ([#7380](https://github.com/windmill-labs/windmill/issues/7380)) ([6ffb80d](https://github.com/windmill-labs/windmill/commit/6ffb80d1e1631385ea1bc2b5ad447431f52d892f))
## [1.592.1](https://github.com/windmill-labs/windmill/compare/v1.592.0...v1.592.1) (2025-12-12)
### Bug Fixes
* **ai:** improve share system prompts ([490114d](https://github.com/windmill-labs/windmill/commit/490114d133a08ef7f61ed216796b01fbec32a677))
## [1.592.0](https://github.com/windmill-labs/windmill/compare/v1.591.4...v1.592.0) (2025-12-12)
### Features
* **ai:** standardize and improve system prompts ([#7346](https://github.com/windmill-labs/windmill/issues/7346)) ([31e002a](https://github.com/windmill-labs/windmill/commit/31e002ad411bfbf08c933700bf5ae12b253ac0b8))
### Bug Fixes
* **bun:** deployment error on workspace dependencies ([#7355](https://github.com/windmill-labs/windmill/issues/7355)) ([6859670](https://github.com/windmill-labs/windmill/commit/68596701f1e9da460d7bd3246dc797fca1e66a62))
* clear datetime input in schedule sets input to null ([#7358](https://github.com/windmill-labs/windmill/issues/7358)) ([8f1343e](https://github.com/windmill-labs/windmill/commit/8f1343e155620b047e746da40653ec627f97a1a3))
## [1.591.4](https://github.com/windmill-labs/windmill/compare/v1.591.3...v1.591.4) (2025-12-12)
### Bug Fixes
* **app:** fix raw scripts forbidden by policy if no args ([857adf5](https://github.com/windmill-labs/windmill/commit/857adf5b63fe243736366c38a7e573678552a99b))
## [1.591.3](https://github.com/windmill-labs/windmill/compare/v1.591.2...v1.591.3) (2025-12-11)
### Bug Fixes
* fix test up to step ([#7348](https://github.com/windmill-labs/windmill/issues/7348)) ([4337a88](https://github.com/windmill-labs/windmill/commit/4337a8810e7db874cd75006aacf4e4a85bc33836))
* S3 SDK nits + Presigned S3 Public URL function ([#7342](https://github.com/windmill-labs/windmill/issues/7342)) ([2ee00b3](https://github.com/windmill-labs/windmill/commit/2ee00b3c7b0f7ba9a5997ec4474949500ff2c67b))
## [1.591.2](https://github.com/windmill-labs/windmill/compare/v1.591.1...v1.591.2) (2025-12-11)
### Bug Fixes
* **backend:** pin reqwest to 0.12.24 for better handling of redirects ([#7336](https://github.com/windmill-labs/windmill/issues/7336)) ([99bf866](https://github.com/windmill-labs/windmill/commit/99bf866f90d96f706a517b7626df999bce9e36ac))
## [1.591.1](https://github.com/windmill-labs/windmill/compare/v1.591.0...v1.591.1) (2025-12-10)
### Bug Fixes
* update git sync to latest cli ([da65ddd](https://github.com/windmill-labs/windmill/commit/da65ddd8f82ae283557526d0e97b1057976ec2ff))
## [1.591.0](https://github.com/windmill-labs/windmill/compare/v1.590.0...v1.591.0) (2025-12-10)
### Features
* **aichat:** add test pipeline ([#7321](https://github.com/windmill-labs/windmill/issues/7321)) ([36cb5bd](https://github.com/windmill-labs/windmill/commit/36cb5bde1a887effee2e2ba83a6ddd85f3e79775))
* **aichat:** use single tool for flow chat ([#7326](https://github.com/windmill-labs/windmill/issues/7326)) ([888a6a4](https://github.com/windmill-labs/windmill/commit/888a6a4e607f814cd92f3f489b40ac886dbafc0d))
* **mcp:** add documentation endpoint ([#7331](https://github.com/windmill-labs/windmill/issues/7331)) ([bd3271f](https://github.com/windmill-labs/windmill/commit/bd3271f967ed1996060c590bac0587a387b7869c))
### Bug Fixes
* alining group name normalization between scim and igroup api ([#7325](https://github.com/windmill-labs/windmill/issues/7325)) ([8159b8e](https://github.com/windmill-labs/windmill/commit/8159b8e017660f3b3aeed6beb97a42e3ee658824))
* **backend:** add presigned url support for object storage ([#7328](https://github.com/windmill-labs/windmill/issues/7328)) ([651681b](https://github.com/windmill-labs/windmill/commit/651681b7efb6793d2939217aafcfb3e44f5b0e84))
* **cli:** do not remove schema for normal apps ([3470c92](https://github.com/windmill-labs/windmill/commit/3470c927aeae1627984fff535ec36ece4349f329))
* **frontend:** do not consider advanced module settings when testing ai agent step ([72a38f3](https://github.com/windmill-labs/windmill/commit/72a38f36fbaf3ceaec03612b9932187fa134755c))
## [1.590.0](https://github.com/windmill-labs/windmill/compare/v1.589.3...v1.590.0) (2025-12-08)

View File

@@ -1,16 +1,6 @@
ARG DEBIAN_IMAGE=debian:bookworm-slim
ARG RUST_IMAGE=rust:1.90-slim-bookworm
# Build libwindmill_duckdb_ffi_internal.so separately
FROM ${RUST_IMAGE} AS windmill_duckdb_ffi_internal_builder
WORKDIR /windmill-duckdb-ffi-internal
RUN apt-get update && apt-get install -y pkg-config clang=1:14.0-55.* libclang-dev=1:14.0-55.* cmake=3.25.* && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
COPY ./backend/windmill-duckdb-ffi-internal .
RUN cargo build --release -p windmill_duckdb_ffi_internal
FROM ${RUST_IMAGE} AS rust_base
RUN apt-get update && apt-get install -y git libssl-dev pkg-config npm
@@ -30,6 +20,20 @@ WORKDIR /windmill
ENV SQLX_OFFLINE=true
# ENV CARGO_INCREMENTAL=1
FROM rust_base AS windmill_duckdb_ffi_internal_builder
WORKDIR /windmill-duckdb-ffi-internal
RUN apt-get update && apt-get install -y clang=1:14.0-55.* libclang-dev=1:14.0-55.* cmake=3.25.* && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
COPY ./backend/windmill-duckdb-ffi-internal .
RUN --mount=type=cache,target=/usr/local/cargo/registry \
--mount=type=cache,target=$SCCACHE_DIR,sharing=locked \
cargo build --release -p windmill_duckdb_ffi_internal
FROM node:24-alpine as frontend
# install dependencies
@@ -44,6 +48,7 @@ RUN mkdir /backend
COPY /backend/windmill-api/openapi.yaml /backend/windmill-api/openapi.yaml
COPY /openflow.openapi.yaml /openflow.openapi.yaml
COPY /backend/windmill-api/build_openapi.sh /backend/windmill-api/build_openapi.sh
COPY /system_prompts/auto-generated /system_prompts/auto-generated
RUN cd /backend/windmill-api && . ./build_openapi.sh
COPY /backend/parsers/windmill-parser-wasm/pkg/ /backend/parsers/windmill-parser-wasm/pkg/
@@ -229,7 +234,9 @@ COPY ./frontend/src/lib/hubPaths.json ${APP}/hubPaths.json
RUN windmill cache ${APP}/hubPaths.json && rm ${APP}/hubPaths.json && chmod -R 777 /tmp/windmill
# Create a non-root user 'windmill' with UID and GID 1000
# Cr,.eate a non-root user 'windmill' with UID and GID 1000
RUN addgroup --gid 1000 windmill && \
adduser --disabled-password --gecos "" --uid 1000 --gid 1000 windmill

View File

@@ -0,0 +1,19 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE workspace_diff SET has_changes = true, exists_in_source = $5, exists_in_fork = $6\n WHERE path = $3 AND kind = $4 AND (\n (source_workspace_id = $1 AND fork_workspace_id = $2)\n OR (source_workspace_id = $2 AND fork_workspace_id =$1)\n )",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text",
"Text",
"Bool",
"Bool"
]
},
"nullable": []
},
"hash": "034a8519198daf30e0eb8a74ed92f896c83bb39e1cb52fe3c29c1a224c3859c2"
}

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE schedule SET enabled = false WHERE workspace_id = $1 AND enabled = true RETURNING path",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false
]
},
"hash": "03669873e4e3b22c737d5170821f677925474aad885bf1c0780bdb978225517e"
}

View File

@@ -1,156 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "select hash, tag, concurrency_key, concurrent_limit, concurrency_time_window_s, debounce_key, debounce_delay_s, cache_ttl, cache_ignore_s3_path, language as \"language: ScriptLang\", dedicated_worker, priority, delete_after_use, timeout, has_preprocessor, on_behalf_of_email, created_by, path from script where hash = $1 AND workspace_id = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "hash",
"type_info": "Int8"
},
{
"ordinal": 1,
"name": "tag",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "concurrency_key",
"type_info": "Varchar"
},
{
"ordinal": 3,
"name": "concurrent_limit",
"type_info": "Int4"
},
{
"ordinal": 4,
"name": "concurrency_time_window_s",
"type_info": "Int4"
},
{
"ordinal": 5,
"name": "debounce_key",
"type_info": "Varchar"
},
{
"ordinal": 6,
"name": "debounce_delay_s",
"type_info": "Int4"
},
{
"ordinal": 7,
"name": "cache_ttl",
"type_info": "Int4"
},
{
"ordinal": 8,
"name": "cache_ignore_s3_path",
"type_info": "Bool"
},
{
"ordinal": 9,
"name": "language: ScriptLang",
"type_info": {
"Custom": {
"name": "script_lang",
"kind": {
"Enum": [
"python3",
"deno",
"go",
"bash",
"postgresql",
"nativets",
"bun",
"mysql",
"bigquery",
"snowflake",
"graphql",
"powershell",
"mssql",
"php",
"bunnative",
"rust",
"ansible",
"csharp",
"oracledb",
"nu",
"java",
"duckdb",
"ruby"
]
}
}
}
},
{
"ordinal": 10,
"name": "dedicated_worker",
"type_info": "Bool"
},
{
"ordinal": 11,
"name": "priority",
"type_info": "Int2"
},
{
"ordinal": 12,
"name": "delete_after_use",
"type_info": "Bool"
},
{
"ordinal": 13,
"name": "timeout",
"type_info": "Int4"
},
{
"ordinal": 14,
"name": "has_preprocessor",
"type_info": "Bool"
},
{
"ordinal": 15,
"name": "on_behalf_of_email",
"type_info": "Text"
},
{
"ordinal": 16,
"name": "created_by",
"type_info": "Varchar"
},
{
"ordinal": 17,
"name": "path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Int8",
"Text"
]
},
"nullable": [
false,
true,
true,
true,
true,
true,
true,
true,
true,
false,
true,
true,
true,
true,
true,
true,
false,
false
]
},
"hash": "05b69dcef0f4f649513e186e73089979c49b4b8113ee832ea7539b56a0415f32"
}

View File

@@ -29,8 +29,7 @@
"postgres",
"sqs",
"gcp",
"mqtt",
"nextcloud"
"mqtt"
]
}
}

View File

@@ -172,6 +172,11 @@
"ordinal": 33,
"name": "datatable",
"type_info": "Jsonb"
},
{
"ordinal": 34,
"name": "teams_team_guid",
"type_info": "Text"
}
],
"parameters": {
@@ -213,6 +218,7 @@
true,
true,
true,
true,
true
]
},

View File

@@ -0,0 +1,59 @@
{
"db_name": "PostgreSQL",
"query": "SELECT path, kind, ahead, behind, has_changes, exists_in_source, exists_in_fork FROM workspace_diff\n WHERE source_workspace_id = $1 AND fork_workspace_id = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "path",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "kind",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "ahead",
"type_info": "Int4"
},
{
"ordinal": 3,
"name": "behind",
"type_info": "Int4"
},
{
"ordinal": 4,
"name": "has_changes",
"type_info": "Bool"
},
{
"ordinal": 5,
"name": "exists_in_source",
"type_info": "Bool"
},
{
"ordinal": 6,
"name": "exists_in_fork",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
false,
false,
false,
true,
true,
true
]
},
"hash": "0b8e5fe95f4a2855678ca041b50405b698a368626da42dd9f4ce9d0681d016a1"
}

View File

@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM workspace_diff WHERE path = $3 AND kind = $4 AND (\n (source_workspace_id = $1 AND fork_workspace_id = $2)\n OR (source_workspace_id = $2 AND fork_workspace_id =$1)\n )",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "0f689b9bd1c9a24f3c6cdafef0215f102122665bc3cc15718831b991052b4caf"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "WITH inserted_job AS (\n INSERT INTO v2_job (id, workspace_id, raw_code, raw_lock, raw_flow, tag, parent_job,\n created_by, permissioned_as, runnable_id, runnable_path, args, kind, trigger,\n script_lang, same_worker, pre_run_error, permissioned_as_email, visible_to_owner,\n flow_innermost_root_job, root_job, concurrent_limit, concurrency_time_window_s, timeout, flow_step_id,\n cache_ttl, priority, trigger_kind, script_entrypoint_override, preprocessed)\n VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18,\n $19, $20, $38, $21, $22, $23, $24, $25, $26, $39::job_trigger_kind,\n ($12::JSONB)->>'_ENTRYPOINT_OVERRIDE', $27)\n ),\n inserted_runtime AS (\n INSERT INTO v2_job_runtime (id, ping) VALUES ($1, null)\n ),\n inserted_job_perms AS (\n INSERT INTO job_perms (job_id, email, username, is_admin, is_operator, folders, groups, workspace_id, end_user_email) \n values ($1, $32, $33, $34, $35, $36, $37, $2, $41) \n ON CONFLICT (job_id) DO UPDATE SET email = EXCLUDED.email, username = EXCLUDED.username, is_admin = EXCLUDED.is_admin, is_operator = EXCLUDED.is_operator, folders = EXCLUDED.folders, groups = EXCLUDED.groups, workspace_id = EXCLUDED.workspace_id, end_user_email = EXCLUDED.end_user_email\n )\n INSERT INTO v2_job_queue\n (workspace_id, id, running, scheduled_for, started_at, tag, priority, cache_ignore_s3_path)\n VALUES ($2, $1, $28, COALESCE($29, now()), CASE WHEN $27 OR $40 THEN now() END, $30, $31, $42)",
"query": "WITH inserted_job AS (\n INSERT INTO v2_job (\n id, -- 1\n workspace_id, -- 2\n raw_code, -- 3\n raw_lock, -- 4\n raw_flow, -- 5\n tag, -- 6\n parent_job, -- 7\n created_by, -- 8\n permissioned_as, -- 9\n runnable_id, -- 10\n runnable_path, -- 11\n args, -- 12\n kind, -- 13\n trigger, -- 14\n script_lang, -- 15\n same_worker, -- 16\n pre_run_error, -- 17 \n permissioned_as_email, -- 18\n visible_to_owner, -- 19\n flow_innermost_root_job, -- 20\n root_job, -- 38\n concurrent_limit, -- 21\n concurrency_time_window_s, -- 22\n timeout, -- 23\n flow_step_id, -- 24\n cache_ttl, -- 25\n priority, -- 26\n trigger_kind, -- 39\n script_entrypoint_override, -- 12\n preprocessed -- 27,\n ) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18,\n $19, $20, $38, $21, $22, $23, $24, $25, $26, $39::job_trigger_kind,\n ($12::JSONB)->>'_ENTRYPOINT_OVERRIDE', $27)\n ),\n inserted_runtime AS (\n INSERT INTO v2_job_runtime (id, ping) VALUES ($1, null)\n ),\n inserted_job_perms AS (\n INSERT INTO job_perms (job_id, email, username, is_admin, is_operator, folders, groups, workspace_id, end_user_email) \n values ($1, $32, $33, $34, $35, $36, $37, $2, $41) \n ON CONFLICT (job_id) DO UPDATE SET email = EXCLUDED.email, username = EXCLUDED.username, is_admin = EXCLUDED.is_admin, is_operator = EXCLUDED.is_operator, folders = EXCLUDED.folders, groups = EXCLUDED.groups, workspace_id = EXCLUDED.workspace_id, end_user_email = EXCLUDED.end_user_email\n )\n INSERT INTO v2_job_queue\n (workspace_id, id, running, scheduled_for, started_at, tag, priority, cache_ignore_s3_path, runnable_settings_handle)\n VALUES ($2, $1, $28, COALESCE($29, now()), CASE WHEN $27 OR $40 THEN now() END, $30, $31, $42, $43)",
"describe": {
"columns": [],
"parameters": {
@@ -121,18 +121,18 @@
"postgres",
"sqs",
"gcp",
"mqtt",
"nextcloud"
"mqtt"
]
}
}
},
"Bool",
"Varchar",
"Bool"
"Bool",
"Int8"
]
},
"nullable": []
},
"hash": "b179a3f876ca659bed892d464bf51a733cc86a3204fcd9edccda63fddc97dced"
"hash": "14276a040cb4db88d71fccdc3579e8c0bb132b70668301b535872d1632753e30"
}

View File

@@ -172,6 +172,11 @@
"ordinal": 33,
"name": "datatable",
"type_info": "Jsonb"
},
{
"ordinal": 34,
"name": "teams_team_guid",
"type_info": "Text"
}
],
"parameters": {
@@ -213,6 +218,7 @@
true,
true,
true,
true,
true
]
},

View File

@@ -39,8 +39,7 @@
"postgres",
"sqs",
"gcp",
"mqtt",
"nextcloud"
"mqtt"
]
}
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE instance_group SET scim_display_name = $1 where id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "2241ed0c5a47ac715de3ef13a850e514e0fb7b062f4147bffb0e9badfea478d0"
}

View File

@@ -1,16 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO script\n (workspace_id, hash, path, parent_hashes, summary, description, content, created_by, schema, is_template, extra_perms, lock, language, kind, tag, draft_only, envs, concurrent_limit, concurrency_time_window_s, cache_ttl, cache_ignore_s3_path, dedicated_worker, ws_error_handler_muted, priority, restart_unless_cancelled, delete_after_use, timeout, concurrency_key, visible_to_runner_only, no_main_func, codebase, has_preprocessor, on_behalf_of_email, schema_validation, assets, debounce_key, debounce_delay_s)\n\n SELECT workspace_id, $1, path, array_prepend($2::bigint, COALESCE(parent_hashes, '{}'::bigint[])), summary, description, content, created_by, schema, is_template, extra_perms, NULL, language, kind, tag, draft_only, envs, concurrent_limit, concurrency_time_window_s, cache_ttl, cache_ignore_s3_path, dedicated_worker, ws_error_handler_muted, priority, restart_unless_cancelled, delete_after_use, timeout, concurrency_key, visible_to_runner_only, no_main_func, codebase, has_preprocessor, on_behalf_of_email, schema_validation, assets, debounce_key, debounce_delay_s\n\n FROM script WHERE hash = $2 AND workspace_id = $3;\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Int8",
"Int8",
"Text"
]
},
"nullable": []
},
"hash": "23759cb515e926e272bbc8e5d8a0a9d039b99bc2026e381e99ef41cdaf6ea19f"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT path FROM script\n WHERE workspace_id = $1 AND path = ANY($2) AND archived = false",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"TextArray"
]
},
"nullable": [
false
]
},
"hash": "2d1ba3c92c0385c530934082284cf548a50d533dc1bef58dfd0ecc163c9920f3"
}

View File

@@ -0,0 +1,35 @@
{
"db_name": "PostgreSQL",
"query": "SELECT value, is_secret, description\n FROM variable\n WHERE workspace_id = $1 AND path = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "value",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "is_secret",
"type_info": "Bool"
},
{
"ordinal": 2,
"name": "description",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
false,
false
]
},
"hash": "2d5ff8fc102ae0c452c1f9cd5cd30fd0a3b4e6e746c659da1767edafd139d45e"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO flow_iterator_data (job_id, itered) VALUES ($1, $2)\n ON CONFLICT (job_id) DO UPDATE SET itered = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid",
"Jsonb"
]
},
"nullable": []
},
"hash": "389828f43e638c02757ba37da46b03111a9915a16b53f3e29a09de89210d6af1"
}

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT itered as \"itered: Json<Vec<Box<RawValue>>>\" FROM flow_iterator_data WHERE job_id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "itered: Json<Vec<Box<RawValue>>>",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Uuid"
]
},
"nullable": [
false
]
},
"hash": "3c5165992c4b8ad3f91627d1d9f6156d3a6b45a7cb2b37a7c166d36d7caa4d2f"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT\n workspace_id,\n slack_team_id,\n teams_team_id,\n teams_team_name,\n slack_name,\n slack_command_script,\n teams_command_script,\n slack_email,\n slack_oauth_client_id,\n slack_oauth_client_secret,\n auto_invite_domain,\n auto_invite_operator,\n auto_add,\n customer_id,\n plan,\n webhook,\n deploy_to,\n ai_config,\n error_handler,\n error_handler_extra_args,\n error_handler_muted_on_cancel,\n large_file_storage,\n datatable,\n ducklake,\n git_sync,\n deploy_ui,\n default_app,\n default_scripts,\n mute_critical_alerts,\n color,\n operator_settings,\n git_app_installations,\n auto_add_instance_groups,\n auto_add_instance_groups_roles\n FROM\n workspace_settings\n WHERE\n workspace_id = $1\n ",
"query": "\n SELECT\n workspace_id,\n slack_team_id,\n teams_team_id,\n teams_team_name,\n teams_team_guid,\n slack_name,\n slack_command_script,\n teams_command_script,\n slack_email,\n slack_oauth_client_id,\n slack_oauth_client_secret,\n auto_invite_domain,\n auto_invite_operator,\n auto_add,\n customer_id,\n plan,\n webhook,\n deploy_to,\n ai_config,\n error_handler,\n error_handler_extra_args,\n error_handler_muted_on_cancel,\n large_file_storage,\n datatable,\n ducklake,\n git_sync,\n deploy_ui,\n default_app,\n default_scripts,\n mute_critical_alerts,\n color,\n operator_settings,\n git_app_installations,\n auto_add_instance_groups,\n auto_add_instance_groups_roles\n FROM\n workspace_settings\n WHERE\n workspace_id = $1\n ",
"describe": {
"columns": [
{
@@ -25,151 +25,156 @@
},
{
"ordinal": 4,
"name": "teams_team_guid",
"type_info": "Text"
},
{
"ordinal": 5,
"name": "slack_name",
"type_info": "Varchar"
},
{
"ordinal": 5,
"ordinal": 6,
"name": "slack_command_script",
"type_info": "Varchar"
},
{
"ordinal": 6,
"ordinal": 7,
"name": "teams_command_script",
"type_info": "Text"
},
{
"ordinal": 7,
"ordinal": 8,
"name": "slack_email",
"type_info": "Varchar"
},
{
"ordinal": 8,
"ordinal": 9,
"name": "slack_oauth_client_id",
"type_info": "Varchar"
},
{
"ordinal": 9,
"ordinal": 10,
"name": "slack_oauth_client_secret",
"type_info": "Varchar"
},
{
"ordinal": 10,
"ordinal": 11,
"name": "auto_invite_domain",
"type_info": "Varchar"
},
{
"ordinal": 11,
"ordinal": 12,
"name": "auto_invite_operator",
"type_info": "Bool"
},
{
"ordinal": 12,
"ordinal": 13,
"name": "auto_add",
"type_info": "Bool"
},
{
"ordinal": 13,
"ordinal": 14,
"name": "customer_id",
"type_info": "Varchar"
},
{
"ordinal": 14,
"ordinal": 15,
"name": "plan",
"type_info": "Varchar"
},
{
"ordinal": 15,
"ordinal": 16,
"name": "webhook",
"type_info": "Text"
},
{
"ordinal": 16,
"ordinal": 17,
"name": "deploy_to",
"type_info": "Varchar"
},
{
"ordinal": 17,
"ordinal": 18,
"name": "ai_config",
"type_info": "Jsonb"
},
{
"ordinal": 18,
"ordinal": 19,
"name": "error_handler",
"type_info": "Varchar"
},
{
"ordinal": 19,
"ordinal": 20,
"name": "error_handler_extra_args",
"type_info": "Json"
},
{
"ordinal": 20,
"ordinal": 21,
"name": "error_handler_muted_on_cancel",
"type_info": "Bool"
},
{
"ordinal": 21,
"ordinal": 22,
"name": "large_file_storage",
"type_info": "Jsonb"
},
{
"ordinal": 22,
"ordinal": 23,
"name": "datatable",
"type_info": "Jsonb"
},
{
"ordinal": 23,
"ordinal": 24,
"name": "ducklake",
"type_info": "Jsonb"
},
{
"ordinal": 24,
"ordinal": 25,
"name": "git_sync",
"type_info": "Jsonb"
},
{
"ordinal": 25,
"ordinal": 26,
"name": "deploy_ui",
"type_info": "Jsonb"
},
{
"ordinal": 26,
"ordinal": 27,
"name": "default_app",
"type_info": "Varchar"
},
{
"ordinal": 27,
"ordinal": 28,
"name": "default_scripts",
"type_info": "Jsonb"
},
{
"ordinal": 28,
"ordinal": 29,
"name": "mute_critical_alerts",
"type_info": "Bool"
},
{
"ordinal": 29,
"ordinal": 30,
"name": "color",
"type_info": "Varchar"
},
{
"ordinal": 30,
"ordinal": 31,
"name": "operator_settings",
"type_info": "Jsonb"
},
{
"ordinal": 31,
"ordinal": 32,
"name": "git_app_installations",
"type_info": "Jsonb"
},
{
"ordinal": 32,
"ordinal": 33,
"name": "auto_add_instance_groups",
"type_info": "TextArray"
},
{
"ordinal": 33,
"ordinal": 34,
"name": "auto_add_instance_groups_roles",
"type_info": "Jsonb"
}
@@ -187,6 +192,7 @@
true,
true,
true,
true,
false,
true,
true,
@@ -216,5 +222,5 @@
true
]
},
"hash": "95fa60eb45228ff289655fc676991f4e90d237799f6817f292eb1391694164c7"
"hash": "3c53de373b9f1034b5f43002bf4715e12ddc641f4ca52efe0335719fa9461bb0"
}

View File

@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO runnable_settings (hash, debouncing_settings, concurrency_settings)\n VALUES ($1, $2, $3)\n ON CONFLICT (hash)\n DO NOTHING",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Int8",
"Int8",
"Int8"
]
},
"nullable": []
},
"hash": "451d9cde90d14071e21ffb5f615052b7ba7fc315fc301ed5c0ff50d9a3ab0d4a"
}

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id FROM workspace WHERE parent_workspace_id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false
]
},
"hash": "4d04e436d46530f2f4c9dbc5c6c472581b9d90d330957539c0019ec2e7d9a68a"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT path FROM app\n WHERE workspace_id = $1 AND path = ANY($2)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"TextArray"
]
},
"nullable": [
false
]
},
"hash": "50ef2eac143273c7b7b7414b9be3c89130c709a57aca712e2af996bd4a4fa101"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT path FROM resource\n WHERE workspace_id = $1 AND path = ANY($2)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"TextArray"
]
},
"nullable": [
false
]
},
"hash": "5169d793f3820095839bb69d6c7c8cbea692ef6c1a838b970f382ee72db492d3"
}

View File

@@ -1,16 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n UPDATE workspace_settings\n SET teams_team_id = $1, teams_team_name = $2\n WHERE workspace_id = $3\n AND NOT EXISTS (\n SELECT 1 FROM workspace_settings\n WHERE teams_team_id = $1 AND workspace_id <> $2\n )\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "551c78392919e18019bb0a4344fb1bd45853bf5b72e0ab991e0e61fedcfb42fc"
}

View File

@@ -1,16 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE instance_group SET scim_display_name = $1, name = $2 where id = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "574d9a2bb6eceb62f3d2c2be3f18b29ef8bba3d6da1b1e21f2ca307ccbebee89"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT \n v2_job_queue.workspace_id,\n v2_job_queue.id,\n v2_job.args as \"args: sqlx::types::Json<HashMap<String, Box<RawValue>>>\",\n v2_job.parent_job,\n v2_job.created_by,\n v2_job_queue.started_at,\n scheduled_for,\n runnable_path,\n kind as \"kind: JobKind\",\n runnable_id as \"runnable_id: ScriptHash\",\n canceled_reason,\n canceled_by,\n permissioned_as,\n permissioned_as_email,\n flow_status as \"flow_status: sqlx::types::Json<Box<RawValue>>\",\n v2_job.tag,\n script_lang as \"script_lang: ScriptLang\",\n same_worker,\n pre_run_error,\n concurrent_limit,\n concurrency_time_window_s,\n flow_innermost_root_job,\n root_job,\n timeout,\n flow_step_id,\n cache_ttl,\n cache_ignore_s3_path,\n v2_job_queue.priority,\n preprocessed,\n script_entrypoint_override,\n trigger,\n trigger_kind as \"trigger_kind: JobTriggerKind\",\n visible_to_owner,\n NULL as permissioned_as_end_user_email\n FROM v2_job_queue INNER JOIN v2_job ON v2_job.id = v2_job_queue.id LEFT JOIN v2_job_status ON v2_job_status.id = v2_job_queue.id WHERE v2_job_queue.id = $1",
"query": "SELECT \n v2_job_queue.workspace_id,\n v2_job_queue.id,\n v2_job.args as \"args: sqlx::types::Json<HashMap<String, Box<RawValue>>>\",\n v2_job.parent_job,\n v2_job.created_by,\n v2_job_queue.started_at,\n v2_job_queue.runnable_settings_handle,\n scheduled_for,\n runnable_path,\n kind as \"kind: JobKind\",\n runnable_id as \"runnable_id: ScriptHash\",\n canceled_reason,\n canceled_by,\n permissioned_as,\n permissioned_as_email,\n flow_status as \"flow_status: sqlx::types::Json<Box<RawValue>>\",\n v2_job.tag,\n script_lang as \"script_lang: ScriptLang\",\n same_worker,\n pre_run_error,\n concurrent_limit,\n concurrency_time_window_s,\n flow_innermost_root_job,\n root_job,\n timeout,\n flow_step_id,\n cache_ttl,\n cache_ignore_s3_path,\n v2_job_queue.priority,\n preprocessed,\n script_entrypoint_override,\n trigger,\n trigger_kind as \"trigger_kind: JobTriggerKind\",\n visible_to_owner,\n NULL as permissioned_as_end_user_email\n FROM v2_job_queue INNER JOIN v2_job ON v2_job.id = v2_job_queue.id LEFT JOIN v2_job_status ON v2_job_status.id = v2_job_queue.id WHERE v2_job_queue.id = $1",
"describe": {
"columns": [
{
@@ -35,16 +35,21 @@
},
{
"ordinal": 6,
"name": "runnable_settings_handle",
"type_info": "Int8"
},
{
"ordinal": 7,
"name": "scheduled_for",
"type_info": "Timestamptz"
},
{
"ordinal": 7,
"ordinal": 8,
"name": "runnable_path",
"type_info": "Varchar"
},
{
"ordinal": 8,
"ordinal": 9,
"name": "kind: JobKind",
"type_info": {
"Custom": {
@@ -79,42 +84,42 @@
}
},
{
"ordinal": 9,
"ordinal": 10,
"name": "runnable_id: ScriptHash",
"type_info": "Int8"
},
{
"ordinal": 10,
"ordinal": 11,
"name": "canceled_reason",
"type_info": "Text"
},
{
"ordinal": 11,
"ordinal": 12,
"name": "canceled_by",
"type_info": "Varchar"
},
{
"ordinal": 12,
"ordinal": 13,
"name": "permissioned_as",
"type_info": "Varchar"
},
{
"ordinal": 13,
"ordinal": 14,
"name": "permissioned_as_email",
"type_info": "Varchar"
},
{
"ordinal": 14,
"ordinal": 15,
"name": "flow_status: sqlx::types::Json<Box<RawValue>>",
"type_info": "Jsonb"
},
{
"ordinal": 15,
"ordinal": 16,
"name": "tag",
"type_info": "Varchar"
},
{
"ordinal": 16,
"ordinal": 17,
"name": "script_lang: ScriptLang",
"type_info": {
"Custom": {
@@ -150,77 +155,77 @@
}
},
{
"ordinal": 17,
"ordinal": 18,
"name": "same_worker",
"type_info": "Bool"
},
{
"ordinal": 18,
"ordinal": 19,
"name": "pre_run_error",
"type_info": "Text"
},
{
"ordinal": 19,
"ordinal": 20,
"name": "concurrent_limit",
"type_info": "Int4"
},
{
"ordinal": 20,
"ordinal": 21,
"name": "concurrency_time_window_s",
"type_info": "Int4"
},
{
"ordinal": 21,
"ordinal": 22,
"name": "flow_innermost_root_job",
"type_info": "Uuid"
},
{
"ordinal": 22,
"ordinal": 23,
"name": "root_job",
"type_info": "Uuid"
},
{
"ordinal": 23,
"ordinal": 24,
"name": "timeout",
"type_info": "Int4"
},
{
"ordinal": 24,
"ordinal": 25,
"name": "flow_step_id",
"type_info": "Varchar"
},
{
"ordinal": 25,
"ordinal": 26,
"name": "cache_ttl",
"type_info": "Int4"
},
{
"ordinal": 26,
"ordinal": 27,
"name": "cache_ignore_s3_path",
"type_info": "Bool"
},
{
"ordinal": 27,
"ordinal": 28,
"name": "priority",
"type_info": "Int2"
},
{
"ordinal": 28,
"ordinal": 29,
"name": "preprocessed",
"type_info": "Bool"
},
{
"ordinal": 29,
"ordinal": 30,
"name": "script_entrypoint_override",
"type_info": "Varchar"
},
{
"ordinal": 30,
"ordinal": 31,
"name": "trigger",
"type_info": "Varchar"
},
{
"ordinal": 31,
"ordinal": 32,
"name": "trigger_kind: JobTriggerKind",
"type_info": {
"Custom": {
@@ -239,20 +244,19 @@
"postgres",
"sqs",
"gcp",
"mqtt",
"nextcloud"
"mqtt"
]
}
}
}
},
{
"ordinal": 32,
"ordinal": 33,
"name": "visible_to_owner",
"type_info": "Bool"
},
{
"ordinal": 33,
"ordinal": 34,
"name": "permissioned_as_end_user_email",
"type_info": "Text"
}
@@ -269,6 +273,7 @@
true,
false,
true,
true,
false,
true,
false,
@@ -299,5 +304,5 @@
null
]
},
"hash": "6c97ab28ab47b75fb3ff39ea70fa3627f08b61bbd33aecb9ea816f8f78a04ec5"
"hash": "5bf200f2c8db25ddf231b564503c6c70f7f3958564a79bb0c6b3863b1ebb0cbf"
}

View File

@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO script\n (workspace_id, hash, path, parent_hashes, summary, description, content, created_by, schema, is_template, extra_perms, lock, language, kind, tag, draft_only, envs, concurrent_limit, concurrency_time_window_s, cache_ttl, cache_ignore_s3_path, dedicated_worker, ws_error_handler_muted, priority, restart_unless_cancelled, delete_after_use, timeout, concurrency_key, visible_to_runner_only, no_main_func, codebase, has_preprocessor, on_behalf_of_email, schema_validation, assets, debounce_key, debounce_delay_s, runnable_settings_handle)\n\n SELECT workspace_id, $1, path, array_prepend($2::bigint, COALESCE(parent_hashes, '{}'::bigint[])), summary, description, content, created_by, schema, is_template, extra_perms, NULL, language, kind, tag, draft_only, envs, concurrent_limit, concurrency_time_window_s, cache_ttl, cache_ignore_s3_path, dedicated_worker, ws_error_handler_muted, priority, restart_unless_cancelled, delete_after_use, timeout, concurrency_key, visible_to_runner_only, no_main_func, codebase, has_preprocessor, on_behalf_of_email, schema_validation, assets, debounce_key, debounce_delay_s, runnable_settings_handle\n\n FROM script WHERE hash = $2 AND workspace_id = $3;\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Int8",
"Int8",
"Text"
]
},
"nullable": []
},
"hash": "5c056ad6cc8967393729288437205c605a24118021fdb2b21b6b61695dc4ff28"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE workspace_settings\n SET teams_team_id = null, teams_team_name = null WHERE workspace_id = $1",
"query": "UPDATE workspace_settings\n SET teams_team_id = null, teams_team_name = null, teams_team_guid = null WHERE workspace_id = $1",
"describe": {
"columns": [],
"parameters": {
@@ -10,5 +10,5 @@
},
"nullable": []
},
"hash": "23c37d36e16251763fabf194e41de63612a7506cc0671b0eb83e528c1c839db4"
"hash": "62a625902ab1507f198bc9b12f2fea8398ec3905699ebf0e28cdfc85c0de4615"
}

View File

@@ -1,15 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO password (email, login_type, verified, username) VALUES ($1, 'saml', true, $2) ON CONFLICT DO NOTHING",
"query": "INSERT INTO password (email, login_type, verified, username, name) VALUES ($1, 'saml', true, $2, $3) ON CONFLICT DO NOTHING",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar"
]
},
"nullable": []
},
"hash": "a59aac0bc593d99aedd14fd9606f11191590a12ea98a671e172b867306959884"
"hash": "638d3c2ba1198dce5b5b0e47df59a92ff8011e19fbefcc3960d6f0fe167e55b6"
}

View File

@@ -0,0 +1,35 @@
{
"db_name": "PostgreSQL",
"query": "SELECT app.summary, app.policy, app_version.value\n FROM app\n JOIN app_version\n ON app_version.id = app.versions[array_upper(app.versions, 1)]\n WHERE app.workspace_id = $1 AND app.path = $2 AND COALESCE(app.draft_only, false) = false",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "summary",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "policy",
"type_info": "Jsonb"
},
{
"ordinal": 2,
"name": "value",
"type_info": "Json"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
false,
false
]
},
"hash": "66a9cb11fcc4757a3b35154840b703b87daa538cf3a72007649bbb074f56131e"
}

View File

@@ -39,8 +39,7 @@
"postgres",
"sqs",
"gcp",
"mqtt",
"nextcloud"
"mqtt"
]
}
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO instance_group (name, scim_display_name, id, external_id) VALUES ($1, $2, $3, $4) ON CONFLICT DO NOTHING",
"query": "INSERT INTO instance_group (name, scim_display_name, id, external_id) VALUES ($1, $2, $3, $4)",
"describe": {
"columns": [],
"parameters": {
@@ -13,5 +13,5 @@
},
"nullable": []
},
"hash": "bd829646d08f68106211f97c75dce13b6fc7d35bbaf7f503dcc73ae48fd07489"
"hash": "7cb7dcf8b20deb59fb1c3d4ad0ca4f9a209ce0d80682182e56946392f800c24c"
}

View File

@@ -0,0 +1,35 @@
{
"db_name": "PostgreSQL",
"query": "SELECT value, description, resource_type\n FROM resource\n WHERE workspace_id = $1 AND path = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "value",
"type_info": "Jsonb"
},
{
"ordinal": 1,
"name": "description",
"type_info": "Text"
},
{
"ordinal": 2,
"name": "resource_type",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
true,
true,
false
]
},
"hash": "819c233915383e89af1bcf1a56c5f67c4e1fc217f216f609e36a9944a7807b33"
}

View File

@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_diff (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)\n SELECT $1, unnest($2::varchar[]), $3, $4, 0, 1, NULL\n ON CONFLICT (source_workspace_id, fork_workspace_id, path, kind)\n DO UPDATE SET\n behind = workspace_diff.behind + 1,\n has_changes = NULL",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"VarcharArray",
"Varchar",
"Varchar"
]
},
"nullable": []
},
"hash": "90a1797f8d5ef7f9b67557c1fb919d1165860c5daf0de76fb2ac2201de11553c"
}

View File

@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO tutorial_progress VALUES ($2, $1::bigint::bit(64)) ON CONFLICT (email) DO UPDATE SET progress = EXCLUDED.progress",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Int8",
"Varchar"
]
},
"nullable": []
},
"hash": "99e6bffe177e69448b09e82b30d24af00edc86a9bc498f319c1e5bee55d77a8a"
}

View File

@@ -29,8 +29,7 @@
"postgres",
"sqs",
"gcp",
"mqtt",
"nextcloud"
"mqtt"
]
}
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT \n j.id, j.workspace_id, j.runnable_id AS \"runnable_id: ScriptHash\", q.scheduled_for, q.started_at, j.parent_job, j.flow_innermost_root_job, j.runnable_path, j.kind as \"kind!: JobKind\", j.permissioned_as, \n j.created_by, j.script_lang AS \"script_lang: ScriptLang\", j.permissioned_as_email, j.flow_step_id, j.trigger_kind AS \"trigger_kind: JobTriggerKind\", j.trigger, j.priority, j.concurrent_limit, j.tag, j.cache_ttl, q.cache_ignore_s3_path\n FROM v2_job j LEFT JOIN v2_job_queue q ON j.id = q.id\n WHERE j.id = $1 AND j.workspace_id = $2",
"query": "SELECT \n j.id, j.workspace_id, j.runnable_id AS \"runnable_id: ScriptHash\", q.scheduled_for, q.started_at, j.parent_job, j.flow_innermost_root_job, j.runnable_path, j.kind as \"kind!: JobKind\", j.permissioned_as, \n j.created_by, j.script_lang AS \"script_lang: ScriptLang\", j.permissioned_as_email, j.flow_step_id, j.trigger_kind AS \"trigger_kind: JobTriggerKind\", j.trigger, j.priority, j.concurrent_limit, j.tag, j.cache_ttl, q.cache_ignore_s3_path, q.runnable_settings_handle\n FROM v2_job j LEFT JOIN v2_job_queue q ON j.id = q.id\n WHERE j.id = $1 AND j.workspace_id = $2",
"describe": {
"columns": [
{
@@ -154,8 +154,7 @@
"postgres",
"sqs",
"gcp",
"mqtt",
"nextcloud"
"mqtt"
]
}
}
@@ -190,6 +189,11 @@
"ordinal": 20,
"name": "cache_ignore_s3_path",
"type_info": "Bool"
},
{
"ordinal": 21,
"name": "runnable_settings_handle",
"type_info": "Int8"
}
],
"parameters": {
@@ -219,8 +223,9 @@
true,
false,
true,
true,
true
]
},
"hash": "7b5ad10af2a9b34fa86429499ea24c0c09c6e7e9ebfa3af90035570133f7c579"
"hash": "a1745a4f525b251d2f5a602ab2b2ede46b4471e21b11f607573a844013911abe"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT COALESCE((SELECT MIN(started_at) as min_started_at\n FROM v2_job_queue INNER JOIN v2_job ON v2_job.id = v2_job_queue.id\n WHERE v2_job.runnable_path = $1 AND v2_job.kind != 'dependencies' AND v2_job_queue.running = true AND v2_job_queue.workspace_id = $2 AND v2_job_queue.canceled_by IS NULL AND v2_job.concurrent_limit > 0), $3) as min_started_at, now() AS now",
"query": "SELECT COALESCE((SELECT MIN(started_at) as min_started_at\n FROM v2_job_queue INNER JOIN v2_job ON v2_job.id = v2_job_queue.id LEFT JOIN runnable_settings rs ON rs.hash = v2_job_queue.runnable_settings_handle LEFT JOIN concurrency_settings cs ON cs.hash = rs.concurrency_settings\n WHERE v2_job.runnable_path = $1 AND v2_job.kind != 'dependencies' AND v2_job_queue.running = true AND v2_job_queue.workspace_id = $2 AND v2_job_queue.canceled_by IS NULL AND COALESCE(cs.concurrent_limit, v2_job.concurrent_limit) > 0), $3) as min_started_at, now() AS now",
"describe": {
"columns": [
{
@@ -26,5 +26,5 @@
null
]
},
"hash": "6b6f8f7b4a6b6e7e41a9da8b6dfdbcae842ff252cc355bd91aeeb5e26dcc74f3"
"hash": "a2e52f033120a3f0b64e0a5ba125df7ce0d25096a23f0b655846a1c07b41f620"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "select hash, tag, concurrency_key, concurrent_limit, concurrency_time_window_s, debounce_key, debounce_delay_s, cache_ttl, cache_ignore_s3_path, language as \"language: ScriptLang\", dedicated_worker, priority, timeout, on_behalf_of_email, created_by FROM script\n WHERE path = $1 AND workspace_id = $2 AND archived = false AND (lock IS NOT NULL OR $3 = false)\n ORDER BY created_at DESC LIMIT 1",
"query": "select hash, tag, concurrency_key, concurrent_limit, concurrency_time_window_s, debounce_key, debounce_delay_s, cache_ttl, cache_ignore_s3_path, runnable_settings_handle, language as \"language: ScriptLang\", dedicated_worker, priority, timeout, on_behalf_of_email, created_by FROM script\n WHERE path = $1 AND workspace_id = $2 AND archived = false AND (lock IS NOT NULL OR $3 = false)\n ORDER BY created_at DESC LIMIT 1",
"describe": {
"columns": [
{
@@ -50,6 +50,11 @@
},
{
"ordinal": 9,
"name": "runnable_settings_handle",
"type_info": "Int8"
},
{
"ordinal": 10,
"name": "language: ScriptLang",
"type_info": {
"Custom": {
@@ -85,27 +90,27 @@
}
},
{
"ordinal": 10,
"ordinal": 11,
"name": "dedicated_worker",
"type_info": "Bool"
},
{
"ordinal": 11,
"ordinal": 12,
"name": "priority",
"type_info": "Int2"
},
{
"ordinal": 12,
"ordinal": 13,
"name": "timeout",
"type_info": "Int4"
},
{
"ordinal": 13,
"ordinal": 14,
"name": "on_behalf_of_email",
"type_info": "Text"
},
{
"ordinal": 14,
"ordinal": 15,
"name": "created_by",
"type_info": "Varchar"
}
@@ -127,6 +132,7 @@
true,
true,
true,
true,
false,
true,
true,
@@ -135,5 +141,5 @@
false
]
},
"hash": "7f9b7ab9bec6a0f745273d0cd5602ceab46a7ec9fd225f7b9d16a2ddb9bad7b3"
"hash": "a33673ebc4d1eb4c3513987dbc43e2c80974598e1d9fe7203145bfc29928ba65"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE folder SET owners = array_remove(owners, $1::varchar) WHERE name = $2 AND workspace_id = $3 RETURNING name",
"query": "UPDATE folder SET owners = array_remove(owners, $1::varchar) WHERE name = $2 AND workspace_id = $3 AND $1 = ANY(owners) RETURNING name",
"describe": {
"columns": [
{
@@ -11,7 +11,7 @@
],
"parameters": {
"Left": [
"Varchar",
"Text",
"Text",
"Text"
]
@@ -20,5 +20,5 @@
false
]
},
"hash": "e500a422fa986faf5c612c08f4a139e10b25a3106551e7c92f43dcf9758a5ec7"
"hash": "a3ca0af5d84acba93776828c090a79bf6554180b9db4951b69f7070f769b2962"
}

View File

@@ -184,8 +184,7 @@
"postgres",
"sqs",
"gcp",
"mqtt",
"nextcloud"
"mqtt"
]
}
}

View File

@@ -0,0 +1,59 @@
{
"db_name": "PostgreSQL",
"query": "SELECT hash, created_at, content, summary, description, lock, schema\n FROM script\n WHERE workspace_id = $1 AND path = $2 AND archived = false\n ORDER BY created_at DESC\n LIMIT 1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "hash",
"type_info": "Int8"
},
{
"ordinal": 1,
"name": "created_at",
"type_info": "Timestamptz"
},
{
"ordinal": 2,
"name": "content",
"type_info": "Text"
},
{
"ordinal": 3,
"name": "summary",
"type_info": "Text"
},
{
"ordinal": 4,
"name": "description",
"type_info": "Text"
},
{
"ordinal": 5,
"name": "lock",
"type_info": "Text"
},
{
"ordinal": 6,
"name": "schema",
"type_info": "Json"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
false,
false,
false,
false,
true,
true
]
},
"hash": "a6fe41e36e06c88e1387abaea17107ceab2c5b17258958bde03b3e492bb2790e"
}

View File

@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT progress::bigint FROM tutorial_progress WHERE email = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "progress",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
null
]
},
"hash": "a82eec879838b02e3e0722352fba9f537374dc0658733f7e15bfe42f646d22e2"
}

View File

@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_diff (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)\n VALUES ($1, $2, $3, $4, 1, 0, NULL)\n ON CONFLICT (source_workspace_id, fork_workspace_id, path, kind)\n DO UPDATE SET\n ahead = workspace_diff.ahead + 1,\n has_changes = NULL",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar",
"Varchar"
]
},
"nullable": []
},
"hash": "ac3001bd72248efe36ae3c96af5450f0721f888d18475fae862d675d40779a30"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "\n UPDATE workspace_settings\n SET git_app_installations = (\n SELECT COALESCE(jsonb_agg(elem), '[]'::jsonb)\n FROM jsonb_array_elements(git_app_installations) AS elem\n WHERE (elem->>'installation_id')::bigint != $1\n )\n WHERE workspace_id = $2\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Int8",
"Text"
]
},
"nullable": []
},
"hash": "ad288f9f242f930fc571d04d5ba217f653437f60c890b6b73059fdaa7eb45302"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM workspace_diff WHERE has_changes = false AND (\n (source_workspace_id = $1 AND fork_workspace_id = $2)\n OR (source_workspace_id = $2 AND fork_workspace_id =$1)\n )",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": []
},
"hash": "af6aea925527c486e56375b6cd215b0e36938a5853285be485647bc8cc846973"
}

View File

@@ -0,0 +1,49 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id, changed_by, changed_at, change_type, member_affected\n FROM group_permission_history\n WHERE workspace_id = $1 AND group_name = $2\n ORDER BY id DESC\n LIMIT $3 OFFSET $4",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Int8"
},
{
"ordinal": 1,
"name": "changed_by",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "changed_at",
"type_info": "Timestamptz"
},
{
"ordinal": 3,
"name": "change_type",
"type_info": "Varchar"
},
{
"ordinal": 4,
"name": "member_affected",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Int8",
"Int8"
]
},
"nullable": [
false,
false,
false,
false,
true
]
},
"hash": "b11034489786626184e2f6b7f99469a945a4bbfe0e6a33c5a3e009dadd56d52c"
}

View File

@@ -104,8 +104,7 @@
"postgres",
"sqs",
"gcp",
"mqtt",
"nextcloud"
"mqtt"
]
}
}

View File

@@ -0,0 +1,49 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id, changed_by, changed_at, change_type, affected\n FROM folder_permission_history\n WHERE workspace_id = $1 AND folder_name = $2\n ORDER BY id DESC\n LIMIT $3 OFFSET $4",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Int8"
},
{
"ordinal": 1,
"name": "changed_by",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "changed_at",
"type_info": "Timestamptz"
},
{
"ordinal": 3,
"name": "change_type",
"type_info": "Varchar"
},
{
"ordinal": 4,
"name": "affected",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Int8",
"Int8"
]
},
"nullable": [
false,
false,
false,
false,
true
]
},
"hash": "b24eeafc2fc26664ef38d15d3cb47ca19549bcd9372d7a61ec2e28d82897bacf"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id, q.workspace_id, j.runnable_id as \"runnable_id: ScriptHash\", scheduled_for, parent_job, flow_innermost_root_job, runnable_path, kind as \"kind: JobKind\", started_at, permissioned_as, created_by, script_lang as \"script_lang: ScriptLang\", \n permissioned_as_email, flow_step_id, trigger_kind as \"trigger_kind: JobTriggerKind\", trigger, q.priority, concurrent_limit, q.tag, cache_ttl, cache_ignore_s3_path, r.ping as last_ping, worker, memory_peak, running\n FROM v2_job_queue q JOIN v2_job j USING (id) LEFT JOIN v2_job_runtime r USING (id) LEFT JOIN v2_job_status s USING (id)\n WHERE j.id = $1",
"query": "SELECT\n id,\n q.runnable_settings_handle,\n q.workspace_id,\n j.runnable_id as \"runnable_id: ScriptHash\",\n scheduled_for,\n parent_job,\n flow_innermost_root_job,\n runnable_path,\n kind as \"kind: JobKind\",\n started_at,\n permissioned_as,\n created_by,\n script_lang as \"script_lang: ScriptLang\",\n permissioned_as_email,\n flow_step_id,\n trigger_kind as \"trigger_kind: JobTriggerKind\",\n trigger,\n q.priority,\n concurrent_limit,\n q.tag,\n cache_ttl,\n cache_ignore_s3_path,\n r.ping as last_ping,\n worker,\n memory_peak,\n running\n FROM v2_job_queue q\n JOIN v2_job j USING (id)\n LEFT JOIN v2_job_runtime r USING (id)\n LEFT JOIN v2_job_status s USING (id)\n WHERE j.id = $1",
"describe": {
"columns": [
{
@@ -10,36 +10,41 @@
},
{
"ordinal": 1,
"name": "runnable_settings_handle",
"type_info": "Int8"
},
{
"ordinal": 2,
"name": "workspace_id",
"type_info": "Varchar"
},
{
"ordinal": 2,
"ordinal": 3,
"name": "runnable_id: ScriptHash",
"type_info": "Int8"
},
{
"ordinal": 3,
"ordinal": 4,
"name": "scheduled_for",
"type_info": "Timestamptz"
},
{
"ordinal": 4,
"ordinal": 5,
"name": "parent_job",
"type_info": "Uuid"
},
{
"ordinal": 5,
"ordinal": 6,
"name": "flow_innermost_root_job",
"type_info": "Uuid"
},
{
"ordinal": 6,
"ordinal": 7,
"name": "runnable_path",
"type_info": "Varchar"
},
{
"ordinal": 7,
"ordinal": 8,
"name": "kind: JobKind",
"type_info": {
"Custom": {
@@ -74,22 +79,22 @@
}
},
{
"ordinal": 8,
"ordinal": 9,
"name": "started_at",
"type_info": "Timestamptz"
},
{
"ordinal": 9,
"ordinal": 10,
"name": "permissioned_as",
"type_info": "Varchar"
},
{
"ordinal": 10,
"ordinal": 11,
"name": "created_by",
"type_info": "Varchar"
},
{
"ordinal": 11,
"ordinal": 12,
"name": "script_lang: ScriptLang",
"type_info": {
"Custom": {
@@ -125,17 +130,17 @@
}
},
{
"ordinal": 12,
"ordinal": 13,
"name": "permissioned_as_email",
"type_info": "Varchar"
},
{
"ordinal": 13,
"ordinal": 14,
"name": "flow_step_id",
"type_info": "Varchar"
},
{
"ordinal": 14,
"ordinal": 15,
"name": "trigger_kind: JobTriggerKind",
"type_info": {
"Custom": {
@@ -154,60 +159,59 @@
"postgres",
"sqs",
"gcp",
"mqtt",
"nextcloud"
"mqtt"
]
}
}
}
},
{
"ordinal": 15,
"ordinal": 16,
"name": "trigger",
"type_info": "Varchar"
},
{
"ordinal": 16,
"ordinal": 17,
"name": "priority",
"type_info": "Int2"
},
{
"ordinal": 17,
"ordinal": 18,
"name": "concurrent_limit",
"type_info": "Int4"
},
{
"ordinal": 18,
"ordinal": 19,
"name": "tag",
"type_info": "Varchar"
},
{
"ordinal": 19,
"ordinal": 20,
"name": "cache_ttl",
"type_info": "Int4"
},
{
"ordinal": 20,
"ordinal": 21,
"name": "cache_ignore_s3_path",
"type_info": "Bool"
},
{
"ordinal": 21,
"ordinal": 22,
"name": "last_ping",
"type_info": "Timestamptz"
},
{
"ordinal": 22,
"ordinal": 23,
"name": "worker",
"type_info": "Varchar"
},
{
"ordinal": 23,
"ordinal": 24,
"name": "memory_peak",
"type_info": "Int4"
},
{
"ordinal": 24,
"ordinal": 25,
"name": "running",
"type_info": "Bool"
}
@@ -219,6 +223,7 @@
},
"nullable": [
false,
true,
false,
true,
false,
@@ -245,5 +250,5 @@
false
]
},
"hash": "a84e67035584bbdb02482026b9cc0808086c50f78947d43bb88628a481f41a1d"
"hash": "b3771b690c5966272b1f42c9965bb6a8f961c119516e4c33dc928cd3b4f4edbc"
}

View File

@@ -18,8 +18,8 @@
"Left": []
},
"nullable": [
false,
true
true,
false
]
},
"hash": "b3dbdfb50ee8118bdaed3164b210cb549a34b96554ae1872355b90304f5dcb76"

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO script (workspace_id, hash, path, parent_hashes, summary, description, content, created_by, schema, is_template, extra_perms, lock, language, kind, tag, draft_only, envs, concurrent_limit, concurrency_time_window_s, cache_ttl, dedicated_worker, ws_error_handler_muted, priority, restart_unless_cancelled, delete_after_use, timeout, concurrency_key, visible_to_runner_only, no_main_func, codebase, has_preprocessor, on_behalf_of_email, schema_validation, assets, debounce_key, debounce_delay_s, cache_ignore_s3_path) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9::text::json, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21, $22, $23, $24, $25, $26, $27, $28, $29, $30, $31, $32, $33, $34, $35, $36, $37)",
"query": "INSERT INTO script (workspace_id, hash, path, parent_hashes, summary, description, content, created_by, schema, is_template, extra_perms, lock, language, kind, tag, draft_only, envs, concurrent_limit, concurrency_time_window_s, cache_ttl, dedicated_worker, ws_error_handler_muted, priority, restart_unless_cancelled, delete_after_use, timeout, concurrency_key, visible_to_runner_only, no_main_func, codebase, has_preprocessor, on_behalf_of_email, schema_validation, assets, debounce_key, debounce_delay_s, cache_ignore_s3_path, runnable_settings_handle) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9::text::json, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21, $22, $23, $24, $25, $26, $27, $28, $29, $30, $31, $32, $33, $34, $35, $36, $37, $38)",
"describe": {
"columns": [],
"parameters": {
@@ -59,7 +59,9 @@
"failure",
"command",
"approval",
"preprocessor"
"preprocessor",
"schedule_handler_old",
"dynamic_skip"
]
}
}
@@ -86,10 +88,11 @@
"Jsonb",
"Varchar",
"Int4",
"Bool"
"Bool",
"Int8"
]
},
"nullable": []
},
"hash": "3d05d9d7e087eb6e1c14c2b8a20598581e6c7493ed99cb9ad1c2ee5d0b212d38"
"hash": "b4eb72b0274cbdce7490f63c36d0d16ee847294fadc138593a1baa417cbb3652"
}

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id FROM v2_job_queue WHERE workspace_id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Uuid"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false
]
},
"hash": "b9c520fdc29b17977bf1c0b30311c536ebea8a76b75cda188ec91fcc0fc274a7"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT path FROM flow\n WHERE workspace_id = $1 AND path = ANY($2) AND archived = false",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"TextArray"
]
},
"nullable": [
false
]
},
"hash": "c5897802334f09596a0297f9b9a1e24f3c6f8b1628965438a9360c073d70924e"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT\n jsonb_strip_nulls(jsonb_build_object(\n 'path', asset.path,\n 'kind', asset.kind,\n 'usages', ARRAY_AGG(jsonb_build_object(\n 'path', asset.usage_path,\n 'kind', asset.usage_kind,\n 'access_type', asset.usage_access_type\n )),\n 'metadata', (CASE\n WHEN asset.kind = 'resource' THEN\n jsonb_build_object('resource_type', resource.resource_type)\n ELSE\n NULL\n END\n )\n )) as \"list!: _\"\n FROM asset\n LEFT JOIN resource ON asset.kind = 'resource' AND asset.path = resource.path AND resource.workspace_id = $1\n WHERE asset.workspace_id = $1\n AND (asset.kind <> 'resource' OR resource.path IS NOT NULL)\n AND (asset.usage_kind <> 'flow' OR asset.usage_path = ANY(SELECT path FROM flow WHERE workspace_id = $1))\n AND (asset.usage_kind <> 'script' OR asset.usage_path = ANY(SELECT path FROM script WHERE workspace_id = $1))\n GROUP BY asset.path, asset.kind, resource.resource_type\n ORDER BY asset.path, asset.kind",
"query": "SELECT\n jsonb_strip_nulls(jsonb_build_object(\n 'path', asset.path,\n 'kind', asset.kind,\n 'usages', ARRAY_AGG(jsonb_build_object(\n 'path', asset.usage_path,\n 'kind', asset.usage_kind,\n 'access_type', asset.usage_access_type\n )),\n 'metadata', (CASE\n WHEN asset.kind = 'resource' THEN\n jsonb_build_object('resource_type', resource.resource_type)\n ELSE\n NULL\n END\n )\n )) as \"list!: _\"\n FROM asset\n LEFT JOIN resource ON asset.kind = 'resource'\n AND array_to_string((string_to_array(asset.path, '/'))[1:3], '/') = resource.path -- With specific table, asset path can be e.g u/diego/pg_db/table_name\n AND resource.workspace_id = $1\n WHERE asset.workspace_id = $1\n AND (asset.kind <> 'resource' OR resource.path IS NOT NULL)\n AND (asset.usage_kind <> 'flow' OR asset.usage_path = ANY(SELECT path FROM flow WHERE workspace_id = $1))\n AND (asset.usage_kind <> 'script' OR asset.usage_path = ANY(SELECT path FROM script WHERE workspace_id = $1))\n GROUP BY asset.path, asset.kind, resource.resource_type\n ORDER BY asset.path, asset.kind",
"describe": {
"columns": [
{
@@ -18,5 +18,5 @@
null
]
},
"hash": "92f03f4df5e86eb40b255ad0f2cc85e0302c37b0f312366098104cd280a91ef6"
"hash": "c94cd50ff1233025b170efee6489e6637676a8b0435a4612a7efbfff6ea2543d"
}

View File

@@ -0,0 +1,41 @@
{
"db_name": "PostgreSQL",
"query": "SELECT value, summary, description, schema\n FROM flow\n WHERE workspace_id = $1 AND path = $2 AND archived = false",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "value",
"type_info": "Jsonb"
},
{
"ordinal": 1,
"name": "summary",
"type_info": "Text"
},
{
"ordinal": 2,
"name": "description",
"type_info": "Text"
},
{
"ordinal": 3,
"name": "schema",
"type_info": "Json"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
false,
false,
true
]
},
"hash": "dc6ffc9946325e794f3a9d1cf621a9c07d211351195c399f6ff25c1dc7adb01a"
}

View File

@@ -0,0 +1,20 @@
{
"db_name": "PostgreSQL",
"query": "SELECT COUNT(*) FROM instance_group",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "count",
"type_info": "Int8"
}
],
"parameters": {
"Left": []
},
"nullable": [
null
]
},
"hash": "e1c1e25053ae4b1635780c7e472c9d86806fa8cc762786b7a29bd121837c8ebc"
}

View File

@@ -184,8 +184,7 @@
"postgres",
"sqs",
"gcp",
"mqtt",
"nextcloud"
"mqtt"
]
}
}

View File

@@ -0,0 +1,28 @@
{
"db_name": "PostgreSQL",
"query": "SELECT concurrency_settings, debouncing_settings FROM runnable_settings WHERE hash = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "concurrency_settings",
"type_info": "Int8"
},
{
"ordinal": 1,
"name": "debouncing_settings",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Int8"
]
},
"nullable": [
true,
true
]
},
"hash": "ebbbd069e0f33be9609604025d159fe1ecbefc2e9c11f7c4900b7121d4367e01"
}

View File

@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "\n UPDATE workspace_settings\n SET teams_team_id = $1, teams_team_name = $2, teams_team_guid = $3\n WHERE workspace_id = $4\n AND NOT EXISTS (\n SELECT 1 FROM workspace_settings\n WHERE teams_team_id = $1 AND workspace_id <> $4\n )\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "f034f7b0118ad467c7399c5554eb916d5a9716ca0d638e3bc65509b476db378e"
}

View File

@@ -0,0 +1,18 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO folder_permission_history\n (workspace_id, folder_name, changed_by, change_type, affected)\n VALUES ($1, $2, $3, $4, $5)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar",
"Varchar",
"Varchar"
]
},
"nullable": []
},
"hash": "f1206b00c4b81e59943385646d653330efba9cd1e731621f7efc91f04567841f"
}

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id FROM instance_group WHERE name = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
true
]
},
"hash": "f2eb05fe3581772d985e2ace82706d824b208ee9a1cb65a85be389e14672620c"
}

View File

@@ -0,0 +1,18 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO group_permission_history\n (workspace_id, group_name, changed_by, change_type, member_affected)\n VALUES ($1, $2, $3, $4, $5)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar",
"Varchar",
"Varchar"
]
},
"nullable": []
},
"hash": "f95358255e55d68dd453d173e23ab3cb1f1c2ba5b1cfc78f706b0b014f045477"
}

View File

@@ -104,8 +104,7 @@
"postgres",
"sqs",
"gcp",
"mqtt",
"nextcloud"
"mqtt"
]
}
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT path FROM variable\n WHERE workspace_id = $1 AND path = ANY($2)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"TextArray"
]
},
"nullable": [
false
]
},
"hash": "ff24cfb1f603b4ea612e48cee5880fc516207c8a474446a1ee09e8324f85ba09"
}

293
backend/Cargo.lock generated
View File

@@ -559,6 +559,22 @@ dependencies = [
"syn 2.0.111",
]
[[package]]
name = "astral-tokio-tar"
version = "0.5.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ec179a06c1769b1e42e1e2cbe74c7dcdb3d6383c838454d063eaac5bbb7ebbe5"
dependencies = [
"filetime",
"futures-core",
"libc",
"portable-atomic",
"rustc-hash 2.1.1",
"tokio",
"tokio-stream",
"xattr",
]
[[package]]
name = "async-broadcast"
version = "0.7.2"
@@ -788,9 +804,9 @@ dependencies = [
[[package]]
name = "aws-credential-types"
version = "1.2.10"
version = "1.2.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b01c9521fa01558f750d183c8c68c81b0155b9d193a4ba7f84c36bd1b6d04a06"
checksum = "3cd362783681b15d136480ad555a099e82ecd8e2d10a841e14dfd0078d67fee3"
dependencies = [
"aws-smithy-async",
"aws-smithy-runtime-api",
@@ -800,9 +816,9 @@ dependencies = [
[[package]]
name = "aws-lc-rs"
version = "1.15.1"
version = "1.15.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6b5ce75405893cd713f9ab8e297d8e438f624dde7d706108285f7e17a25a180f"
checksum = "6a88aab2464f1f25453baa7a07c84c5b7684e274054ba06817f382357f77a288"
dependencies = [
"aws-lc-sys",
"zeroize",
@@ -810,9 +826,9 @@ dependencies = [
[[package]]
name = "aws-lc-sys"
version = "0.34.0"
version = "0.35.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "179c3777a8b5e70e90ea426114ffc565b2c1a9f82f6c4a0c5a34aa6ef5e781b6"
checksum = "b45afffdee1e7c9126814751f88dddc747f41d91da16c9551a0f1e8a11e788a1"
dependencies = [
"cc",
"cmake",
@@ -822,9 +838,9 @@ dependencies = [
[[package]]
name = "aws-runtime"
version = "1.5.16"
version = "1.5.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7ce527fb7e53ba9626fc47824f25e256250556c40d8f81d27dd92aa38239d632"
checksum = "d81b5b2898f6798ad58f484856768bca817e3cd9de0974c24ae0f1113fe88f1b"
dependencies = [
"aws-credential-types",
"aws-sigv4",
@@ -895,9 +911,9 @@ dependencies = [
[[package]]
name = "aws-sdk-rds"
version = "1.119.0"
version = "1.121.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "70fbf24cf9f84d18f6296857982dd970dd1c23a54a3bac208c7e9abc13a35c1b"
checksum = "15640ecd44797fa9396c66793bd06e57f63f66419117f83518f4d51057b01945"
dependencies = [
"aws-credential-types",
"aws-runtime",
@@ -1009,9 +1025,9 @@ dependencies = [
[[package]]
name = "aws-sigv4"
version = "1.3.6"
version = "1.3.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c35452ec3f001e1f2f6db107b6373f1f48f05ec63ba2c5c9fa91f07dad32af11"
checksum = "69e523e1c4e8e7e8ff219d732988e22bfeae8a1cafdbe6d9eca1546fa080be7c"
dependencies = [
"aws-credential-types",
"aws-smithy-eventstream",
@@ -1032,9 +1048,9 @@ dependencies = [
[[package]]
name = "aws-smithy-async"
version = "1.2.6"
version = "1.2.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "127fcfad33b7dfc531141fda7e1c402ac65f88aca5511a4d31e2e3d2cd01ce9c"
checksum = "9ee19095c7c4dda59f1697d028ce704c24b2d33c6718790c7f1d5a3015b4107c"
dependencies = [
"futures-util",
"pin-project-lite",
@@ -1043,9 +1059,9 @@ dependencies = [
[[package]]
name = "aws-smithy-eventstream"
version = "0.60.13"
version = "0.60.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e29a304f8319781a39808847efb39561351b1bb76e933da7aa90232673638658"
checksum = "dc12f8b310e38cad85cf3bef45ad236f470717393c613266ce0a89512286b650"
dependencies = [
"aws-smithy-types",
"bytes",
@@ -1054,9 +1070,9 @@ dependencies = [
[[package]]
name = "aws-smithy-http"
version = "0.62.5"
version = "0.62.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "445d5d720c99eed0b4aa674ed00d835d9b1427dd73e04adaf2f94c6b2d6f9fca"
checksum = "826141069295752372f8203c17f28e30c464d22899a43a0c9fd9c458d469c88b"
dependencies = [
"aws-smithy-eventstream",
"aws-smithy-runtime-api",
@@ -1076,9 +1092,9 @@ dependencies = [
[[package]]
name = "aws-smithy-http-client"
version = "1.1.4"
version = "1.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "623254723e8dfd535f566ee7b2381645f8981da086b5c4aa26c0c41582bb1d2c"
checksum = "59e62db736db19c488966c8d787f52e6270be565727236fd5579eaa301e7bc4a"
dependencies = [
"aws-smithy-async",
"aws-smithy-runtime-api",
@@ -1106,27 +1122,27 @@ dependencies = [
[[package]]
name = "aws-smithy-json"
version = "0.61.7"
version = "0.61.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2db31f727935fc63c6eeae8b37b438847639ec330a9161ece694efba257e0c54"
checksum = "49fa1213db31ac95288d981476f78d05d9cbb0353d22cdf3472cc05bb02f6551"
dependencies = [
"aws-smithy-types",
]
[[package]]
name = "aws-smithy-observability"
version = "0.1.4"
version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2d1881b1ea6d313f9890710d65c158bdab6fb08c91ea825f74c1c8c357baf4cc"
checksum = "17f616c3f2260612fe44cede278bafa18e73e6479c4e393e2c4518cf2a9a228a"
dependencies = [
"aws-smithy-runtime-api",
]
[[package]]
name = "aws-smithy-query"
version = "0.60.8"
version = "0.60.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d28a63441360c477465f80c7abac3b9c4d075ca638f982e605b7dc2a2c7156c9"
checksum = "ae5d689cf437eae90460e944a58b5668530d433b4ff85789e69d2f2a556e057d"
dependencies = [
"aws-smithy-types",
"urlencoding",
@@ -1134,9 +1150,9 @@ dependencies = [
[[package]]
name = "aws-smithy-runtime"
version = "1.9.4"
version = "1.9.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0bbe9d018d646b96c7be063dd07987849862b0e6d07c778aad7d93d1be6c1ef0"
checksum = "65fda37911905ea4d3141a01364bc5509a0f32ae3f3b22d6e330c0abfb62d247"
dependencies = [
"aws-smithy-async",
"aws-smithy-http",
@@ -1158,9 +1174,9 @@ dependencies = [
[[package]]
name = "aws-smithy-runtime-api"
version = "1.9.2"
version = "1.9.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ec7204f9fd94749a7c53b26da1b961b4ac36bf070ef1e0b94bb09f79d4f6c193"
checksum = "ab0d43d899f9e508300e587bf582ba54c27a452dd0a9ea294690669138ae14a2"
dependencies = [
"aws-smithy-async",
"aws-smithy-types",
@@ -1175,9 +1191,9 @@ dependencies = [
[[package]]
name = "aws-smithy-types"
version = "1.3.4"
version = "1.3.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "25f535879a207fce0db74b679cfc3e91a3159c8144d717d55f5832aea9eef46e"
checksum = "905cb13a9895626d49cf2ced759b062d913834c7482c38e49557eac4e6193f01"
dependencies = [
"base64-simd 0.8.0",
"bytes",
@@ -1201,9 +1217,9 @@ dependencies = [
[[package]]
name = "aws-smithy-types-convert"
version = "0.60.10"
version = "0.60.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c99945c7033e37bfe8dc76c19862ebd7544b14b4f34a8551ccf2690b433eba0e"
checksum = "b70bc27e41d5ed80b376602ff4becdab6ea8489403fad3abbfea2c9c825c1e1e"
dependencies = [
"aws-smithy-types",
"chrono",
@@ -1211,18 +1227,18 @@ dependencies = [
[[package]]
name = "aws-smithy-xml"
version = "0.60.12"
version = "0.60.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "eab77cdd036b11056d2a30a7af7b775789fb024bf216acc13884c6c97752ae56"
checksum = "11b2f670422ff42bf7065031e72b45bc52a3508bd089f743ea90731ca2b6ea57"
dependencies = [
"xmlparser",
]
[[package]]
name = "aws-types"
version = "1.3.10"
version = "1.3.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d79fb68e3d7fe5d4833ea34dc87d2e97d26d3086cb3da660bb6b1f76d98680b6"
checksum = "1d980627d2dd7bfc32a3c025685a033eeab8d365cc840c631ef59d1b8f428164"
dependencies = [
"aws-credential-types",
"aws-smithy-async",
@@ -1793,9 +1809,9 @@ dependencies = [
[[package]]
name = "bumpalo"
version = "3.19.0"
version = "3.19.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "46c5e41b57b8bba42a04676d81cb89e9ee8e859a1a66f80a5a72e1cb76b34d43"
checksum = "5dd9dc738b7a8311c7ade152424974d8115f2cdad61e8dab8dac9f2362298510"
dependencies = [
"allocator-api2",
]
@@ -2008,9 +2024,9 @@ dependencies = [
[[package]]
name = "cc"
version = "1.2.49"
version = "1.2.50"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "90583009037521a116abf44494efecd645ba48b6622457080f080b85544e2215"
checksum = "9f50d563227a1c37cc0a263f64eca3334388c01c5e4c4861a9def205c614383c"
dependencies = [
"find-msvc-tools",
"jobserver",
@@ -2166,9 +2182,9 @@ dependencies = [
[[package]]
name = "cmake"
version = "0.1.54"
version = "0.1.57"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e7caa3f9de89ddbe2c607f4101924c5abec803763ae9534e4f4d7d8f84aa81f0"
checksum = "75443c44cd6b379beb8c5b45d85d0773baf31cce901fe7bb252f4eff3008ef7d"
dependencies = [
"cc",
]
@@ -2822,7 +2838,7 @@ dependencies = [
"parquet",
"rand 0.8.5",
"regex",
"sqlparser",
"sqlparser 0.55.0",
"tempfile",
"tokio",
"url",
@@ -2899,7 +2915,7 @@ dependencies = [
"parquet",
"paste",
"recursive",
"sqlparser",
"sqlparser 0.55.0",
"tokio",
"web-time",
]
@@ -3075,7 +3091,7 @@ dependencies = [
"paste",
"recursive",
"serde_json",
"sqlparser",
"sqlparser 0.55.0",
]
[[package]]
@@ -3371,7 +3387,7 @@ dependencies = [
"log",
"recursive",
"regex",
"sqlparser",
"sqlparser 0.55.0",
]
[[package]]
@@ -3404,7 +3420,7 @@ dependencies = [
"once_cell",
"percent-encoding",
"serde",
"sourcemap 9.3.0",
"sourcemap 9.3.1",
"swc_atoms",
"swc_common",
"swc_config",
@@ -5368,9 +5384,9 @@ checksum = "1d674e81391d1e1ab681a28d99df07927c6d4aa5b027d7da16ba32d1d21ecd99"
[[package]]
name = "flatbuffers"
version = "25.9.23"
version = "25.12.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "09b6620799e7340ebd9968d2e0708eb82cf1971e9a16821e2091b6d6e475eed5"
checksum = "35f6839d7b3b98adde531effaf34f0c2badc6f4735d26fe74709d8e513a96ef3"
dependencies = [
"bitflags 2.9.4",
"rustc_version 0.4.1",
@@ -6922,9 +6938,9 @@ checksum = "7aedcccd01fc5fe81e6b489c15b247b8b0690feb23304303a9e560f37efc560a"
[[package]]
name = "icu_properties"
version = "2.1.1"
version = "2.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e93fcd3157766c0c8da2f8cff6ce651a31f0810eaa1c51ec363ef790bbb5fb99"
checksum = "020bfc02fe870ec3a66d93e677ccca0562506e5872c650f893269e08615d74ec"
dependencies = [
"icu_collections",
"icu_locale_core",
@@ -6936,9 +6952,9 @@ dependencies = [
[[package]]
name = "icu_properties_data"
version = "2.1.1"
version = "2.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "02845b3647bb045f1100ecd6480ff52f34c35f82d9880e029d329c21d1054899"
checksum = "616c294cf8d725c6afcd8f55abc17c56464ef6211f9ed59cccffe534129c77af"
[[package]]
name = "icu_provider"
@@ -7723,13 +7739,13 @@ dependencies = [
[[package]]
name = "libredox"
version = "0.1.10"
version = "0.1.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "416f7e718bdb06000964960ffa43b4335ad4012ae8b99060261aa4a8088d5ccb"
checksum = "df15f6eac291ed1cf25865b1ee60399f57e7c227e7f51bdbd4c5270396a9ed50"
dependencies = [
"bitflags 2.9.4",
"libc",
"redox_syscall 0.5.18",
"redox_syscall 0.6.0",
]
[[package]]
@@ -7766,9 +7782,9 @@ dependencies = [
[[package]]
name = "libz-rs-sys"
version = "0.5.3"
version = "0.5.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8b484ba8d4f775eeca644c452a56650e544bf7e617f1d170fe7298122ead5222"
checksum = "15413ef615ad868d4d65dce091cb233b229419c7c0c4bcaa746c0901c49ff39c"
dependencies = [
"zlib-rs",
]
@@ -8602,15 +8618,6 @@ dependencies = [
"minimal-lexical",
]
[[package]]
name = "nom"
version = "8.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df9761775871bdef83bee530e60050f7e54b1105350d6884eb0fb4f46c2f9405"
dependencies = [
"memchr",
]
[[package]]
name = "notify"
version = "6.1.1"
@@ -8632,9 +8639,9 @@ dependencies = [
[[package]]
name = "ntapi"
version = "0.4.1"
version = "0.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e8a3895c6391c39d7fe7ebc444a87eb2991b2a0bc718fdabd071eec617fc68e4"
checksum = "c70f219e21142367c70c0b30c6a9e3a14d55b4d12a204d897fbec83a0363f081"
dependencies = [
"winapi",
]
@@ -9810,9 +9817,9 @@ dependencies = [
[[package]]
name = "portable-atomic"
version = "1.11.1"
version = "1.12.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f84267b20a16ea918e43c6a88433c2d54fa145c92a811b5b047ccbe153674483"
checksum = "f59e70c4aef1e55797c2e8fd94a4f2a973fc972cfde0e0b05f683667b0cd39dd"
[[package]]
name = "postgres-native-tls"
@@ -10563,18 +10570,18 @@ dependencies = [
[[package]]
name = "redox_syscall"
version = "0.3.5"
version = "0.5.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "567664f262709473930a4bf9e51bf2ebf3348f2e748ccc50dea20646858f8f29"
checksum = "ed2bf2547551a7053d6fdfafda3f938979645c44812fbfcda098faae3f1a362d"
dependencies = [
"bitflags 1.3.2",
"bitflags 2.9.4",
]
[[package]]
name = "redox_syscall"
version = "0.5.18"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed2bf2547551a7053d6fdfafda3f938979645c44812fbfcda098faae3f1a362d"
checksum = "ec96166dafa0886eb81fe1c0a388bece180fbef2135f97c1e2cf8302e74b43b5"
dependencies = [
"bitflags 2.9.4",
]
@@ -11233,9 +11240,9 @@ dependencies = [
[[package]]
name = "rustls-pki-types"
version = "1.13.1"
version = "1.13.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "708c0f9d5f54ba0272468c1d306a52c495b31fa155e91bc25371e6df7996908c"
checksum = "21e6f2ab2928ca4291b86736a8bd920a277a399bba1589409d72154ff87c1282"
dependencies = [
"web-time",
"zeroize",
@@ -12175,9 +12182,9 @@ dependencies = [
[[package]]
name = "sourcemap"
version = "9.3.0"
version = "9.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c8131753a3c444a6177b92fc9e1bd0d2ecaf1c8953e6a41cc599e8e7ca317ef2"
checksum = "37ccaaa78a0ca68b20f8f711eaa2522a00131c48a3de5b892ca5c36cec1ce9bb"
dependencies = [
"base64-simd 0.8.0",
"bitvec",
@@ -12264,6 +12271,17 @@ dependencies = [
"sqlparser_derive",
]
[[package]]
name = "sqlparser"
version = "0.59.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4591acadbcf52f0af60eafbb2c003232b2b4cd8de5f0e9437cb8b1b59046cc0f"
dependencies = [
"log",
"recursive",
"sqlparser_derive",
]
[[package]]
name = "sqlparser_derive"
version = "0.3.0"
@@ -12625,9 +12643,9 @@ dependencies = [
[[package]]
name = "supports-hyperlinks"
version = "3.1.0"
version = "3.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "804f44ed3c63152de6a9f90acbea1a110441de43006ea51bcce8f436196a288b"
checksum = "e396b6523b11ccb83120b115a0b7366de372751aa6edf19844dfb13a6af97e91"
[[package]]
name = "supports-unicode"
@@ -12691,7 +12709,7 @@ dependencies = [
"rustc-hash 1.1.0",
"serde",
"siphasher 0.3.11",
"sourcemap 9.3.0",
"sourcemap 9.3.1",
"swc_allocator",
"swc_atoms",
"swc_eq_ignore_macros",
@@ -12755,7 +12773,7 @@ dependencies = [
"num-bigint",
"once_cell",
"serde",
"sourcemap 9.3.0",
"sourcemap 9.3.1",
"swc_allocator",
"swc_atoms",
"swc_common",
@@ -13814,21 +13832,6 @@ dependencies = [
"tokio",
]
[[package]]
name = "tokio-tar"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9d5714c010ca3e5c27114c1cdeb9d14641ace49874aa5626d7149e47aedace75"
dependencies = [
"filetime",
"futures-core",
"libc",
"redox_syscall 0.3.5",
"tokio",
"tokio-stream",
"xattr",
]
[[package]]
name = "tokio-tungstenite"
version = "0.24.0"
@@ -13938,9 +13941,9 @@ dependencies = [
[[package]]
name = "toml_parser"
version = "1.0.4"
version = "1.0.6+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c0cbe268d35bdb4bb5a56a2de88d0ad0eb70af5384a99d648cd4b3d04039800e"
checksum = "a3198b4b0a8e11f09dd03e133c0280504d0801269e9afa46362ffde1cbeebf44"
dependencies = [
"winnow 0.7.14",
]
@@ -14074,9 +14077,9 @@ checksum = "8df9b6e13f2d32c91b9bd719c00d1958837bc7dec474d94952798cc8e69eeec3"
[[package]]
name = "tracing"
version = "0.1.43"
version = "0.1.44"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2d15d90a0b5c19378952d479dc858407149d7bb45a14de0142f6c534b16fc647"
checksum = "63e71662fa4b2a2c3a26f570f037eb95bb1f85397f3cd8076caed2f026a6d100"
dependencies = [
"log",
"pin-project-lite",
@@ -14109,9 +14112,9 @@ dependencies = [
[[package]]
name = "tracing-core"
version = "0.1.35"
version = "0.1.36"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7a04e24fab5c89c6a36eb8558c9656f30d81de51dfa4d3b45f26b21d61fa0a6c"
checksum = "db97caf9d906fbde555dd62fa95ddba9eecfd14cb388e4f491a66d74cd5fb79a"
dependencies = [
"once_cell",
"valuable",
@@ -14242,9 +14245,9 @@ dependencies = [
[[package]]
name = "tree-sitter-language"
version = "0.1.5"
version = "0.1.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c4013970217383f67b18aef68f6fb2e8d409bc5755227092d32efb0422ba24b8"
checksum = "4ae62f7eae5eb549c71b76658648b72cc6111f2d87d24a1e31fa907f4943e3ce"
[[package]]
name = "tree-sitter-ruby"
@@ -15163,7 +15166,7 @@ checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
[[package]]
name = "windmill"
version = "1.590.0"
version = "1.596.0"
dependencies = [
"anyhow",
"aws-sdk-config",
@@ -15184,7 +15187,6 @@ dependencies = [
"lazy_static",
"libloading 0.8.9",
"memchr",
"nom 8.0.0",
"object_store",
"once_cell",
"pep440_rs",
@@ -15200,6 +15202,7 @@ dependencies = [
"sha1",
"sha2 0.10.9",
"size",
"sql-builder",
"sqlx",
"strum 0.27.2",
"systemstat",
@@ -15226,10 +15229,11 @@ dependencies = [
[[package]]
name = "windmill-api"
version = "1.590.0"
version = "1.596.0"
dependencies = [
"anyhow",
"argon2",
"astral-tokio-tar",
"async-nats",
"async-oauth2",
"async-recursion",
@@ -15237,6 +15241,7 @@ dependencies = [
"async-trait",
"async_zip",
"aws-config",
"aws-credential-types",
"aws-sdk-config",
"aws-sdk-sqs",
"aws-sdk-sso",
@@ -15319,7 +15324,6 @@ dependencies = [
"tokio-postgres 0.7.11",
"tokio-postgres 0.7.13",
"tokio-stream",
"tokio-tar",
"tokio-tungstenite",
"tokio-util",
"tonic",
@@ -15347,7 +15351,7 @@ dependencies = [
[[package]]
name = "windmill-api-client"
version = "1.590.0"
version = "1.596.0"
dependencies = [
"base64 0.22.1",
"chrono",
@@ -15362,7 +15366,7 @@ dependencies = [
[[package]]
name = "windmill-audit"
version = "1.590.0"
version = "1.596.0"
dependencies = [
"chrono",
"lazy_static",
@@ -15376,7 +15380,7 @@ dependencies = [
[[package]]
name = "windmill-autoscaling"
version = "1.590.0"
version = "1.596.0"
dependencies = [
"anyhow",
"axum",
@@ -15395,7 +15399,7 @@ dependencies = [
[[package]]
name = "windmill-common"
version = "1.590.0"
version = "1.596.0"
dependencies = [
"anyhow",
"async-recursion",
@@ -15433,6 +15437,7 @@ dependencies = [
"lazy_static",
"magic-crypt",
"mail-send",
"native-tls",
"object_store",
"once_cell",
"openidconnect",
@@ -15444,6 +15449,7 @@ dependencies = [
"pep440_rs",
"phf 0.11.3",
"pin-project-lite",
"postgres-native-tls 0.5.1",
"prometheus",
"quick_cache",
"rand 0.9.0",
@@ -15468,6 +15474,7 @@ dependencies = [
"thiserror 2.0.17",
"tikv-jemalloc-ctl",
"tokio",
"tokio-postgres 0.7.13",
"tokio-stream",
"tokio-util",
"tonic",
@@ -15488,7 +15495,7 @@ dependencies = [
[[package]]
name = "windmill-git-sync"
version = "1.590.0"
version = "1.596.0"
dependencies = [
"regex",
"serde",
@@ -15503,9 +15510,10 @@ dependencies = [
[[package]]
name = "windmill-indexer"
version = "1.590.0"
version = "1.596.0"
dependencies = [
"anyhow",
"astral-tokio-tar",
"bytes",
"chrono",
"const_format",
@@ -15519,7 +15527,6 @@ dependencies = [
"tantivy",
"tempfile",
"tokio",
"tokio-tar",
"tracing",
"uuid",
"windmill-common",
@@ -15527,7 +15534,7 @@ dependencies = [
[[package]]
name = "windmill-macros"
version = "1.590.0"
version = "1.596.0"
dependencies = [
"itertools 0.14.0",
"lazy_static",
@@ -15543,7 +15550,7 @@ dependencies = [
[[package]]
name = "windmill-parser"
version = "1.590.0"
version = "1.596.0"
dependencies = [
"convert_case 0.6.0",
"serde",
@@ -15552,7 +15559,7 @@ dependencies = [
[[package]]
name = "windmill-parser-bash"
version = "1.590.0"
version = "1.596.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -15564,7 +15571,7 @@ dependencies = [
[[package]]
name = "windmill-parser-csharp"
version = "1.590.0"
version = "1.596.0"
dependencies = [
"anyhow",
"serde_json",
@@ -15576,7 +15583,7 @@ dependencies = [
[[package]]
name = "windmill-parser-go"
version = "1.590.0"
version = "1.596.0"
dependencies = [
"anyhow",
"gosyn",
@@ -15588,7 +15595,7 @@ dependencies = [
[[package]]
name = "windmill-parser-graphql"
version = "1.590.0"
version = "1.596.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -15600,7 +15607,7 @@ dependencies = [
[[package]]
name = "windmill-parser-java"
version = "1.590.0"
version = "1.596.0"
dependencies = [
"anyhow",
"serde_json",
@@ -15612,7 +15619,7 @@ dependencies = [
[[package]]
name = "windmill-parser-nu"
version = "1.590.0"
version = "1.596.0"
dependencies = [
"anyhow",
"nu-parser",
@@ -15623,7 +15630,7 @@ dependencies = [
[[package]]
name = "windmill-parser-php"
version = "1.590.0"
version = "1.596.0"
dependencies = [
"anyhow",
"itertools 0.14.0",
@@ -15634,7 +15641,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py"
version = "1.590.0"
version = "1.596.0"
dependencies = [
"anyhow",
"itertools 0.14.0",
@@ -15642,11 +15649,12 @@ dependencies = [
"rustpython-parser",
"serde_json",
"windmill-parser",
"windmill-parser-sql",
]
[[package]]
name = "windmill-parser-py-imports"
version = "1.590.0"
version = "1.596.0"
dependencies = [
"anyhow",
"async-recursion",
@@ -15670,7 +15678,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ruby"
version = "1.590.0"
version = "1.596.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -15684,7 +15692,7 @@ dependencies = [
[[package]]
name = "windmill-parser-rust"
version = "1.590.0"
version = "1.596.0"
dependencies = [
"anyhow",
"convert_case 0.6.0",
@@ -15701,21 +15709,21 @@ dependencies = [
[[package]]
name = "windmill-parser-sql"
version = "1.590.0"
version = "1.596.0"
dependencies = [
"anyhow",
"lazy_static",
"nom 8.0.0",
"regex",
"regex-lite",
"serde",
"serde_json",
"sqlparser 0.59.0",
"windmill-parser",
]
[[package]]
name = "windmill-parser-ts"
version = "1.590.0"
version = "1.596.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -15729,11 +15737,12 @@ dependencies = [
"triomphe",
"wasm-bindgen",
"windmill-parser",
"windmill-parser-sql",
]
[[package]]
name = "windmill-parser-yaml"
version = "1.590.0"
version = "1.596.0"
dependencies = [
"anyhow",
"serde",
@@ -15744,7 +15753,7 @@ dependencies = [
[[package]]
name = "windmill-queue"
version = "1.590.0"
version = "1.596.0"
dependencies = [
"anyhow",
"async-recursion",
@@ -15781,7 +15790,7 @@ dependencies = [
[[package]]
name = "windmill-sql-datatype-parser-wasm"
version = "1.590.0"
version = "1.596.0"
dependencies = [
"wasm-bindgen",
"wasm-bindgen-test",
@@ -15791,7 +15800,7 @@ dependencies = [
[[package]]
name = "windmill-worker"
version = "1.590.0"
version = "1.596.0"
dependencies = [
"anyhow",
"async-once-cell",
@@ -16697,9 +16706,9 @@ dependencies = [
[[package]]
name = "zlib-rs"
version = "0.5.3"
version = "0.5.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "36134c44663532e6519d7a6dfdbbe06f6f8192bde8ae9ed076e9b213f0e31df7"
checksum = "51f936044d677be1a1168fae1d03b583a285a5dd9d8cbf7b24c23aa1fc775235"
[[package]]
name = "zstd"

View File

@@ -1,6 +1,6 @@
[package]
name = "windmill"
version = "1.590.0"
version = "1.596.0"
authors.workspace = true
edition.workspace = true
@@ -33,7 +33,7 @@ members = [
exclude = ["./windmill-duckdb-ffi-internal"]
[workspace.package]
version = "1.590.0"
version = "1.596.0"
authors = ["Ruben Fiszel <ruben@windmill.dev>"]
edition = "2021"
@@ -125,6 +125,7 @@ windmill-autoscaling = { workspace = true, optional = true }
futures.workspace = true
tracing.workspace = true
sqlx.workspace = true
sql-builder.workspace = true
rand.workspace = true
chrono.workspace = true
git-version.workspace = true
@@ -158,7 +159,6 @@ kube.workspace = true
k8s-openapi.workspace = true
libloading.workspace = true
bitflags.workspace = true
nom.workspace = true
globset.workspace = true
@@ -256,11 +256,11 @@ mail-send = { version = "0.4.0", features = ["builder"], default-features=false
urlencoding = "^2"
url = { version = "^2" , features = ["serde"]}
async-oauth2 = "0.5.1"
reqwest = { version = "^0.12", features = ["json", "stream", "gzip", "multipart"] }
reqwest = { version = "=0.12.24", features = ["json", "stream", "gzip", "multipart"] }
eventsource-stream = "0.2.3"
time = "^0"
serde_urlencoded = "^0"
tokio-tar = "^0"
astral-tokio-tar = "^0.5.6"
tempfile = "^3"
tokio-util = { version = "^0", features = ["io"] }
json-pointer = "^0"
@@ -378,7 +378,6 @@ pg_escape = "0.1.1"
async-nats = "0.38.0"
nkeys = "0.4.4"
nu-parser = { version = "0.101.0", default-features = false }
nom = "8.0.0"
globset = "0.4.16"
process-wrap = { version = "8.2.1", features = ["tokio1"] }

View File

@@ -1 +1 @@
0d72102809e766d5285a162dcc30055847700ee2
576abf6519d1aa12a2b989a58a123501206284fb

View File

@@ -43,7 +43,7 @@ def load_openapi_spec(file_path: str) -> Dict[str, Any]:
print(f"Error loading OpenAPI spec: {e}", file=sys.stderr)
sys.exit(1)
def extract_separate_schemas(parameters: List[Dict[str, Any]], request_body: Optional[Dict[str, Any]], spec: Dict[str, Any], required_fields: Optional[List[str]] = None) -> tuple:
def extract_separate_schemas(parameters: List[Dict[str, Any]], request_body: Optional[Dict[str, Any]], spec: Dict[str, Any], required_fields: Optional[List[str]] = None, base_path: str = "") -> tuple:
"""Extract separate schemas for path parameters, query parameters, and request body."""
path_params_schema = {
"type": "object",
@@ -63,16 +63,16 @@ def extract_separate_schemas(parameters: List[Dict[str, Any]], request_body: Opt
for param in parameters:
# Resolve $ref if present
if '$ref' in param:
param = resolve_schema_refs(param, spec)
param = resolve_schema_refs(param, spec, base_path)
param_name = param.get('name', '')
param_schema = param.get('schema', {'type': 'string'})
param_required = param.get('required', False)
param_description = param.get('description', '')
param_in = param.get('in', 'query')
# Resolve any refs in the parameter schema
param_schema = resolve_schema_refs(param_schema, spec)
param_schema = resolve_schema_refs(param_schema, spec, base_path)
# Add description if available
if param_description:
@@ -93,7 +93,7 @@ def extract_separate_schemas(parameters: List[Dict[str, Any]], request_body: Opt
# Process request body if present
if request_body:
body_schema = extract_request_body_schema(request_body, spec)
body_schema = extract_request_body_schema(request_body, spec, base_path)
# If we have required fields specified and a body schema, update the required array
if body_schema and required_fields:
@@ -115,68 +115,109 @@ def extract_separate_schemas(parameters: List[Dict[str, Any]], request_body: Opt
return (path_params_schema, query_params_schema, body_schema)
def resolve_ref(ref_path: str, spec: Dict[str, Any]) -> Optional[Dict[str, Any]]:
"""Resolve a $ref path to the actual schema definition."""
if not ref_path.startswith('#/'):
# Cache for loaded external files
_external_file_cache: Dict[str, Dict[str, Any]] = {}
def load_external_file(file_path: str, base_path: str) -> Optional[Dict[str, Any]]:
"""Load an external YAML file relative to the base path."""
if file_path in _external_file_cache:
return _external_file_cache[file_path]
try:
import yaml
from pathlib import Path
# Resolve the path relative to the base file
base_dir = Path(base_path).parent
full_path = (base_dir / file_path).resolve()
with open(full_path, 'r', encoding='utf-8') as f:
content = yaml.safe_load(f)
_external_file_cache[file_path] = content
return content
except Exception as e:
print(f"Warning: Could not load external file {file_path}: {e}", file=sys.stderr)
return None
def resolve_ref(ref_path: str, spec: Dict[str, Any], base_path: str = "") -> tuple:
"""Resolve a $ref path to the actual schema definition.
Handles both internal refs (#/...) and external file refs (file.yaml#/...).
Returns a tuple of (resolved_schema, resolved_spec) where resolved_spec is the spec
that should be used for resolving any nested refs within the resolved schema.
"""
# Check if this is an external file reference
if '#' in ref_path and not ref_path.startswith('#'):
# External file reference: "../../openflow.openapi.yaml#/components/schemas/Retry"
file_part, fragment = ref_path.split('#', 1)
external_spec = load_external_file(file_part, base_path)
if external_spec is None:
return None, spec
# Resolve the fragment within the external file, and return external_spec for nested refs
resolved, _ = resolve_ref('#' + fragment, external_spec, base_path)
return resolved, external_spec
if not ref_path.startswith('#/'):
return None, spec
# Remove the '#/' prefix and split by '/'
path_parts = ref_path[2:].split('/')
# Navigate through the spec following the path
current = spec
for part in path_parts:
if isinstance(current, dict) and part in current:
current = current[part]
else:
return None
return current if isinstance(current, dict) else None
return None, spec
def resolve_schema_refs(schema: Dict[str, Any], spec: Dict[str, Any]) -> Dict[str, Any]:
return (current if isinstance(current, dict) else None), spec
def resolve_schema_refs(schema: Dict[str, Any], spec: Dict[str, Any], base_path: str = "") -> Dict[str, Any]:
"""Recursively resolve all $ref references in a schema."""
if not isinstance(schema, dict):
return schema
# If this is a $ref, resolve it
if '$ref' in schema:
ref_path = schema['$ref']
resolved = resolve_ref(ref_path, spec)
resolved, resolved_spec = resolve_ref(ref_path, spec, base_path)
if resolved:
# Recursively resolve any refs in the resolved schema
return resolve_schema_refs(resolved, spec)
# Recursively resolve any refs in the resolved schema using the appropriate spec
return resolve_schema_refs(resolved, resolved_spec, base_path)
else:
print(f"Warning: Could not resolve $ref: {ref_path}")
return schema
# Recursively process all values in the schema
resolved_schema = {}
for key, value in schema.items():
if isinstance(value, dict):
resolved_schema[key] = resolve_schema_refs(value, spec)
resolved_schema[key] = resolve_schema_refs(value, spec, base_path)
elif isinstance(value, list):
resolved_schema[key] = [
resolve_schema_refs(item, spec) if isinstance(item, dict) else item
resolve_schema_refs(item, spec, base_path) if isinstance(item, dict) else item
for item in value
]
else:
resolved_schema[key] = value
return resolved_schema
def extract_request_body_schema(request_body: Dict[str, Any], spec: Dict[str, Any]) -> Optional[Dict[str, Any]]:
def extract_request_body_schema(request_body: Dict[str, Any], spec: Dict[str, Any], base_path: str = "") -> Optional[Dict[str, Any]]:
"""Extract request body schema from OpenAPI requestBody definition and resolve refs."""
if not request_body:
return None
content = request_body.get('content', {})
json_content = content.get('application/json', {})
schema = json_content.get('schema', {})
if schema:
# Resolve any $ref references in the schema
return resolve_schema_refs(schema, spec)
return resolve_schema_refs(schema, spec, base_path)
return None
def http_method_to_rust(method: str) -> str:
@@ -221,7 +262,7 @@ def find_mcp_tools(spec: Dict[str, Any]) -> List[Dict[str, Any]]:
return tools
def generate_typescript_code(tools: List[Dict[str, Any]], spec: Dict[str, Any]) -> str:
def generate_typescript_code(tools: List[Dict[str, Any]], spec: Dict[str, Any], base_path: str = "") -> str:
"""Generate TypeScript code with MCP endpoint tools."""
if not tools:
return """// Auto-generated MCP tools from OpenAPI specification
@@ -252,7 +293,7 @@ export const mcpEndpointTools: EndpointTool[] = [];
# Generate separate schemas
path_params_schema, query_params_schema, body_schema = extract_separate_schemas(
tool['parameters'], tool['requestBody'], spec, tool['required_fields']
tool['parameters'], tool['requestBody'], spec, tool['required_fields'], base_path
)
# Convert schemas to TypeScript - use 'as const' for better type inference
@@ -297,7 +338,7 @@ export const mcpEndpointTools: EndpointTool[] = [
return typescript_code
def generate_rust_code(tools: List[Dict[str, Any]], spec: Dict[str, Any]) -> str:
def generate_rust_code(tools: List[Dict[str, Any]], spec: Dict[str, Any], base_path: str = "") -> str:
"""Generate the complete Rust code with MCP tools."""
if not tools:
return """// No MCP tools found in the OpenAPI specification
@@ -320,7 +361,7 @@ pub fn all_tools() -> Vec<EndpointTool> {
# Generate separate schemas
path_params_schema, query_params_schema, body_schema = extract_separate_schemas(
tool['parameters'], tool['requestBody'], spec, tool['required_fields']
tool['parameters'], tool['requestBody'], spec, tool['required_fields'], base_path
)
path_params_rust = schema_to_rust_value(path_params_schema)
@@ -386,7 +427,7 @@ def main():
# Generate and write Rust code
print(f"Generating Rust code...")
rust_code = generate_rust_code(tools, spec)
rust_code = generate_rust_code(tools, spec, str(openapi_file))
print(f"Writing Rust code to: {rust_output_file}")
rust_output_file.parent.mkdir(parents=True, exist_ok=True)
@@ -395,7 +436,7 @@ def main():
# Generate and write TypeScript code
print(f"Generating TypeScript code...")
typescript_code = generate_typescript_code(tools, spec)
typescript_code = generate_typescript_code(tools, spec, str(openapi_file))
print(f"Writing TypeScript code to: {ts_output_file}")
ts_output_file.parent.mkdir(parents=True, exist_ok=True)

View File

@@ -0,0 +1,6 @@
-- Add down migration script here
DROP INDEX IF EXISTS idx_group_perm_history_workspace_group;
DROP TABLE IF EXISTS group_permission_history;
DROP INDEX IF EXISTS idx_folder_perm_history_workspace_folder;
DROP TABLE IF EXISTS folder_permission_history;

View File

@@ -0,0 +1,90 @@
-- Add up migration script here
-- Folder permission changes history
CREATE TABLE IF NOT EXISTS folder_permission_history (
id BIGSERIAL PRIMARY KEY,
workspace_id VARCHAR(50) NOT NULL,
folder_name VARCHAR(255) NOT NULL,
changed_by VARCHAR(50) NOT NULL,
changed_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(),
change_type VARCHAR(50) NOT NULL,
affected VARCHAR(100),
FOREIGN KEY (workspace_id, folder_name) REFERENCES folder(workspace_id, name) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_folder_perm_history_workspace_folder
ON folder_permission_history(workspace_id, folder_name, id DESC);
-- Group permission changes history
CREATE TABLE IF NOT EXISTS group_permission_history (
id BIGSERIAL PRIMARY KEY,
workspace_id VARCHAR(50) NOT NULL,
group_name VARCHAR(255) NOT NULL,
changed_by VARCHAR(50) NOT NULL,
changed_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(),
change_type VARCHAR(50) NOT NULL,
member_affected VARCHAR(100),
FOREIGN KEY (workspace_id, group_name) REFERENCES group_(workspace_id, name) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_group_perm_history_workspace_group
ON group_permission_history(workspace_id, group_name, id DESC);
GRANT ALL ON TABLE folder_permission_history TO windmill_user;
GRANT ALL ON TABLE group_permission_history TO windmill_user;
GRANT ALL ON TABLE folder_permission_history TO windmill_admin;
GRANT ALL ON TABLE group_permission_history TO windmill_admin;
-- Enable RLS on folder_permission_history
ALTER TABLE folder_permission_history ENABLE ROW LEVEL SECURITY;
-- Admin policies (windmill_admin can always do everything)
CREATE POLICY admin_all ON folder_permission_history FOR ALL TO windmill_admin USING (true) WITH CHECK (true);
CREATE POLICY admin_all ON group_permission_history FOR ALL TO windmill_admin USING (true) WITH CHECK (true);
-- Enable RLS on group_permission_history
ALTER TABLE group_permission_history ENABLE ROW LEVEL SECURITY;
-- RLS policies for folder_permission_history
-- Anyone can insert
CREATE POLICY allow_insert ON folder_permission_history FOR INSERT TO windmill_user WITH CHECK (true);
-- Select requires being in extra_perms (as user or via group)
CREATE POLICY see_extra_perms_user ON folder_permission_history FOR SELECT TO windmill_user
USING (EXISTS (
SELECT 1 FROM folder f
WHERE f.workspace_id = folder_permission_history.workspace_id
AND f.name = folder_permission_history.folder_name
AND f.extra_perms ? CONCAT('u/', current_setting('session.user'))
));
CREATE POLICY see_extra_perms_groups ON folder_permission_history FOR SELECT TO windmill_user
USING (EXISTS (
SELECT 1 FROM folder f
WHERE f.workspace_id = folder_permission_history.workspace_id
AND f.name = folder_permission_history.folder_name
AND f.extra_perms ?| regexp_split_to_array(current_setting('session.pgroups'), ',')::text[]
));
-- RLS policies for group_permission_history
-- Anyone can insert
CREATE POLICY allow_insert ON group_permission_history FOR INSERT TO windmill_user WITH CHECK (true);
-- Select requires being in extra_perms (as user or via group)
CREATE POLICY see_extra_perms_user ON group_permission_history FOR SELECT TO windmill_user
USING (EXISTS (
SELECT 1 FROM group_ g
WHERE g.workspace_id = group_permission_history.workspace_id
AND g.name = group_permission_history.group_name
AND (g.extra_perms ->> CONCAT('u/', current_setting('session.user')))::boolean
));
CREATE POLICY see_extra_perms_groups ON group_permission_history FOR SELECT TO windmill_user
USING (EXISTS (
SELECT 1 FROM group_ g, jsonb_each_text(g.extra_perms) f
WHERE g.workspace_id = group_permission_history.workspace_id
AND g.name = group_permission_history.group_name
AND SPLIT_PART(f.key, '/', 1) = 'g'
AND f.key = ANY(regexp_split_to_array(current_setting('session.pgroups'), ',')::text[])
AND f.value::boolean
));

View File

@@ -0,0 +1,3 @@
-- Add down migration script here
DROP TABLE IF EXISTS skip_workspace_diff_tally;
DROP TABLE IF EXISTS workspace_diff;

View File

@@ -0,0 +1,24 @@
-- Add up migration script here
CREATE TABLE workspace_diff (
source_workspace_id VARCHAR(50) NOT NULL,
fork_workspace_id VARCHAR(50) NOT NULL,
path VARCHAR(255) NOT NULL,
kind VARCHAR(50) NOT NULL,
ahead INTEGER NOT NULL DEFAULT 0,
behind INTEGER NOT NULL DEFAULT 0,
has_changes BOOLEAN DEFAULT NULL,
exists_in_source BOOLEAN DEFAULT NULL,
exists_in_fork BOOLEAN DEFAULT NULL,
PRIMARY KEY (source_workspace_id, fork_workspace_id, path, kind)
);
-- Create table to track workspaces that should be excluded from diff tallying
-- Old workspaces that are linked but have already diverged need to be skipped
CREATE TABLE skip_workspace_diff_tally (
workspace_id VARCHAR(50) PRIMARY KEY,
added_at TIMESTAMP NOT NULL DEFAULT NOW()
);
-- Populate with all existing workspaces to exclude them from new tallying logic
INSERT INTO skip_workspace_diff_tally (workspace_id)
SELECT id FROM workspace;

View File

@@ -1 +1,2 @@
-- Add down migration script here
ALTER TABLE email_trigger DROP COLUMN enabled;

View File

@@ -0,0 +1,11 @@
ALTER TABLE v2_job_queue
DROP COLUMN runnable_settings_handle;
ALTER TABLE script
DROP COLUMN runnable_settings_handle;
DROP TABLE IF EXISTS job_settings;
DROP TABLE IF EXISTS runnable_settings;
DROP TABLE IF EXISTS concurrency_settings;
DROP TABLE IF EXISTS debouncing_settings;

View File

@@ -0,0 +1,42 @@
CREATE TABLE IF NOT EXISTS concurrency_settings(
hash BIGINT PRIMARY KEY,
concurrency_key VARCHAR(255),
concurrent_limit INTEGER,
concurrency_time_window_s INTEGER
);
CREATE TABLE IF NOT EXISTS debouncing_settings(
hash BIGINT PRIMARY KEY,
debounce_key VARCHAR(255),
debounce_delay_s INTEGER,
max_total_debouncing_time INTEGER,
max_total_debounces_amount INTEGER,
debounce_args_to_accumulate TEXT[]
);
CREATE TABLE IF NOT EXISTS runnable_settings(
hash BIGINT PRIMARY KEY,
debouncing_settings BIGINT DEFAULT NULL,
concurrency_settings BIGINT DEFAULT NULL
);
CREATE TABLE IF NOT EXISTS job_settings(
job_id UUID PRIMARY KEY,
runnable_settings BIGINT DEFAULT NULL
);
ALTER TABLE script
ADD COLUMN runnable_settings_handle BIGINT DEFAULT NULL;
ALTER TABLE v2_job_queue
ADD COLUMN runnable_settings_handle BIGINT DEFAULT NULL;
GRANT ALL ON concurrency_settings TO windmill_admin;
GRANT ALL ON concurrency_settings TO windmill_user;
GRANT ALL ON debouncing_settings TO windmill_admin;
GRANT ALL ON debouncing_settings TO windmill_user;
GRANT ALL ON runnable_settings TO windmill_admin;
GRANT ALL ON runnable_settings TO windmill_user;
GRANT ALL ON job_settings TO windmill_admin;
GRANT ALL ON job_settings TO windmill_user;

View File

@@ -0,0 +1,38 @@
-- Grant CREATE privilege on all databases where custom_instance_user has CONNECT
-- This allows custom_instance_user to create schemas in databases it can already access
DO $$
DECLARE
db_record RECORD;
grant_command TEXT;
BEGIN
-- Find all databases where custom_instance_user has CONNECT privilege
-- We check if the datacl array contains an entry for custom_instance_user with 'c' (CONNECT) privilege
FOR db_record IN
SELECT d.datname
FROM pg_database d
WHERE d.datname NOT IN ('template0', 'template1') -- Skip template databases
AND d.datallowconn = true -- Only consider databases that allow connections
AND d.datacl IS NOT NULL -- Has ACL entries
AND EXISTS (
SELECT 1
FROM unnest(d.datacl) AS acl_entry
WHERE acl_entry::text LIKE 'custom_instance_user=c/%' -- 'c' is the privilege code for CONNECT
)
LOOP
BEGIN
-- Grant CREATE privilege on the database
EXECUTE format('GRANT CREATE ON DATABASE %I TO custom_instance_user', db_record.datname);
RAISE NOTICE 'Granted CREATE on database % to custom_instance_user', db_record.datname;
EXCEPTION
WHEN others THEN
RAISE NOTICE 'Failed to grant CREATE on database %: %', db_record.datname, SQLERRM;
END;
END LOOP;
RAISE NOTICE 'Completed granting CREATE privileges to custom_instance_user on all accessible databases';
EXCEPTION
WHEN others THEN
RAISE NOTICE 'Error in custom_instance_user CREATE privilege migration: %', SQLERRM;
-- Continue without failing the migration
END
$$;

View File

@@ -0,0 +1 @@
-- Add down migration script here

View File

@@ -0,0 +1,6 @@
-- Add up migration script here
GRANT ALL ON SEQUENCE folder_permission_history_id_seq TO windmill_user;
GRANT ALL ON SEQUENCE folder_permission_history_id_seq TO windmill_admin;
GRANT ALL ON SEQUENCE group_permission_history_id_seq TO windmill_user;
GRANT ALL ON SEQUENCE group_permission_history_id_seq TO windmill_admin;

View File

@@ -0,0 +1,2 @@
-- Drop flow_iterator_data table
DROP TABLE IF EXISTS flow_iterator_data;

View File

@@ -0,0 +1,9 @@
-- Create separate table for storing flow iterator data (itered arrays)
-- This avoids expensive JSONB_SET operations on large itered arrays during parallel loop execution
CREATE TABLE IF NOT EXISTS flow_iterator_data (
job_id UUID PRIMARY KEY REFERENCES v2_job_queue (id) ON DELETE CASCADE NOT NULL,
itered JSONB NOT NULL,
created_at TIMESTAMP WITH TIME ZONE DEFAULT now()
);
-- Index not needed beyond primary key since all lookups are by job_id

View File

@@ -0,0 +1 @@
ALTER TABLE workspace_settings DROP COLUMN teams_team_guid;

View File

@@ -0,0 +1,3 @@
-- Add teams_team_guid column to store the GUID (used for MS Graph API calls)
-- The existing teams_team_id column stores the internal_id (used for webhook matching)
ALTER TABLE workspace_settings ADD COLUMN teams_team_guid TEXT;

View File

@@ -10,6 +10,7 @@ path = "./src/lib.rs"
[dependencies]
windmill-parser.workspace = true
windmill-parser-sql.workspace = true
rustpython-parser.workspace = true
itertools.workspace = true
serde_json.workspace = true

View File

@@ -2,12 +2,12 @@ use rustpython_ast::{Constant, Expr, ExprConstant, Visitor};
use rustpython_parser::{ast::Suite, Parse};
use std::collections::HashMap;
use windmill_parser::asset_parser::{
detect_sql_access_type, merge_assets, parse_asset_syntax, AssetKind, AssetUsageAccessType,
ParseAssetsResult,
asset_was_used, merge_assets, parse_asset_syntax, AssetKind, AssetUsageAccessType,
ParseAssetsOutput, ParseAssetsResult,
};
use AssetUsageAccessType::*;
pub fn parse_assets(input: &str) -> anyhow::Result<Vec<ParseAssetsResult<String>>> {
pub fn parse_assets(input: &str) -> anyhow::Result<ParseAssetsOutput> {
let ast = Suite::parse(input, "main.py")
.map_err(|e| anyhow::anyhow!("Error parsing code: {}", e.to_string()))?;
@@ -15,67 +15,51 @@ pub fn parse_assets(input: &str) -> anyhow::Result<Vec<ParseAssetsResult<String>
ast.into_iter()
.for_each(|stmt| assets_finder.visit_stmt(stmt));
for (kind, name) in assets_finder.var_identifiers.into_values() {
for (kind, path, _) in assets_finder.var_identifiers.into_values() {
// if a db = wmill.datatable() was never used (e.g db.query(...)),
// we still want to register the asset as unknown access type
if assets_finder
.assets
.iter()
.all(|a| !(a.kind == kind && a.path == name))
{
if asset_was_used(&assets_finder.assets, (kind, &path)) == false {
assets_finder
.assets
.push(ParseAssetsResult { kind, access_type: None, path: name });
.push(ParseAssetsResult { kind, access_type: None, path });
}
}
Ok(merge_assets(assets_finder.assets))
Ok(ParseAssetsOutput { assets: merge_assets(assets_finder.assets), ..Default::default() })
}
type VarAssetName = String;
type VarAssetSchema = Option<String>;
struct AssetsFinder {
assets: Vec<ParseAssetsResult<String>>,
var_identifiers: HashMap<String, (AssetKind, String)>,
assets: Vec<ParseAssetsResult>,
var_identifiers: HashMap<String, (AssetKind, VarAssetName, VarAssetSchema)>,
}
impl Visitor for AssetsFinder {
// Handle assignment statements like: x = wmill.datatable('name')
fn visit_stmt_assign(&mut self, node: rustpython_ast::StmtAssign) {
// Check if the value is a call to a tracked function
if let Some((kind, name)) = self.extract_asset_from_call(&node.value) {
// Track all target variables
for target in &node.targets {
if let Expr::Name(name_expr) = target {
let Ok(var_name) = name_expr.id.parse::<String>();
self.var_identifiers
.insert(var_name, (kind.clone(), name.clone()));
}
}
} else {
// If not wmill.datatable or similar, remove any tracked variables
// It means the identifier is no longer refering to an asset
for target in &node.targets {
if let Expr::Name(name_expr) = target {
let Ok(var_name) = name_expr.id.parse::<String>();
let removed = self.var_identifiers.remove(&var_name);
// if a db = wmill.datatable() or similar was removed, but never used (e.g db.query(...)),
// we still want to register the asset as unknown access type
match removed {
Some((kind, name)) => {
if self
.assets
.iter()
.all(|a| !(a.kind == kind && a.path == name))
{
self.assets.push(ParseAssetsResult {
kind,
access_type: None,
path: name,
});
}
}
None => {}
if let Some(Expr::Name(expr_name)) = node.targets.first() {
// Remove any tracked variables with that name in case of reassignment
let Ok(var_name) = expr_name.id.parse::<String>();
let removed = self.var_identifiers.remove(&var_name);
// if a db = wmill.datatable() or similar was removed, but never used (e.g db.query(...)),
// we still want to register the asset as unknown access type
match removed {
Some((kind, path, _)) => {
if !asset_was_used(&self.assets, (kind, &path)) {
self.assets
.push(ParseAssetsResult { kind, access_type: None, path });
}
}
None => {}
}
if let Some((kind, name, schema)) = self.extract_asset_from_call(&node.value) {
// Track target variable
let Ok(var_name) = expr_name.id.parse::<String>();
self.var_identifiers
.insert(var_name, (kind.clone(), name.clone(), schema.clone()));
}
}
// Continue with generic visit to catch any other assets in the expression
@@ -87,7 +71,7 @@ impl Visitor for AssetsFinder {
fn visit_expr_constant(&mut self, node: ExprConstant) {
match node.value {
Constant::Str(s) => {
if let Some((kind, path)) = parse_asset_syntax(&s) {
if let Some((kind, path)) = parse_asset_syntax(&s, false) {
self.assets.push(ParseAssetsResult {
kind,
path: path.to_string(),
@@ -108,7 +92,7 @@ impl Visitor for AssetsFinder {
if let Expr::Constant(ExprConstant { value: Constant::Str(s), .. }) =
&keyword.value
{
if let Some((kind, path)) = parse_asset_syntax(s) {
if let Some((kind, path)) = parse_asset_syntax(s, false) {
self.assets.push(ParseAssetsResult {
kind,
path: path.to_string(),
@@ -125,7 +109,10 @@ impl Visitor for AssetsFinder {
impl AssetsFinder {
/// Extract asset info from calls like wmill.datatable('name'), wmill.ducklake('name'), etc.
fn extract_asset_from_call(&self, expr: &Expr) -> Option<(AssetKind, String)> {
fn extract_asset_from_call(
&self,
expr: &Expr,
) -> Option<(AssetKind, VarAssetName, VarAssetSchema)> {
let call = expr.as_call_expr()?;
// Check for wmill.datatable, wmill.ducklake pattern
@@ -161,10 +148,21 @@ impl AssetsFinder {
} else {
None
}
})
.unwrap_or_else(|| "main".to_string());
});
let (name, schema) = match name {
None => ("main".to_string(), None),
Some(name) => {
if let Some((name, s)) = name.split_once(':') {
let schema = Some(s.to_string());
let name = if name.is_empty() { "main" } else { name };
(name.to_string(), schema)
} else {
(name, None)
}
}
};
Some((kind, name))
Some((kind, name, schema))
}
fn visit_expr_call_inner(&mut self, node: &rustpython_ast::ExprCall) -> Result<(), ()> {
@@ -184,8 +182,8 @@ impl AssetsFinder {
value, ..
}) = node.func.as_ref()
{
if let Expr::Name(name_expr) = value.as_ref() {
name_expr.id.parse().map_err(|_| ())?
if let Expr::Name(expr_name) = value.as_ref() {
expr_name.id.parse().map_err(|_| ())?
} else {
return Err(());
}
@@ -195,28 +193,45 @@ impl AssetsFinder {
if obj_name == "wmill" {
// Continue
} else if let Some((kind, ref path)) = self.var_identifiers.get(&obj_name) {
} else if let Some((kind, ref path, ref schema)) = self.var_identifiers.get(&obj_name) {
if ident == "query" {
let name_expr = node.args.get(0).or_else(|| {
let expr_name = node.args.get(0).or_else(|| {
node.keywords
.iter()
.find(|kw| kw.arg.as_deref() == Some("name"))
.map(|kw| &kw.value)
});
match name_expr {
Some(Expr::Constant(ExprConstant {
value: Constant::Str(sql_query), ..
})) => {
let access_type = detect_sql_access_type(&sql_query);
self.assets.push(ParseAssetsResult {
kind: *kind,
path: path.to_string(),
access_type,
});
return Ok(());
}
let sql = match expr_name {
Some(Expr::Constant(ExprConstant { value: Constant::Str(sql), .. })) => sql,
_ => return Err(()),
};
let duckdb_conn_prefix = match kind {
AssetKind::DataTable => "datatable",
AssetKind::Ducklake => "ducklake",
_ => return Ok(()),
};
let sql = format!("ATTACH '{duckdb_conn_prefix}://{path}' AS dt; USE dt; {sql}");
// We use the SQL parser to detect if it's a read or write query
match windmill_parser_sql::parse_assets(&sql) {
Ok(mut sql_assets) => {
if let Some(schema_name) = schema {
for asset in &mut sql_assets.assets {
if asset.kind == *kind && asset.path.starts_with(path.as_str()) {
asset.path = format!(
"{}/{}.{}",
path,
schema_name,
&asset.path[path.len() + 1..]
);
}
}
}
self.assets.extend(sql_assets.assets);
}
_ => {}
}
return Ok(());
} else {
return Err(());
}
@@ -249,7 +264,9 @@ impl AssetsFinder {
match arg_val {
Some(Expr::Constant(ExprConstant { value: Constant::Str(value), .. })) => {
let path = parse_asset_syntax(&value).map(|(_, p)| p).unwrap_or(&value);
let path = parse_asset_syntax(&value, false)
.map(|(_, p)| p)
.unwrap_or(&value);
self.assets
.push(ParseAssetsResult { kind, path: path.to_string(), access_type });
}
@@ -259,5 +276,233 @@ impl AssetsFinder {
}
}
struct Arg(usize, &'static str);
// Positional arguments in python can also be used by their name
struct Arg(usize, &'static str);
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_py_asset_parser_load_s3() {
let input = r#"
import wmill
def main():
wmill.load_s3_file('s3:///test.csv')
"#;
let s = parse_assets(input).map(|o| o.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::S3Object,
path: "/test.csv".to_string(),
access_type: Some(R)
},])
);
}
#[test]
fn test_py_asset_parser_unused_sql() {
let input = r#"
import wmill
def main():
db = wmill.datatable()
"#;
let s = parse_assets(input).map(|o| o.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "main".to_string(),
access_type: None
},])
);
}
#[test]
fn test_py_asset_parser_sql_read() {
let input = r#"
import wmill
def main(x: int):
db = wmill.datatable('dt')
return db.query('SELECT * FROM friends WHERE age = $1', x).fetch()
"#;
let s = parse_assets(input).map(|o| o.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/friends".to_string(),
access_type: Some(R)
},])
);
}
#[test]
fn test_py_asset_parser_sql_read_write() {
let input = r#"
import wmill
def main(x: int):
db = wmill.datatable('dt')
db.query('UPDATE friends SET x = $1', x).fetch()
db.query('SELECT * FROM friends WHERE age = $1', x).fetch_one()
db.query('SELECT * FROM analytics').fetch()
"#;
let s = parse_assets(input).map(|o| o.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![
ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/analytics".to_string(),
access_type: Some(R)
},
ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/friends".to_string(),
access_type: Some(RW)
},
])
);
}
#[test]
fn test_py_asset_parser_multiple_sql_scopes() {
let input = r#"
import wmill
def main():
def f(x: int):
db = wmill.datatable()
return db.query('SELECT * FROM friends WHERE age = $1', x)
db = wmill.datatable('another1')
return db.query('INSERT INTO customers VALUES ($1)', 0)
def g():
db = wmill.ducklake('another2')
"#;
let s = parse_assets(input).map(|o| o.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![
ParseAssetsResult {
kind: AssetKind::DataTable,
path: "another1/customers".to_string(),
access_type: Some(W)
},
ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "another2".to_string(),
access_type: None
},
ParseAssetsResult {
kind: AssetKind::DataTable,
path: "main/friends".to_string(),
access_type: Some(R)
},
])
);
}
#[test]
fn test_py_asset_parser_overriden_var_identifier() {
let input = r#"
import wmill
def main():
db = wmill.datatable('another1')
def g():
db = wmill.ducklake()
"#;
let s = parse_assets(input).map(|o| o.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![
ParseAssetsResult {
kind: AssetKind::DataTable,
path: "another1".to_string(),
access_type: None
},
ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "main".to_string(),
access_type: None
},
])
);
}
#[test]
fn test_py_asset_parser_datatable_with_schema() {
let input = r#"
import wmill
def main(x: int):
db = wmill.datatable('dt:public')
return db.query('SELECT * FROM friends WHERE age = $1', x).fetch()
"#;
let s = parse_assets(input).map(|o| o.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/public.friends".to_string(),
access_type: Some(R)
},])
);
}
#[test]
fn test_py_asset_parser_ducklake_with_schema() {
let input = r#"
import wmill
def main():
db = wmill.ducklake('lake1:analytics')
return db.query('SELECT * FROM metrics').fetch()
"#;
let s = parse_assets(input).map(|o| o.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "lake1/analytics.metrics".to_string(),
access_type: Some(R)
},])
);
}
#[test]
fn test_py_asset_parser_schema_with_write() {
let input = r#"
import wmill
def main(x: int):
db = wmill.datatable('dt:public')
db.query('INSERT INTO users VALUES ($1)', x).fetch()
return db.query('SELECT * FROM users').fetch()
"#;
let s = parse_assets(input).map(|o| o.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/public.users".to_string(),
access_type: Some(RW)
},])
);
}
#[test]
fn test_py_asset_parser_unused_datatable_with_schema() {
let input = r#"
import wmill
def main():
db = wmill.datatable('dt:public')
"#;
let s = parse_assets(input).map(|o| o.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt".to_string(),
access_type: None
},])
);
}
}

View File

@@ -15,7 +15,7 @@ use windmill_parser::{json_to_typ, Arg, MainArgSignature, ObjectType, Typ};
use rustpython_parser::{
ast::{
Constant, Expr, ExprConstant, ExprDict, ExprList, ExprName, Stmt, StmtFunctionDef, Suite,
Constant, Expr, ExprAttribute, ExprConstant, ExprDict, ExprList, ExprName, Stmt, StmtAssign, StmtClassDef, StmtFunctionDef, Suite,
},
Parse,
};
@@ -60,6 +60,166 @@ fn filter_non_main(code: &str, main_name: &str) -> String {
return filtered_code;
}
/// Data extracted from parsing the Python code
struct CodeMetadata {
enums: HashMap<String, EnumInfo>,
descriptions: HashMap<String, String>,
}
/// Information about an Enum class
struct EnumInfo {
values: Vec<String>,
members: HashMap<String, String>,
}
fn has_enum_keyword(code: &str) -> bool {
code.contains("Enum")
}
/// Extract only class and function definitions from code (prepass filtering)
fn filter_relevant_statements(code: &str) -> String {
let mut result = Vec::new();
let mut lines = code.lines().peekable();
while let Some(line) = lines.next() {
let trimmed = line.trim_start();
if trimmed.starts_with("class ") || trimmed.starts_with("def ") {
result.push(line);
let base_indent = line.len() - trimmed.len();
while let Some(&next_line) = lines.peek() {
let next_trimmed = next_line.trim_start();
let next_indent = next_line.len() - next_trimmed.len();
if next_trimmed.is_empty() || next_indent > base_indent {
result.push(lines.next().unwrap());
} else {
break;
}
}
}
}
result.join("\n")
}
/// Extract Enum definitions and docstring descriptions lazily.
/// Only parses AST if relevant keywords are present.
fn extract_code_metadata(code: &str, main_name: &str) -> CodeMetadata {
let mut enums = HashMap::new();
let mut descriptions = HashMap::new();
let has_enum = has_enum_keyword(code);
let has_docstring = code.contains("Args:");
if !has_enum && !has_docstring {
return CodeMetadata { enums, descriptions };
}
let filtered_code = filter_relevant_statements(code);
let ast = match Suite::parse(&filtered_code, "main.py") {
Ok(ast) => ast,
Err(_) => return CodeMetadata { enums, descriptions },
};
for stmt in ast {
match stmt {
Stmt::ClassDef(StmtClassDef { name, body, bases, .. }) if has_enum => {
let is_enum = bases.iter().any(|base| {
matches!(base, Expr::Name(ExprName { id, .. })
if id == "Enum" || id == "IntEnum" || id == "StrEnum"
|| id == "Flag" || id == "IntFlag")
});
if is_enum {
let mut values = Vec::new();
let mut members = HashMap::new();
for item in body {
if let Stmt::Assign(StmtAssign { targets, value, .. }) = item {
if let Some(Expr::Name(ExprName { id: target_name, .. })) = targets.first() {
if !target_name.starts_with('_') {
if let Expr::Constant(ExprConstant { value: Constant::Str(val), .. }) = value.as_ref() {
values.push(val.to_string());
members.insert(target_name.to_string(), val.to_string());
}
}
}
}
}
if !values.is_empty() {
enums.insert(name.to_string(), EnumInfo { values, members });
}
}
},
Stmt::FunctionDef(StmtFunctionDef { name: func_name, body, .. }) if has_docstring => {
if &func_name == main_name {
if let Some(Stmt::Expr(expr_stmt)) = body.first() {
if let Expr::Constant(ExprConstant { value: Constant::Str(docstring), .. }) = expr_stmt.value.as_ref() {
descriptions = parse_docstring_args(docstring);
}
}
}
},
_ => {}
}
}
CodeMetadata { enums, descriptions }
}
/// Parse docstring Args: section (format: "param_name (type): Description")
fn parse_docstring_args(docstring: &str) -> HashMap<String, String> {
let mut descriptions = HashMap::new();
let mut in_args_section = false;
let mut base_indent: Option<usize> = None;
for line in docstring.lines() {
let trimmed = line.trim();
if trimmed == "Args:" {
in_args_section = true;
base_indent = None;
continue;
}
if in_args_section {
if trimmed.is_empty() {
continue;
}
let indent = line.len() - line.trim_start().len();
if base_indent.is_none() && !trimmed.is_empty() {
base_indent = Some(indent);
}
if let Some(base) = base_indent {
if indent < base && trimmed.ends_with(':') {
break;
}
}
if let Some(colon_pos) = trimmed.find(':') {
let before_colon = &trimmed[..colon_pos];
let description = trimmed[colon_pos + 1..].trim();
if let Some(paren_pos) = before_colon.find('(') {
let param_name = before_colon[..paren_pos].trim();
descriptions.insert(param_name.to_string(), description.to_string());
} else {
descriptions.insert(before_colon.trim().to_string(), description.to_string());
}
}
}
}
descriptions
}
/// skip_params is a micro optimization for when we just want to find the main
/// function without parsing all the params.
pub fn parse_python_signature(
@@ -91,27 +251,61 @@ pub fn parse_python_signature(
if !skip_params && params.is_some() {
let params = params.unwrap();
//println!("{:?}", params);
let def_arg_start = params.args.len() - params.defaults().count();
// Two-pass approach for lazy metadata extraction:
// Pass 1: Parse types without enum info to determine if metadata is needed
// Pass 2: Re-parse unknown types with metadata only if necessary
// This ensures zero overhead for scripts without enums/docstrings
let empty_enums = HashMap::new();
let args_first_pass: Vec<_> = params
.args
.iter()
.enumerate()
.map(|(i, x)| {
let arg_name = x.as_arg().arg.to_string();
let (typ, has_default) = x
.as_arg()
.annotation
.as_ref()
.map_or((Typ::Unknown, false), |e| parse_expr(e, &empty_enums));
(i, arg_name, typ, has_default)
})
.collect();
// Determine if we need to extract metadata from the code
let has_potential_enums = args_first_pass
.iter()
.any(|(_, _, typ, _)| matches!(typ, Typ::Resource(_)));
let metadata = if has_potential_enums || code.contains("Args:") {
extract_code_metadata(code, &main_name)
} else {
CodeMetadata {
enums: HashMap::new(),
descriptions: HashMap::new(),
}
};
// Build final args, re-parsing Resource types as enums if metadata was extracted
Ok(MainArgSignature {
star_args: params.vararg.is_some(),
star_kwargs: params.kwarg.is_some(),
args: params
.args
.iter()
.enumerate()
.map(|(i, x)| {
let (mut typ, has_default) = x
.as_arg()
.annotation
.as_ref()
.map_or((Typ::Unknown, false), |e| parse_expr(e));
args: args_first_pass
.into_iter()
.map(|(i, arg_name, mut typ, mut has_default)| {
if matches!(typ, Typ::Resource(_)) && !metadata.enums.is_empty() {
if let Some(annotation) = params.args[i].as_arg().annotation.as_ref() {
(typ, has_default) = parse_expr(annotation, &metadata.enums);
}
}
let default = if i >= def_arg_start {
params
.defaults()
.nth(i - def_arg_start)
.map(to_value)
.map(|expr| to_value(expr, &metadata.enums))
.flatten()
} else {
None
@@ -140,8 +334,8 @@ pub fn parse_python_signature(
}
Arg {
otyp: None,
name: x.as_arg().arg.to_string(),
otyp: metadata.descriptions.get(&arg_name).map(|d| d.to_string()),
name: arg_name,
typ,
has_default: has_default || default.is_some(),
default,
@@ -163,15 +357,15 @@ pub fn parse_python_signature(
}
}
fn parse_expr(e: &Box<Expr>) -> (Typ, bool) {
fn parse_expr(e: &Box<Expr>, enums: &HashMap<String, EnumInfo>) -> (Typ, bool) {
match e.as_ref() {
Expr::Name(ExprName { id, .. }) => (parse_typ(id.as_ref()), false),
Expr::Name(ExprName { id, .. }) => (parse_typ(id.as_ref(), enums), false),
Expr::Attribute(x) => {
if x.value
.as_name_expr()
.is_some_and(|x| x.id.as_str() == "wmill")
{
(parse_typ(x.attr.as_str()), false)
(parse_typ(x.attr.as_str(), enums), false)
} else {
(Typ::Unknown, false)
}
@@ -181,7 +375,7 @@ fn parse_expr(e: &Box<Expr>) -> (Typ, bool) {
x.right.as_ref(),
Expr::Constant(ExprConstant { value: Constant::None, .. })
) {
(parse_expr(&x.left).0, true)
(parse_expr(&x.left, enums).0, true)
} else {
(Typ::Unknown, false)
}
@@ -210,8 +404,8 @@ fn parse_expr(e: &Box<Expr>) -> (Typ, bool) {
};
(Typ::Str(values), false)
}
"List" | "list" => (Typ::List(Box::new(parse_expr(&x.slice).0)), false),
"Optional" => (parse_expr(&x.slice).0, true),
"List" | "list" => (Typ::List(Box::new(parse_expr(&x.slice, enums).0)), false),
"Optional" => (parse_expr(&x.slice, enums).0, true),
_ => (Typ::Unknown, false),
},
_ => (Typ::Unknown, false),
@@ -220,7 +414,11 @@ fn parse_expr(e: &Box<Expr>) -> (Typ, bool) {
}
}
fn parse_typ(id: &str) -> Typ {
fn parse_typ(id: &str, enums: &HashMap<String, EnumInfo>) -> Typ {
if let Some(enum_info) = enums.get(id) {
return Typ::Str(Some(enum_info.values.clone()));
}
match id {
"str" => Typ::Str(None),
"float" => Typ::Float,
@@ -249,7 +447,7 @@ fn map_resource_name(x: &str) -> String {
}
}
fn to_value<R>(et: &Expr<R>) -> Option<serde_json::Value> {
fn to_value<R>(et: &Expr<R>, enums: &HashMap<String, EnumInfo>) -> Option<serde_json::Value> {
match et {
Expr::Constant(ExprConstant { value, .. }) => Some(constant_to_value(value)),
Expr::Dict(ExprDict { keys, values, .. }) => {
@@ -259,22 +457,35 @@ fn to_value<R>(et: &Expr<R>) -> Option<serde_json::Value> {
.map(|(k, v)| {
let key = k
.as_ref()
.map(to_value)
.map(|e| to_value(e, enums))
.flatten()
.and_then(|x| match x {
serde_json::Value::String(s) => Some(s),
_ => None,
})
.unwrap_or_else(|| "no_key".to_string());
(key, to_value(&v))
(key, to_value(&v, enums))
})
.collect::<HashMap<String, _>>();
Some(json!(v))
}
Expr::List(ExprList { elts, .. }) => {
let v = elts.into_iter().map(|x| to_value(&x)).collect::<Vec<_>>();
let v = elts.into_iter().map(|x| to_value(&x, enums)).collect::<Vec<_>>();
Some(json!(v))
}
Expr::Attribute(ExprAttribute { value, attr, .. }) => {
// Handle Enum.MEMBER: returns enum value ("red") not member name ("RED")
if let Expr::Name(ExprName { id: enum_name, .. }) = value.as_ref() {
if let Some(enum_info) = enums.get(enum_name.as_str()) {
if let Some(enum_value) = enum_info.members.get(attr.as_str()) {
return Some(json!(enum_value));
}
}
Some(json!(attr.as_str()))
} else {
None
}
}
Expr::Call { .. } => Some(json!(FUNCTION_CALL)),
_ => None,
}
@@ -751,4 +962,35 @@ def main(a: str, b: Optional[str], c: str | None): return
Ok(())
}
#[test]
fn test_parse_python_sig_enum() -> anyhow::Result<()> {
let code = r#"
from enum import Enum
class Color(str, Enum):
RED = 'red'
GREEN = 'green'
BLUE = 'blue'
def main(color: Color = Color.RED):
"""
Test enum parsing
Args:
color (Color): Color selection from Color enum
"""
return {"color": color}
"#;
let result = parse_python_signature(code, None, false)?;
assert_eq!(result.args.len(), 1);
assert_eq!(result.args[0].name, "color");
assert_eq!(
result.args[0].typ,
Typ::Str(Some(vec!["red".to_string(), "green".to_string(), "blue".to_string()]))
);
assert_eq!(result.args[0].default, Some(json!("red")));
assert_eq!(result.args[0].otyp, Some("Color selection from Color enum".to_string()));
Ok(())
}
}

View File

@@ -20,4 +20,4 @@ anyhow.workspace = true
lazy_static.workspace = true
serde_json.workspace = true
serde.workspace = true
nom.workspace = true
sqlparser = { version = "0.59.0", features = ["visitor"] }

View File

@@ -1,151 +1,676 @@
use std::collections::HashMap;
use sqlparser::{
ast::{
CopyTarget, Expr, ObjectName, TableFactor, TableObject, Value, ValueWithSpan, Visit,
Visitor,
},
dialect::DuckDbDialect,
parser::Parser,
};
use windmill_parser::asset_parser::{
merge_assets, AssetKind, AssetUsageAccessType, ParseAssetsResult,
asset_was_used, merge_assets, parse_asset_syntax, AssetKind, AssetUsageAccessType,
ParseAssetsOutput, ParseAssetsResult,
};
use AssetUsageAccessType::*;
use nom::{
branch::alt,
bytes::complete::{tag, tag_no_case, take_while},
character::complete::{char, multispace0},
combinator::opt,
sequence::preceded,
IResult, Parser,
};
pub fn parse_assets(input: &str) -> anyhow::Result<ParseAssetsOutput> {
let statements = Parser::parse_sql(&DuckDbDialect, input)?;
pub fn parse_assets<'a>(input: &str) -> anyhow::Result<Vec<ParseAssetsResult<&str>>> {
let mut assets = Vec::new();
let mut remaining = input;
let mut collector = AssetCollector::new();
for statement in statements {
let _ = statement.visit(&mut collector);
}
while !remaining.trim().is_empty() {
if let Ok((rest, _)) = parse_comment(remaining) {
remaining = rest; // skip comment
}
if let Ok((rest, res)) = parse_asset(remaining) {
assets.push(res);
remaining = rest;
} else {
remaining = &remaining[1..]; // skip 1 char and continue
for (_, (kind, path)) in collector.var_identifiers {
if !asset_was_used(&collector.assets, (kind, &path)) {
collector
.assets
.push(ParseAssetsResult { kind, access_type: None, path: path });
}
}
Ok(merge_assets(assets))
Ok(ParseAssetsOutput { assets: merge_assets(collector.assets), ..Default::default() })
}
fn parse_asset(input: &str) -> IResult<&str, ParseAssetsResult<&str>> {
alt((
parse_s3_object_read.map(|path| ParseAssetsResult {
path,
kind: AssetKind::S3Object,
access_type: Some(R),
}),
parse_s3_object_write.map(|path| ParseAssetsResult {
path,
kind: AssetKind::S3Object,
access_type: Some(W),
}),
// Parse ambiguous access_types at the end if we could not find precisely read or copy
parse_s3_object_lit.map(|path| ParseAssetsResult {
path,
kind: AssetKind::S3Object,
access_type: None,
}),
parse_resource_lit.map(|path| ParseAssetsResult {
path,
kind: AssetKind::Resource,
access_type: None,
}),
parse_ducklake_lit.map(|path| ParseAssetsResult {
path,
kind: AssetKind::Ducklake,
access_type: None,
}),
parse_datatable_lit.map(|path| ParseAssetsResult {
path,
kind: AssetKind::DataTable,
access_type: None,
}),
))
.parse(input)
/// Visitor that collects S3 asset literals from SQL statements
struct AssetCollector {
assets: Vec<ParseAssetsResult>,
// e.g set to Read when we are inside a SELECT ... FROM ... statement
current_access_type_stack: Vec<AssetUsageAccessType>,
// e.g ATTACH 'ducklake://a' AS dl; => { "dl": (Ducklake, "a") }
var_identifiers: HashMap<String, (AssetKind, String)>,
// e.g USE dl;
currently_used_asset: Option<(AssetKind, String)>,
}
/// Any expression that reads an s3 asset
fn parse_s3_object_read(input: &str) -> IResult<&str, &str> {
alt((parse_s3_object_read_fn, parse_s3_object_select_from)).parse(input)
impl AssetCollector {
fn new() -> Self {
Self {
assets: Vec::new(),
current_access_type_stack: Vec::with_capacity(8),
var_identifiers: HashMap::new(),
currently_used_asset: None,
}
}
// Detect when we do 'a.b' and 'a' is associated with an asset in var_identifiers
// Or when we access 'b' and we did USE a;
fn get_associated_asset_from_obj_name(&self, name: &ObjectName) -> Option<ParseAssetsResult> {
let access_type = self.current_access_type_stack.last().copied();
if let Some((kind, path)) = &self.currently_used_asset {
// We don't want to infer that any simple identifier refers to an asset if
// we are not in a known R/W context
if access_type.is_none() {
return None;
}
if name.0.len() == 1 || name.0.len() == 2 {
if name
.0
.iter()
.any(|id| id.as_ident().and_then(|id| id.quote_style).is_some())
{
return None;
}
let specific_table = &name
.0
.iter()
.map(|id| id.as_ident().map(|id| id.value.clone()))
.collect::<Option<Vec<String>>>()?
.join(".");
let path = format!("{}/{}", path, specific_table);
return Some(ParseAssetsResult { kind: *kind, access_type, path });
}
}
// Check if the first part of the name (the a in a.b) is associated with an asset
if name.0.len() < 2 {
return None;
}
let ident = name.0.first()?.as_ident()?;
let (kind, path) = self.var_identifiers.get(&ident.value)?;
let path = if name.0.len() == 2 || name.0.len() == 3 {
let specific_table = &name.0[1..]
.iter()
.map(|id| id.as_ident().map(|id| id.value.clone()))
.collect::<Option<Vec<String>>>()?
.join(".");
format!("{}/{}", path, specific_table)
} else {
path.clone()
};
Some(ParseAssetsResult { kind: *kind, access_type, path })
}
fn handle_string_literal(&mut self, s: &str) {
// Check if the string matches our asset syntax patterns
if let Some((kind, path)) = parse_asset_syntax(s, false) {
if kind == AssetKind::S3Object {
self.assets.push(ParseAssetsResult {
kind,
path: path.to_string(),
access_type: self.current_access_type_stack.last().copied(),
});
}
}
}
fn handle_obj_name_pre(&mut self, name: &ObjectName) {
if let Some(fname) = get_trivial_obj_name(name) {
if is_read_fn(fname) {
self.current_access_type_stack.push(R);
}
}
if let Some(str_lit) = get_str_lit_from_obj_name(name) {
self.handle_string_literal(str_lit);
}
// Writes to tables should be handled directly when visiting the statement
if self.current_access_type_stack.last() == Some(&R) {
if let Some(asset) = self.get_associated_asset_from_obj_name(name) {
self.assets.push(asset);
}
}
}
fn handle_obj_name_post(&mut self, name: &ObjectName) {
if self.current_access_type_stack.is_empty() {
return;
}
if let Some(fname) = get_trivial_obj_name(name) {
if is_read_fn(fname) {
self.current_access_type_stack.pop();
}
}
}
fn handle_table_with_joins(&mut self, table_with_joins: &sqlparser::ast::TableWithJoins) {
if let TableFactor::Table { name, .. } = &table_with_joins.relation {
if let Some(asset) = self.get_associated_asset_from_obj_name(name) {
self.assets.push(asset);
}
}
for join in &table_with_joins.joins {
if let TableFactor::Table { name, .. } = &join.relation {
if let Some(asset) = self.get_associated_asset_from_obj_name(name) {
self.assets.push(asset);
}
}
}
}
}
/// Any expression that writes to an s3 asset
fn parse_s3_object_write(input: &str) -> IResult<&str, &str> {
// COPY (...) TO 's3://...'
let (input, _) = (tag_no_case("TO"), multispace0).parse(input)?;
let (input, path) = parse_s3_object_lit(input)?;
Ok((input, path))
impl Visitor for AssetCollector {
type Break = ();
fn pre_visit_table_factor(
&mut self,
table_factor: &TableFactor,
) -> std::ops::ControlFlow<Self::Break> {
match table_factor {
TableFactor::Table { name, args, .. } => {
if args.is_none() {
// Avoid Table Functions
self.handle_obj_name_pre(name);
}
}
_ => {}
}
std::ops::ControlFlow::Continue(())
}
fn post_visit_table_factor(
&mut self,
table_factor: &TableFactor,
) -> std::ops::ControlFlow<Self::Break> {
match table_factor {
TableFactor::Table { name, .. } => self.handle_obj_name_post(name),
_ => {}
}
std::ops::ControlFlow::Continue(())
}
fn pre_visit_expr(&mut self, expr: &Expr) -> std::ops::ControlFlow<Self::Break> {
match expr {
Expr::Value(ValueWithSpan { value: Value::SingleQuotedString(s), .. }) => {
self.handle_string_literal(s)
}
Expr::Value(ValueWithSpan { value: Value::DoubleQuotedString(s), .. }) => {
self.handle_string_literal(s);
}
_ => {}
}
std::ops::ControlFlow::Continue(())
}
fn post_visit_expr(&mut self, expr: &Expr) -> std::ops::ControlFlow<Self::Break> {
match expr {
Expr::Function(func) => self.handle_obj_name_post(&func.name),
_ => {}
}
std::ops::ControlFlow::Continue(())
}
fn pre_visit_statement(
&mut self,
statement: &sqlparser::ast::Statement,
) -> std::ops::ControlFlow<Self::Break> {
match statement {
sqlparser::ast::Statement::Query(_) => {
// don't forget pop() in post_visit_statement
self.current_access_type_stack.push(R);
}
sqlparser::ast::Statement::Insert(insert) => {
let access_type = if insert.returning.is_some() { RW } else { W };
self.current_access_type_stack.push(access_type);
match insert.table {
TableObject::TableName(ref name) => {
if let Some(asset) = self.get_associated_asset_from_obj_name(name) {
self.assets.push(asset);
}
}
_ => {}
}
self.current_access_type_stack.pop();
}
sqlparser::ast::Statement::Update { returning, table, from, .. } => {
if let Some(from_tables) = from {
let from_tables = match from_tables {
sqlparser::ast::UpdateTableFromKind::AfterSet(tables) => tables,
sqlparser::ast::UpdateTableFromKind::BeforeSet(tables) => tables,
};
self.current_access_type_stack.push(R);
for table_with_joins in from_tables {
self.handle_table_with_joins(table_with_joins);
}
self.current_access_type_stack.pop();
}
let access_type = if returning.is_some() { RW } else { W };
self.current_access_type_stack.push(access_type);
self.handle_table_with_joins(table);
self.current_access_type_stack.pop();
}
sqlparser::ast::Statement::Delete(delete) => {
let access_type = if delete.returning.is_some() { RW } else { W };
self.current_access_type_stack.push(access_type);
for name in &delete.tables {
if let Some(asset) = self.get_associated_asset_from_obj_name(name) {
self.assets.push(asset);
}
}
let tables = match &delete.from {
sqlparser::ast::FromTable::WithFromKeyword(tables) => tables,
sqlparser::ast::FromTable::WithoutKeyword(tables) => tables,
};
for table_with_joins in tables {
self.handle_table_with_joins(table_with_joins);
}
self.current_access_type_stack.pop();
}
sqlparser::ast::Statement::CreateTable(create_table) => {
self.current_access_type_stack.push(W);
if let Some(asset) = self.get_associated_asset_from_obj_name(&create_table.name) {
self.assets.push(asset);
}
self.current_access_type_stack.pop();
}
sqlparser::ast::Statement::CreateView { name, .. } => {
self.current_access_type_stack.push(W);
if let Some(asset) = self.get_associated_asset_from_obj_name(name) {
self.assets.push(asset);
}
self.current_access_type_stack.pop();
}
sqlparser::ast::Statement::Copy { target: CopyTarget::File { filename }, .. } => {
self.current_access_type_stack.push(W);
self.handle_string_literal(filename);
self.current_access_type_stack.pop();
}
sqlparser::ast::Statement::AttachDuckDBDatabase {
database_path,
database_alias,
..
} => {
if let Some((kind, path)) = parse_asset_syntax(&database_path.value, true) {
if kind == AssetKind::Ducklake
|| kind == AssetKind::DataTable
|| kind == AssetKind::Resource
{
if let Some(database_alias) = database_alias {
self.var_identifiers
.insert(database_alias.value.clone(), (kind, path.to_string()));
}
}
}
}
sqlparser::ast::Statement::DetachDuckDBDatabase { database_alias, .. } => {
let asset = self.var_identifiers.remove(&database_alias.value);
if self.currently_used_asset == asset {
self.currently_used_asset = None;
}
}
sqlparser::ast::Statement::Use(sqlparser::ast::Use::Object(obj_name)) => {
if let Some((kind, path)) = self.var_identifiers.get(&obj_name.to_string()) {
self.currently_used_asset = Some((*kind, path.clone()));
} else {
self.currently_used_asset = None;
}
}
_ => {}
}
std::ops::ControlFlow::Continue(())
}
fn post_visit_statement(
&mut self,
statement: &sqlparser::ast::Statement,
) -> std::ops::ControlFlow<Self::Break> {
match statement {
sqlparser::ast::Statement::Query(_) => {
self.current_access_type_stack.pop();
}
_ => {}
}
std::ops::ControlFlow::Continue(())
}
fn pre_visit_query(
&mut self,
_query: &sqlparser::ast::Query,
) -> std::ops::ControlFlow<Self::Break> {
self.current_access_type_stack.push(R);
std::ops::ControlFlow::Continue(())
}
fn post_visit_query(
&mut self,
_query: &sqlparser::ast::Query,
) -> std::ops::ControlFlow<Self::Break> {
self.current_access_type_stack.pop();
std::ops::ControlFlow::Continue(())
}
// We do not use pre_visit_relation because we cannot know if an ObjectName is a table or a function
}
/// read_parquet('s3://...')
fn parse_s3_object_read_fn(input: &str) -> IResult<&str, &str> {
let (input, _) = alt((
tag_no_case("read_parquet"),
tag_no_case("read_csv"),
tag_no_case("read_json"),
))
.parse(input)?;
let (input, _) = multispace0(input)?;
let (input, _) = char('(')(input)?;
let (input, _) = multispace0(input)?;
let (input, path) = parse_s3_object_lit(input)?;
let (input, _) = multispace0(input)?;
let (input, _) = char(')')(input)?;
Ok((input, path))
fn is_read_fn(fname: &str) -> bool {
fname.eq_ignore_ascii_case("read_parquet")
|| fname.eq_ignore_ascii_case("read_csv")
|| fname.eq_ignore_ascii_case("read_json")
}
/// SELECT ... FROM 's3://...'
fn parse_s3_object_select_from(input: &str) -> IResult<&str, &str> {
let (input, _) = tag_no_case("FROM").parse(input)?;
let (input, _) = multispace0(input)?;
let (input, path) = parse_s3_object_lit(input)?;
Ok((input, path))
}
/// 's3://...'
fn parse_s3_object_lit(input: &str) -> IResult<&str, &str> {
let (input, _) = quote(input)?;
let (input, _) = tag("s3://").parse(input)?;
let (input, path) = take_while(|c| c != '\'' && c != '"')(input)?;
let (input, _) = quote(input)?;
Ok((input, path))
fn get_trivial_obj_name(name: &sqlparser::ast::ObjectName) -> Option<&str> {
if name.0.len() != 1 {
return None;
}
Some(name.0.first()?.as_ident()?.value.as_str())
}
fn quote(input: &str) -> IResult<&str, char> {
alt((char('\''), char('\"'))).parse(input)
fn get_str_lit_from_obj_name(name: &ObjectName) -> Option<&str> {
if name.0.len() != 1 {
return None;
}
let ident = name.0.first()?.as_ident()?;
if ident.quote_style != Some('\'') {
return None;
}
Some(ident.value.as_str())
}
fn parse_resource_lit(input: &str) -> IResult<&str, &str> {
let (input, _) = quote(input)?;
let (input, _) = alt((tag("$res:"), tag("res://"))).parse(input)?;
let (input, path) = take_while(|c| c != '\'' && c != '"')(input)?;
let (input, _) = quote(input)?;
Ok((input, path))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_sql_asset_parser_s3_literals() {
let input = r#"
SELECT * FROM read_parquet('s3:///a.parquet');
COPY (SELECT * FROM 's3://snd/b.parquet') TO 's3:///c.parquet';
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![
ParseAssetsResult {
kind: AssetKind::S3Object,
path: "/a.parquet".to_string(),
access_type: Some(R)
},
ParseAssetsResult {
kind: AssetKind::S3Object,
path: "/c.parquet".to_string(),
access_type: Some(W)
},
ParseAssetsResult {
kind: AssetKind::S3Object,
path: "snd/b.parquet".to_string(),
access_type: Some(R)
},
])
);
}
fn parse_ducklake_lit(input: &str) -> IResult<&str, &str> {
let (input, _) = quote(input)?;
let (input, _) = tag("ducklake").parse(input)?;
let (input, path) =
opt(preceded(tag("://"), take_while(|c| c != '\'' && c != '"'))).parse(input)?;
let (input, _) = quote(input)?;
Ok((input, path.unwrap_or("main")))
}
#[test]
fn test_sql_asset_parser_attach_no_usage_is_registered_as_unknown() {
let input = r#"
ATTACH 'ducklake://my_dl' AS dl;
SELECT 2;
USE dl;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "my_dl".to_string(),
access_type: None
},])
);
}
fn parse_datatable_lit(input: &str) -> IResult<&str, &str> {
let (input, _) = quote(input)?;
let (input, _) = tag("datatable").parse(input)?;
let (input, path) =
opt(preceded(tag("://"), take_while(|c| c != '\'' && c != '"'))).parse(input)?;
let (input, _) = quote(input)?;
Ok((input, path.unwrap_or("main")))
}
#[test]
fn test_sql_asset_parser_attach_dot_notation_read() {
let input = r#"
ATTACH 'ducklake://my_dl' AS dl;
SELECT * FROM dl.table1;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "my_dl/table1".to_string(),
access_type: Some(R)
},])
);
}
fn parse_comment(input: &str) -> IResult<&str, &str> {
let (input, _) = tag("--").parse(input)?;
let (input, comment) = take_while(|c| c != '\n')(input)?;
Ok((input, comment))
#[test]
fn test_sql_asset_parser_attach_dot_notation_write() {
let input = r#"
ATTACH 'datatable://my_dt' AS dt;
SELECT dt.read_bait FROM unrelated_table; -- dt. doesn't access the asset
INSERT INTO dt.table1 VALUES ('test');
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "my_dt/table1".to_string(),
access_type: Some(W)
},])
);
}
#[test]
fn test_sql_asset_parser_detach() {
let input = r#"
ATTACH 'ducklake://my_dl' AS dl;
DETACH dl;
SELECT * FROM dl.table1;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(s.map_err(|e| e.to_string()), Ok(vec![]));
}
#[test]
fn test_sql_asset_parser_implicit_use_asset() {
let input = r#"
ATTACH 'ducklake://my_dl' AS dl;
USE dl;
INSERT INTO table1 VALUES ('test');
USE memory;
SELECT * FROM table1;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "my_dl/table1".to_string(),
access_type: Some(W)
},])
);
}
#[test]
fn test_sql_asset_parser_default_main() {
let input = r#"
ATTACH 'datatable' AS dl;
INSERT INTO dl.table1 VALUES ('test');
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "main/table1".to_string(),
access_type: Some(W)
},])
);
}
#[test]
fn test_sql_asset_parser_create_table() {
let input = r#"
ATTACH 'ducklake' AS dl; USE dl;
CREATE TABLE friends (
name text,
age int
);
INSERT INTO friends VALUES ($name, $age);
SELECT * FROM friends;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "main/friends".to_string(),
access_type: Some(RW)
},])
);
}
// Make sure a_function is not detected as main/a_function
#[test]
fn test_sql_asset_parser_function_table() {
let input = r#"
ATTACH 'ducklake' AS dl; USE dl;
SELECT * FROM a_function('');
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "main".to_string(),
access_type: None
},])
);
}
#[test]
fn test_sql_asset_parser_delete() {
let input = r#"
ATTACH 'ducklake' AS dl;
USE dl;
DELETE FROM table1;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "main/table1".to_string(),
access_type: Some(W)
},])
);
}
#[test]
fn test_sql_asset_parser_update() {
let input = r#"
ATTACH 'ducklake' AS dl;
USE dl;
UPDATE table1 SET id = NULL;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "main/table1".to_string(),
access_type: Some(W)
},])
);
}
#[test]
fn test_sql_asset_parser_resource() {
let input = r#"
ATTACH 'res://u/user/pg_resource' AS db (TYPE postgres);
USE db;
SELECT * FROM table1;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Resource,
path: "u/user/pg_resource/table1".to_string(),
access_type: Some(R)
},])
);
}
#[test]
fn test_sql_asset_parser_update_with_dot_notation() {
let input = r#"
ATTACH 'ducklake' AS dl;
UPDATE dl.table1 SET id = NULL;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "main/table1".to_string(),
access_type: Some(W)
},])
);
}
#[test]
fn test_sql_asset_parser_table_with_schema() {
let input = r#"
ATTACH 'ducklake' AS dl;
UPDATE dl.sch.table1 SET id = NULL;
SELECT * FROM dl.sch.table1;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "main/sch.table1".to_string(),
access_type: Some(RW)
},])
);
}
#[test]
fn test_sql_asset_parser_table_with_schema_implicit() {
let input = r#"
ATTACH 'ducklake' AS dl;
USE dl;
UPDATE sch.table1 SET id = NULL;
SELECT * FROM sch.table1;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "main/sch.table1".to_string(),
access_type: Some(RW)
},])
);
}
}

View File

@@ -15,6 +15,7 @@ serde-wasm-bindgen.workspace = true
[dependencies]
windmill-parser.workspace = true
windmill-parser-sql.workspace = true
swc_common.workspace = true
triomphe.workspace = true
swc_ecma_parser.workspace = true

View File

@@ -1,16 +1,16 @@
use std::collections::HashMap;
use swc_common::{sync::Lrc, FileName, SourceMap};
use swc_common::{sync::Lrc, FileName, SourceMap, Spanned};
use swc_ecma_ast::{CallExpr, Expr, Lit, MemberExpr, MemberProp, Str};
use swc_ecma_parser::{lexer::Lexer, Parser, StringInput, Syntax, TsSyntax};
use swc_ecma_visit::{Visit, VisitWith};
use windmill_parser::asset_parser::{
detect_sql_access_type, merge_assets, parse_asset_syntax, AssetKind, AssetUsageAccessType,
ParseAssetsResult,
asset_was_used, merge_assets, parse_asset_syntax, AssetKind, AssetUsageAccessType,
ParseAssetsOutput, ParseAssetsResult, SqlQueryDetails,
};
use AssetUsageAccessType::*;
pub fn parse_assets(code: &str) -> anyhow::Result<Vec<ParseAssetsResult<String>>> {
pub fn parse_assets(code: &str) -> anyhow::Result<ParseAssetsOutput> {
let cm: Lrc<SourceMap> = Default::default();
let fm = cm.new_source_file(FileName::Custom("main.ts".into()).into(), code.into());
let lexer = Lexer::new(
@@ -35,13 +35,19 @@ pub fn parse_assets(code: &str) -> anyhow::Result<Vec<ParseAssetsResult<String>>
anyhow::anyhow!("Error while parsing code, it is invalid TypeScript: {err_s}, {e:?}")
})?
.body;
let mut assets_finder = AssetsFinder { assets: vec![], var_identifiers: HashMap::new() };
let mut assets_finder =
AssetsFinder { assets: vec![], sql_queries: vec![], var_identifiers: HashMap::new() };
assets_finder.visit_module_items(&ast);
Ok(merge_assets(assets_finder.assets))
Ok(ParseAssetsOutput {
assets: merge_assets(assets_finder.assets),
sql_queries: assets_finder.sql_queries,
})
}
type VarAssetName = String;
type VarAssetSchema = Option<String>;
struct AssetsFinder {
assets: Vec<ParseAssetsResult<String>>,
assets: Vec<ParseAssetsResult>,
// The user will write code like:
// let sql = wmill.datatable('main')
@@ -49,16 +55,64 @@ struct AssetsFinder {
// The goal is to remember that the identifier "sql" corresponds to the datatable "main"
// so that when we see a tagged template expression with tag "sql" we know which datatable it
// corresponds to. This allows us to infer if a datatable is Read or Write based on the SQL query.
var_identifiers: HashMap<String, (AssetKind, String)>,
var_identifiers: HashMap<String, (AssetKind, VarAssetName, VarAssetSchema)>,
sql_queries: Vec<SqlQueryDetails>,
}
/// Helper function to extract wmill.datatable() or wmill.ducklake() calls,
/// Returns (AssetKind, asset_name, optional_schema_name)
fn extract_wmill_datatable_call(expr: &Expr) -> Option<(AssetKind, String, Option<String>)> {
if let Expr::Call(call_expr) = expr {
if let Some(Expr::Member(member)) = call_expr.callee.as_expr().map(AsRef::as_ref) {
// Check if object is "wmill"
let is_wmill = matches!(
member.obj.as_ref(),
Expr::Ident(ident) if ident.sym.as_str() == "wmill"
);
if is_wmill {
if let MemberProp::Ident(prop) = &member.prop {
// Get the asset name from first arg, default to "main"
let asset_name = call_expr
.args
.first()
.and_then(|arg| match arg.expr.as_ref() {
Expr::Lit(Lit::Str(s)) => Some(s.value.to_string()),
_ => None,
})
.unwrap_or_else(|| "main".to_string());
let (asset_name, schema_name) = asset_name.split_once(':').map_or_else(
|| (asset_name.clone(), None),
|(name, schema)| {
(
(if name.is_empty() { "main" } else { name }).to_string(),
Some(schema.to_string()),
)
},
);
let kind = match prop.sym.as_str() {
"datatable" => Some(AssetKind::DataTable),
"ducklake" => Some(AssetKind::Ducklake),
_ => None,
};
return kind.map(|k| (k, asset_name, schema_name));
}
}
}
}
None
}
impl Visit for AssetsFinder {
// visit_call_expr will not recurse if it detects an asset,
// so this will only be called when no further context was found
fn visit_lit(&mut self, node: &swc_ecma_ast::Lit) {
match node {
swc_ecma_ast::Lit::Str(str) => {
if let Some((kind, path)) = parse_asset_syntax(str.value.as_str()) {
if let Some((kind, path)) = parse_asset_syntax(str.value.as_str(), false) {
self.assets.push(ParseAssetsResult {
kind,
path: path.to_string(),
@@ -77,6 +131,37 @@ impl Visit for AssetsFinder {
}
}
fn visit_assign_expr(&mut self, node: &swc_ecma_ast::AssignExpr) {
// Handle reassignments like: sql = wmill.datatable('main')
// Extract the variable name from the left side
let var_name = match &node.left {
swc_ecma_ast::AssignTarget::Simple(simple_target) => match simple_target {
swc_ecma_ast::SimpleAssignTarget::Ident(ident_binding) => {
ident_binding.id.sym.as_str().to_string()
}
_ => {
node.visit_children_with(self);
return;
}
},
_ => {
node.visit_children_with(self);
return;
}
};
// Check if right side is a wmill.datatable() or wmill.ducklake() call
if let Some((kind, asset_name, schema)) = extract_wmill_datatable_call(node.right.as_ref())
{
self.var_identifiers
.insert(var_name, (kind, asset_name, schema));
return;
}
// Default: visit children
node.visit_children_with(self);
}
fn visit_block_stmt(&mut self, node: &swc_ecma_ast::BlockStmt) {
// Save current state before entering the block
let saved_var_identifiers = self.var_identifiers.clone();
@@ -84,19 +169,12 @@ impl Visit for AssetsFinder {
// Visit children (this may add new identifiers)
node.visit_children_with(self);
// If we find 'let sql = wmill.datatable(...)',
// but no sql`` tagged templates were used, we add
// the asset with unknown access type
for var in self.var_identifiers.keys() {
if saved_var_identifiers.contains_key(var) {
continue;
}
let (kind, ref path) = self.var_identifiers[var];
if self
.assets
.iter()
.any(|a| a.kind == kind && &a.path == path)
{
let (kind, ref path, _) = self.var_identifiers[var];
if asset_was_used(&self.assets, (kind, path)) {
continue;
}
self.assets
@@ -118,43 +196,12 @@ impl Visit for AssetsFinder {
};
// Check if init is a call to wmill.datatable(...) or wmill.ducklake(...)
// optionally with .schema() chained
if let Some(init) = &node.init {
if let Expr::Call(call_expr) = init.as_ref() {
if let Some(Expr::Member(member)) = call_expr.callee.as_expr().map(AsRef::as_ref) {
// Check if object is "wmill"
let is_wmill = matches!(
member.obj.as_ref(),
Expr::Ident(ident) if ident.sym.as_str() == "wmill"
);
if is_wmill {
if let MemberProp::Ident(prop) = &member.prop {
// Get the asset name from first arg, default to "main"
let asset_name = call_expr
.args
.first()
.and_then(|arg| match arg.expr.as_ref() {
Expr::Lit(Lit::Str(s)) => Some(s.value.to_string()),
_ => None,
})
.unwrap_or_else(|| "main".to_string());
match prop.sym.as_str() {
"datatable" => {
self.var_identifiers
.insert(var_name, (AssetKind::DataTable, asset_name));
return;
}
"ducklake" => {
self.var_identifiers
.insert(var_name, (AssetKind::Ducklake, asset_name));
return;
}
_ => {}
}
}
}
}
if let Some((kind, asset_name, schema)) = extract_wmill_datatable_call(init.as_ref()) {
self.var_identifiers
.insert(var_name, (kind, asset_name, schema));
return;
}
}
@@ -173,27 +220,67 @@ impl Visit for AssetsFinder {
};
// Check if it's a known identifier
let (kind, asset_name) = if let Some((kind, name)) = self.var_identifiers.get(tag_name) {
(*kind, name.clone())
} else {
let Some((kind, asset_name, schema)) = self.var_identifiers.get(tag_name) else {
node.visit_children_with(self);
return;
};
// Extract the SQL query from the template quasis (string parts)
// Substitute ${} with $1, $2, etc.
let sql: String = node
.tpl
.quasis
.iter()
.map(|quasi| quasi.raw.as_str())
.collect::<Vec<_>>()
.join(" ");
.enumerate()
.fold(String::new(), |acc, (i, s)| {
if i == 0 {
s.to_string()
} else {
format!("{}${}{}", acc, i, s)
}
});
// Determine access type based on SQL keywords
let access_type = detect_sql_access_type(&sql);
let duckdb_conn_prefix = match kind {
AssetKind::DataTable => "datatable",
AssetKind::Ducklake => "ducklake",
_ => return,
};
self.assets
.push(ParseAssetsResult { kind, path: asset_name, access_type });
// Capture SQL query details before transforming for SQL parser
let span = node.span();
let span_tuple = (span.lo.0, span.hi.0);
self.sql_queries.push(SqlQueryDetails {
query_string: sql.clone(),
span: span_tuple,
source_kind: *kind,
source_name: asset_name.clone(),
source_schema: schema.clone(),
});
let sql_with_attach =
format!("ATTACH '{duckdb_conn_prefix}://{asset_name}' AS dt; USE dt; {sql}");
// We use the SQL parser to detect if it's a read or write query
match windmill_parser_sql::parse_assets(&sql_with_attach) {
Ok(mut sql_assets) => {
if let Some(schema) = schema {
for asset in &mut sql_assets.assets {
if asset.kind == *kind && asset.path.starts_with(asset_name) {
asset.path = format!(
"{}/{}.{}",
asset_name,
schema,
&asset.path[asset_name.len() + 1..]
);
}
}
}
self.assets.extend(sql_assets.assets);
}
_ => {}
}
}
}
@@ -221,7 +308,9 @@ impl AssetsFinder {
match arg_value.map(|e| e.expr.as_ref()) {
Some(Expr::Lit(Lit::Str(Str { value, .. }))) => {
let path = parse_asset_syntax(&value).map(|(_, p)| p).unwrap_or(&value);
let path = parse_asset_syntax(&value, false)
.map(|(_, p)| p)
.unwrap_or(&value);
self.assets
.push(ParseAssetsResult { kind, path: path.to_string(), access_type });
}
@@ -230,3 +319,365 @@ impl AssetsFinder {
Ok(())
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_ts_asset_parser_load_s3() {
let input = r#"
import * as wmill from "windmill-client"
export async function main() {
wmill.loadS3File('s3:///test.csv')
}
"#;
let s = parse_assets(input);
assert_eq!(
s.map(|r| r.assets).map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::S3Object,
path: "/test.csv".to_string(),
access_type: Some(R)
},])
);
}
#[test]
fn test_ts_asset_parser_unused_sql() {
let input = r#"
import * as wmill from "windmill-client"
export async function main() {
let sql = wmill.datatable('dt')
}
"#;
let s = parse_assets(input);
assert_eq!(
s.map(|r| r.assets).map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt".to_string(),
access_type: None
},])
);
}
#[test]
fn test_ts_asset_parser_sql_read() {
let input = r#"
import * as wmill from "windmill-client"
export async function main(x: number) {
let sql = wmill.datatable('dt')
return await sql`SELECT * FROM friends WHERE age = ${x}`.fetch()
}
"#;
let s = parse_assets(input);
assert_eq!(
s.map(|r| r.assets).map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/friends".to_string(),
access_type: Some(R)
},])
);
}
#[test]
fn test_ts_asset_parser_sql_read_write() {
let input = r#"
import * as wmill from "windmill-client"
export async function main(x: number) {
let sql = wmill.datatable('dt')
await sql`UPDATE friends SET name = 'Pierre' WHERE age = ${x}`.fetch()
let pierre = await sql`SELECT * FROM friends WHERE age = ${x}`.fetchOne()
return await sql`SELECT * FROM analytics`.fetch()
}
"#;
let s = parse_assets(input);
assert_eq!(
s.map(|r| r.assets).map_err(|e| e.to_string()),
Ok(vec![
ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/analytics".to_string(),
access_type: Some(R)
},
ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/friends".to_string(),
access_type: Some(RW)
},
])
);
}
#[test]
fn test_ts_asset_parser_multiple_sql_scopes() {
let input = r#"
import * as wmill from "windmill-client"
export async function main() {
async function f(x: number) {
let sql = wmill.datatable()
return await sql`SELECT * FROM friends WHERE age = ${x}`.fetch()
}
let sql = wmill.datatable('another1')
return await sql`INSERT INTO customers VALUES (${0})`.fetch()
}
function unused() {
let sql = wmill.ducklake('another2')
}
"#;
let s = parse_assets(input);
assert_eq!(
s.map(|r| r.assets).map_err(|e| e.to_string()),
Ok(vec![
ParseAssetsResult {
kind: AssetKind::DataTable,
path: "another1/customers".to_string(),
access_type: Some(W)
},
ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "another2".to_string(),
access_type: None
},
ParseAssetsResult {
kind: AssetKind::DataTable,
path: "main/friends".to_string(),
access_type: Some(R)
},
])
);
}
#[test]
fn test_ts_asset_parser_overriden_var_identifier() {
let input = r#"
import * as wmill from "windmill-client"
export async function main() {
let sql = wmill.datatable('another1')
}
function g() {
let sql = wmill.ducklake()
}
"#;
let s = parse_assets(input);
assert_eq!(
s.map(|r| r.assets).map_err(|e| e.to_string()),
Ok(vec![
ParseAssetsResult {
kind: AssetKind::DataTable,
path: "another1".to_string(),
access_type: None
},
ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "main".to_string(),
access_type: None
},
])
);
}
#[test]
fn test_ts_asset_parser_datatable_with_schema() {
let input = r#"
import * as wmill from "windmill-client"
export async function main(x: number) {
let sql = wmill.datatable(':myschema')
return await sql`SELECT * FROM friends WHERE age = ${x}`.fetch()
}
"#;
let s = parse_assets(input);
assert_eq!(
s.map(|r| r.assets).map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "main/myschema.friends".to_string(),
access_type: Some(R)
},])
);
}
#[test]
fn test_ts_asset_parser_schema_with_write() {
let input = r#"
import * as wmill from "windmill-client"
export async function main(x: number) {
let sql = wmill.datatable('dt:public')
await sql`INSERT INTO users VALUES (${x})`.fetch()
return await sql`SELECT * FROM users`.fetch()
}
"#;
let s = parse_assets(input);
assert_eq!(
s.map(|r| r.assets).map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/public.users".to_string(),
access_type: Some(RW)
},])
);
}
#[test]
fn test_ts_asset_parser_unused_datatable_with_schema() {
let input = r#"
import * as wmill from "windmill-client"
export async function main() {
let sql = wmill.datatable('dt:myschema')
}
"#;
let s = parse_assets(input);
assert_eq!(
s.map(|r| r.assets).map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt".to_string(),
access_type: None
},])
);
}
#[test]
fn test_ts_asset_parser_reassignment() {
let input = r#"
import * as wmill from "windmill-client"
export async function main(x: number) {
let sql;
sql = wmill.datatable('dt')
return await sql`SELECT * FROM users WHERE id = ${x}`.fetch()
}
"#;
let s = parse_assets(input);
assert_eq!(
s.map(|r| r.assets).map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/users".to_string(),
access_type: Some(R)
},])
);
}
#[test]
fn test_ts_asset_parser_reassignment_with_schema() {
let input = r#"
import * as wmill from "windmill-client"
export async function main(x: number) {
let sql = wmill.datatable('dt')
await sql`INSERT INTO test VALUES ('')`.fetch()
sql = wmill.datatable('dt:private')
return await sql`SELECT * FROM users WHERE id = ${x}`.fetch()
}
"#;
let s = parse_assets(input);
assert_eq!(
s.map(|r| r.assets).map_err(|e| e.to_string()),
Ok(vec![
ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/private.users".to_string(),
access_type: Some(R)
},
ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/test".to_string(),
access_type: Some(W)
},
])
);
}
#[test]
fn test_ts_asset_parser_sql_query_details() {
let input = r#"
import * as wmill from "windmill-client"
export async function main(x: number) {
let sql = wmill.datatable('dt')
return await sql`SELECT * FROM friends WHERE age = ${x}`.fetch()
}
"#;
let result = parse_assets(input).unwrap();
// Check assets
assert_eq!(result.assets.len(), 1);
assert_eq!(result.assets[0].kind, AssetKind::DataTable);
assert_eq!(result.assets[0].path, "dt/friends");
// Check SQL query details
assert_eq!(result.sql_queries.len(), 1);
let query_detail = &result.sql_queries[0];
assert_eq!(
query_detail.query_string,
"SELECT * FROM friends WHERE age = $1"
);
assert_eq!(query_detail.source_kind, AssetKind::DataTable);
assert_eq!(query_detail.source_name, "dt");
assert_eq!(query_detail.source_schema, None);
// Span should be non-zero
assert!(query_detail.span.0 > 0);
assert!(query_detail.span.1 > query_detail.span.0);
}
#[test]
fn test_ts_asset_parser_sql_query_details_with_schema() {
let input = r#"
import * as wmill from "windmill-client"
export async function main(x: number) {
let sql = wmill.datatable('dt:public')
await sql`INSERT INTO users VALUES (${x})`.fetch()
return await sql`SELECT * FROM users`.fetch()
}
"#;
let result = parse_assets(input).unwrap();
// Check SQL query details
assert_eq!(result.sql_queries.len(), 2);
// First query (INSERT)
assert_eq!(
result.sql_queries[0].query_string,
"INSERT INTO users VALUES ($1)"
);
assert_eq!(result.sql_queries[0].source_kind, AssetKind::DataTable);
assert_eq!(result.sql_queries[0].source_name, "dt");
assert_eq!(
result.sql_queries[0].source_schema,
Some("public".to_string())
);
// Second query (SELECT)
assert_eq!(result.sql_queries[1].query_string, "SELECT * FROM users");
assert_eq!(result.sql_queries[1].source_kind, AssetKind::DataTable);
assert_eq!(result.sql_queries[1].source_name, "dt");
assert_eq!(
result.sql_queries[1].source_schema,
Some("public".to_string())
);
}
#[test]
fn test_ts_asset_parser_sql_query_details_ducklake() {
let input = r#"
import * as wmill from "windmill-client"
export async function main() {
let sql = wmill.ducklake('my_lake')
return await sql`SELECT id, name FROM products LIMIT 10`.fetch()
}
"#;
let result = parse_assets(input).unwrap();
// Check SQL query details
assert_eq!(result.sql_queries.len(), 1);
let query_detail = &result.sql_queries[0];
assert_eq!(
query_detail.query_string,
"SELECT id, name FROM products LIMIT 10"
);
assert_eq!(query_detail.source_kind, AssetKind::Ducklake);
assert_eq!(query_detail.source_name, "my_lake");
assert_eq!(query_detail.source_schema, None);
}
}

Some files were not shown because too many files have changed in this diff Show More