Commit Graph

54 Commits

Author SHA1 Message Date
Diego Imbert
75fdc2cdc9 feat: data table schemas (#7353)
* data tables settings ui

* install runed

* zod 4 fixes

* use new toJSONSchema

* Migrate ducklake catalogs to more generic custom instance databases

* fix compilation

* Safety conversion for old duckdb ffi

* data tables settings

* ts client basis

* inline run works

* datatables work

* Revert "datatables work"

This reverts commit 6e1588d59e.

* datatables work (without leaking pg credentials)

* println

* separate sqlUtils.ts

* nit

* Separate custom instance db Select and Wizard components

* nit

* nit wording

* add tags to custom instance dbs

* error when trying to use ducklake as datatable or opposite

* show status in dropdown

* data table instance setup works

* sqk function for ducklake

* factorize logic

* fix temp reactivity

* Data table assetexplore

* Migrate S3 permissions to modal

* Revert "Migrate S3 permissions to modal"

This reverts commit 0631d03cb0.

* nit query -> fetch

* Custom instance setup new look

* run_language_executor separate fn

* run_inline param

* nit wording

* Better typed client

* Data tables display as assets in frontend

* asset db icon

* nit

* cleaner errors

* nit

* Fix sed calls in mac

* run_inline_script_preview in python client

* basic python datatable client

* datatable and datalake parser in python

* ducklake client python

* nit fix

* Fix migration producing NULL instead of {} when no custom databases

* merge conflict fail

* python ducklake client arg fix

* parse or infer sql types in ts client

* ts asset parser, detect datatable & ducklake R/W

* fix sql repl for other read ops than select

* export type SqlTemplateFunction

* rename list_custom_instance_pg_databases

* typecheck datatable and ducklake name in Typescript

* Fix typecheck datatable and ducklake in TS

* declare module overriding instead of extending

* infer_sql_type in python client

* SqlQuery object in python

* fix merge conflicts

* update const_format

* CI fix

* factor out to var_identifiers

* sqlx prepare

* unnecessary security (admin is required)

* clearer comment

* ee repo ref

* nit snake case

* claude step 1: detect var declarations

* move detect_sql_access_type to common mod

* claude step 2: detect when saved vars are queried

* Revert "claude step 2: detect when saved vars are queried"

This reverts commit 1e1f930568.

* Revert "claude step 1: detect var declarations"

This reverts commit f866f4819d.

* remove ducklake/datatable and default

* detect data table assigns in var_identifiers

* Python parser successfully infers R/W/RW from ducklake / datatable

* still register ducklake/datatable if not used as unknown R/W

* Go to settings button in Assets Dropdown on not found

* nit

* sqlx prepare fail

* manual fix, somehow sqlx prepare won't do it

* fix frontend ci

* ee repo ref

* ducklake_user doesnt exist in unit tests

* nit fix

* ui nit

* nit

* nit missing clone

* fork ducklakes and datatables

* fix surface hover bug

* stupid mistake

* better deeply reactive mutable derived

* Ducklake picker

* Editor bar data tables

* DuckDB supports datatables

* datatable in duckdb asset parser

* duckdb asset parser var_identifiers

* Revert "duckdb asset parser var_identifiers"

This reverts commit 88068b1a77.

* sqlx prepare

* Box pin in test_workflow_as_code to fix stack overflow

* stash

* sql asset parser parses most s3 literals

* nit

* Detect attach + handle returning RW

* detect assets used with dot notation

* detect implicit access with USE dl; syntax

* Add assets as unknown if var was never used

* Support default ducklake/datatable main in parser

* ignore asset parsing errors in frontend (avoid flow layout shift)

* super weird duplication (merge conflict ?)

* nits

* fix duckdb parser detecting too much as asset when RW ctx is unknown

* fix transparent assets btn

* missing arg

* nit styling

* asset parser specific table parsing

* fix resource specific table parsing

* More concise asset display in flows + better icons

* fix assets page filtering out resources with added table

* Fix frontend to support specific table assets

* Open DB Manager to specific table

* Specific table parser in Python and TS + unit tests

* Fix UPDATE setting access to None

* fix flow edge rendering on top of output picker

* python parser fix var override bug

* add ts test

* fix compilation

* sqlx prepare

* update parsers version

* fix missing schema key onDelete

* Grant permission to create schemas in custom instance databases

* Update pg query to return empty schemas

* Create schema

* Select nits

* support schemas in sql parser

* ts parser handle schema with sql parser result

* detect .schema() syntax

* detect schema syntax in python

* support .schema() in ts and py SDK

* open db manager to specific schema

* support reassignment in ts parser

* nit better unitest

* : syntax in ts

* datatable:schema syntax in python

* fix client py

* nit select dropdown darkmode

* object | null fetchOne

* ts client nits

* parse_sql_client_name fn

* getImportWmillTsStatement refactor in EditorBar

* text to json() in python client

* update parser versions

* pkg lock
2025-12-16 21:16:42 +00:00
Diego Imbert
c6c7f3415a Specific asset tables (#7323)
* data tables settings ui

* install runed

* zod 4 fixes

* use new toJSONSchema

* Migrate ducklake catalogs to more generic custom instance databases

* fix compilation

* Safety conversion for old duckdb ffi

* data tables settings

* ts client basis

* inline run works

* datatables work

* Revert "datatables work"

This reverts commit 6e1588d59e.

* datatables work (without leaking pg credentials)

* println

* separate sqlUtils.ts

* nit

* Separate custom instance db Select and Wizard components

* nit

* nit wording

* add tags to custom instance dbs

* error when trying to use ducklake as datatable or opposite

* show status in dropdown

* data table instance setup works

* sqk function for ducklake

* factorize logic

* fix temp reactivity

* Data table assetexplore

* Migrate S3 permissions to modal

* Revert "Migrate S3 permissions to modal"

This reverts commit 0631d03cb0.

* nit query -> fetch

* Custom instance setup new look

* run_language_executor separate fn

* run_inline param

* nit wording

* Better typed client

* Data tables display as assets in frontend

* asset db icon

* nit

* cleaner errors

* nit

* Fix sed calls in mac

* run_inline_script_preview in python client

* basic python datatable client

* datatable and datalake parser in python

* ducklake client python

* nit fix

* Fix migration producing NULL instead of {} when no custom databases

* merge conflict fail

* python ducklake client arg fix

* parse or infer sql types in ts client

* ts asset parser, detect datatable & ducklake R/W

* fix sql repl for other read ops than select

* export type SqlTemplateFunction

* rename list_custom_instance_pg_databases

* typecheck datatable and ducklake name in Typescript

* Fix typecheck datatable and ducklake in TS

* declare module overriding instead of extending

* infer_sql_type in python client

* SqlQuery object in python

* fix merge conflicts

* update const_format

* CI fix

* factor out to var_identifiers

* sqlx prepare

* unnecessary security (admin is required)

* clearer comment

* ee repo ref

* nit snake case

* claude step 1: detect var declarations

* move detect_sql_access_type to common mod

* claude step 2: detect when saved vars are queried

* Revert "claude step 2: detect when saved vars are queried"

This reverts commit 1e1f930568.

* Revert "claude step 1: detect var declarations"

This reverts commit f866f4819d.

* remove ducklake/datatable and default

* detect data table assigns in var_identifiers

* Python parser successfully infers R/W/RW from ducklake / datatable

* still register ducklake/datatable if not used as unknown R/W

* Go to settings button in Assets Dropdown on not found

* nit

* sqlx prepare fail

* manual fix, somehow sqlx prepare won't do it

* fix frontend ci

* ee repo ref

* ducklake_user doesnt exist in unit tests

* nit fix

* ui nit

* nit

* nit missing clone

* fork ducklakes and datatables

* fix surface hover bug

* stupid mistake

* better deeply reactive mutable derived

* Ducklake picker

* Editor bar data tables

* DuckDB supports datatables

* datatable in duckdb asset parser

* duckdb asset parser var_identifiers

* Revert "duckdb asset parser var_identifiers"

This reverts commit 88068b1a77.

* sqlx prepare

* Box pin in test_workflow_as_code to fix stack overflow

* stash

* sql asset parser parses most s3 literals

* nit

* Detect attach + handle returning RW

* detect assets used with dot notation

* detect implicit access with USE dl; syntax

* Add assets as unknown if var was never used

* Support default ducklake/datatable main in parser

* ignore asset parsing errors in frontend (avoid flow layout shift)

* super weird duplication (merge conflict ?)

* nits

* fix duckdb parser detecting too much as asset when RW ctx is unknown

* fix transparent assets btn

* missing arg

* nit styling

* asset parser specific table parsing

* fix resource specific table parsing

* More concise asset display in flows + better icons

* fix assets page filtering out resources with added table

* Fix frontend to support specific table assets

* Open DB Manager to specific table

* Specific table parser in Python and TS + unit tests

* Fix UPDATE setting access to None

* fix flow edge rendering on top of output picker

* python parser fix var override bug

* add ts test

* fix compilation

* sqlx prepare

* update parsers version

* fix missing schema key onDelete
2025-12-11 13:34:17 +00:00
Diego Imbert
df884e6135 Custom instance db password safety (#7313)
* update parsers version

* safety custom instance db migration

* Refresh custom instance pwd button

* nit indent

* nit button accent

* Prevent leave without saving
2025-12-08 16:35:39 +00:00
Ruben Fiszel
0e63dd301f fix: raw app v0.5 (#7310) 2025-12-06 16:06:30 +00:00
Diego Imbert
9bbab3321e feat: Data tables (#7226)
* data tables settings ui

* install runed

* zod 4 fixes

* use new toJSONSchema

* Migrate ducklake catalogs to more generic custom instance databases

* fix compilation

* Safety conversion for old duckdb ffi

* data tables settings

* ts client basis

* inline run works

* datatables work

* Revert "datatables work"

This reverts commit 6e1588d59e.

* datatables work (without leaking pg credentials)

* println

* separate sqlUtils.ts

* nit

* Separate custom instance db Select and Wizard components

* nit

* nit wording

* add tags to custom instance dbs

* error when trying to use ducklake as datatable or opposite

* show status in dropdown

* data table instance setup works

* sqk function for ducklake

* factorize logic

* fix temp reactivity

* Data table assetexplore

* Migrate S3 permissions to modal

* Revert "Migrate S3 permissions to modal"

This reverts commit 0631d03cb0.

* nit query -> fetch

* Custom instance setup new look

* run_language_executor separate fn

* run_inline param

* nit wording

* Better typed client

* Data tables display as assets in frontend

* asset db icon

* nit

* cleaner errors

* nit

* Fix sed calls in mac

* run_inline_script_preview in python client

* basic python datatable client

* datatable and datalake parser in python

* ducklake client python

* nit fix

* Fix migration producing NULL instead of {} when no custom databases

* merge conflict fail

* python ducklake client arg fix

* parse or infer sql types in ts client

* ts asset parser, detect datatable & ducklake R/W

* fix sql repl for other read ops than select

* export type SqlTemplateFunction

* rename list_custom_instance_pg_databases

* typecheck datatable and ducklake name in Typescript

* Fix typecheck datatable and ducklake in TS

* declare module overriding instead of extending

* infer_sql_type in python client

* SqlQuery object in python

* fix merge conflicts

* update const_format

* CI fix

* factor out to var_identifiers

* sqlx prepare

* unnecessary security (admin is required)

* clearer comment

* ee repo ref

* nit snake case

* claude step 1: detect var declarations

* move detect_sql_access_type to common mod

* claude step 2: detect when saved vars are queried

* Revert "claude step 2: detect when saved vars are queried"

This reverts commit 1e1f930568.

* Revert "claude step 1: detect var declarations"

This reverts commit f866f4819d.

* remove ducklake/datatable and default

* detect data table assigns in var_identifiers

* Python parser successfully infers R/W/RW from ducklake / datatable

* still register ducklake/datatable if not used as unknown R/W

* Go to settings button in Assets Dropdown on not found

* nit

* sqlx prepare fail

* manual fix, somehow sqlx prepare won't do it

* fix frontend ci

* ee repo ref

* ducklake_user doesnt exist in unit tests

* nit fix

* ui nit

* nit

* nit missing clone

* fork ducklakes and datatables

* fix surface hover bug

* stupid mistake

* better deeply reactive mutable derived

* Ducklake picker

* Editor bar data tables

* DuckDB supports datatables

* datatable in duckdb asset parser

* duckdb asset parser var_identifiers

* Revert "duckdb asset parser var_identifiers"

This reverts commit 88068b1a77.

* sqlx prepare

* Box pin in test_workflow_as_code to fix stack overflow

* go to settings button

* ee repo ref

* fix compilation

* wording nit
2025-12-05 23:08:58 +00:00
Pyra
ef031cd3c0 nit(workspace-dependencies): better deployment warning (#7265)
* nit(workspace-dependencies): better deployment warning

Signed-off-by: pyranota <pyra@duck.com>

* ci

Signed-off-by: pyranota <pyra@duck.com>

* fix npm check

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
2025-12-01 11:18:37 +00:00
Pyra
d38c96db36 feat: workspace dependencies (#7124)
* commit raw requirements

Signed-off-by: pyranota <pyra@duck.com>

* raw requirements

Signed-off-by: pyranota <pyra@duck.com>

* implement `parse_annotation`

Signed-off-by: pyranota <pyra@duck.com>

* more progress on wdeps

Signed-off-by: pyranota <pyra@duck.com>

* more progress

Signed-off-by: pyranota <pyra@duck.com>

* fixes

Signed-off-by: pyranota <pyra@duck.com>

* more progress

Signed-off-by: pyranota <pyra@duck.com>

* fixes

Signed-off-by: pyranota <pyra@duck.com>

* cli improvements + raw deps

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fix python versions

Signed-off-by: pyranota <pyra@duck.com>

* progress

Signed-off-by: pyranota <pyra@duck.com>

* update :)

Signed-off-by: pyranota <pyra@duck.com>

* add MaybeLock

Signed-off-by: pyranota <pyra@duck.com>

* go WIP

Signed-off-by: pyranota <pyra@duck.com>

* fix python ignoring py version from requirements

Signed-off-by: pyranota <pyra@duck.com>

* optimize php

Signed-off-by: pyranota <pyra@duck.com>

* require admin to alter

Signed-off-by: pyranota <pyra@duck.com>

* fix(cli): flow generateLocks raw deps

Signed-off-by: pyranota <pyra@duck.com>

* progress in checklist

Signed-off-by: pyranota <pyra@duck.com>

* fix agent workers

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* nit: remove default features

Signed-off-by: pyranota <pyra@duck.com>

* oh-wow

Signed-off-by: pyranota <pyra@duck.com>

* remove dbg!

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* add indexes

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* remove todos

Signed-off-by: pyranota <pyra@duck.com>

* fix cli

Signed-off-by: pyranota <pyra@duck.com>

* add debug flag

Signed-off-by: pyranota <pyra@duck.com>

* cli: remove noise

Signed-off-by: pyranota <pyra@duck.com>

* fix cli

Signed-off-by: pyranota <pyra@duck.com>

* remove todos

Signed-off-by: pyranota <pyra@duck.com>

* trigger deps correctly

Signed-off-by: pyranota <pyra@duck.com>

* fix frontend

Signed-off-by: pyranota <pyra@duck.com>

* fix frontend again

Signed-off-by: pyranota <pyra@duck.com>

* finally fix frontend

Signed-off-by: pyranota <pyra@duck.com>

* ee repo ref

Signed-off-by: pyranota <pyra@duck.com>

* fix all

Signed-off-by: pyranota <pyra@duck.com>

* more fixes...

Signed-off-by: pyranota <pyra@duck.com>

* remove test

Signed-off-by: pyranota <pyra@duck.com>

* Update backend-test.yml

* comment out legacy test

Signed-off-by: pyranota <pyra@duck.com>

* fix ci

Signed-off-by: pyranota <pyra@duck.com>

* fix ci?

Signed-off-by: pyranota <pyra@duck.com>

* comment out thing

Signed-off-by: pyranota <pyra@duck.com>

* ignore test

Signed-off-by: pyranota <pyra@duck.com>

* ci

Signed-off-by: pyranota <pyra@duck.com>

* base fixture

Signed-off-by: pyranota <pyra@duck.com>

* fix regression

Signed-off-by: pyranota <pyra@duck.com>

* fix docs links

Signed-off-by: pyranota <pyra@duck.com>

* update min version

Signed-off-by: pyranota <pyra@duck.com>

* simplify

* implement cache for get_latest

Signed-off-by: pyranota <pyra@duck.com>

* move to workspace settings

Signed-off-by: pyranota <pyra@duck.com>

* sqlx + migration

Signed-off-by: pyranota <pyra@duck.com>

* more migrations

Signed-off-by: pyranota <pyra@duck.com>

* use box pin

Signed-off-by: pyranota <pyra@duck.com>

* nit

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-11-28 16:00:11 +00:00
Diego Imbert
2cae72c9db fix: fix parse_postgres_uri not decoding password (#7157)
* nit overscroll auto

* decode password in parse_postgres_url

* don't encode host in pg_executor

* nit
2025-11-17 11:29:44 +00:00
Ruben Fiszel
f1029d0f14 s3 endpoints improvements 2025-11-15 19:07:13 +00:00
wendrul
806a168e18 Add link to job + update git sync script tip if applicable on fork fail (#7117)
* Add link to job + update git sync script tip if applicable on fork fail

* Format
2025-11-12 12:32:36 +00:00
wendrul
192fecc86f fix: create git branch right before creating the workspace fork to catch errors and have a coherent fork point (#7073)
* Workspace forks: add endpoint to create a branch before creating a fork

* Update hubPaths + create branch before creating fork on frontend

* Update tmp ee-repo-ref

* Remove debug hubPath

* Prepare sqlx

* Fix ee imports

* Update ee-ref

* Update ee-repo-ref final

* Prepare sqlx
2025-11-06 16:45:34 +00:00
Alexander Petric
6b99bf5003 nit: git_branches -> gitBranches (#7059) 2025-11-04 20:01:14 +00:00
Alexander Petric
bcf4d5d78a nit: formatting cli commands in windmill git sync ui (#7058) 2025-11-04 18:18:06 +00:00
wendrul
663bc3b19a feat: is_admin non propagation + change endpoint for forks (#6958)
* fix is_admin non propagation + change endpoint

* Add deprecated endpoint to tell users to update cli

* Fix endpoint definitions

* fix

* fix

* fix

* fix

* fix

* fix

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-10-27 21:51:19 +00:00
Diego Imbert
f33ae2a4b1 Remove whitespace-nowrap from Button (#6955) 2025-10-27 15:49:27 +00:00
centdix
949f96d66a chore(aichat): display tool usage earlier (#6917)
* feat(copilot): display tool calls immediately in loading state during streaming

Display tool calls in loading state as soon as they are parsed during
OpenAI streaming, rather than waiting until processToolCall is invoked.

Changes:
- parseOpenAICompletion: Track initialized tool calls and display them
  immediately when we have complete tool info (id + function.name)
- processToolCall: Updated comment to clarify it merges with existing
  loading state set during parsing

This provides better UX by showing tool execution progress progressively
as the stream is parsed.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat(copilot): display Anthropic tool calls immediately in loading state

Apply the same immediate tool call display pattern to Anthropic streaming
that was implemented for OpenAI.

Changes:
- parseAnthropicCompletion: Display tool calls immediately in loading state
  when tool_use blocks are received in the message event

This ensures consistent UX across both OpenAI and Anthropic providers,
showing tool execution progress as soon as tool calls are detected.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat(copilot): show Anthropic tool calls even earlier with temp displays

Display temporary loading states for Anthropic tool calls as soon as
inputJson events are received (when tool input starts streaming), then
replace them with real tool displays when complete tool_use blocks
arrive in the message event.

Changes:
- ToolCallbacks: Added removeToolStatus method to clean up temp displays
- AIChatManager: Implemented removeToolStatus to remove tool messages
  from displayMessages array
- anthropic.ts:
  * Display temp tool on first inputJson event (earliest indicator)
  * Flush pending text message before showing temp tool (proper ordering)
  * Remove temp display when complete tool_use block arrives
  * Replace with real tool display via preAction

This provides the earliest possible feedback for Anthropic tool calls,
showing loading states as soon as the model starts generating tool
inputs rather than waiting for complete blocks.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* cleaning

* cleaning

* cleaning

* fix icon

* nit

* handle error

* nit

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-23 20:14:34 +00:00
centdix
b5d0f23f4f feat(aichat): add user-level custom system prompts (#6884)
* feat(aichat): create reusable CustomAIPrompts component

Extract custom AI prompts UI into a reusable component that can be
used in both workspace settings and user settings. Component includes:
- AI mode selector with visual indicators for set prompts
- Textarea with character limit
- Customizable title, description, and hint messages

Co-authored-by: centdix <centdix@users.noreply.github.com>

* refactor(aichat): update workspace AISettings to use reusable component

Replace inline custom prompts UI with the reusable CustomAIPrompts
component. Add hint about user-level custom prompts being available
in account settings and how they combine with workspace prompts.

Co-authored-by: centdix <centdix@users.noreply.github.com>

* feat(aichat): add user-level custom AI prompts in account settings

Add collapsible section in user settings for custom AI prompts:
- Stored in localStorage (key: userCustomAIPrompts)
- Collapsible UI to save space
- Visual indicator when prompts are configured
- Hint about prompt combination with workspace settings
- Prompts apply across all workspaces for the user

Co-authored-by: centdix <centdix@users.noreply.github.com>

* feat(aichat): combine workspace and user custom prompts

Update AIChatManager to combine workspace-level and user-level custom
prompts. Prompts are combined in order: workspace first, then user.

Add helper functions in aiStore.ts:
- getUserCustomPrompts(): retrieves user prompts from localStorage
- getCombinedCustomPrompt(mode): combines workspace + user prompts

All AI modes (script, flow, navigator, ask, API) now use combined
prompts, allowing users to append their own instructions to workspace
settings across all workspaces.

Co-authored-by: centdix <centdix@users.noreply.github.com>

* fix: remove unused imports

Remove unused imports to fix svelte-check errors:
- Remove unused 'get' from svelte/store in AIChatManager
- Remove unused 'copilotInfo' from aiStore in AIChatManager
- Remove unused 'AIMode' from AISettings

Co-authored-by: centdix <centdix@users.noreply.github.com>

* simplify

* nit

* fix

* fix

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: centdix <centdix@users.noreply.github.com>
2025-10-23 08:35:51 +00:00
Guilhem
032f0c1f8c feat: UX/UI full overhaul to meet new design system 2025-10-22 15:56:36 +00:00
centdix
cb8818796d feat(settings): add unsaved changes warning for workspace settings (#6813)
* feat(settings): add unsaved changes warning on windmill ai tab

Add dialog to warn users when leaving the Windmill AI settings tab with
unsaved changes, allowing them to save or cancel their changes.

Changes:
- Track initial AI config state in workspace settings
- Compare current vs initial state to detect unsaved changes
- Integrate UnsavedConfirmationModal with beforeNavigate guard
- Update initial state after successful save via onSave callback

Implements request from issue #6812

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* also confirm on tab changes

* fix

* fix

* fix

* clean tabs usage

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-14 16:50:19 +00:00
Diego Imbert
f798ff4535 feat: Database manager for Ducklake instance catalogs (#6785)
* OVERRIDE_DATA_PATH

* DB Manager for Ducklake instance catalog debugging
2025-10-09 16:28:11 +00:00
Diego Imbert
258b275f9b fix: better ducklake setup (#6763)
* stash

* Much better ducklake setup UX

* nits

* mistake

* sqlx prepare
2025-10-07 07:00:00 +00:00
Diego Imbert
686cf8dd69 Timestamp input in db manager (#6754)
* nullable by default

* unrelated nits

* native timezone input postgres

* Fix timestamptz NaiveDateTime conversion is pg_executor

* remove consolelog
2025-10-06 15:10:17 +00:00
Ruben Fiszel
9ddde9aa94 split store and aiStore 2025-10-04 09:59:10 +00:00
centdix
047420e5ad feat(flow): add option to turn flow into chat (#6658)
* add toggle option + chat interface

* backend impl

* draft

* put info in schema

* Revert "backend impl"

This reverts commit c534eeb49986424e2c12e2c5642be4e17ba380d1.

* chat interface in flow input

* cleaning

* add logic for running flow + styling

* handle historic args

* fix frontend changes

* add tables

* add conv list

* add endpoints

* adapt frontend

* list message logic

* save message in db

* save response in db

* cleaning

* better migrations

* refresh on new conv

* better logic for messages

* nit

* genere conversation uuid from frontend

* store chat mode info in flow status

* better ui for chat

* collapse chat

* ui

* infinite scroll on convs

* infinite scroll on messages

* fix ui

* new chat entry on new

* cleaning

* change setting logic

* fix test logic from flow input

* move toggle to input

* add warning modal when enabling chat mode

* add summary and explanation on inline script

* add hint for chat mode on user_message desc

* show chat message instead of input in graph

* add warning for triggers

* one logo when not expanded

* use infinitelist for conversations

* add warning when deployment in progress

* full width button

* better icon for menu

* better input + nits

* put toggle in action

* use waitjob

* cleaning

* cleaning

* scroll on new + cleaning

* use enum

* fix logic

* full screen

* cleaning

* exit on updatesqlx error

* Update SQLx metadata

* fix

* cleaning

* add for wait result endpoint

* add missing drop

* delete cascade

* fix: use macro version of query_as in flow_conversations.rs

Use sqlx::query_as! macro instead of query_as function for compile-time
SQL validation and better type safety

Co-authored-by: centdix <centdix@users.noreply.github.com>

* fix: update comment to clarify conversation message update condition

The comment now accurately reflects that the update happens when
it's a flow and it's done (flow_is_done)

Co-authored-by: centdix <centdix@users.noreply.github.com>

* fix: only parse chat_input_enabled if conditions are met

Move the parse_chat_input_enabled() call inside the condition check
to avoid unnecessary parsing when the flow is not done or unsuccessful

Co-authored-by: centdix <centdix@users.noreply.github.com>

* fix: use the same transaction for conversation creation

Pass transaction to get_or_create_conversation_with_id instead of
creating a new one, ensuring all operations are atomic

Co-authored-by: centdix <centdix@users.noreply.github.com>

* fix: remove update trigger and handle updated_at in application code

Remove the database trigger that automatically updates conversation
timestamp and instead update it explicitly when creating messages.
This gives better control and consistency.

Co-authored-by: centdix <centdix@users.noreply.github.com>

* Update SQLx metadata

* cleaning

* feat(aiagent): handle memory (#6719)

* implement memory

* s3 logic for memory

* fix typo

* much cleaner

* cleaning

* cleaning

* only if chat

* display nit

* nit

* fix stack overflow

* cleaning

* use len arg from input

* cleaning

* change order

* delete memory when conv deleted

* cleaning

* nit

* show description in expr mode

* opti

* opti

* updatee ref

* store string as simple string

* use markdown

* do not wait for deletion

* add delete loading

* fix logic

* fix markdown

* Update ee-repo-ref.txt

* Update SQLx metadata

* fix in test interface

* nit

* nit

* fix layout

* use memory_id to store memory

* shorter description

* rls + grant

* fix text overflow

* extract output from res

* cleaning

* handle streaming

* cleaning

* fix tool error

* nit

* update ref

* fix

* Update SQLx metadata

* nit

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: centdix <centdix@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-10-03 14:04:21 +00:00
Diego Imbert
9f40ce2da9 SelectDropdown reverse animation + ui nits (#6733)
* text-secondary nit

* Select Dropdown reverse animation

* nit misalignment
2025-10-02 20:31:05 +00:00
Diego Imbert
7cd5f26f70 Clarify Ducklake manual setup instructions (#6737) 2025-10-02 20:30:52 +00:00
Diego Imbert
f545b1d572 Fix ducklake instance DB + Add manual instructions (#6736)
* same auth method than worker for tokio_postgres

* Add manual setup instructions for Ducklake

* clarify instruction
2025-10-02 16:15:22 +00:00
Diego Imbert
a00991a293 Fix save btn disabled on deletion (#6735) 2025-10-02 14:26:12 +00:00
Diego Imbert
12a2b58704 misalignment fix (#6726) 2025-10-01 16:16:15 +00:00
Diego Imbert
72b744c4e1 feat: UX improvements (all inputs)
* Monaco transparent bg

* started improving input transform form

* always show static/f selector

* fix connecting btn changing size

* pretty Fill Inputs button

* ResizeTransitionWrapper

* Prettier TemplateEditor

* prevent double onpointerdown when clicking button to close

* text hint

* force focus border for TemplateEditor

* styling in js mode

* update select style

* fix jittery fake monaco placeholder

* select nits

* aiproviderpicker + nits

* smaller ${...} badge

* no-default-style

* select dropdown slide

* nit

* Refresh button in flow picker quick

* jsonEditor pretty

* ai provider toggle button more

* change resource edit button pos

* ResourcePicker Add and Refresh btn

* fix scrollbar

* Fix FileInput and S3 Arg Input

* fix textarea styling

* nicer refresh button in Test This Step

* fix togglebutton border in darkmode

* rounded nit

* Fix multiselect styling

* Prevent crash when selecting dyn-multiselect

* missing $derived and $state => reactivity issue when switching between DynSelect and DynMultiselect

* forgot $effect.pre

* fix nested objects

* nits

* prettier json toggle and array inputs

* array input nits

* nit

* fix json toggle appearing in fileinputs

* nit

* started updating PropertyEditor

* (stash) fix select dropdown animation teleporting from bottom to top

* nit

* nit

* resize transition in schemaform

* nit

* nit typo

* nit enableFlyTransition

* shadow nit

* small consistency changes

* user setting nit

* resize transition in module preview form

* more space

* nit readability on hover

* DateTimeInput new style

* nit fix

* remove yPadding in template and simple editor

* nits

* Revert "remove yPadding in template and simple editor"

This reverts commit 8f27c8d0b8.

* nit

* Fix proppicker border

* fix inconsistent spacing btw arginput and input transform form field headers

* consistent add item button

* nit

* s3 settings nits

* RunsFilter fix

* gray ${...} badge

* border fix darkmode

* nit

* nit app editor consistency

* fix step input gen style

* nit fix

* nits

* toggle border

* nit toggle button

* nit font-medium

* nit font-medium

* nit font-medium

* nit font-medium

* nit

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-09-27 21:52:13 +00:00
centdix
d837badf2c feat(aichat): add max tokens settings (#6613)
* add max tokens settings

* higher max

* fixes

* save max tokens in workspace settings

* cleaning

* cleaning

* cleaning

* feat(ai): add collapsible sections to ModelTokenLimits component

- Add collapsible/expandable sections for each AI provider
- Display 'Modified' badge when providers have custom settings
- Use ChevronDown/ChevronUp icons for visual feedback
- Add smooth slide transitions for better UX
- Reduce vertical space usage in workspace settings

Co-authored-by: centdix <centdix@users.noreply.github.com>

* adjust

* nit

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: centdix <centdix@users.noreply.github.com>
2025-09-17 16:55:30 +00:00
Diego Imbert
e146ac2530 Fix EE check with S3 advanced permissions (#6623)
* Advanced permission rules UI

* stash

* first iteration for s3 rule parsing

* Move to glob based approach

* cache expiry

* fix popover positioning

* ee

* unused imports

* forgot windmill_uploads

* Check S3 permissions for apps

* nit

* typo

* ee repo ref

* forgot get_workspace_s3_resource_and_check_paths in oss

* Restrict to EE
2025-09-17 13:19:56 +00:00
Diego Imbert
673b4d2a4c Advanced S3 permissionning (#6617)
* Advanced permission rules UI

* stash

* first iteration for s3 rule parsing

* Move to glob based approach

* cache expiry

* fix popover positioning

* ee

* unused imports

* forgot windmill_uploads

* Check S3 permissions for apps

* nit

* typo

* ee repo ref

* forgot get_workspace_s3_resource_and_check_paths in oss
2025-09-17 12:15:33 +00:00
Ruben Fiszel
49e687b00d internal: rolldown vite (#6584) 2025-09-11 16:12:36 +00:00
wendrul
3dadcbe865 feat: forkables workspaces v0 (#6479)
* Add create_ephemeral workspace endpoint

* Add cli devShell

* List ephemeral workspaces + improve endpoint

* Add postgres function to clone a workspace (to be revisited)

* Clone workspace using the postgres function

* Add first iteration of ephemeral workspaces command

* Update display of forked workspaces

* Remove SQLX_OFFLINE

* Add UI to create ephemeral workspace

* Add option to exclude repository from being inherited to forks

* WIP: reworking cloning logic

* Fix cloning

* Fix redirect after creating fork

* Clean up cloning behaviour

* Rename ephemeral to fork

* emove ephemeral_workspaces table in favour of columns in  workspaces

* Fix display of forked workspaces

* Fix skip inherit git sync repo setting

* Fix fork invite display + creating fork as user

* Fix SideMenu bug

* Fix alignment

* Simplify migrations

* Update deletion of workspaces

* Delete forked workspace from cli

* Deleting fork workspaces from the UI as non-admin

* Update cli sync and fork creation to adapt to branches and forks

* Update fork prefix

* Remove skip tracking toggle

* Fix npm check warnings

* Fix last npm check

* fix: force stdin to Stdio::null for all user code execution (#6575)

Set stdin to Stdio::null for all Commands that execute user code across all supported languages to prevent unwanted input consumption. This affects Python, Deno, Bash, PowerShell, Go, Rust, PHP, Ruby, Java, C#, Ansible, Nu, and Bun executors.

The dedicated worker handler was intentionally left unchanged as it requires stdin for inter-process communication.

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* Update ee-repo ref

* Update SQLx metadata

* Fix typos

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-09-10 17:42:45 +00:00
centdix
c0dbbe942c feat(flows): allow all providers for ai agent steps (#6529)
* add ui for structured output

* implement backend logic for openai models

* simulate having non required props

* cleaning

* implement logic for anthropic

* cleaning

* cleaning

* cleaning

* avoid name clash

* return object

* focus new field when adding

* fix condition

* small opti

* use box raw value

* avoid unnecessary parsing + return error if parsing fails

* update ui to show all providers

* cleaning

* adapt backend

* fix

* styling fix

* fix for anthropic with openrouter

* nit styling

* avoid race condition

* fix merge

* fix

* fix import

* make one component for whole provider selection

* fix resource logic

* rename

* clean

* remove console log

* fix bad copy
2025-09-05 08:11:23 +00:00
Diego Imbert
420be5a1f8 feat: use S3 Proxy for duckdb instead of direct connection (#6505)
* s3 proxy works with get (no auth yet)

* nit

* support s3:// syntax

* Support s3:// syntax and fix vite api proxy normalizing double slashes in URI

* s3 checks authed

* nit

* PUT works

* delete file works

* Derive the JWT signature from the backend

* Authorize s3 correctly (JWT signature is never sent in cleartext)

* convert object store error to wmill error for correct status code

* stash

* fix

* POST first request proxy works

* s3 put for duckdb

* factor out direct proxy code

* Fix Issue with backend proxy and wrong signature due to Host header mismatch

* Add _default_ syntax to solve URI normalization issues with signing

* restricted to user paths toggle

* user path restriction works !

* change restriction to allow

* fix

* factor out code

* better permissions UX in object storage settings

* Revert to restrict_to_user_paths

* check permissions in old s3 api

* DuckDB now uses S3 Proxy and no longer needs LFS query

* implement todo

* fix hardcoded w_id

* s3 proxy size limit

* s3_proxy is ee

* nit

* add Google Cloud Storage as option to secondary storage

* GCS secret in duckdb

* fix toolchain compile

* Remove user permissions for v0

* fix ci 2

* fix CI OSS

* fix missing feature flag

* fix unused warning

* integration test fails bc rustc 1.85.0

* ee ref

* fix ci ...

* update ee ref
2025-09-03 15:03:44 +00:00
centdix
7df5f5453f feat(aichat): allow custom system prompt for each mode (#6500)
* add ui to add custom system prompt

* implement backend changes

* add custom prompt in system prompt

* add in openapi

* add max length

* add backend validation
2025-09-02 09:27:37 +00:00
Alexander Petric
cb649b2689 feat: git sync v2 + cli git_branches support (#6327)
* adding branch support

* gitsync settings refactor

* promotionOverride

* require branches and initialize them

* branches -> git_branches

* profiles

* format

* format

* only warn instead of error when branches: not set in git context

* error handling

* formatting

* modal job handling improvement

* respect --yes

* logging + apply effective settings

* nit

* ui updates

* npm check

* Update frontend/src/lib/components/git_sync/PullWorkspaceModal.svelte

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* hubpaths

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-08-06 23:03:03 +00:00
Diego Imbert
e6f1211d31 feat: Ducklake native support (#6268)
* upgrade duckdb

* basic ducklake works

* ducklake works with custom db catalogs

* fix: pwsh skip already installed modules outside of cache (#6037)

* improve query performance of user stats

* separate ducklake_catalog db

* ducklake settings

* DucklakeSettings frontend

* Ducklake ws settings saved in backend

* fetch ducklake catalog resource

* Ducklake works with configured s3 storage

* Ducklake as asset

* ducklake asset icon

* Fix duckdb array and object args not working properly (#6254)

* Fix bug with comments in duckdb

* Avoid multiple queries when doing ATTACH ducklake

* trunc sig no longer needed now that comments are trimmed

* cache DuckdbConnectionSettingsResponse

* duplicated code

* transform_attach_ducklake contributes to duckdb_connection_settings_cache

* eliminate the need for used_storages

* nit

* cleaner management of the bigquery credentials file

* DBManagerDrawer refactor to prepare for Ducklake

* get ducklake schema

* implement delete for ducklake

* load column metadata for ducklake

* Select query works for ducklake, basic db explorer works !

* duckdb count query

* Support all db ops for ducklake

* clean migrations

* SQL repl for Ducklake

* fix broken database studio

* nit

* assert function

* Ducklake in Editor Bar

* default ducklake syntax + allow extra args

* DucklakeCatalogWizard UI

* nit + remove extra $

* modal when databases do not exist

* cannot be windmill

* Ducklake works safely with instance database

* Avoid sending instance db credentials on network

* resource leak security

* remove fetch_attach_db_conn_str

* prevent instance pg password leak

* hide asset usage count when not available

* case unsensitivity duckdb

* warnings

* disable instance catalog

* use shorthand syntax when inserting with EditorBar

* Instance ducklake catalog is now safe to use

* use safer argon2 pwd

* update package json parsers

* update package json

* better msgs

* tooltips

* disable explore button until saved

* nit

* fix warnings

* better ducklake_user password management

* nit

* Sanitize passwords from errors in ducklake

* DisplayResult broken in job result

* remove superadmin requirement to check databases_exist

* duckdb_connection_settings_v2_inner

* Ducklake works on agent worker (finally)

* ci

* #[allow(dead_code)]

* fix openapi missing response

* Separate +Database button for DuckDB in EditorBar

* Fix dropdown in ducklake settings

* Attempt to fix migration race condition in CI

* update sqlx failing for some offline queries

* avoid temp password for ducklake_user

* nits

* ducklake settings nits

* update duckdb default script

* fix sql repl resetting text on refresh

* avoid pgcrypto extension

---------

Co-authored-by: HugoCasa <hugo@casademont.ch>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-08-06 13:55:36 +00:00
Alexander Petric
dcc20c4fc1 git sync UI improvements (#6303)
* ui improvements round 1

* modal cleanup

* init

* UI refactor

* UI cleanup + refactor

* legacy cleanup

* success model -> github actions, non-ee warnings

* sqlx

* npm check

* ee warning everywhere

* last comments

* formatting

* no hardcoded theme

* claude review improvemenets
2025-07-30 21:36:32 +00:00
claude[bot]
16757f3bb5 nit(aichat): recommend anthropic as provider (#6265)
* feat(aichat): add recommendation alert for Claude latest model in Anthropic provider settings

Add info alert in AI workspace settings recommending Claude latest model for better reliability of AI chat when using Anthropic provider.

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

* fix

* nit

* nit

* use badge

* use blue

* add tooltip

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
Co-authored-by: centdix <farhadg110@gmail.com>
2025-07-23 15:14:51 +00:00
Alexander Petric
aa37f643e7 feat: git sync improvements (#6182)
* init checkpoint

* ui second pass...

* round 1 backend + saving settings + detecting changes...

* checkpoint

* fix openapi

* saving + correct wmill.yaml diff

* cli refactor

* cli and tests refactor done

* cli multi workspace support

* cli support skip core types to align with ui

* new test framework

* sqlx

* openapi spec

* frontend

* sync + settings changes

* some fixes

* some fixes

* security: Remove hardcoded EE license key, use environment variable only

- Remove hardcoded license key from containerized test backend
- Environment variable EE_LICENSE_KEY now required for EE features
- License key no longer stored in database during tests

* sqlx

* tests

* fixing tests

* fix tests

* checkpoint

* checkpoint

* cli build

* frontend - cli exchange

* settings match

* ee repo ref

* npm check

* openapi

* tests

* checkpoint

* cli + tests

* reset to preview on changes

* merge issue ee

* cleanup

* hubscript

* simplifications

* ee repo ref

* cli fixes

* fix sync and add tests

* extra test

* git sync settings / key change aware

* ee-repo ref

* ee-repo ref

* ee repo ref

* ee ref

* review 1

* ee ref

* Update frontend/src/lib/components/PullGitRepoPopover.svelte

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update frontend/src/routes/(root)/(logged)/workspace_settings/+page.svelte

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* ee ref

* remove extra includes from ui

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-07-15 16:25:31 +00:00
HugoCasa
328ef605ad feat: triggers error handler and retry (#6138)
* feat: http triggers error handling and retries

* feat: handle sync http triggers

* feat: triggers retry and error handling

* feat: frontend and preprocessor

* fix build

* minimum allow dead code

* check workspace tags in schedules

* fix flow preprocesor + hub urls

* update ee ref

* nit

* nit

* final nits

* schedule UI nits

* migrate workspace settings to svelte 5
2025-07-11 18:55:23 +02:00
Ruben Fiszel
c2a09fb9de resource picker nits 2025-07-09 09:26:34 +00:00
windmill-internal-app[bot]
c51e128920 fix: add support for GCS object storage (#6083)
* AI: Updates to files (run 15923369101)

* all

* ee ref

* unneeded

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: HugoCasa <hugo@casademont.ch>
2025-06-30 20:11:57 +00:00
Ruben Fiszel
467b21c223 nit fix 2025-06-27 21:58:13 +02:00
HugoCasa
431437c344 feat: use FIM for code autocomplete (#6081)
* feat: use FIM for code autocomplete

* nits
2025-06-27 08:09:09 +00:00
Diego Imbert
fa8d1b47db fix: new MultiSelect component (#5979)
* rename all multiselects to Legacy

* move Select

* Separate SelectDropdown

* fix click outside for select with portal

* multiselect v0

* multiselect clear btn

* move filterText logic to SelectDropdown

* ui nits

* console.log

* draggable

* Draggable multiselect

* multiselect search

* nit refacto

* autofocus multiselect input

* Replace in AppMultiSelectV2

* search icon

* app multi select nits

* arginput update multiselect

* fix autofocus scrolling up

* replace High priority tags multiselect

* autoscaling config editor multiselect replace

* fix clear btn not in border

* replace multiselect in cron input

* replace multiselect in savedinputs

* replace EventHandlerItem multiselect

* select dropdown shadow

* more multiselect migration

* hover opacity on drag

* TokensTable UI fixes + replace multiselect

* ai settings replace multiselect

* DefaultTags Multiselect replace

* prevent multiselect from opening on drag

* nit

* app multiselect css + simplify

* console log

* safeSelectItems cleanup

* Remove svelte-multiselect

* clip when wrap not allowed

* hide duplicate app components

* CSS works better  with multiselect in app editor

* allowOverflow

* allowClear

* fix custom createText messed up with search
2025-06-19 14:37:17 +02:00
centdix
7490e883d7 feat: use provider api to list available AI models in workspace settings (#5947)
* use open router of model lists

* draft

* allow get in ai proxy

* add fetch available models function

* use func

* fix for anthropic

* fix

* fetch on mount

* fix ai settings

* fix

* handle azure
2025-06-17 08:48:47 +00:00