Commit Graph

10021 Commits

Author SHA1 Message Date
Diego Imbert
e6f1211d31 feat: Ducklake native support (#6268)
* upgrade duckdb

* basic ducklake works

* ducklake works with custom db catalogs

* fix: pwsh skip already installed modules outside of cache (#6037)

* improve query performance of user stats

* separate ducklake_catalog db

* ducklake settings

* DucklakeSettings frontend

* Ducklake ws settings saved in backend

* fetch ducklake catalog resource

* Ducklake works with configured s3 storage

* Ducklake as asset

* ducklake asset icon

* Fix duckdb array and object args not working properly (#6254)

* Fix bug with comments in duckdb

* Avoid multiple queries when doing ATTACH ducklake

* trunc sig no longer needed now that comments are trimmed

* cache DuckdbConnectionSettingsResponse

* duplicated code

* transform_attach_ducklake contributes to duckdb_connection_settings_cache

* eliminate the need for used_storages

* nit

* cleaner management of the bigquery credentials file

* DBManagerDrawer refactor to prepare for Ducklake

* get ducklake schema

* implement delete for ducklake

* load column metadata for ducklake

* Select query works for ducklake, basic db explorer works !

* duckdb count query

* Support all db ops for ducklake

* clean migrations

* SQL repl for Ducklake

* fix broken database studio

* nit

* assert function

* Ducklake in Editor Bar

* default ducklake syntax + allow extra args

* DucklakeCatalogWizard UI

* nit + remove extra $

* modal when databases do not exist

* cannot be windmill

* Ducklake works safely with instance database

* Avoid sending instance db credentials on network

* resource leak security

* remove fetch_attach_db_conn_str

* prevent instance pg password leak

* hide asset usage count when not available

* case unsensitivity duckdb

* warnings

* disable instance catalog

* use shorthand syntax when inserting with EditorBar

* Instance ducklake catalog is now safe to use

* use safer argon2 pwd

* update package json parsers

* update package json

* better msgs

* tooltips

* disable explore button until saved

* nit

* fix warnings

* better ducklake_user password management

* nit

* Sanitize passwords from errors in ducklake

* DisplayResult broken in job result

* remove superadmin requirement to check databases_exist

* duckdb_connection_settings_v2_inner

* Ducklake works on agent worker (finally)

* ci

* #[allow(dead_code)]

* fix openapi missing response

* Separate +Database button for DuckDB in EditorBar

* Fix dropdown in ducklake settings

* Attempt to fix migration race condition in CI

* update sqlx failing for some offline queries

* avoid temp password for ducklake_user

* nits

* ducklake settings nits

* update duckdb default script

* fix sql repl resetting text on refresh

* avoid pgcrypto extension

---------

Co-authored-by: HugoCasa <hugo@casademont.ch>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-08-06 13:55:36 +00:00
Ruben Fiszel
24febaead3 nit 2025-08-06 07:31:54 +00:00
dieriba
1bcb0431c4 feat: dynamic select in flow (#6315)
* update

* add dyn select for flows

* nits

* fix typo

* done

* update .sqlx

* better

* nits

* fix

* nits

* nits

* fix

* improvements

* update

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-08-06 07:20:03 +00:00
centdix
69c2a7c1c8 chore(cli): better folder structure + add config utils (#6319)
* organize in folders

* add config command

* fix

* cleaning

* move utility functions

* merge

* only show token with option

* only show token with option

* fix

* remove config command

* add config utils

* change paths

* nit

* clean path assigner

---------

Co-authored-by: Alexander Petric <alpetric@users.noreply.github.com>
2025-08-05 13:59:07 +00:00
pyranota
83aa0d0267 fix(rust-sdk): revert openapi-generator-cli to 7.10.0 (#6325)
In > 7.10.0 regression was introduced breaking our build process.

This commit reverts the change and fixes tests

Signed-off-by: pyranota <pyra@duck.com>
2025-08-05 11:08:16 +00:00
Ruben Fiszel
254c6eda9f chore(main): release 1.518.2 (#6323)
* chore(main): release 1.518.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
v1.518.2
2025-08-05 07:30:34 +00:00
Ruben Fiszel
c351b7ba87 fix: fix edit button for script in flow module 2025-08-05 07:22:30 +00:00
Ruben Fiszel
86df8ce240 fix: github apps popup appear for superadmins 2025-08-04 22:53:54 +00:00
Ruben Fiszel
5e8919483b chore(main): release 1.518.1 (#6322)
* chore(main): release 1.518.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
v1.518.1
2025-08-04 22:17:08 +00:00
centdix
82ac747791 fix(cli): revert renaming of inline scripts (#6320)
* use old path assigner

* fix

* fix

* fix
2025-08-04 22:13:52 +00:00
Ruben Fiszel
4fbf36d7d3 chore(main): release 1.518.0 (#6317)
* chore(main): release 1.518.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
v1.518.0
2025-08-02 14:08:00 +00:00
Ruben Fiszel
0903d7c733 fix checks 2025-08-01 21:56:06 +00:00
Alexander Petric
8d05696809 fix: delete empty git connection (#6318) 2025-08-01 16:35:40 +00:00
Ruben Fiszel
523bc2023b fix: sanitize XSS on login error 2025-08-01 16:24:18 +00:00
Ruben Fiszel
d45ddecf8f fix: add wm_labels to tracing spans 2025-08-01 12:24:00 +00:00
Ruben Fiszel
c279154409 fix: add disabled support to resource picker in schema forms 2025-08-01 11:54:56 +00:00
Ruben Fiszel
87dd52296f fix: cleanup concurrency_counter automatically + remove orphans keys automatically 2025-08-01 11:26:26 +00:00
centdix
493707668b feat(utils): add flow.yaml validation function (#6316)
* add validateflow function

* cleaner code

* preprocess json

* cleaning

* create specific package

* cleaning

* add tests
2025-08-01 11:14:39 +00:00
Ruben Fiszel
6ea2029265 cleanup 2025-08-01 05:27:54 +00:00
centdix
1af671246d fix typo on cli build (#6314) 2025-07-31 18:41:14 +00:00
Ruben Fiszel
1cb7653651 chore(main): release 1.517.0 (#6310)
* chore(main): release 1.517.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
v1.517.0
2025-07-31 18:13:54 +00:00
wendrul
8887707d41 fix: use with_capacity back presusre for tantivy directory multipart writes (#6313)
* use with capacity for tantivy directory multi part uploads

* Update ee repo ref

* Update ee-repo-ref

* Update ee-repo-ref
2025-07-31 18:09:27 +00:00
centdix
dd379d0a69 fix styling on aichatinput (#6312) 2025-07-31 16:52:37 +00:00
Diego Imbert
2ed2ff1c03 DisplayResult broken in job result (#6311) 2025-07-31 16:50:48 +00:00
Ruben Fiszel
ef3e2353a7 feat(cli): wmill-lock.yaml v2 for easier git merge diffs 2025-07-31 14:23:38 +00:00
wendrul
205674a72a Add dependencies to run sqlx prepare to nix flake (#6309) 2025-07-31 13:23:52 +00:00
centdix
17ae1ee160 chore: use windmill-utils-internal for cli (#6297)
* add utils package

* naming

* cleaning

* simplify assignPath

* rename old files

* same for locks

* create on confirm

* default true

* use replaceinlinescripts from utils

* use extractscriptfromflows

* make it compile

* cleaning

* use argsigtojson

* fix

* fix missing await

* cleaner

* cleaning

* cleaning

* use in frontend

* add docs

* testing

* remove log

* use autogenerated types

* remove old

* fix

* cleaning

* adapt usage

* draft

* better build script

* fix build

* revert to default creation

* add docs

* remove and rename

* make everything work

* add await

* only if not installed

* add vs code setting

* add to publish action

* fix bc

* safer use of sep

* fix

* do not rename on push

* no publish on release

* use published package on frontend

* nit
2025-07-31 13:23:39 +00:00
Ruben Fiszel
18912e6bab indexer improvements 2025-07-31 12:35:08 +00:00
Ruben Fiszel
656482b973 improve indexer 2025-07-31 12:10:05 +00:00
Ruben Fiszel
74f41ab53b fix oss build 2025-07-31 09:08:55 +00:00
Ruben Fiszel
2b8860d2b0 nit warnings 2025-07-31 08:56:10 +00:00
Ruben Fiszel
30394dffbf use tantivy fork 2025-07-31 08:44:16 +00:00
Ruben Fiszel
5c58abc864 upgrade tantivy to 0.24.2 2025-07-31 08:29:28 +00:00
Ruben Fiszel
d81e190cf9 indexer improvements 2025-07-31 07:40:54 +00:00
Ruben Fiszel
d71811d384 put size limits of job in query 2025-07-31 01:11:09 +00:00
Ruben Fiszel
9116205a1c chore(main): release 1.516.0 (#6298)
* chore(main): release 1.516.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
v1.516.0
2025-07-31 00:15:05 +00:00
centdix
7f56651b13 chore: add windmill-utils-internal package (#6299)
* add utils package

* naming

* cleaning

* add docs

* remove log

* use autogenerated types

* remove old

* fix

* cleaning

* add docs
2025-07-31 00:10:58 +00:00
Ruben Fiszel
a8e3b5e595 nits 2025-07-30 23:43:46 +00:00
Ruben Fiszel
ff112e408e handle_deployment_metadata in a task 2025-07-30 23:30:33 +00:00
Ruben Fiszel
faa5e65d7e handle metadata for new scripts happen after commit 2025-07-30 23:20:22 +00:00
dieriba
993e80955b fix: sqs oidc authentication disconnect #6307 2025-07-30 23:15:33 +00:00
Ruben Fiszel
576156b0cc fix: no process relative imports for scripts with codebase 2025-07-30 22:50:22 +00:00
Alexander Petric
dcc20c4fc1 git sync UI improvements (#6303)
* ui improvements round 1

* modal cleanup

* init

* UI refactor

* UI cleanup + refactor

* legacy cleanup

* success model -> github actions, non-ee warnings

* sqlx

* npm check

* ee warning everywhere

* last comments

* formatting

* no hardcoded theme

* claude review improvemenets
2025-07-30 21:36:32 +00:00
Diego Imbert
b04a203e9a default to false if field not present operator settings (#6301) 2025-07-30 20:47:57 +00:00
Diego Imbert
cc97347d74 fix broken audit logs filter (#6304)
* rename to from to

* goto fix
2025-07-30 20:46:59 +00:00
pyranota
af73244b44 fix rust sdk build error (#6305)
Signed-off-by: pyranota <pyra@duck.com>
2025-07-30 18:13:11 +00:00
wendrul
77c8f17fdf fix: indexer collection of job logs before indexing (#6300)
* Add flume as dependecy for indexer

* Update ee-repo-ref

* Remove flags from cargo.toml

* Update ee-repo-ref

* Update ee-repo-ref
2025-07-30 15:31:07 +00:00
claude[bot]
a460e131c7 feat: add CA certificate update at startup via environment variable (#6280)
* feat: add CA certificate update at startup via environment variable

Add support for running 'update-ca-certificates' at binary startup
when RUN_UPDATE_CA_CERTIFICATE_AT_START environment variable is set to "true".

- Check for RUN_UPDATE_CA_CERTIFICATE_AT_START env var on startup
- Execute update-ca-certificates command if env var is set to "true"
- Log success/failure appropriately with tracing
- Continue startup even if CA certificate update fails
- Non-blocking implementation with proper error handling

Fixes #6279

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

* refactor: extract CA certificate update logic into separate function

Extract the CA certificate update logic from windmill_main() into a
dedicated update_ca_certificates_if_requested() function for better
code organization and maintainability.

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

* improvements

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
Co-authored-by: Alexander Petric <alpetric@users.noreply.github.com>
Co-authored-by: Alexander Petric <alex@windmill.dev>
2025-07-30 14:39:26 +00:00
Ruben Fiszel
25aa982318 nit naming 2025-07-30 12:58:33 +00:00
Ruben Fiszel
4b9683f146 feat: prevent too large results (>500Mb) from OOMing database 2025-07-30 12:55:39 +00:00