Ruben Fiszel
d9d58a2060
extend oauth refresh account size
2025-11-29 11:28:51 +00:00
Pyra
d36760bff7
feat: workspace dependencies ( #7124 )
...
* commit raw requirements
Signed-off-by: pyranota <pyra@duck.com >
* raw requirements
Signed-off-by: pyranota <pyra@duck.com >
* implement `parse_annotation`
Signed-off-by: pyranota <pyra@duck.com >
* more progress on wdeps
Signed-off-by: pyranota <pyra@duck.com >
* more progress
Signed-off-by: pyranota <pyra@duck.com >
* fixes
Signed-off-by: pyranota <pyra@duck.com >
* more progress
Signed-off-by: pyranota <pyra@duck.com >
* fixes
Signed-off-by: pyranota <pyra@duck.com >
* cli improvements + raw deps
Signed-off-by: pyranota <pyra@duck.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* fix python versions
Signed-off-by: pyranota <pyra@duck.com >
* progress
Signed-off-by: pyranota <pyra@duck.com >
* update :)
Signed-off-by: pyranota <pyra@duck.com >
* add MaybeLock
Signed-off-by: pyranota <pyra@duck.com >
* go WIP
Signed-off-by: pyranota <pyra@duck.com >
* fix python ignoring py version from requirements
Signed-off-by: pyranota <pyra@duck.com >
* optimize php
Signed-off-by: pyranota <pyra@duck.com >
* require admin to alter
Signed-off-by: pyranota <pyra@duck.com >
* fix(cli): flow generateLocks raw deps
Signed-off-by: pyranota <pyra@duck.com >
* progress in checklist
Signed-off-by: pyranota <pyra@duck.com >
* fix agent workers
Signed-off-by: pyranota <pyra@duck.com >
* nits
Signed-off-by: pyranota <pyra@duck.com >
* nits
Signed-off-by: pyranota <pyra@duck.com >
* nit: remove default features
Signed-off-by: pyranota <pyra@duck.com >
* oh-wow
Signed-off-by: pyranota <pyra@duck.com >
* remove dbg!
Signed-off-by: pyranota <pyra@duck.com >
* nits
Signed-off-by: pyranota <pyra@duck.com >
* add indexes
Signed-off-by: pyranota <pyra@duck.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* nits
Signed-off-by: pyranota <pyra@duck.com >
* remove todos
Signed-off-by: pyranota <pyra@duck.com >
* fix cli
Signed-off-by: pyranota <pyra@duck.com >
* add debug flag
Signed-off-by: pyranota <pyra@duck.com >
* cli: remove noise
Signed-off-by: pyranota <pyra@duck.com >
* fix cli
Signed-off-by: pyranota <pyra@duck.com >
* remove todos
Signed-off-by: pyranota <pyra@duck.com >
* trigger deps correctly
Signed-off-by: pyranota <pyra@duck.com >
* fix frontend
Signed-off-by: pyranota <pyra@duck.com >
* fix frontend again
Signed-off-by: pyranota <pyra@duck.com >
* finally fix frontend
Signed-off-by: pyranota <pyra@duck.com >
* ee repo ref
Signed-off-by: pyranota <pyra@duck.com >
* fix all
Signed-off-by: pyranota <pyra@duck.com >
* more fixes...
Signed-off-by: pyranota <pyra@duck.com >
* remove test
Signed-off-by: pyranota <pyra@duck.com >
* Update backend-test.yml
* comment out legacy test
Signed-off-by: pyranota <pyra@duck.com >
* fix ci
Signed-off-by: pyranota <pyra@duck.com >
* fix ci?
Signed-off-by: pyranota <pyra@duck.com >
* comment out thing
Signed-off-by: pyranota <pyra@duck.com >
* ignore test
Signed-off-by: pyranota <pyra@duck.com >
* ci
Signed-off-by: pyranota <pyra@duck.com >
* base fixture
Signed-off-by: pyranota <pyra@duck.com >
* fix regression
Signed-off-by: pyranota <pyra@duck.com >
* fix docs links
Signed-off-by: pyranota <pyra@duck.com >
* update min version
Signed-off-by: pyranota <pyra@duck.com >
* simplify
* implement cache for get_latest
Signed-off-by: pyranota <pyra@duck.com >
* move to workspace settings
Signed-off-by: pyranota <pyra@duck.com >
* sqlx + migration
Signed-off-by: pyranota <pyra@duck.com >
* more migrations
Signed-off-by: pyranota <pyra@duck.com >
* use box pin
Signed-off-by: pyranota <pyra@duck.com >
* nit
Signed-off-by: pyranota <pyra@duck.com >
---------
Signed-off-by: pyranota <pyra@duck.com >
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
2025-11-28 16:00:11 +00:00
Alexander Petric
1fe463cf94
feat(security): unshare pid of worker job process ( #7106 )
...
* feat(security): unbind pid for worker jobs to create extra process isolation
* review
* simplify
* cleanup + compose files
* re-add removed comments from nu executor
* simplify
* fail immediately
* updates
* update ping backend
* nsjail / unshare in workers page
* migrations
* frontend + sqlx
* frontend
* frontend
* fix error message
* undo example changes
2025-11-18 23:04:31 +00:00
hugocasa
8ed55897f3
count external jwts for telemetry
...
* feat: count external jwts
* nits
2025-11-18 23:03:35 +00:00
Ruben Fiszel
9e897dd693
fix: fix custom email triggers enabled ( #7164 )
2025-11-17 22:26:47 +00:00
dieriba
f982e990da
feat: allow http trigger to be disabled ( #6976 )
...
* done
* update sqlx
* fix
* fix openapi spec
* fix
* Update backend/windmill-api/openapi.yaml
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* nits
* nits
* fix
* fix
* update sqlx
* unneeded
* ok
* update sqlx
---------
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-11-17 08:51:35 +00:00
Diego Imbert
6ecd4954b2
SQL result_collection flag ( #7041 )
...
* Return array of results duckdb
* Migration script to add result_collection=legacy
* migration script fixes
* app_version_lite not necessary
* New annotations macro that supports custom types
* pass unit tests
* nit style
* result_collection almost works for postgres
* fix last_statement
* frontend suggesitons
* fix column_order making columns disappear
* added version check for duckdb FFI lib to avoid crashes when changing FFI interface on Windows agent workers
* result_collection for duckdb
* Correct legacy behavior in DuckDB
* mysql result_collection
* mssql collection_strategy
* result_collection for oracle
* snowflake result_collection
* fix errors
* mistake, .clone() causing deadlock
* fix frontend errors on oracle / mssql
* fix legacy behavior on mssql
* add exception catch in migration
* missing app_script update
* change cache layout to avoid worker pulling old scripts
2025-11-12 15:32:07 +00:00
centdix
3ae4b2dd0a
chore(aiagent): store memory in db ( #7053 )
...
* store to db
* no warning in context
* warning in frontend
* remove log
* 100kb + alert
* update sqlx
* update eeref
* Update ee-repo-ref.txt
* cleaning
---------
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-11-07 15:27:00 +00:00
Alexander Petric
665a3b82a9
feat: slack app on workspace level ( #6992 )
...
* feat: slack app on workspace level
* frontend
* ee repo ref
* Update SQLx metadata
* repo ref
* ee repo ref
* ee repo ref
* sqlx
* ee repo ref
* ee ref + sqlx
---------
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-30 16:24:44 +00:00
Pyra
38c868e5a9
null debounce_delay_s if 0 ( #7013 )
...
Signed-off-by: pyranota <pyra@duck.com >
2025-10-30 16:21:08 +00:00
Ruben Fiszel
e9dc9960fa
fix: add grant all to concurrency_counter
2025-10-28 10:17:47 +00:00
hugocasa
df2a0f8843
chore: update cli version in hub sync ( #6950 )
2025-10-27 15:49:43 +00:00
Pyra
00953e3a3f
fix(debouncing): fix perf issues and re-enable debouncing ( #6932 )
...
* fix(debouncing): fix perf issues and re-enable debouncing
Signed-off-by: pyranota <pyra@duck.com >
* add debounce data in clone_script
Signed-off-by: pyranota <pyra@duck.com >
* update sqlx cache
Signed-off-by: pyranota <pyra@duck.com >
---------
Signed-off-by: pyranota <pyra@duck.com >
2025-10-24 15:24:46 +00:00
Pyra
242a57f840
feat: job debouncing ( #6878 )
...
* feat(EE): job debouncing
Signed-off-by: pyranota <pyra@duck.com >
* remove 'script' file
Signed-off-by: pyranota <pyra@duck.com >
* more work
Signed-off-by: pyranota <pyra@duck.com >
* properly gate it behind enterprise
Signed-off-by: pyranota <pyra@duck.com >
* update ee repo ref
Signed-off-by: pyranota <pyra@duck.com >
* change ee repo ref again
Signed-off-by: pyranota <pyra@duck.com >
* remove unused variable
Signed-off-by: pyranota <pyra@duck.com >
* feat(EE): implement TODOs and enhance tracing for job debouncing
- Add database index on script(workspace_id, debounce_key) for efficient lookups
- Update minimum version requirement to 1.564.0 throughout codebase
- Add tracing warnings when debouncing is disabled due to worker version mismatch
- Fix all documentation links from TODO placeholders to proper URLs
- Replace Gauge icon with Timer icon for debouncing UI elements
- Update placeholder text and tooltips with clear descriptions
Co-authored-by: Pyra <pyranota@users.noreply.github.com >
* create -> crate
Signed-off-by: pyranota <pyra@duck.com >
* remove index
Signed-off-by: pyranota <pyra@duck.com >
* some updates
Signed-off-by: pyranota <pyra@duck.com >
* fix once more
Signed-off-by: pyranota <pyra@duck.com >
* fix it once more
Signed-off-by: pyranota <pyra@duck.com >
* Remove flow step debouncing, keep top-level flow debouncing
- Remove debounce fields from RawScript and FlowScript FlowModuleValue variants
- Remove debounce fields from JobPayload::FlowScript and RawCode
- Update raw_script_to_payload function signature
- Remove debouncing UI from flow step runtime settings
- Remove debouncing toggle handler and indicator badge
- Preserve top-level flow debouncing in FlowSettings
Co-authored-by: Pyra <pyranota@users.noreply.github.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* fixup claude's work
Signed-off-by: pyranota <pyra@duck.com >
* cleanup: remove dbg! statements, update min version to 1.566.0, add comprehensive comments
- Removed all dbg! macro calls from production code
- Updated MIN_VERSION_SUPPORTS_DEBOUNCING from 1.564.0 to 1.566.0
- Added comprehensive documentation comments explaining:
- Debouncing feature purpose and mechanics
- Database schema for debounce_key and debounce_stale_data tables
- Version check logic and guard functions
- Improved code clarity and maintainability
Co-authored-by: Pyra <pyranota@users.noreply.github.com >
* improve fallback
Signed-off-by: pyranota <pyra@duck.com >
* remove comments from old migration
Signed-off-by: pyranota <pyra@duck.com >
* fix pull
Signed-off-by: pyranota <pyra@duck.com >
* fix once more
Signed-off-by: pyranota <pyra@duck.com >
* Update frontend/src/lib/components/ScriptBuilder.svelte
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
* add sqlx cache
Signed-off-by: pyranota <pyra@duck.com >
* fix: frontend CI - fix debounce field names and remove leftover flow step debouncing
- Fix ScriptBuilder.svelte: change custom_debounce_key to debounce_key
- Add debounce_key and debounce_delay_s fields to NewScript schema in openapi.yaml
- Regenerate frontend types from OpenAPI spec
- Remove leftover flow step debouncing code from FlowModuleComponent.svelte
- Remove debounce fields from RawScript in openflow.openapi.yaml
- Remove unused Timer import from FlowModuleHeader.svelte
All frontend checks now passing (0 errors, 0 warnings)
Co-authored-by: Pyra <pyranota@users.noreply.github.com >
* fix ci
Signed-off-by: pyranota <pyra@duck.com >
* remove unused import
Signed-off-by: pyranota <pyra@duck.com >
* fix ci again
Signed-off-by: pyranota <pyra@duck.com >
* udpate ee repo ref
Signed-off-by: pyranota <pyra@duck.com >
* CI doesn't want to be fixed but I still try
Signed-off-by: pyranota <pyra@duck.com >
* nits
Signed-off-by: pyranota <pyra@duck.com >
* ci...
Signed-off-by: pyranota <pyra@duck.com >
* Update ee-repo-ref.txt
* safer migration
Signed-off-by: pyranota <pyra@duck.com >
* reduce noise in logs
Signed-off-by: pyranota <pyra@duck.com >
* fix cli for scripts
Signed-off-by: pyranota <pyra@duck.com >
* nit
Signed-off-by: pyranota <pyra@duck.com >
---------
Signed-off-by: pyranota <pyra@duck.com >
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Pyra <pyranota@users.noreply.github.com >
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-23 20:18:08 +00:00
hugocasa
4c74211487
feat: http routes streaming ( #6834 )
...
* feat: http routes streaming
* improve webhook config section for SSE
* feat: add request type config to http routes
* fix check
* sqlx
---------
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
2025-10-17 14:05:17 +00:00
Ruben Fiszel
1343109420
fix select grant
2025-10-17 13:59:58 +00:00
Ruben Fiszel
08e1807dfe
fix: add grant select on debounce_key
2025-10-17 13:55:02 +00:00
Ruben Fiszel
b702c91135
fix migration error
2025-10-17 03:42:41 +00:00
Diego Imbert
8536a4c64e
create ducklake_user_pg_pwd if deleted by CLI ( #6845 )
2025-10-16 21:54:00 +00:00
Ruben Fiszel
691ef6babd
remove safety migration
2025-10-16 21:29:44 +00:00
Diego Imbert
b779cb7e02
ducklake safety for instance_settings.yaml users ( #6844 )
2025-10-16 20:39:11 +00:00
Pyra
4674191e90
feat: dependency job debouncing ( #6769 )
...
* v0
Signed-off-by: pyranota <pyra@duck.com >
* optimize relocks
* make it work with relative relative imports
Signed-off-by: pyranota <pyra@duck.com >
* use fallback
Signed-off-by: pyranota <pyra@duck.com >
* remove dbg and todos
Signed-off-by: pyranota <pyra@duck.com >
* future proof a bit
Signed-off-by: pyranota <pyra@duck.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* more cleanup
Signed-off-by: pyranota <pyra@duck.com >
* remove final TODO
Signed-off-by: pyranota <pyra@duck.com >
* do not use bytemuck
Signed-off-by: pyranota <pyra@duck.com >
* optimize hashing
Signed-off-by: pyranota <pyra@duck.com >
* implementation 1
Signed-off-by: pyranota <pyra@duck.com >
* almost v0
Signed-off-by: pyranota <pyra@duck.com >
* v0
Signed-off-by: pyranota <pyra@duck.com >
* add comments and use fallback
Signed-off-by: pyranota <pyra@duck.com >
* call dissolve for apps
Signed-off-by: pyranota <pyra@duck.com >
* add comms
Signed-off-by: pyranota <pyra@duck.com >
* refactor v0 (partially tested + dirty)
Signed-off-by: pyranota <pyra@duck.com >
* finishing
Signed-off-by: pyranota <pyra@duck.com >
* remove TODO
Signed-off-by: pyranota <pyra@duck.com >
* Update SQLx metadata
* silence unused argument
Signed-off-by: pyranota <pyra@duck.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* implement rebuild_map endpoint
Signed-off-by: pyranota <pyra@duck.com >
* update windmill api client
Signed-off-by: pyranota <pyra@duck.com >
* almost finish with tests
Signed-off-by: pyranota <pyra@duck.com >
* add proper testing
Signed-off-by: pyranota <pyra@duck.com >
* remove unused fixtures
Signed-off-by: pyranota <pyra@duck.com >
* Update SQLx metadata
* partial cleanup
Signed-off-by: pyranota <pyra@duck.com >
* Update backend/windmill-worker/src/scoped_dependency_map.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* Update backend/windmill-common/src/scripts.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* evil doings
Signed-off-by: pyranota <pyra@duck.com >
* more cleanup
* Update SQLx metadata
* more cleanup
Signed-off-by: pyranota <pyra@duck.com >
* fixing CI
Signed-off-by: pyranota <pyra@duck.com >
* remove python from default features
Signed-off-by: pyranota <pyra@duck.com >
* feat: dependency job debouncing
* checkpoint
Signed-off-by: pyranota <pyra@duck.com >
* more improvements
Signed-off-by: pyranota <pyra@duck.com >
* refactor: clean up dependency job debouncing implementation
- Add comprehensive comments explaining the debouncing mechanism
- Replace debug statements (dbg!) with proper tracing calls
- Extract helper functions to reduce code duplication:
- extract_to_relock_from_args() for extracting nodes/components
- accumulate_debounce_stale_data() for updating stale data
- Improve code readability and maintainability
Co-authored-by: Pyra <pyranota@users.noreply.github.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* Update SQLx metadata
* Update backend/windmill-common/src/jobs.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* Update backend/windmill-common/src/jobs.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* Update backend/windmill-queue/src/jobs.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* better error handling for helper
Signed-off-by: pyranota <pyra@duck.com >
* update ee-ref
Signed-off-by: pyranota <pyra@duck.com >
* test: add race condition test for dependency job debouncing
- Implement test_2 for scripts to test the race condition edge case
- Add comprehensive documentation comments to all test functions
- Remove empty test_2 stubs for flows (not needed)
- Keep test_2 stub for apps with TODO comment
The race condition test simulates the scenario where a job is marked
as running but debounce_key hasn't been cleaned up yet, forcing the
system to create a new job while reusing the existing debounce_key.
This edge case can occur due to the lack of transactions in the pull
function for performance reasons (see jobs.rs:4415-4425).
Co-authored-by: Pyra <pyranota@users.noreply.github.com >
* test: implement race condition test for dependency job debouncing
- Add comprehensive test_2 for script module that tests the race condition edge case
- Remove empty test_2 stubs from flows and apps modules
- Fix unused variable warning in worker_lockfiles.rs
- Add detailed comments explaining the race condition scenario and test logic
Co-authored-by: Pyra <pyranota@users.noreply.github.com >
* implement fallback
Signed-off-by: pyranota <pyra@duck.com >
* make it mostly work
* all tests are almost working
Signed-off-by: pyranota <pyra@duck.com >
* add comments a bit
Signed-off-by: pyranota <pyra@duck.com >
* Update backend/windmill-common/src/scripts.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* Update backend/windmill-worker/src/worker_lockfiles.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* Update backend/windmill-worker/src/worker_lockfiles.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* feat: improve debouncing documentation, tracing, and error handling
- Add comprehensive 3-phase debouncing documentation explaining push/accumulation/pull
- Enhance all tracing with structured logging (job_id, workspace_id, node_count, etc.)
- Add proper error handling with .map_err() and contextual messages
- Replace dbg!() with proper tracing::debug!()
- Replace todo!() with proper error handling
- Fix typos: 'and edge case' → 'an edge case', 'bc' → 'because'
- Fix debug variable name: 'debounce_job_id_0' → 'debounce_job_id_o'
- Add documentation for debounce cleanup and stale data retrieval
- Add trace-level logging for non-error paths to reduce noise
Co-authored-by: Pyra <pyranota@users.noreply.github.com >
* do some work for future improvements
Signed-off-by: pyranota <pyra@duck.com >
* fix tests
Signed-off-by: pyranota <pyra@duck.com >
* clippy
Signed-off-by: pyranota <pyra@duck.com >
* update sqlx
Signed-off-by: pyranota <pyra@duck.com >
* clippy
Signed-off-by: pyranota <pyra@duck.com >
* update ee ref
Signed-off-by: pyranota <pyra@duck.com >
* flag tests behind the feature, add timeout
Signed-off-by: pyranota <pyra@duck.com >
* fix timeout + cleanup
Signed-off-by: pyranota <pyra@duck.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* row lock debounce_key
Signed-off-by: pyranota <pyra@duck.com >
* addressing TODOs
Signed-off-by: pyranota <pyra@duck.com >
* fix test
Signed-off-by: pyranota <pyra@duck.com >
* ee ref
Signed-off-by: pyranota <pyra@duck.com >
* ee repo
Signed-off-by: pyranota <pyra@duck.com >
---------
Signed-off-by: pyranota <pyra@duck.com >
Co-authored-by: Pyra <92104930+pyranye@users.noreply.github.com >
Co-authored-by: GitHub Action <action@github.com >
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Pyra <pyranota@users.noreply.github.com >
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
2025-10-16 16:49:35 +00:00
Alexander Petric
922704ea12
fix: always create instance groups with uuid ( #6826 )
...
* fix: always create instance groups with uuid
* Update SQLx metadata
* repo ref
---------
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-15 22:30:09 +00:00
centdix
2218b41ced
feat(flow): show tool usage in flow conversations ( #6771 )
...
* save each assitant and tool message
* fix result assistant message logic
* nit
* poll conversation messages
* styling and backend nits
* stream tools
* fix tools streaming
* cleaning
* cleaning + handle tool errors
* flag flow errors
* cleaning
* use separate manager
* cleaning
* fix refresh
* cleaning
* cleaning
* Update SQLx metadata
* fix migration
* reuse memory id + add timeout to polling
* cleaning
* no error if stream complete
* fix memory for nested agents
* fix chat enabled for nested agents
* cleaning
* Update SQLx metadata
* nit
---------
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-09 05:07:18 +00:00
Alexander Petric
82c83ff658
feat: add dynamic skip for schedules ( #6739 )
...
* feat: add dynamic skip for schedules
* npm check
* accidental rename
* fixing tests
* Update SQLx metadata
* simplify api / queries
---------
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-08 09:18:15 +00:00
Diego Imbert
963c5a5965
fix: better ducklake setup ( #6763 )
...
* stash
* Much better ducklake setup UX
* nits
* mistake
* sqlx prepare
2025-10-07 07:00:00 +00:00
hugocasa
f9a035831f
feat: end user email env var ( #6750 )
...
* feat: end user email env var
* nit
* nits
2025-10-03 17:27:58 +00:00
Ruben Fiszel
7a1a9648fb
raw app storage on db + s3 ( #6749 )
2025-10-03 16:30:06 +00:00
centdix
84bd856a83
feat(flow): add option to turn flow into chat ( #6658 )
...
* add toggle option + chat interface
* backend impl
* draft
* put info in schema
* Revert "backend impl"
This reverts commit c534eeb49986424e2c12e2c5642be4e17ba380d1.
* chat interface in flow input
* cleaning
* add logic for running flow + styling
* handle historic args
* fix frontend changes
* add tables
* add conv list
* add endpoints
* adapt frontend
* list message logic
* save message in db
* save response in db
* cleaning
* better migrations
* refresh on new conv
* better logic for messages
* nit
* genere conversation uuid from frontend
* store chat mode info in flow status
* better ui for chat
* collapse chat
* ui
* infinite scroll on convs
* infinite scroll on messages
* fix ui
* new chat entry on new
* cleaning
* change setting logic
* fix test logic from flow input
* move toggle to input
* add warning modal when enabling chat mode
* add summary and explanation on inline script
* add hint for chat mode on user_message desc
* show chat message instead of input in graph
* add warning for triggers
* one logo when not expanded
* use infinitelist for conversations
* add warning when deployment in progress
* full width button
* better icon for menu
* better input + nits
* put toggle in action
* use waitjob
* cleaning
* cleaning
* scroll on new + cleaning
* use enum
* fix logic
* full screen
* cleaning
* exit on updatesqlx error
* Update SQLx metadata
* fix
* cleaning
* add for wait result endpoint
* add missing drop
* delete cascade
* fix: use macro version of query_as in flow_conversations.rs
Use sqlx::query_as! macro instead of query_as function for compile-time
SQL validation and better type safety
Co-authored-by: centdix <centdix@users.noreply.github.com >
* fix: update comment to clarify conversation message update condition
The comment now accurately reflects that the update happens when
it's a flow and it's done (flow_is_done)
Co-authored-by: centdix <centdix@users.noreply.github.com >
* fix: only parse chat_input_enabled if conditions are met
Move the parse_chat_input_enabled() call inside the condition check
to avoid unnecessary parsing when the flow is not done or unsuccessful
Co-authored-by: centdix <centdix@users.noreply.github.com >
* fix: use the same transaction for conversation creation
Pass transaction to get_or_create_conversation_with_id instead of
creating a new one, ensuring all operations are atomic
Co-authored-by: centdix <centdix@users.noreply.github.com >
* fix: remove update trigger and handle updated_at in application code
Remove the database trigger that automatically updates conversation
timestamp and instead update it explicitly when creating messages.
This gives better control and consistency.
Co-authored-by: centdix <centdix@users.noreply.github.com >
* Update SQLx metadata
* cleaning
* feat(aiagent): handle memory (#6719 )
* implement memory
* s3 logic for memory
* fix typo
* much cleaner
* cleaning
* cleaning
* only if chat
* display nit
* nit
* fix stack overflow
* cleaning
* use len arg from input
* cleaning
* change order
* delete memory when conv deleted
* cleaning
* nit
* show description in expr mode
* opti
* opti
* updatee ref
* store string as simple string
* use markdown
* do not wait for deletion
* add delete loading
* fix logic
* fix markdown
* Update ee-repo-ref.txt
* Update SQLx metadata
* fix in test interface
* nit
* nit
* fix layout
* use memory_id to store memory
* shorter description
* rls + grant
* fix text overflow
* extract output from res
* cleaning
* handle streaming
* cleaning
* fix tool error
* nit
* update ref
* fix
* Update SQLx metadata
* nit
---------
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: centdix <centdix@users.noreply.github.com >
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
2025-10-03 14:04:21 +00:00
hugocasa
a08268337b
feat(backend): job result stream optimization ( #6673 )
...
* feat(backend): job result stream optimization
* get offset locally instead of from db
* fix: agent worker result stream
* update ref
* nit
* remove foreign key on job
* fix build
2025-09-26 11:26:50 +00:00
dieriba
d682eac9ac
feat: websocket trigger send runnable result even if error ( #6664 )
...
* done
* nits
* update sqlx
---------
Co-authored-by: hugocasa <hugo@casademont.ch >
2025-09-23 19:22:54 +00:00
dieriba
ee6f0f0357
fix: add ack deadline gcp ( #6625 )
...
* update ref
* add ack deadline
* update ref
2025-09-17 16:48:37 +00:00
wendrul
0bf200d26c
feat: forkables workspaces v0 ( #6479 )
...
* Add create_ephemeral workspace endpoint
* Add cli devShell
* List ephemeral workspaces + improve endpoint
* Add postgres function to clone a workspace (to be revisited)
* Clone workspace using the postgres function
* Add first iteration of ephemeral workspaces command
* Update display of forked workspaces
* Remove SQLX_OFFLINE
* Add UI to create ephemeral workspace
* Add option to exclude repository from being inherited to forks
* WIP: reworking cloning logic
* Fix cloning
* Fix redirect after creating fork
* Clean up cloning behaviour
* Rename ephemeral to fork
* emove ephemeral_workspaces table in favour of columns in workspaces
* Fix display of forked workspaces
* Fix skip inherit git sync repo setting
* Fix fork invite display + creating fork as user
* Fix SideMenu bug
* Fix alignment
* Simplify migrations
* Update deletion of workspaces
* Delete forked workspace from cli
* Deleting fork workspaces from the UI as non-admin
* Update cli sync and fork creation to adapt to branches and forks
* Update fork prefix
* Remove skip tracking toggle
* Fix npm check warnings
* Fix last npm check
* fix: force stdin to Stdio::null for all user code execution (#6575 )
Set stdin to Stdio::null for all Commands that execute user code across all supported languages to prevent unwanted input consumption. This affects Python, Deno, Bash, PowerShell, Go, Rust, PHP, Ruby, Java, C#, Ansible, Nu, and Bun executors.
The dedicated worker handler was intentionally left unchanged as it requires stdin for inter-process communication.
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* Update ee-repo ref
* Update SQLx metadata
* Fix typos
---------
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-09-10 17:42:45 +00:00
hugocasa
5f5968f2de
feat: email triggers ( #6548 )
...
* feat: email triggers
* Change down migration to drop email_trigger table
* email triggers UI
* bug fix
* Apply suggestion from @ellipsis-dev[bot]
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* cli and git sync
* Revert "cli and git sync"
This reverts commit 220fd50d13ruben@windmill.dev >
2025-09-09 21:04:23 +00:00
claude[bot]
bc6a2e6305
feat: add worker_group_job_stats table for efficient job metrics aggregation ( #6527 )
...
* feat: add worker_group_job_stats table for job metrics aggregation
- Add new table with hour timestamp, worker group, script lang, workspace_id, job count and total duration
- Workers accumulate stats in memory and update hourly via sum aggregation
- Monitor.rs cleans up rows older than 60 days periodically
- Stats are flushed on worker shutdown to prevent data loss
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* all
* all
* ee-repo-ref
* nits
* nits
---------
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
2025-09-08 17:27:58 +00:00
pyranota
668283a859
fix(flows): fix relative imports cache invalidation ( #6519 )
...
* fix(flows): fix relative imports cache invalidation
Signed-off-by: pyranota <pyra@duck.com >
* update flow_versions in the end
Signed-off-by: pyranota <pyra@duck.com >
* comms + fallback
Signed-off-by: pyranota <pyra@duck.com >
* add warn if fallback is used
Signed-off-by: pyranota <pyra@duck.com >
* include referenced scripts' hashes in flow_node id
Signed-off-by: pyranota <pyra@duck.com >
* improve fallback + TODOs
Signed-off-by: pyranota <pyra@duck.com >
* perf: optimize trigger
Signed-off-by: pyranota <pyra@duck.com >
* reduce TODOs
Signed-off-by: pyranota <pyra@duck.com >
* remove more TODOs
Signed-off-by: pyranota <pyra@duck.com >
* test thing
Signed-off-by: pyranota <pyra@duck.com >
* remove TODO
Signed-off-by: pyranota <pyra@duck.com >
* drop old flow trigger
Signed-off-by: pyranota <pyra@duck.com >
* add comments to migration
Signed-off-by: pyranota <pyra@duck.com >
---------
Signed-off-by: pyranota <pyra@duck.com >
2025-09-05 16:06:51 +00:00
Ruben Fiszel
88646fd1ee
fix extra migration startup issue
2025-09-02 20:15:37 +00:00
Ruben Fiszel
a86681edd0
add variable caching api
2025-09-02 14:34:46 +00:00
Ruben Fiszel
bca9f899a0
improve the notify mechanism for resource change
2025-09-02 11:08:05 +00:00
claude[bot]
02d5dab51b
feat(perf): add 60-second local cache for variables and resources( #6511 )
...
* feat: add 60-second cache for variables and resources with PostgreSQL invalidation
- Add new var_resource_cache module with 60-second TTL
- Implement PostgreSQL NOTIFY/LISTEN for immediate cache invalidation
- Cache get_variable() for non-secret, non-encrypted reads
- Cache get_resource_value() for all read operations
- Add database triggers on variable/resource table changes
- Initialize cache system in main.rs after database connection
- Add Clone derive to ListableVariable for cache compatibility
Performance benefits:
- Avoids database queries on cache hits
- Immediate invalidation ensures data consistency
- Selective caching respects security constraints
🤖 Generated with [Claude Code](https://claude.ai/code )
Co-Authored-By: Claude <noreply@anthropic.com >
* refactor: move cache to API layer with allow_cache query param
- Move variable/resource cache from windmill-common to windmill-api
- Add allow_cache query parameter to variables and resources endpoints
- Follow raw script cache pattern with timestamp + value structure
- Create proper database migration for notification triggers
- Include encrypted values in cache when appropriate
- Only activate caching when explicitly requested
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
* refactor: move cache to API layer with allow_cache query param
Move PostgreSQL LISTEN logic to main.rs following established pattern
- Remove custom listener initialization from cache module
- Add cache invalidation channels to centralized notification handler
- Simplified cache module to only handle cache operations
- Follow raw script cache pattern for notification handling
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
---------
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
2025-09-02 07:48:03 +00:00
claude[bot]
1f1bd8cae7
feat(perf): add 300-second local cache for variable crypt retrieval ( #6483 )
...
* feat: add 60-second cache for workspace key retrieval
This implements a cache with 60-second staleness for the get_workspace_key
function to reduce database queries for workspace encryption keys. The cache
follows the same pattern as the existing CUSTOM_ENVS_CACHE but with a
shorter expiration time.
Requested by @rubenfiszel
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* Extend cache staleness from 60 to 300 seconds
* feat: add cache invalidation notifications for workspace keys
Add PostgreSQL LISTEN/NOTIFY mechanism to invalidate workspace key cache
across all servers and workers when workspace keys change.
- Add database migration with trigger function for workspace_key changes
- Add notification handler in main.rs to remove from WORKSPACE_KEY_CACHE
- Follow same pattern as existing workspace environment cache invalidation
- Ensures distributed cache consistency for workspace encryption keys
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
* finish
---------
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
2025-08-31 12:19:59 +00:00
hugocasa
734fb3f1f7
prevent past due team plans to overuse ( #6487 )
...
* prevent past due team plans to overuse
* nit
* update ee ref
2025-08-28 16:56:57 +00:00
pyranota
a1b702f2a8
fix: fix relative imports cache invalidation ( #6468 )
...
* fix: new script on deps job for secondary scripts
Signed-off-by: pyranota <pyra@duck.com >
* make it work (dirty)
Signed-off-by: pyranota <pyra@duck.com >
* reduce db calls
* remove `triggered_by_relative_import`
Signed-off-by: pyranota <pyra@duck.com >
* add comment to common_dependency_path
Signed-off-by: pyranota <pyra@duck.com >
* add fallback to old behavior
Signed-off-by: pyranota <pyra@duck.com >
* remove TODOs
Signed-off-by: pyranota <pyra@duck.com >
* pass deployed hash to git sync handler function
Signed-off-by: pyranota <pyra@duck.com >
* fix ci
Signed-off-by: pyranota <pyra@duck.com >
---------
Signed-off-by: pyranota <pyra@duck.com >
2025-08-27 17:44:35 +00:00
hugocasa
7e5b31b124
feat: ai agent steps ( #6393 )
...
* feat: ai agent steps base
* better backend and graph
* feat: anthropic, log viewer
* nit
* fix(frontend): hide tool nodes from timeline
* move ai agent actions from flow status to flow status module
* nits and workspace/hub scripts support
* tmp ref
* fix merge
* feat: display agent tools status in the graph
* fix reactivity
* fix flow status
* nit
2025-08-20 22:40:57 +00:00
pyranota
4341c14f8e
feat: add ruby support ( #5939 )
...
* implement single line pin
* make panic-safe
* use pin even if multiple modules imported withing single statement
* add repins and make imports respect pins
* keep all pins
* Allow multiple pins
* add comments + handle stuff more safely
* fix fully qualified imports
* remove ignore
* sort nested
* apply unique to output requirements list
* fix typo
* remove mut
* update sqlx
* sort imports
* sort imports
* fix formatter and format
* feat: add ruby support
* fix
* update sqlx
* add ruby to flake.nix
Signed-off-by: pyranota <pyra@duck.com >
* upgrade par_install_language_dependencies helper
rename fields + allow passing custom payload
Signed-off-by: pyranota <pyra@duck.com >
* change migrations
Signed-off-by: pyranota <pyra@duck.com >
* make full pipeline work!
Signed-off-by: pyranota <pyra@duck.com >
* do a bit fixing
* stage
Signed-off-by: pyranota <pyra@duck.com >
* add ruby in dockerfiles
Signed-off-by: pyranota <pyra@duck.com >
* implement auto-require + 'windmill/inline'
Signed-off-by: pyranota <pyra@duck.com >
* create 'windmill/inline' only if does not exist
Signed-off-by: pyranota <pyra@duck.com >
* add nsjail
Signed-off-by: pyranota <pyra@duck.com >
* update global settings
Signed-off-by: pyranota <pyra@duck.com >
* add ruby icon
* generate lockfile on deploy (but broken)
Signed-off-by: pyranota <pyra@duck.com >
* fix(frontend): display deployed script lockfile and other info
Signed-off-by: pyranota <pyra@duck.com >
* feat: add mini-wmill client
Methods:
- get_resource
- get_variable
Additionally add shortcuts on frontend
Signed-off-by: pyranota <pyra@duck.com >
* add nit warning
Signed-off-by: pyranota <pyra@duck.com >
* support private repos credentials
Signed-off-by: pyranota <pyra@duck.com >
* fix compilation error
Signed-off-by: pyranota <pyra@duck.com >
* found rustc bug + refactor universal installer
Signed-off-by: pyranota <pyra@duck.com >
* more refactor + fixes
Signed-off-by: pyranota <pyra@duck.com >
* bug fixing + polishing
Signed-off-by: pyranota <pyra@duck.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* change 'windmill/inline'
Signed-off-by: pyranota <pyra@duck.com >
* delegate env_clear to universal installer
Signed-off-by: pyranota <pyra@duck.com >
* update init script
Signed-off-by: pyranota <pyra@duck.com >
* add cli support
Signed-off-by: pyranota <pyra@duck.com >
* unhardcode frontend
Signed-off-by: pyranota <pyra@duck.com >
* fix cli
Signed-off-by: pyranota <pyra@duck.com >
* fix init script
Signed-off-by: pyranota <pyra@duck.com >
* make it work on ms windows
Signed-off-by: pyranota <pyra@duck.com >
* do not clean env in the end
Signed-off-by: pyranota <pyra@duck.com >
* fix windows (again)
Signed-off-by: pyranota <pyra@duck.com >
* include PROXY_ENVS (untested)
Signed-off-by: pyranota <pyra@duck.com >
* feat: self-signed certs + proxy support (tested)
Signed-off-by: pyranota <pyra@duck.com >
* flake: pin ruby to 3.4
* merge follow-ups
Signed-off-by: pyranota <pyra@duck.com >
* remove obsolete libc crate
Signed-off-by: pyranota <pyra@duck.com >
* cleanup parser
Signed-off-by: pyranota <pyra@duck.com >
* fix java ci
Signed-off-by: pyranota <pyra@duck.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* fix nits
Signed-off-by: pyranota <pyra@duck.com >
* Update backend/windmill-worker/src/universal_pkg_installer.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* Update backend/windmill-worker/src/ruby_executor.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* add nsjail for lock stage
Signed-off-by: pyranota <pyra@duck.com >
* fix merge
Signed-off-by: pyranota <pyra@duck.com >
* Update common.rs
* regen windmill-api-client
Signed-off-by: pyranota <pyra@duck.com >
* fix tests
Signed-off-by: pyranota <pyra@duck.com >
* fix tests again
Signed-off-by: pyranota <pyra@duck.com >
* reference published ruby parser
Signed-off-by: pyranota <pyra@duck.com >
---------
Signed-off-by: pyranota <pyra@duck.com >
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-08-15 13:03:32 +00:00
Alexander Petric
44e044f44e
feat: instance groups workspace ( #6380 )
...
* feat: instancd groups mapping to workspace
* npm run check
* nits
* improve apis
* sqlx prepare and ee-repo ref
* adding workspace assignment in groups page
* nits
* correct rollback
* sqlx
* ee repo ref
* revert accidental ee-repo-ref commit to main
* ee repo ref
* revert accidental ee-repo-ref commit to main
2025-08-14 09:06:00 +00:00
dieriba
9be04984b6
feat: togglable manual acknowledgement for gcp trigger ( #6321 )
...
* update
* done
* update repo ref
* nits
* update repo ref
* fix
* all
---------
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
2025-08-07 17:45:50 +00:00
dieriba
4759c42da2
feat: add email workspace error handler if smtp is configured ( #6267 )
...
* detecting trigger kind on job error
* v0 done
* done
* updat .sqlx and add mgrations
* fix: set back vs code settings and unused
* fix unused import
* add feature flag
* update ref
* nits
* nits
* done
* update repo ref
* remove
* add isCloudHosted
* update sqlx
* big
* fix
* use extra args instead of new col
* nits
* update .sqllx
* update ref
* update email
* fix
* fix
* fix
* nits
* update script and use env base_internal_url
* update sqlx and fix query
* remove
* fix unused import
* update ref
2025-08-07 16:46:53 +02:00
claude[bot]
82620098d2
feat: add native result streaming ( #6242 )
...
* feat: add stream output feature to SSE job updates
Adds stream_output field to JobUpdate struct that extracts log lines
starting with '[wm_stream]:' from job logs. Regular logs now exclude
stream lines, which are captured separately for specialized handling.
- Added stream_output: Option<String> field to JobUpdate struct
- Created extract_stream_output_from_logs() function to filter stream lines
- Modified get_job_update_data() to use stream extraction logic
- SSE clients now receive both new_logs and stream_output in job updates
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
* feat: rename stream_output to stream and handle newlines as \n
- Renamed `stream_output` field to `stream` in JobUpdate struct
- Updated extract_stream_output_from_logs to extract_stream_from_logs
- Changed stream output to join with literal \n instead of actual newlines
- Stream lines are properly excluded from regular new_logs
🤖 Generated with [Claude Code](https://claude.ai/code )
Co-Authored-By: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
* decision tree nits
* push ee ref
* push ee ref
* fix: fix id renaming in apps
* remove duplicate caching (#6285 )
* feat: migrate audit log ids to bigints (blocking migration for EE)
* fix(mcp): add proper check for mcp routes (#6282 )
* add proper check for mcp routes
* cleaner
* apply to flow
* fix add checks scopes
---------
Co-authored-by: dieriba <dieriba.pro@gmail.com >
* chore(main): release 1.514.0 (#6283 )
* chore(main): release 1.514.0
* Apply automatic changes
---------
Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com >
* fix: pin tokio to 1.46.1 and aws-sdks-ts
* pin rustls to 0.23.29 + pin aws-sdk
* chore(main): release 1.514.1 (#6288 )
* chore(main): release 1.514.1
* Apply automatic changes
---------
Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com >
* fix: improve docker logs collection in docker mode
* support $res: string in form inputs of arrays
* fix import nit
* fix: fix DynSelect
* nits
* fix: resource-type-ts-parser (#6289 )
* fix: resource types as arg in typescript handle imported defined types
* Update nix flake (#6291 )
* merge
* Small UI fixes (#6294 )
* fix step history not refreshing with staticInputs
* fix array of obj not showing up in json editor in test this step
* datatable scales correctly in DisplayResult and scrolling is much more usable
* avoid next button disapearing and changing layout / hurting ux
* nits
* fix bug when renaming module A to B then module C to A, C takes the schema of A
* fix bug with comments in sql repl
* fix aggrid theme randomly not loading
* bindable script
* better delete button in db manager
* property select doesnt exist
* fix all warnings
* delete $flowStateStore[id] on delete
* feat(cli): generate cursor rules on init (#6270 )
* create cursor rules on init
* change gen
* add missing resource-type command
* add resource type command in guidance
* add schema option
* revert
* nit
* nit
* add flow guidance
* nit
* chore(main): release 1.515.0 (#6292 )
* chore(main): release 1.515.0
* Apply automatic changes
---------
Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com >
* fix: improved logs for script
* nits logs
* chore(main): release 1.515.1 (#6295 )
* chore(main): release 1.515.1
* Apply automatic changes
---------
Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com >
* merge
* even more indexer tracings
* add more tracing logs
* feat: prevent too large results (>500Mb) from OOMing database
* nit naming
* feat: add CA certificate update at startup via environment variable (#6280 )
* feat: add CA certificate update at startup via environment variable
Add support for running 'update-ca-certificates' at binary startup
when RUN_UPDATE_CA_CERTIFICATE_AT_START environment variable is set to "true".
- Check for RUN_UPDATE_CA_CERTIFICATE_AT_START env var on startup
- Execute update-ca-certificates command if env var is set to "true"
- Log success/failure appropriately with tracing
- Continue startup even if CA certificate update fails
- Non-blocking implementation with proper error handling
Fixes #6279
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
* refactor: extract CA certificate update logic into separate function
Extract the CA certificate update logic from windmill_main() into a
dedicated update_ca_certificates_if_requested() function for better
code organization and maintainability.
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
* improvements
---------
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
Co-authored-by: Alexander Petric <alpetric@users.noreply.github.com >
Co-authored-by: Alexander Petric <alex@windmill.dev >
* fix: indexer collection of job logs before indexing (#6300 )
* Add flume as dependecy for indexer
* Update ee-repo-ref
* Remove flags from cargo.toml
* Update ee-repo-ref
* Update ee-repo-ref
* fix rust sdk build error (#6305 )
Signed-off-by: pyranota <pyra@duck.com >
* fix broken audit logs filter (#6304 )
* rename to from to
* goto fix
* default to false if field not present operator settings (#6301 )
* git sync UI improvements (#6303 )
* ui improvements round 1
* modal cleanup
* init
* UI refactor
* UI cleanup + refactor
* legacy cleanup
* success model -> github actions, non-ee warnings
* sqlx
* npm check
* ee warning everywhere
* last comments
* formatting
* no hardcoded theme
* claude review improvemenets
* fix: no process relative imports for scripts with codebase
* fix: sqs oidc authentication disconnect #6307
* handle metadata for new scripts happen after commit
* handle_deployment_metadata in a task
* nits
* chore: add windmill-utils-internal package (#6299 )
* add utils package
* naming
* cleaning
* add docs
* remove log
* use autogenerated types
* remove old
* fix
* cleaning
* add docs
* chore(main): release 1.516.0 (#6298 )
* chore(main): release 1.516.0
* Apply automatic changes
---------
Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com >
* merge
* indexer improvements
* upgrade tantivy to 0.24.2
* use tantivy fork
* nit warnings
* fix oss build
* improve indexer
* chore: use windmill-utils-internal for cli (#6297 )
* add utils package
* naming
* cleaning
* simplify assignPath
* rename old files
* same for locks
* create on confirm
* default true
* use replaceinlinescripts from utils
* use extractscriptfromflows
* make it compile
* cleaning
* use argsigtojson
* fix
* fix missing await
* cleaner
* cleaning
* cleaning
* use in frontend
* add docs
* testing
* remove log
* use autogenerated types
* remove old
* fix
* cleaning
* adapt usage
* draft
* better build script
* fix build
* revert to default creation
* add docs
* remove and rename
* make everything work
* add await
* only if not installed
* add vs code setting
* add to publish action
* fix bc
* safer use of sep
* fix
* do not rename on push
* no publish on release
* use published package on frontend
* nit
* Add dependencies to run sqlx prepare to nix flake (#6309 )
* feat(cli): wmill-lock.yaml v2 for easier git merge diffs
* merge
* merge
* all
* all
* rm warnings
* fix styling on aichatinput (#6312 )
* fix: use with_capacity back presusre for tantivy directory multipart writes (#6313 )
* use with capacity for tantivy directory multi part uploads
* Update ee repo ref
* Update ee-repo-ref
* Update ee-repo-ref
* chore(main): release 1.517.0 (#6310 )
* chore(main): release 1.517.0
* Apply automatic changes
---------
Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com >
* fix typo on cli build (#6314 )
* cleanup
* feat(utils): add flow.yaml validation function (#6316 )
* add validateflow function
* cleaner code
* preprocess json
* cleaning
* create specific package
* cleaning
* add tests
* fix: cleanup concurrency_counter automatically + remove orphans keys automatically
* fix: add disabled support to resource picker in schema forms
* fix: add wm_labels to tracing spans
* all
* merge
* all
* fix: delete empty git connection (#6318 )
* fix checks
* bun handling
* all
* all?
* all
* all
* update
* all
* update
* check
* fix history
* all
* all
* all
* Remove leftover debug tracing statements
- Remove commented debug trace in jobs.rs for stream output
- Remove commented debug trace in result_stream.rs for stream processing
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
* fix test
* all
* handle iter
* fix
---------
Signed-off-by: pyranota <pyra@duck.com >
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
Co-authored-by: centdix <40307056+centdix@users.noreply.github.com >
Co-authored-by: dieriba <dieriba.pro@gmail.com >
Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com >
Co-authored-by: wendrul <53628737+wendrul@users.noreply.github.com >
Co-authored-by: Diego Imbert <70353967+diegoimbert@users.noreply.github.com >
Co-authored-by: Alexander Petric <alpetric@users.noreply.github.com >
Co-authored-by: Alexander Petric <alex@windmill.dev >
Co-authored-by: pyranota <92104930+pyranota@users.noreply.github.com >
2025-08-06 22:40:42 +00:00
Diego Imbert
a45c9ca525
feat: Ducklake native support ( #6268 )
...
* upgrade duckdb
* basic ducklake works
* ducklake works with custom db catalogs
* fix: pwsh skip already installed modules outside of cache (#6037 )
* improve query performance of user stats
* separate ducklake_catalog db
* ducklake settings
* DucklakeSettings frontend
* Ducklake ws settings saved in backend
* fetch ducklake catalog resource
* Ducklake works with configured s3 storage
* Ducklake as asset
* ducklake asset icon
* Fix duckdb array and object args not working properly (#6254 )
* Fix bug with comments in duckdb
* Avoid multiple queries when doing ATTACH ducklake
* trunc sig no longer needed now that comments are trimmed
* cache DuckdbConnectionSettingsResponse
* duplicated code
* transform_attach_ducklake contributes to duckdb_connection_settings_cache
* eliminate the need for used_storages
* nit
* cleaner management of the bigquery credentials file
* DBManagerDrawer refactor to prepare for Ducklake
* get ducklake schema
* implement delete for ducklake
* load column metadata for ducklake
* Select query works for ducklake, basic db explorer works !
* duckdb count query
* Support all db ops for ducklake
* clean migrations
* SQL repl for Ducklake
* fix broken database studio
* nit
* assert function
* Ducklake in Editor Bar
* default ducklake syntax + allow extra args
* DucklakeCatalogWizard UI
* nit + remove extra $
* modal when databases do not exist
* cannot be windmill
* Ducklake works safely with instance database
* Avoid sending instance db credentials on network
* resource leak security
* remove fetch_attach_db_conn_str
* prevent instance pg password leak
* hide asset usage count when not available
* case unsensitivity duckdb
* warnings
* disable instance catalog
* use shorthand syntax when inserting with EditorBar
* Instance ducklake catalog is now safe to use
* use safer argon2 pwd
* update package json parsers
* update package json
* better msgs
* tooltips
* disable explore button until saved
* nit
* fix warnings
* better ducklake_user password management
* nit
* Sanitize passwords from errors in ducklake
* DisplayResult broken in job result
* remove superadmin requirement to check databases_exist
* duckdb_connection_settings_v2_inner
* Ducklake works on agent worker (finally)
* ci
* #[allow(dead_code)]
* fix openapi missing response
* Separate +Database button for DuckDB in EditorBar
* Fix dropdown in ducklake settings
* Attempt to fix migration race condition in CI
* update sqlx failing for some offline queries
* avoid temp password for ducklake_user
* nits
* ducklake settings nits
* update duckdb default script
* fix sql repl resetting text on refresh
* avoid pgcrypto extension
---------
Co-authored-by: HugoCasa <hugo@casademont.ch >
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
2025-08-06 13:55:36 +00:00
