kalugin66/windmill
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
10,576 Commits 513 Branches 1,362 Tags
f35c9ffe63cd8a79bbdfe498dbb08e53efd9e5a8
Commit Graph

1360 Commits

Author SHA1 Message Date
Diego Imbert
f35c9ffe63 S3 Proxy better errors + DuckDB S3 fix (#6740) 
* Support for MinIO and other S3 impls in S3 Proxy

* Nice S3 Proxy error messages in DuckDB executor

* nit

* useless code

* super nit

* ee repo ref
 2025-10-03 11:06:46 +00:00
hugocasa
cdb7524118 fix(backend): concurrency limits preprocessor (#6727) 
* fix(backend): concurrency limits preprocessor

* refactor

* fix concurrency on agent workers

* nit

* nit

* update ref

* fix ci

* chore: update ee ref
 2025-10-02 20:31:31 +00:00
Diego Imbert
59cdb141c3 NULL Toggle in InsertRow drawer (#6729) 
* NULL toggle in InsertRow

* fix long type parsing in postgres

* nits

* graphite catch

* lazy_static

* support for time/timestamp/tz long forms in pg parser

* graphite suggestion
 2025-10-02 10:10:53 +00:00
claude[bot]
c7f6ff14af correct typos in user-facing text (#6725) 
- Fixed 'occured' -> 'occurred' (10 instances)
- Fixed 'commited' -> 'committed' (2 instances)

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
 2025-10-01 16:04:15 +00:00
Ruben Fiszel
e2e3ae9f02 fix: fix job duration unwrap crash 2025-10-01 15:35:09 +00:00
Ruben Fiszel
ee00964a06 nit 2025-10-01 09:57:24 +00:00
Ruben Fiszel
5689411199 nit exit status 2025-10-01 09:18:43 +00:00
Ruben Fiszel
d382ea7c8b feat: support esm mode for codebase bundles (#6709) 
* rawAppsS3

* make fn common

* all

* merge

* nit

* fix ingress

* all

* all

* all

* all
 2025-09-30 14:45:30 +00:00
hugocasa
898eb6231b feat(backend): array and object params support in pwsh (#6706) 
* feat(backend): array and object params support in pwsh

* add bool and improve arg conversion

* fix bash

* update parser

* sqlx
 2025-09-30 12:25:12 +00:00
Ruben Fiszel
2055e536a7 fix: fix c# with nsjail 2025-09-29 16:34:29 +00:00
hugocasa
4bbbeb956f feat: powershell private repo support (#6684) 
* feat: powershell private repo support

* nits
 2025-09-29 13:24:41 +00:00
Pyra
5f63ce6dd8 fix(uv): log stdout on uv pip install error (#6702) 
Signed-off-by: pyranota <pyra@duck.com>
 2025-09-29 12:12:06 +00:00
Osman Mesut OZCAN
629a378320 Fix typo (#6696) 2025-09-29 07:38:15 +00:00
hugocasa
f990107c45 feat: ai agent streaming (#6644) 
* feat: ai agent step streaming

* refactor

* all

* nits

* fix other providers

* nits

* adapt to new streaming process
 2025-09-26 17:09:34 +00:00
Ruben Fiszel
db4bc7ee69 fix: fix agent_workers completed job back-compatibility deserialization 2025-09-26 15:27:21 +00:00
Diego Imbert
4828926be6 Fix SEGFAULT with duckdb and jemalloc (#6680) 2025-09-26 13:57:28 +00:00
hugocasa
8f4fef9804 feat(backend): job result stream optimization (#6673) 
* feat(backend): job result stream optimization

* get offset locally instead of from db

* fix: agent worker result stream

* update ref

* nit

* remove foreign key on job

* fix build
 2025-09-26 11:26:50 +00:00
Pyra
ed806bf9d0 fix(backend): rework dependency_map handling (#6598) 
* v0

Signed-off-by: pyranota <pyra@duck.com>

* optimize relocks

* make it work with relative relative imports

Signed-off-by: pyranota <pyra@duck.com>

* use fallback

Signed-off-by: pyranota <pyra@duck.com>

* remove dbg and todos

Signed-off-by: pyranota <pyra@duck.com>

* future proof a bit

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* more cleanup

Signed-off-by: pyranota <pyra@duck.com>

* remove final TODO

Signed-off-by: pyranota <pyra@duck.com>

* do not use bytemuck

Signed-off-by: pyranota <pyra@duck.com>

* optimize hashing

Signed-off-by: pyranota <pyra@duck.com>

* implementation 1

Signed-off-by: pyranota <pyra@duck.com>

* almost v0

Signed-off-by: pyranota <pyra@duck.com>

* v0

Signed-off-by: pyranota <pyra@duck.com>

* add comments and use fallback

Signed-off-by: pyranota <pyra@duck.com>

* call dissolve for apps

Signed-off-by: pyranota <pyra@duck.com>

* add comms

Signed-off-by: pyranota <pyra@duck.com>

* refactor v0 (partially tested + dirty)

Signed-off-by: pyranota <pyra@duck.com>

* finishing

Signed-off-by: pyranota <pyra@duck.com>

* remove TODO

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* silence unused argument

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* implement rebuild_map endpoint

Signed-off-by: pyranota <pyra@duck.com>

* update windmill api client

Signed-off-by: pyranota <pyra@duck.com>

* almost finish with tests

Signed-off-by: pyranota <pyra@duck.com>

* add proper testing

Signed-off-by: pyranota <pyra@duck.com>

* remove unused fixtures

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* partial cleanup

Signed-off-by: pyranota <pyra@duck.com>

* Update backend/windmill-worker/src/scoped_dependency_map.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-common/src/scripts.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* evil doings

Signed-off-by: pyranota <pyra@duck.com>

* more cleanup

* Update SQLx metadata

* more cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fixing CI

Signed-off-by: pyranota <pyra@duck.com>

* remove python from default features

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: Pyra <92104930+pyranye@users.noreply.github.com>
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
 2025-09-24 13:40:39 +00:00
Ruben Fiszel
2de7134b85 fix: improve scripts duplicity error in global search 2025-09-23 15:03:41 +00:00
Ruben Fiszel
27881c672d rm warn 2025-09-20 13:44:35 +00:00
Ruben Fiszel
81ff0dcd8c fix: retry python relative imports on errno 104 2025-09-20 13:41:15 +00:00
Ruben Fiszel
1afb7a2156 improve job cancelling with new flow jobs locks 2025-09-20 11:00:45 +00:00
Ruben Fiszel
9a21107848 improve job cancelling with new flow jobs locks 2025-09-20 10:17:10 +00:00
Ruben Fiszel
88b4bc2c9b remove tracing err 2025-09-20 08:11:19 +00:00
Ruben Fiszel
9d8641cdd0 fix flow jobs 2025-09-20 08:07:41 +00:00
Ruben Fiszel
fb93024d3e vacuum skip_locked 2025-09-20 00:42:43 +00:00
Ruben Fiszel
56cdb69e59 fix: add termination handler earlier in lifecycle 2025-09-20 00:22:15 +00:00
hugocasa
d365014882 build on macos with deno_core (#6642) 2025-09-19 17:33:05 +00:00
Ruben Fiszel
f71f9b0894 feat: load for loop jobs timeline directly from for loop flow status (#6646) 2025-09-19 17:32:09 +00:00
hugocasa
993baf46bd feat(backend): flow streaming (#6520) 
* feat(backend): flow streaming

* all streaming languages + sync api

* sqlx

* fix build

* UI and nits

* nit

* feat: stream last flow step

* sqlx

* nit

* use get for stream endpoints + add snippet in UI

* refactor

* nits

* Update backend/windmill-worker/src/common.rs

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>

* nits

---------

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
 2025-09-18 15:49:35 +00:00
centdix
709a937ac2 fix(aiagent): fix endpoint for azure (#6633) 
* fix ai agent for azure

* cleaning

* add parentheses
 2025-09-18 13:22:24 +00:00
centdix
3199f9fffd feat(ai agent): allow multiple images input for ai agent + code cleaning (#6591) 
* add in frontend

* draft openai handling

* upload to s3

* simpler output

* return s3 directly if any

* low quality

* implement for gemini

* handle imagen model

* handle image input

* cleaning

* remove base64 from output

* cleaning

* fix timeout

* handle openrouter

* remove log

* allow image input when creating image

* cleaning

* increase stack size

* inline everything

* revert stack size

* refactor: move AI executor types to separate module

- Created ai module structure with types.rs
- Moved all type definitions from ai_executor.rs to ai/types.rs
- No functional changes, just code organization

* refactor: add QueryBuilder trait and provider detection utilities

- Created QueryBuilder trait for abstracting provider-specific logic
- Added helper functions for provider detection (is_anthropic_provider)
- Implemented placeholder QueryBuilder for all providers
- Updated OpenAIRequest to use slices instead of Vec references
- All providers now have QueryBuilder implementations (using default for now)

* feat: implement OpenAI query builder with image support foundation

- Created proper OpenAI query builder implementation
- Added image_handler module for S3 upload/download utilities
- Separated text and image request building logic
- Added prepare_messages_for_api to handle S3Object conversion
- Foundation laid for supporting tools with image output

* refactor(ai): complete AI executor refactoring with query builder pattern

- Created modular structure under ai/ module
- Moved all types to ai/types.rs
- Created QueryBuilder trait for provider abstraction
- Implemented OpenAI query builder with image+tools support
- Added unified agent runner supporting both text and image outputs with tools
- Refactored run_agent to delegate to new unified implementation
- Added image handler utilities for S3 operations
- Improved code organization and maintainability

* cleaning

* feat(ai): implement remaining provider query builders

- Added Anthropic query builder with proper message conversion
- Added Google AI query builder with Gemini API support
- Added OpenRouter query builder delegating to OpenAI for compatibility
- Added missing Anthropic and Gemini types to types.rs
- Fixed type references and compilation errors
- All providers now support the unified query builder interface

* fixes

* fixes

* mime type + cleaning

* image to images

* handle mutlitple images

* fix

* remove agent_runner file

* clean query builder logic

* cleaning

* cleaning

* hide structured_output based on output type

* fix

* user images and not nested

* better descriptions
 2025-09-17 16:55:54 +00:00
wendrul
9325f5636c fix: Don't reencrypt secrets on workspace forking (#6622) 
* Do not reencrypt variable when performing a workspace fork

* Update SQLx metadata

* Update SQLx metadata

* Remove unused

---------

Co-authored-by: GitHub Action <action@github.com>
 2025-09-17 16:47:27 +00:00
hugocasa
c24c629317 fix: run preprocessor even if empty flow (#6609) 2025-09-15 12:13:43 +00:00
centdix
20f48e6ded feat(ai agent): handle images in ai agent (#6572) 
* add in frontend

* draft openai handling

* upload to s3

* simpler output

* return s3 directly if any

* low quality

* implement for gemini

* handle imagen model

* handle image input

* cleaning

* remove base64 from output

* cleaning

* fix timeout

* handle openrouter

* remove log

* allow image input when creating image

* cleaning

* increase stack size

* inline everything

* revert stack size

* cleaning

* fix for openai

* better mime type

* add descriptions
 2025-09-15 07:51:09 +00:00
Ruben Fiszel
aa68ac8336 more tracing on archiving scripts 2025-09-11 17:34:54 +00:00
pyranota
c4ccc4ba14 fix(apps): fix relative imports cache invalidation (#6564) 
* v0

Signed-off-by: pyranota <pyra@duck.com>

* optimize relocks

* make it work with relative relative imports

Signed-off-by: pyranota <pyra@duck.com>

* use fallback

Signed-off-by: pyranota <pyra@duck.com>

* remove dbg and todos

Signed-off-by: pyranota <pyra@duck.com>

* future proof a bit

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* more cleanup

Signed-off-by: pyranota <pyra@duck.com>

* remove final TODO

Signed-off-by: pyranota <pyra@duck.com>

* do not use bytemuck

Signed-off-by: pyranota <pyra@duck.com>

* optimize hashing

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
 2025-09-11 10:46:01 +00:00
claude[bot]
fcd58191d4 fix: force stdin to Stdio::null for all user code execution (#6575) 
Set stdin to Stdio::null for all Commands that execute user code across all supported languages to prevent unwanted input consumption. This affects Python, Deno, Bash, PowerShell, Go, Rust, PHP, Ruby, Java, C#, Ansible, Nu, and Bun executors.

The dedicated worker handler was intentionally left unchanged as it requires stdin for inter-process communication.

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
 2025-09-10 16:14:25 +00:00
Diego Imbert
304f953710 Separate duckdb crate to fix c++ build/link issues (#6551) 
* call ffi

* remove duckdb dep

* rename windmill_duckdb_ffi_internal

* static lib

* ci

* back to dylib, bug isn't fixed in static

* feature flag and copy dynamic lib

* fix dynlib in docker

* load libwindmill_duckdb_ffi_internal at runtime on usage

* lazy static deadlocks

* Cache dynamic library handles

* update auto s3 path insert from editor bar

* Fix duckdb S3 freezing worker because of blocking task in tokio async context

* build dll windows GH workflow

* try fix windows build

* revert build.rs

* nit fixes CI

* Dockerfile update (not tested yet

* build dev sh for duckdb lib

* mistake

* attach windmill_duckdb_ffi_internal.so artefact

* rhel9

* docker fixes

* fix dockerfile

* better err msg

* forgot lib prefix .so

* add column_order

* fix column_order
 2025-09-10 14:43:46 +00:00
Ruben Fiszel
cf70265ee9 rust tests cleanup (#6573) 
* test improvements

* improve tests

* improve tests
 2025-09-10 14:43:10 +00:00
hugocasa
36bbde6239 feat: email triggers (#6548) 
* feat: email triggers

* Change down migration to drop email_trigger table

* email triggers UI

* bug fix

* Apply suggestion from @ellipsis-dev[bot]

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* cli and git sync

* Revert "cli and git sync"

This reverts commit 220fd50d13.

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
 2025-09-09 21:04:23 +00:00
Ruben Fiszel
4f9e5badf9 fix: add request duration to relative imports loading in python 2025-09-09 09:09:33 +00:00
claude[bot]
20f4086a77 feat: add worker_group_job_stats table for efficient job metrics aggregation (#6527) 
* feat: add worker_group_job_stats table for job metrics aggregation

- Add new table with hour timestamp, worker group, script lang, workspace_id, job count and total duration
- Workers accumulate stats in memory and update hourly via sum aggregation
- Monitor.rs cleans up rows older than 60 days periodically
- Stats are flushed on worker shutdown to prevent data loss

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* all

* all

* ee-repo-ref

* nits

* nits

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
 2025-09-08 17:27:58 +00:00
dieriba
c210a404e0 feat: timeout as expression (#6509) 
* done

* add ctx and flow input

* backward compatible

* fix typo

* fix
 2025-09-08 15:42:21 +00:00
Ruben Fiszel
83fff3a590 fix: fix db pool corruption by nativets logs 2025-09-06 14:15:18 +00:00
pyranota
f50ab6387a fix(flows): fix relative imports cache invalidation [ext] (#6546) 
* fix: follow up on prev pr

Signed-off-by: pyranota <pyra@duck.com>

* Update worker_lockfiles.rs

---------

Signed-off-by: pyranota <pyra@duck.com>
 2025-09-05 17:07:13 +00:00
pyranota
d0982432e2 fix(flows): fix relative imports cache invalidation (#6519) 
* fix(flows): fix relative imports cache invalidation

Signed-off-by: pyranota <pyra@duck.com>

* update flow_versions in the end

Signed-off-by: pyranota <pyra@duck.com>

* comms + fallback

Signed-off-by: pyranota <pyra@duck.com>

* add warn if fallback is used

Signed-off-by: pyranota <pyra@duck.com>

* include referenced scripts' hashes in flow_node id

Signed-off-by: pyranota <pyra@duck.com>

* improve fallback + TODOs

Signed-off-by: pyranota <pyra@duck.com>

* perf: optimize trigger

Signed-off-by: pyranota <pyra@duck.com>

* reduce TODOs

Signed-off-by: pyranota <pyra@duck.com>

* remove more TODOs

Signed-off-by: pyranota <pyra@duck.com>

* test thing

Signed-off-by: pyranota <pyra@duck.com>

* remove TODO

Signed-off-by: pyranota <pyra@duck.com>

* drop old flow trigger

Signed-off-by: pyranota <pyra@duck.com>

* add comments to migration

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
 2025-09-05 16:06:51 +00:00
centdix
832d120e6c chore: better error message for ai agent + add back .ts in utils-internal (#6544) 
* better error message

* add back ext after publishing

* add back ext
 2025-09-05 14:52:41 +00:00
Ruben Fiszel
1611404e95 fix flow push schedule on flow update (#6538) 
* fix flow schedule

* fix flow schedule
 2025-09-05 10:09:17 +00:00
centdix
c0dbbe942c feat(flows): allow all providers for ai agent steps (#6529) 
* add ui for structured output

* implement backend logic for openai models

* simulate having non required props

* cleaning

* implement logic for anthropic

* cleaning

* cleaning

* cleaning

* avoid name clash

* return object

* focus new field when adding

* fix condition

* small opti

* use box raw value

* avoid unnecessary parsing + return error if parsing fails

* update ui to show all providers

* cleaning

* adapt backend

* fix

* styling fix

* fix for anthropic with openrouter

* nit styling

* avoid race condition

* fix merge

* fix

* fix import

* make one component for whole provider selection

* fix resource logic

* rename

* clean

* remove console log

* fix bad copy
 2025-09-05 08:11:23 +00:00