Commit Graph

252 Commits

Author SHA1 Message Date
Ruben Fiszel
26bec054a3 fix: flow steps use their tags if any specific when used as subflow 2025-06-17 02:27:24 +02:00
Ruben Fiszel
e8836a393a fix: improve concurrent job parallelism performance 2025-06-09 16:59:07 +02:00
dieriba
f2526571a3 feat: ssh repl like direct to workers hosts machine (#5809)
* feat(backend): add repl for worker

* feat(frontend): add repl to interact with worker and missing packages

* nits

* Update backend/windmill-worker/src/result_processor.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-worker/src/result_processor.rs

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>

* Update frontend/src/lib/components/WorkerRepl.svelte

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>

* fix typo

* fix cd

* nits

* refactoring

* feat: handle other agent worker to also access repl feature, fix bugs

* update repo ref

* nits and match function args

* nits

* typo

* remove struct AgentWorkerData

* update repo ref

* nits

* impl new for authed client and revert to async

* update ref

* updage ee repo ref

* fix missing method/imports small bugs

* update ee repo ref

* update .sqlx

* test

* clean wait for interactive_shell future

* free call stack

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
2025-06-03 18:41:14 +02:00
Diego Imbert
0e316239dd EE Refactor (#5844)
* app compiles with every ee substituted

* Replace all oss files content

* Revert "Replace all oss files content"

This reverts commit ea4017d59f.

* delete all ee

* hide all _ee files under private flag

* hide every oss stuff when private flag set

* pub use *

* gitignore and substitute script

* pub mod for ee needed for ee repo

* small mistakes

* remove oidc_oss impl

* ee ref (temp)

* ee ref

* fix --all-features selecting private in OSS CI

* ee repo ref

* allow unused
2025-06-02 22:12:33 +02:00
Ruben Fiszel
5b96bccedd feat: add aws oidc support for instance s3 storage (#5810)
* backend

* iterate

* all

* all

* all

* iterate

* revert

* all

* add tracing to get of authed client

* all

* all

* lal

* all

* update

* fix

* push

* all

* all

* revert

* frontend

* fix checks

* avoid deadlock

* safer

* fix

* fix
2025-05-25 14:03:38 +02:00
Ruben Fiszel
beb658757f nit error 2025-05-17 10:56:16 +02:00
Ruben Fiszel
59afa493fa fix: improve handling of custom concurrency key/tag with preprocessors (#5762)
* fix

* rm spurrious diff

* handle failing preprocessors

* nits

* update

* update
2025-05-17 10:52:28 +02:00
Ruben Fiszel
e87d4f3c1a fix: avoid deadlocks in sending completed job to result processors (#5742)
* buffer

* unbounded and bounded

* fix

* buffer

* impl clone

* flume it all

* update

* update

* update
2025-05-15 09:13:09 +02:00
Ruben Fiszel
5b26498e09 add more duration warnings to logs 2025-05-14 10:13:14 +02:00
HugoCasa
254c3cf8ef feat: preprocessor refactor (#5629)
* feat: preprocessor refactor

* nit

* oneOf kind support + new capture format + fix build

* fix build

* fix build

* update preprocessor templates

* cache runnable version info

* update ee ref

* update hub scripts && ee ref
2025-05-09 09:50:01 +02:00
dieriba
242a565428 feat: raise error if end early in flow (#5653)
* update front

* handle err_msg display if enabled when stop condition is met

* nits

* Update backend/windmill-worker/src/worker_flow.rs

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>

* Update openflow.openapi.yaml

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update worker_flow.rs

* Update worker_flow.rs

* update flow test

* update condition

* remove println

* nits

* fix flow test

* nits and missing property on struct definition

* nits

* fix unsaved state

* update .sqlx

* cleanup sqlx

---------

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-05-07 18:23:40 +02:00
Ruben Fiszel
a7b2b51444 fix: improve skip failure on parallel branchall 2025-04-27 00:37:25 +02:00
Ruben Fiszel
12ba15c928 improve benchmarks 2025-04-12 11:09:21 +00:00
Ruben Fiszel
63fa499015 feat: agent workers v2 using http (#5588) 2025-04-10 00:56:11 +00:00
Ruben Fiszel
3edca4bc91 perf: optimize number of queries needed for job run (#5504)
* optPerf

* update sqlx

* update sqlx
2025-03-24 20:12:26 +01:00
Ruben Fiszel
fb532e0b85 nit move away from using views 2025-03-19 10:48:43 +01:00
Ruben Fiszel
880db319e8 feat: filter by worker + backend perf opt (#5489)
* all

* all

* all

* refactor

* sqlx

* ref

* impr

* fix

* fix

* fix

* ee private
2025-03-19 09:55:53 +01:00
Ruben Fiszel
1ef482e8ae feat: add ready endpoints for workers to enterprise 2025-02-18 21:49:35 +01:00
Ruben Fiszel
77a8eedc96 fix: display branch chosen even if emoty branch 2025-02-14 04:23:25 +01:00
Ruben Fiszel
e7d4582789 fix: improve handling of empty branches and loops 2025-02-14 04:12:55 +01:00
Ruben Fiszel
2015e79ff0 fix: better handling of null pre-processor return values 2025-02-12 22:10:53 +01:00
Lucas Abel
bf206515e8 fix(backend): improve schedule queries plan to leverage indices better for performance (#5273)
* backend: fix schedule queries plan

* rework

* fix skipped as non success

* update sqlx

* update

* all

* fix npm run check

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-02-12 16:34:03 +01:00
Lucas Abel
4a744ff313 backend: move bench.rs from common to worker crate (#5253) 2025-02-10 11:39:59 +01:00
Ruben Fiszel
bf7f67e42f load previous flow steps even for just test this steps 2025-02-10 04:38:29 +01:00
Ruben Fiszel
403826fca9 fix: worker name in job + better timeout handling for same_worker jobs (#5248) 2025-02-08 22:20:06 +01:00
Lucas Abel
d51c3080f7 backend: use v2 tables through views where possible (v2 phase 3) (#5119) 2025-02-06 12:43:24 +01:00
Lucas Abel
e0d99ab703 backend: check queries to queue and completed_job where possible (v2 phase 2) (#5125) 2025-02-06 12:39:32 +01:00
Lucas Abel
3e8e0c653e backend: migrate job tables to v2 schema (v2 phase 1) (#5084) 2025-02-06 12:37:11 +01:00
Ruben Fiszel
90ba65ae20 improve error messages for internal err 2025-02-05 16:32:01 +01:00
Alexander Petric
03ad0385ad improve audit aprovals (#5161)
* update audit log to track aproval and cancellation of job

* approval to resource field

* audit resource as json

* refactor aproval audit logs

* moving audit logs to worker_flow

* striping u/ from auditor name

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-01-30 09:13:44 +01:00
HugoCasa
04fbda28e8 feat: on behalf of (#5058)
* feat: on behalf of

* fix

* revert mistakke

* temp ee ref

* renaming + CLI

* sqlx

* nits + ee ref
2025-01-14 18:32:25 +01:00
Lucas Abel
8586476a13 backend: do not force get_schedule_opt to be within a transaction (#5019) 2025-01-06 09:36:57 +01:00
Lucas Abel
83d24cb9ab cache: better cache system (#4951)
* cache: consider source as untrusted

* cache: better scoped cache

* cache: non-lazy flow value parse error

* fixup! cache: consider source as untrusted

* fixup! cache: consider source as untrusted

* fixup! cache: better scoped cache
2024-12-20 11:19:57 +01:00
Ruben Fiszel
bc4d13f5ff nits cleanup 2024-12-18 11:49:44 +01:00
Ruben Fiszel
13be0cd1c8 fix: fix relative imports in cached flow scripts 2024-12-18 11:34:07 +01:00
Ruben Fiszel
1b008409e2 flow editor improvements 2024-12-18 00:21:25 +01:00
Lucas Abel
977ac5c3f3 feat(cache): remove persistent raw values from queue (#4866)
* feat(cache): remove persistent raw values from queue

* test: add tests and fix discovered regressions

* fix csharp executor after rebase

* invalidate script cache after locking

* fix lite flow version race

* fix flow dependencies lock race

* update version flag to `v.1.440`
2024-12-16 17:10:16 +01:00
Lucas Abel
af5cca1b00 feat(cache): re-work job results cache (#4898) 2024-12-11 12:01:48 +01:00
Lucas Abel
e77ae1c25b fmt: re-format Rust leftovers (#4897) 2024-12-11 08:57:00 +01:00
Ruben Fiszel
6d047449e2 feat: add otlp support (#4869)
* all

* all

* add otel

* add otel

* update docker-image

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update
2024-12-07 16:29:20 +01:00
Ruben Fiszel
e5efefc7e8 improve retries max value handling 2024-12-06 01:03:46 +01:00
Ruben Fiszel
db62a2c53c simplify warn_after_seconds 2024-12-04 18:28:28 +01:00
Ruben Fiszel
bb9dd7d368 simplify warn_after_seconds 2024-12-04 18:24:31 +01:00
Ruben Fiszel
b534ef6074 improve transparency for slow queries 2024-12-04 17:30:58 +01:00
Ruben Fiszel
aa1c3702d7 fix empty branchall subflow 2024-11-29 19:02:47 +01:00
Lucas Abel
2911775d0a feat(backend): store flow inner modules into flow_node table (#4778) 2024-11-27 16:08:31 +01:00
Lucas Abel
d86ea30221 enable is_simple optimization for FlowScript (#4799)
* backend: remove some allocations in `worker_flow.rs`

* feat(backend): enable `is_simple` optimization for `FlowScript`

* fix: fix node retrieval for `flowscript` inner step
2024-11-27 10:06:19 +01:00
Lucas Abel
e4784e89da feat(backend): implement flow scripts (#4748)
* feat(backend): implement flow scripts

* conditionally resolve reduced flow value

* conditionally use flow scripts based on the min worker version

* grant all to `flow_{version_lite,node}`

* do not silently discard errors

* env: add flow script feature flag

* flow script on cloud automatically

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-11-25 23:20:51 +01:00
Ruben Fiszel
6768e5bbbd fix: retry on inserting completed job (#4784)
* fix: retry on pushing next scheduled job of schedule

* all

* improve error handling

* update sqlx
2024-11-25 01:33:44 +01:00
Ruben Fiszel
89456f502f remove rsmq 2024-11-24 11:04:54 +01:00