Ruben Fiszel
094a9deccd
fix: improve cancel for flows with many substeps
2025-03-24 22:00:15 +01:00
Ruben Fiszel
123b4405ed
perf: optimize number of queries needed for job run ( #5504 )
...
* optPerf
* update sqlx
* update sqlx
2025-03-24 20:12:26 +01:00
HugoCasa
884aba4811
perf: improve perf of get completed flow node ( #5418 )
...
* perf: improve perf of get completed flow node
* better
* nit
* nits
---------
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
2025-03-19 10:51:30 +01:00
Ruben Fiszel
eefb77f445
nit move away from using views
2025-03-19 10:48:43 +01:00
Ruben Fiszel
aaa4cde349
feat: filter by worker + backend perf opt ( #5489 )
...
* all
* all
* all
* refactor
* sqlx
* ref
* impr
* fix
* fix
* fix
* ee private
2025-03-19 09:55:53 +01:00
Ruben Fiszel
5134deefad
improve concurrency limits II ( #5485 )
2025-03-17 17:24:50 +01:00
Ruben Fiszel
fcaee6fef5
fix: improve concurrency limit check performances
2025-03-17 00:53:14 +01:00
Ruben Fiszel
c15e699150
DISABLE_CONCURRENCY_LIMIT 2
2025-03-16 18:22:41 +01:00
Ruben Fiszel
18ecb261c3
DISABLE_CONCURRENCY_LIMIT
2025-03-16 17:45:40 +01:00
Ruben Fiszel
fe6c2abd59
fix: improve row lock contention on concurrency counter
2025-03-16 17:20:39 +01:00
Ruben Fiszel
5765d4cf75
nit zombie job monitoring
2025-03-11 07:08:28 +01:00
Ruben Fiszel
d7879e237b
fix: improve cancel performance
2025-03-03 23:50:13 +01:00
Ruben Fiszel
4ae8cad4d4
fix: schedules do not accept 5 units cron syntax on update/create anymore
2025-02-20 17:59:32 +01:00
Ruben Fiszel
60a3cc2e65
make benchmarks even more stable
2025-02-18 08:43:03 +01:00
Ruben Fiszel
a034c3ff28
fix: improve que job indices for faster performances
2025-02-17 21:26:46 +01:00
Lucas Abel
ecbec1fe50
fix(backend): improve schedule queries plan to leverage indices better for performance ( #5273 )
...
* backend: fix schedule queries plan
* rework
* fix skipped as non success
* update sqlx
* update
* all
* fix npm run check
---------
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
2025-02-12 16:34:03 +01:00
Ruben Fiszel
5f46d79f26
fix: fix concurrent limit jobs non restarting
2025-02-10 11:33:19 +01:00
Ruben Fiszel
7bf5e381ac
fix: worker name in job + better timeout handling for same_worker jobs ( #5248 )
2025-02-08 22:20:06 +01:00
Lucas Abel
17a86e6907
backend: fix workflow as code ( #5239 )
...
* backend: improve `/get_job_update` after v2
* backend: insert missing `workflow_as_code_status` on completion
also insert `flow_status` from so we can remove the query when `_duration` is above 500
* backend: fix workflow_as_code after v2
* backend: add `workflow_as_code` worker test
2025-02-07 12:27:43 +01:00
Lucas Abel
6ded2c598e
backend: use v2 tables through views where possible (v2 phase 3) ( #5119 )
2025-02-06 12:43:24 +01:00
Lucas Abel
8374377958
backend: check queries to queue and completed_job where possible (v2 phase 2) ( #5125 )
2025-02-06 12:39:32 +01:00
Lucas Abel
cecc19dafc
backend: migrate job tables to v2 schema (v2 phase 1) ( #5084 )
2025-02-06 12:37:11 +01:00
Ruben Fiszel
098cd72840
fix uv install for nsjail ( #5222 )
2025-02-06 00:00:33 +01:00
Ruben Fiszel
dd3fb0f955
improve error messages for internal err
2025-02-05 16:32:01 +01:00
dieriba
900a8d675e
feat: postgres triggers ( #4860 )
...
* feat: init database triggers
* feat: ✨ wip: database_triggers
* feat: ✨ add database triggers front view
* feat: 🚧 database_triggers
* feat: 🚧 add definition in yaml, updated backend code and added migration
* feat: 🚧 updated migration file, update openapi.yml, updated database_triggers page and backend function
* fix: struct rust
* feat: 🚧 update migrate, database trigger backend function fixed
* feat: 🚧 add resource picker front, update backend function
* feat: 🚧 edit inner database inner, update triggers
* feat: 🚧 database_triggers
* feat: 🚧 update openapi yaml, prettied websocker trigger
* feat: 🚧 database_triggers
* feat: 🚧 add resource module, update variable file, working on main loop for database_trigger
* feat: 🐛 working sqlx query
* feat: 🚧 fix query with sqlx, added main loop
* feat: 🚧 add new column database_trigg
* feat: 🛂 run jobs works
* feat: 🚧 handling slot name and replication
* feat: restructring triggers, decoding trigger message on work
* feat: 🚧 database_trigger
* feat: 🚧
* feat: 🚧 converter done, work on custom script
* feat: 🚧 multiple trigger
* feat: 🚧 adding new argument function
* feat: 🚧 database_trigger
* feat: 🚧 add generate template for front, update script picker
* feat: 🚧 template script fix bug, work on restructing backend logic
* feat: 🚧 update autogenerated script, add persistence state for template script
* feat: 🚧 update structure client
* feat: 🚧 rewrited crud function
* feat: 🚧 added publication handler
* feat: 🚧 new ui finished
* feat: 🚧 added slot function hanlder finish front ux
* feat: ✨ ux improvement done, backend logic done
* feat: 🐛 fix where clause
* feat:
* feat:
* chore: update .sqlx and remove empty package
* fix: update publication and remove unneccessary print
* feat: finish converter to json, remove save button, remove unused crate
* feat: update migration for database trigger, fixed query, fixed frontend error with missing feature on back
* chore: update .sqlx
* chore: update sqlx
* chore: .sqlx
* chore: add unused
* fix: use right database resource in back and front
* fix build
* refactor:
* fix sqlx
* nits: retrieve all script from database trigger editor
* fix: template script object, fix route name
* merge main
* feat: rework ux
* feat/finish ux and update backend logic
* feat: database_trigger
* feat: database_trigger
* chore: update sqlx
* chore: fix ci
* nits: fix worker.rs
* feat: database_trigger
* Update frontend/src/lib/components/triggers/database/DatabaseTriggersPanel.svelte
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* feat:
* fix ci
* nits: database to postgres
* nits: fix .sqlx
* feat: add link to original code and open tab functionallity
* nits: update .sqlx
* nits: fix request type and yaml file
* nits: add abel suggestion
* fix: update .sqlx
* fix: update .sqlx
* nits: add comment to hex.rs
* nits: remove features, rename database to postgres name
* nits: replace database name by postgres
* nits: database name to postgres
* nits: update .sqlx
* nits: update .sqlx
* nit
* nit
* nits
* nit
* nits
* nits
* nits
* nits
* fix sqlx
* nits: update .vscode
---------
Co-authored-by: HugoCasa <hugo@casademont.ch >
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
2025-01-24 15:12:41 +01:00
Lucas Abel
7591edea4c
backend: always prioritize flow steps ( #5108 )
...
When a flow start, his flow steps have lower priority than already scheduled jobs, hence steps are only executed when the queue is drained which considerably increase flow execution time when queue size is significant.
This add a very low priority to flow steps in order to prioritize finishing a started flow over new jobs.
2025-01-21 09:27:40 +01:00
HugoCasa
175379fbe9
feat: on behalf of ( #5058 )
...
* feat: on behalf of
* fix
* revert mistakke
* temp ee ref
* renaming + CLI
* sqlx
* nits + ee ref
2025-01-14 18:32:25 +01:00
HugoCasa
a2f05d0e36
prevent cancelling schedule next tick + remove need of flow_status inside add_completed_job ( #5022 )
2025-01-13 14:26:28 +01:00
HugoCasa
a6f78aed81
fea(backend): return 500 on sync webhook/route error ( #5049 )
...
* fea(backend): return 500 on sync webhook/route error
* fix sqlx
2025-01-13 12:25:06 +01:00
Lucas Abel
3f0a818fa3
backend: do not force get_schedule_opt to be within a transaction ( #5019 )
2025-01-06 09:36:57 +01:00
Lucas Abel
dede390fab
fix(backend): fix has_failure_module only looking for raw_flow ( #5018 )
2025-01-06 09:21:17 +01:00
Lucas Abel
12d18beb32
cache: better cache system ( #4951 )
...
* cache: consider source as untrusted
* cache: better scoped cache
* cache: non-lazy flow value parse error
* fixup! cache: consider source as untrusted
* fixup! cache: consider source as untrusted
* fixup! cache: better scoped cache
2024-12-20 11:19:57 +01:00
Lucas Abel
dada363097
fix: fix redeploying flows with attached schedules
2024-12-18 18:56:30 +01:00
Lucas Abel
8502ff8225
pass cargo check ( #4949 )
2024-12-18 16:21:37 +01:00
Lucas Abel
e32bcd3383
backend: pass cargo check --all-features ( #4946 )
2024-12-18 12:37:31 +01:00
Ruben Fiszel
a857314f59
fix: fix relative imports in cached flow scripts
2024-12-18 11:34:07 +01:00
Lucas Abel
dab7b39917
feat(cache): remove persistent raw values from queue ( #4866 )
...
* feat(cache): remove persistent raw values from queue
* test: add tests and fix discovered regressions
* fix csharp executor after rebase
* invalidate script cache after locking
* fix lite flow version race
* fix flow dependencies lock race
* update version flag to `v.1.440`
2024-12-16 17:10:16 +01:00
HugoCasa
aaae1ea642
feat: add multipart/form-data support ( #4927 )
...
* feat: add multipart/form-data support
* nits
2024-12-15 10:53:12 +01:00
Ruben Fiszel
75ec668276
nit
2024-12-15 01:00:44 +01:00
Lucas Abel
c430c95886
cleanup is_flow usage and impl ( #4923 )
2024-12-13 18:08:01 +01:00
Ruben Fiszel
bd56ab8d5f
fix: flow node default tag
2024-12-13 17:51:08 +01:00
HugoCasa
a8d95434cc
feat: accept direct file upload for webhook/http (s3) ( #4903 )
...
* feat: accept direct file upload for webhook/http (s3)
* nit
* nit
2024-12-12 00:33:19 +01:00
Lucas Abel
ed71efa6da
fmt: re-format Rust leftovers ( #4897 )
2024-12-11 08:57:00 +01:00
Ruben Fiszel
8a5ba115c6
feat: add otlp support ( #4869 )
...
* all
* all
* add otel
* add otel
* update docker-image
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
* update
2024-12-07 16:29:20 +01:00
Lucas Abel
ef08fc8e29
feat: add db storage for app inline scripts ( #4837 )
2024-12-06 13:37:45 +01:00
Ruben Fiszel
19946deb82
simplify warn_after_seconds
2024-12-04 18:28:28 +01:00
Lucas Abel
0a65b90049
feat(cache): implement flow node caching ( #4808 )
...
* feat(cache): implement flow node caching
* feat(cache): implement script caching
* feat(cache): improve cache
2024-11-29 09:35:00 +01:00
Lucas Abel
4e52e54314
feat(backend): store flow inner modules into flow_node table ( #4778 )
2024-11-27 16:08:31 +01:00
Ruben Fiszel
e3ee592a2f
nit
2024-11-25 23:32:14 +01:00
Lucas Abel
ce121fcf37
feat(backend): implement flow scripts ( #4748 )
...
* feat(backend): implement flow scripts
* conditionally resolve reduced flow value
* conditionally use flow scripts based on the min worker version
* grant all to `flow_{version_lite,node}`
* do not silently discard errors
* env: add flow script feature flag
* flow script on cloud automatically
---------
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com >
2024-11-25 23:20:51 +01:00
