Commit Graph

677 Commits

Author SHA1 Message Date
Ruben Fiszel
a55aad3003 feat: flow concurrency limits support custom concurrency key 2024-04-09 22:03:39 +02:00
Ruben Fiszel
a4442d4d3d fix: improve performance of for-loops with parralelism 2024-04-09 19:47:50 +02:00
Ruben Fiszel
29422f156d fix: improve performance of for-loops with parralelism 2024-04-09 19:41:56 +02:00
Ruben Fiszel
8cddef62f2 prevent parallelism from timing out lock monitor 2024-04-09 18:50:19 +02:00
Ruben Fiszel
0c824de4cd fix: improve deadlocks for parallel branches with parallelism constraints 2024-04-09 11:52:40 +02:00
Ruben Fiszel
74631922f2 more error messages 2024-04-09 10:27:14 +02:00
Ruben Fiszel
8dd117528c fix: improve deadlocks for parallel branches with parallelism constraints 2024-04-09 01:38:55 +02:00
Ruben Fiszel
9131d5cc40 fix: parallel flow with parallelism constraint could deadlock 2024-04-06 13:25:34 +02:00
Ruben Fiszel
d0a0dd0f7e nit global cache python improvements 2024-04-04 22:07:14 +02:00
Ruben Fiszel
0b3e6b9f1c fix: improve global cache deps 2024-04-04 21:33:44 +02:00
Ruben Fiszel
2e3b6f66e3 fix: improve chars splitting for logs with utf-8 chars 2024-04-04 13:38:19 +02:00
Ruben Fiszel
826757b33a fix: fix char excess size module for logs truncation 2024-04-04 03:25:11 +02:00
Ruben Fiszel
5594afd0a2 add checksum log to s3 piptar pull 2024-04-02 20:55:31 +02:00
Ruben Fiszel
ff26c8e42d feat: while loop as new flow primitive 2024-04-02 17:38:11 +02:00
Ruben Fiszel
cc5d1b405e continue on error option for steps 2024-04-02 10:32:47 +02:00
Ruben Fiszel
5fdfa28cb9 fix: improve git sync for script renames 2024-03-26 18:21:58 +01:00
Ruben Fiszel
ef1ce83279 feat: add set_flow_user_states and get_flow_user_states api and sdk support 2024-03-26 17:31:17 +01:00
Ruben Fiszel
c15eeff3ea improve vacuum queue 2024-03-24 17:40:21 +01:00
Ruben Fiszel
e51c63d81c improve vacuum queue 2024-03-24 17:28:15 +01:00
Ruben Fiszel
d2afc05703 fix: add support for azure storage on distributed cache/logs 2024-03-23 18:42:38 +01:00
Ruben Fiszel
6bf20838ae feat(frontend): fetch logs just-in-time only when necessary 2024-03-23 15:57:23 +01:00
HugoCasa
283d55008c feat: add workspace environment vars (custom contextual vars) (#3455)
* feat: custom contextual variables

* chore: sqlx prepare

* fix: main merge
2024-03-23 11:40:55 +01:00
Ruben Fiszel
4bada8a69f nit newline 2024-03-23 11:33:10 +01:00
Ruben Fiszel
75e9e67d7a feat: large log disk and distributed storage compaction 2024-03-23 10:55:29 +01:00
HugoCasa
522f32c6c0 feat keep sql columns ordering (#3444)
* feat: keep pg columns ordering

* feat: pass column order through flow status

* feat: sql ordering for all sql langs

* fix: sqlx build on mac + rename to _metadata

* Update JobPreview.svelte

* Update +page.svelte

* Update +page.svelte

* Update LogPanel.svelte

* Update +page.svelte

* Update LogPanel.svelte

* Update JobPreview.svelte

* fix: build

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2024-03-22 10:34:32 +01:00
Ruben Fiszel
030f6c553e generalize cache object store 2024-03-21 08:50:10 +01:00
Ruben Fiszel
eeca92f100 fix compile on non ee 2024-03-20 16:55:06 +01:00
Ruben Fiszel
94ddd4ba6d fix compile on non ee 2024-03-20 16:41:23 +01:00
Ruben Fiszel
52d00d9c58 fix compile on non ee 2024-03-20 15:34:25 +01:00
Ruben Fiszel
5e656c52ea remove tmpcache 2024-03-20 15:32:55 +01:00
Ruben Fiszel
72c105268e fix compile on non ee 2024-03-20 15:18:54 +01:00
Ruben Fiszel
745bd4d170 fix compile on non ee 2024-03-20 15:09:28 +01:00
Ruben Fiszel
174ead0473 feat: s3 cache config added to instance settings + parallelized tar pulling 2024-03-20 14:19:34 +01:00
HugoCasa
4557e7beb4 fix: ai prevent python async + max db schema length (#3440) 2024-03-19 08:17:51 +01:00
Ruben Fiszel
5689567585 avoid caching binary on cloud 2024-03-17 19:13:51 +01:00
Ruben Fiszel
ca88047312 fix: allow multiple PIP_SECRETS 2024-03-17 15:15:36 +01:00
Ruben Fiszel
b3e53de947 removing pulling all piptars at start 2024-03-17 11:21:43 +01:00
Ruben Fiszel
e0235d9e9e fix: disapproval does not trigger flow error handler anymore 2024-03-17 11:08:02 +01:00
Ruben Fiszel
57c0c95fca log nits 2024-03-16 19:01:58 +01:00
Ruben Fiszel
ffbfba9323 bash nits 2024-03-16 18:52:17 +01:00
Ruben Fiszel
1e9dc754d2 nits logging 2024-03-16 18:30:51 +01:00
Ruben Fiszel
e5366b84bb feat: refactor logging to improve performance by order of magnitude for longer jobs 2024-03-16 17:55:34 +01:00
Ruben Fiszel
ae661365e4 fix: s3 cache is an instance settings (#3421)
* s3

* s3
2024-03-15 15:09:05 +01:00
Ruben Fiszel
71f7299b6c fix: improve pip_secret interpolation 2024-03-15 11:17:41 +01:00
Ruben Fiszel
89516b922f fix: avoid unecessary re-schedule for retyied flows 2024-03-15 09:53:58 +01:00
Ruben Fiszel
e5f7bcf1a6 fix jobs metrics 2024-03-14 19:53:38 +01:00
Ruben Fiszel
47d96c0d97 add NO_LOGS_AT_ALL 2024-03-14 19:47:46 +01:00
Ruben Fiszel
df910d7441 fix: be less agressive with log streaming for long jobs 2024-03-14 19:42:41 +01:00
Ruben Fiszel
97d4f1cc26 feat: better locking (#3412)
* earlyBackendStop

* fix(frontend): Flow step input description (#3409)

* fix(frontend): Add missing descrpition in a flow step input

* fix(frontend): Add missing descrpition in a flow step input

* fix(frontend): Fix color picker layout (#3411)

* feat: add bit support in pg (#3407)

* fix sqlx

* fix: remove dependency on semver intersect for bun

* sqlx fix

* improve early kill behavior

---------

Co-authored-by: Faton Ramadani <faton.ramadani14@gmail.com>
Co-authored-by: HugoCasa <hugo@casademont.ch>
2024-03-14 13:57:54 +01:00
HugoCasa
3730566844 feat: add bit support in pg (#3407) 2024-03-14 10:40:20 +01:00