Commit Graph

725 Commits

Author SHA1 Message Date
Ruben Fiszel
1eeaa277f0 add base_internal_url to dedicated python workers 2024-04-30 22:48:40 +02:00
Ruben Fiszel
fa62a93c9f add stderr to debug log of dedicated workers 2024-04-30 21:10:05 +02:00
Ruben Fiszel
eb873c6119 more logs for dedicated workers 2024-04-30 17:19:34 +02:00
Ruben Fiszel
cf6ffef48a fix(python): supports folders starting with numbers for execution 2024-04-30 11:23:48 +02:00
Ruben Fiszel
1f983c2414 fix: fix console logs in REST scripts 2024-04-30 10:46:20 +02:00
HugoCasa
3956d012ad feat: add resource and variable picker for rest scripts (#3628)
* feat: add resource and variable picker for rest scripts

* feat: add reserved variables to rest scripts
2024-04-29 12:13:21 +02:00
Ruben Fiszel
22226e883e feat(cli): split inline sscripts for apps like for flows 2024-04-28 14:55:33 +02:00
Ruben Fiszel
de2bea613a rm go bin if cloud hosted 2024-04-26 13:42:24 +02:00
Ruben Fiszel
e3869693ec rm go bin if cloud hosted 2024-04-26 13:40:52 +02:00
HugoCasa
fee22fc060 feat: pg add json support (#3620) 2024-04-26 13:28:15 +02:00
Ruben Fiszel
c58342ddfd fix backend tests 2024-04-26 13:27:24 +02:00
Ruben Fiszel
45cad34222 small fixes 2024-04-26 11:38:44 +02:00
Ruben Fiszel
4188383d3e feat: add distributed global cache for go 2024-04-26 11:31:21 +02:00
HugoCasa
ae7f978180 feat: add support for readonly application intent in mssql (#3591) 2024-04-23 14:08:51 +02:00
Ruben Fiszel
48ba709627 fix: improve re-scheduled for estimation in concurrency limits 2024-04-21 12:41:30 +02:00
Ruben Fiszel
5504114732 increase pg size limit for cloud 2024-04-21 10:20:50 +02:00
wendrul
ac841400ba Instrument logs (#3566)
* Add spans to show worker_name and job_id

Always show worker_name and job_id when relevant, by creating spans that
live as long as the worker and the job.

* Remove redundant worker names from fn run_worker

Remove the now redundant worker_name tracing, as the information now
lives at the span level

* Use span when spawning new thread

* Instrument more pertinent functions

Move spans to more pertinent functions. Remove more redundant info, and
change worker_name to just worker

* Change tracing subscriber to log path:line_no

Change from logging target (module name usually) to file path and line
number
2024-04-18 01:11:14 +02:00
Ruben Fiszel
f84a902445 fix: measure memory usage on postgres scripts 2024-04-18 01:05:16 +02:00
Ruben Fiszel
071a0ae927 feat: show more for logs on s3 directly possible from browser log viewer 2024-04-17 21:42:46 +02:00
HugoCasa
7c85cd808d fix: add mysql decimal support (#3557) 2024-04-15 14:26:17 +02:00
Ruben Fiszel
a55aad3003 feat: flow concurrency limits support custom concurrency key 2024-04-09 22:03:39 +02:00
Ruben Fiszel
a4442d4d3d fix: improve performance of for-loops with parralelism 2024-04-09 19:47:50 +02:00
Ruben Fiszel
29422f156d fix: improve performance of for-loops with parralelism 2024-04-09 19:41:56 +02:00
Ruben Fiszel
8cddef62f2 prevent parallelism from timing out lock monitor 2024-04-09 18:50:19 +02:00
Ruben Fiszel
0c824de4cd fix: improve deadlocks for parallel branches with parallelism constraints 2024-04-09 11:52:40 +02:00
Ruben Fiszel
74631922f2 more error messages 2024-04-09 10:27:14 +02:00
Ruben Fiszel
8dd117528c fix: improve deadlocks for parallel branches with parallelism constraints 2024-04-09 01:38:55 +02:00
Ruben Fiszel
9131d5cc40 fix: parallel flow with parallelism constraint could deadlock 2024-04-06 13:25:34 +02:00
Ruben Fiszel
d0a0dd0f7e nit global cache python improvements 2024-04-04 22:07:14 +02:00
Ruben Fiszel
0b3e6b9f1c fix: improve global cache deps 2024-04-04 21:33:44 +02:00
Ruben Fiszel
2e3b6f66e3 fix: improve chars splitting for logs with utf-8 chars 2024-04-04 13:38:19 +02:00
Ruben Fiszel
826757b33a fix: fix char excess size module for logs truncation 2024-04-04 03:25:11 +02:00
Ruben Fiszel
5594afd0a2 add checksum log to s3 piptar pull 2024-04-02 20:55:31 +02:00
Ruben Fiszel
ff26c8e42d feat: while loop as new flow primitive 2024-04-02 17:38:11 +02:00
Ruben Fiszel
cc5d1b405e continue on error option for steps 2024-04-02 10:32:47 +02:00
Ruben Fiszel
c265593099 fix powershell with nsjail 2024-03-26 23:11:26 +01:00
Ruben Fiszel
5fdfa28cb9 fix: improve git sync for script renames 2024-03-26 18:21:58 +01:00
Ruben Fiszel
ef1ce83279 feat: add set_flow_user_states and get_flow_user_states api and sdk support 2024-03-26 17:31:17 +01:00
Ruben Fiszel
c15eeff3ea improve vacuum queue 2024-03-24 17:40:21 +01:00
Ruben Fiszel
e51c63d81c improve vacuum queue 2024-03-24 17:28:15 +01:00
Ruben Fiszel
d2afc05703 fix: add support for azure storage on distributed cache/logs 2024-03-23 18:42:38 +01:00
Ruben Fiszel
6bf20838ae feat(frontend): fetch logs just-in-time only when necessary 2024-03-23 15:57:23 +01:00
HugoCasa
283d55008c feat: add workspace environment vars (custom contextual vars) (#3455)
* feat: custom contextual variables

* chore: sqlx prepare

* fix: main merge
2024-03-23 11:40:55 +01:00
Ruben Fiszel
4bada8a69f nit newline 2024-03-23 11:33:10 +01:00
Ruben Fiszel
75e9e67d7a feat: large log disk and distributed storage compaction 2024-03-23 10:55:29 +01:00
HugoCasa
522f32c6c0 feat keep sql columns ordering (#3444)
* feat: keep pg columns ordering

* feat: pass column order through flow status

* feat: sql ordering for all sql langs

* fix: sqlx build on mac + rename to _metadata

* Update JobPreview.svelte

* Update +page.svelte

* Update +page.svelte

* Update LogPanel.svelte

* Update +page.svelte

* Update LogPanel.svelte

* Update JobPreview.svelte

* fix: build

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2024-03-22 10:34:32 +01:00
Ruben Fiszel
030f6c553e generalize cache object store 2024-03-21 08:50:10 +01:00
Ruben Fiszel
eeca92f100 fix compile on non ee 2024-03-20 16:55:06 +01:00
Ruben Fiszel
94ddd4ba6d fix compile on non ee 2024-03-20 16:41:23 +01:00
Ruben Fiszel
52d00d9c58 fix compile on non ee 2024-03-20 15:34:25 +01:00