Commit Graph

1640 Commits

Author SHA1 Message Date
Ruben Fiszel
e60a4b251e feat(backend): cache hub scripts in the worker cache 2023-05-09 12:45:54 +02:00
Ruben Fiszel
796b346bfb add cairo to openbb build 2023-05-09 11:11:09 +02:00
Ruben Fiszel
baacfd00e4 feat(backend): in python, if a value is bytes, it is encoded to base64 automaticaly 2023-05-09 09:36:07 +02:00
Ruben Fiszel
143a30661c feat: add support for full fleged apps (react, svelte, vue) (#1536)
* add react support

* use whoami

* update

* update

* update

* create react app

* all

* sqlx
2023-05-08 19:10:45 +02:00
Ruben Fiszel
62fb31ad28 fix default value in python 2023-05-05 10:03:28 +02:00
Ruben Fiszel
96ff45ecb5 feat(backend): default parameters are used in python if missing from args 2023-05-05 09:44:53 +02:00
Ruben Fiszel
f3861ebd2f fix(backend): fix python transformers 2023-05-03 23:27:14 +02:00
Ruben Fiszel
e21208de31 feat(backend): add EXIT_AFTER_NO_JOB_FOR_SECS for ephemeral workers 2023-04-28 00:09:48 +02:00
Ruben Fiszel
1764c4114e feat(backend): expose tag in the job (#1486)
* fix

* update tag

* update tag

* update tag

* fix
2023-04-27 19:09:53 +02:00
Ruben Fiszel
1d1c812e36 feat(backend): add JOB_RETENTION_SECS to delete completed jobs completed after expiration period 2023-04-26 19:34:54 +02:00
Ruben Fiszel
02ba82064f feat(backend): worker groups (#1452)
* tagged scripts

* tagged scripts

* fix

* fix

* fix

* all

* update sqlx
2023-04-23 16:57:23 +02:00
Ruben Fiszel
c9d6a47b31 fix(backend): avoid potential conflict between pull from tar and background sync 2023-04-21 23:49:14 +02:00
Ruben Fiszel
a3487f18c4 fix(backend): global cache synco only start if all piptars have been downloaded 2023-04-21 22:21:04 +02:00
Ruben Fiszel
1ba0e24155 feat(backend): only run fully deployed scripts 2023-04-21 12:49:47 +02:00
Ruben Fiszel
394c6fa92d feat(backend): workers are instantly ready and sync with global cache in background 2023-04-21 09:45:25 +02:00
Ruben Fiszel
d8ceaa5bcc improve log line for cache syncing 2023-04-21 01:54:58 +02:00
Ruben Fiszel
dd6602284e fix untar_all_piptars 2023-04-21 01:02:51 +02:00
Ruben Fiszel
ecca1e2c3e better rclone filters 2023-04-21 00:31:39 +02:00
Ruben Fiszel
ee1599166c better rclone filters 2023-04-20 23:45:31 +02:00
Ruben Fiszel
d8da7b2883 set relative folder for archive creation 2023-04-20 23:33:53 +02:00
Ruben Fiszel
8fda8bbaf8 remove deno/gen/windmill from tmp cache 2023-04-20 21:04:17 +02:00
Ruben Fiszel
e74f8f8248 --s3-no-cache-buckets 2023-04-20 20:50:02 +02:00
Ruben Fiszel
242b2ae899 feat(backend): global cache refactor for pip using tar for each dependency (#1443)
* cache refactor

* exclude tar from being synced to bucket

* run

* update

* update
2023-04-20 20:05:12 +02:00
Ruben Fiszel
ccda9850b6 fix python install on save with nsjail 2023-04-20 10:33:11 +02:00
Ruben Fiszel
f0b939cd3a add barrier when num workers > 1 2023-04-19 17:00:03 +02:00
Ruben Fiszel
26c7e9e8c3 add barrier when num workers > 1 2023-04-19 12:46:25 +02:00
Ruben Fiszel
fb0983bcb3 feat(ee): sync cache in background 2023-04-19 01:15:41 +02:00
Ruben Fiszel
89d0bea9fa feat(ee): sync cache in background 2023-04-19 00:55:51 +02:00
Ruben Fiszel
4934dc454f only cache on first worker 2023-04-18 21:23:29 +02:00
Ruben Fiszel
5e2ca89137 log how much time took each command child 2023-04-18 08:55:26 +02:00
Ruben Fiszel
66c532affc improve bash output buffering 2023-04-18 01:42:28 +02:00
Ruben Fiszel
732d61f190 performance: process completed job in background (#1396)
* compile

* improve
2023-04-15 11:06:51 +02:00
Ruben Fiszel
556dbce239 feat(backend): install python scripts on save 2023-04-13 22:57:53 +02:00
Ruben Fiszel
c2b66fef4f do a test run on first iteration 2023-04-13 21:54:42 +02:00
Ruben Fiszel
be41e45130 split worker.rs into more files 2023-04-13 17:44:47 +02:00
Ruben Fiszel
94098c23e0 fix refreshOn 2023-04-13 09:22:52 +02:00
Ruben Fiszel
90ce302f2d fix(backend): deno uses --no-check 2023-04-13 00:22:46 +02:00
Ruben Fiszel
4dc1773dc7 fix(backend): do not fail on schedule not existing anymore 2023-04-12 20:06:50 +02:00
Ruben Fiszel
53c4c744c4 exclude deno/gen/file from global cache 2023-04-12 19:18:48 +02:00
Ruben Fiszel
e8611ba8c1 add GOPRIVATE and NETRC for private modules 2023-04-11 19:55:38 +02:00
Ruben Fiszel
c4366b9ef4 fix relative url deno loading2 2023-04-11 15:57:59 +02:00
Ruben Fiszel
df8236a03f fix(backend): nested deno relative imports 2023-04-11 15:16:55 +02:00
Kai Jellinghaus
63102b036c feat(backend): Redis based queue (#1324)
* Merge?

* Fix V8 breaking change

* WIP

* WIP

* Cleanup

* Move to git reference

* Fix Merge conflict

* update

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2023-04-11 12:24:13 +02:00
Ruben Fiszel
66d2ae813a fix tests + scheduled_for uses now instead of system time 2023-04-11 11:48:14 +02:00
Ruben Fiszel
613cd64003 improve metrics 2023-04-10 11:58:44 +02:00
Ruben Fiszel
621bb282af improve metrics 2023-04-10 11:53:32 +02:00
Ruben Fiszel
415be7c70f perf: parallelize more operations for deno jobs 2023-04-10 10:55:56 +02:00
Ruben Fiszel
b7b26dc9c7 improve token lock 2023-04-10 08:00:41 +02:00
Ruben Fiszel
6ab4f30026 fetch token in the background 2023-04-10 01:16:15 +02:00
Ruben Fiszel
a23a14f9f2 make create_token_for_owner not a transaction anymore 2023-04-09 21:47:05 +02:00