Commit Graph

17 Commits

Author SHA1 Message Date
Ruben Fiszel
c9d6a47b31 fix(backend): avoid potential conflict between pull from tar and background sync 2023-04-21 23:49:14 +02:00
Ruben Fiszel
a3487f18c4 fix(backend): global cache synco only start if all piptars have been downloaded 2023-04-21 22:21:04 +02:00
Ruben Fiszel
1ba0e24155 feat(backend): only run fully deployed scripts 2023-04-21 12:49:47 +02:00
Ruben Fiszel
394c6fa92d feat(backend): workers are instantly ready and sync with global cache in background 2023-04-21 09:45:25 +02:00
Ruben Fiszel
d8ceaa5bcc improve log line for cache syncing 2023-04-21 01:54:58 +02:00
Ruben Fiszel
dd6602284e fix untar_all_piptars 2023-04-21 01:02:51 +02:00
Ruben Fiszel
ecca1e2c3e better rclone filters 2023-04-21 00:31:39 +02:00
Ruben Fiszel
ee1599166c better rclone filters 2023-04-20 23:45:31 +02:00
Ruben Fiszel
d8da7b2883 set relative folder for archive creation 2023-04-20 23:33:53 +02:00
Ruben Fiszel
8fda8bbaf8 remove deno/gen/windmill from tmp cache 2023-04-20 21:04:17 +02:00
Ruben Fiszel
e74f8f8248 --s3-no-cache-buckets 2023-04-20 20:50:02 +02:00
Ruben Fiszel
242b2ae899 feat(backend): global cache refactor for pip using tar for each dependency (#1443)
* cache refactor

* exclude tar from being synced to bucket

* run

* update

* update
2023-04-20 20:05:12 +02:00
Ruben Fiszel
f0b939cd3a add barrier when num workers > 1 2023-04-19 17:00:03 +02:00
Ruben Fiszel
26c7e9e8c3 add barrier when num workers > 1 2023-04-19 12:46:25 +02:00
Ruben Fiszel
fb0983bcb3 feat(ee): sync cache in background 2023-04-19 01:15:41 +02:00
Ruben Fiszel
89d0bea9fa feat(ee): sync cache in background 2023-04-19 00:55:51 +02:00
Ruben Fiszel
be41e45130 split worker.rs into more files 2023-04-13 17:44:47 +02:00