Commit Graph

5945 Commits

Author SHA1 Message Date
Ruben Fiszel
e96c5ca670 early stop now bubble up to the top 2022-10-30 02:32:35 +02:00
Ruben Fiszel
bb4a620645 revert switching metrics to tiny-http 2022-10-30 01:55:44 +02:00
Ruben Fiszel
12de3e700f handle more error edge cases 2022-10-29 13:59:50 +02:00
Ruben Fiszel
6ceac4280e feat(python): type is automatically inferred from default parameters 2022-10-29 12:48:50 +02:00
Kai Jellinghaus
6263e90735 restructure the entire backend layout using workspaces (#815) 2022-10-29 11:58:06 +02:00
Ruben Fiszel
1936a60b4a UX flow improvements + fix collecting results for loops in frontend + fix Identity behavior in backend 2022-10-27 18:50:32 +02:00
Ruben Fiszel
dc103696a7 update resource spec 2022-10-27 00:59:25 +02:00
Ruben Fiszel
574df78421 fix export to tarball 2022-10-27 00:43:51 +02:00
Ruben Fiszel
256612bbc5 improve python parsing of imports 2022-10-27 00:19:17 +02:00
Ruben Fiszel
14fb4551f4 feat: payload capture of json to initialize flow input (#655)
* capture table and some endpoints for json pastebin

* only retain user's most recent captures

* merge conflict

* sqlx data

* fix migration

Co-authored-by: sqwishy <somebody@froghat.ca>
2022-10-25 00:54:53 +02:00
Ruben Fiszel
786e59af1f chore(deps): backend update 2022-10-24 21:28:25 +02:00
Ruben Fiszel
16afde6806 approval flow with idendity of the approver 2022-10-24 16:37:09 +02:00
Ruben Fiszel
eae96274e3 add identity job_kind to openapi 2022-10-24 14:51:25 +02:00
Ruben Fiszel
02f4113234 add resume_id to job_resume + add identity module 2022-10-24 14:48:28 +02:00
Ruben Fiszel
3ad64ba6eb fix get_suspended_job_flow module finding 2022-10-24 10:37:21 +02:00
Ruben Fiszel
da71358090 fix get_suspended_job_flow 2022-10-24 10:18:23 +02:00
Ruben Fiszel
1353c7b8da chore(main): release 1.41.0 (#795)
* chore(main): release 1.41.0

* Apply automatic changes

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2022-10-24 02:38:12 +02:00
Ruben Fiszel
77c114f037 fix(deno-client): add approver 2022-10-24 02:24:44 +02:00
Ruben Fiszel
7a4bcd98f6 feat: add approver to approval step 2022-10-24 01:50:39 +02:00
Ruben Fiszel
b942f0b603 chore(main): release 1.40.1 (#794)
* chore(main): release 1.40.1

* Apply automatic changes

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2022-10-22 16:21:10 +02:00
Ruben Fiszel
81549837ba chore(main): release 1.40.0 (#780)
* chore(main): release 1.40.0

* Update CHANGELOG.md

* Apply automatic changes

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2022-10-22 14:06:23 +02:00
Ruben Fiszel
a9359dada7 fix: improve approval flow with approval page 2022-10-22 13:55:12 +02:00
Ruben Fiszel
f8fce1fa78 fix send tx for kill 2022-10-22 00:34:36 +02:00
Ruben Fiszel
960b701cf6 feat(backend): propagate cancel instantly to all flow jobs if any 2022-10-21 22:01:07 +02:00
Ruben Fiszel
68a0376fc3 feat(backend): propagate cancel instantly to all flow jobs if any 2022-10-21 22:00:52 +02:00
Ruben Fiszel
a9bf7a4835 fix import script from hub 2022-10-21 19:43:53 +02:00
Ruben Fiszel
26cd5ec6f2 fix import script from hub 2022-10-21 19:38:33 +02:00
Kai Jellinghaus
93b1432625 fix(backend): avoid mem leak on interval #786 2022-10-21 12:15:03 +02:00
Ruben Fiszel
6d4d8e2d64 feat: support running and publishing go, python scripts to the hub (#779) 2022-10-20 20:16:08 +02:00
Ruben Fiszel
71a75a57b6 chore(main): release 1.39.0 (#747)
* chore(main): release 1.39.0

* Apply automatic changes

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2022-10-20 16:38:34 +02:00
Kai Jellinghaus
438563d5ea Add more tracing (#773) 2022-10-20 07:40:58 +02:00
Kai Jellinghaus
1537a1b756 Cargo config improvements (#775)
* Move cargo config to correct location

* Add --no-rosegment to build params.
This helps with stack traces, and is necessary for some profilers.
2022-10-20 07:38:49 +02:00
Kai Jellinghaus
bce59a197e More Metrics (#735)
* Expose Worker Tokio Metrics

* Add metrics tracking the job queue

This can be used to (approximate) the queue length
This can be used to (approximate) the running jobs

* Remove testing code

* Rename metrics to exclude TOTAL_

* Sleep in worker metrics loop

* Add jobs_executed

* Add zombie job metrics

* Add uptime metric

* Fix metric naming

* Remove poll stats

* unify worker execution metrics

* Rename jobs_* metrics

* Rename metrics to match exposed name

* Remove leftover import

* Rename variables to match further

* Fix Merge error

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2022-10-19 10:15:42 +02:00
Ruben Fiszel
d2fd81e542 remove erronous pull created_by 2022-10-19 04:10:41 +02:00
Ruben Fiszel
dd33179d70 make get job more reliable 2022-10-19 03:54:41 +02:00
Ruben Fiszel
b728041770 nits 2022-10-19 03:41:56 +02:00
Ruben Fiszel
d1657383c7 feat(backend): atomic moving queue -> complete and delete (#771)
* feat(backend): atomic move to queue + delete

* fix

* fix

* fix

* fix
2022-10-19 03:16:15 +02:00
Ruben Fiszel
8aa4c7691b fix(backend): reschedule flow at first step end (#746)
* fix(backend): reschedule flow at first step end

* fix

* fix

* fix

* fix

* sqlx prepare
2022-10-18 17:06:39 +02:00
Ruben Fiszel
d1780c8c6b feat: add result by id to fetch result from any node (#769)
* feat: add result by id to fetch result from any node

* sqlx prepare

* fix

* fix

* review
2022-10-18 17:04:13 +02:00
Ruben Fiszel
584f168f8f feat: add ids to modules + input_transform lowered to flowmodulevalue (#768)
* feat: add ids to modules + inputTransforms from module to module value

* all

* delete obsolete input transforms
2022-10-17 22:34:12 +02:00
dependabot[bot]
2d69b42441 chore(deps): bump magic-crypt from 3.1.10 to 3.1.12 in /backend (#762)
Bumps [magic-crypt](https://github.com/magiclen/rust-magiccrypt) from 3.1.10 to 3.1.12.
- [Release notes](https://github.com/magiclen/rust-magiccrypt/releases)
- [Commits](https://github.com/magiclen/rust-magiccrypt/compare/v3.1.10...v3.1.12)

---
updated-dependencies:
- dependency-name: magic-crypt
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-17 19:12:52 +00:00
dependabot[bot]
261d7147c3 chore(deps): bump swc_ecma_parser from 0.122.7 to 0.122.13 in /backend (#758)
Bumps [swc_ecma_parser](https://github.com/swc-project/swc) from 0.122.7 to 0.122.13.
- [Release notes](https://github.com/swc-project/swc/releases)
- [Changelog](https://github.com/swc-project/swc/blob/main/CHANGELOG.md)
- [Commits](https://github.com/swc-project/swc/commits)

---
updated-dependencies:
- dependency-name: swc_ecma_parser
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-17 18:46:00 +00:00
Ruben Fiszel
5f3646fb37 feat(backend): add branch all (#751)
* feat(backend): add branch all

* progress

* all
2022-10-17 17:31:31 +02:00
Ruben Fiszel
ffad0c30f0 feat: InProgress forloop_jobs -> flow_jobs to unify with branchAll 2022-10-16 16:04:24 +02:00
Ruben Fiszel
4929a50286 feat(backend): rework forloop flow job arg passing + reimplement branchone using flows 2022-10-16 15:56:41 +02:00
Ruben Fiszel
9fe4229b63 chore(main): release 1.38.5 (#745)
* chore(main): release 1.38.5

* Apply automatic changes

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2022-10-15 18:37:41 +02:00
Ruben Fiszel
85c41372be chore(main): release 1.38.4 (#744)
* chore(main): release 1.38.4

* Apply automatic changes

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2022-10-15 18:27:04 +02:00
Jakub Kołodziejczak
89e6b1a63a chore(backend): allow disabling nsjail for unit tests (#742)
UC: as a developer I want to be able to test code w/o nsjail

bug: flaky test `test_go_job`
```
running 1 test
test worker::tests::test_go_job ... FAILED

failures:

---- worker::tests::test_go_job stdout ----
received killpill for worker 0
thread 'worker::tests::test_go_job' panicked at 'assertion failed: `(left == right)`
  left: `Object {"error": String("Error during execution of the script:\n\ngo: error obtaining buildID for go tool compile: pipe2: too many open files\ngo: error obtaining buildID for go tool compile: pipe2: too many open files\ngo: error obtaining buildID for go tool compile: pipe2: too many open files")}`,
 right: `String("hello world")`', src/worker.rs:2434:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```

workaround: disable nsjail
2022-10-15 16:08:05 +02:00
Ruben Fiszel
9aabb163ad chore(main): release 1.38.3 (#740)
* chore(main): release 1.38.3

* Apply automatic changes

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2022-10-15 11:35:49 +02:00
Ruben Fiszel
133a661d88 chore(main): release 1.38.1 (#734)
* chore(main): release 1.38.1

* Apply automatic changes

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2022-10-15 08:49:34 +02:00