Commit Graph

323 Commits

Author SHA1 Message Date
Ruben Fiszel
13be0cd1c8 fix: fix relative imports in cached flow scripts 2024-12-18 11:34:07 +01:00
Lucas Abel
977ac5c3f3 feat(cache): remove persistent raw values from queue (#4866)
* feat(cache): remove persistent raw values from queue

* test: add tests and fix discovered regressions

* fix csharp executor after rebase

* invalidate script cache after locking

* fix lite flow version race

* fix flow dependencies lock race

* update version flag to `v.1.440`
2024-12-16 17:10:16 +01:00
HugoCasa
83a60cbc51 feat: add multipart/form-data support (#4927)
* feat: add multipart/form-data support

* nits
2024-12-15 10:53:12 +01:00
Ruben Fiszel
b169ddb9d3 nit 2024-12-15 01:00:44 +01:00
Lucas Abel
74a89552c7 cleanup is_flow usage and impl (#4923) 2024-12-13 18:08:01 +01:00
Ruben Fiszel
3634ade41b fix: flow node default tag 2024-12-13 17:51:08 +01:00
HugoCasa
563a492008 feat: accept direct file upload for webhook/http (s3) (#4903)
* feat: accept direct file upload for webhook/http (s3)

* nit

* nit
2024-12-12 00:33:19 +01:00
Lucas Abel
e77ae1c25b fmt: re-format Rust leftovers (#4897) 2024-12-11 08:57:00 +01:00
Ruben Fiszel
6d047449e2 feat: add otlp support (#4869)
* all

* all

* add otel

* add otel

* update docker-image

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update
2024-12-07 16:29:20 +01:00
Lucas Abel
2bc4934c4f feat: add db storage for app inline scripts (#4837) 2024-12-06 13:37:45 +01:00
Ruben Fiszel
db62a2c53c simplify warn_after_seconds 2024-12-04 18:28:28 +01:00
Lucas Abel
3fbb2bfc8a feat(cache): implement flow node caching (#4808)
* feat(cache): implement flow node caching

* feat(cache): implement script caching

* feat(cache): improve cache
2024-11-29 09:35:00 +01:00
Lucas Abel
2911775d0a feat(backend): store flow inner modules into flow_node table (#4778) 2024-11-27 16:08:31 +01:00
Ruben Fiszel
2427d64aa8 nit 2024-11-25 23:32:14 +01:00
Lucas Abel
e4784e89da feat(backend): implement flow scripts (#4748)
* feat(backend): implement flow scripts

* conditionally resolve reduced flow value

* conditionally use flow scripts based on the min worker version

* grant all to `flow_{version_lite,node}`

* do not silently discard errors

* env: add flow script feature flag

* flow script on cloud automatically

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-11-25 23:20:51 +01:00
Ruben Fiszel
6768e5bbbd fix: retry on inserting completed job (#4784)
* fix: retry on pushing next scheduled job of schedule

* all

* improve error handling

* update sqlx
2024-11-25 01:33:44 +01:00
Ruben Fiszel
5ed0ae697b improve backoff for schedules 2024-11-24 18:48:01 +01:00
Ruben Fiszel
89456f502f remove rsmq 2024-11-24 11:04:54 +01:00
Ruben Fiszel
278f593358 fix: retry on pushing next scheduled job of schedule 2024-11-24 08:52:51 +01:00
Alexander Petric
8909bea935 feat(schedule): support for extended cron syntax (#4754)
* init

* croner with fall back on scheduling failure

* adding packages

* fallback mechanism v1

* removing debug console log

* improving cargo organization

* queue cargo.toml

* use workspace cargo-tz

* test next occurrence when creating new croner schedule

* toggle label

* treat legacy edit requests where no cron_version is provided as v1

* fix edit schedule
2024-11-21 19:24:38 +01:00
Lucas Abel
e6c7fe7027 backend: rework raw values fetching logic (#4770) 2024-11-21 13:59:42 +01:00
Lucas Abel
97457a5679 feat(backend): move some static fields out of job tables (#4689)
* feat(backend): move some static fields out of job tables

* backend: re-use `flow_value` if any and avoid clone

* update minimal version to 1.427.0
2024-11-19 11:27:55 +01:00
Alexander Petric
c32038a76d feat(monitoring): workspace critical alerts (#4684)
* critical alert ui

* updating ui, backend logic

* revert

* type check fix npm

* checking out cli files from main

* moving alert icon

* adding sqlx mock data

* more sqlx changes

* feat(frontend): nodes from flow can be connected directly in expr input through a plug icon (#4652)

* Add flow prop picker

# Conflicts:
#	frontend/src/lib/components/propertyPicker/PropPicker.svelte

* fix unwanted copy

* cleaning

* Fix unset context

* move button and always display input

* fix unwanted proppicker display

* update

* update

* clean all

* clean all

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>

* replace hide/show with toggle

* adding mutable setting and navigation to settings to configure channels

* merge fix

* ee non ee changees

* auto-acknowledge when muted

* pr comments

* fix bad log

* user inner modal component

* update unaknowledge alerts after acknowledging from modal

* aknowledge -> acknowledge

* format

* adding backend support for workspace critical alerts

* immediately check for alerts

* immediately check for alerts

* adding openapi changes

* simplify loading of superadmin/ee

* update modal logic

* frontend logic update

* sqlx prepare

* adding mute functionality for workspace critical alerts + show alerts button for instance and workspace settings

* npm / rust warnings & errors

* reverting non-mac cargo toml

* reverting non-mac cargo toml

* mute toggles in critical alert ui

* ui polish

* adding acknowledged_workspace column

* sqlx prepare

* make sure we wait for all stores to be loaded

* adding workspace mute logic to report_critical alert

* auto ack recovered alerts

* toggle workspace as superadmin and critical alerts in logs menu

* adding critical alert button if width < 786px

* refresh on change

* Move notification to logs (#4698)

* Move notification to logs

* removing unused button

---------

Co-authored-by: Alexander Petric <petric.al@gmail.com>

* tailwind typo

* auto ack on mute

* don't change deref

* sqlx prep

* avoid renaming db column, keep acknowledged instead of acknowledged_global

* hide critical alert menu when not ee

* if workspace muted, also acknowledge global when workspace set

* removing save button for mute setting and improve ux/responsiveness

* fix: deployment callbacks have a concurrency limit of 1 on same path

* sqlx prep

* ee-repo ref

* z-[9999] for modal

---------

Co-authored-by: Guilhem <guilhemlemouel@gmail.com>
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2024-11-15 09:48:48 +01:00
Ruben Fiszel
c44fc4f5ab fix: deployment callbacks have a concurrency limit of 1 on same path 2024-11-14 21:15:26 +01:00
Lucas Abel
2d572695ef nit: generalize usage of has_failure_module (#4706) 2024-11-14 08:49:12 +01:00
Guilhem
22ab51e991 feat(frontend): improve and simplify scheduled poll flows (#4560) 2024-11-04 23:18:09 +01:00
HugoCasa
62d88436b2 feat: flow step skipping (#4461) 2024-09-30 16:30:59 +02:00
HugoCasa
41ff40f777 send stats when renewing key if last >24h (#4430)
* feat: send stats when renewing key if last >24h

* nits

* fix: sqlx

* nit

* renewal reason

* stats reason

* update ee ref

* Update ee-repo-ref.txt

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2024-09-30 12:23:07 +02:00
Ruben Fiszel
f189aaad5a fix: improve suspend_first behavior and frequency 2024-09-28 15:46:50 +02:00
Ruben Fiszel
da969da96f improve benchmarking tools (#4450) 2024-09-27 10:48:04 +02:00
HugoCasa
118ffc57fb do not call workspace error handler if flow has error handler (#4434)
* do not call workspace error handler if flow has error handler

* optimize + UI improvements for error handler
2024-09-25 09:10:41 +02:00
HugoCasa
ef74468506 fix: allow no body in job requests (#4413) 2024-09-20 18:26:50 +02:00
HugoCasa
ae8f29a4cb feat: http routing (#4339)
* feat: http routing

* all

* feat: improve UI

* final stuff

* fix: sqlx

* fix: nit

* fix: nits

* fix: error handler display

* fix: routes panel perms

* all

* fix: improve ability to paste from macos in vscode extension

* fix lock-write for deno

* all

* cleaning

* fix

* cli preprocessor

* nits

* nits

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-09-20 15:38:38 +02:00
pyranota
d3e0b1c636 feat: Allow setProgress and getProgress from within the script (#4400)
* Allow setting progress explicitly from script body.

This feature exposes:
 * `getProgress`
 * `setProgress`
 * `incProgress`

API in TypeScript client (python is coming soon).

NOTE: Progress cannot be out of range 0..100 and cannot decrease.

With exposed APIs there is also UI changes, so progress can be shown for individual jobs as well.

For optimization reasons, jobs start to ask for progress only after N-seconds of execution.

* feat: Add `shell.nix`

If you dont have anything but nix, dont worry, run nix-shell in root, or activate with direnv and get all needed dependencies

NOTE: You will still need docker

* feat: Add `dev.nu` to typescript client

Little helper function, allowing developer to work on ts client easier.

To use:
  `./dev.nu watch`

Now add import of windmill in body of your script and `//nobundle` on top of the file

Edit ts client in your favourite editor and hit save. Script will do the rest.

* Cleanup files

* Fix: Failed to deserialize query string: missing field `get_progress`

* perf: Implement non-naive polling mechanism for getting job progress

* Add independant delay for getProgress

Problem in `TestJobLoader`:

There should be 2 delays:

    One until we find our first progress (every 5s)
    Once we found our first progress, we can do it every second

* nit: Use `query_scalar!` instead of `query_as`

* Fix: Sql error, no rows returned by a query that expected to return at least one row

* refactor: Remove global CSS for JobProgressBar

* Change UI for progress of flow subjobs

* Replace `Step 1` with `Running` in ProgressBar for individual jobs

* Remove `incProgress`

incProgress is not very usefull and error-prone

* perf: Set metric only for jobs that are actually using it

(https://github.com/windmill-labs/windmill/pull/4373#discussion_r1759843773)

* Offload registering progress from clients to server

* Add `jobId?` argument to typescript-client's `setProgress` and `getProgress`

Allows to set progress of other jobs and flows,
if jobId specified, than flow id will be inferred automatically.

Could be used by SDK.

* Add `Error::MetricNotFound` for better error handling

* Fix: Make `JobProgressBar` display in red when failed

* Add persistant progress bar

Now you can reload the page after job is done and progress will be still there

* Allow succeeded individual job's progress bar stick to 100%

* Add python support 

* nit: Remove usage of undefined variable in python-client

* Add `async` in ts client (for error handling)

* nit(frontend): Remove unused import

* Dont load JobProgressBar when it is not needed

* nit: npm check fix

* cargo sqlx prepare

* fix sqlx

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-09-18 01:16:19 +02:00
Ruben Fiszel
7a29c93bf8 fix: same worker is transitive on nested flows 2024-09-11 08:59:01 +02:00
Ruben Fiszel
458dcfe1e8 fix: handle better same_worker flow monitor 2024-09-11 01:29:40 +02:00
Ruben Fiszel
16e9f063f7 fix: bun scripts cached in docker image have their dependencies pre-loaded 2024-09-09 13:52:12 +02:00
HugoCasa
a009c8df23 feat: schedule success handler (#4346) 2024-09-06 19:03:09 +02:00
Ruben Fiszel
9bbfb20ae3 nits error message 2024-09-02 00:46:26 +02:00
HugoCasa
c3ce68066c feat: min workers in worker group alert + zombie job critical alert (#4307)
* feat: min workers in worker group alert + zombie job critical alert

* updatee ee ref
2024-08-30 17:25:53 +02:00
HugoCasa
6a99d999a2 feat: improve early stop (#4257) 2024-08-21 00:47:34 +02:00
Ruben Fiszel
c2313a8c8d feat: add wrap_body header to webhooks 2024-08-20 22:58:17 +02:00
Ruben Fiszel
b80257720a fix: handle more gracefully worker without tags 2024-08-20 16:27:01 +02:00
HugoCasa
904b5b4ea1 feat: cache common hub scripts in image (#4249)
* feat: cache common hub scripts in image

* Delete backend/src/hubPaths.json

* fix: missing file

* fix: dockerfile

* fix: cache by path

* fix

* fix

* precreate cache folder
2024-08-16 17:30:44 +02:00
HugoCasa
44cf3f886f feat: opt-in job args in audit logs (#4241)
* feat: job args in audit logs

* update ee ref
2024-08-14 19:00:52 +02:00
HugoCasa
e46c2ea513 fix: improve license key ui (#4220)
* fix: improve license key ui

* fix: stop schedules on key expiration

* update ee ref
2024-08-11 01:01:24 +02:00
Ruben Fiszel
553c95d71b improve queue indices 2024-07-31 23:44:40 +02:00
Ruben Fiszel
cd4a424456 improve job result deser 2024-07-31 02:32:33 +02:00
HugoCasa
e2388989bb feat: get completed flow node result by api/download + eval list result json path optim (#4108)
* feat: get completed flow node result by api

* fix: missing file

* feat: fallback to completed job + fix dfs

* feat: download all results + eval improvement

* fix: nit regex

* fix: nit
2024-07-29 20:30:55 +02:00
Ruben Fiszel
12ec75551e feat: remove nativets in favor of bun with native pragma 2024-07-29 19:20:00 +02:00