Commit Graph

126 Commits

Author SHA1 Message Date
HugoCasa
d17397b070 feat: test trigger connection (#5145)
* feat: test trigger connection

* fix build

* fix build

* update ee ref

* nit

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-01-27 22:40:51 +01:00
dieriba
316a2167e7 feat: postgres triggers (#4860)
* feat: init database triggers

* feat:  wip: database_triggers

* feat:  add database triggers front view

* feat: 🚧 database_triggers

* feat: 🚧 add definition in yaml, updated backend code and added migration

* feat: 🚧 updated migration file, update openapi.yml, updated database_triggers page and backend function

* fix: struct rust

* feat: 🚧 update migrate, database trigger backend function fixed

* feat: 🚧 add resource picker front, update backend function

* feat: 🚧 edit inner database inner, update triggers

* feat: 🚧 database_triggers

* feat: 🚧 update openapi yaml, prettied websocker trigger

* feat: 🚧 database_triggers

* feat: 🚧 add resource module, update variable file, working on main loop for database_trigger

* feat: 🐛 working sqlx query

* feat: 🚧 fix query with sqlx, added main loop

* feat: 🚧 add new column database_trigg

* feat: 🛂 run jobs works

* feat: 🚧 handling slot name and replication

* feat: restructring triggers, decoding trigger message on work

* feat: 🚧 database_trigger

* feat: 🚧

* feat: 🚧 converter done, work on custom script

* feat: 🚧 multiple trigger

* feat: 🚧 adding new argument function

* feat: 🚧 database_trigger

* feat: 🚧 add generate template for front, update script picker

* feat: 🚧 template script fix bug, work on restructing backend logic

* feat: 🚧 update autogenerated script, add persistence state for template script

* feat: 🚧 update structure client

* feat: 🚧 rewrited crud function

* feat: 🚧 added publication handler

* feat: 🚧 new ui finished

* feat: 🚧 added slot function hanlder finish front ux

* feat:  ux improvement done, backend logic done

* feat: 🐛 fix where clause

* feat:

* feat:

* chore: update .sqlx and remove empty package

* fix: update publication and remove unneccessary print

* feat: finish converter to json, remove save button, remove unused crate

* feat: update migration for database trigger, fixed query, fixed frontend error with missing feature on back

* chore: update .sqlx

* chore: update sqlx

* chore: .sqlx

* chore: add unused

* fix: use right database resource in back and front

* fix build

* refactor:

* fix sqlx

* nits: retrieve all script from database trigger editor

* fix: template script object, fix route name

* merge main

* feat: rework ux

* feat/finish ux and update backend logic

* feat: database_trigger

* feat: database_trigger

* chore: update sqlx

* chore: fix ci

* nits: fix worker.rs

* feat: database_trigger

* Update frontend/src/lib/components/triggers/database/DatabaseTriggersPanel.svelte

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* feat:

* fix ci

* nits: database to postgres

* nits: fix .sqlx

* feat: add link to original code and open tab functionallity

* nits: update .sqlx

* nits: fix request type and yaml file

* nits: add abel suggestion

* fix: update .sqlx

* fix: update .sqlx

* nits: add comment to hex.rs

* nits: remove features, rename database to postgres name

* nits: replace database name by postgres

* nits: database name to postgres

* nits: update .sqlx

* nits: update .sqlx

* nit

* nit

* nits

* nit

* nits

* nits

* nits

* nits

* fix sqlx

* nits: update .vscode

---------

Co-authored-by: HugoCasa <hugo@casademont.ch>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-01-24 15:12:41 +01:00
Alexander Petric
a88fbb238a feat: msft teams support for critical alerts (#5113)
* feat: msft teams support for critical alerts

* ee changes

* ee

* sqlx prep

* multiple teams channels

* commit file, not symlink

* improve reactivity

* docs link

* Update ee-repo-ref.txt
2025-01-23 19:57:13 +01:00
HugoCasa
e66fcf9270 feat: nats triggers (#5039)
* feat: nats triggers

* missing files

* fix build

* update feature flags

* event stream fixes

* update ee ref
2025-01-10 19:47:17 +01:00
HugoCasa
01c6f84520 feat: captures (#4807)
* feat: captures

* flow UI and improvements

* fix: build

* fix sqlx

* Move Capture WIP

* Add capture to webhook and websocket

* Move connection status viewer in the head

* change trigger section label

* Add popover capture picker using melt ui

* Add shortcut to triggers capture from input form

* remove capture tab in input

* Allways show capture

* remove useless logs

* Add email capture

* Add kafka capture into triggers

* Add edit option in capture table

* use light header for arg input

* Add prefilled group id

* Change button label

* fix logic

* Open resource drawer if prototype has fields

* fix default completion

* Change name Prototype

* Dissociate Editor Mode for triggers

* Fix bug for script

* Fix apply args

* fix apply schema

* Add capture table to script

* fix apply args

* Add capture button for script

* Delete capture tab

* Set capture on when opening triggers capture

* fix connection indicator

* fix minor issues

* Add preprocessor logic

* Use slot in log Panel for captures

* handle capture refresh in script

* Delete capture tab from script editor

* reset kafka resource on toggle static

* fix minor issue

* Allow resource in kafka capture

* use simple capture button in flow

* Remove capture panel

* Polish route trigger editor

* Fix resource saving

* Remove excessive padding

* merge nits

* add workflow_dispatch to build

* add workflow_dispatch to build

* better capture UI

* fix sqlx

* fix build

* fix build

* build

* make initial_messages optional in line with db

* fix npm check

* better handle args for capture webhook and http

* improve migration + http capture fixes

* fix sqlx

* update ee ref

---------

Co-authored-by: Guilhem <guilhemlemouel@gmail.com>
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-01-06 12:07:09 +01:00
Alexander Petric
6308bf0dcb feat: interactive slack approvals (#4942)
* feat: interactive slack approvals

* move form creation logic to backend

* adding python client

* polish messages

* date time picker and default values

* Initial commit

* Initial commit

* refactor

* cleanup

* cleanup

* cleanup

* cleanup, treating numbers/integers/booleans

* adding slack to ts dependencies

* sqlx prepare

* gitignore, adding package.lock

* ellipsis comments

* Update typescript-client/client.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* refactor to move everything serverside + modal

* sqlx prep

* reverting slack dependency

* python client update

* fix build errors

* Update pyproject.toml

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2024-12-20 21:47:45 +01:00
Ruben Fiszel
34392ef66f split workspace and auth logic 2024-12-15 13:13:09 +01:00
Ruben Fiszel
51c0373b4c feature gates more feature for faster compilation (#4928)
* cleanup

* cleanup

* all

* all

* all

* all
2024-12-15 00:00:37 +01:00
HugoCasa
7dce9ab003 fix unused variable (#4926) 2024-12-14 13:43:51 +01:00
HugoCasa
90ece755f7 more feature flags (#4918)
* more feature flags

* include python in tests

* smtp feature flag

* del

* add back file

* fix openapi int
2024-12-14 13:21:28 +01:00
HugoCasa
563a492008 feat: accept direct file upload for webhook/http (s3) (#4903)
* feat: accept direct file upload for webhook/http (s3)

* nit

* nit
2024-12-12 00:33:19 +01:00
HugoCasa
1ec6c6f765 feat: app custom paths (#4828)
* feat: app custom paths

* nit

* make ee only + fix sqlx

* fix: custom http routes auth

* nits

* fix auth + nits

* apps_ee

* move custom path to ee

* fix app jwt

* update ee ref
2024-12-04 16:50:17 +01:00
Ruben Fiszel
89456f502f remove rsmq 2024-11-24 11:04:54 +01:00
HugoCasa
88b8ffab90 feat: kafka triggers (#4713)
* feat: kafka triggers

* sqlx

* fix build

* improve error messages on windows

* nit

* fix build

* missing action

* nit

* maybe fix ssl on windows

* nit

* update ee ref

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2024-11-18 17:03:50 +01:00
dieriba
556b4a41a1 feat: Support mistral anthropic for ai (#4692)
* wip: openai proxy and other ai proxy integration

* fixing migration script

* wip: support different ai provider in front, fix proxy openai

* wip: adding frontend ai provider

* updated copilot types

* wip: working on anthropic integration

* done AI proxy front

* adding new type and support for anthropic

* updating gitignore

* adding streaming response

* added streaming prompt

* push lib/gen

* wip: fixing anthropic

* anthropic fully supported

* fix backend missing var error and fully support stream event for anthropic

* remove gen directory

* fixing openapi file

* add support for mistral, and update create workspace components

* remove deref.json

* remove package-json

* openapi

* fix ui enable code

* added utility function for init workspace ai provider

* fix workspace switch bug

* update anthropic property and fixed frontend error

* fix workspace settings

* update error message and fix typo migration file

* chore: update openapi file

* fix dev file

* add .sqlx

* all

* update sqlx

---------

Co-authored-by: dieriba <t.dieriba@gmail.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-11-14 14:24:35 +01:00
Lucas Abel
50ff183bae feat(backend): monitor minimal version of living workers (#4704) 2024-11-14 12:33:50 +01:00
wendrul
77735d859c feat: Add full-text search on windmill service logs (#4576)
* Add service_log indexer file (ee)

* Rename to completed_run

* rename to service_logs

* Update lib.rs

* Make indexing of logs and simple frontend

* Add common ee file

* Update lib.rs

* Make log search global to fix openapi unused path param

* Prepare sqlx

* Show last indexed also when no jobs are found

* Adapt feature flags to service logs indexer

* Update api and main

* Build service log search page

* scope query to the min and max time

* Migrate modal, bug fixes

* Remove unused import

* Make the service logs page take the full screen

* improve quick access menu to service_logs, fix date stuff, hide 0 matches

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2024-11-05 09:52:07 +01:00
Ruben Fiszel
fd4fecf945 split better oss and ee 2024-11-02 16:31:03 +01:00
HugoCasa
7fdb524fc6 improve key renewal (#4519)
* improve key renewal

* sqlx and fix

* fix build

* improve everything

* nits

* nits

* update ee ref

* nit
2024-10-29 09:26:45 +01:00
HugoCasa
1741ce7823 fix ws killpill (#4551)
* websocket killpill

* oups

* nit

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2024-10-19 01:17:15 +02:00
Ruben Fiszel
0e45cca78d adapt websocket triggers to multi-tenant cloud 2024-10-18 00:52:23 +02:00
HugoCasa
b0fbcd8305 feat: websocket triggers (#4505)
* feat: websocket triggers

* fix

* frontend nits

* nits

* new trigger UI

* sqlx
2024-10-18 00:21:52 +02:00
Faton Ramadani
258445e523 feat(frontend): unify all triggers UX and simplify flow settings (#4259)
* feat(frontend): added list of triggers in the flow graph

* feat(frontend): added list of triggers in the flow graph

* feat(frontend): clean up

* feat(frontend): improve UX

* feat(frontend): triggers

* feat(frontend): triggers

* feat(frontend): done

* feat(frontend): fix trigger when position when a preprocessor is presetn

* Glm/rework flow settings v2 (#4497)

* fat(frontend): simplify flow settings menu

* improve scroll

* changing mute toggle

* Add advanced settings badge

* Add nord theme colors

* Add bage for advanced options

* fix minor issue

* fix minor issue

* Add triggers menu to flow settings

* Add quick trigger access

* remove triggers in flow settings

* fix minor issue

* Move triggers settings to flow right panel

* polishing

* fix unset store

* remove save up to for triggers

* fix padding

* reset default tag color

* remove custom select component

* revert path change

* revert section modif

* Revert unused feature

---------

Co-authored-by: Guilhem <guilhem@mbp-de-windmill.home>

* Connect top bar cron to schedules settings

* Turn copilot into node

* fix copilot placement

* remove useless import

* fix center copilot

* fix binding

* remove copilot on top of preprocessor

* render copilot node on condition

* quickfix

* remove copilot node

* fix minor issues

* fix route count update

* fix schedule sync

* harmonize colors

* fix alignment and add edges

* recenter node summary

* fix schedules sync

* Add id title

* all

* all

* all

* iteration

* all

* all

* done

* fix

* more fixes

---------

Co-authored-by: Guilhem <guilhemlemouel@gmail.com>
Co-authored-by: Guilhem <guilhem@mbp-de-windmill.home>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-10-16 17:58:15 +02:00
HugoCasa
41ff40f777 send stats when renewing key if last >24h (#4430)
* feat: send stats when renewing key if last >24h

* nits

* fix: sqlx

* nit

* renewal reason

* stats reason

* update ee ref

* Update ee-repo-ref.txt

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2024-09-30 12:23:07 +02:00
Ruben Fiszel
bd6ddc166b feat(cli): add queues, workers and worker-groups commands (#4439)
* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all
2024-09-26 11:27:37 +02:00
HugoCasa
ef74468506 fix: allow no body in job requests (#4413) 2024-09-20 18:26:50 +02:00
HugoCasa
ae8f29a4cb feat: http routing (#4339)
* feat: http routing

* all

* feat: improve UI

* final stuff

* fix: sqlx

* fix: nit

* fix: nits

* fix: error handler display

* fix: routes panel perms

* all

* fix: improve ability to paste from macos in vscode extension

* fix lock-write for deno

* all

* cleaning

* fix

* cli preprocessor

* nits

* nits

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-09-20 15:38:38 +02:00
Ruben Fiszel
7d29651d3e fix: graceful worker exits for same worker jobs (#4371)
* all

* fix compile
2024-09-11 18:32:08 +02:00
Ruben Fiszel
c62e152d27 feat: service logs (#4244) 2024-08-31 14:57:35 +02:00
HugoCasa
80a41669b2 feat: email triggers (#4163)
* feat: email triggers v0

* update docker compose to nginx with tcp reverse proxy + move smtp to private

* fix: open source build

* test: update ee ref for testing

* feat: use caddy with layer4

* fix: nit

* feat: configurable email domain

* fix: nit

* fix: nit

* fix: get l4 from main

* fix: default email domain to mail.domain

* update ee ref
2024-08-06 16:17:33 +02:00
Ruben Fiszel
caecbde99a better split tantivy 2024-07-24 08:15:14 +02:00
HugoCasa
f76f639ee8 feat: use jwks for external jwt auth (#4089)
* feat: use jwks for external jwts

* fix: build

* update ee ref

* fix: nit error msg
2024-07-23 16:22:36 +02:00
wendrul
af444c8b36 feat: Full-text search on runs using tantivy and command palette for quick actions (#4046)
* Add indexer crate and files

* POC searcher

incomplete schema
only indexes at startup

* POC search component frontend

* Demo of the frontend element

* add Results and Args as text

* minimal functionality

* Make jump to scripts by name

also flows and apps

* Add button on sidebar to open search

* Update lock on indexer after merge

* Make arrow key navigation compatible with scrol

* Show empty result screen and log as a coming feat

* Add summary to script searchable items

* Catch `parts is undefined` error (uFuzzy)

* Index refreshing using tokio interval

* Fix JobLoader workspace being wrongly defined

* Fix click outside

* Add debouncing for completed run search

* Binary mode working + job index tracker

* Warning for no license + fix height scrollbars on content search

* Make it compile without EE files

* remove panic to use errors

* Move global search

* Cleanup UI, no more tab switcher but clear placeholders and actions

* Add tantivy feature flag for windmill-api

* Rework indexer mode

* Mac compatibility for shortcut

* Update test for new run_server

* Prepare sqlx

* Mac compatibility

* Fix openapi yaml

* Fix frontend

* Frontend api fix

* Update docker-compose.yml and caddyfile

With the (by default deactivated) container and reverse proxy to use the
windmill indexer

* fix feature flag for tests

* fix feature falg for running tests

* fix feature flag for running tests

* Make content search use search modal instead

* Add tantivy feature to ee build steps

* Remove old Content search

* change volume location for indexer

* Update dependencies

* Prepare sqlx

* Uncomment line on docker compose

* Add line between input and results

* Update ee repo ref
2024-07-11 11:13:27 +02:00
wendrul
8dd422d986 feat: towards cloud events webhook compliance (#3883)
* Cloud events webhook compliance

* Only on OPTIONS
2024-06-07 14:24:04 +02:00
Ruben Fiszel
e666f837cd fix: further restrict job api to logged in workspace users or anonymous jobs only" (#3860) 2024-06-03 15:13:15 +02:00
Ruben Fiszel
6a332bc30f fix: make all error strings more verbose 2024-05-25 22:15:15 +02:00
wendrul
60b036c1ae Add concurrency limits observability (#3586)
* Add concurrency_key table and field to payload

Add a new table custom_concurrency_key for storing concurrency keys on a
per job basis. Add the field to the job payload to populate the table on
creation of a new job.

* Rename to custom_concurrency_key

Be more explicit about the nature of the option variable by calling it
a more appropriate name.

* Delete concurrency_keys on delete_expire_items

* Propagate db errors up

* Add ref to EE changes

* update sqlx

* Add concurrency_key to tests

* Revert renaming of FlowValue field

FlowValue field concurrency_key must be named concurrency_key for
serialization purposes.

* Remove debug print statement

* Get concurrency_key from Job

Whether it is a Completed job or a queued one, add a ways to get the
associated concurrency key

* Add endpoint to get concurrency_key

* Add endpoint and button to get job concurrency_key

Add a line with a link on the FlowMetadata that goes to the
concurrency_groups page. add endpoint to get the concurrency_key of a
job

* Merge concurrency_key tables into one

migration + change all related querrys

* Add concurrency plot to the runs page

* Prepare sqlx

* Add concurrency Key filter on UI

* Prepare sqlx

* Prepare sqlx

* Fix filter and order query

The limit makes more sense if we cut the older rows, so order by
started_at

* Factor interpolation to reuse on concurrency key

Factor the arg interpolation logic into a function and finish the
processing of concurrency key before insertion

* Remove old concurrency key processing logic

* Second transaction with userdb

To send all concurrency intervals but revealing only uuids of accessible
jobs, make a second transaction with the userdb.

* Remove old second endpoint

the intervals endpoint now also gets the concurrencyt key information
for all jobs

* Show external jobs

* Put filters into a dropdown

Create a ToggleButtonMore and put elements into a dropdown

* Add toggle between two graphs

* Add filter functionality to concurrency graph

* Improve concurrency graph front

* Fix concurrency groups page

* Prepare sqlx

* Add ref to ee

* Change migration to create new table instead

Instead of renaming the custom_concurrency_key_ended atble, we create a
new table and we will delete custom_concurrency_key_ended in the future
when it is no longer linked to any jobs

* Do small UI improvements

* Fix range fusion by not filtering past jobs

Instead of filtering jobs in the backend on a startedAfter value, limit
the query to 1000 and query all possible towards the past to get a good
context for the graph in most sane situations

* Improve frontend UI
2024-05-15 20:15:14 +02:00
Ruben Fiszel
1f64715043 feat: local typescript codebase as bundle (#3694)
* codebases

* codebases

* foo

* foo

* foo

* foo

* fix(frontend): Disable the insert button when required fields are empty strings (#3659)

* fix(frontend): use normal password mask for the sensitive fields of the resource editor

* handle super admins in password arg input

* chore(main): release 1.323.2 (#3660)

* chore(main): release 1.323.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>

* foo

* all

* s3_helpers move

* progress

* all

* progress

* progress

* progress

* progress

* codebase final

* update without ee

* sqlx

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* update

* all

* all

* all

---------

Co-authored-by: Faton Ramadani <faton.ramadani14@gmail.com>
Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-05-09 18:35:50 +02:00
HugoCasa
1f206e51fb feat: disable response logs based on env var (#3685) 2024-05-07 15:57:41 +02:00
Ruben Fiszel
b2362c8263 add scalar at openapi2.html 2024-04-08 01:42:43 +02:00
Ruben Fiszel
442d40ac06 fix: remove reqwest_11 2024-04-06 22:13:35 +02:00
HugoCasa
9b6ed95ef2 feat: private hub (#3491)
* feat: private hub v0

* feat: add UI setting

* Update ee.rs

* Update ee.rs

* fix: remove ee symlink

* fix: add back CE ee.rs

* feat: disable s3 embedding loading when private hub

* feat: reload embeddings on hub url change

* fix: only reload embeddings db on server mode

* patch: set default hub url const

* fix: nit
2024-04-04 04:16:53 +02:00
Ruben Fiszel
7714221f3e feat: large log disk and distributed storage compaction 2024-03-23 10:55:29 +01:00
Ruben Fiszel
16b3b659e9 chore: update deps 2024-03-21 13:37:17 +01:00
Ruben Fiszel
ad79ef636c reactive embedding feature 2024-02-24 17:28:36 +01:00
Ruben Fiszel
c3ef987e28 chore: fasten compilation time 2024-02-24 17:10:55 +01:00
Ruben Fiszel
82c8b1d1dd chore: fasten compilation time 2024-02-24 16:29:35 +01:00
Ruben Fiszel
879a6cda00 chore: remove warnings 2024-02-18 14:03:00 +01:00
Ruben Fiszel
1a8285a06f feat: scim token and saml metadata setting in UI directly
chore(main): release 1.271.0 (#3237)

* chore(main): release 1.271.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>

fix test

revert ee changes

revert ee changes
2024-02-18 07:26:12 +01:00
Ruben Fiszel
4b3df8ae2e chore: s3 helpers EE 2024-02-17 15:05:10 +01:00