Commit Graph

159 Commits

Author SHA1 Message Date
Pyra
1dfb0f0078 refactor: move concurrency and debouncing settings to their own structs (#7286)
* move concurrency and debouncing settings to it's own structs

Signed-off-by: pyranota <pyra@duck.com>

* cleanup queue::jobs::pull

Signed-off-by: pyranota <pyra@duck.com>

* done

Signed-off-by: pyranota <pyra@duck.com>

* migrated the rest

Signed-off-by: pyranota <pyra@duck.com>

* implement todo!

* impmenet todo! v2

* more cleanup

* fix all

Signed-off-by: pyranota <pyra@duck.com>

* remove unrelated code

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* do not serialize rawcode

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fixes

Signed-off-by: pyranota <pyra@duck.com>

* update ee repo ref

Signed-off-by: pyranota <pyra@duck.com>

* update ee repo

Signed-off-by: pyranota <pyra@duck.com>

* fix tests

Signed-off-by: pyranota <pyra@duck.com>

* Update ee-repo-ref.txt

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-04 10:11:54 +00:00
claude[bot]
e9f13065bf fix: show related job when deployment is in progress (#7294)
* feat: show related job when deployment is in progress

- Added job_id column to deployment_metadata table to track current deployment jobs
- Updated backend to store job_id when creating dependency jobs for scripts and flows
- Modified deployment status API endpoints to include job_id in responses
- Updated frontend to display clickable job link in "Deployment in progress" badge
- Added OpenAPI schema updates for new job_id field

Resolves #7293

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* update

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-12-04 09:50:07 +00:00
Ruben Fiszel
60d5ea8579 fix: get flow by id doesn't require path (#7278) 2025-12-02 19:45:30 +00:00
hugocasa
93c34c6d76 feat: flow for loop squashing (#7107)
* feat: flow for loop optimization

* fix: make dedicated flows work with flow nodes

* fix: transform dedicated inputs + better error handling + differentiate squash from same worker

* fix builds

* fix build

* fix build

* make dedicated worker private

* update ee ref

* frontend nits

* nit

* add trace to dedicated worker

* update ee ref

* Update ee-repo-ref.txt

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-11-25 17:25:34 +01:00
Ruben Fiszel
80c7911641 fix: listScripts include description with opt-out query arg (#7210)
* description jobs

* all
2025-11-24 08:37:55 +00:00
Guilhem
cfeb294308 feat(frontend): add notes to flow (#6628)
* Add note component

* save note size and position

* move add note button up

* nit

* Add markdown support

* wip

* fix add sticky note button

* fix text update

* Add sticky note to saved flow data

* add note color picker

* Introduce node multiselect

* Add group notes

* Adapt layout to group node

* create a note manager class

* clean reactivity

* clean

* improve adaptive layout to group note

* modify layout based on cached text height

* fined grained graph rendering for notes

* separate noteManager into editor and render

* separate noteManager into editor and render

* create a note change observer

* render note node from context

* simplify note state managment

* show note in flow viewer

* clean dirty changes

* clean selection manager

* fix layout check

* improve bg surface select

* Handle z-index for stacked group notes

* clean selection manager

* exclude notes from rect select

* Allow switch between selection modes with keyboard keys

* improve selection box styling

* prevent dragging note when editing

* nit

* Simplify selection using svelte flow built in feature

* handle note selection separately

* Add min size for notes

* improve selection toggle

* improve mode switch

* make size and position optional for group notes

* Improve initial viewport position

* Add context menu for the canevas

* nit

* Add node context menu

* improve note select

* use clickoutside for note deselect

* use pointerdown outside to close context menu

* nit

* fix selection issues

* make edges non selectable

* improve color palette

* fix backend

* fix backend check

* cargo lock restore

* Add toggle to display notes

* fix note selection

* nit

* account for css offset in for loop

* fix multiple selection pannel styling

* clear flow selection when creating note

* Improve placeholder and note default text

* Escape note edit mode when pressing Esc

* Allow note edition in local dev

* clean

* Handle subflow selection

* prevent group note resizing

* nit

* allow notes in flow expand

* Improve multi select panel

* Allow context menu in note mode

* Add event listenner to fix pane click deselect

* prevent zoom in text area in notes

* improve bounding box styling

* Use control for box selection for non mac users

* nit

* clean notes groups

* nit

* use portal for note actions

* handle assets node when computing note layout

* Simplify layout compute for notes

* use smart color choice for notes

* Switch display note when adding a new note

* clean code

* improve group note bound size calculation

* simplify AI tool nodes and asset handling

* nit

* nit

* improve flow centering

* create group note button

* Improve selection of nodes

* Revert "Improve selection of nodes"

This reverts commit d2c40d82b1.

* refert backend changes

* nit

* improve graph selection

* clean

* make backend work except job runs

* fix notSelectable

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-11-19 21:25:57 +00:00
dieriba
09cdfb4556 feat: webhook by flow version (#7062)
* ok

* done

* update sqlx and repo ref

* fix
2025-11-14 14:36:24 +00:00
Diego Imbert
0639623af6 Trigger kind filter (#7123)
* Pass JobTriggerKind to push() to store it in v2_job

* fix compiltion

* frontend job trigger kind filter

* SQL trigger_kind filtering

* fix animated button

* ee repo ref

* Fix "Show schedule" + trigger kind = schedule

* duplicate email

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-11-13 10:50:04 +00:00
dieriba
c59183f5c3 feat: env var in flow (#6852) 2025-11-07 18:38:55 +00:00
dieriba
b5e341fde7 perf: parse flow value only if needed (#7025)
* nits

* renames
2025-11-01 15:08:34 +01:00
dieriba
5f6aabb667 nits: warning retry (#6760)
* done

* ok

* ok

* better

* update sqlx

* raw val
2025-11-01 10:40:43 +00:00
Pyra
83c0c82124 fix: auto-unarchived scripts (#6998)
* fix: auto-unarchived scripts

Signed-off-by: pyranota <pyra@duck.com>

* also do for archive by hash

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
2025-10-29 21:09:36 +00:00
Pyra
edece035f8 feat: job debouncing (#6878)
* feat(EE): job debouncing

Signed-off-by: pyranota <pyra@duck.com>

* remove 'script' file

Signed-off-by: pyranota <pyra@duck.com>

* more work

Signed-off-by: pyranota <pyra@duck.com>

* properly gate it behind enterprise

Signed-off-by: pyranota <pyra@duck.com>

* update ee repo ref

Signed-off-by: pyranota <pyra@duck.com>

* change ee repo ref again

Signed-off-by: pyranota <pyra@duck.com>

* remove unused variable

Signed-off-by: pyranota <pyra@duck.com>

* feat(EE): implement TODOs and enhance tracing for job debouncing

- Add database index on script(workspace_id, debounce_key) for efficient lookups
- Update minimum version requirement to 1.564.0 throughout codebase
- Add tracing warnings when debouncing is disabled due to worker version mismatch
- Fix all documentation links from TODO placeholders to proper URLs
- Replace Gauge icon with Timer icon for debouncing UI elements
- Update placeholder text and tooltips with clear descriptions

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* create -> crate

Signed-off-by: pyranota <pyra@duck.com>

* remove index

Signed-off-by: pyranota <pyra@duck.com>

* some updates

Signed-off-by: pyranota <pyra@duck.com>

* fix once more

Signed-off-by: pyranota <pyra@duck.com>

* fix it once more

Signed-off-by: pyranota <pyra@duck.com>

* Remove flow step debouncing, keep top-level flow debouncing

- Remove debounce fields from RawScript and FlowScript FlowModuleValue variants
- Remove debounce fields from JobPayload::FlowScript and RawCode
- Update raw_script_to_payload function signature
- Remove debouncing UI from flow step runtime settings
- Remove debouncing toggle handler and indicator badge
- Preserve top-level flow debouncing in FlowSettings

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fixup claude's work

Signed-off-by: pyranota <pyra@duck.com>

* cleanup: remove dbg! statements, update min version to 1.566.0, add comprehensive comments

- Removed all dbg! macro calls from production code
- Updated MIN_VERSION_SUPPORTS_DEBOUNCING from 1.564.0 to 1.566.0
- Added comprehensive documentation comments explaining:
  - Debouncing feature purpose and mechanics
  - Database schema for debounce_key and debounce_stale_data tables
  - Version check logic and guard functions
- Improved code clarity and maintainability

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* improve fallback

Signed-off-by: pyranota <pyra@duck.com>

* remove comments from old migration

Signed-off-by: pyranota <pyra@duck.com>

* fix pull

Signed-off-by: pyranota <pyra@duck.com>

* fix once more

Signed-off-by: pyranota <pyra@duck.com>

* Update frontend/src/lib/components/ScriptBuilder.svelte

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* add sqlx cache

Signed-off-by: pyranota <pyra@duck.com>

* fix: frontend CI - fix debounce field names and remove leftover flow step debouncing

- Fix ScriptBuilder.svelte: change custom_debounce_key to debounce_key
- Add debounce_key and debounce_delay_s fields to NewScript schema in openapi.yaml
- Regenerate frontend types from OpenAPI spec
- Remove leftover flow step debouncing code from FlowModuleComponent.svelte
- Remove debounce fields from RawScript in openflow.openapi.yaml
- Remove unused Timer import from FlowModuleHeader.svelte

All frontend checks now passing (0 errors, 0 warnings)

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* fix ci

Signed-off-by: pyranota <pyra@duck.com>

* remove unused import

Signed-off-by: pyranota <pyra@duck.com>

* fix ci again

Signed-off-by: pyranota <pyra@duck.com>

* udpate ee repo ref

Signed-off-by: pyranota <pyra@duck.com>

* CI doesn't want to be fixed but I still try

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* ci...

Signed-off-by: pyranota <pyra@duck.com>

* Update ee-repo-ref.txt

* safer migration

Signed-off-by: pyranota <pyra@duck.com>

* reduce noise in logs

Signed-off-by: pyranota <pyra@duck.com>

* fix cli for scripts

Signed-off-by: pyranota <pyra@duck.com>

* nit

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Pyra <pyranota@users.noreply.github.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-23 20:18:08 +00:00
Pyra
defb6c9694 feat: dependency job debouncing (#6769)
* v0

Signed-off-by: pyranota <pyra@duck.com>

* optimize relocks

* make it work with relative relative imports

Signed-off-by: pyranota <pyra@duck.com>

* use fallback

Signed-off-by: pyranota <pyra@duck.com>

* remove dbg and todos

Signed-off-by: pyranota <pyra@duck.com>

* future proof a bit

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* more cleanup

Signed-off-by: pyranota <pyra@duck.com>

* remove final TODO

Signed-off-by: pyranota <pyra@duck.com>

* do not use bytemuck

Signed-off-by: pyranota <pyra@duck.com>

* optimize hashing

Signed-off-by: pyranota <pyra@duck.com>

* implementation 1

Signed-off-by: pyranota <pyra@duck.com>

* almost v0

Signed-off-by: pyranota <pyra@duck.com>

* v0

Signed-off-by: pyranota <pyra@duck.com>

* add comments and use fallback

Signed-off-by: pyranota <pyra@duck.com>

* call dissolve for apps

Signed-off-by: pyranota <pyra@duck.com>

* add comms

Signed-off-by: pyranota <pyra@duck.com>

* refactor v0 (partially tested + dirty)

Signed-off-by: pyranota <pyra@duck.com>

* finishing

Signed-off-by: pyranota <pyra@duck.com>

* remove TODO

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* silence unused argument

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* implement rebuild_map endpoint

Signed-off-by: pyranota <pyra@duck.com>

* update windmill api client

Signed-off-by: pyranota <pyra@duck.com>

* almost finish with tests

Signed-off-by: pyranota <pyra@duck.com>

* add proper testing

Signed-off-by: pyranota <pyra@duck.com>

* remove unused fixtures

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* partial cleanup

Signed-off-by: pyranota <pyra@duck.com>

* Update backend/windmill-worker/src/scoped_dependency_map.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-common/src/scripts.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* evil doings

Signed-off-by: pyranota <pyra@duck.com>

* more cleanup

* Update SQLx metadata

* more cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fixing CI

Signed-off-by: pyranota <pyra@duck.com>

* remove python from default features

Signed-off-by: pyranota <pyra@duck.com>

* feat: dependency job debouncing

* checkpoint

Signed-off-by: pyranota <pyra@duck.com>

* more improvements

Signed-off-by: pyranota <pyra@duck.com>

* refactor: clean up dependency job debouncing implementation

- Add comprehensive comments explaining the debouncing mechanism
- Replace debug statements (dbg!) with proper tracing calls
- Extract helper functions to reduce code duplication:
  - extract_to_relock_from_args() for extracting nodes/components
  - accumulate_debounce_stale_data() for updating stale data
- Improve code readability and maintainability

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* Update backend/windmill-common/src/jobs.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-common/src/jobs.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-queue/src/jobs.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* better error handling for helper

Signed-off-by: pyranota <pyra@duck.com>

* update ee-ref

Signed-off-by: pyranota <pyra@duck.com>

* test: add race condition test for dependency job debouncing

- Implement test_2 for scripts to test the race condition edge case
- Add comprehensive documentation comments to all test functions
- Remove empty test_2 stubs for flows (not needed)
- Keep test_2 stub for apps with TODO comment

The race condition test simulates the scenario where a job is marked
as running but debounce_key hasn't been cleaned up yet, forcing the
system to create a new job while reusing the existing debounce_key.

This edge case can occur due to the lack of transactions in the pull
function for performance reasons (see jobs.rs:4415-4425).

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* test: implement race condition test for dependency job debouncing

- Add comprehensive test_2 for script module that tests the race condition edge case
- Remove empty test_2 stubs from flows and apps modules
- Fix unused variable warning in worker_lockfiles.rs
- Add detailed comments explaining the race condition scenario and test logic

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* implement fallback

Signed-off-by: pyranota <pyra@duck.com>

* make it mostly work

* all tests are almost working

Signed-off-by: pyranota <pyra@duck.com>

* add comments a bit

Signed-off-by: pyranota <pyra@duck.com>

* Update backend/windmill-common/src/scripts.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-worker/src/worker_lockfiles.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-worker/src/worker_lockfiles.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* feat: improve debouncing documentation, tracing, and error handling

- Add comprehensive 3-phase debouncing documentation explaining push/accumulation/pull
- Enhance all tracing with structured logging (job_id, workspace_id, node_count, etc.)
- Add proper error handling with .map_err() and contextual messages
- Replace dbg!() with proper tracing::debug!()
- Replace todo!() with proper error handling
- Fix typos: 'and edge case' → 'an edge case', 'bc' → 'because'
- Fix debug variable name: 'debounce_job_id_0' → 'debounce_job_id_o'
- Add documentation for debounce cleanup and stale data retrieval
- Add trace-level logging for non-error paths to reduce noise

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* do some work for future improvements

Signed-off-by: pyranota <pyra@duck.com>

* fix tests

Signed-off-by: pyranota <pyra@duck.com>

* clippy

Signed-off-by: pyranota <pyra@duck.com>

* update sqlx

Signed-off-by: pyranota <pyra@duck.com>

* clippy

Signed-off-by: pyranota <pyra@duck.com>

* update ee ref

Signed-off-by: pyranota <pyra@duck.com>

* flag tests behind the feature, add timeout

Signed-off-by: pyranota <pyra@duck.com>

* fix timeout + cleanup

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* row lock debounce_key

Signed-off-by: pyranota <pyra@duck.com>

* addressing TODOs

Signed-off-by: pyranota <pyra@duck.com>

* fix test

Signed-off-by: pyranota <pyra@duck.com>

* ee ref

Signed-off-by: pyranota <pyra@duck.com>

* ee repo

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: Pyra <92104930+pyranye@users.noreply.github.com>
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Pyra <pyranota@users.noreply.github.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-10-16 16:49:35 +00:00
Ruben Fiszel
aeb6829011 use excluded where relevant 2025-10-16 09:19:44 +00:00
Alexander Petric
ae8d37fc34 feat: add dynamic skip for schedules (#6739)
* feat: add dynamic skip for schedules

* npm check

* accidental rename

* fixing tests

* Update SQLx metadata

* simplify api / queries

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-08 09:18:15 +00:00
hugocasa
3907c9f951 feat: end user email env var (#6750)
* feat: end user email env var

* nit

* nits
2025-10-03 17:27:58 +00:00
centdix
047420e5ad feat(flow): add option to turn flow into chat (#6658)
* add toggle option + chat interface

* backend impl

* draft

* put info in schema

* Revert "backend impl"

This reverts commit c534eeb49986424e2c12e2c5642be4e17ba380d1.

* chat interface in flow input

* cleaning

* add logic for running flow + styling

* handle historic args

* fix frontend changes

* add tables

* add conv list

* add endpoints

* adapt frontend

* list message logic

* save message in db

* save response in db

* cleaning

* better migrations

* refresh on new conv

* better logic for messages

* nit

* genere conversation uuid from frontend

* store chat mode info in flow status

* better ui for chat

* collapse chat

* ui

* infinite scroll on convs

* infinite scroll on messages

* fix ui

* new chat entry on new

* cleaning

* change setting logic

* fix test logic from flow input

* move toggle to input

* add warning modal when enabling chat mode

* add summary and explanation on inline script

* add hint for chat mode on user_message desc

* show chat message instead of input in graph

* add warning for triggers

* one logo when not expanded

* use infinitelist for conversations

* add warning when deployment in progress

* full width button

* better icon for menu

* better input + nits

* put toggle in action

* use waitjob

* cleaning

* cleaning

* scroll on new + cleaning

* use enum

* fix logic

* full screen

* cleaning

* exit on updatesqlx error

* Update SQLx metadata

* fix

* cleaning

* add for wait result endpoint

* add missing drop

* delete cascade

* fix: use macro version of query_as in flow_conversations.rs

Use sqlx::query_as! macro instead of query_as function for compile-time
SQL validation and better type safety

Co-authored-by: centdix <centdix@users.noreply.github.com>

* fix: update comment to clarify conversation message update condition

The comment now accurately reflects that the update happens when
it's a flow and it's done (flow_is_done)

Co-authored-by: centdix <centdix@users.noreply.github.com>

* fix: only parse chat_input_enabled if conditions are met

Move the parse_chat_input_enabled() call inside the condition check
to avoid unnecessary parsing when the flow is not done or unsuccessful

Co-authored-by: centdix <centdix@users.noreply.github.com>

* fix: use the same transaction for conversation creation

Pass transaction to get_or_create_conversation_with_id instead of
creating a new one, ensuring all operations are atomic

Co-authored-by: centdix <centdix@users.noreply.github.com>

* fix: remove update trigger and handle updated_at in application code

Remove the database trigger that automatically updates conversation
timestamp and instead update it explicitly when creating messages.
This gives better control and consistency.

Co-authored-by: centdix <centdix@users.noreply.github.com>

* Update SQLx metadata

* cleaning

* feat(aiagent): handle memory (#6719)

* implement memory

* s3 logic for memory

* fix typo

* much cleaner

* cleaning

* cleaning

* only if chat

* display nit

* nit

* fix stack overflow

* cleaning

* use len arg from input

* cleaning

* change order

* delete memory when conv deleted

* cleaning

* nit

* show description in expr mode

* opti

* opti

* updatee ref

* store string as simple string

* use markdown

* do not wait for deletion

* add delete loading

* fix logic

* fix markdown

* Update ee-repo-ref.txt

* Update SQLx metadata

* fix in test interface

* nit

* nit

* fix layout

* use memory_id to store memory

* shorter description

* rls + grant

* fix text overflow

* extract output from res

* cleaning

* handle streaming

* cleaning

* fix tool error

* nit

* update ref

* fix

* Update SQLx metadata

* nit

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: centdix <centdix@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-10-03 14:04:21 +00:00
Ruben Fiszel
87861301f2 fix: make schedule more resilient in case of pg clock shifts 2025-09-26 18:02:34 +00:00
hugocasa
5aac5fa136 feat(backend): prefix matching for list_paths_from_workspace_runnable + bulk delete endpoints for scripts/variables/resources (#6542)
* feat(backend): accept wildcard for list_paths_from_workspace_runnable + bulk delete endpoints for scripts/variables

* nit

* better result

* safer runnables list

* nit

* nits

* bulk delete resources as well

* batch sql queries

* nits
2025-09-08 13:41:01 +02:00
pyranota
d0982432e2 fix(flows): fix relative imports cache invalidation (#6519)
* fix(flows): fix relative imports cache invalidation

Signed-off-by: pyranota <pyra@duck.com>

* update flow_versions in the end

Signed-off-by: pyranota <pyra@duck.com>

* comms + fallback

Signed-off-by: pyranota <pyra@duck.com>

* add warn if fallback is used

Signed-off-by: pyranota <pyra@duck.com>

* include referenced scripts' hashes in flow_node id

Signed-off-by: pyranota <pyra@duck.com>

* improve fallback + TODOs

Signed-off-by: pyranota <pyra@duck.com>

* perf: optimize trigger

Signed-off-by: pyranota <pyra@duck.com>

* reduce TODOs

Signed-off-by: pyranota <pyra@duck.com>

* remove more TODOs

Signed-off-by: pyranota <pyra@duck.com>

* test thing

Signed-off-by: pyranota <pyra@duck.com>

* remove TODO

Signed-off-by: pyranota <pyra@duck.com>

* drop old flow trigger

Signed-off-by: pyranota <pyra@duck.com>

* add comments to migration

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
2025-09-05 16:06:51 +00:00
dieriba
f10cac1c4b feat: conditional retry workflow (#6461)
* feat retry

* fix

* fix

* fix and nits

* fix query

* fix test

* nits and perf

* update sqlx

* once

* remove

* update sqlx
2025-09-03 20:24:38 +00:00
dieriba
f3fd1e90b0 feat: refactor trigger crud (#6472)
* base

* base crate

* websocket updated

* crud refactored

* fix and nits

* fix compiler warning, nits and update repo ref

* fix oss

* fix compilation

* update ref

* fix

* update feature

* listener base

* add listener

* refactor logic done and implemented for postgres

* fix capture

* websocket

* implem for all triggers

* update sqlx,repo ref and fix

* fix oss

* unify struct fix websocket

* nits and fix oss runtime axum error

* perf cache query

* update .sqlx

* update .sqlx

* fix

* fix unused

* fix

* fix

* nits and fix http handler update endpoint

* update .sqlx

* update repo ref

* nits

* fix

* update ref

* fix

* update .sqlx
2025-09-02 11:28:06 +00:00
hugocasa
fbd942f179 fix(backend): retrieve root_job for MiniPulledJob + fix root job for flow jobs (#6490)
* feat(backend): retrieve root_job for MiniPulledJob + fix root job for flow jobs

* nit

* nits

* set root_job only if additional information meaning if not equal to innermost or parent

* nit

* feat: add root job env var

* nits

* fix build

* nit

* fix build

* nit

* sqlx
2025-09-02 09:44:09 +00:00
hugocasa
958e8af782 feat: ai agent steps (#6393)
* feat: ai agent steps base

* better backend and graph

* feat: anthropic, log viewer

* nit

* fix(frontend): hide tool nodes from timeline

* move ai agent actions from flow status to flow status module

* nits and workspace/hub scripts support

* tmp ref

* fix merge

* feat: display agent tools status in the graph

* fix reactivity

* fix flow status

* nit
2025-08-20 22:40:57 +00:00
dieriba
1bcb0431c4 feat: dynamic select in flow (#6315)
* update

* add dyn select for flows

* nits

* fix typo

* done

* update .sqlx

* better

* nits

* fix

* nits

* nits

* fix

* improvements

* update

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-08-06 07:20:03 +00:00
Diego Imbert
2ab5345e61 Assets refactor (#6217)
* Moved logic to FlowAssetsProvider

* Remove assetsMap in flow

* do not parse everything on mount + only check for missing assets fields

* add assets field in backend

* remove fallbackAccessTypes

* better structure and less queries / parsing

* Fix assets not showing when pulling raw_flow from jobs

* flow assets ctx for job run

* Fix transitive assets fetching

* Fix input args asset node

* enablePathScriptAndFlowAssets flag

* edit btn for variable

* untrack refresh

* move parseInputArgsAssets

* Assets tab in runs

* Update FlowStatusViewerInner to svelte 5 + fix asset sync bug

* avoid toast error on bad resource

* fetch res metadata for input arg asset

* Job assets viewer in run page

* r/w selector

* remove indigo badge

* store alt_access_type state in ScriptEditor

* Don't parse assets in flow script editor

* Add alt_access_type in backend

* show Read as selected by default to avoid giving the feeling of having made a decision

* keep alt_access_type when reparsing in flow raw scripts

* Remove variable asset kind, and save assets for scripts

* remove all backend asset parsing

* R/W/RW selector button nits

* fix insert into assets not saving alt access type

* support named arguments in python asset parser

* improve asset usage drawer R/W indicator

* update legacy $res: syntax

* reactivity issue

* remove last variable asset stuff

* sqlx prepare

* tooltip explainer

* deprecated variable asset nit
2025-07-17 22:15:01 +00:00
dieriba
5f364100f3 feat: granular token scopes (#6093)
* base

* add scopes in the UI

* remove legacy scope, unified create token into a components

* fix layout, convert old scope to new scope

* update scope

* update ui, and clear scope

* remove desc

* almsot there

* fix path

* fix delete and scope resource path matching

* update scope

* update scope and error message

* nits and fix logic

* nits and fix

* added multiple resource and jobs scope for script and flow

* add check_scope for endpoint

* clean front and improve fronetend code

* fix resource validation logic and backward compatibility with old scope

* fix frontend state and scope checks logic

* update scopes

* fix height

* nits: better_naming

* fix route

* fix add missing import

* fix import and move fn

* update repo ref

* fix import

* fix query and nits

* nits

* fix ,missing import

* fix

* revert add admin protection

* handle run action correctly

* fix

* add check scopes to some endpoint

* fix and nits

* nits

* remove unused import

* nits

* add chevron when domain is exapanded

* fix border bottom

* nits adds resource path button

* nits

* fix

* nits

* nits

* nits

* fix merge

* fix

* UI nits

* update repo ref

* add lazy static

* update scopes

---------

Co-authored-by: HugoCasa <hugo@casademont.ch>
2025-07-17 15:18:40 +00:00
HugoCasa
328ef605ad feat: triggers error handler and retry (#6138)
* feat: http triggers error handling and retries

* feat: handle sync http triggers

* feat: triggers retry and error handling

* feat: frontend and preprocessor

* fix build

* minimum allow dead code

* check workspace tags in schedules

* fix flow preprocesor + hub urls

* update ee ref

* nit

* nit

* final nits

* schedule UI nits

* migrate workspace settings to svelte 5
2025-07-11 18:55:23 +02:00
Diego Imbert
433341b295 feat: assets as a primary concept (#6125)
* assets migration

* parse assets (duckdb)

* iterate on assets

* S3 object Preview

* remove pagination

* filterText

* better occurence list

* tweak

* assets in JobPreview

* clone impl

* AssetsDetectedBadge

* improve DbManagerButton + asset dropdown button

* edit resource btn

* warning when incorrect resource

* +Resource in DuckDB

* +S3 Object editor bar

* nit fix rename

* flow asset badge

* More Generic OnChange

* Highlight assets used in modules

* Show occurence count in flow

* Better UX, avoid moving parts

* nit

* Asset nodes

* move to dedicated Asset ctx

* fix layoutNodes not handling first assetsMap

* explore asset btn in flow asset node

* correct offset

* single computeAssetNodes function

* Fix y positioning of nodes with assets

* resource editor

* write mode node (ui)

* accessType in ctx + fix insert button positioning

* right positioning when mixing read and write nodes

* right positioning when mixing R and W assets

* Better layout fix algorithm

* listAssetsByUsage and asset nodes on transitive usages

* refactor + remove linkAssets

* Refactor to allow for custom R/W modes

* AssetsDropdownButton in flow script editor

* R/W/RW selection and changes node pos in flow

* layoutNodes doesnt need recompute now

* fix wrong assumption that nodes recompute when assets change

* r/w/rw multi toggle

* MultiToggle cool animation + clearable

* rename + 1px nit

* remove mini toggle button group, use ToggleButtonGroup

* Combinator parser that detects R / W asset context

* nit fix missing flex-1

* missing order by

* better ui indication for access type

* special x offset case when only one asset node for clarity

* parse getResource in TS with swc ecma parser

* support load and write s3 detection in TS

* Python asset parser

* support wmill api calls without special $res: or s3:// syntax

* detect out of context asset uris python

* do not use access type override when not ambiguous in flow graph

* parse_assets match case in rust

* AsRef<str> refactor

* From impl

* Save flow assets

* Save script asset usages + fixes + save fallback access types

* asset sub icon

* max total asset node width to avoid overlap

* small refactor

* don't parse comments in duckdb assets

* fix assets clearing on parse error

* fix script asset save in wrong place

* load initial asset fallback access types

* support variables

* ui fixes

* Support S3Object as URI in TS client

* support new syntax in python client

* Support +S3Object in EditorBar for TS and python

* Reduce resource requests in assets page

* import windmill client when necessary

* update s3Types.d.ts

* nit fix

* Show input resources and s3 objects as assets

* improve asset icons

* DarkModeObserver refactor

* asset page tabs

* Moved resource variables and s3object pages to assets tabs

* fetch resource usages

* Get variables usages

* move assets usage dropdown to component

* Revert "move assets usage dropdown to component"

This reverts commit 622ea4ab12.

* Revert "Get variables usages"

This reverts commit b11ced4e29.

* Revert "fetch resource usages"

This reverts commit aa5187ad4b.

* Revert "Moved resource variables and s3object pages to assets tabs"

This reverts commit 4430487be4.

* Revert "asset page tabs"

This reverts commit dacc2f0da5.

* move assets usage dropdown to component

* asset icon in asset pages

* tooltip

* details

* Storage selector in S3 File Picker

* make edge less opaque

* Refactor computeAssetNodes to separate in and out nodes

* AssetsOverflowedNode

* nits

* fix assets not being parsed in flows sometimes

* show asset kind and resource_type

* ui nits

* support res:// in duckdb

* add banner for old deployments

* Fix permissionning

* fix broken disable /enable all

* assets page view permission for operators

* Disable ExploreAssetButton for operators

* asset kind as subtitle

* do not spam getResource in assets page. prob. revert fail

* update assets page on workspace change

* reload storage names on ws change

* delete assets on archive / deletion

* sqlx prepare

* missing update when updating user

* add indexes on asset

* better message

* missing loadInit: false

* dead code

* use transaction

* typo

* update package.json

* update package.json

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-07-11 09:56:53 +00:00
wendrul
835645643e feat: Better tracing for audit logs, including a graph to visualize them (#6078)
* Migrate audit log page to svelte 5

* Add email and span cols to audit table

* Add token_prefixs to audit logs (into AuditAuthorable trait)

* Add audit logs graph (wip)

* Add audit span on push and jwt

* Unify same job audit into the same audit span

* Improve the graph visually

* Fix typo

* functioning graph with svelte issue

* Fix leak

* feat: migrate AuditLogsTable from DataTable to VirtualList for performance

- Replace DataTable component with VirtualList for handling thousands of rows
- Migrate to Svelte 5 runes ($props, $bindable, $derived, $state)
- Implement flattenLogs() for virtual scrolling with grouped date headers
- Add sticky indices and dynamic height calculation
- Update parent component to use callback prop pattern instead of events
- Preserve all existing functionality: filtering, selection, pagination
- Follows RunsTable.svelte implementation pattern

Resolves performance issues when displaying large audit log datasets.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

* Fix remaining virtual list issues

* WIP graph

* Fix chart styling

* Fix npm check

* Fix missing audit_span arguments

* Update sqlx

* use varchar 255 for email as in other tables

* Remove syntax inconsistency

* Match struct with ee crate

* Update ee-repo-ref.txt

* Update worker_flow.rs

* Remove redefinition of trait to prevent shadowing

* Re add trait on oss but only when no `private` flag

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-07-03 21:02:45 +00:00
Ruben Fiszel
18cb8324ed add more quotas to prevent abuse on cloud 2025-06-24 06:42:42 +02:00
dieriba
aba8c01d7f feat: windmill http triggers and webhooks to openapi spec (#5918)
* add sum and description to http routes

* add to openapi spec

* add subsection

* add collaspable and action button

* added path rendering,

* migrate logic to backend

* remover server and handle different format generation

* add filter for http route and document generated

* remove print

* handled webhook and integrated server component

* done

* add download and copy

* nits

* update .sqlx

* remove .vscode

* fix npm check

* add summary description and fix toggle

* added security handling and nits

* update .sqlx

* nits and rename key

* remove unused code

* update ref

* nits rename var

* nits

* nits

* nits

* nits

* nits

* add token generation for cURL command

* create token

* update label and remove section

* update repo ref

* clean

* let brower start download before cleaning up resource

* handle sync/async for webhook

* format fix

* nits

* nits

* reset sum and description
2025-06-17 19:49:00 +02:00
Ruben Fiszel
8b7aefb3bc improve tag of new flow dependency job 2025-06-17 09:12:23 +02:00
Diego Imbert
0e316239dd EE Refactor (#5844)
* app compiles with every ee substituted

* Replace all oss files content

* Revert "Replace all oss files content"

This reverts commit ea4017d59f.

* delete all ee

* hide all _ee files under private flag

* hide every oss stuff when private flag set

* pub use *

* gitignore and substitute script

* pub mod for ee needed for ee repo

* small mistakes

* remove oidc_oss impl

* ee ref (temp)

* ee ref

* fix --all-features selecting private in OSS CI

* ee repo ref

* allow unused
2025-06-02 22:12:33 +02:00
dieriba
242a565428 feat: raise error if end early in flow (#5653)
* update front

* handle err_msg display if enabled when stop condition is met

* nits

* Update backend/windmill-worker/src/worker_flow.rs

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>

* Update openflow.openapi.yaml

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update worker_flow.rs

* Update worker_flow.rs

* update flow test

* update condition

* remove println

* nits

* fix flow test

* nits and missing property on struct definition

* nits

* fix unsaved state

* update .sqlx

* cleanup sqlx

---------

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-05-07 18:23:40 +02:00
Diego Imbert
e868fe2bf5 feat: list references upon renaming a script or a flow (#5487)
* Refactored flow_workspace_runnables to more generic workspace_runnable_dependencies

* list flows referencing an item upon renaming it

* Refactor with two exclusive columns to avoid breaking FK constraints

* Show apps depending on item upon renaming

* sqlx prepare

* list-disc instead of •

* on delete and on update cascade

* displayPathChangedWarning oneOf check instead noneOf

* combine migrations + add "on update cascade" to flow fk

* unique index on app dependencies to avoid duplicates

* create new workspace_runnable_dependencies instead of renaming old table

* Add "looking for references" loading msg

* Revert "create new workspace_runnable_dependencies instead of renaming old table"

This reverts commit 015c38ca8f.

* flow_workspace_runnables view for backwards compatibility

* Add warning for script imports on rename

* support import dependency tracking in deno

* number of using scripts / flows / apps tooltip

* forgot sqlx prepare

* delete app-related rows in down migration
2025-03-24 22:08:19 +01:00
HugoCasa
4f43b1984f feat: keep captures across drafts and deploys (#5482) 2025-03-14 15:17:48 +01:00
HugoCasa
7bf9e25ede feat: track workspace runnables used in flows (#5369)
* feat: track workspace runnables used in flows

* track script hash

* weird

* do it with lock

* Revert "feat: add support for | None and Optional in python (#5361)"

This reverts commit 9736355d5f.

* Revert "Revert "feat: add support for | None and Optional in python (#5361)""

This reverts commit bb8f709894.

* update openapi

* delete old in lock_modules + don't track hub scripts
2025-02-27 10:01:12 +01:00
Ruben Fiszel
f90841ac5a fix missing lock error logs 2025-02-23 09:18:50 +01:00
Ruben Fiszel
b8e6d0da79 feat: make flow lock deployment error visible in UI 2025-02-22 15:02:16 +01:00
Lucas Abel
d51c3080f7 backend: use v2 tables through views where possible (v2 phase 3) (#5119) 2025-02-06 12:43:24 +01:00
Ruben Fiszel
90ba65ae20 improve error messages for internal err 2025-02-05 16:32:01 +01:00
HugoCasa
04fbda28e8 feat: on behalf of (#5058)
* feat: on behalf of

* fix

* revert mistakke

* temp ee ref

* renaming + CLI

* sqlx

* nits + ee ref
2025-01-14 18:32:25 +01:00
HugoCasa
01c6f84520 feat: captures (#4807)
* feat: captures

* flow UI and improvements

* fix: build

* fix sqlx

* Move Capture WIP

* Add capture to webhook and websocket

* Move connection status viewer in the head

* change trigger section label

* Add popover capture picker using melt ui

* Add shortcut to triggers capture from input form

* remove capture tab in input

* Allways show capture

* remove useless logs

* Add email capture

* Add kafka capture into triggers

* Add edit option in capture table

* use light header for arg input

* Add prefilled group id

* Change button label

* fix logic

* Open resource drawer if prototype has fields

* fix default completion

* Change name Prototype

* Dissociate Editor Mode for triggers

* Fix bug for script

* Fix apply args

* fix apply schema

* Add capture table to script

* fix apply args

* Add capture button for script

* Delete capture tab

* Set capture on when opening triggers capture

* fix connection indicator

* fix minor issues

* Add preprocessor logic

* Use slot in log Panel for captures

* handle capture refresh in script

* Delete capture tab from script editor

* reset kafka resource on toggle static

* fix minor issue

* Allow resource in kafka capture

* use simple capture button in flow

* Remove capture panel

* Polish route trigger editor

* Fix resource saving

* Remove excessive padding

* merge nits

* add workflow_dispatch to build

* add workflow_dispatch to build

* better capture UI

* fix sqlx

* fix build

* fix build

* build

* make initial_messages optional in line with db

* fix npm check

* better handle args for capture webhook and http

* improve migration + http capture fixes

* fix sqlx

* update ee ref

---------

Co-authored-by: Guilhem <guilhemlemouel@gmail.com>
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-01-06 12:07:09 +01:00
Lucas Abel
2911775d0a feat(backend): store flow inner modules into flow_node table (#4778) 2024-11-27 16:08:31 +01:00
Ruben Fiszel
89456f502f remove rsmq 2024-11-24 11:04:54 +01:00
Guilhem
22ab51e991 feat(frontend): improve and simplify scheduled poll flows (#4560) 2024-11-04 23:18:09 +01:00
Ruben Fiszel
2a136effe7 nit test serializer 2024-10-29 09:09:08 +01:00
pyranota
11623682c2 Deploy collision warning fix (#4544)
* feat: Multiplayer deploy collision warning

Show warning with diff viewer if 2+ users try to deploy flow/script/app at the same time

In other words, if user A and user B started editing same flow/script/app and user A deployed it,
user B will get warning if they try to deploy as well.

* Add warning for scripts

* Add `deployedBy` to Apps

* Format

* Fix advanced deployment on scripts

* Write comments and cleanup

* feat(frontend): unify all triggers UX and simplify flow settings (#4259)

* feat(frontend): added list of triggers in the flow graph

* feat(frontend): added list of triggers in the flow graph

* feat(frontend): clean up

* feat(frontend): improve UX

* feat(frontend): triggers

* feat(frontend): triggers

* feat(frontend): done

* feat(frontend): fix trigger when position when a preprocessor is presetn

* Glm/rework flow settings v2 (#4497)

* fat(frontend): simplify flow settings menu

* improve scroll

* changing mute toggle

* Add advanced settings badge

* Add nord theme colors

* Add bage for advanced options

* fix minor issue

* fix minor issue

* Add triggers menu to flow settings

* Add quick trigger access

* remove triggers in flow settings

* fix minor issue

* Move triggers settings to flow right panel

* polishing

* fix unset store

* remove save up to for triggers

* fix padding

* reset default tag color

* remove custom select component

* revert path change

* revert section modif

* Revert unused feature

---------

Co-authored-by: Guilhem <guilhem@mbp-de-windmill.home>

* Connect top bar cron to schedules settings

* Turn copilot into node

* fix copilot placement

* remove useless import

* fix center copilot

* fix binding

* remove copilot on top of preprocessor

* render copilot node on condition

* quickfix

* remove copilot node

* fix minor issues

* fix route count update

* fix schedule sync

* harmonize colors

* fix alignment and add edges

* recenter node summary

* fix schedules sync

* Add id title

* all

* all

* all

* iteration

* all

* all

* done

* fix

* more fixes

---------

Co-authored-by: Guilhem <guilhemlemouel@gmail.com>
Co-authored-by: Guilhem <guilhem@mbp-de-windmill.home>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>

* Update ScriptBuilder.svelte

* Remove `onMount` for flows

* Use version instead of last_updated_at in flows

* Use only versions for apps

* Fetch latest data in Diffs in Apps

* Optimize with (script/flow/app)GetLatestVersion

Create several new endpoints, that returns just latest version without rest of the history

* Sync Diffs with deployed

* Improve Diff's data

* Use `getFlowLatestVersion`

* fix failing endpoints for deployed collision warning

---------

Co-authored-by: Faton Ramadani <faton.ramadani14@gmail.com>
Co-authored-by: Guilhem <guilhemlemouel@gmail.com>
Co-authored-by: Guilhem <guilhem@mbp-de-windmill.home>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-10-17 18:05:44 +02:00