Commit Graph

688 Commits

Author SHA1 Message Date
Ruben Fiszel
f1029d0f14 s3 endpoints improvements 2025-11-15 19:07:13 +00:00
dieriba
09cdfb4556 feat: webhook by flow version (#7062)
* ok

* done

* update sqlx and repo ref

* fix
2025-11-14 14:36:24 +00:00
centdix
3dd293e450 Revert "feat(ai): handle aws bedrock as provider (#7131)" (#7141)
This reverts commit 30eb9aae25.
2025-11-14 12:31:16 +00:00
centdix
30eb9aae25 feat(ai): handle aws bedrock as provider (#7131)
* backend draft

* fix for tool and streaming

* do frontend side

* working

* working tools

* rm

* handle list endpoint

* handle for ai agents

* fix for models requiring inference id

* cleaning

* fix desc issue

* fix tool usage

* fix structured output

* cleaning

* fix for api

* rm

* fix input images
2025-11-14 12:22:51 +00:00
Diego Imbert
c079155e3e SQL result_collection flag (#7041)
* Return array of results duckdb

* Migration script to add result_collection=legacy

* migration script fixes

* app_version_lite not necessary

* New annotations macro that supports custom types

* pass unit tests

* nit style

* result_collection almost works for postgres

* fix last_statement

* frontend suggesitons

* fix column_order making columns disappear

* added version check for duckdb FFI lib to avoid crashes when changing FFI interface on Windows agent workers

* result_collection for duckdb

* Correct legacy behavior in DuckDB

* mysql result_collection

* mssql collection_strategy

* result_collection for oracle

* snowflake result_collection

* fix errors

* mistake, .clone() causing deadlock

* fix frontend errors on oracle / mssql

* fix legacy behavior on mssql

* add exception catch in migration

* missing app_script update

* change cache layout to avoid worker pulling old scripts
2025-11-12 15:32:07 +00:00
Pyra
84992cd8ff fix: proper error handling in pulled job preprocessor (#7098)
* fix: proper error handling in pulled job preprocessor

Signed-off-by: pyranota <pyra@duck.com>

* follow up for merge

Signed-off-by: pyranota <pyra@duck.com>

* make it safe

Signed-off-by: pyranota <pyra@duck.com>

* clippy

Signed-off-by: pyranota <pyra@duck.com>

* remove unused import

Signed-off-by: pyranota <pyra@duck.com>

* use String instead of Value

Signed-off-by: pyranota <pyra@duck.com>

* update ee ref

Signed-off-by: pyranota <pyra@duck.com>

* implement Error for PulledJobResultToErr

Signed-off-by: pyranota <pyra@duck.com>

* updatesqlx

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
2025-11-10 13:43:50 +00:00
dieriba
c59183f5c3 feat: env var in flow (#6852) 2025-11-07 18:38:55 +00:00
centdix
49524d5b28 chore(aiagent): store memory in db (#7053)
* store to db

* no warning in context

* warning in frontend

* remove log

* 100kb + alert

* update sqlx

* update eeref

* Update ee-repo-ref.txt

* cleaning

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-11-07 15:27:00 +00:00
Ruben Fiszel
ad43680bb9 fix: dep jobs improvements (#7081)
* dep jobs improvements

* update
2025-11-07 11:04:53 +00:00
Ruben Fiszel
ad861f5312 formalize ai input transform (#7078)
* ai input transform

* fix
2025-11-07 08:03:44 +00:00
hugocasa
1edfdce0dd feat: hub actions and better search (#7056)
* feat: hub actions and better search

* nit
2025-11-05 11:10:48 +00:00
Ruben Fiszel
4a6d02960c nits 2025-11-04 19:13:09 +00:00
centdix
e6d700878f fix typo (#7044) 2025-11-04 11:36:04 +00:00
Ruben Fiszel
2d1c1d81ba internal(backend): mini completed job + improvements (#7042)
* mini completed job

* mini completed job

* sqlx

* restore dedi

* restore dedi

* nits

* fix
2025-11-04 09:31:11 +00:00
hugocasa
16317a4714 fix: consider duckdb as a normal tag (#7035) 2025-11-03 10:55:13 +00:00
hugocasa
c04489c463 fix: include missing tags from default/native consts (#7034) 2025-11-03 10:31:33 +00:00
dieriba
b5e341fde7 perf: parse flow value only if needed (#7025)
* nits

* renames
2025-11-01 15:08:34 +01:00
dieriba
5f6aabb667 nits: warning retry (#6760)
* done

* ok

* ok

* better

* update sqlx

* raw val
2025-11-01 10:40:43 +00:00
hugocasa
cf86881528 fix(backend): add locks to inline preprocessor/failure/tool modules in flows (#6825)
* fix(backend): add locks to inline preprocessor/failure/tool modules in flows

* fix merge

* fix tests

* fix tests

* add test for ai agent tool

* nit

* fix build

* fix locking
2025-10-30 10:33:36 +00:00
hugocasa
cdf8fdf4a4 fix(backend): more efficient worker telemetry (#6997)
* fix(backend): more efficient worker telemetry

* update ref

* update ee ref

* Update ee-repo-ref.txt

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-29 21:01:32 +00:00
Ruben Fiszel
5b315bf2c8 fix: improve jobs runs page performance (#6942) 2025-10-27 14:29:53 +00:00
Ruben Fiszel
5b1ab610b7 improve stack size usage (#6936)
* oom

* final

* final
2025-10-24 22:08:54 +00:00
Pyra
999aaaacd8 fix(debouncing): fix perf issues and re-enable debouncing (#6932)
* fix(debouncing): fix perf issues and re-enable debouncing

Signed-off-by: pyranota <pyra@duck.com>

* add debounce data in clone_script

Signed-off-by: pyranota <pyra@duck.com>

* update sqlx cache

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
2025-10-24 15:24:46 +00:00
Ruben Fiszel
7348b5df99 temporarily disable debouncing 2025-10-24 10:49:38 +00:00
hugocasa
7b46491578 fix(cloud): better errors when failing to get team plan status (#6908)
* fix(cloud): better errors when failing to get team plan status

* better errors

* fix build

* fix build
2025-10-23 20:19:10 +00:00
Pyra
edece035f8 feat: job debouncing (#6878)
* feat(EE): job debouncing

Signed-off-by: pyranota <pyra@duck.com>

* remove 'script' file

Signed-off-by: pyranota <pyra@duck.com>

* more work

Signed-off-by: pyranota <pyra@duck.com>

* properly gate it behind enterprise

Signed-off-by: pyranota <pyra@duck.com>

* update ee repo ref

Signed-off-by: pyranota <pyra@duck.com>

* change ee repo ref again

Signed-off-by: pyranota <pyra@duck.com>

* remove unused variable

Signed-off-by: pyranota <pyra@duck.com>

* feat(EE): implement TODOs and enhance tracing for job debouncing

- Add database index on script(workspace_id, debounce_key) for efficient lookups
- Update minimum version requirement to 1.564.0 throughout codebase
- Add tracing warnings when debouncing is disabled due to worker version mismatch
- Fix all documentation links from TODO placeholders to proper URLs
- Replace Gauge icon with Timer icon for debouncing UI elements
- Update placeholder text and tooltips with clear descriptions

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* create -> crate

Signed-off-by: pyranota <pyra@duck.com>

* remove index

Signed-off-by: pyranota <pyra@duck.com>

* some updates

Signed-off-by: pyranota <pyra@duck.com>

* fix once more

Signed-off-by: pyranota <pyra@duck.com>

* fix it once more

Signed-off-by: pyranota <pyra@duck.com>

* Remove flow step debouncing, keep top-level flow debouncing

- Remove debounce fields from RawScript and FlowScript FlowModuleValue variants
- Remove debounce fields from JobPayload::FlowScript and RawCode
- Update raw_script_to_payload function signature
- Remove debouncing UI from flow step runtime settings
- Remove debouncing toggle handler and indicator badge
- Preserve top-level flow debouncing in FlowSettings

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fixup claude's work

Signed-off-by: pyranota <pyra@duck.com>

* cleanup: remove dbg! statements, update min version to 1.566.0, add comprehensive comments

- Removed all dbg! macro calls from production code
- Updated MIN_VERSION_SUPPORTS_DEBOUNCING from 1.564.0 to 1.566.0
- Added comprehensive documentation comments explaining:
  - Debouncing feature purpose and mechanics
  - Database schema for debounce_key and debounce_stale_data tables
  - Version check logic and guard functions
- Improved code clarity and maintainability

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* improve fallback

Signed-off-by: pyranota <pyra@duck.com>

* remove comments from old migration

Signed-off-by: pyranota <pyra@duck.com>

* fix pull

Signed-off-by: pyranota <pyra@duck.com>

* fix once more

Signed-off-by: pyranota <pyra@duck.com>

* Update frontend/src/lib/components/ScriptBuilder.svelte

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* add sqlx cache

Signed-off-by: pyranota <pyra@duck.com>

* fix: frontend CI - fix debounce field names and remove leftover flow step debouncing

- Fix ScriptBuilder.svelte: change custom_debounce_key to debounce_key
- Add debounce_key and debounce_delay_s fields to NewScript schema in openapi.yaml
- Regenerate frontend types from OpenAPI spec
- Remove leftover flow step debouncing code from FlowModuleComponent.svelte
- Remove debounce fields from RawScript in openflow.openapi.yaml
- Remove unused Timer import from FlowModuleHeader.svelte

All frontend checks now passing (0 errors, 0 warnings)

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* fix ci

Signed-off-by: pyranota <pyra@duck.com>

* remove unused import

Signed-off-by: pyranota <pyra@duck.com>

* fix ci again

Signed-off-by: pyranota <pyra@duck.com>

* udpate ee repo ref

Signed-off-by: pyranota <pyra@duck.com>

* CI doesn't want to be fixed but I still try

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* ci...

Signed-off-by: pyranota <pyra@duck.com>

* Update ee-repo-ref.txt

* safer migration

Signed-off-by: pyranota <pyra@duck.com>

* reduce noise in logs

Signed-off-by: pyranota <pyra@duck.com>

* fix cli for scripts

Signed-off-by: pyranota <pyra@duck.com>

* nit

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Pyra <pyranota@users.noreply.github.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-23 20:18:08 +00:00
Alexander Petric
fc7b65b193 nit: also log force ipv4 in configure_client (#6924) 2025-10-23 14:58:45 +00:00
Alexander Petric
9b956902ae fix windows typo (#6900) 2025-10-22 17:48:38 +00:00
Ruben Fiszel
9b1a7f5a3a fix(windows): improve 2nd shutdown monitor 2025-10-22 08:42:36 +00:00
Pyra
caa21fcb33 fix: debounce_key automatic deletion (#6885)
* fix: debounce_key automatic deletion

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* better solution

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-21 18:31:05 +00:00
Alexander Petric
d777c7798b feat: allow optionally forcing ipv4 for reqwest (#6883) 2025-10-21 15:46:41 +00:00
centdix
97ac1be036 feat(aiagent): allow mcp as tools (#6790)
* draft mcp client

* testing

* fix

* cleaning

* mcp resource in inputtransforms

* cleaning

* big cleaning

* cleaning

* no arc

* add utils file

* refactor tools

* add mcp actions

* draft frontend

* send arguments from backend

* better frontend

* cleaning

* use token for auth

* add logo

* rm

* fix

* fix

* chore: refactor mcp for ai agents (#6829)

* Add Tool enum for AIAgent with backward compatibility

- Created Tool enum that can be either Windmill (FlowModule) or Mcp (resource reference)
- Created McpToolRef struct to hold MCP resource path
- Implemented custom Deserialize for Tool with backward compatibility:
  - New format: {type: 'windmill'|'mcp', ...}
  - Old format: FlowModule objects (automatically wrapped in Tool::Windmill)
- Updated AIAgent to use Vec<Tool> instead of Vec<FlowModule>
- Updated FlowValue::traverse_leafs to handle Tool enum
- Backward compatible: old flows with Vec<FlowModule> will deserialize correctly

* Refactor AI executor to process Tool enum instead of extracting MCP from input_transforms

- Separate Windmill tools and MCP resource paths from tools list
- Process Windmill FlowModules into Tool definitions
- Load MCP tools from resource paths in Tool::Mcp variants
- Remove old logic that extracted mcp_resources from input_transforms
- Import FlowModule, remove unused InputTransform
- Fix type issues: use .as_str() for path and handle Option<bool> properly

* handle in args

* mcp as flowmodule

* frontend

* config for mcp

* simplify logic

* fix ai executor logic

* cleaning

* clean frontend

* fix

* better resource picker

* fix and styling

* add endpoint to fetch tools

* apply tool filtering

* fix name validation

* better ui

* use cache

* fix

* fix merge

* refactor: Separate MCP tools from FlowModule in AIAgent

- Add new AgentTool, ToolValue, and McpToolValue types
- Update AIAgent to use Vec<AgentTool> instead of Vec<FlowModule>
- Implement From traits for clean conversion between AgentTool and FlowModule
- Add backward compatibility via custom deserializer for AgentTool
- Simplify resolve_module logic by reusing existing resolve_modules function
- Update traverse_leafs to handle AgentTool structure

This refactoring separates MCP tools from FlowModule tools, making the
type system clearer and eliminating the need to treat MCP servers as
a special case of FlowModule.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: Update ai_executor and worker_lockfiles for AgentTool

- Update ai_executor.rs to handle new AgentTool structure
  - Separate MCP tools from FlowModule tools using ToolValue enum
  - Convert AgentTool to FlowModule for backward compatibility
  - Add imports for AgentTool and ToolValue types

- Update worker_lockfiles.rs for lazy loading optimization
  - Convert AgentTool <-> FlowModule in insert_flow_modules
  - Preserve lazy loading for FlowModule tools via modules_node
  - Keep MCP tools inline (lightweight, no need for lazy loading)
  - Maintain backward compatibility with existing flows

This enables the lazy loading optimization for FlowModule tools while
keeping MCP tools inline, balancing performance and simplicity.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* cleaning

* adapt frontend

* cleaning

* cleaning

* type fix

* cleaning

* fix back comp

* move mcp button position

* nit

* cleaning

* fix nested removal

* cleaning

* opti

* fix chat markdown display

* fix chat messages layout

* fix back comp frontend

* fix deserializer

* nit

* simpler serializer

* use if else

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-21 15:28:52 +00:00
Pyra
defb6c9694 feat: dependency job debouncing (#6769)
* v0

Signed-off-by: pyranota <pyra@duck.com>

* optimize relocks

* make it work with relative relative imports

Signed-off-by: pyranota <pyra@duck.com>

* use fallback

Signed-off-by: pyranota <pyra@duck.com>

* remove dbg and todos

Signed-off-by: pyranota <pyra@duck.com>

* future proof a bit

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* more cleanup

Signed-off-by: pyranota <pyra@duck.com>

* remove final TODO

Signed-off-by: pyranota <pyra@duck.com>

* do not use bytemuck

Signed-off-by: pyranota <pyra@duck.com>

* optimize hashing

Signed-off-by: pyranota <pyra@duck.com>

* implementation 1

Signed-off-by: pyranota <pyra@duck.com>

* almost v0

Signed-off-by: pyranota <pyra@duck.com>

* v0

Signed-off-by: pyranota <pyra@duck.com>

* add comments and use fallback

Signed-off-by: pyranota <pyra@duck.com>

* call dissolve for apps

Signed-off-by: pyranota <pyra@duck.com>

* add comms

Signed-off-by: pyranota <pyra@duck.com>

* refactor v0 (partially tested + dirty)

Signed-off-by: pyranota <pyra@duck.com>

* finishing

Signed-off-by: pyranota <pyra@duck.com>

* remove TODO

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* silence unused argument

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* implement rebuild_map endpoint

Signed-off-by: pyranota <pyra@duck.com>

* update windmill api client

Signed-off-by: pyranota <pyra@duck.com>

* almost finish with tests

Signed-off-by: pyranota <pyra@duck.com>

* add proper testing

Signed-off-by: pyranota <pyra@duck.com>

* remove unused fixtures

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* partial cleanup

Signed-off-by: pyranota <pyra@duck.com>

* Update backend/windmill-worker/src/scoped_dependency_map.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-common/src/scripts.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* evil doings

Signed-off-by: pyranota <pyra@duck.com>

* more cleanup

* Update SQLx metadata

* more cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fixing CI

Signed-off-by: pyranota <pyra@duck.com>

* remove python from default features

Signed-off-by: pyranota <pyra@duck.com>

* feat: dependency job debouncing

* checkpoint

Signed-off-by: pyranota <pyra@duck.com>

* more improvements

Signed-off-by: pyranota <pyra@duck.com>

* refactor: clean up dependency job debouncing implementation

- Add comprehensive comments explaining the debouncing mechanism
- Replace debug statements (dbg!) with proper tracing calls
- Extract helper functions to reduce code duplication:
  - extract_to_relock_from_args() for extracting nodes/components
  - accumulate_debounce_stale_data() for updating stale data
- Improve code readability and maintainability

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* Update backend/windmill-common/src/jobs.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-common/src/jobs.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-queue/src/jobs.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* better error handling for helper

Signed-off-by: pyranota <pyra@duck.com>

* update ee-ref

Signed-off-by: pyranota <pyra@duck.com>

* test: add race condition test for dependency job debouncing

- Implement test_2 for scripts to test the race condition edge case
- Add comprehensive documentation comments to all test functions
- Remove empty test_2 stubs for flows (not needed)
- Keep test_2 stub for apps with TODO comment

The race condition test simulates the scenario where a job is marked
as running but debounce_key hasn't been cleaned up yet, forcing the
system to create a new job while reusing the existing debounce_key.

This edge case can occur due to the lack of transactions in the pull
function for performance reasons (see jobs.rs:4415-4425).

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* test: implement race condition test for dependency job debouncing

- Add comprehensive test_2 for script module that tests the race condition edge case
- Remove empty test_2 stubs from flows and apps modules
- Fix unused variable warning in worker_lockfiles.rs
- Add detailed comments explaining the race condition scenario and test logic

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* implement fallback

Signed-off-by: pyranota <pyra@duck.com>

* make it mostly work

* all tests are almost working

Signed-off-by: pyranota <pyra@duck.com>

* add comments a bit

Signed-off-by: pyranota <pyra@duck.com>

* Update backend/windmill-common/src/scripts.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-worker/src/worker_lockfiles.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-worker/src/worker_lockfiles.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* feat: improve debouncing documentation, tracing, and error handling

- Add comprehensive 3-phase debouncing documentation explaining push/accumulation/pull
- Enhance all tracing with structured logging (job_id, workspace_id, node_count, etc.)
- Add proper error handling with .map_err() and contextual messages
- Replace dbg!() with proper tracing::debug!()
- Replace todo!() with proper error handling
- Fix typos: 'and edge case' → 'an edge case', 'bc' → 'because'
- Fix debug variable name: 'debounce_job_id_0' → 'debounce_job_id_o'
- Add documentation for debounce cleanup and stale data retrieval
- Add trace-level logging for non-error paths to reduce noise

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* do some work for future improvements

Signed-off-by: pyranota <pyra@duck.com>

* fix tests

Signed-off-by: pyranota <pyra@duck.com>

* clippy

Signed-off-by: pyranota <pyra@duck.com>

* update sqlx

Signed-off-by: pyranota <pyra@duck.com>

* clippy

Signed-off-by: pyranota <pyra@duck.com>

* update ee ref

Signed-off-by: pyranota <pyra@duck.com>

* flag tests behind the feature, add timeout

Signed-off-by: pyranota <pyra@duck.com>

* fix timeout + cleanup

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* row lock debounce_key

Signed-off-by: pyranota <pyra@duck.com>

* addressing TODOs

Signed-off-by: pyranota <pyra@duck.com>

* fix test

Signed-off-by: pyranota <pyra@duck.com>

* ee ref

Signed-off-by: pyranota <pyra@duck.com>

* ee repo

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: Pyra <92104930+pyranye@users.noreply.github.com>
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Pyra <pyranota@users.noreply.github.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-10-16 16:49:35 +00:00
hugocasa
3b5c962473 fix(backend): revert flow node opti for ai agents (#6840)
* Revert "feat(backend): use flow nodes opti for ai agent steps (#6808)"

This reverts commit 8d5acda340.

* fix(backend): revert flow node opti for ai agents

* keep standard base64
2025-10-16 14:05:06 +00:00
Ruben Fiszel
aeb6829011 use excluded where relevant 2025-10-16 09:19:44 +00:00
wendrul
32fae7a10c feat: ansible playbook execution git repo mode (repo viewer + UI utils) (#6831)
* Improve minio on flake.nix

* Add first asset parsing logic for ansible

* Correct html gt sign

* Decouple s3 file picker from drawer

* Factor duplicate code into snippet

* Update S3FilePickerInner to be compatible

* Fix pane shrinking issue

* Git repo viewer

* Change GitRepoViewer

* Endpoints for git repo visualizer

* Move git repo viewer to its own component

* Add button to populate git repo viewer

* Update parser yaml for new ansisble features (repo viewer)

* Reflect parser changes for ansible

* Add button to add the git repo mode of declaration for ansible

* Factor function

* Playbook + inventories into the drawer

* Add button to add inventories from s3

* Move tests to lib.rs

* Inventory loading from s3

* Move get github app token logic to be reused by ansible

* Update parser and ansible executor

* Use the correct path for inventories

* Add nushell to flake for wasm builds

* Add published parser

* Update hubPaths with clone and upload to s3

* Update ee-repo to the branch ref

* Fix npm run check

* Update cargo.lock

* Change labels on buttons

* Remove debug log

* Update ee-repo-ref

* Fix ee issues

* Update ee-repo ref

* Fix typo

* Fix ee

* Update ee-repo-ref

* Fix missing imports

* Unused var

* Fix typo

* Layout improvents

* Fix typos

* Remove unused function and log

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-10-16 09:08:18 +00:00
Ruben Fiszel
3d5631938f error 2025-10-15 22:12:09 +00:00
hugocasa
8d5acda340 feat(backend): use flow nodes opti for ai agent steps (#6808) 2025-10-14 18:06:27 +00:00
centdix
dc4582a1bb feat(flow): show tool usage in flow conversations (#6771)
* save each assitant and tool message

* fix result assistant message logic

* nit

* poll conversation messages

* styling and backend nits

* stream tools

* fix tools streaming

* cleaning

* cleaning + handle tool errors

* flag flow errors

* cleaning

* use separate manager

* cleaning

* fix refresh

* cleaning

* cleaning

* Update SQLx metadata

* fix migration

* reuse memory id + add timeout to polling

* cleaning

* no error if stream complete

* fix memory for nested agents

* fix chat enabled for nested agents

* cleaning

* Update SQLx metadata

* nit

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-09 05:07:18 +00:00
Ruben Fiszel
8073e5daeb fix: better handle already completed jobs cases 2025-10-08 10:06:15 +00:00
hugocasa
e1d2eb2870 feat: support for protected private hub (#6762)
* feat: support for protected private hub

* nits

* nits
2025-10-08 09:33:28 +00:00
Alexander Petric
ae8d37fc34 feat: add dynamic skip for schedules (#6739)
* feat: add dynamic skip for schedules

* npm check

* accidental rename

* fixing tests

* Update SQLx metadata

* simplify api / queries

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-08 09:18:15 +00:00
dieriba
27d7809ef9 feat: parallel loop as expr (#6743)
* ok

* revert

* backward compatible

* nits

* nits

* perf

* ok

* nits
2025-10-08 06:57:22 +00:00
Alexander Petric
c9dae9580a fix: correct otel log levels (#6772)
* fix: correct otel log levels

* oss
2025-10-08 06:47:05 +00:00
Diego Imbert
258b275f9b fix: better ducklake setup (#6763)
* stash

* Much better ducklake setup UX

* nits

* mistake

* sqlx prepare
2025-10-07 07:00:00 +00:00
centdix
820437c897 chore(flow): clean flow conversation code (#6755)
* use own folder for memory

* fixes

* better chat interface

* fix export tab

* move in folder

* dont show flow graph if chat mode

* fix

* fix too long title

* fix user message

* fix

* fix

* remove from server

* cleaner

* cleaning

* cleaning

* cleaning
2025-10-06 15:10:29 +00:00
Ruben Fiszel
0bc822dd10 warn for slow queries in push scheduled job 2025-10-03 17:30:40 +00:00
hugocasa
3907c9f951 feat: end user email env var (#6750)
* feat: end user email env var

* nit

* nits
2025-10-03 17:27:58 +00:00
centdix
047420e5ad feat(flow): add option to turn flow into chat (#6658)
* add toggle option + chat interface

* backend impl

* draft

* put info in schema

* Revert "backend impl"

This reverts commit c534eeb49986424e2c12e2c5642be4e17ba380d1.

* chat interface in flow input

* cleaning

* add logic for running flow + styling

* handle historic args

* fix frontend changes

* add tables

* add conv list

* add endpoints

* adapt frontend

* list message logic

* save message in db

* save response in db

* cleaning

* better migrations

* refresh on new conv

* better logic for messages

* nit

* genere conversation uuid from frontend

* store chat mode info in flow status

* better ui for chat

* collapse chat

* ui

* infinite scroll on convs

* infinite scroll on messages

* fix ui

* new chat entry on new

* cleaning

* change setting logic

* fix test logic from flow input

* move toggle to input

* add warning modal when enabling chat mode

* add summary and explanation on inline script

* add hint for chat mode on user_message desc

* show chat message instead of input in graph

* add warning for triggers

* one logo when not expanded

* use infinitelist for conversations

* add warning when deployment in progress

* full width button

* better icon for menu

* better input + nits

* put toggle in action

* use waitjob

* cleaning

* cleaning

* scroll on new + cleaning

* use enum

* fix logic

* full screen

* cleaning

* exit on updatesqlx error

* Update SQLx metadata

* fix

* cleaning

* add for wait result endpoint

* add missing drop

* delete cascade

* fix: use macro version of query_as in flow_conversations.rs

Use sqlx::query_as! macro instead of query_as function for compile-time
SQL validation and better type safety

Co-authored-by: centdix <centdix@users.noreply.github.com>

* fix: update comment to clarify conversation message update condition

The comment now accurately reflects that the update happens when
it's a flow and it's done (flow_is_done)

Co-authored-by: centdix <centdix@users.noreply.github.com>

* fix: only parse chat_input_enabled if conditions are met

Move the parse_chat_input_enabled() call inside the condition check
to avoid unnecessary parsing when the flow is not done or unsuccessful

Co-authored-by: centdix <centdix@users.noreply.github.com>

* fix: use the same transaction for conversation creation

Pass transaction to get_or_create_conversation_with_id instead of
creating a new one, ensuring all operations are atomic

Co-authored-by: centdix <centdix@users.noreply.github.com>

* fix: remove update trigger and handle updated_at in application code

Remove the database trigger that automatically updates conversation
timestamp and instead update it explicitly when creating messages.
This gives better control and consistency.

Co-authored-by: centdix <centdix@users.noreply.github.com>

* Update SQLx metadata

* cleaning

* feat(aiagent): handle memory (#6719)

* implement memory

* s3 logic for memory

* fix typo

* much cleaner

* cleaning

* cleaning

* only if chat

* display nit

* nit

* fix stack overflow

* cleaning

* use len arg from input

* cleaning

* change order

* delete memory when conv deleted

* cleaning

* nit

* show description in expr mode

* opti

* opti

* updatee ref

* store string as simple string

* use markdown

* do not wait for deletion

* add delete loading

* fix logic

* fix markdown

* Update ee-repo-ref.txt

* Update SQLx metadata

* fix in test interface

* nit

* nit

* fix layout

* use memory_id to store memory

* shorter description

* rls + grant

* fix text overflow

* extract output from res

* cleaning

* handle streaming

* cleaning

* fix tool error

* nit

* update ref

* fix

* Update SQLx metadata

* nit

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: centdix <centdix@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-10-03 14:04:21 +00:00
Diego Imbert
f35c9ffe63 S3 Proxy better errors + DuckDB S3 fix (#6740)
* Support for MinIO and other S3 impls in S3 Proxy

* Nice S3 Proxy error messages in DuckDB executor

* nit

* useless code

* super nit

* ee repo ref
2025-10-03 11:06:46 +00:00