Commit Graph

10716 Commits

Author SHA1 Message Date
centdix
1afa36ceeb feat(aiagent): allow mcp as tools (#6790)
* draft mcp client

* testing

* fix

* cleaning

* mcp resource in inputtransforms

* cleaning

* big cleaning

* cleaning

* no arc

* add utils file

* refactor tools

* add mcp actions

* draft frontend

* send arguments from backend

* better frontend

* cleaning

* use token for auth

* add logo

* rm

* fix

* fix

* chore: refactor mcp for ai agents (#6829)

* Add Tool enum for AIAgent with backward compatibility

- Created Tool enum that can be either Windmill (FlowModule) or Mcp (resource reference)
- Created McpToolRef struct to hold MCP resource path
- Implemented custom Deserialize for Tool with backward compatibility:
  - New format: {type: 'windmill'|'mcp', ...}
  - Old format: FlowModule objects (automatically wrapped in Tool::Windmill)
- Updated AIAgent to use Vec<Tool> instead of Vec<FlowModule>
- Updated FlowValue::traverse_leafs to handle Tool enum
- Backward compatible: old flows with Vec<FlowModule> will deserialize correctly

* Refactor AI executor to process Tool enum instead of extracting MCP from input_transforms

- Separate Windmill tools and MCP resource paths from tools list
- Process Windmill FlowModules into Tool definitions
- Load MCP tools from resource paths in Tool::Mcp variants
- Remove old logic that extracted mcp_resources from input_transforms
- Import FlowModule, remove unused InputTransform
- Fix type issues: use .as_str() for path and handle Option<bool> properly

* handle in args

* mcp as flowmodule

* frontend

* config for mcp

* simplify logic

* fix ai executor logic

* cleaning

* clean frontend

* fix

* better resource picker

* fix and styling

* add endpoint to fetch tools

* apply tool filtering

* fix name validation

* better ui

* use cache

* fix

* fix merge

* refactor: Separate MCP tools from FlowModule in AIAgent

- Add new AgentTool, ToolValue, and McpToolValue types
- Update AIAgent to use Vec<AgentTool> instead of Vec<FlowModule>
- Implement From traits for clean conversion between AgentTool and FlowModule
- Add backward compatibility via custom deserializer for AgentTool
- Simplify resolve_module logic by reusing existing resolve_modules function
- Update traverse_leafs to handle AgentTool structure

This refactoring separates MCP tools from FlowModule tools, making the
type system clearer and eliminating the need to treat MCP servers as
a special case of FlowModule.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: Update ai_executor and worker_lockfiles for AgentTool

- Update ai_executor.rs to handle new AgentTool structure
  - Separate MCP tools from FlowModule tools using ToolValue enum
  - Convert AgentTool to FlowModule for backward compatibility
  - Add imports for AgentTool and ToolValue types

- Update worker_lockfiles.rs for lazy loading optimization
  - Convert AgentTool <-> FlowModule in insert_flow_modules
  - Preserve lazy loading for FlowModule tools via modules_node
  - Keep MCP tools inline (lightweight, no need for lazy loading)
  - Maintain backward compatibility with existing flows

This enables the lazy loading optimization for FlowModule tools while
keeping MCP tools inline, balancing performance and simplicity.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* cleaning

* adapt frontend

* cleaning

* cleaning

* type fix

* cleaning

* fix back comp

* move mcp button position

* nit

* cleaning

* fix nested removal

* cleaning

* opti

* fix chat markdown display

* fix chat messages layout

* fix back comp frontend

* fix deserializer

* nit

* simpler serializer

* use if else

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-21 15:28:52 +00:00
Ruben Fiszel
ed3ac2d928 fix: fix column def sync for evalv2 2025-10-21 14:11:55 +00:00
Ruben Fiszel
6c9bc283ab fix: delete workspace_env on workspace deletion 2025-10-21 11:14:36 +00:00
Guilhem
0464a6ebc9 fix suspended count display (#6871) 2025-10-20 14:32:53 +00:00
Ruben Fiszel
6f6b5be833 chore(main): release 1.563.4 (#6865)
* chore(main): release 1.563.4

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-10-20 10:14:04 +00:00
Ruben Fiszel
2f208b5a72 fix(cli): improve generate-flow with raw requirements 2025-10-20 10:03:14 +00:00
Ruben Fiszel
6da9fd9ff1 tailwind classes editor chrome fix 2025-10-19 23:16:26 +00:00
Ruben Fiszel
02a220aece refactor: improve usage table behavior to eliminate synchronous row locks (#6863)
* refactor: improve usage table behavior to eliminate synchronous row locks

Replace synchronous INSERT...RETURNING with SELECT + async UPDATE pattern:
- Add check_usage_limits() to read current usage without row locks
- Add increment_usage_async() to update usage in background task
- Refactor job push logic to use optimistic validation
- Simplify job completion tracking with better error handling

This eliminates blocking row locks on the usage table during job creation,
significantly improving throughput and reducing contention.

Note: Requires running 'cargo sqlx prepare' with database access to update
the query cache in .sqlx/ directory.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

* Update SQLx metadata

* refactor: optimize cloud usage checks with caching and conditional queries

- Add 60s cache for superadmin status checks to reduce DB load
- Skip unnecessary user usage query for premium workspaces
- Use existing team plan status cache (already implemented in windmill-common)
- Update check_usage_limits to accept check_user_usage parameter
- Add sqlx query cache for conditional user usage query

This optimization eliminates redundant database queries during job creation,
particularly for premium workspaces where user usage tracking is not needed.

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-18 07:47:31 +00:00
Ruben Fiszel
0e39366ded fix: set jwt auth for custom apps 2025-10-18 07:10:09 +00:00
Ruben Fiszel
37d6d3c84d chore(main): release 1.563.3 (#6864)
* chore(main): release 1.563.3

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-10-18 06:29:11 +00:00
Ruben Fiszel
84c4b3292a nit opt 2025-10-18 06:28:40 +00:00
Ruben Fiszel
4c9ef3e563 fix(cli): when generating flow locks with new inline content, also generate separate content file 2025-10-18 06:20:05 +00:00
Ruben Fiszel
e00daa9bc8 chore(main): release 1.563.2 (#6862)
* chore(main): release 1.563.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-10-17 18:30:16 +00:00
Ruben Fiszel
cbea1e0652 fix: monitor less frequent cleanup process periodicity 2025-10-17 18:27:11 +00:00
Ruben Fiszel
fd5e294284 chore(main): release 1.563.1 (#6861)
* chore(main): release 1.563.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-10-17 17:21:32 +00:00
Ruben Fiszel
4136a42b48 re-route agent workers job if needed prior to applying concurrency limits 2025-10-17 17:15:05 +00:00
Ruben Fiszel
ceada62b5d fix: fix concurrency limit behavior with remote agents dep jobs 2025-10-17 16:59:29 +00:00
Ruben Fiszel
fe3ee35dde chore(main): release 1.563.0 (#6856)
* chore(main): release 1.563.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-10-17 14:27:49 +00:00
hugocasa
e2f1e6e023 fix(frontend): interaction with code instance settings on chrome (#6859) 2025-10-17 14:22:19 +00:00
Ruben Fiszel
cbb9e4f98c nits cache 2025-10-17 14:21:24 +00:00
hugocasa
4c74211487 feat: http routes streaming (#6834)
* feat: http routes streaming

* improve webhook config section for SSE

* feat: add request type config to http routes

* fix check

* sqlx

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-10-17 14:05:17 +00:00
Ruben Fiszel
1343109420 fix select grant 2025-10-17 13:59:58 +00:00
wendrul
189fb507da Fix ansible delegate mode only working with ssh (#6858) 2025-10-17 13:58:00 +00:00
Ruben Fiszel
08e1807dfe fix: add grant select on debounce_key 2025-10-17 13:55:02 +00:00
Ruben Fiszel
b6a65a4b22 fix: flow dev mode improvements 2025-10-17 13:36:05 +00:00
Pyra
6c533d5070 perf: remove unnecessary db call (#6853)
* perf: remove unnecessary db call

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-17 11:51:55 +00:00
centdix
36cd49fb41 feat(aiagent): Store AI provider config in localStorage (#6854)
* feat(aiagent): Store AI provider config in localStorage

- Added localStorage persistence for AI provider, resource, and model selections
- Configuration is loaded as default values on component initialization
- Automatically saves whenever selections change
- Validates stored provider is still available before loading
- Uses storage key: windmill_ai_provider_config

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* better

* fix logic

* Update toggle option text for default setting

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-10-17 11:48:27 +00:00
Ruben Fiszel
deb4fc6ede remove benchmark from all_features_oss 2025-10-17 08:28:06 +00:00
centdix
ab8a942200 add svelte 5 mcp (#6847) 2025-10-17 07:24:57 +00:00
Ruben Fiszel
b702c91135 fix migration error 2025-10-17 03:42:41 +00:00
Ruben Fiszel
80d4d40fd1 chore(main): release 1.562.0 (#6841)
* chore(main): release 1.562.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-10-16 21:54:50 +00:00
Diego Imbert
8536a4c64e create ducklake_user_pg_pwd if deleted by CLI (#6845) 2025-10-16 21:54:00 +00:00
Ruben Fiszel
691ef6babd remove safety migration 2025-10-16 21:29:44 +00:00
Ruben Fiszel
6e15b0e44d nit 2025-10-16 20:53:32 +00:00
hugocasa
ac1599727e feat: add support for sage intacct oauth (#6794)
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-10-16 20:40:25 +00:00
Diego Imbert
b779cb7e02 ducklake safety for instance_settings.yaml users (#6844) 2025-10-16 20:39:11 +00:00
Ruben Fiszel
c3e284bc96 split main async with a box pin 2025-10-16 20:31:46 +00:00
Ruben Fiszel
51c89573bb fix rare stack overflow bc of async size 2025-10-16 20:05:09 +00:00
Ruben Fiszel
c2b22b3df8 fix rare stack overflow bc of async size 2025-10-16 20:01:30 +00:00
wendrul
4006409296 Ansible repo update hubPath + fixes (#6843)
* Allow ssh auth

* ssh changes pt2

* Fix unused

* Update package.json and hubPaths
2025-10-16 18:24:41 +00:00
Ruben Fiszel
b5fb4a6e54 nit test colors 2025-10-16 18:06:44 +00:00
Pyra
bdf21d340f remove dbg! leftovers (#6842)
Signed-off-by: pyranota <pyra@duck.com>
2025-10-16 17:29:57 +00:00
Ruben Fiszel
1fdcf7a87b fix compile 2025-10-16 17:21:09 +00:00
Pyra
4674191e90 feat: dependency job debouncing (#6769)
* v0

Signed-off-by: pyranota <pyra@duck.com>

* optimize relocks

* make it work with relative relative imports

Signed-off-by: pyranota <pyra@duck.com>

* use fallback

Signed-off-by: pyranota <pyra@duck.com>

* remove dbg and todos

Signed-off-by: pyranota <pyra@duck.com>

* future proof a bit

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* more cleanup

Signed-off-by: pyranota <pyra@duck.com>

* remove final TODO

Signed-off-by: pyranota <pyra@duck.com>

* do not use bytemuck

Signed-off-by: pyranota <pyra@duck.com>

* optimize hashing

Signed-off-by: pyranota <pyra@duck.com>

* implementation 1

Signed-off-by: pyranota <pyra@duck.com>

* almost v0

Signed-off-by: pyranota <pyra@duck.com>

* v0

Signed-off-by: pyranota <pyra@duck.com>

* add comments and use fallback

Signed-off-by: pyranota <pyra@duck.com>

* call dissolve for apps

Signed-off-by: pyranota <pyra@duck.com>

* add comms

Signed-off-by: pyranota <pyra@duck.com>

* refactor v0 (partially tested + dirty)

Signed-off-by: pyranota <pyra@duck.com>

* finishing

Signed-off-by: pyranota <pyra@duck.com>

* remove TODO

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* silence unused argument

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* implement rebuild_map endpoint

Signed-off-by: pyranota <pyra@duck.com>

* update windmill api client

Signed-off-by: pyranota <pyra@duck.com>

* almost finish with tests

Signed-off-by: pyranota <pyra@duck.com>

* add proper testing

Signed-off-by: pyranota <pyra@duck.com>

* remove unused fixtures

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* partial cleanup

Signed-off-by: pyranota <pyra@duck.com>

* Update backend/windmill-worker/src/scoped_dependency_map.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-common/src/scripts.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* evil doings

Signed-off-by: pyranota <pyra@duck.com>

* more cleanup

* Update SQLx metadata

* more cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fixing CI

Signed-off-by: pyranota <pyra@duck.com>

* remove python from default features

Signed-off-by: pyranota <pyra@duck.com>

* feat: dependency job debouncing

* checkpoint

Signed-off-by: pyranota <pyra@duck.com>

* more improvements

Signed-off-by: pyranota <pyra@duck.com>

* refactor: clean up dependency job debouncing implementation

- Add comprehensive comments explaining the debouncing mechanism
- Replace debug statements (dbg!) with proper tracing calls
- Extract helper functions to reduce code duplication:
  - extract_to_relock_from_args() for extracting nodes/components
  - accumulate_debounce_stale_data() for updating stale data
- Improve code readability and maintainability

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* Update backend/windmill-common/src/jobs.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-common/src/jobs.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-queue/src/jobs.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* better error handling for helper

Signed-off-by: pyranota <pyra@duck.com>

* update ee-ref

Signed-off-by: pyranota <pyra@duck.com>

* test: add race condition test for dependency job debouncing

- Implement test_2 for scripts to test the race condition edge case
- Add comprehensive documentation comments to all test functions
- Remove empty test_2 stubs for flows (not needed)
- Keep test_2 stub for apps with TODO comment

The race condition test simulates the scenario where a job is marked
as running but debounce_key hasn't been cleaned up yet, forcing the
system to create a new job while reusing the existing debounce_key.

This edge case can occur due to the lack of transactions in the pull
function for performance reasons (see jobs.rs:4415-4425).

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* test: implement race condition test for dependency job debouncing

- Add comprehensive test_2 for script module that tests the race condition edge case
- Remove empty test_2 stubs from flows and apps modules
- Fix unused variable warning in worker_lockfiles.rs
- Add detailed comments explaining the race condition scenario and test logic

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* implement fallback

Signed-off-by: pyranota <pyra@duck.com>

* make it mostly work

* all tests are almost working

Signed-off-by: pyranota <pyra@duck.com>

* add comments a bit

Signed-off-by: pyranota <pyra@duck.com>

* Update backend/windmill-common/src/scripts.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-worker/src/worker_lockfiles.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-worker/src/worker_lockfiles.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* feat: improve debouncing documentation, tracing, and error handling

- Add comprehensive 3-phase debouncing documentation explaining push/accumulation/pull
- Enhance all tracing with structured logging (job_id, workspace_id, node_count, etc.)
- Add proper error handling with .map_err() and contextual messages
- Replace dbg!() with proper tracing::debug!()
- Replace todo!() with proper error handling
- Fix typos: 'and edge case' → 'an edge case', 'bc' → 'because'
- Fix debug variable name: 'debounce_job_id_0' → 'debounce_job_id_o'
- Add documentation for debounce cleanup and stale data retrieval
- Add trace-level logging for non-error paths to reduce noise

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* do some work for future improvements

Signed-off-by: pyranota <pyra@duck.com>

* fix tests

Signed-off-by: pyranota <pyra@duck.com>

* clippy

Signed-off-by: pyranota <pyra@duck.com>

* update sqlx

Signed-off-by: pyranota <pyra@duck.com>

* clippy

Signed-off-by: pyranota <pyra@duck.com>

* update ee ref

Signed-off-by: pyranota <pyra@duck.com>

* flag tests behind the feature, add timeout

Signed-off-by: pyranota <pyra@duck.com>

* fix timeout + cleanup

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* row lock debounce_key

Signed-off-by: pyranota <pyra@duck.com>

* addressing TODOs

Signed-off-by: pyranota <pyra@duck.com>

* fix test

Signed-off-by: pyranota <pyra@duck.com>

* ee ref

Signed-off-by: pyranota <pyra@duck.com>

* ee repo

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: Pyra <92104930+pyranye@users.noreply.github.com>
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Pyra <pyranota@users.noreply.github.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-10-16 16:49:35 +00:00
Ruben Fiszel
a427feb543 fix compile 2025-10-16 16:28:04 +00:00
Ruben Fiszel
6c52db97ca nit compile 2025-10-16 16:24:36 +00:00
Ruben Fiszel
76643ae900 fix: add configurable timeout sse stream 2025-10-16 15:29:49 +00:00
Ruben Fiszel
34e118c6ad chore(main): release 1.561.0 (#6838)
* chore(main): release 1.561.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-10-16 15:07:59 +00:00
Ruben Fiszel
7d3214ac57 Is Password nit 2025-10-16 14:41:12 +00:00
Ruben Fiszel
e87d0ab94d fix: fix job loader in public apps with jwt token 2025-10-16 14:17:45 +00:00