Commit Graph

5945 Commits

Author SHA1 Message Date
Ruben Fiszel
86d04e4f65 chore(main): release 1.566.0 (#6904)
* chore(main): release 1.566.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-10-24 19:44:57 +00:00
hugocasa
f0bbdd4aa8 feat(backend): pass all headers and query to webhook preprocessor (#6931) 2025-10-24 19:40:44 +00:00
Ruben Fiszel
04890a7a42 tmp overflow fix 2025-10-24 19:38:35 +00:00
Ruben Fiszel
4cee047ebc fix agent worker nits 2025-10-24 19:32:47 +00:00
Pyra
999aaaacd8 fix(debouncing): fix perf issues and re-enable debouncing (#6932)
* fix(debouncing): fix perf issues and re-enable debouncing

Signed-off-by: pyranota <pyra@duck.com>

* add debounce data in clone_script

Signed-off-by: pyranota <pyra@duck.com>

* update sqlx cache

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
2025-10-24 15:24:46 +00:00
Ruben Fiszel
583b870983 temporarily disable debouncing 2025-10-24 10:50:51 +00:00
Ruben Fiszel
7348b5df99 temporarily disable debouncing 2025-10-24 10:49:38 +00:00
hugocasa
f43dee1952 feat: support wildcards in http routes (#6927) 2025-10-24 09:37:03 +00:00
Ruben Fiszel
5773f66a3a fork fix 2025-10-23 22:14:40 +00:00
Ruben Fiszel
25cf7fb0b1 fix build 2025-10-23 21:52:59 +00:00
Alexander Petric
1d3245eea2 fix: also auto add/del igroup members to workspaces where configured (#6888)
* fix: also auto add/del igroup members to workspaces where configured

* Update SQLx metadata

* feature flags

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-23 21:34:21 +00:00
hugocasa
7b46491578 fix(cloud): better errors when failing to get team plan status (#6908)
* fix(cloud): better errors when failing to get team plan status

* better errors

* fix build

* fix build
2025-10-23 20:19:10 +00:00
Pyra
edece035f8 feat: job debouncing (#6878)
* feat(EE): job debouncing

Signed-off-by: pyranota <pyra@duck.com>

* remove 'script' file

Signed-off-by: pyranota <pyra@duck.com>

* more work

Signed-off-by: pyranota <pyra@duck.com>

* properly gate it behind enterprise

Signed-off-by: pyranota <pyra@duck.com>

* update ee repo ref

Signed-off-by: pyranota <pyra@duck.com>

* change ee repo ref again

Signed-off-by: pyranota <pyra@duck.com>

* remove unused variable

Signed-off-by: pyranota <pyra@duck.com>

* feat(EE): implement TODOs and enhance tracing for job debouncing

- Add database index on script(workspace_id, debounce_key) for efficient lookups
- Update minimum version requirement to 1.564.0 throughout codebase
- Add tracing warnings when debouncing is disabled due to worker version mismatch
- Fix all documentation links from TODO placeholders to proper URLs
- Replace Gauge icon with Timer icon for debouncing UI elements
- Update placeholder text and tooltips with clear descriptions

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* create -> crate

Signed-off-by: pyranota <pyra@duck.com>

* remove index

Signed-off-by: pyranota <pyra@duck.com>

* some updates

Signed-off-by: pyranota <pyra@duck.com>

* fix once more

Signed-off-by: pyranota <pyra@duck.com>

* fix it once more

Signed-off-by: pyranota <pyra@duck.com>

* Remove flow step debouncing, keep top-level flow debouncing

- Remove debounce fields from RawScript and FlowScript FlowModuleValue variants
- Remove debounce fields from JobPayload::FlowScript and RawCode
- Update raw_script_to_payload function signature
- Remove debouncing UI from flow step runtime settings
- Remove debouncing toggle handler and indicator badge
- Preserve top-level flow debouncing in FlowSettings

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fixup claude's work

Signed-off-by: pyranota <pyra@duck.com>

* cleanup: remove dbg! statements, update min version to 1.566.0, add comprehensive comments

- Removed all dbg! macro calls from production code
- Updated MIN_VERSION_SUPPORTS_DEBOUNCING from 1.564.0 to 1.566.0
- Added comprehensive documentation comments explaining:
  - Debouncing feature purpose and mechanics
  - Database schema for debounce_key and debounce_stale_data tables
  - Version check logic and guard functions
- Improved code clarity and maintainability

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* improve fallback

Signed-off-by: pyranota <pyra@duck.com>

* remove comments from old migration

Signed-off-by: pyranota <pyra@duck.com>

* fix pull

Signed-off-by: pyranota <pyra@duck.com>

* fix once more

Signed-off-by: pyranota <pyra@duck.com>

* Update frontend/src/lib/components/ScriptBuilder.svelte

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* add sqlx cache

Signed-off-by: pyranota <pyra@duck.com>

* fix: frontend CI - fix debounce field names and remove leftover flow step debouncing

- Fix ScriptBuilder.svelte: change custom_debounce_key to debounce_key
- Add debounce_key and debounce_delay_s fields to NewScript schema in openapi.yaml
- Regenerate frontend types from OpenAPI spec
- Remove leftover flow step debouncing code from FlowModuleComponent.svelte
- Remove debounce fields from RawScript in openflow.openapi.yaml
- Remove unused Timer import from FlowModuleHeader.svelte

All frontend checks now passing (0 errors, 0 warnings)

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* fix ci

Signed-off-by: pyranota <pyra@duck.com>

* remove unused import

Signed-off-by: pyranota <pyra@duck.com>

* fix ci again

Signed-off-by: pyranota <pyra@duck.com>

* udpate ee repo ref

Signed-off-by: pyranota <pyra@duck.com>

* CI doesn't want to be fixed but I still try

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* ci...

Signed-off-by: pyranota <pyra@duck.com>

* Update ee-repo-ref.txt

* safer migration

Signed-off-by: pyranota <pyra@duck.com>

* reduce noise in logs

Signed-off-by: pyranota <pyra@duck.com>

* fix cli for scripts

Signed-off-by: pyranota <pyra@duck.com>

* nit

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Pyra <pyranota@users.noreply.github.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-23 20:18:08 +00:00
Alexander Petric
b33e3d8505 fix: apify oauth (#6902)
* apify init

* apify svg

* Rename OAuth provider from apify_oauth to apify

Co-authored-by: Alexander Petric <alpetric@users.noreply.github.com>

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
2025-10-23 20:15:34 +00:00
centdix
53d8fbe508 feat(flow chat): display image outputs (#6880)
* support displaying images output in flow chat

* opti

* simplify

* better code

* fix
2025-10-23 17:19:57 +00:00
Alexander Petric
fc7b65b193 nit: also log force ipv4 in configure_client (#6924) 2025-10-23 14:58:45 +00:00
Ruben Fiszel
b5ce75e3d2 docs: add critical warning against SELECT * in worker queries (#6916)
Never use SELECT * in queries that workers might execute to ensure
backwards compatibility when workers are running behind API server
version. New database columns would break outdated workers.

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-23 10:54:25 +00:00
hugocasa
e81d629705 feat(backend): stream early return (#6896)
* feat(backend): stream early return

* revert early return change

* nit

* sqlx

* fix build
2025-10-23 08:36:21 +00:00
hugocasa
4b26def0cd chore(parsers): publish bash parser with CRLF handling (#6905) 2025-10-23 08:33:41 +00:00
Vivek Chavan
fa76105d14 fix: detect preprocessor in re-exported named exports (#6899)
Fixes #6894

The TypeScript parser now correctly detects preprocessor functions
that are re-exported from other modules using named exports like:
  export { preprocessor } from "./other_module";

Previously, only function declarations were detected. Now the parser
also checks ExportNamed AST nodes for any specifier named 'preprocessor'.

This allows developers to easily reuse preprocessor functions across
multiple scripts without the workaround of wrapping them in a new function.

Added comprehensive tests covering:
- Simple re-export: export { preprocessor } from "./other"
- Re-export with renaming: export { preprocessor as preprocessor }
- Mixed exports: export { foo, preprocessor, bar }
- Negative case: exports without preprocessor
2025-10-23 07:51:57 +00:00
Ruben Fiszel
dc887e5f43 remove erronous error log in agent mode 2025-10-23 07:41:31 +00:00
Ruben Fiszel
623392de9e chore(main): release 1.565.0 (#6890)
* chore(main): release 1.565.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-10-22 18:09:57 +00:00
centdix
2170d8dd32 feat(ai agent): handle inputTransforms for tools arguments (#6873)
* show inputs

* filter filled args

* merge args with input transforms

* working expr

* handle results expr

* cleaning

* cleaning

* cleaning

* cleaning

* Update SQLx metadata

* fix no previous step

* add ai option in frontend

* cleaning

* cleaning

* Update SQLx metadata

* fix

* fix reactive unmount issue with tool id

* use existing func

* only fetch if needed

* fix

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-22 18:04:23 +00:00
Alexander Petric
9b956902ae fix windows typo (#6900) 2025-10-22 17:48:38 +00:00
Ruben Fiszel
9b1a7f5a3a fix(windows): improve 2nd shutdown monitor 2025-10-22 08:42:36 +00:00
Pyra
b9eaff3688 debouncing edge case (#6891)
* fix: debouncing edge case

* Update SQLx metadata

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-22 07:15:28 +00:00
hugocasa
cadf32cf03 fix(backend): batch rerun jobs with preprocessor (#6875) 2025-10-22 06:27:32 +00:00
Vivek Chavan
5529736784 fix(parser): Handle CRLF line endings in bash and PowerShell parsers (#6889)
Fixes #6867

The bash and PowerShell parsers were not properly handling CRLF (Windows-style)
line endings when parsing script arguments. The regex patterns were only
matching LF line endings, causing scripts with CRLF to fail metadata generation.

Changes:
- Updated RE_BASH regex to optionally match \r before line end (\r?$)
- Updated RE_POWERSHELL_ARGS regex to optionally match \r (\r?)
- Added test case test_parse_bash_sig_with_crlf() to verify CRLF handling

This fix ensures that bash scripts created on Windows (or with editors
using CRLF) will correctly parse arguments and generate proper metadata
via 'wmill script generate-metadata' command.

Python parser was verified to work correctly as it uses an AST parser
that inherently handles line endings properly.
2025-10-22 06:21:38 +00:00
Ruben Fiszel
b893af16e7 chore(main): release 1.564.0 (#6877)
* chore(main): release 1.564.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-10-21 21:18:03 +02:00
Pyra
caa21fcb33 fix: debounce_key automatic deletion (#6885)
* fix: debounce_key automatic deletion

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* better solution

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-21 18:31:05 +00:00
Alexander Petric
d777c7798b feat: allow optionally forcing ipv4 for reqwest (#6883) 2025-10-21 15:46:41 +00:00
centdix
97ac1be036 feat(aiagent): allow mcp as tools (#6790)
* draft mcp client

* testing

* fix

* cleaning

* mcp resource in inputtransforms

* cleaning

* big cleaning

* cleaning

* no arc

* add utils file

* refactor tools

* add mcp actions

* draft frontend

* send arguments from backend

* better frontend

* cleaning

* use token for auth

* add logo

* rm

* fix

* fix

* chore: refactor mcp for ai agents (#6829)

* Add Tool enum for AIAgent with backward compatibility

- Created Tool enum that can be either Windmill (FlowModule) or Mcp (resource reference)
- Created McpToolRef struct to hold MCP resource path
- Implemented custom Deserialize for Tool with backward compatibility:
  - New format: {type: 'windmill'|'mcp', ...}
  - Old format: FlowModule objects (automatically wrapped in Tool::Windmill)
- Updated AIAgent to use Vec<Tool> instead of Vec<FlowModule>
- Updated FlowValue::traverse_leafs to handle Tool enum
- Backward compatible: old flows with Vec<FlowModule> will deserialize correctly

* Refactor AI executor to process Tool enum instead of extracting MCP from input_transforms

- Separate Windmill tools and MCP resource paths from tools list
- Process Windmill FlowModules into Tool definitions
- Load MCP tools from resource paths in Tool::Mcp variants
- Remove old logic that extracted mcp_resources from input_transforms
- Import FlowModule, remove unused InputTransform
- Fix type issues: use .as_str() for path and handle Option<bool> properly

* handle in args

* mcp as flowmodule

* frontend

* config for mcp

* simplify logic

* fix ai executor logic

* cleaning

* clean frontend

* fix

* better resource picker

* fix and styling

* add endpoint to fetch tools

* apply tool filtering

* fix name validation

* better ui

* use cache

* fix

* fix merge

* refactor: Separate MCP tools from FlowModule in AIAgent

- Add new AgentTool, ToolValue, and McpToolValue types
- Update AIAgent to use Vec<AgentTool> instead of Vec<FlowModule>
- Implement From traits for clean conversion between AgentTool and FlowModule
- Add backward compatibility via custom deserializer for AgentTool
- Simplify resolve_module logic by reusing existing resolve_modules function
- Update traverse_leafs to handle AgentTool structure

This refactoring separates MCP tools from FlowModule tools, making the
type system clearer and eliminating the need to treat MCP servers as
a special case of FlowModule.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: Update ai_executor and worker_lockfiles for AgentTool

- Update ai_executor.rs to handle new AgentTool structure
  - Separate MCP tools from FlowModule tools using ToolValue enum
  - Convert AgentTool to FlowModule for backward compatibility
  - Add imports for AgentTool and ToolValue types

- Update worker_lockfiles.rs for lazy loading optimization
  - Convert AgentTool <-> FlowModule in insert_flow_modules
  - Preserve lazy loading for FlowModule tools via modules_node
  - Keep MCP tools inline (lightweight, no need for lazy loading)
  - Maintain backward compatibility with existing flows

This enables the lazy loading optimization for FlowModule tools while
keeping MCP tools inline, balancing performance and simplicity.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* cleaning

* adapt frontend

* cleaning

* cleaning

* type fix

* cleaning

* fix back comp

* move mcp button position

* nit

* cleaning

* fix nested removal

* cleaning

* opti

* fix chat markdown display

* fix chat messages layout

* fix back comp frontend

* fix deserializer

* nit

* simpler serializer

* use if else

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-21 15:28:52 +00:00
Ruben Fiszel
7d1ee0662e fix: delete workspace_env on workspace deletion 2025-10-21 11:14:36 +00:00
Ruben Fiszel
e10ac1a93e chore(main): release 1.563.4 (#6865)
* chore(main): release 1.563.4

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-10-20 10:14:04 +00:00
Ruben Fiszel
6ccccbcf9e fix(cli): improve generate-flow with raw requirements 2025-10-20 10:03:14 +00:00
Ruben Fiszel
dfd08d8a4b refactor: improve usage table behavior to eliminate synchronous row locks (#6863)
* refactor: improve usage table behavior to eliminate synchronous row locks

Replace synchronous INSERT...RETURNING with SELECT + async UPDATE pattern:
- Add check_usage_limits() to read current usage without row locks
- Add increment_usage_async() to update usage in background task
- Refactor job push logic to use optimistic validation
- Simplify job completion tracking with better error handling

This eliminates blocking row locks on the usage table during job creation,
significantly improving throughput and reducing contention.

Note: Requires running 'cargo sqlx prepare' with database access to update
the query cache in .sqlx/ directory.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

* Update SQLx metadata

* refactor: optimize cloud usage checks with caching and conditional queries

- Add 60s cache for superadmin status checks to reduce DB load
- Skip unnecessary user usage query for premium workspaces
- Use existing team plan status cache (already implemented in windmill-common)
- Update check_usage_limits to accept check_user_usage parameter
- Add sqlx query cache for conditional user usage query

This optimization eliminates redundant database queries during job creation,
particularly for premium workspaces where user usage tracking is not needed.

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-18 07:47:31 +00:00
Ruben Fiszel
4cc2166c10 chore(main): release 1.563.3 (#6864)
* chore(main): release 1.563.3

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-10-18 06:29:11 +00:00
Ruben Fiszel
200242aa3b chore(main): release 1.563.2 (#6862)
* chore(main): release 1.563.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-10-17 18:30:16 +00:00
Ruben Fiszel
f581ce6248 fix: monitor less frequent cleanup process periodicity 2025-10-17 18:27:11 +00:00
Ruben Fiszel
4fdbd9184e chore(main): release 1.563.1 (#6861)
* chore(main): release 1.563.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-10-17 17:21:32 +00:00
Ruben Fiszel
c8d2a036b6 re-route agent workers job if needed prior to applying concurrency limits 2025-10-17 17:15:05 +00:00
Ruben Fiszel
6faea9adad fix: fix concurrency limit behavior with remote agents dep jobs 2025-10-17 16:59:29 +00:00
Ruben Fiszel
c3299e8b8b chore(main): release 1.563.0 (#6856)
* chore(main): release 1.563.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-10-17 14:27:49 +00:00
hugocasa
8cd0006498 feat: http routes streaming (#6834)
* feat: http routes streaming

* improve webhook config section for SSE

* feat: add request type config to http routes

* fix check

* sqlx

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-10-17 14:05:17 +00:00
Ruben Fiszel
d2ae0533f4 fix select grant 2025-10-17 13:59:58 +00:00
wendrul
e719be54e0 Fix ansible delegate mode only working with ssh (#6858) 2025-10-17 13:58:00 +00:00
Ruben Fiszel
60e17e5068 fix: add grant select on debounce_key 2025-10-17 13:55:02 +00:00
Pyra
2868eeda26 perf: remove unnecessary db call (#6853)
* perf: remove unnecessary db call

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-17 11:51:55 +00:00
Ruben Fiszel
5c02af11c3 remove benchmark from all_features_oss 2025-10-17 08:28:06 +00:00
Ruben Fiszel
34cd68676b fix migration error 2025-10-17 03:42:41 +00:00