Commit Graph

229 Commits

Author SHA1 Message Date
hugocasa
6f24f1939d feat: google native triggers (#7837)
* feat: google native triggers

* nit skill

* better native trigger abstraction

* use resources for workspace integrations

* better and better

* better tests

* update native trigger skill

* sqlx

* less tx and google update fix

* refactor a bit the external logic

* nits

* fix

* fix google native trigger update

* fix oauth

* review fixes

* sqlx fix

* nit

* chore: update ee-repo-ref to a10eda4251610cceee67fbe05463b8be82ffa9e0

This commit updates the EE repository reference after PR #416 was merged in windmill-ee-private.

Previous ee-repo-ref: bf3696d5f2a39a3cb84dbbee81e092155f2a8c75

New ee-repo-ref: a10eda4251610cceee67fbe05463b8be82ffa9e0

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-02-16 15:52:02 +00:00
Diego Imbert
3476ef4b9c fix: fix DB Manager not working with db resources with 4+ path segments (#7809)
* support more than 3 path segments

* Fix explore db resource not working with 4+ path segments

* don't assume 3 segments

* ?table= syntax impl

* update parsers

* more nit fixes

* fix sql query

* claude nit

* Update SQLx metadata

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-02-05 14:00:15 +00:00
Diego Imbert
a3d75ba10a feat: favorite datatable and ducklake tables + asset page nits (#7795)
* update cf worker hostname

* set remote_url cookie from param

* ephemeral backends v1

* nit

* Run queue server

* ntis

* timeout

* better db process management

* commit hash and worktree

* nit use map

* nit

* err handling

* Revert "err handling"

This reverts commit 19de00c0c0.

* nits

* auto cleanup

* Ephemeral backend command action

* remove checkout

* checkout ee repo

* nits

* process.env.GIT_EE_DEPLOY_KEY_FILE

* resumeURLs logic

* nit

* use windmill flow for ephemeral backend action

* fixes

* new token

* worktree pools

* Delete GH secret on cleanup

* linux deploy

* nit

* nit

* unhandled promises

* nit

* fix docker bridge IP on linux

* pass cf_frontend_url to wmill flow

* git fetch

* release worktree when binary started

* send error

* logger

* logging

* logging 2

* delete log files periodically

* redirect to raw app with logs

* CORS

* MANAGER_AUTH_TOKEN

* Check organization membership

* nit

* bwrap

* nit

* return timeoutAt in resumeUrl

* nit

* Change password

* nit remove https

* Settings icon instead of plain text

* doc link

* favorite icon

* Better favorites state logic

* Asset favorites

* nit fixes and better label display

* global db manager drawer

* Favorites menu open db manager

* favorites in assets page + backend refactor

* Delete favorite dropdown

* parseFavoriteAsset

* git fail

* nit

* git fail

* nit

* nits

* git fail

* Table2 instead of pyramid
2026-02-04 18:48:51 +00:00
Diego Imbert
0caa533fbd feat: column-level asset tracking for ducklake and datatables (#7774)
* Refactor 1

* claude tmp1

* fixes1

* support for insert and update

* Fix returning

* docs: add nuanced mutex selection guidance to Rust backend skill (#7737)

Add "Mutex Selection in Async Code" section explaining when to use
std::sync::Mutex vs tokio::sync::Mutex based on official Tokio docs.
std::sync::Mutex is preferred for data protection as it's faster;
tokio::sync::Mutex only needed when holding locks across .await points.

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): revert findCodebase change that broke ../shared codebases (#7740)

* fix(cli): revert findCodebase relative_path check that broke ../shared codebases

The previous change added a check to ensure script paths start with the
codebase's relative_path. However, this broke cases where relative_path
uses parent directory references (e.g., "../shared") because:

1. path.join normalizes paths, so "/project/../shared/f/script.ts" becomes
   "/shared/f/script.ts"
2. FSFSElement strips the cwd prefix, resulting in "f/script.ts"
3. The check "f/script.ts".startsWith("../shared/") failed

The original behavior was correct - relative_path indicates where to find
codebase files, while includes/excludes patterns match against the normalized
paths that get passed during sync.

Fixes regression reported in #7729 comments.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test(cli): add preview test for codebase with imports

Tests that codebase bundling correctly includes imported modules,
which is the key functionality needed for ../shared codebases.
The test creates a helper module and a main script that imports
from it, then verifies the bundled script executes correctly.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* chore(main): release 1.621.2 (#7735)

* chore(main): release 1.621.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* feat: workspace dedicated workers (#7741)

* feat: workspace dedicated workers

* ref

* chore: update ee-repo-ref to a18ac31062ac092cb9a5fc87629e217d97f4911d

This commit updates the EE repository reference after PR #398 was merged in windmill-ee-private.

Previous ee-repo-ref: 98cfe3fef764d9d815d326d5056c734a03689d33

New ee-repo-ref: a18ac31062ac092cb9a5fc87629e217d97f4911d

Automated by sync-ee-ref workflow.

* fix(frontend): workspace script in flow steps

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* fix: remove uuid-ossp extension requirement for RDS compatibility

The uuid-ossp extension was created in the first migration but never
actually used - the codebase uses gen_random_uuid() which is built-in
to PostgreSQL 13+. This allows Windmill to run on AWS RDS where
application users may not have CREATE SCHEMA privileges.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: indexer build error (#7744)

* fix: indexer build error

* prepare sqlx

* Remove changes from Cargo.toml

* refactor: remove seed parameter from AI chat completions (#7745)

* better claude

* refactor: remove seed parameter from AI chat completions

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* fix: visibility bug on deployment UI (issue when renaming items) + add tracking of folders and resource types (#7739)

* fix: Raw apps deployment UI (and merge UI)

* Add folders and resource tpyes to merge UI

* claude first pass on adding the new arg for h_deploy_metadata

* Add missing argument to handle_deployment_metadata in all its calls

* Add support for folders and resource types in merge UI

* Update eereporef for CI

* Update ee repo

* Add migration to reset cached diff with potential artifacts

* fix type in frontend

* Preapare sqlx

* Remove unused import and logs

* update ee-repo

* Update eerepo

* chore: update ee-repo-ref to aca38475afd2cafaf63f4bbffc65be9437d57d86

This commit updates the EE repository reference after PR #397 was merged in windmill-ee-private.

Previous ee-repo-ref: 19c64cf8c61d83f45047b37660054b29658cd403

New ee-repo-ref: aca38475afd2cafaf63f4bbffc65be9437d57d86

Automated by sync-ee-ref workflow.

* Make integration  test for workspace comparisons

* Update SQLx metadata

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* fix: require AGENT_TOKEN and BASE_INTERNAL_URL for agent mode

- Add AgentConfig struct to validate required env vars on startup
- Change build_agent_http_client to require explicit token and URL
- Remove DEFAULT_BASE_INTERNAL_URL fallback (no more silent localhost:8000)
- Exit immediately if agent cannot connect to server on initial load
- Update integration tests to use dynamic port for BASE_INTERNAL_URL

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: forward teams error to client (#7746)

* fix: forward teams error to client

* chore: update ee-repo-ref to 9a3d71f2c6a41ed4d17111a8c05d8e1d4933898d

This commit updates the EE repository reference after PR #400 was merged in windmill-ee-private.

Previous ee-repo-ref: 25d35a8de1cd70e281dc876e51cd30402580b5c0

New ee-repo-ref: 9a3d71f2c6a41ed4d17111a8c05d8e1d4933898d

Automated by sync-ee-ref workflow.

* fix

* fix

* fix

* al

* sqlx

* sqlx

* all

* all

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>

* feat: add token usage tracking to AI agent output (#7738)

* feat: add token usage tracking to AI agent output

Add TokenUsage struct to track input/output/cache tokens from AI providers.
Currently implemented for Bedrock provider, with infrastructure in place
for other providers. Usage is included in the AI agent result alongside
output and messages when available.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add token usage extraction for Anthropic provider

Extract usage from message_delta SSE event and convert to TokenUsage.
Includes input_tokens, output_tokens, cache_read_input_tokens, and
cache_creation_input_tokens (mapped to cache_write_input_tokens).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add token usage extraction for Google AI/Gemini provider

Extract usage from usageMetadata in Gemini SSE events and convert to TokenUsage.
Maps promptTokenCount -> input_tokens, candidatesTokenCount -> output_tokens,
totalTokenCount -> total_tokens.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add token usage extraction for OpenAI Responses API provider

Extract usage from response.completed SSE event and convert to TokenUsage.
Maps input_tokens, output_tokens, and total_tokens directly.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add token usage extraction for Azure OpenAI / Chat Completions API

Add stream_options.include_usage to request and parse usage from final
SSE chunk for providers using the standard OpenAI Chat Completions API
(Azure OpenAI, Mistral, DeepSeek, Groq, TogetherAI, CustomAI).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: cleanup token usage tracking - remove unused Image usage field and accumulate across iterations

- Remove unused `usage` field from ParsedResponse::Image variant
- Add TokenUsage::accumulate() method to sum usage across agent iterations
- Accumulate input/output/total/cache tokens instead of replacing with last iteration

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: remove verbose debug logging from AI providers

Remove tracing::info!("[debug] ...") statements that were too verbose
for production. These logged raw events on every streaming event.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: add retry mechanism for OpenAI-compatible providers without stream_options support

Some OpenAI-compatible providers don't support the stream_options parameter
for usage tracking. This adds a retry mechanism that:
- First attempts the request with stream_options.include_usage
- If it fails with 400 and error mentions stream_options/include_usage,
  automatically retries without the parameter

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: remove unnecessary text parsing overhead in image response handlers

Revert debugging changes that read response as text before parsing JSON.
Using response.json() directly is more efficient.

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: centralize TokenUsage conversion with constructor methods

Add new(), from_input_output(), and with_cache() constructors to TokenUsage
to eliminate duplicate conversion logic across providers. Also fixes potential
truncation in Bedrock cache token conversion by using i32::try_from with
fallback to i32::MAX.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: simplify Anthropic usage extraction and add Default derive

- Use idiomatic `if let` pattern instead of `is_some()` check for usage extraction
- Add Default derive to OpenAIChatUsage for consistency with other usage structs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use saturating_add to prevent overflow in token accumulation

In long-running agents with many iterations, token counts could
potentially overflow. Using saturating_add ensures values cap at
i32::MAX instead of wrapping around.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* better claude

* nit

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* chore(main): release 1.622.0 (#7742)

* chore(main): release 1.622.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* fix npm check

* fix: add schema compatibility layer for MCP clients like n8n (#7747)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* nit ui nextcloud triggers (#7749)

* feat: add PocketID OAuth provider support (#7318)

* feat(oauth): add Pocket-ID OAuth provider component

- Implements PocketIdSetting.svelte following Keycloak pattern
- Configures OIDC endpoints for Pocket-ID (/authorize, /api/oidc/*)
- Supports standard OIDC scopes (openid, profile, email)
- Uses passkey-only authentication via Pocket-ID

Refs #5678

* feat(oauth): register Pocket-ID in SSO provider list

- Import PocketIdSetting component
- Add Pocket-ID to provider list in SSO tab
- Update exclusion filter to prevent duplicate custom entries

Refs #5678

* fix(oauth): add missing PocketID icon and fix component integration

- Create PocketIdIcon.svelte component with user profile icon
- Register pocket-id in APP_TO_ICON_COMPONENT mapping
- Fix PocketIdSetting to use IconedResourceType pattern matching other OAuth providers

This resolves the issue where PocketID toggle was not appearing in SSO settings.

Refs #5678

* refactor: migrate PocketIdSetting to Svelte 5 runes syntax

- Use $props() with $bindable() for reactive prop binding
- Use $state() for local reactive state
- Use $derived() for computed values
- Use $effect() for reactive side effects
- Replace on:change with onchange event handler
- Pre-populate base URL from existing config when editing
- Clean up bracket notation to dot notation for value properties

Addresses reviewer feedback

* fix: rename pocket-id to pocketid for naming convention compliance

Change identifier from 'pocket-id' to 'pocketid' to match Windmill's naming convention.
No OAuth provider uses hyphens - all custom SSO providers (keycloak, authentik, authelia,
kanidm, zitadel) use no separator.

Changes:
- AuthSettings.svelte: oauths['pocket-id'] → oauths['pocketid'] (2 locations)
- PocketIdSetting.svelte: name={'pocket-id'} → name={'pocketid'}
- icons/index.ts: 'pocket-id': PocketIdIcon → pocketid: PocketIdIcon

Note: PocketID does not need oauth_connect.json entry as it's a custom SSO provider
with user-configured endpoints, similar to Keycloak/Authentik.

Addresses reviewer feedback

* fix: use TextInput component for consistency

---------

Co-authored-by: hugocasa <hugo@casademont.ch>

* fix: preserve script envs field during sync push

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit frontend fix

* chore(main): release 1.623.0 (#7748)

* chore(main): release 1.623.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* fix: resolve infinite effect loop in PocketIdSetting component (#7753)

* fix: prevent retention cleanup from deleting jobs of active flows (#7755)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* chore(main): release 1.623.1 (#7754)

* chore(main): release 1.623.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* feat: default to quickjs on ce for flow eval (#7756)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* feat: runtime assets (#7656)

* Runtime assets

* Nits

* Revert "Nits"

This reverts commit 3031a2ddd1.

* detection_kinds

* don't delete runtime assets

* Show latest executions

* conditional unique idx

* nit status

* refactor

* nit refactor

* prepare sql

* Detect assets in complex JSON input objects

* false positive prevent

* nit

* redundant idx

* Update frontend/src/lib/components/assets/AssetsUsageDrawer.svelte

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* Update backend/migrations/20260122134517_runtime_assets.up.sql

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* runtime assets are inserted in a loop

* nit

* nit fix

* Don't use lazy static

* fix compilation

* nits

* missing on conflict do nothing

* add index

* Fix max n logic

* created at

* nits

* remove pagination

* sqlx prepare

* Only detect resource assets in input

* get_runtime_asset_sender()

* use global get_runtime_asset_sender to avoid prop drilling

* nit refactor : register_runtime_asset

* get job_id from token

* job as a usage kind

* fixes

* ee

* nit refactor

* merge access types when same job uses same asset multiple times

* Refactor to support wmill s3 API

* nit

* parse_wmill_sdk_sql_assets refactor

* Detect datatable and ducklake usage

* nit order by

* Join with v2_job

* better UI

* add sequential id for cursor pagination

* useInfiniteQuery

* useScrollToBottom

* sql index

* claude code stash

* migration fixes

* Infinite scroll UI

* nit

* style nit

* runtime asset created at

* Asset filters

* fix usage kind filter

* also check runnable_path for jobs when filtering

* better filters

* avoid flickering

* debounced filters

* nit

* tooltips

* fix: update AssetUsage type to match new ListAssetsResponse structure

The ListAssetsResponse changed from an array to an object with an 'assets' property.
Updated the type extraction accordingly.

Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com>

* sqlx prepare

* Delete .claude/hooks/.symlink-manifest

* unnecessary dep

* nit refactor

* nit comment

* nit naming

* CI fix attempt 1

* ee ref

* nit remove alerts

* nit

* chore: update ee-repo-ref to 138a4f5f868f3bded5bb7cb77b222b532c07e4af

This commit updates the EE repository reference after PR #395 was merged in windmill-ee-private.

Previous ee-repo-ref: 7d3a21d53066726e97dfea9f117373299bc9318c

New ee-repo-ref: 138a4f5f868f3bded5bb7cb77b222b532c07e4af

Automated by sync-ee-ref workflow.

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* fix: remove $schema field from Google AI output schema requests (#7765)

* fix: remove $schema field from Google AI output schema requests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: add $schema field to all output schema integration tests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: remove $schema field from Google AI tool parameter schemas

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: add workspace script tool test for AI agents

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* fix: Fix app multiselect not refreshing result when creating element (#7766)

* Fix returning

* asset columns are saved and displayed in the assets page

* runtime assets column detectionz

* frontend nits

* update regex parsers

* UI nits

* Display asset columns in flow graph

* Column hint directly in asset node

* nit bg

* sqlx prepare

* ee repo ref

* chore: update ee-repo-ref to 66a68df97e8c65c498b28f302a365ab8687cad9e

This commit updates the EE repository reference after PR #402 was merged in windmill-ee-private.

Previous ee-repo-ref: 0a32bc104cbaec9664a4d7cb1565823722c875a1

New ee-repo-ref: 66a68df97e8c65c498b28f302a365ab8687cad9e

Automated by sync-ee-ref workflow.

---------

Co-authored-by: centdix <40307056+centdix@users.noreply.github.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
Co-authored-by: hugocasa <hugo@casademont.ch>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: wendrul <53628737+wendrul@users.noreply.github.com>
Co-authored-by: Alexander Petric <alpetric@users.noreply.github.com>
Co-authored-by: Devdatta Talele <50290838+devdattatalele@users.noreply.github.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com>
2026-02-04 16:27:53 +00:00
Guilhem
3b5c1657c7 fix(frontend): improve runs detail page (#7694)
* separate flow and status with splitpane

* fix autoscroll behavior

* Flow log viewer nit

* not graph viewer

* Improve flow job result

* Create job detail header to replace metadata

* Remove FlowPreviewResult

* Create compact job header

* Use job header in runs page

* Improve runs page run preview

* Use flow header in detail section

* Show logs for script steps

* Clean old schedule status

* Limit result height

* Script run detail improvement

* Script run preview improvement

* Fix csv table overflow

* surface tertiary as background for Inputs

* nit

* Improve runs detail skeleton

* fix check

* nit

* Improve node definition

* fix flow module component overflow

* Use component DataTable for flow schema viewer

* Add language icon to step detail

* improve run header

* Improve Job detail header

* nit

* restore isOwner logic

* Handle resume flows

* restore execution status in run preview

* restore flow execution status in the preview

* flow preview, add status bar

* nit module status

* nit

* Remove flor preview result

* nit

* nit

* fix flow result card

* nit

* nit

* nit

* improve field selection on runs detail based on job type

* Improve column layout

* create JobStatusIcon component

* remove job status badge icons

* improve compact version

* use shared job field display

* improve job detail field display

* fix badge alignment

* increase padding

* nit

* nit

* improve compact display

* make background darker for metadata

* use auto layout

* fix auto layout

* improve display

* fix truncate logic

* fix compact

* improve flex adaptibility

* improve responsive layout

* improve extra compact header

* nit

* remove unused icons

* nit

* Improve flow result display

* nit

* merge progressbar and execution status

* handle canceled flow better
2026-02-03 17:20:35 +00:00
dieriba
6418c4bcc6 feat: nextcloud native triggers (#6797) 2026-01-26 16:49:52 +00:00
Guilhem
7385726741 fix(frontend): Improve flow detail page (#7647)
* improve flow detail page

* Do not display seconds for last edit

* expand graph when possible

* nit

* make flowGraph min height reactive

* Add flow graph tab when chat mode enabled

* improve script detail layout

* nit

* nit

* Update frontend/src/lib/components/TimeAgo.svelte

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* Update frontend/src/lib/components/TimeAgo.svelte

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* Update frontend/src/lib/components/TimeAgo.svelte

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* Update frontend/src/lib/components/TimeAgo.svelte

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* nit

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
2026-01-22 15:55:18 +00:00
claude[bot]
6be060bea8 feat(ai): add websearch tool for AI agents (#7399)
* refactor(ai): use responses API for OpenAI/Azure, create 'other' provider for completion endpoint

- Created new 'other.rs' provider using the OpenAI-compatible completion endpoint
- Refactored 'openai.rs' to use the Responses API for both text and image output
- Updated query_builder to route OpenAI/AzureOpenAI to OpenAIQueryBuilder
- All other providers (Mistral, DeepSeek, Groq, etc.) now use OtherQueryBuilder
- Updated OpenRouter to delegate to OtherQueryBuilder instead of OpenAIQueryBuilder

This prepares the codebase for adding websearch tool support using the Responses API.

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* feat(ai): add websearch as AI agent tool type

- Added WebsearchToolValue to ToolValue enum in flows.rs
- Updated all pattern matches to handle websearch tool type
- Added has_websearch parameter to run_agent function
- Websearch tools don't require additional configuration

This prepares the backend for provider-specific websearch implementations.

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* feat(frontend): add websearch as tool option in AI agent UI

- Added WebsearchTool type and utility functions to agentToolUtils.ts
- Added "Web Search" option in tool insertion menu (InsertModuleInner)
- Updated NewAIToolNode to handle pickWebsearchTool event
- Updated AIToolNode to render websearch tools properly
- Updated FlowModuleSchemaMap to create websearch tools

Users can now add websearch tools to AI agents through the UI.

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* feat(ai): implement websearch for OpenAI provider

- Added has_websearch parameter to BuildRequestArgs
- OpenAI provider now adds web_search tool to requests when enabled
- Uses OpenAI Responses API web_search tool type
- Websearch tool is added before other custom tools in the request

Implements websearch functionality for OpenAI and Azure OpenAI providers.

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* feat(ai): implement websearch for Anthropic provider

- Created anthropic.rs provider with native Anthropic API format
- Added web_search tool to Anthropic requests when enabled
- Anthropic uses /messages endpoint with x-api-key authentication
- Updated query_builder to route Anthropic to new provider

Implements websearch functionality for Anthropic Claude models.

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* feat(ai): implement websearch for Gemini/GoogleAI provider

- GoogleAI now uses completion endpoint (other.rs) for text instead of responses API
- Added Google Search grounding when websearch is enabled
- Uses google_search_retrieval tool in request when has_websearch is true
- Updated parse methods to use OtherQueryBuilder for completion endpoint

Implements websearch functionality for Google Gemini models.

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* fix frontend

* fix anthropic and openai

* better for gemini

* structured output

* cleaning

* fix validate tool

* fixes

* cleaning

* cleaning

* fix for openai

* no responses api for azure

* fixes

* fix

* add tests for ai agent

* avoid panic

* better tests

* test user images

* fix tool choice

* always use streaming backend side

* big cleaning

* show annotations plus agent action for open ai websearch use

* show annotations plus agent action for anthropic websearch use

* show annotations plus agent action for google websearch use

* nit forntend

* rm

* fix

* add test for image ouptut

* fix for azure

* add in openflow

* fix

* fix

* nit tests

* fixes

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: centdix <farhadg110@gmail.com>
2025-12-22 11:41:28 +00:00
Diego Imbert
c6c7f3415a Specific asset tables (#7323)
* data tables settings ui

* install runed

* zod 4 fixes

* use new toJSONSchema

* Migrate ducklake catalogs to more generic custom instance databases

* fix compilation

* Safety conversion for old duckdb ffi

* data tables settings

* ts client basis

* inline run works

* datatables work

* Revert "datatables work"

This reverts commit 6e1588d59e.

* datatables work (without leaking pg credentials)

* println

* separate sqlUtils.ts

* nit

* Separate custom instance db Select and Wizard components

* nit

* nit wording

* add tags to custom instance dbs

* error when trying to use ducklake as datatable or opposite

* show status in dropdown

* data table instance setup works

* sqk function for ducklake

* factorize logic

* fix temp reactivity

* Data table assetexplore

* Migrate S3 permissions to modal

* Revert "Migrate S3 permissions to modal"

This reverts commit 0631d03cb0.

* nit query -> fetch

* Custom instance setup new look

* run_language_executor separate fn

* run_inline param

* nit wording

* Better typed client

* Data tables display as assets in frontend

* asset db icon

* nit

* cleaner errors

* nit

* Fix sed calls in mac

* run_inline_script_preview in python client

* basic python datatable client

* datatable and datalake parser in python

* ducklake client python

* nit fix

* Fix migration producing NULL instead of {} when no custom databases

* merge conflict fail

* python ducklake client arg fix

* parse or infer sql types in ts client

* ts asset parser, detect datatable & ducklake R/W

* fix sql repl for other read ops than select

* export type SqlTemplateFunction

* rename list_custom_instance_pg_databases

* typecheck datatable and ducklake name in Typescript

* Fix typecheck datatable and ducklake in TS

* declare module overriding instead of extending

* infer_sql_type in python client

* SqlQuery object in python

* fix merge conflicts

* update const_format

* CI fix

* factor out to var_identifiers

* sqlx prepare

* unnecessary security (admin is required)

* clearer comment

* ee repo ref

* nit snake case

* claude step 1: detect var declarations

* move detect_sql_access_type to common mod

* claude step 2: detect when saved vars are queried

* Revert "claude step 2: detect when saved vars are queried"

This reverts commit 1e1f930568.

* Revert "claude step 1: detect var declarations"

This reverts commit f866f4819d.

* remove ducklake/datatable and default

* detect data table assigns in var_identifiers

* Python parser successfully infers R/W/RW from ducklake / datatable

* still register ducklake/datatable if not used as unknown R/W

* Go to settings button in Assets Dropdown on not found

* nit

* sqlx prepare fail

* manual fix, somehow sqlx prepare won't do it

* fix frontend ci

* ee repo ref

* ducklake_user doesnt exist in unit tests

* nit fix

* ui nit

* nit

* nit missing clone

* fork ducklakes and datatables

* fix surface hover bug

* stupid mistake

* better deeply reactive mutable derived

* Ducklake picker

* Editor bar data tables

* DuckDB supports datatables

* datatable in duckdb asset parser

* duckdb asset parser var_identifiers

* Revert "duckdb asset parser var_identifiers"

This reverts commit 88068b1a77.

* sqlx prepare

* Box pin in test_workflow_as_code to fix stack overflow

* stash

* sql asset parser parses most s3 literals

* nit

* Detect attach + handle returning RW

* detect assets used with dot notation

* detect implicit access with USE dl; syntax

* Add assets as unknown if var was never used

* Support default ducklake/datatable main in parser

* ignore asset parsing errors in frontend (avoid flow layout shift)

* super weird duplication (merge conflict ?)

* nits

* fix duckdb parser detecting too much as asset when RW ctx is unknown

* fix transparent assets btn

* missing arg

* nit styling

* asset parser specific table parsing

* fix resource specific table parsing

* More concise asset display in flows + better icons

* fix assets page filtering out resources with added table

* Fix frontend to support specific table assets

* Open DB Manager to specific table

* Specific table parser in Python and TS + unit tests

* Fix UPDATE setting access to None

* fix flow edge rendering on top of output picker

* python parser fix var override bug

* add ts test

* fix compilation

* sqlx prepare

* update parsers version

* fix missing schema key onDelete
2025-12-11 13:34:17 +00:00
centdix
3d5b79c154 feat(aichat): simplify flow mode edits (#6981)
* draft

* Phase 1: Remove deprecated granular flow AI tools

Simplify AI chat flow mode to use only YAML-based editing:
- Remove all commented-out granular tools (add_step, remove_step, set_code, etc.)
- Clean up FlowAIChatHelpers interface to only essential methods
- Update system prompts to focus on YAML-only workflow
- Remove unused imports and type definitions

This is part of a larger refactoring to simplify the flow editing
experience to a single YAML editing tool with automatic diff visualization.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* use minified json

* use openflow in system prompt

* handle inputs

* cleaning

* cleaning

* diffmode in flowgraph

* remove acceptrejectmodule

* use new diff mode

* cleaning

* better props

* better logic

* cleaning

* accept reject logic

* use get set

* draft manager

* use diff manager

* draft

* Refactor flowDiffManager to be instance-based with auto-computation

- Remove singleton export, making it instantiable per FlowGraphV2
- Add afterFlow state tracking for auto-diff computation
- Add beforeInputSchema/afterInputSchema for schema change tracking
- Add $effect for reactive auto-computation when beforeFlow/afterFlow changes
- Add setAfterFlow() and setInputSchemas() methods
- Simplify accept/reject methods to just mark pending=false
- Add validation to throw error when accepting/rejecting without beforeFlow
- Update setSnapshot to accept undefined for clearing

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Refactor FlowGraphV2 to own diffManager instance

- Import and create diffManager instance per FlowGraphV2
- Remove onAcceptModule and onRejectModule props
- Add validation $effect to error if both diffBeforeFlow and moduleActions provided
- Add $effect to sync props (diffBeforeFlow or moduleActions) to diffManager
- Add $effect to watch current flow changes and update afterFlow
- Replace computedDiff with diffManager.moduleActions
- Use raw modules instead of merged flow (diffManager handles merging)
- Expose getDiffManager() and setBeforeFlow() methods
- Pass diffManager to graph context instead of callbacks
- Remove $inspect for removed props

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Update FlowModuleSchemaMap to use FlowGraphV2's diffManager

- Remove import of flowDiffManager singleton
- Update setBeforeFlow to call graph.setBeforeFlow()
- Update setModuleActions and getModuleActions to use graph.getDiffManager()
- Add getDiffManager() proxy method
- Simplify handleAcceptModule and handleRejectModule to use new API
- Handle editor state separately from diff operations
- Remove diffBeforeFlow, moduleActions, onAcceptModule, onRejectModule props passed to FlowGraphV2
- Remove onAcceptModule and onRejectModule from Props interface and destructured props

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Update FlowAIChat to use flowModuleSchemaMap's diffManager

- Remove import of flowDiffManager singleton
- Update revertToSnapshot to use flowModuleSchemaMap.getDiffManager()
- Add null check for diffManager before using

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Verify FlowGraphDiffViewer compatibility with refactored architecture

FlowGraphDiffViewer already uses the correct prop patterns:
- Before graph: moduleActions prop (display-only mode)
- After graph: diffBeforeFlow prop (full diff mode with auto-computation)

Each FlowGraphV2 instance creates its own diffManager, making the side-by-side
view work correctly with independent diff state per graph.

No code changes required.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Update graph components to use diffManager instead of callbacks

- Update graphBuilder.svelte.ts to pass diffManager instead of onAcceptModule/onRejectModule
- Update InputNode and ModuleN type definitions with diffManager
- Update ModuleNode.svelte to pass diffManager to MapItem
- Update MapItem.svelte to pass diffManager to FlowModuleSchemaItem
- Update FlowModuleSchemaItem.svelte to use diffManager directly for accept/reject
- Replace callback-based accept/reject with direct diffManager calls
- Only show accept/reject buttons when beforeFlow exists and action is pending

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix removed modules not showing in diff viewer

Problem: After refactoring, removed modules were no longer appearing in the
diff viewer because we changed effectiveModules from using the merged flow
(which includes removed modules) to using raw modules.

Solution:
- Add mergedFlow state to flowDiffManager to store timeline's merged flow
- Add markRemovedAsShadowed parameter support for side-by-side view
- Store timeline.mergedFlow in auto-computation $effect
- Add getter for mergedFlow and setMarkRemovedAsShadowed method
- Clear mergedFlow in clearSnapshot()
- Update FlowGraphV2 to set markRemovedAsShadowed in diffManager
- Update effectiveModules/FailureModule/PreprocessorModule to use mergedFlow

The merged flow contains all modules including removed ones, enabling:
- Unified view: Removed modules appear in red with "removed" badge
- Side-by-side view: Removed modules show as shadowed in After graph

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Simplify accept/reject logic by removing actions instead of toggling pending state

Previously, accepting or rejecting a module action would set pending to false but keep the action in the moduleActions map. This caused a bug where the $effect would overwrite moduleActions with fresh actions having pending: true, making accept/reject buttons reappear on previously handled modules.

Now, when a user accepts or rejects a module action, we remove it entirely from the moduleActions map. This is simpler and fixes the button reappearing issue.

Changes:
- acceptModule: Remove action from moduleActions instead of setting pending: false
- rejectModule: Remove action from moduleActions instead of setting pending: false
- checkAndClearSnapshot: Check if moduleActions is empty instead of checking pending states
- Fix typo: getModuleFromFrom → getModuleFromFlow

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* cleaning

* fix logic

* make diff drawer part of manager

* accept submodules

* fixes

* Phase 4: Add checkAndApplyChanges() helper to flowDiffManager

- Added new checkAndApplyChanges() function to apply mergedFlow to flowStore when all changes are decided
- This replaces the old checkAndClearSnapshot() behavior and ensures flowStore is updated atomically
- Handles both flow structure and input schema updates

* Phase 2: Simplify acceptModule() - only modify mergedFlow

- Remove flowStore mutations from acceptModule()
- For removed modules: just delete the shadowed (__prefix) version from mergedFlow
- For added/modified: no action needed (already correct in mergedFlow)
- Call checkAndApplyChanges() to apply changes when all decided

* Phase 3: Simplify rejectModule() - only modify mergedFlow

- Remove all flowStore mutations from rejectModule()
- For added modules: delete from mergedFlow
- For removed modules: replace shadowed (__) module with original from beforeFlow
- For modified modules: restore old version in mergedFlow
- For Input schema: revert afterInputSchema
- Call checkAndApplyChanges() to apply changes when all decided

* Phase 5: Verify acceptAll/rejectAll work with new architecture

- acceptAll() and rejectAll() already pass options correctly to acceptModule/rejectModule
- They will automatically benefit from checkAndApplyChanges()
- No changes needed for this phase

* Phase 6: Remove FlowGraphV2 reactive effect that updates afterFlow

- Removed the  (lines 252-266) that continuously updated afterFlow
- This effect created reactive loops when flowStore changed
- afterFlow should only be set once when AI generates changes via setFlowYaml()
- The initial sync effect (lines 226-250) is kept for prop-driven diff mode

* Phase 7: Update FlowAIChat setFlowYaml to use diffManager

- Changed setFlowYaml() to use diffManager.setAfterFlow() instead of modifying flowStore
- flowStore remains unchanged during AI review phase
- Changes are staged in mergedFlow for user review
- Only applied to flowStore when all changes are accepted/rejected
- Added error handling for missing diffManager

* Fix linter warnings

- Remove unused FlowTimeline type import
- Fix ChangeTracker initialization with proper type parameter
- Keep deleteModuleFromFlow and checkAndClearSnapshot for potential future use

* Update plan document with implementation status

- Mark all phases as complete
- Add commit references
- Update file checklist
- Add implementation summary at top of document

* Add comprehensive implementation summary document

- Detailed overview of architecture changes
- Before/after comparisons for each file
- Complete testing scenarios checklist
- Troubleshooting guide
- Migration notes and backwards compatibility info

* Show pending modules in editor panel

- Pass diffManager from FlowModuleSchemaMap to FlowEditorPanel
- Add effectiveModules derived value that uses mergedFlow when in diff mode
- Update module iteration to use effectiveModules instead of flowStore
- Allows users to view added/modified modules during AI review
- Fixes issue where clicking on pending modules showed nothing

* Add implementation summary for show pending modules feature

* fix

* shorter system prompt

* Fix Input schema diff mode issues

- Add Accept/Reject buttons to Input node (previously only showed Diff button)
- Pass diffManager to FlowInput component
- Add effectiveSchema derived value that uses afterInputSchema when in diff mode
- Add effectiveDisabled to prevent editing Input when reviewing AI changes
- Update FlowInputViewer to show pending schema changes
- Fixes issue where Input schema changes couldn't be accepted/rejected
- Fixes issue where pending Input schema wasn't visible in the panel

* Disable delete and move buttons when in pending mode

- Add effectiveDeletable derived value that checks diffManager.hasPendingChanges
- Replace all instances of deletable with effectiveDeletable in template
- Prevents delete/move operations when AI changes are being reviewed
- Delete and move buttons are hidden when there are pending changes
- Buttons reappear once all changes are accepted or rejected
- Prevents conflicting operations during review phase

* no move or delte when reviewing

* use context

* inline script reduction

* use json

* rollback to direct modif

* fix merge

* cleaning

* fix reject removed

* add set step code tool

* better prompt

* add back relevant tools

* add back accept reject

* use edit mode for pending

* fix input

* remove unneeded effect

* cleaner + bug fix

* fix failure and preprocessor

* fix show diff for failure module

* fix accept reject on failre module

* no auto add module to context

* cleaning

* add back effect

* cleaning

* fix multiple setflowjson

* track effectivemoduleactions for graph rendering

* nit prompt

* styling

* rm md files

* rm flake copy

* cleaning

* fix z index

* fix revert

* only change before after

* use add remove modify tools

* input + failure + preproc tools

* parsing issues

* nit

* use raw schema for tools

* resolve ref for gemini

* fix schema

* show test on graph

* much cleaner logic

* ignore empty assets

* Remove debug console.log statements from production code

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Remove debug $inspect calls from FlowGraphV2

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add error logging to setFlowJson before re-throwing

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Standardize null/undefined handling to prefer null

- Use .nullable().optional() instead of .nullish() in Zod schemas
- Simplify addModuleToFlow signature to use string | null
- Coerce undefined to null when extracting parsed args
- Simplify null checks to only check !== null

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Remove debug console.log from AI tool functions

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Extract special module IDs to constants

Add SPECIAL_MODULE_IDS constant with INPUT, PREPROCESSOR, and FAILURE
to avoid magic strings throughout the flow AI chat code.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add cleanup for diffDrawer reference on unmount

Prevents potential memory leaks by clearing the diffDrawer reference
when the FlowGraphV2 component is destroyed.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Use structuredClone instead of JSON.parse(JSON.stringify())

structuredClone is more efficient and type-safe for deep cloning objects.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Cache module lookups in reconstructMergedFlow

Move getAllModulesMap and getAllModuleIds calls outside the loop to avoid
redundant recomputation. Track merged IDs incrementally as modules are added.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Revert "Use structuredClone instead of JSON.parse(JSON.stringify())"

This reverts commit a62ba5b980.

* cleaning

* allow delete

* better openflow for ai agents + truncate system prompt

* handle ai agent tools

* fix set code for tool

* fix wrong cancel request called

* mark tool calls as canceled

* get lang instructions

* use streamiing args

* give db url to claude

* fix revert

* save and clear when leaving editor

* keep whitespace in user message

* uniformize colors

* fix diff button

* remove db from backend claude

* remove move module tool

* no failure and preprocessor

* fix error given to llm

* fix z index

* fix ts errors

* cleaning

* fix add module logic

* fix(copilot): add 'tools' to branchPath description for aiagent containers

The branchPath parameter description was missing 'tools' option for aiagent
containers and didn't mention branchall support.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(copilot): correct AI agent tool IDs and summaries documentation

Tool summaries CAN contain spaces (they're human-readable descriptions).
Only tool IDs must avoid spaces.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(copilot): remove reference to non-existent set_flow_json tool

The set_module_code tool description referenced set_flow_json which
doesn't exist as an exposed tool (it's an internal helper).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(copilot): clarify inspect_inline_script is read-only

The tool description incorrectly suggested it could modify code.
This tool only inspects - use set_module_code to modify.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(copilot): clarify afterId behavior for AI agent tools

Updated wording to clarify that afterId can be used but is optional
for AI agent tools since tool order doesn't affect execution.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor(copilot): remove unused id param from get_instructions_for_code_generation

The id parameter was only used to check for preprocessor, which is no
longer needed. Simplified the tool to only require the language param.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* docs(copilot): add result format to search_scripts tool description

Helps AI understand what data format to expect from the tool.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* docs(copilot): add result format to resource_type tool description

Helps AI understand what data format to expect from the tool and
provides example resource type names.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* nit

* Add support for adding branches to branchall/branchone via add_module

Previously, add_module could only add modules inside existing branches.
Now, using insideId with branchPath=null will add a NEW branch to a
branchall or branchone container.

API:
- add_module({ insideId: "my_branchall", branchPath: null, value: { summary: "New Branch", skip_failure: false, modules: [] } })
- add_module({ insideId: "my_branchone", branchPath: null, value: { summary: "Condition", expr: "...", modules: [] } })

Changes:
- Extended addModuleToFlow to handle branchPath=null case
- Updated validation to allow branchPath=null when adding branches
- Updated tool descriptions and system prompt documentation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* nit

* add remove branch tool

* check all ids for duplicates

* no dup

* nit

* cleaning

* fix dup ids

* split core.ts

* only mount diff drawer if useful

* remove wrong logic

* update exprs

* fix

* chore(flow): Add unit tests to flow diff manager (#7291)

* setup

* add basic tests for flowdiff

* add complex tests

* fix branch issue

* more complex tests

* add flow diff manager tests

* add utils

* better handling of moved case

* more tests for move case

* add buggy test case

* rm

* rework config

* cleaning

* fix config

* rm

* fix for reverting type change module

* all good

* rm

* add missing testmode

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-12-08 19:08:14 +01:00
hugocasa
a8d40733f4 fix(frontend): use right workspace script version in flow status (#7308) 2025-12-05 18:02:41 +00:00
Diego Imbert
6886ba72d1 fix InsertModuleButton sometimes disappearing when waiting events (#7246) 2025-11-27 17:42:13 +00:00
Guilhem
c37dde096c fix(frontend): show trigger table when no trigger selected (#7219)
* improve triggers pannel

* nit

* nit
2025-11-25 12:14:19 +00:00
Guilhem
a2d3297343 fix(frontend): improve preprocessor discoverability (#7214)
* add tooltip

* show payload tooltip

* Add preprocessor button

* nit

* improve preprocessor template

* fix add preprocessor

* fix oneof wrapping

* clean

* Use funnel cog icon for preprocessor step

* nit
2025-11-24 19:43:14 +00:00
Guilhem
054aeb3327 fix(frontend): missing node Result id migration (#7182)
* fix missing id changes

* fix ai tool selection
2025-11-20 11:33:10 +00:00
Guilhem
cfeb294308 feat(frontend): add notes to flow (#6628)
* Add note component

* save note size and position

* move add note button up

* nit

* Add markdown support

* wip

* fix add sticky note button

* fix text update

* Add sticky note to saved flow data

* add note color picker

* Introduce node multiselect

* Add group notes

* Adapt layout to group node

* create a note manager class

* clean reactivity

* clean

* improve adaptive layout to group note

* modify layout based on cached text height

* fined grained graph rendering for notes

* separate noteManager into editor and render

* separate noteManager into editor and render

* create a note change observer

* render note node from context

* simplify note state managment

* show note in flow viewer

* clean dirty changes

* clean selection manager

* fix layout check

* improve bg surface select

* Handle z-index for stacked group notes

* clean selection manager

* exclude notes from rect select

* Allow switch between selection modes with keyboard keys

* improve selection box styling

* prevent dragging note when editing

* nit

* Simplify selection using svelte flow built in feature

* handle note selection separately

* Add min size for notes

* improve selection toggle

* improve mode switch

* make size and position optional for group notes

* Improve initial viewport position

* Add context menu for the canevas

* nit

* Add node context menu

* improve note select

* use clickoutside for note deselect

* use pointerdown outside to close context menu

* nit

* fix selection issues

* make edges non selectable

* improve color palette

* fix backend

* fix backend check

* cargo lock restore

* Add toggle to display notes

* fix note selection

* nit

* account for css offset in for loop

* fix multiple selection pannel styling

* clear flow selection when creating note

* Improve placeholder and note default text

* Escape note edit mode when pressing Esc

* Allow note edition in local dev

* clean

* Handle subflow selection

* prevent group note resizing

* nit

* allow notes in flow expand

* Improve multi select panel

* Allow context menu in note mode

* Add event listenner to fix pane click deselect

* prevent zoom in text area in notes

* improve bounding box styling

* Use control for box selection for non mac users

* nit

* clean notes groups

* nit

* use portal for note actions

* handle assets node when computing note layout

* Simplify layout compute for notes

* use smart color choice for notes

* Switch display note when adding a new note

* clean code

* improve group note bound size calculation

* simplify AI tool nodes and asset handling

* nit

* nit

* improve flow centering

* create group note button

* Improve selection of nodes

* Revert "Improve selection of nodes"

This reverts commit d2c40d82b1.

* refert backend changes

* nit

* improve graph selection

* clean

* make backend work except job runs

* fix notSelectable

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-11-19 21:25:57 +00:00
Diego Imbert
2b826cee5a Hide 'show assets' toggle when there are no assets (#7037) 2025-11-03 13:43:05 +00:00
Guilhem
1d7ba8b1e1 always show subflow expand button (#7033)
* fix unwanted tab change

* expand subflow is always visible

* nit

* nit
2025-11-03 12:23:01 +00:00
centdix
04d2ef419d feat(flow): Add graph diff visualizer (#6948)
* graph mode

* show colors

* show module diff viewer button

* better diff logic

* small width merge graph diff

* invert logic

* simplify

* better logic

* put removed modules in initial position

* nit

* nit

* fix conflicting ids

* fix

* add shadowed for after

* better position logic

* fix

* cleaning

* use splitpanes

* add toggle

* fix

* sync move

* icons

* handle zoom

* left header snippet

* cleaning

* cleaning

* remove stats

* big cleaning

* fix

* fix

* fix

* remove not working logic

* invert logic

* nit

* use in deploymentui

* fix typo

* no custom style

* handle nested

* simpler logic

* fix

* fix
2025-11-01 15:17:56 +01:00
Guilhem
b4d081306b fix (frontend): overflow in tool picker (#7014)
* fix overflow in tool picker

* nit
2025-10-30 18:13:39 +00:00
Ruben Fiszel
a807670589 proper script debounce editor 2025-10-30 17:04:16 +00:00
Guilhem
94e5c21e25 glm/fix-prop-picker-position (#6991)
* fix button fill container

* fix popover overflow

* Update script picker to new guidelines

* fix expand sublow button

* fix scroll

* fix popover dark mode

* fix refresh button

* nit

* fix popover oferflow

* nit
2025-10-29 21:03:12 +00:00
Diego Imbert
c44ac70b35 Show Assets toggle (#6985)
* Show Assets toggle

* Fix flow graph not updating when manuallly changing ambiguous asset R/W
2025-10-29 21:02:09 +00:00
Diego Imbert
d474679277 nit: Asset node popover readability (#6974)
* nit asset node popover

* nit Load secret value btn
2025-10-28 11:31:15 +00:00
Guilhem
a12c2788ec fix(frontend): add transparency to color palette (#6947)
* fix input padding

* fix surface hover on surface hover

* fix metadata gen

* fix border selected darkmode
2025-10-27 21:21:21 +00:00
Diego Imbert
0b82ff4ebb New color palettes (#6945)
* debug runs text fix

* New color palette

* Remove luminance-blue

* replace hardcoded flow node colors

* nit
2025-10-27 15:46:28 +00:00
Guilhem
032f0c1f8c feat: UX/UI full overhaul to meet new design system 2025-10-22 15:56:36 +00:00
centdix
97ac1be036 feat(aiagent): allow mcp as tools (#6790)
* draft mcp client

* testing

* fix

* cleaning

* mcp resource in inputtransforms

* cleaning

* big cleaning

* cleaning

* no arc

* add utils file

* refactor tools

* add mcp actions

* draft frontend

* send arguments from backend

* better frontend

* cleaning

* use token for auth

* add logo

* rm

* fix

* fix

* chore: refactor mcp for ai agents (#6829)

* Add Tool enum for AIAgent with backward compatibility

- Created Tool enum that can be either Windmill (FlowModule) or Mcp (resource reference)
- Created McpToolRef struct to hold MCP resource path
- Implemented custom Deserialize for Tool with backward compatibility:
  - New format: {type: 'windmill'|'mcp', ...}
  - Old format: FlowModule objects (automatically wrapped in Tool::Windmill)
- Updated AIAgent to use Vec<Tool> instead of Vec<FlowModule>
- Updated FlowValue::traverse_leafs to handle Tool enum
- Backward compatible: old flows with Vec<FlowModule> will deserialize correctly

* Refactor AI executor to process Tool enum instead of extracting MCP from input_transforms

- Separate Windmill tools and MCP resource paths from tools list
- Process Windmill FlowModules into Tool definitions
- Load MCP tools from resource paths in Tool::Mcp variants
- Remove old logic that extracted mcp_resources from input_transforms
- Import FlowModule, remove unused InputTransform
- Fix type issues: use .as_str() for path and handle Option<bool> properly

* handle in args

* mcp as flowmodule

* frontend

* config for mcp

* simplify logic

* fix ai executor logic

* cleaning

* clean frontend

* fix

* better resource picker

* fix and styling

* add endpoint to fetch tools

* apply tool filtering

* fix name validation

* better ui

* use cache

* fix

* fix merge

* refactor: Separate MCP tools from FlowModule in AIAgent

- Add new AgentTool, ToolValue, and McpToolValue types
- Update AIAgent to use Vec<AgentTool> instead of Vec<FlowModule>
- Implement From traits for clean conversion between AgentTool and FlowModule
- Add backward compatibility via custom deserializer for AgentTool
- Simplify resolve_module logic by reusing existing resolve_modules function
- Update traverse_leafs to handle AgentTool structure

This refactoring separates MCP tools from FlowModule tools, making the
type system clearer and eliminating the need to treat MCP servers as
a special case of FlowModule.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: Update ai_executor and worker_lockfiles for AgentTool

- Update ai_executor.rs to handle new AgentTool structure
  - Separate MCP tools from FlowModule tools using ToolValue enum
  - Convert AgentTool to FlowModule for backward compatibility
  - Add imports for AgentTool and ToolValue types

- Update worker_lockfiles.rs for lazy loading optimization
  - Convert AgentTool <-> FlowModule in insert_flow_modules
  - Preserve lazy loading for FlowModule tools via modules_node
  - Keep MCP tools inline (lightweight, no need for lazy loading)
  - Maintain backward compatibility with existing flows

This enables the lazy loading optimization for FlowModule tools while
keeping MCP tools inline, balancing performance and simplicity.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* cleaning

* adapt frontend

* cleaning

* cleaning

* type fix

* cleaning

* fix back comp

* move mcp button position

* nit

* cleaning

* fix nested removal

* cleaning

* opti

* fix chat markdown display

* fix chat messages layout

* fix back comp frontend

* fix deserializer

* nit

* simpler serializer

* use if else

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-21 15:28:52 +00:00
Ruben Fiszel
963e0fb356 fix: flow dev mode improvements 2025-10-17 13:36:05 +00:00
Ruben Fiszel
75ceba81d9 nits disableAi 2025-10-04 16:54:59 +00:00
centdix
047420e5ad feat(flow): add option to turn flow into chat (#6658)
* add toggle option + chat interface

* backend impl

* draft

* put info in schema

* Revert "backend impl"

This reverts commit c534eeb49986424e2c12e2c5642be4e17ba380d1.

* chat interface in flow input

* cleaning

* add logic for running flow + styling

* handle historic args

* fix frontend changes

* add tables

* add conv list

* add endpoints

* adapt frontend

* list message logic

* save message in db

* save response in db

* cleaning

* better migrations

* refresh on new conv

* better logic for messages

* nit

* genere conversation uuid from frontend

* store chat mode info in flow status

* better ui for chat

* collapse chat

* ui

* infinite scroll on convs

* infinite scroll on messages

* fix ui

* new chat entry on new

* cleaning

* change setting logic

* fix test logic from flow input

* move toggle to input

* add warning modal when enabling chat mode

* add summary and explanation on inline script

* add hint for chat mode on user_message desc

* show chat message instead of input in graph

* add warning for triggers

* one logo when not expanded

* use infinitelist for conversations

* add warning when deployment in progress

* full width button

* better icon for menu

* better input + nits

* put toggle in action

* use waitjob

* cleaning

* cleaning

* scroll on new + cleaning

* use enum

* fix logic

* full screen

* cleaning

* exit on updatesqlx error

* Update SQLx metadata

* fix

* cleaning

* add for wait result endpoint

* add missing drop

* delete cascade

* fix: use macro version of query_as in flow_conversations.rs

Use sqlx::query_as! macro instead of query_as function for compile-time
SQL validation and better type safety

Co-authored-by: centdix <centdix@users.noreply.github.com>

* fix: update comment to clarify conversation message update condition

The comment now accurately reflects that the update happens when
it's a flow and it's done (flow_is_done)

Co-authored-by: centdix <centdix@users.noreply.github.com>

* fix: only parse chat_input_enabled if conditions are met

Move the parse_chat_input_enabled() call inside the condition check
to avoid unnecessary parsing when the flow is not done or unsuccessful

Co-authored-by: centdix <centdix@users.noreply.github.com>

* fix: use the same transaction for conversation creation

Pass transaction to get_or_create_conversation_with_id instead of
creating a new one, ensuring all operations are atomic

Co-authored-by: centdix <centdix@users.noreply.github.com>

* fix: remove update trigger and handle updated_at in application code

Remove the database trigger that automatically updates conversation
timestamp and instead update it explicitly when creating messages.
This gives better control and consistency.

Co-authored-by: centdix <centdix@users.noreply.github.com>

* Update SQLx metadata

* cleaning

* feat(aiagent): handle memory (#6719)

* implement memory

* s3 logic for memory

* fix typo

* much cleaner

* cleaning

* cleaning

* only if chat

* display nit

* nit

* fix stack overflow

* cleaning

* use len arg from input

* cleaning

* change order

* delete memory when conv deleted

* cleaning

* nit

* show description in expr mode

* opti

* opti

* updatee ref

* store string as simple string

* use markdown

* do not wait for deletion

* add delete loading

* fix logic

* fix markdown

* Update ee-repo-ref.txt

* Update SQLx metadata

* fix in test interface

* nit

* nit

* fix layout

* use memory_id to store memory

* shorter description

* rls + grant

* fix text overflow

* extract output from res

* cleaning

* handle streaming

* cleaning

* fix tool error

* nit

* update ref

* fix

* Update SQLx metadata

* nit

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: centdix <centdix@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-10-03 14:04:21 +00:00
Ruben Fiszel
311b410f2f fix: improve performance of flow viewer 2025-09-27 08:48:01 +00:00
Ruben Fiszel
7add57499c fix: improve graph rendering performances 2025-09-27 08:23:09 +00:00
Ruben Fiszel
1afb7a2156 improve job cancelling with new flow jobs locks 2025-09-20 11:00:45 +00:00
Ruben Fiszel
f71f9b0894 feat: load for loop jobs timeline directly from for loop flow status (#6646) 2025-09-19 17:32:09 +00:00
Diego Imbert
5cab802c42 fix: fix flow quick picker stuck (#6638)
* Fix flow quick picker stuck

* fix Ci
2025-09-18 18:56:59 +00:00
Ruben Fiszel
49e687b00d internal: rolldown vite (#6584) 2025-09-11 16:12:36 +00:00
Ruben Fiszel
196746223a nit check 2025-09-11 11:03:13 +00:00
Ruben Fiszel
21602f125f fix scheduled poll 2025-09-11 08:54:05 +00:00
Ruben Fiszel
eb9fbb999f updates 2025-09-11 07:57:01 +00:00
hugocasa
36bbde6239 feat: email triggers (#6548)
* feat: email triggers

* Change down migration to drop email_trigger table

* email triggers UI

* bug fix

* Apply suggestion from @ellipsis-dev[bot]

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* cli and git sync

* Revert "cli and git sync"

This reverts commit 220fd50d13.

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-09-09 21:04:23 +00:00
Guilhem
561bda2cce fix(frontend): improve flow step buttons layout (#6507)
* improve flow step buttons layout

* nit

* nit

* harmonize branche delete
2025-09-02 06:52:18 +00:00
Ruben Fiszel
e28c9df60f fix: fix preprocessor not displaying immediately on addition 2025-08-29 06:57:01 +00:00
hugocasa
c13747cda9 fix(frontend): ai agent flow status + UI nits (#6447)
* fix(frontend): ai agent flow status

* nit: prevent undefined node issue

* feat: UI nits + flow status select iter fix

* nit ai agent color in picker
2025-08-22 18:51:13 +01:00
hugocasa
28f1d61164 fix(frontend): graph cache of ai agent step tools (#6431) 2025-08-21 13:59:16 +02:00
hugocasa
958e8af782 feat: ai agent steps (#6393)
* feat: ai agent steps base

* better backend and graph

* feat: anthropic, log viewer

* nit

* fix(frontend): hide tool nodes from timeline

* move ai agent actions from flow status to flow status module

* nits and workspace/hub scripts support

* tmp ref

* fix merge

* feat: display agent tools status in the graph

* fix reactivity

* fix flow status

* nit
2025-08-20 22:40:57 +00:00
Diego Imbert
51568eee02 fix: improve computeAssetNodes rendering caching and performance (#6414)
* better caching for computeAssetNodes

* only pass required fields

* unecessary returns

* type not necessary
2025-08-19 21:12:34 +00:00
Ruben Fiszel
205618af0a nit module tracker 2025-08-19 07:56:17 +00:00
Ruben Fiszel
5e73c49ab6 fix: flow status reactivity improvement (#6402) 2025-08-19 02:12:57 +00:00
Ruben Fiszel
eae0c09979 fix: improve flow layout for more complex flow 2025-08-14 21:43:39 +00:00