* refactor: replace email with permissioned_as for triggers/schedules
Add a new `permissioned_as` column (format: `u/{username}`, `g/{group}`,
or raw email) to all trigger tables and schedule. This value is used
directly for job permission checks, removing the need for email lookups
when creating/updating triggers.
- Migration: add permissioned_as to all 9 trigger tables + schedule,
drop email from trigger tables (schedule keeps it for backwards compat)
- Backend: resolve_email() (async, DB) -> resolve_permissioned_as() (sync)
- Email cache: get_email_from_permissioned_as() with quick_cache for
places that still need email (fetch_api_authed, schedule backwards compat)
- Frontend: rename email/preserve_email -> permissioned_as/preserve_permissioned_as
in deploy data and OpenAPI schemas
- Tests updated for new field names and u/{username} format
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix sqlx/build
* update ee ref
* refactor: simplify resolve_edited_by to always use authed username
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix compile + migration
* update ref
* test: add trigger trait method tests for permissioned_as queries
Add tests that call TriggerCrud and Listener trait methods directly
to verify dynamic SQL correctly references the permissioned_as column.
Covers get_trigger_by_path, list_triggers, set_trigger_mode, and
fetch_enabled_unlistened_triggers for all trigger types.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* update sqlx
* fix: use permissioned_as directly for schedules and fix audit RLS for groups
- Schedule: permissioned_as only set on create, not on edit/set_enabled
- Schedule: stop reading email column, use get_email_from_permissioned_as
- Triggers: use fetch_api_authed_from_permissioned_as instead of edited_by
- Triggers: rename listener fields for clarity (username -> edited_by)
- Fix audit author username for group permissioned_as (g/test -> group-test)
to match session.user, preventing RLS policy violations on audit_partitioned
- OpenAPI: remove permissioned_as/preserve_permissioned_as from EditSchedule
- Add backwards-compat comments for schedule email writes
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: regenerate system prompts for permissioned_as field
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix build
* refactor: generalize onBehalfOf naming, add permissioned_as to EditSchedule
- Frontend: rename onBehalfOfPermissionedAs -> onBehalfOf with comments
explaining it carries emails for flows/scripts and permissioned_as for
triggers/schedules
- Frontend: rename getOnBehalfOfEmail -> getOnBehalfOf,
getOnBehalfOfPermissionedAsForDeploy -> getOnBehalfOfForDeploy,
customOnBehalfOfEmails -> customOnBehalfOf
- Backend: add optional permissioned_as/preserve_permissioned_as to
EditSchedule with COALESCE (only updates when provided)
- Backend: add on_behalf_of audit log for schedule edit
- Backend: remove unused resolve_on_behalf_of_permissioned_as
- Tests: remove email assertions from schedule update test (email is
just backwards compat, only permissioned_as matters)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: preserve email column when permissioned_as is preserved on schedule edit
Derive email from the preserved permissioned_as via cache lookup instead
of always writing authed.email. This keeps the email column consistent
with the old behavior for backwards compat with old workers.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: update deploy UI labels from "edited by" to "run as" for triggers
Triggers now use permissioned_as (not edited_by) for permissions, so
update the deploy UI wording to reflect this. Also update wm_deployers
group description to mention schedules and permissioned_as.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use u/username format for custom trigger/schedule deploy selection
When picking a custom user for trigger/schedule deployment, store
u/${username} (permissioned_as format) instead of the email. Flows/scripts
continue to use email format for on_behalf_of_email.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: show u/username format for "me" option in trigger deploy selector
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: simplify OnBehalfOfSelector to return the right format per kind
OnBehalfOfSelector now handles the email vs permissioned_as format
internally based on kind:
- triggers: returns u/username, displays u/username in all options
- flows/scripts/apps: returns email, displays username
The onSelect callback now takes (choice, value?) where value is already
in the correct format. Parent components just store it directly without
needing to know about the format difference.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: always show u/username format in OnBehalfOfSelector for all kinds
Display is now consistent: all kinds show u/username in the selector.
The returned value still differs (email for flows/scripts, u/username
for triggers) since the backend APIs expect different formats.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: replace email with permissioned_as in http_trigger test insert
The email column was dropped from trigger tables in the migration.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: review fixes — migration, app policy, capture cleanup, naming
- Migration: remove DEFAULT '', use nullable → populate → SET NOT NULL
- App policy: set both on_behalf_of and on_behalf_of_email for all choices
- OnBehalfOfSelector: return OnBehalfOfDetails {email, permissionedAs} instead of ambiguous value
- Remove unused email field from Capture struct and query
- Rename getSourceEmail/getTargetEmail → getSourceOnBehalfOf/getTargetOnBehalfOf
- Rename test functions from preserve_email to preserve_permissioned_as
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add permissioned_as to all test schedule INSERTs
Since the migration no longer uses DEFAULT '', all INSERTs must
explicitly provide permissioned_as. Updated test fixtures and
schedule_push tests.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: strip permissioned_as from exports/sync, fix OpenAPI required field
- Add permissioned_as to workspace export strip list (like edited_by)
- Add permissioned_as to CLI TriggerFile Omit list
- Fix TriggerExtraProperty.required: email → permissioned_as
- Regenerate frontend and CLI types
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: remove accidentally committed generated files
These directories are gitignored and should not be tracked.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: regenerate system prompts for permissioned_as schema changes
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: remove permissioned_as from CLI TriggerFile Omit list
Already stripped in workspace export, no need to also omit from the type.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: optimize email cache key and revert TriggerFile Omit change
- Use single concatenated string for cache key instead of (String, String) tuple
- Remove permissioned_as from CLI TriggerFile Omit (already stripped in export)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: zero-allocation email cache lookups using Equivalent trait
Use a borrowed EmailCacheKey(&str, &str) for cache lookups via
quick_cache's Equivalent support. Only allocates (String, String)
on cache miss for insert. This is called on every trigger fire
and schedule push.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add permissioned_as to Schedule required fields in OpenAPI spec
The backend always returns permissioned_as (non-optional String),
so the schema should reflect that.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: handle group- prefix in migration UPDATE statements
edited_by can be 'group-{name}' for group-owned triggers/schedules.
The migration now correctly maps these to 'g/{name}' format instead
of incorrectly producing 'u/group-{name}'.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Revert "fix: handle group- prefix in migration UPDATE statements"
This reverts commit 0971392b38.
* fix: use superadmin email to resolve permissioned_as in schedule migration
For users upgrading from older versions where edited_by may not reflect
the actual schedule owner, check if the email belongs to a superadmin
and look up their username. Otherwise fall back to edited_by.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: fall back to superadmin email when not in workspace usr table
If the superadmin isn't a member of the workspace, use their email
as raw permissioned_as instead of falling back to edited_by.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: always update permissioned_as and email on schedule edit
Consistent with pre-refactor behavior where email and edited_by
were always updated on every edit. permissioned_as is now always
set (to editing user or preserved value), removing the COALESCE
that previously preserved it when not provided.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add schedule permission tests and centralize group prefix constants
Tests: schedule create/update for normal user, workspace admin, and
superadmin not in workspace. Verifies schedule fields (email,
permissioned_as, edited_by) and pushed job fields (permissioned_as,
permissioned_as_email).
Constants: centralize "u/", "g/", "group-" as PERMISSIONED_AS_USER_PREFIX,
PERMISSIONED_AS_GROUP_PREFIX, USERNAME_GROUP_PREFIX.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use @unknown.windmill.dev for synthetic email fallback
Prevents privilege escalation: a user with username like
'superadmin_secret' would get superadmin via the synthetic
email matching SUPERADMIN_SECRET_EMAIL. Using a different
subdomain avoids any collision with hardcoded @windmill.dev emails.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* update ee ref
* sqlx
* chore: regenerate system prompts after main merge
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: update ee-repo-ref to bda51bc33bcb573659e7ff07d0a23ff6e23b8148
This commit updates the EE repository reference after PR #468 was merged in windmill-ee-private.
Previous ee-repo-ref: 8cf1802f8fe183f430830590b4f3172a50207843
New ee-repo-ref: bda51bc33bcb573659e7ff07d0a23ff6e23b8148
Automated by sync-ee-ref workflow.
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* refactor: extract object store code into windmill-object-store crate with filesystem backend
Consolidate all object_store-dependent code from windmill-common into a new
windmill-object-store crate. Add a filesystem-backed object store implementation
using LocalFileSystem for dev/testing without cloud credentials. Includes 30
comprehensive tests covering render_endpoint, lfs_to_object_store_resource,
duckdb_connection_settings, error mapping, and filesystem-backed integration tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* all
* all
* all
* all
* fix: fix raw_app hardcoded path, add missing ObjectStoreResource import, and add tests
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: move S3ModeFormat to windmill-types, make windmill-parser-sql optional, restore debug logs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* all
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Add windmill-operator crate for Kubernetes CRD-based instance config
Introduces a new `windmill-operator` crate that enables declarative
management of Windmill instance configuration via a Kubernetes
`WindmillInstance` CRD. The operator watches CRD resources and performs
full declarative sync of global_settings and worker configs to the
database, supporting GitOps workflows for instance-level configuration.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Add tests for windmill-operator CRD and db_sync
- 9 unit tests for CRD serialization, deserialization, metadata, and
status field behavior
- 15 integration tests for db_sync using #[sqlx::test] with full
declarative sync coverage: upsert, delete, protected keys,
idempotency, worker config prefix handling, and end-to-end sync
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Replace untyped BTreeMap CRD fields with typed structs for schema validation
GlobalSettings, SmtpSettings, IndexerSettings, and WorkerGroupConfig now
have explicit typed fields with serde(flatten) catch-all for forward
compatibility. The generated CRD YAML includes a full OpenAPI v3 schema
that Kubernetes validates on kubectl apply.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Type opaque serde_json::Value CRD fields with real structs
Replace most remaining serde_json::Value fields in WindmillInstance CRD
with properly typed structs derived from the codebase:
- oauths: BTreeMap<String, OAuthClient>
- otel: OtelSettings
- otel_tracing_proxy: OtelTracingProxySettings with ScriptLang enum
- critical_error_channels: Vec<CriticalErrorChannel> (untagged enum)
- critical_alerts_on_db_oversize: DbOversizeAlert
- ducklake_settings: DucklakeSettings with nested catalog/storage types
- custom_instance_pg_databases: CustomInstancePgDatabases
- autoscaling (worker config): AutoscalingConfig with integration struct
- custom_tags, default_tags_workspaces: Vec<String>
- default_tags_per_workspace: bool
Still opaque (serde_json::Value): object_store_cache_config (kube-core
can't generate schemas for internally-tagged enums), secret_backend
(EE-private), slack, teams (no clear struct definitions).
Regenerated CRD YAML with full OpenAPI schema (352→703 lines).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Unify instance config types and add bulk GET/PUT API
Move all typed settings (GlobalSettings, WorkerGroupConfig, etc.) from
windmill-operator/crd.rs into windmill-common/instance_config.rs so both
the API server and operator share a single source of truth. Add diff/apply
logic (Merge mode for UI, Replace mode for operator) and InstanceConfig::from_db().
Add GET/PUT /settings/instance_config endpoints so the frontend loads all
settings in 1 call instead of 42, and saves with a single bulk PUT. The
backend handles the diff internally, running pre-write hooks for changed keys.
Refactor windmill-operator/db_sync.rs to use the shared diff+apply functions
and slim crd.rs down to the CRD wrapper with re-exports.
Includes 32 unit tests and 30 integration tests covering serialization,
diff logic, DB roundtrips, protected settings, and edge cases.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add Form/YAML toggle to instance settings UI
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: show Form/YAML toggle regardless of hideTabs prop
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* refactor: replace toggle button group with simple YAML toggle
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat: mask sensitive fields in YAML view with show/hide toggle
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat: hide internal settings and mask sensitive fields in YAML view
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat: hide jwt_secret and min_keep_alive_version from API and config exports
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* all
* feat: add secretKeyRef support for sensitive fields in operator CRD
Allow sensitive fields (license_key, hub_api_secret, scim_token,
smtp_password, OAuthClient.secret, custom PG user_pwd) to reference
Kubernetes Secrets via the standard secretKeyRef pattern instead of
inlining values as plaintext YAML. The reconciler resolves all refs
by reading K8s Secrets before syncing to the database.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* all
* all
* all
* fix: merge main and update dev environment docs
Resolve merge conflicts from origin/main, fix duplicate
UV_INDEX_STRATEGY_SETTING import, and add Playwright MCP
testing instructions to CLAUDE.md.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* all
* fix: init tracing for CLI subcommands and deduplicate setting side-effects
Initialize tracing subscriber before early-return CLI paths (sync-config,
operator) so tracing calls are not silently dropped. Refactor
set_global_setting_internal to call run_setting_pre_write_hook instead of
duplicating the side-effect logic.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add `wmill instance get-config` CLI command
Dumps the current instance config (global settings + worker configs) as
YAML. Supports --output-file to write to a file instead of stdout.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* all
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: make aws-config and related deps optional in windmill-common
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: gate python version listing on inline_preview feature
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* check endpoint
* use flag for cred check on worker
* use sdk for auth chat side
* cleaning
* cleaning
* also handle GET
* Add session token support in BedrockClient::from_credentials
Users with temporary STS credentials need session tokens for AWS
authentication. This adds the optional session_token parameter to
support these use cases.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Cache AWS SDK config loading to avoid repeated I/O
aws_config::load_defaults() performs environment variable lookups and
potentially file I/O on each call. Cache the result in a static OnceCell
to improve performance on high-frequency Bedrock requests.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add debouncing to Bedrock credential check button
Prevent rapid clicks from spawning multiple concurrent flow preview
jobs for the worker credential check.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Remove orphaned SigV4 signing comment
The manual SigV4 signing code was removed in favor of the AWS SDK,
but this documentation comment was left behind.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Remove verbose tracing info and change-related comments
- Remove per-request auth method info logs (too verbose for production)
- Simplify from_env log to single debug-level message
- Remove comments describing future changes rather than current behavior
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* cleaning
* Fix Bedrock feature flag compilation warnings
Gate AWS-specific imports, struct fields, and methods behind the bedrock
feature flag to eliminate dead code warnings when building without the
bedrock feature enabled.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* bedrock flag
* cleaning
* Move BedrockCredentialsCheck from edit drawer to add drawer
The component was incorrectly shown when editing resources instead of
when adding new ones. Moved it from ResourceEditor.svelte to
ApiConnectForm.svelte.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* cleaning
* cleaning
* make aws-config optional
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
* data tables settings ui
* install runed
* zod 4 fixes
* use new toJSONSchema
* Migrate ducklake catalogs to more generic custom instance databases
* fix compilation
* Safety conversion for old duckdb ffi
* data tables settings
* ts client basis
* inline run works
* datatables work
* Revert "datatables work"
This reverts commit 6e1588d59e.
* datatables work (without leaking pg credentials)
* println
* separate sqlUtils.ts
* nit
* Separate custom instance db Select and Wizard components
* nit
* nit wording
* add tags to custom instance dbs
* error when trying to use ducklake as datatable or opposite
* show status in dropdown
* data table instance setup works
* sqk function for ducklake
* factorize logic
* fix temp reactivity
* Data table assetexplore
* Migrate S3 permissions to modal
* Revert "Migrate S3 permissions to modal"
This reverts commit 0631d03cb0.
* nit query -> fetch
* Custom instance setup new look
* run_language_executor separate fn
* run_inline param
* nit wording
* Better typed client
* Data tables display as assets in frontend
* asset db icon
* nit
* cleaner errors
* nit
* Fix sed calls in mac
* run_inline_script_preview in python client
* basic python datatable client
* datatable and datalake parser in python
* ducklake client python
* nit fix
* Fix migration producing NULL instead of {} when no custom databases
* merge conflict fail
* python ducklake client arg fix
* parse or infer sql types in ts client
* ts asset parser, detect datatable & ducklake R/W
* fix sql repl for other read ops than select
* export type SqlTemplateFunction
* rename list_custom_instance_pg_databases
* typecheck datatable and ducklake name in Typescript
* Fix typecheck datatable and ducklake in TS
* declare module overriding instead of extending
* infer_sql_type in python client
* SqlQuery object in python
* fix merge conflicts
* update const_format
* CI fix
* factor out to var_identifiers
* sqlx prepare
* unnecessary security (admin is required)
* clearer comment
* ee repo ref
* nit snake case
* claude step 1: detect var declarations
* move detect_sql_access_type to common mod
* claude step 2: detect when saved vars are queried
* Revert "claude step 2: detect when saved vars are queried"
This reverts commit 1e1f930568.
* Revert "claude step 1: detect var declarations"
This reverts commit f866f4819d.
* remove ducklake/datatable and default
* detect data table assigns in var_identifiers
* Python parser successfully infers R/W/RW from ducklake / datatable
* still register ducklake/datatable if not used as unknown R/W
* Go to settings button in Assets Dropdown on not found
* nit
* sqlx prepare fail
* manual fix, somehow sqlx prepare won't do it
* fix frontend ci
* ee repo ref
* ducklake_user doesnt exist in unit tests
* nit fix
* ui nit
* nit
* nit missing clone
* fork ducklakes and datatables
* fix surface hover bug
* stupid mistake
* better deeply reactive mutable derived
* Ducklake picker
* Editor bar data tables
* DuckDB supports datatables
* datatable in duckdb asset parser
* duckdb asset parser var_identifiers
* Revert "duckdb asset parser var_identifiers"
This reverts commit 88068b1a77.
* sqlx prepare
* Box pin in test_workflow_as_code to fix stack overflow
* go to settings button
* ee repo ref
* fix compilation
* wording nit
* backend draft
* fix for tool and streaming
* do frontend side
* working
* working tools
* rm
* handle list endpoint
* handle for ai agents
* fix for models requiring inference id
* cleaning
* fix desc issue
* fix tool usage
* fix structured output
* cleaning
* fix for api
* rm
* fix input images
* cleaning
* chore: use aws sdk (#7156)
* feat(ai): Add AWS SDK dependencies for Bedrock integration
- Add aws-sdk-bedrockruntime v1.113.0
- Add aws-credential-types for bearer token authentication
- Update rustls to v0.23.35 for compatibility
- Dependencies added to windmill-common for AI features
* feat(ai): Add bearer token provider for Bedrock authentication
- Implement BearerTokenProvider using aws_credential_types
- Simple token-based auth using API keys from Windmill resources
- Add basic unit tests for provider creation
- Export bedrock_auth module in lib.rs
* feat(ai): Add Bedrock client wrapper with region extraction
- Implement BedrockClient wrapper around AWS SDK client
- Bearer token authentication integration
- Extract AWS region from Bedrock base URL automatically
- Comprehensive unit tests for region extraction
- Make aws-config non-optional dependency for AI features
- Update feature flags to reflect new dependency structure
* cargo
* feat(ai): Implement non-streaming Bedrock via AWS SDK
Use official AWS SDK instead of manual HTTP requests for better type safety and maintainability. Implements the Bedrock converse() API for non-streaming requests with proper bearer token authentication and message format conversion between OpenAI and Bedrock formats.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* refactor(ai): Eliminate Simple* conversion types for Bedrock SDK
- Move AI types to windmill-common/src/ai_types.rs for shared access
- Update bedrock_converters to work directly with OpenAI types
- Remove ~200 lines of conversion boilerplate from ai_executor.rs and bedrock.rs
- Remove unused imports to clean compilation warnings
- Benefits: 50% fewer conversion steps, no information loss, easier maintenance
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(ai): Add streaming support for AWS Bedrock SDK
- Implement converse_stream() for Bedrock streaming responses
- Use EventReceiver.recv() to process stream events
- Extract text deltas using bedrock_stream_event_to_text()
- Send TokenDelta events to StreamEventProcessor for real-time updates
- Refactor request building to eliminate duplication between streaming and non-streaming
- Clean, minimal implementation following AWS SDK patterns
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* revert flake change
* fix
* feat(ai): Add tool calls and image support for Bedrock streaming
**Phase 1: Streaming Tool Call Support**
- Add stream event processing functions in bedrock_converters.rs:
- bedrock_stream_event_to_tool_start() - Extract tool use start from ContentBlockStart
- bedrock_stream_event_to_tool_delta() - Extract tool input deltas from ContentBlockDelta
- bedrock_stream_event_is_block_stop() - Detect ContentBlockStop events
- streaming_tool_calls_to_openai() - Convert accumulated tool calls to OpenAI format
- Update ai_executor.rs streaming loop with tool call accumulator (HashMap)
- Track current tool use ID during streaming
- Send ToolCallArguments events to StreamEventProcessor
- Return accumulated tool calls instead of empty vector
**Phase 2: Image Input Support**
- Add parse_image_data_url() to extract format and base64 data from data URLs
- Add content_part_to_block() to convert ContentPart to Bedrock ContentBlock
- Refactor convert_message() to handle multi-part content with images
- Support ImageUrl conversion to Bedrock ImageBlock with proper format (png/jpeg/gif/webp)
- Import AWS SDK image types: ImageBlock, ImageSource, ImageFormat
- Keep content_to_text() helper for system message text extraction
**Benefits**:
- ✅ Tool calling now works in both streaming and non-streaming modes
- ✅ Images are properly converted instead of being silently dropped
- ✅ Structured output works in streaming (uses tool calling)
- ✅ Full feature parity with manual HTTP implementation
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* cleaning
* fix(ai): Add S3 image support and structured output for Bedrock
**Fixes:**
1. **S3 Image Support**: Call prepare_messages_for_api() before Bedrock SDK path to convert S3Objects to ImageUrls
- Downloads images from S3 and encodes as base64 data URLs
- Ensures images are properly handled in both streaming and non-streaming modes
2. **Structured Output**: Add ToolChoice::Any when structured output tool is present
- Forces Bedrock to call the structured_output tool
- Ensures JSON schema compliance for structured output
- Works in both streaming and non-streaming modes
**Changes:**
- ai_executor.rs: Call prepare_messages_for_api() for Bedrock SDK path
- ai_executor.rs: Set tool_choice to Any when structured_output_tool_name is present
- aws_bedrock.rs: Remove unused ToolChoice imports (used via full path in worker)
**Testing:**
- ✅ S3 images are now downloaded and converted before API call
- ✅ Structured output now forces tool usage with ToolChoice::Any
- ✅ Both work in streaming and non-streaming modes
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* cleaning
* cleaning
* cleaning
* better error
* cleaning
* cleaning
* rm
* rename
* apply region
---------
Co-authored-by: Claude <noreply@anthropic.com>
* fix default
* no panic
* no print
* use utils file
* cleaning
---------
Co-authored-by: Claude <noreply@anthropic.com>
* draft mcp client
* testing
* fix
* cleaning
* mcp resource in inputtransforms
* cleaning
* big cleaning
* cleaning
* no arc
* add utils file
* refactor tools
* add mcp actions
* draft frontend
* send arguments from backend
* better frontend
* cleaning
* use token for auth
* add logo
* rm
* fix
* fix
* chore: refactor mcp for ai agents (#6829)
* Add Tool enum for AIAgent with backward compatibility
- Created Tool enum that can be either Windmill (FlowModule) or Mcp (resource reference)
- Created McpToolRef struct to hold MCP resource path
- Implemented custom Deserialize for Tool with backward compatibility:
- New format: {type: 'windmill'|'mcp', ...}
- Old format: FlowModule objects (automatically wrapped in Tool::Windmill)
- Updated AIAgent to use Vec<Tool> instead of Vec<FlowModule>
- Updated FlowValue::traverse_leafs to handle Tool enum
- Backward compatible: old flows with Vec<FlowModule> will deserialize correctly
* Refactor AI executor to process Tool enum instead of extracting MCP from input_transforms
- Separate Windmill tools and MCP resource paths from tools list
- Process Windmill FlowModules into Tool definitions
- Load MCP tools from resource paths in Tool::Mcp variants
- Remove old logic that extracted mcp_resources from input_transforms
- Import FlowModule, remove unused InputTransform
- Fix type issues: use .as_str() for path and handle Option<bool> properly
* handle in args
* mcp as flowmodule
* frontend
* config for mcp
* simplify logic
* fix ai executor logic
* cleaning
* clean frontend
* fix
* better resource picker
* fix and styling
* add endpoint to fetch tools
* apply tool filtering
* fix name validation
* better ui
* use cache
* fix
* fix merge
* refactor: Separate MCP tools from FlowModule in AIAgent
- Add new AgentTool, ToolValue, and McpToolValue types
- Update AIAgent to use Vec<AgentTool> instead of Vec<FlowModule>
- Implement From traits for clean conversion between AgentTool and FlowModule
- Add backward compatibility via custom deserializer for AgentTool
- Simplify resolve_module logic by reusing existing resolve_modules function
- Update traverse_leafs to handle AgentTool structure
This refactoring separates MCP tools from FlowModule tools, making the
type system clearer and eliminating the need to treat MCP servers as
a special case of FlowModule.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* refactor: Update ai_executor and worker_lockfiles for AgentTool
- Update ai_executor.rs to handle new AgentTool structure
- Separate MCP tools from FlowModule tools using ToolValue enum
- Convert AgentTool to FlowModule for backward compatibility
- Add imports for AgentTool and ToolValue types
- Update worker_lockfiles.rs for lazy loading optimization
- Convert AgentTool <-> FlowModule in insert_flow_modules
- Preserve lazy loading for FlowModule tools via modules_node
- Keep MCP tools inline (lightweight, no need for lazy loading)
- Maintain backward compatibility with existing flows
This enables the lazy loading optimization for FlowModule tools while
keeping MCP tools inline, balancing performance and simplicity.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* cleaning
* adapt frontend
* cleaning
* cleaning
* type fix
* cleaning
* fix back comp
* move mcp button position
* nit
* cleaning
* fix nested removal
* cleaning
* opti
* fix chat markdown display
* fix chat messages layout
* fix back comp frontend
* fix deserializer
* nit
* simpler serializer
* use if else
---------
Co-authored-by: Claude <noreply@anthropic.com>
* s3 proxy works with get (no auth yet)
* nit
* support s3:// syntax
* Support s3:// syntax and fix vite api proxy normalizing double slashes in URI
* s3 checks authed
* nit
* PUT works
* delete file works
* Derive the JWT signature from the backend
* Authorize s3 correctly (JWT signature is never sent in cleartext)
* convert object store error to wmill error for correct status code
* stash
* fix
* POST first request proxy works
* s3 put for duckdb
* factor out direct proxy code
* Fix Issue with backend proxy and wrong signature due to Host header mismatch
* Add _default_ syntax to solve URI normalization issues with signing
* restricted to user paths toggle
* user path restriction works !
* change restriction to allow
* fix
* factor out code
* better permissions UX in object storage settings
* Revert to restrict_to_user_paths
* check permissions in old s3 api
* DuckDB now uses S3 Proxy and no longer needs LFS query
* implement todo
* fix hardcoded w_id
* s3 proxy size limit
* s3_proxy is ee
* nit
* add Google Cloud Storage as option to secondary storage
* GCS secret in duckdb
* fix toolchain compile
* Remove user permissions for v0
* fix ci 2
* fix CI OSS
* fix missing feature flag
* fix unused warning
* integration test fails bc rustc 1.85.0
* ee ref
* fix ci ...
* update ee ref
* upgrade duckdb
* basic ducklake works
* ducklake works with custom db catalogs
* fix: pwsh skip already installed modules outside of cache (#6037)
* improve query performance of user stats
* separate ducklake_catalog db
* ducklake settings
* DucklakeSettings frontend
* Ducklake ws settings saved in backend
* fetch ducklake catalog resource
* Ducklake works with configured s3 storage
* Ducklake as asset
* ducklake asset icon
* Fix duckdb array and object args not working properly (#6254)
* Fix bug with comments in duckdb
* Avoid multiple queries when doing ATTACH ducklake
* trunc sig no longer needed now that comments are trimmed
* cache DuckdbConnectionSettingsResponse
* duplicated code
* transform_attach_ducklake contributes to duckdb_connection_settings_cache
* eliminate the need for used_storages
* nit
* cleaner management of the bigquery credentials file
* DBManagerDrawer refactor to prepare for Ducklake
* get ducklake schema
* implement delete for ducklake
* load column metadata for ducklake
* Select query works for ducklake, basic db explorer works !
* duckdb count query
* Support all db ops for ducklake
* clean migrations
* SQL repl for Ducklake
* fix broken database studio
* nit
* assert function
* Ducklake in Editor Bar
* default ducklake syntax + allow extra args
* DucklakeCatalogWizard UI
* nit + remove extra $
* modal when databases do not exist
* cannot be windmill
* Ducklake works safely with instance database
* Avoid sending instance db credentials on network
* resource leak security
* remove fetch_attach_db_conn_str
* prevent instance pg password leak
* hide asset usage count when not available
* case unsensitivity duckdb
* warnings
* disable instance catalog
* use shorthand syntax when inserting with EditorBar
* Instance ducklake catalog is now safe to use
* use safer argon2 pwd
* update package json parsers
* update package json
* better msgs
* tooltips
* disable explore button until saved
* nit
* fix warnings
* better ducklake_user password management
* nit
* Sanitize passwords from errors in ducklake
* DisplayResult broken in job result
* remove superadmin requirement to check databases_exist
* duckdb_connection_settings_v2_inner
* Ducklake works on agent worker (finally)
* ci
* #[allow(dead_code)]
* fix openapi missing response
* Separate +Database button for DuckDB in EditorBar
* Fix dropdown in ducklake settings
* Attempt to fix migration race condition in CI
* update sqlx failing for some offline queries
* avoid temp password for ducklake_user
* nits
* ducklake settings nits
* update duckdb default script
* fix sql repl resetting text on refresh
* avoid pgcrypto extension
---------
Co-authored-by: HugoCasa <hugo@casademont.ch>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
* assets migration
* parse assets (duckdb)
* iterate on assets
* S3 object Preview
* remove pagination
* filterText
* better occurence list
* tweak
* assets in JobPreview
* clone impl
* AssetsDetectedBadge
* improve DbManagerButton + asset dropdown button
* edit resource btn
* warning when incorrect resource
* +Resource in DuckDB
* +S3 Object editor bar
* nit fix rename
* flow asset badge
* More Generic OnChange
* Highlight assets used in modules
* Show occurence count in flow
* Better UX, avoid moving parts
* nit
* Asset nodes
* move to dedicated Asset ctx
* fix layoutNodes not handling first assetsMap
* explore asset btn in flow asset node
* correct offset
* single computeAssetNodes function
* Fix y positioning of nodes with assets
* resource editor
* write mode node (ui)
* accessType in ctx + fix insert button positioning
* right positioning when mixing read and write nodes
* right positioning when mixing R and W assets
* Better layout fix algorithm
* listAssetsByUsage and asset nodes on transitive usages
* refactor + remove linkAssets
* Refactor to allow for custom R/W modes
* AssetsDropdownButton in flow script editor
* R/W/RW selection and changes node pos in flow
* layoutNodes doesnt need recompute now
* fix wrong assumption that nodes recompute when assets change
* r/w/rw multi toggle
* MultiToggle cool animation + clearable
* rename + 1px nit
* remove mini toggle button group, use ToggleButtonGroup
* Combinator parser that detects R / W asset context
* nit fix missing flex-1
* missing order by
* better ui indication for access type
* special x offset case when only one asset node for clarity
* parse getResource in TS with swc ecma parser
* support load and write s3 detection in TS
* Python asset parser
* support wmill api calls without special $res: or s3:// syntax
* detect out of context asset uris python
* do not use access type override when not ambiguous in flow graph
* parse_assets match case in rust
* AsRef<str> refactor
* From impl
* Save flow assets
* Save script asset usages + fixes + save fallback access types
* asset sub icon
* max total asset node width to avoid overlap
* small refactor
* don't parse comments in duckdb assets
* fix assets clearing on parse error
* fix script asset save in wrong place
* load initial asset fallback access types
* support variables
* ui fixes
* Support S3Object as URI in TS client
* support new syntax in python client
* Support +S3Object in EditorBar for TS and python
* Reduce resource requests in assets page
* import windmill client when necessary
* update s3Types.d.ts
* nit fix
* Show input resources and s3 objects as assets
* improve asset icons
* DarkModeObserver refactor
* asset page tabs
* Moved resource variables and s3object pages to assets tabs
* fetch resource usages
* Get variables usages
* move assets usage dropdown to component
* Revert "move assets usage dropdown to component"
This reverts commit 622ea4ab12.
* Revert "Get variables usages"
This reverts commit b11ced4e29.
* Revert "fetch resource usages"
This reverts commit aa5187ad4b.
* Revert "Moved resource variables and s3object pages to assets tabs"
This reverts commit 4430487be4.
* Revert "asset page tabs"
This reverts commit dacc2f0da5.
* move assets usage dropdown to component
* asset icon in asset pages
* tooltip
* details
* Storage selector in S3 File Picker
* make edge less opaque
* Refactor computeAssetNodes to separate in and out nodes
* AssetsOverflowedNode
* nits
* fix assets not being parsed in flows sometimes
* show asset kind and resource_type
* ui nits
* support res:// in duckdb
* add banner for old deployments
* Fix permissionning
* fix broken disable /enable all
* assets page view permission for operators
* Disable ExploreAssetButton for operators
* asset kind as subtitle
* do not spam getResource in assets page. prob. revert fail
* update assets page on workspace change
* reload storage names on ws change
* delete assets on archive / deletion
* sqlx prepare
* missing update when updating user
* add indexes on asset
* better message
* missing loadInit: false
* dead code
* use transaction
* typo
* update package.json
* update package.json
---------
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
* app compiles with every ee substituted
* Replace all oss files content
* Revert "Replace all oss files content"
This reverts commit ea4017d59f.
* delete all ee
* hide all _ee files under private flag
* hide every oss stuff when private flag set
* pub use *
* gitignore and substitute script
* pub mod for ee needed for ee repo
* small mistakes
* remove oidc_oss impl
* ee ref (temp)
* ee ref
* fix --all-features selecting private in OSS CI
* ee repo ref
* allow unused
* backend
* iterate
* all
* all
* all
* iterate
* revert
* all
* add tracing to get of authed client
* all
* all
* lal
* all
* update
* fix
* push
* all
* all
* revert
* frontend
* fix checks
* avoid deadlock
* safer
* fix
* fix
* add tracing to get of authed client
* internal: Trigger claude when commenting with /aider (#5783)
* add claude instructions files
* call claude too when using aider
* fix
* add draft for linear claude integration
* fix: workspace preprocessor fixes
* tmp ee ref
* fix build
* update ee ref
* fix: hub script preprocessor handling
* fix build
* good ref
---------
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: centdix <40307056+centdix@users.noreply.github.com>
* stream to s3 boilerplate
* S3 works with new syntax
* snowflake s3 streaming support
* postgres s3 support
* fix postgres stream format
* mysql s3 streaming
* mssql s3 streaming
* new s3 mode syntax
* optional folder param
* rename folder to prefix
* json_stream_arr_values
* cargo toml rollback
* convert_ndjson with datafusion
* format conversion kinda works
* Fixed not finishing the datafusion writer
* support for pg and mssql
* fix file ext
* bigquery conversion and works with s3 streaming
* fix s3 flag parser
* snowflake s3 streaming support
* factor out duplicate code
* remove anyhow
* Err case for parse s3 mode
* Send error to mpsc
* bigquery s3 streaming fix for huge queries
* remove extra stuff
* snowflake s3 streaming support
* small regex mistake
* cfg(not(feature = "parquet"))
* fix CI (unused import)
* error handling fix (graphite)
* feat: critical alert if disk near full
* update logic to cover edge-case
* update logic
* windows support
* Update README.md
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* make it periodically
* add extra safety
* respect killpills
* do not check every 3 secs
* move to monitor.rs
* rework
* alter readme
* extending functionality
* fix worker_mode
* fix compilation
* fix typo
* make use of AI suggestion
* update ee-repo-ref
* update ee ref
* logs are CE
* update ee repo ref
* remove systemstat from worker crate
* fix comp error + sync cargo.lock
* more comptime fixing
* Update ee-repo-ref.txt
* fix compilation error
---------
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
* feat: add nu (nushell) support
* add worker tests
* deactivate tables and non-any types below top-level
full support will come in V1
for V0 it's better to keep things minimal and simple
* add syntax highlighting
used python's grammar, since nushell isn't supported by monaco nor svelte-highlights
for V1 nu will get it`s own grammar
* add logo
* partially implement plugin support
* change logo + ability to deploy + nsjail draft
* static variables + get_resource + get_variable
* lsp/dev.nu + initial nu lsp (not working yet)
* make it work with nsjail
* nullguard
* Much more flexible signature parsing and better error-messages
* add init script
* rename nulsp to nu
* install nu to dockerfile
* fix merge
* implement Default for MainArgSignature
* stage NU_CACHE_DIR
* improve dockerfiles
* dev.nu for parser-wasm + flake.nix
* update code for windows
* add nushell to flake
* upload Cargo.lock
* make build.sh work on nixos
* build wasm cli parsers
* add docs to README_DEV.md
* add helper script docker/dev.nu
* improve docker/dev.nu
* fix windows
* commit frontend/package(lock).json
* update cargo.lock
* correctly update cargo.lock
* remove lsp
* update flake.nix to include svelte server and nushell
* Revert base.sql to main
* remove PLUGIN_USE_RE
* make CARGO_PATH private
* add nu to cli
* Change flags to build wasm-nu-parser
* remove flake.nix from parser-wasm
* update wasm-build target
* remove unused import
* add cli support for nu
* update github workflows
* wasm-build 0.17 -> 0.19
* update build script
* update cargo.lock
* Fix typographical error
* start working on java
* do java boilerplate
* implement parser for java
* update Cargo.lock
* update ENV_SETTINGS
* use published nu parser
* update package.lock
* java is S3 + Caching enabled
* install nsjail backup
* commit v0
* fix nsjail
* v0.1
* rewrite parser in tree-sitter
* implement parser from scratch
* polishing
* change init script to match new parser
* fix imports
* fix cli build
* fix cli build
* refactor install phase
* implement .valid.windmill atomic verification
* implement java init functionality
* remove quick-xml
* fix windows not recognizing 'mvn'
* create empty settings.xml if there is no config provided
* clean up
* change default settings.xml
* change classpath format for windows
* docs to helper
* java copy bin cache instead of symlink
* remove comments
* merge
* fix package.json
* fix package.json 2
* minor fixing
* migrate to Coursier
* update misc
* Http(s) Proxy + CA certs
* remove unused .wasm
* make requirements insensitive to spaces
* update handle_child refs
* rework save_cache for directories
* fix s3 bug
* compile .wasm for cli
* remove uuid import
* fix compilation
* use reference
* fix zero-dep failure
* removing unsafe stuff
* remove unneeded imports
* revert: we still need winapi
* remove nix store from nsjail
* do not create cache_nomount
* add java to dnt
* remove duplicated dependency in init script
* fix typos
* fix CI
* use published parser
* feat(cache): remove persistent raw values from queue
* test: add tests and fix discovered regressions
* fix csharp executor after rebase
* invalidate script cache after locking
* fix lite flow version race
* fix flow dependencies lock race
* update version flag to `v.1.440`
