* feat: add auto_commit option to Kafka triggers with manual commit API
Add ability to disable auto-commit on Kafka triggers so users can
manually commit offsets after processing messages. This prevents
message loss when processing fails.
Changes:
- Add `auto_commit` column to kafka_trigger table (default true)
- Add POST /kafka_triggers/commit_offsets/{path} endpoint using
BaseConsumer with manual assign() to avoid rebalance
- Enrich trigger_info payload with partition and offset fields
- Conditionally commit based on auto_commit setting
- Add auto-commit toggle to frontend Kafka trigger config
- Add commitKafkaOffsets helpers to Python and TypeScript SDKs
- Add integration tests for auto_commit DB defaults
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: use DB-based pending commits for kafka manual offset commit
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat: pass trigger_path to all v2 preprocessors, secure commit_offsets endpoint, fix commit semantics
- Add trigger_path to v2 preprocessor event for all trigger types (kafka, nats, sqs, mqtt, gcp, postgres, websocket, http, email)
- Secure commit_offsets endpoint: infer trigger from job token (OptJobAuthed) instead of requiring trigger path parameter
- Fix auto_commit: only commit offset after successful job push
- Fix pending commits: commit offset+1 (Kafka semantics) and use CommitMode::Sync
- Update TS/Python clients and frontend preprocessor templates
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add advanced section badges and reorganize kafka trigger settings
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: remove dead wm_trigger assertions from kafka e2e test
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* sqlx
* refactor: remove unused advancedCollapsed state from all trigger editors
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* update ref
* chore: update ee-repo-ref to ed2c9d360e6fab866b9744cc79f50038d1fc7152
This commit updates the EE repository reference after PR #452 was merged in windmill-ee-private.
Previous ee-repo-ref: 5b31116a1d5a042c6a780732901cfd89584d1773
New ee-repo-ref: ed2c9d360e6fab866b9744cc79f50038d1fc7152
Automated by sync-ee-ref workflow.
* fix: use path-based auth for kafka commit_offsets endpoint
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: update ee-repo-ref to fcd3ea52b0cc94fbe1159baf662a38da947456de
This commit updates the EE repository reference after PR #457 was merged in windmill-ee-private.
Previous ee-repo-ref: b3a5c33c92cb1b2caf7a65986d71da291ff72a35
New ee-repo-ref: fcd3ea52b0cc94fbe1159baf662a38da947456de
Automated by sync-ee-ref workflow.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
* test: add E2E git sync integration tests with Gitea
Add 7 end-to-end tests that verify the full git sync pipeline:
deploy objects in Windmill → DeploymentCallback job runs hub sync script →
correct files appear in a Gitea git repository.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: auto-manage git sync script version locked to Windmill release
- Add LATEST_GIT_SYNC_SCRIPT_PATH constant as single source of truth
- Backend auto-fills empty script_path with latest on save
- New repos use empty script_path (auto-managed by backend)
- Existing repos with pinned versions show warning with opt-in button
- cache_hub_scripts always caches the latest constant
- Rename hubPaths.json gitSync entries to deprecated_ prefix
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: update ee-repo-ref.txt for git-sync-tests branch
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: update ee-repo-ref.txt
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: remove upgrade_git_sync_script_paths from save path
Empty script_path is now resolved to latest at job dispatch time in EE,
not on save. Users opt in via the UI button.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: use Option<String> for git sync script_path
None means auto-managed (uses LATEST_GIT_SYNC_SCRIPT_PATH),
Some(path) means pinned to a specific script. Resolution happens
at job dispatch time via effective_script_path().
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: separate git sync tests into dedicated CI workflow
- Remove git_sync_test from default integration test suite
- Move gitea service to dedicated docker-compose.git-sync.yml
- Add run_git_sync.sh script
- New workflow triggers on changes to git sync crate, hub paths,
ee-repo-ref, or the test files themselves
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: add Rust integration tests for git sync filtering logic
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: run git sync E2E tests via cargo run instead of docker image
Build from source and run Windmill directly, start Gitea as a
standalone container. Tests run against localhost — no pre-built
Docker image needed, works on PRs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: add relevance check before running git sync E2E tests
Only run the expensive build+test when actually relevant:
- Direct git sync file changes: always run
- ee-repo-ref.txt changed: check if EE diff touches windmill-git-sync/
- Unrelated changes to workspaces.rs or other files: skip
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: remove broad path triggers from git sync workflow
Remove workspaces.rs and wmill_integration_test_utils.py from path
triggers - they change too often for unrelated reasons. Keep only
git-sync-specific paths + ee-repo-ref.txt (filtered by check-relevance).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: rewrite git sync E2E tests with full coverage and fix test infra
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* chore: remove accidentally committed gen files
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: remove unit/integration tests for git sync filtering (covered by E2E)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: use correct build features and pass license key to test step in CI
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: add workflow_dispatch trigger to git sync test workflow
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Update commit reference in ee-repo-ref.txt
* fix: update stats_oss stubs to match EE telemetry signature changes
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: disable -D warnings for git sync e2e build step
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: allow test connection button in auto-managed git sync mode
The test connection button was disabled and runTestJob() bailed out
when script_path was unset. The test job uses a separate hub script
(gitSyncTest), not the sync script, so the guard was wrong.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: update ee-repo-ref to include auto-managed script_path fix
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: use full SHA in ee-repo-ref.txt
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address PR review feedback
- Initialize current_count before loop in wait_for_sync_jobs
- Clean up temp directories in clone helpers with addCleanup
- Fail CI startup steps if Gitea/Windmill never become ready
- Assert exact job count in exclude_path test
- Remove docs/git-sync-tests-plan.md (stale planning doc)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: remove orphaned git_sync.sql fixture
No longer referenced after Rust integration tests were removed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: assert old file removal in rename test
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: update ee-repo-ref to 612d96a66f9d0cfdae335ef3eb4881f3444ce7cd
This commit updates the EE repository reference after PR #442 was merged in windmill-ee-private.
Previous ee-repo-ref: a05004a7c82f3d1ee5f6863bb9f5a33827d30032
New ee-repo-ref: 612d96a66f9d0cfdae335ef3eb4881f3444ce7cd
Automated by sync-ee-ref workflow.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* feat: add git sync support for workspace dependencies
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: implement git sync for workspace dependencies
Signed-off-by: pyranota <pyra@duck.com>
* remove deno.lock
Signed-off-by: pyranota <pyra@duck.com>
* update ee
Signed-off-by: pyranota <pyra@duck.com>
* add tests to cli
Signed-off-by: pyranota <pyra@duck.com>
* sqlx
* chore: update ee-repo-ref to 09dfb247f6f59c61b7f2431932c4557fb26c22d8
This commit updates the EE repository reference after PR #446 was merged in windmill-ee-private.
Previous ee-repo-ref: 8a8832ae5d7efab85b3a57a740308ececa0e2aac
New ee-repo-ref: 09dfb247f6f59c61b7f2431932c4557fb26c22d8
Automated by sync-ee-ref workflow.
* fix test
* fix ci test
Signed-off-by: pyranota <pyra@duck.com>
---------
Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Add a new "fileset" resource type that represents a collection of files
stored as a relpath→content map. This enables resource types to manage
multiple files (e.g., config directories, template sets) instead of just
a single file.
Backend:
- Add is_fileset column to resource_type table
- Update CRUD operations and workspace duplication to handle is_fileset
- Add integration tests for fileset resource types
Frontend:
- Add FilesetEditor component with file explorer + Monaco editor
- Extract shared FileExplorer component from RawAppSidebar (dedup)
- Add fileset toggle to EditableSchemaWrapper
- Show fileset editor in ResourceEditor and ApiConnectForm
- Show folder icon for fileset resource types in IconedResourceType
CLI:
- Support fileset resources in sync pull (expand to .fileset/ directory)
- Support fileset resources in sync push (reconstruct from directory)
- Handle !inline_fileset YAML tag in resource resolution
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: replace leftover common:: references in dependency_map test
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: add missing deno_core/mcp features and gate dead code in permissions test
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
