Compare commits

..

29 Commits

Author SHA1 Message Date
centdix
f7961e39ad bump version 2026-02-03 17:07:13 +01:00
centdix
4f70b5c1d3 dual build for utils-internal 2026-02-03 16:55:58 +01:00
Guilhem
d47c1d31db update resource drawer to design guidelines (#7770)
* fix(frontend): update resource drawer to design guidelines

* nit
2026-02-03 13:06:18 +00:00
Diego Imbert
56c88361b8 nit frontend undefined fixes (#7772) 2026-02-03 13:05:59 +00:00
Guilhem
861b167a14 replace hard hat icon with server cog (#7771) 2026-02-03 12:26:05 +00:00
Diego Imbert
3a719cea6b fix: Fix app multiselect not refreshing result when creating element (#7766) 2026-02-03 10:02:18 +00:00
centdix
18d85f1412 fix: remove $schema field from Google AI output schema requests (#7765)
* fix: remove $schema field from Google AI output schema requests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: add $schema field to all output schema integration tests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: remove $schema field from Google AI tool parameter schemas

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: add workspace script tool test for AI agents

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 01:04:36 +00:00
Diego Imbert
635a24f82c feat: runtime assets (#7656)
* Runtime assets

* Nits

* Revert "Nits"

This reverts commit 3031a2ddd1.

* detection_kinds

* don't delete runtime assets

* Show latest executions

* conditional unique idx

* nit status

* refactor

* nit refactor

* prepare sql

* Detect assets in complex JSON input objects

* false positive prevent

* nit

* redundant idx

* Update frontend/src/lib/components/assets/AssetsUsageDrawer.svelte

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* Update backend/migrations/20260122134517_runtime_assets.up.sql

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* runtime assets are inserted in a loop

* nit

* nit fix

* Don't use lazy static

* fix compilation

* nits

* missing on conflict do nothing

* add index

* Fix max n logic

* created at

* nits

* remove pagination

* sqlx prepare

* Only detect resource assets in input

* get_runtime_asset_sender()

* use global get_runtime_asset_sender to avoid prop drilling

* nit refactor : register_runtime_asset

* get job_id from token

* job as a usage kind

* fixes

* ee

* nit refactor

* merge access types when same job uses same asset multiple times

* Refactor to support wmill s3 API

* nit

* parse_wmill_sdk_sql_assets refactor

* Detect datatable and ducklake usage

* nit order by

* Join with v2_job

* better UI

* add sequential id for cursor pagination

* useInfiniteQuery

* useScrollToBottom

* sql index

* claude code stash

* migration fixes

* Infinite scroll UI

* nit

* style nit

* runtime asset created at

* Asset filters

* fix usage kind filter

* also check runnable_path for jobs when filtering

* better filters

* avoid flickering

* debounced filters

* nit

* tooltips

* fix: update AssetUsage type to match new ListAssetsResponse structure

The ListAssetsResponse changed from an array to an object with an 'assets' property.
Updated the type extraction accordingly.

Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com>

* sqlx prepare

* Delete .claude/hooks/.symlink-manifest

* unnecessary dep

* nit refactor

* nit comment

* nit naming

* CI fix attempt 1

* ee ref

* nit remove alerts

* nit

* chore: update ee-repo-ref to 138a4f5f868f3bded5bb7cb77b222b532c07e4af

This commit updates the EE repository reference after PR #395 was merged in windmill-ee-private.

Previous ee-repo-ref: 7d3a21d53066726e97dfea9f117373299bc9318c

New ee-repo-ref: 138a4f5f868f3bded5bb7cb77b222b532c07e4af

Automated by sync-ee-ref workflow.

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-02-02 10:09:40 +00:00
Ruben Fiszel
bdf9447e82 feat: default to quickjs on ce for flow eval (#7756)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 08:04:37 +00:00
Ruben Fiszel
790ead082c chore(main): release 1.623.1 (#7754)
* chore(main): release 1.623.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-02-01 21:02:32 +00:00
Ruben Fiszel
50b6c199e7 Merge branch 'fix/retention-active-flows' 2026-02-01 20:53:50 +00:00
Ruben Fiszel
799db94683 fix: prevent retention cleanup from deleting jobs of active flows (#7755)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 20:52:25 +00:00
Ruben Fiszel
4226ec8260 fix: prevent retention cleanup from deleting jobs of active flows
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 20:46:14 +00:00
hugocasa
a8523f552c fix: resolve infinite effect loop in PocketIdSetting component (#7753) 2026-02-01 16:12:10 +00:00
Ruben Fiszel
5c9b95e786 chore(main): release 1.623.0 (#7748)
* chore(main): release 1.623.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-02-01 08:05:05 +00:00
Ruben Fiszel
6e824a6289 nit frontend fix 2026-01-31 22:38:47 +00:00
Ruben Fiszel
f405dff2e2 fix: preserve script envs field during sync push
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-31 22:36:00 +00:00
Devdatta Talele
720e3c5436 feat: add PocketID OAuth provider support (#7318)
* feat(oauth): add Pocket-ID OAuth provider component

- Implements PocketIdSetting.svelte following Keycloak pattern
- Configures OIDC endpoints for Pocket-ID (/authorize, /api/oidc/*)
- Supports standard OIDC scopes (openid, profile, email)
- Uses passkey-only authentication via Pocket-ID

Refs #5678

* feat(oauth): register Pocket-ID in SSO provider list

- Import PocketIdSetting component
- Add Pocket-ID to provider list in SSO tab
- Update exclusion filter to prevent duplicate custom entries

Refs #5678

* fix(oauth): add missing PocketID icon and fix component integration

- Create PocketIdIcon.svelte component with user profile icon
- Register pocket-id in APP_TO_ICON_COMPONENT mapping
- Fix PocketIdSetting to use IconedResourceType pattern matching other OAuth providers

This resolves the issue where PocketID toggle was not appearing in SSO settings.

Refs #5678

* refactor: migrate PocketIdSetting to Svelte 5 runes syntax

- Use $props() with $bindable() for reactive prop binding
- Use $state() for local reactive state
- Use $derived() for computed values
- Use $effect() for reactive side effects
- Replace on:change with onchange event handler
- Pre-populate base URL from existing config when editing
- Clean up bracket notation to dot notation for value properties

Addresses reviewer feedback

* fix: rename pocket-id to pocketid for naming convention compliance

Change identifier from 'pocket-id' to 'pocketid' to match Windmill's naming convention.
No OAuth provider uses hyphens - all custom SSO providers (keycloak, authentik, authelia,
kanidm, zitadel) use no separator.

Changes:
- AuthSettings.svelte: oauths['pocket-id'] → oauths['pocketid'] (2 locations)
- PocketIdSetting.svelte: name={'pocket-id'} → name={'pocketid'}
- icons/index.ts: 'pocket-id': PocketIdIcon → pocketid: PocketIdIcon

Note: PocketID does not need oauth_connect.json entry as it's a custom SSO provider
with user-configured endpoints, similar to Keycloak/Authentik.

Addresses reviewer feedback

* fix: use TextInput component for consistency

---------

Co-authored-by: hugocasa <hugo@casademont.ch>
2026-01-30 17:27:44 +00:00
hugocasa
1f1ef9ee94 nit ui nextcloud triggers (#7749) 2026-01-30 14:06:53 +00:00
centdix
297aa23ed4 fix: add schema compatibility layer for MCP clients like n8n (#7747)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 13:57:59 +00:00
Ruben Fiszel
9d2785bece fix npm check 2026-01-30 08:17:58 +00:00
Ruben Fiszel
45aa9ab746 chore(main): release 1.622.0 (#7742)
* chore(main): release 1.622.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-01-29 23:49:09 +00:00
centdix
ce23f21c0e feat: add token usage tracking to AI agent output (#7738)
* feat: add token usage tracking to AI agent output

Add TokenUsage struct to track input/output/cache tokens from AI providers.
Currently implemented for Bedrock provider, with infrastructure in place
for other providers. Usage is included in the AI agent result alongside
output and messages when available.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add token usage extraction for Anthropic provider

Extract usage from message_delta SSE event and convert to TokenUsage.
Includes input_tokens, output_tokens, cache_read_input_tokens, and
cache_creation_input_tokens (mapped to cache_write_input_tokens).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add token usage extraction for Google AI/Gemini provider

Extract usage from usageMetadata in Gemini SSE events and convert to TokenUsage.
Maps promptTokenCount -> input_tokens, candidatesTokenCount -> output_tokens,
totalTokenCount -> total_tokens.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add token usage extraction for OpenAI Responses API provider

Extract usage from response.completed SSE event and convert to TokenUsage.
Maps input_tokens, output_tokens, and total_tokens directly.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add token usage extraction for Azure OpenAI / Chat Completions API

Add stream_options.include_usage to request and parse usage from final
SSE chunk for providers using the standard OpenAI Chat Completions API
(Azure OpenAI, Mistral, DeepSeek, Groq, TogetherAI, CustomAI).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: cleanup token usage tracking - remove unused Image usage field and accumulate across iterations

- Remove unused `usage` field from ParsedResponse::Image variant
- Add TokenUsage::accumulate() method to sum usage across agent iterations
- Accumulate input/output/total/cache tokens instead of replacing with last iteration

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: remove verbose debug logging from AI providers

Remove tracing::info!("[debug] ...") statements that were too verbose
for production. These logged raw events on every streaming event.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: add retry mechanism for OpenAI-compatible providers without stream_options support

Some OpenAI-compatible providers don't support the stream_options parameter
for usage tracking. This adds a retry mechanism that:
- First attempts the request with stream_options.include_usage
- If it fails with 400 and error mentions stream_options/include_usage,
  automatically retries without the parameter

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: remove unnecessary text parsing overhead in image response handlers

Revert debugging changes that read response as text before parsing JSON.
Using response.json() directly is more efficient.

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: centralize TokenUsage conversion with constructor methods

Add new(), from_input_output(), and with_cache() constructors to TokenUsage
to eliminate duplicate conversion logic across providers. Also fixes potential
truncation in Bedrock cache token conversion by using i32::try_from with
fallback to i32::MAX.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: simplify Anthropic usage extraction and add Default derive

- Use idiomatic `if let` pattern instead of `is_some()` check for usage extraction
- Add Default derive to OpenAIChatUsage for consistency with other usage structs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use saturating_add to prevent overflow in token accumulation

In long-running agents with many iterations, token counts could
potentially overflow. Using saturating_add ensures values cap at
i32::MAX instead of wrapping around.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* better claude

* nit

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 23:45:46 +00:00
Alexander Petric
ca8dbc0676 fix: forward teams error to client (#7746)
* fix: forward teams error to client

* chore: update ee-repo-ref to 9a3d71f2c6a41ed4d17111a8c05d8e1d4933898d

This commit updates the EE repository reference after PR #400 was merged in windmill-ee-private.

Previous ee-repo-ref: 25d35a8de1cd70e281dc876e51cd30402580b5c0

New ee-repo-ref: 9a3d71f2c6a41ed4d17111a8c05d8e1d4933898d

Automated by sync-ee-ref workflow.

* fix

* fix

* fix

* al

* sqlx

* sqlx

* all

* all

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-01-29 23:44:42 +00:00
Ruben Fiszel
6c84a89053 fix: require AGENT_TOKEN and BASE_INTERNAL_URL for agent mode
- Add AgentConfig struct to validate required env vars on startup
- Change build_agent_http_client to require explicit token and URL
- Remove DEFAULT_BASE_INTERNAL_URL fallback (no more silent localhost:8000)
- Exit immediately if agent cannot connect to server on initial load
- Update integration tests to use dynamic port for BASE_INTERNAL_URL

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 23:12:18 +00:00
wendrul
998f11a10d fix: visibility bug on deployment UI (issue when renaming items) + add tracking of folders and resource types (#7739)
* fix: Raw apps deployment UI (and merge UI)

* Add folders and resource tpyes to merge UI

* claude first pass on adding the new arg for h_deploy_metadata

* Add missing argument to handle_deployment_metadata in all its calls

* Add support for folders and resource types in merge UI

* Update eereporef for CI

* Update ee repo

* Add migration to reset cached diff with potential artifacts

* fix type in frontend

* Preapare sqlx

* Remove unused import and logs

* update ee-repo

* Update eerepo

* chore: update ee-repo-ref to aca38475afd2cafaf63f4bbffc65be9437d57d86

This commit updates the EE repository reference after PR #397 was merged in windmill-ee-private.

Previous ee-repo-ref: 19c64cf8c61d83f45047b37660054b29658cd403

New ee-repo-ref: aca38475afd2cafaf63f4bbffc65be9437d57d86

Automated by sync-ee-ref workflow.

* Make integration  test for workspace comparisons

* Update SQLx metadata

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-01-29 22:31:49 +00:00
centdix
6a37af09bb refactor: remove seed parameter from AI chat completions (#7745)
* better claude

* refactor: remove seed parameter from AI chat completions

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 20:35:15 +00:00
wendrul
6679ecb9a2 fix: indexer build error (#7744)
* fix: indexer build error

* prepare sqlx

* Remove changes from Cargo.toml
2026-01-29 18:03:40 +00:00
Ruben Fiszel
ad5293c0ed fix: remove uuid-ossp extension requirement for RDS compatibility
The uuid-ossp extension was created in the first migration but never
actually used - the codebase uses gen_random_uuid() which is built-in
to PostgreSQL 13+. This allows Windmill to run on AWS RDS where
application users may not have CREATE SCHEMA privileges.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 17:58:42 +00:00
189 changed files with 5166 additions and 754 deletions

View File

@@ -96,7 +96,6 @@
"enabledPlugins": {
"rust-analyzer-lsp@claude-plugins-official": true,
"typescript-lsp@claude-plugins-official": true,
"code-review@claude-plugins-official": true,
"commit-commands@claude-plugins-official": true
"code-review@claude-plugins-official": true
}
}

View File

@@ -0,0 +1,60 @@
---
name: commit
user_invocable: true
description: Create a git commit with conventional commit format. MUST use anytime you want to commit changes.
---
# Git Commit Skill
Create a focused, single-line commit following conventional commit conventions.
## Instructions
1. **Analyze changes**: Run `git status` and `git diff` to understand what was modified
2. **Stage only modified files**: Add files individually by name. NEVER use `git add -A` or `git add .`
3. **Write commit message**: Follow the conventional commit format as a single line
## Conventional Commit Format
```
<type>: <description>
```
### Types
- `feat`: New feature or capability
- `fix`: Bug fix
- `refactor`: Code change that neither fixes a bug nor adds a feature
- `docs`: Documentation only changes
- `style`: Formatting, missing semicolons, etc (no code change)
- `test`: Adding or correcting tests
- `chore`: Maintenance tasks, dependency updates, etc
- `perf`: Performance improvement
### Rules
- Message MUST be a single line (no multi-line messages)
- Description should be lowercase, imperative mood ("add" not "added")
- No period at the end
- Keep under 72 characters total
### Examples
```
feat: add token usage tracking for AI providers
fix: resolve null pointer in job executor
refactor: extract common validation logic
docs: update API endpoint documentation
chore: upgrade sqlx to 0.7
```
## Execution Steps
1. Run `git status` to see all changes
2. Run `git diff` to understand the changes in detail
3. Run `git log --oneline -5` to see recent commit style
4. Stage ONLY the modified/relevant files: `git add <file1> <file2> ...`
5. Create the commit with conventional format:
```bash
git commit -m "<type>: <description>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>"
```
6. Run `git status` to verify the commit succeeded

View File

@@ -0,0 +1,87 @@
---
name: pr
user_invocable: true
description: Open a draft pull request on GitHub. MUST use when you want to create/open a PR.
---
# Pull Request Skill
Create a draft pull request with a clear title and explicit description of changes.
## Instructions
1. **Analyze branch changes**: Understand all commits since diverging from main
2. **Push to remote**: Ensure all commits are pushed
3. **Create draft PR**: Always open as draft for review before merging
## PR Title Format
Follow conventional commit format for the PR title:
```
<type>: <description>
```
### Types
- `feat`: New feature or capability
- `fix`: Bug fix
- `refactor`: Code restructuring
- `docs`: Documentation changes
- `chore`: Maintenance tasks
- `perf`: Performance improvements
### Title Rules
- Keep under 70 characters
- Use lowercase, imperative mood
- No period at the end
## PR Body Format
The body MUST be explicit about what changed. Structure:
```markdown
## Summary
<Clear description of what this PR does and why>
## Changes
- <Specific change 1>
- <Specific change 2>
- <Specific change 3>
## Test plan
- [ ] <How to verify change 1>
- [ ] <How to verify change 2>
---
Generated with [Claude Code](https://claude.com/claude-code)
```
## Execution Steps
1. Run `git status` to check for uncommitted changes
2. Run `git log main..HEAD --oneline` to see all commits in this branch
3. Run `git diff main...HEAD` to see the full diff against main
4. Check if remote branch exists and is up to date:
```bash
git rev-parse --abbrev-ref --symbolic-full-name @{u} 2>/dev/null || echo "no upstream"
```
5. Push to remote if needed: `git push -u origin HEAD`
6. Create draft PR using gh CLI:
```bash
gh pr create --draft --title "<type>: <description>" --body "$(cat <<'EOF'
## Summary
<description>
## Changes
- <change 1>
- <change 2>
## Test plan
- [ ] <test 1>
- [ ] <test 2>
---
Generated with [Claude Code](https://claude.com/claude-code)
EOF
)"
```
7. Return the PR URL to the user

View File

@@ -1,6 +1,6 @@
---
name: rust-backend
description: Rust coding guidelines for the Windmill backend. Apply when writing or modifying Rust code in the backend directory.
description: Rust coding guidelines for the Windmill backend. MUST use when writing or modifying Rust code in the backend directory.
---
# Rust Backend Coding Guidelines

View File

@@ -1,5 +1,44 @@
# Changelog
## [1.623.1](https://github.com/windmill-labs/windmill/compare/v1.623.0...v1.623.1) (2026-02-01)
### Bug Fixes
* prevent retention cleanup from deleting jobs of active flows ([4226ec8](https://github.com/windmill-labs/windmill/commit/4226ec826084eabbb9fff418ea6e67eb73e27cf0))
* prevent retention cleanup from deleting jobs of active flows ([#7755](https://github.com/windmill-labs/windmill/issues/7755)) ([799db94](https://github.com/windmill-labs/windmill/commit/799db9468395adafe43630d861dac367e5559791))
* resolve infinite effect loop in PocketIdSetting component ([#7753](https://github.com/windmill-labs/windmill/issues/7753)) ([a8523f5](https://github.com/windmill-labs/windmill/commit/a8523f552c39c4bbe3c585f97df5223903013bb2))
## [1.623.0](https://github.com/windmill-labs/windmill/compare/v1.622.0...v1.623.0) (2026-01-31)
### Features
* add PocketID OAuth provider support ([#7318](https://github.com/windmill-labs/windmill/issues/7318)) ([720e3c5](https://github.com/windmill-labs/windmill/commit/720e3c543623c2612b1af704c13d032c53368efb))
### Bug Fixes
* add schema compatibility layer for MCP clients like n8n ([#7747](https://github.com/windmill-labs/windmill/issues/7747)) ([297aa23](https://github.com/windmill-labs/windmill/commit/297aa23ed46315dfd4b034d44361a5bd8aaca884))
* preserve script envs field during sync push ([f405dff](https://github.com/windmill-labs/windmill/commit/f405dff2e22681dc8d4f3a9b7427e278c6cfb0cc))
## [1.622.0](https://github.com/windmill-labs/windmill/compare/v1.621.2...v1.622.0) (2026-01-29)
### Features
* add token usage tracking to AI agent output ([#7738](https://github.com/windmill-labs/windmill/issues/7738)) ([ce23f21](https://github.com/windmill-labs/windmill/commit/ce23f21c0e0bc6365f616ace4c45fa341741c555))
* workspace dedicated workers ([#7741](https://github.com/windmill-labs/windmill/issues/7741)) ([60858d1](https://github.com/windmill-labs/windmill/commit/60858d1e20e68b83fddcdbfc0ff34decaff5d1c5))
### Bug Fixes
* forward teams error to client ([#7746](https://github.com/windmill-labs/windmill/issues/7746)) ([ca8dbc0](https://github.com/windmill-labs/windmill/commit/ca8dbc0676dda619aff6fab7f6ff05ed773738e0))
* indexer build error ([#7744](https://github.com/windmill-labs/windmill/issues/7744)) ([6679ecb](https://github.com/windmill-labs/windmill/commit/6679ecb9a2ead08d2252a64f2a27a6d539fa23e9))
* remove uuid-ossp extension requirement for RDS compatibility ([ad5293c](https://github.com/windmill-labs/windmill/commit/ad5293c0edacfaf1431a3639ef5ea32d9bd761b0))
* require AGENT_TOKEN and BASE_INTERNAL_URL for agent mode ([6c84a89](https://github.com/windmill-labs/windmill/commit/6c84a8905382e29a4bbe0ae947eda794bc4dc566))
* visibility bug on deployment UI (issue when renaming items) + add tracking of folders and resource types ([#7739](https://github.com/windmill-labs/windmill/issues/7739)) ([998f11a](https://github.com/windmill-labs/windmill/commit/998f11a10da45c6d933d8b78ca24ed4f55a53f3b))
## [1.621.2](https://github.com/windmill-labs/windmill/compare/v1.621.1...v1.621.2) (2026-01-29)

View File

@@ -0,0 +1,37 @@
{
"db_name": "PostgreSQL",
"query": "\n DELETE FROM asset\n WHERE (workspace_id, path, kind) IN (\n SELECT workspace_id, path, kind FROM (\n SELECT a.workspace_id, a.path, a.kind, a.usage_kind, ROW_NUMBER() OVER (\n PARTITION BY a.workspace_id, a.path, a.kind\n ORDER BY a.created_at DESC\n ) as rn,\n limits.max_n\n FROM asset a\n INNER JOIN (\n SELECT * FROM UNNEST(\n $1::varchar[], \n $2::varchar[], \n $3::asset_kind[],\n $4::int[]\n ) AS t(workspace_id, path, kind, max_n)\n ) limits\n ON a.workspace_id = limits.workspace_id \n AND a.path = limits.path \n AND a.kind = limits.kind\n WHERE a.usage_kind = 'job'\n ) ranked\n WHERE rn > max_n\n )",
"describe": {
"columns": [],
"parameters": {
"Left": [
"VarcharArray",
"VarcharArray",
{
"Custom": {
"name": "asset_kind[]",
"kind": {
"Array": {
"Custom": {
"name": "asset_kind",
"kind": {
"Enum": [
"s3object",
"resource",
"variable",
"ducklake",
"datatable"
]
}
}
}
}
}
},
"Int4Array"
]
},
"nullable": []
},
"hash": "02e526146f3584cd599dec708e1be48db3b0cd1c74adbfa2e4039377daa016f0"
}

View File

@@ -46,11 +46,11 @@
]
},
"nullable": [
true,
true,
true,
true,
true,
false,
false,
false,
false,
false,
true,
true
]

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_diff\n (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)\n VALUES\n ('test-workspace', 'wm-fork-test-workspace', 'f/shared/old_name', 'resource', 0, 1, NULL),\n ('test-workspace', 'wm-fork-test-workspace', 'f/shared/new_name', 'resource', 1, 0, NULL)",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "14110b9c1b68cf29a6cbdfa737707a44bda831246736e60d63696c3227491adb"
}

View File

@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE flow SET value = $1\n WHERE workspace_id = 'test-workspace' AND path = 'f/shared/original_flow'",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Jsonb"
]
},
"nullable": []
},
"hash": "18272dde939afcd87464d74c03bc3c8ee4395919fbfa50565d8d800e3886911e"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT \n tag\n FROM \n v2_job\n WHERE \n id = $1\n ",
"query": "\n SELECT\n tag\n FROM\n v2_job\n WHERE\n id = $1\n ",
"describe": {
"columns": [
{
@@ -18,5 +18,5 @@
false
]
},
"hash": "49e2430af74ec10857e5df7f7e1ad1b53ba70bb51b0259a1f765f76db9b733ad"
"hash": "1d32bd9309bf2066399b446e8c47502a0ec72ffc07b22593311915ab5e98f80a"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_diff\n (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)\n VALUES\n ('test-workspace', 'wm-fork-test-workspace', 'f/shared/new_in_parent', 'script', 1, 0, NULL),\n ('test-workspace', 'wm-fork-test-workspace', 'new_type', 'resource_type', 1, 0, NULL)",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "1ed8d27979cd903cfd12f52461aada26f807236364fb56bf826228983bdab3ab"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE script SET archived = true\n WHERE workspace_id = 'wm-fork-test-workspace' AND path = 'f/shared/to_delete'",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "21941635afc295e5a42aceffce7aad910477c4537e9fe40b227e4ecc7eae76d3"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_diff\n (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)\n VALUES ('test-workspace', 'wm-fork-test-workspace', 'f/shared/new_in_fork', 'script', 0, 1, NULL)",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "2248e2c5ca8e6fe58704e45066477a1257b15cde7ea44962ce8566d6d7396add"
}

View File

@@ -1,23 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM v2_job_completed\n WHERE id IN (\n SELECT id FROM v2_job_completed\n WHERE completed_at <= now() - ($1::bigint::text || ' s')::interval\n ORDER BY completed_at ASC\n LIMIT $2\n FOR UPDATE SKIP LOCKED\n )\n RETURNING id",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Uuid"
}
],
"parameters": {
"Left": [
"Int8",
"Int8"
]
},
"nullable": [
false
]
},
"hash": "306e0156ee1541710c1c6512ecb4f61baeb3ae6f31ba3fd57a3ec485108a7f49"
}

View File

@@ -0,0 +1,53 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO asset (workspace_id, path, kind, usage_access_type, usage_path, usage_kind, asset_detection_kind, job_id)\n VALUES ($1, $2, $3, $4, $5, $6, 'static', NULL) ON CONFLICT DO NOTHING",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
{
"Custom": {
"name": "asset_kind",
"kind": {
"Enum": [
"s3object",
"resource",
"variable",
"ducklake",
"datatable"
]
}
}
},
{
"Custom": {
"name": "asset_access_type",
"kind": {
"Enum": [
"r",
"w",
"rw"
]
}
}
},
"Varchar",
{
"Custom": {
"name": "asset_usage_kind",
"kind": {
"Enum": [
"script",
"flow"
]
}
}
}
]
},
"nullable": []
},
"hash": "31eada57708ac3347d1e0a4fb5fd412a4a0c6046dbe4cf91849def2eea2e4c62"
}

View File

@@ -0,0 +1,32 @@
{
"db_name": "PostgreSQL",
"query": "SELECT has_changes, exists_in_source, exists_in_fork FROM workspace_diff\n WHERE path = 'f/shared/new_in_parent' AND kind = 'script' AND source_workspace_id = 'test-workspace'",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "has_changes",
"type_info": "Bool"
},
{
"ordinal": 1,
"name": "exists_in_source",
"type_info": "Bool"
},
{
"ordinal": 2,
"name": "exists_in_fork",
"type_info": "Bool"
}
],
"parameters": {
"Left": []
},
"nullable": [
true,
true,
true
]
},
"hash": "338f8f878aba9d9361b17329ee448ee4582194cbbc853d26abd285c259642c60"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE script\n SET content = 'def main(): return \"fork_modified\"', summary = 'Modified in fork'\n WHERE workspace_id = 'wm-fork-test-workspace' AND path = 'f/shared/to_modify_fork'",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "33ad6eb46a13dd49605fdb863a03602878c9d1f40f61e349e06698140133153f"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO script (workspace_id, path, hash, content, summary, description, language, created_by, created_at, archived, schema_validation, ws_error_handler_muted, deleted)\n VALUES\n ('test-workspace', 'f/shared/original_script', 12345, 'def main(): pass', 'Original', '', 'python3', 'test@windmill.dev', NOW(), false, false, false, false),\n ('test-workspace', 'f/shared/to_modify_parent', 22222, 'def main(): return 1', 'To modify in parent', '', 'python3', 'test@windmill.dev', NOW(), false, false, false, false),\n ('test-workspace', 'f/shared/to_modify_fork', 33333, 'def main(): return 2', 'To modify in fork', '', 'python3', 'test@windmill.dev', NOW(), false, false, false, false),\n ('test-workspace', 'f/shared/to_conflict', 44444, 'def main(): return 3', 'To conflict', '', 'python3', 'test@windmill.dev', NOW(), false, false, false, false),\n ('test-workspace', 'f/shared/to_delete', 55555, 'def main(): return 4', 'To delete', '', 'python3', 'test@windmill.dev', NOW(), false, false, false, false)",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "3439a44ea40ea748346af50bc1529dc8fd135c8cb54b7e39d03b326c30a0ab25"
}

View File

@@ -0,0 +1,42 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO asset (workspace_id, path, kind, usage_access_type, usage_path, usage_kind, asset_detection_kind, job_id)\n VALUES ($1, $2, $3, NULL, $4, $5, 'runtime', $6) ON CONFLICT DO NOTHING",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
{
"Custom": {
"name": "asset_kind",
"kind": {
"Enum": [
"s3object",
"resource",
"variable",
"ducklake",
"datatable"
]
}
}
},
"Varchar",
{
"Custom": {
"name": "asset_usage_kind",
"kind": {
"Enum": [
"script",
"flow"
]
}
}
},
"Uuid"
]
},
"nullable": []
},
"hash": "3c6bed058591ad5438428b8c154ba6ee737e59aa033d833843b54e9f58eef3f2"
}

View File

@@ -0,0 +1,24 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM v2_job_completed\n WHERE id IN (\n SELECT jc.id FROM v2_job_completed jc\n LEFT JOIN v2_job j ON j.id = jc.id\n WHERE jc.completed_at <= now() - ($1::bigint::text || ' s')::interval\n AND COALESCE(j.root_job, j.flow_innermost_root_job, jc.id) != ALL($3)\n ORDER BY jc.completed_at ASC\n LIMIT $2\n FOR UPDATE OF jc SKIP LOCKED\n )\n RETURNING id",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Uuid"
}
],
"parameters": {
"Left": [
"Int8",
"Int8",
"UuidArray"
]
},
"nullable": [
false
]
},
"hash": "45997fcb4d9d62c7f7011966bf59bdb86e12ce1d0c8e925e738d2645121a5c1f"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM asset WHERE workspace_id = $1 AND usage_kind = 'script' AND usage_path = (SELECT path FROM script WHERE hash = $2 AND workspace_id = $1) AND asset_detection_kind = 'static'",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Int8"
]
},
"nullable": []
},
"hash": "46395d77f3393a058e25ff4efb0b61c743c58418ba2ae1104bcfc99fae7925e4"
}

View File

@@ -0,0 +1,20 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id FROM app WHERE path = 'f/shared/dashboard' AND workspace_id = 'test-workspace'",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Int8"
}
],
"parameters": {
"Left": []
},
"nullable": [
false
]
},
"hash": "49e4bef9d5e366c179d33794472201bfca768ee5ead967e0232fa87a59b9747c"
}

View File

@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO agent_token_blacklist (token, expires_at, blacklisted_by)\n VALUES ($1, $2, $3)\n ON CONFLICT (token) DO UPDATE SET\n expires_at = EXCLUDED.expires_at,\n blacklisted_at = NOW(),\n blacklisted_by = EXCLUDED.blacklisted_by",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Timestamp",
"Varchar"
]
},
"nullable": []
},
"hash": "4c1dac4ddeca8f053abb35c833acd6993448d0327404cf1ad089ec3df6f2db35"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE script\n SET content = 'def main(): return \"modified\"', summary = 'Modified in parent'\n WHERE workspace_id = 'test-workspace' AND path = 'f/shared/to_modify_parent'",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "4ef7fcfa9a8962497c1eace31e59138cb8b252b207122a823185b36c44c10045"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_diff\n (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)\n VALUES\n ('test-workspace', 'wm-fork-test-workspace', 'f/shared/to_modify_fork', 'script', 0, 1, NULL),\n ('test-workspace', 'wm-fork-test-workspace', 'f/shared/resource_to_modify', 'resource', 0, 1, NULL),\n ('test-workspace', 'wm-fork-test-workspace', 'shared', 'folder', 0, 1, NULL)",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "56e6be86747e953e4eb75b434906bea7f471c22923ac29894feae9c199d78788"
}

View File

@@ -15,7 +15,7 @@
]
},
"nullable": [
null
true
]
},
"hash": "5a219a2532517869578c4504ff3153c43903f929ae5d62fbba12610f89c36d55"

View File

@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_diff (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)\n VALUES ($1, $2, $3, $4, 1, 0, NULL)\n ON CONFLICT (source_workspace_id, fork_workspace_id, path, kind)\n DO UPDATE SET\n ahead = workspace_diff.ahead + 1,\n has_changes = NULL",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar",
"Varchar"
]
},
"nullable": []
},
"hash": "5a3cf6f0958a559c147a37e29458b48eb7488446eb69adeddf2007ae7ae897a7"
}

View File

@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_diff (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)\n SELECT $1, unnest($2::varchar[]), $3, $4, 0, 1, NULL\n ON CONFLICT (source_workspace_id, fork_workspace_id, path, kind)\n DO UPDATE SET\n behind = workspace_diff.behind + 1,\n has_changes = NULL",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"VarcharArray",
"Varchar",
"Varchar"
]
},
"nullable": []
},
"hash": "5b3b4119469b7b110657926e6f99093c8fa5980ad061804cf73dc29151beac4b"
}

View File

@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO resource_type (workspace_id, name, schema, description, created_by)\n VALUES ('test-workspace', 'custom_db', $1, 'Custom DB type', 'test@windmill.dev')",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Jsonb"
]
},
"nullable": []
},
"hash": "6974521dec19adbc4c4497cc50e11a3f3be2a76a68287b3da2b999a925f972b4"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM asset WHERE workspace_id = $1 AND usage_kind = 'script' AND usage_path = $2 AND asset_detection_kind = 'static'",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": []
},
"hash": "752fc50d3374ee5ccdb49f10c11e50bbffeec1088364a64f0849fb9b25e11ef4"
}

View File

@@ -0,0 +1,20 @@
{
"db_name": "PostgreSQL",
"query": "SELECT has_changes FROM workspace_diff\n WHERE path = 'f/shared/original_script' AND kind = 'script' AND source_workspace_id = 'test-workspace'",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "has_changes",
"type_info": "Bool"
}
],
"parameters": {
"Left": []
},
"nullable": [
true
]
},
"hash": "77929b1ab186a01113436b5739240285dc111218efb4df8dbb0c72280a4c6a2c"
}

View File

@@ -0,0 +1,35 @@
{
"db_name": "PostgreSQL",
"query": "SELECT schema, description, format_extension\n FROM resource_type\n WHERE workspace_id = $1 AND name = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "schema",
"type_info": "Jsonb"
},
{
"ordinal": 1,
"name": "description",
"type_info": "Text"
},
{
"ordinal": 2,
"name": "format_extension",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
true,
true,
true
]
},
"hash": "7bc9fc05dbd162866bef1fdd3e7faeb50429881ed1bc962903f06e4b3d5f8d44"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT token, expires_at, blacklisted_at, blacklisted_by \n FROM agent_token_blacklist \n ORDER BY blacklisted_at DESC",
"query": "SELECT token, expires_at, blacklisted_at, blacklisted_by\n FROM agent_token_blacklist\n ORDER BY blacklisted_at DESC",
"describe": {
"columns": [
{
@@ -34,5 +34,5 @@
false
]
},
"hash": "1c5d3556fc8436ddd294f39c5431e1f501a821d6143c5d8aece20814237a6b86"
"hash": "7cbfad812eeb80cff00336697052f266693cf838d62a8b1e581c7239ec42095b"
}

View File

@@ -0,0 +1,34 @@
{
"db_name": "PostgreSQL",
"query": "SELECT\n jsonb_build_object(\n 'path', path,\n 'kind', kind,\n 'access_type', usage_access_type\n ) as \"list!: _\"\n FROM asset\n WHERE workspace_id = $1 AND usage_path = $2 AND usage_kind = $3\n AND asset_detection_kind = 'static'\n ORDER BY path, kind",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "list!: _",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Text",
"Text",
{
"Custom": {
"name": "asset_usage_kind",
"kind": {
"Enum": [
"script",
"flow"
]
}
}
}
]
},
"nullable": [
null
]
},
"hash": "7d73a51d5a83b11c1c4088678dd22ef369e9e26fafb75b351abfdef2b8a7a3e4"
}

View File

@@ -0,0 +1,41 @@
{
"db_name": "PostgreSQL",
"query": "SELECT display_name, owners, extra_perms, summary\n FROM folder\n WHERE workspace_id = $1 AND name = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "display_name",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "owners",
"type_info": "VarcharArray"
},
{
"ordinal": 2,
"name": "extra_perms",
"type_info": "Jsonb"
},
{
"ordinal": 3,
"name": "summary",
"type_info": "Text"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
false,
false,
true
]
},
"hash": "8039b459914ceefe0c0a31b97473a5522e47b4b2fd3ddeff8f221560bc9c6f57"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_diff\n (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)\n VALUES ('test-workspace', 'wm-fork-test-workspace', 'f/shared/lazy_test', 'script', 1, 0, NULL)\n ON CONFLICT DO NOTHING",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "85711709ee4c9684cd46af85c2d21124bfaa6aa07eadeb8d55b4b9b9e3073f10"
}

View File

@@ -0,0 +1,20 @@
{
"db_name": "PostgreSQL",
"query": "SELECT has_changes FROM workspace_diff\n WHERE path = 'f/shared/lazy_test' AND kind = 'script' AND source_workspace_id = 'test-workspace'",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "has_changes",
"type_info": "Bool"
}
],
"parameters": {
"Left": []
},
"nullable": [
true
]
},
"hash": "8c0596e2a1c7cf9eef4e4ce3249a8bdd6cb4ead841591d873e5ec8c9c214b6b5"
}

View File

@@ -0,0 +1,20 @@
{
"db_name": "PostgreSQL",
"query": "SELECT EXISTS(SELECT 1 FROM workspace WHERE id = 'wm-fork-test-workspace')",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "exists",
"type_info": "Bool"
}
],
"parameters": {
"Left": []
},
"nullable": [
null
]
},
"hash": "8d1920926cbdce897305c775b1d286fed8fa2f4258bde54b9d771c91923751ff"
}

View File

@@ -0,0 +1,26 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM asset WHERE workspace_id = $1 AND usage_path = $2 AND usage_kind = $3 AND asset_detection_kind = 'static'",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
{
"Custom": {
"name": "asset_usage_kind",
"kind": {
"Enum": [
"script",
"flow"
]
}
}
}
]
},
"nullable": []
},
"hash": "8ff8b53147d857db7829889ccbb59d2ca0492704a0b4a3cc8d1f4f733e0c794b"
}

View File

@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO resource_type (workspace_id, name, schema, description, created_by)\n VALUES ('test-workspace', 'new_type', $1, 'New type in parent', 'test@windmill.dev')",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Jsonb"
]
},
"nullable": []
},
"hash": "907025d53448ea70760420f90cd95ad1d93ef6d4ebe0295cdc1e70108320f6a9"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_diff\n (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)\n VALUES ('test-workspace', 'wm-fork-test-workspace', 'f/shared/original_script', 'script', 0, 0, NULL)",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "93def20ecf1bcbffe57c54bb12f0a5da22d9d2845d69e998cad8b68954b7a289"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE folder SET display_name = 'Modified Shared Folder'\n WHERE workspace_id = 'wm-fork-test-workspace' AND name = 'shared'",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "96406f7d7a42ec5a614415ed2fbf6f24291e869aa028ef3190390aaeadd10a46"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM asset WHERE workspace_id = $1 AND usage_kind = 'flow' AND usage_path = $2 AND asset_detection_kind = 'static'",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": []
},
"hash": "97101fae34227628efbcd72457a83a683745a6e2fb120329deba5117cc523bf2"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO script (workspace_id, path, hash, content, summary, description, language, created_by, created_at, archived, schema_validation, ws_error_handler_muted, deleted)\n VALUES ('wm-fork-test-workspace', 'f/shared/new_in_fork', 99999, 'def main(): return \"fork\"', 'New in fork', '', 'python3', 'test@windmill.dev', NOW(), false, false, false, false)",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "9775cbe9f8592743575b06ae60b4afdca78d46a5934521314bb7a7d8688f9a35"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO app (workspace_id, path, summary, policy, versions, extra_perms, draft_only)\n VALUES ('test-workspace', 'f/shared/dashboard', 'Dashboard app', '{}', ARRAY[1::bigint], '{}', false)",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "9a7f4786fc29ed2b561d9eb96c274c66da29fdfb3e862e06c10c5deb4a2b5771"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE resource SET path = 'f/shared/new_name'\n WHERE workspace_id = 'test-workspace' AND path = 'f/shared/old_name'",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "9b7d0f980267e62fb50545570e83daae65e4458a88b53b65870523b64d183231"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE variable SET value = 'modified_value'\n WHERE workspace_id = 'test-workspace' AND path = 'f/shared/variable_to_modify'",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "9df747dec3f3c245b1cd3a5039f0a4a69d0667e328ad838c1a2f9bb9fee2d121"
}

View File

@@ -0,0 +1,37 @@
{
"db_name": "PostgreSQL",
"query": "\n DELETE FROM asset\n WHERE id IN (\n SELECT id FROM (\n SELECT a.id, ROW_NUMBER() OVER (\n PARTITION BY a.workspace_id, a.path, a.kind\n ORDER BY a.created_at DESC\n ) as rn,\n limits.max_n\n FROM asset a\n INNER JOIN (\n SELECT * FROM UNNEST(\n $1::varchar[], \n $2::varchar[], \n $3::asset_kind[],\n $4::int[]\n ) AS t(workspace_id, path, kind, max_n)\n ) limits\n ON a.workspace_id = limits.workspace_id \n AND a.path = limits.path \n AND a.kind = limits.kind\n WHERE a.asset_detection_kind = 'runtime'\n ) ranked\n WHERE rn > max_n\n )",
"describe": {
"columns": [],
"parameters": {
"Left": [
"VarcharArray",
"VarcharArray",
{
"Custom": {
"name": "asset_kind[]",
"kind": {
"Array": {
"Custom": {
"name": "asset_kind",
"kind": {
"Enum": [
"s3object",
"resource",
"variable",
"ducklake",
"datatable"
]
}
}
}
}
}
},
"Int4Array"
]
},
"nullable": []
},
"hash": "a14d43c1baebcdf7e8cebcb7c752f78a2212fe55d4247c2321b37d182d2a4386"
}

View File

@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO resource (workspace_id, path, value, resource_type, description, created_by)\n VALUES\n ('test-workspace', 'f/shared/db_config', $1, 'postgresql', '', 'test@windmill.dev'),\n ('test-workspace', 'f/shared/old_name', $2, 'generic', '', 'test@windmill.dev'),\n ('test-workspace', 'f/shared/resource_to_modify', $3, 'generic', '', 'test@windmill.dev')",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Jsonb",
"Jsonb",
"Jsonb"
]
},
"nullable": []
},
"hash": "a17539823e5f2b0ffa8d3f270801d6d41db9e2ef4c33f09565a858d6c55cc3a9"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_diff\n (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)\n VALUES\n ('test-workspace', 'wm-fork-test-workspace', 'f/shared/original_flow', 'flow', 1, 1, NULL),\n ('test-workspace', 'wm-fork-test-workspace', 'f/shared/to_conflict', 'script', 1, 1, NULL)",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "a1f8d5c97404166603de2fe214d965243ddf8a37685a7f62b593718d195dab06"
}

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT\n jsonb_strip_nulls(jsonb_build_object(\n 'path', asset.path,\n 'kind', asset.kind,\n 'usages', ARRAY_AGG(DISTINCT jsonb_build_object(\n 'path', asset.usage_path,\n 'kind', asset.usage_kind,\n 'access_type', asset.usage_access_type,\n 'detection_kinds', (\n SELECT ARRAY_AGG(DISTINCT a2.asset_detection_kind)\n FROM asset a2\n WHERE a2.workspace_id = asset.workspace_id\n AND a2.path = asset.path\n AND a2.kind = asset.kind\n AND a2.usage_path = asset.usage_path\n AND a2.usage_kind = asset.usage_kind\n )\n )),\n 'metadata', (CASE\n WHEN asset.kind = 'resource' THEN\n jsonb_build_object('resource_type', resource.resource_type)\n ELSE\n NULL\n END\n )\n )) as \"list!: _\"\n FROM asset\n LEFT JOIN resource ON asset.kind = 'resource'\n AND array_to_string((string_to_array(asset.path, '/'))[1:3], '/') = resource.path -- With specific table, asset path can be e.g u/diego/pg_db/table_name\n AND resource.workspace_id = $1\n WHERE asset.workspace_id = $1\n AND (asset.kind <> 'resource' OR resource.path IS NOT NULL)\n AND (asset.usage_kind <> 'flow' OR asset.usage_path = ANY(SELECT path FROM flow WHERE workspace_id = $1))\n AND (asset.usage_kind <> 'script' OR asset.usage_path = ANY(SELECT path FROM script WHERE workspace_id = $1))\n GROUP BY asset.path, asset.kind, resource.resource_type\n ORDER BY asset.path, asset.kind",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "list!: _",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
null
]
},
"hash": "a21337dac33ef7a1aff1c6797a3ca47fed231f542d5bd8725d6c86b8dd1a2535"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO script (workspace_id, path, hash, content, summary, description, language, created_by, created_at, archived, schema_validation, ws_error_handler_muted, deleted)\n VALUES ('test-workspace', 'f/shared/new_in_parent', 54321, 'def main(): return \"new\"', 'New in parent', '', 'python3', 'test@windmill.dev', NOW(), false, false, false, false)",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "a962567ecfcbf251a8fcc12db894580b53f209d5f9f771bfaf0453b8d704bf31"
}

View File

@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO flow (workspace_id, path, summary, description, value, schema, edited_by, edited_at, archived)\n VALUES ('test-workspace', 'f/shared/original_flow', 'Flow summary', '', $1, NULL, 'test@windmill.dev', NOW(), false)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Jsonb"
]
},
"nullable": []
},
"hash": "b32c7c24440137d6749e1fc3095512b0f5bc8b81e0b644b01256398664ecf9b1"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT name FROM resource_type\n WHERE workspace_id = $1 AND name = ANY($2)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "name",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"TextArray"
]
},
"nullable": [
false
]
},
"hash": "b95b6ee787c590dd21969965a687fef33fc9fbc80ca87f96aced906570e28c18"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO folder (workspace_id, name, display_name, owners, summary, created_by)\n VALUES ('test-workspace', 'shared', 'Shared Folder', ARRAY['test@windmill.dev']::varchar[], 'Test folder', 'test@windmill.dev')",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "ba3149ec0aa62dba41c459a1bf5be86aebccee68172abefe82f4bb76fc7a80ee"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO app_version (app_id, value, created_by, created_at)\n VALUES ($1, $2, 'test@windmill.dev', NOW())",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Int8",
"Json"
]
},
"nullable": []
},
"hash": "bda182047ae37e263b0011248d86c1b20d8b7908bb8a8ae4c5ed83b51db5446a"
}

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT q.id FROM v2_job_queue q\n JOIN v2_job j ON j.id = q.id\n WHERE j.parent_job IS NULL\n AND j.created_at <= now() - ($1::bigint::text || ' s')::interval",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Uuid"
}
],
"parameters": {
"Left": [
"Int8"
]
},
"nullable": [
false
]
},
"hash": "c01a94ec75990c8fb4068485c91211af295a1d2630861bf53a33966abc4562ec"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO deployment_metadata (workspace_id, path, script_hash, callback_job_ids, deployment_msg) VALUES ($1, $2, $3, $4, $5) \n ON CONFLICT (workspace_id, script_hash) WHERE script_hash IS NOT NULL DO UPDATE SET callback_job_ids = EXCLUDED.callback_job_ids, deployment_msg = EXCLUDED.deployment_msg",
"query": "INSERT INTO deployment_metadata (workspace_id, path, script_hash, callback_job_ids, deployment_msg) VALUES ($1, $2, $3, $4, $5)\n ON CONFLICT (workspace_id, script_hash) WHERE script_hash IS NOT NULL DO UPDATE SET callback_job_ids = EXCLUDED.callback_job_ids, deployment_msg = EXCLUDED.deployment_msg",
"describe": {
"columns": [],
"parameters": {
@@ -14,5 +14,5 @@
},
"nullable": []
},
"hash": "8e750d4b3af9b5844c11b1b92741f2ee9d2d3412d4a2c96c6ccc87ec1c382384"
"hash": "c64288c867ba944e834a44c5a6af7231efd899a148d3607316a5537f4e7b031c"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_diff\n (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)\n VALUES\n ('test-workspace', 'wm-fork-test-workspace', 'f/shared/to_modify_parent', 'script', 1, 0, NULL),\n ('test-workspace', 'wm-fork-test-workspace', 'f/shared/dashboard', 'app', 1, 0, NULL),\n ('test-workspace', 'wm-fork-test-workspace', 'f/shared/variable_to_modify', 'variable', 1, 0, NULL)",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "c925a52d90231f9e6f3af652ddfddd67bff8c3d306794e3583af86818b7054d0"
}

View File

@@ -1,16 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO agent_token_blacklist (token, expires_at, blacklisted_by) \n VALUES ($1, $2, $3) \n ON CONFLICT (token) DO UPDATE SET \n expires_at = EXCLUDED.expires_at,\n blacklisted_at = NOW(),\n blacklisted_by = EXCLUDED.blacklisted_by",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Timestamp",
"Varchar"
]
},
"nullable": []
},
"hash": "c9c040ec228a8fe4fda08439420141bee63339d4e3d5e2d68aabb12009f691c6"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT token, expires_at, blacklisted_at, blacklisted_by \n FROM agent_token_blacklist \n WHERE expires_at > $1 \n ORDER BY blacklisted_at DESC",
"query": "SELECT token, expires_at, blacklisted_at, blacklisted_by\n FROM agent_token_blacklist\n WHERE expires_at > $1\n ORDER BY blacklisted_at DESC",
"describe": {
"columns": [
{
@@ -36,5 +36,5 @@
false
]
},
"hash": "d56722c25877222af9affd5da5bb83b28fa8cbb528a2cfc90684cb10a69e4375"
"hash": "d20d6c43b16b762fb4cdb2cafe1fe9a2920124f398fca5bd54cb805b03b94763"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE app SET summary = 'Modified dashboard app'\n WHERE workspace_id = 'test-workspace' AND path = 'f/shared/dashboard'",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "d6ce9016308da0f20c2118c047349d8110d899121c9c00795c55217b9f1886c2"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO variable (workspace_id, path, value, is_secret, description)\n VALUES\n ('test-workspace', 'f/shared/api_key', 'secret123', false, 'Test key'),\n ('test-workspace', 'f/shared/variable_to_modify', 'original', false, 'To modify')",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "d8c5c64006cdd52570bbc56b0fa5df9b597d9a45353391411efa86959a7512bc"
}

View File

@@ -0,0 +1,37 @@
{
"db_name": "PostgreSQL",
"query": "SELECT COUNT(DISTINCT asset.job_id)::bigint as \"count!\"\n FROM asset\n WHERE asset.workspace_id = $1\n AND asset.path = $2\n AND asset.kind = $3\n AND asset.asset_detection_kind = 'runtime'\n AND asset.job_id IS NOT NULL",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "count!",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text",
"Text",
{
"Custom": {
"name": "asset_kind",
"kind": {
"Enum": [
"s3object",
"resource",
"variable",
"ducklake",
"datatable"
]
}
}
}
]
},
"nullable": [
null
]
},
"hash": "df290155c071e823a1d60573a3cba932d070ed730a37450cd30ba9b90205ed3b"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT name FROM folder\n WHERE workspace_id = $1 AND name = ANY($2)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "name",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"TextArray"
]
},
"nullable": [
false
]
},
"hash": "df804e2a8795af329aa84ccfeb90329383524232276908af181f6ecb3c8fc804"
}

View File

@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE flow SET value = $1\n WHERE workspace_id = 'wm-fork-test-workspace' AND path = 'f/shared/original_flow'",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Jsonb"
]
},
"nullable": []
},
"hash": "e42bb9155be93cbc750702cf7f9a3c54d1a236421861880fe3749052bd2ab43f"
}

View File

@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_diff\n (source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)\n VALUES ('test-workspace', 'wm-fork-test-workspace', 'f/shared/to_delete', 'script', 1, 0, NULL)",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "e881a777b8e67c76cf458828a02f7d5234ae76da567195f5da11cd11d7ebbc05"
}

View File

@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE resource SET value = $1\n WHERE workspace_id = 'wm-fork-test-workspace' AND path = 'f/shared/resource_to_modify'",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Jsonb"
]
},
"nullable": []
},
"hash": "e9e6c8ba034b4f80fada2e07e500a78983aeb1a7b1185b5267229c3197d6f714"
}

View File

@@ -0,0 +1,63 @@
{
"db_name": "PostgreSQL",
"query": "SELECT DISTINCT\n v2_job.id,\n v2_job.created_at,\n v2_job.created_by,\n v2_job.runnable_path,\n CASE\n WHEN v2_job_completed.id IS NOT NULL THEN v2_job_completed.status::text\n ELSE NULL\n END as status\n FROM asset\n INNER JOIN v2_job ON asset.job_id = v2_job.id\n LEFT JOIN v2_job_completed ON v2_job.id = v2_job_completed.id\n WHERE asset.workspace_id = $1\n AND asset.path = $2\n AND asset.kind = $3\n AND asset.asset_detection_kind = 'runtime'\n AND asset.job_id IS NOT NULL\n ORDER BY v2_job.created_at DESC\n LIMIT $4 OFFSET $5",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Uuid"
},
{
"ordinal": 1,
"name": "created_at",
"type_info": "Timestamptz"
},
{
"ordinal": 2,
"name": "created_by",
"type_info": "Varchar"
},
{
"ordinal": 3,
"name": "runnable_path",
"type_info": "Varchar"
},
{
"ordinal": 4,
"name": "status",
"type_info": "Text"
}
],
"parameters": {
"Left": [
"Text",
"Text",
{
"Custom": {
"name": "asset_kind",
"kind": {
"Enum": [
"s3object",
"resource",
"variable",
"ducklake",
"datatable"
]
}
}
},
"Int8",
"Int8"
]
},
"nullable": [
false,
false,
false,
true,
null
]
},
"hash": "fb6656bc8fbd2e1b2c59f9ecbab55e5cdccdd7c99af0b61946694c6e03e7f636"
}

View File

@@ -11,10 +11,11 @@ Windmill uses a workspace-based architecture with multiple crates:
- **windmill-audit**: Audit logging
- Other specialized crates (git-sync, autoscaling, etc.)
## Key References
## Key References (MUST FOLLOW THESE)
- Database schema: @summarized_schema.txt
- API route prefixes: `windmill-api/src/lib.rs`
- You MUST follow best-practices by using the `rust-backend` skill, everytime you write RUST code.
- When working with the database: read `summarized_schema.txt` before starting
- When working with the API routes: you can read `windmill-api/src/lib.rs` to get started
## Adding New Code
@@ -57,8 +58,4 @@ Windmill uses a workspace-based architecture with multiple crates:
- **sqlx**: Database operations
- **serde**: Serialization/deserialization
- **tracing**: Logging and diagnostics
- **reqwest**: HTTP client
## Coding Guidelines
Detailed Rust coding patterns and best practices are provided by the `rust-backend` skill.
- **reqwest**: HTTP client

191
backend/Cargo.lock generated
View File

@@ -828,7 +828,7 @@ dependencies = [
"aws-sdk-ssooidc",
"aws-sdk-sts",
"aws-smithy-async",
"aws-smithy-http",
"aws-smithy-http 0.62.6",
"aws-smithy-json",
"aws-smithy-runtime",
"aws-smithy-runtime-api",
@@ -890,7 +890,7 @@ dependencies = [
"aws-sigv4",
"aws-smithy-async",
"aws-smithy-eventstream",
"aws-smithy-http",
"aws-smithy-http 0.62.6",
"aws-smithy-runtime",
"aws-smithy-runtime-api",
"aws-smithy-types",
@@ -914,7 +914,7 @@ dependencies = [
"aws-credential-types",
"aws-runtime",
"aws-smithy-async",
"aws-smithy-http",
"aws-smithy-http 0.62.6",
"aws-smithy-json",
"aws-smithy-observability",
"aws-smithy-runtime",
@@ -939,7 +939,7 @@ dependencies = [
"aws-sigv4",
"aws-smithy-async",
"aws-smithy-eventstream",
"aws-smithy-http",
"aws-smithy-http 0.62.6",
"aws-smithy-json",
"aws-smithy-observability",
"aws-smithy-runtime",
@@ -963,7 +963,7 @@ dependencies = [
"aws-credential-types",
"aws-runtime",
"aws-smithy-async",
"aws-smithy-http",
"aws-smithy-http 0.62.6",
"aws-smithy-json",
"aws-smithy-runtime",
"aws-smithy-runtime-api",
@@ -987,7 +987,7 @@ dependencies = [
"aws-runtime",
"aws-sigv4",
"aws-smithy-async",
"aws-smithy-http",
"aws-smithy-http 0.62.6",
"aws-smithy-json",
"aws-smithy-observability",
"aws-smithy-query",
@@ -1012,7 +1012,7 @@ dependencies = [
"aws-credential-types",
"aws-runtime",
"aws-smithy-async",
"aws-smithy-http",
"aws-smithy-http 0.62.6",
"aws-smithy-json",
"aws-smithy-runtime",
"aws-smithy-runtime-api",
@@ -1034,7 +1034,7 @@ dependencies = [
"aws-credential-types",
"aws-runtime",
"aws-smithy-async",
"aws-smithy-http",
"aws-smithy-http 0.62.6",
"aws-smithy-json",
"aws-smithy-runtime",
"aws-smithy-runtime-api",
@@ -1056,7 +1056,7 @@ dependencies = [
"aws-credential-types",
"aws-runtime",
"aws-smithy-async",
"aws-smithy-http",
"aws-smithy-http 0.62.6",
"aws-smithy-json",
"aws-smithy-runtime",
"aws-smithy-runtime-api",
@@ -1078,7 +1078,7 @@ dependencies = [
"aws-credential-types",
"aws-runtime",
"aws-smithy-async",
"aws-smithy-http",
"aws-smithy-http 0.62.6",
"aws-smithy-json",
"aws-smithy-query",
"aws-smithy-runtime",
@@ -1100,7 +1100,7 @@ checksum = "69e523e1c4e8e7e8ff219d732988e22bfeae8a1cafdbe6d9eca1546fa080be7c"
dependencies = [
"aws-credential-types",
"aws-smithy-eventstream",
"aws-smithy-http",
"aws-smithy-http 0.62.6",
"aws-smithy-runtime-api",
"aws-smithy-types",
"bytes",
@@ -1117,9 +1117,9 @@ dependencies = [
[[package]]
name = "aws-smithy-async"
version = "1.2.8"
version = "1.2.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9330762ee48c6cecfad2cb37b1506c16c8e858c90638eda2b1a7272b56f88bd5"
checksum = "52eec3db979d18cb807fc1070961cc51d87d069abe9ab57917769687368a8c6c"
dependencies = [
"futures-util",
"pin-project-lite",
@@ -1128,9 +1128,9 @@ dependencies = [
[[package]]
name = "aws-smithy-eventstream"
version = "0.60.15"
version = "0.60.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0810b22ae554f5076c3eabe1fe89b01aee61c354c575789f67e248e83c5f472b"
checksum = "35b9c7354a3b13c66f60fe4616d6d1969c9fd36b1b5333a5dfb3ee716b33c588"
dependencies = [
"aws-smithy-types",
"bytes",
@@ -1160,10 +1160,31 @@ dependencies = [
]
[[package]]
name = "aws-smithy-http-client"
version = "1.1.6"
name = "aws-smithy-http"
version = "0.63.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ec918f18147cec121cb142a91b0038f66d99bbe903e585dccf871920e90b22ab"
checksum = "630e67f2a31094ffa51b210ae030855cb8f3b7ee1329bdd8d085aaf61e8b97fc"
dependencies = [
"aws-smithy-runtime-api",
"aws-smithy-types",
"bytes",
"bytes-utils",
"futures-core",
"futures-util",
"http 1.4.0",
"http-body 1.0.1",
"http-body-util",
"percent-encoding",
"pin-project-lite",
"pin-utils",
"tracing",
]
[[package]]
name = "aws-smithy-http-client"
version = "1.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "12fb0abf49ff0cab20fd31ac1215ed7ce0ea92286ba09e2854b42ba5cabe7525"
dependencies = [
"aws-smithy-async",
"aws-smithy-runtime-api",
@@ -1200,18 +1221,18 @@ dependencies = [
[[package]]
name = "aws-smithy-observability"
version = "0.2.1"
version = "0.2.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a700a7702874cd78b85fecdc9f64f3f72eb22fb713791cb445bcfd2a15bc1ecf"
checksum = "c0a46543fbc94621080b3cf553eb4cbbdc41dd9780a30c4756400f0139440a1d"
dependencies = [
"aws-smithy-runtime-api",
]
[[package]]
name = "aws-smithy-query"
version = "0.60.10"
version = "0.60.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "adc4a6cdc289a37be7fddb7f4365448187d62c603a40e6d46d13c68e5e81900f"
checksum = "0cebbddb6f3a5bd81553643e9c7daf3cc3dc5b0b5f398ac668630e8a84e6fff0"
dependencies = [
"aws-smithy-types",
"urlencoding",
@@ -1219,12 +1240,12 @@ dependencies = [
[[package]]
name = "aws-smithy-runtime"
version = "1.9.8"
version = "1.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bb5b6167fcdf47399024e81ac08e795180c576a20e4d4ce67949f9a88ae37dc1"
checksum = "f3df87c14f0127a0d77eb261c3bc45d5b4833e2a1f63583ebfb728e4852134ee"
dependencies = [
"aws-smithy-async",
"aws-smithy-http",
"aws-smithy-http 0.63.3",
"aws-smithy-http-client",
"aws-smithy-observability",
"aws-smithy-runtime-api",
@@ -1235,6 +1256,7 @@ dependencies = [
"http 1.4.0",
"http-body 0.4.6",
"http-body 1.0.1",
"http-body-util",
"pin-project-lite",
"pin-utils",
"tokio",
@@ -1243,9 +1265,9 @@ dependencies = [
[[package]]
name = "aws-smithy-runtime-api"
version = "1.11.0"
version = "1.11.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5c47b1e62accf759b01aba295e40479d1ba8fb77c2a54f0fed861c809ca49761"
checksum = "49952c52f7eebb72ce2a754d3866cc0f87b97d2a46146b79f80f3a93fb2b3716"
dependencies = [
"aws-smithy-async",
"aws-smithy-types",
@@ -1260,9 +1282,9 @@ dependencies = [
[[package]]
name = "aws-smithy-types"
version = "1.4.0"
version = "1.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c2d447863bdec38c899e5753a48c0abcf590f3ec629e257ad5a9ef8806ad7714"
checksum = "3b3a26048eeab0ddeba4b4f9d51654c79af8c3b32357dc5f336cee85ab331c33"
dependencies = [
"base64-simd 0.8.0",
"bytes",
@@ -1286,9 +1308,9 @@ dependencies = [
[[package]]
name = "aws-smithy-types-convert"
version = "0.60.11"
version = "0.60.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b70bc27e41d5ed80b376602ff4becdab6ea8489403fad3abbfea2c9c825c1e1e"
checksum = "059deaa8583331f9f610b44c7cbc005d0cccec6dec3a7b387de096dbe6c06b8a"
dependencies = [
"aws-smithy-types",
"chrono",
@@ -1947,7 +1969,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8c6d47a4e2961fb8721bcfc54feae6455f2f64e7054f9bc67e875f0e77f4c58d"
dependencies = [
"rust_decimal",
"schemars 1.2.0",
"schemars 1.2.1",
"serde",
"utf8-width",
]
@@ -1976,9 +1998,9 @@ dependencies = [
[[package]]
name = "bytemuck"
version = "1.24.0"
version = "1.25.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1fbdf580320f38b612e485521afda1ee26d10cc9884efaaa750d383e13e3c5f4"
checksum = "c8efb64bd706a16a1bdde310ae86b351e4d21550d98d056f22f8a7f7a2183fec"
dependencies = [
"bytemuck_derive",
]
@@ -2150,9 +2172,9 @@ dependencies = [
[[package]]
name = "cc"
version = "1.2.54"
version = "1.2.55"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6354c81bbfd62d9cfa9cb3c773c2b7b2a3a482d569de977fd0e961f6e7c00583"
checksum = "47b26a0954ae34af09b50f0de26458fa95369a0d478d8236d3f93082b219bd29"
dependencies = [
"find-msvc-tools",
"jobserver",
@@ -5566,9 +5588,9 @@ dependencies = [
[[package]]
name = "find-msvc-tools"
version = "0.1.8"
version = "0.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8591b0bcc8a98a64310a2fae1bb3e9b8564dd10e381e6e28010fde8e8e8568db"
checksum = "5baebc0774151f905a1a2cc41989300b1e6fbb29aff0ceffa1064fdd3088d582"
[[package]]
name = "fixedbitset"
@@ -10054,9 +10076,9 @@ dependencies = [
[[package]]
name = "portable-atomic"
version = "1.13.0"
version = "1.13.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f89776e4d69bb58bc6993e99ffa1d11f228b839984854c7daeb5d37f87cbe950"
checksum = "c33a9471896f1c69cecef8d20cbe2f7accd12527ce60845ff44c153bb2a21b49"
[[package]]
name = "postgres-native-tls"
@@ -11162,7 +11184,7 @@ dependencies = [
"rand 0.9.0",
"reqwest 0.12.28",
"rmcp-macros",
"schemars 1.2.0",
"schemars 1.2.1",
"serde",
"serde_json",
"sse-stream",
@@ -11818,14 +11840,14 @@ dependencies = [
[[package]]
name = "schemars"
version = "1.2.0"
version = "1.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "54e910108742c57a770f492731f99be216a52fadd361b06c8fb59d74ccc267d2"
checksum = "a2b42f36aa1cd011945615b92222f6bf73c599a102a300334cd7f8dbeec726cc"
dependencies = [
"chrono",
"dyn-clone",
"ref-cast",
"schemars_derive 1.2.0",
"schemars_derive 1.2.1",
"serde",
"serde_json",
]
@@ -11844,9 +11866,9 @@ dependencies = [
[[package]]
name = "schemars_derive"
version = "1.2.0"
version = "1.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4908ad288c5035a8eb12cfdf0d49270def0a268ee162b75eeee0f85d155a7c45"
checksum = "7d115b50f4aaeea07e79c1912f645c7513d81715d0420f8bc77a18c6260b307f"
dependencies = [
"proc-macro2",
"quote",
@@ -12153,7 +12175,7 @@ dependencies = [
"indexmap 1.9.3",
"indexmap 2.11.1",
"schemars 0.9.0",
"schemars 1.2.0",
"schemars 1.2.1",
"serde",
"serde_derive",
"serde_json",
@@ -12399,9 +12421,9 @@ dependencies = [
[[package]]
name = "slab"
version = "0.4.11"
version = "0.4.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7a2ae44ef20feb57a68b23d846850f861394c2e02dc425a50098ae8c90267589"
checksum = "0c790de23124f9ab44544d7ac05d60440adc586479ce501c1d6d7da3cd8c9cf5"
[[package]]
name = "slotmap"
@@ -14537,9 +14559,9 @@ dependencies = [
[[package]]
name = "tree-sitter-language"
version = "0.1.6"
version = "0.1.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4ae62f7eae5eb549c71b76658648b72cc6111f2d87d24a1e31fa907f4943e3ce"
checksum = "009994f150cc0cd50ff54917d5bc8bffe8cad10ca10d81c34da2ec421ae61782"
[[package]]
name = "tree-sitter-ruby"
@@ -15466,7 +15488,7 @@ checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
[[package]]
name = "windmill"
version = "1.621.2"
version = "1.623.1"
dependencies = [
"anyhow",
"aws-sdk-config",
@@ -15529,7 +15551,7 @@ dependencies = [
[[package]]
name = "windmill-api"
version = "1.621.2"
version = "1.623.1"
dependencies = [
"anyhow",
"argon2",
@@ -15652,6 +15674,7 @@ dependencies = [
"windmill-parser",
"windmill-parser-py",
"windmill-parser-py-imports",
"windmill-parser-sql",
"windmill-parser-ts",
"windmill-queue",
"windmill-worker",
@@ -15659,7 +15682,7 @@ dependencies = [
[[package]]
name = "windmill-api-client"
version = "1.621.2"
version = "1.623.1"
dependencies = [
"reqwest 0.12.28",
"serde",
@@ -15669,7 +15692,7 @@ dependencies = [
[[package]]
name = "windmill-audit"
version = "1.621.2"
version = "1.623.1"
dependencies = [
"chrono",
"lazy_static",
@@ -15683,7 +15706,7 @@ dependencies = [
[[package]]
name = "windmill-autoscaling"
version = "1.621.2"
version = "1.623.1"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -15702,7 +15725,7 @@ dependencies = [
[[package]]
name = "windmill-common"
version = "1.621.2"
version = "1.623.1"
dependencies = [
"anyhow",
"async-recursion",
@@ -15798,7 +15821,7 @@ dependencies = [
[[package]]
name = "windmill-git-sync"
version = "1.621.2"
version = "1.623.1"
dependencies = [
"regex",
"serde",
@@ -15813,7 +15836,7 @@ dependencies = [
[[package]]
name = "windmill-indexer"
version = "1.621.2"
version = "1.623.1"
dependencies = [
"anyhow",
"astral-tokio-tar",
@@ -15837,7 +15860,7 @@ dependencies = [
[[package]]
name = "windmill-macros"
version = "1.621.2"
version = "1.623.1"
dependencies = [
"itertools 0.14.0",
"lazy_static",
@@ -15853,7 +15876,7 @@ dependencies = [
[[package]]
name = "windmill-mcp"
version = "1.621.2"
version = "1.623.1"
dependencies = [
"anyhow",
"async-trait",
@@ -15873,7 +15896,7 @@ dependencies = [
[[package]]
name = "windmill-oauth"
version = "1.621.2"
version = "1.623.1"
dependencies = [
"anyhow",
"async-oauth2",
@@ -15897,7 +15920,7 @@ dependencies = [
[[package]]
name = "windmill-parser"
version = "1.621.2"
version = "1.623.1"
dependencies = [
"convert_case 0.6.0",
"serde",
@@ -15906,7 +15929,7 @@ dependencies = [
[[package]]
name = "windmill-parser-bash"
version = "1.621.2"
version = "1.623.1"
dependencies = [
"anyhow",
"lazy_static",
@@ -15918,7 +15941,7 @@ dependencies = [
[[package]]
name = "windmill-parser-csharp"
version = "1.621.2"
version = "1.623.1"
dependencies = [
"anyhow",
"serde_json",
@@ -15930,7 +15953,7 @@ dependencies = [
[[package]]
name = "windmill-parser-go"
version = "1.621.2"
version = "1.623.1"
dependencies = [
"anyhow",
"gosyn",
@@ -15942,7 +15965,7 @@ dependencies = [
[[package]]
name = "windmill-parser-graphql"
version = "1.621.2"
version = "1.623.1"
dependencies = [
"anyhow",
"lazy_static",
@@ -15954,7 +15977,7 @@ dependencies = [
[[package]]
name = "windmill-parser-java"
version = "1.621.2"
version = "1.623.1"
dependencies = [
"anyhow",
"serde_json",
@@ -15966,7 +15989,7 @@ dependencies = [
[[package]]
name = "windmill-parser-nu"
version = "1.621.2"
version = "1.623.1"
dependencies = [
"anyhow",
"nu-parser",
@@ -15977,7 +16000,7 @@ dependencies = [
[[package]]
name = "windmill-parser-php"
version = "1.621.2"
version = "1.623.1"
dependencies = [
"anyhow",
"itertools 0.14.0",
@@ -15988,7 +16011,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py"
version = "1.621.2"
version = "1.623.1"
dependencies = [
"anyhow",
"itertools 0.14.0",
@@ -16001,7 +16024,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py-imports"
version = "1.621.2"
version = "1.623.1"
dependencies = [
"anyhow",
"async-recursion",
@@ -16025,7 +16048,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ruby"
version = "1.621.2"
version = "1.623.1"
dependencies = [
"anyhow",
"lazy_static",
@@ -16039,7 +16062,7 @@ dependencies = [
[[package]]
name = "windmill-parser-rust"
version = "1.621.2"
version = "1.623.1"
dependencies = [
"anyhow",
"convert_case 0.6.0",
@@ -16056,7 +16079,7 @@ dependencies = [
[[package]]
name = "windmill-parser-sql"
version = "1.621.2"
version = "1.623.1"
dependencies = [
"anyhow",
"lazy_static",
@@ -16070,7 +16093,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ts"
version = "1.621.2"
version = "1.623.1"
dependencies = [
"anyhow",
"lazy_static",
@@ -16089,7 +16112,7 @@ dependencies = [
[[package]]
name = "windmill-parser-yaml"
version = "1.621.2"
version = "1.623.1"
dependencies = [
"anyhow",
"serde",
@@ -16100,7 +16123,7 @@ dependencies = [
[[package]]
name = "windmill-queue"
version = "1.621.2"
version = "1.623.1"
dependencies = [
"anyhow",
"async-recursion",
@@ -16137,7 +16160,7 @@ dependencies = [
[[package]]
name = "windmill-sql-datatype-parser-wasm"
version = "1.621.2"
version = "1.623.1"
dependencies = [
"wasm-bindgen",
"wasm-bindgen-test",
@@ -16147,7 +16170,7 @@ dependencies = [
[[package]]
name = "windmill-worker"
version = "1.621.2"
version = "1.623.1"
dependencies = [
"anyhow",
"async-once-cell",
@@ -17048,18 +17071,18 @@ dependencies = [
[[package]]
name = "zerocopy"
version = "0.8.35"
version = "0.8.37"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fdea86ddd5568519879b8187e1cf04e24fce28f7fe046ceecbce472ff19a2572"
checksum = "7456cf00f0685ad319c5b1693f291a650eaf345e941d082fc4e03df8a03996ac"
dependencies = [
"zerocopy-derive",
]
[[package]]
name = "zerocopy-derive"
version = "0.8.35"
version = "0.8.37"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0c15e1b46eff7c6c91195752e0eeed8ef040e391cdece7c25376957d5f15df22"
checksum = "1328722bbf2115db7e19d69ebcc15e795719e2d66b60827c6a69a117365e37a0"
dependencies = [
"proc-macro2",
"quote",

View File

@@ -1,6 +1,6 @@
[package]
name = "windmill"
version = "1.621.2"
version = "1.623.1"
authors.workspace = true
edition.workspace = true
@@ -35,7 +35,7 @@ members = [
exclude = ["./windmill-duckdb-ffi-internal"]
[workspace.package]
version = "1.621.2"
version = "1.623.1"
authors = ["Ruben Fiszel <ruben@windmill.dev>"]
edition = "2021"

View File

@@ -1 +1 @@
a18ac31062ac092cb9a5fc87629e217d97f4911d
138a4f5f868f3bded5bb7cb77b222b532c07e4af

View File

@@ -0,0 +1,5 @@
ALTER TABLE asset
DROP COLUMN IF EXISTS created_at,
DROP COLUMN IF EXISTS id;
DELETE FROM asset WHERE usage_kind = 'job';

View File

@@ -0,0 +1,11 @@
ALTER TABLE asset
ADD COLUMN created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
ADD COLUMN id BIGSERIAL UNIQUE;
DO
$do$
BEGIN
ALTER TYPE ASSET_USAGE_KIND ADD VALUE 'job';
EXCEPTION WHEN OTHERS THEN
RAISE NOTICE 'Couldn''t create ASSET_USAGE_KIND::job: %', SQLERRM;
END
$do$;

View File

@@ -0,0 +1,2 @@
DROP INDEX IF EXISTS idx_asset_job_pruning;
DROP INDEX IF EXISTS idx_asset_workspace_created_id;

View File

@@ -0,0 +1,9 @@
-- Postgres requires indexes to be created in a separate migration (transaction) after columns are added.
-- Index for pagination queries that use workspace_id, created_at, and id for cursor pagination
-- Supports: SELECT with GROUP BY path, kind ORDER BY MAX(created_at) DESC, MAX(id) DESC
CREATE INDEX idx_asset_workspace_created_id ON asset (workspace_id, created_at DESC, id DESC);
-- Filtered index for job pruning operations that delete old job assets
-- Supports: DELETE queries with WHERE usage_kind = 'job' and window functions on (workspace_id, path, kind) ORDER BY created_at DESC
CREATE INDEX idx_asset_job_pruning ON asset (workspace_id, path, kind, created_at DESC) WHERE usage_kind = 'job';

View File

@@ -0,0 +1 @@
-- Add down migration script here

View File

@@ -0,0 +1,2 @@
-- Add up migration script here
UPDATE workspace_diff SET has_changes = NULL;

View File

@@ -205,30 +205,15 @@ impl AssetsFinder {
Some(Expr::Constant(ExprConstant { value: Constant::Str(sql), .. })) => sql,
_ => return Err(()),
};
let duckdb_conn_prefix = match kind {
AssetKind::DataTable => "datatable",
AssetKind::Ducklake => "ducklake",
_ => return Ok(()),
};
let sql = format!("ATTACH '{duckdb_conn_prefix}://{path}' AS dt; USE dt; {sql}");
// We use the SQL parser to detect if it's a read or write query
match windmill_parser_sql::parse_assets(&sql) {
Ok(mut sql_assets) => {
if let Some(schema_name) = schema {
for asset in &mut sql_assets.assets {
if asset.kind == *kind && asset.path.starts_with(path.as_str()) {
asset.path = format!(
"{}/{}.{}",
path,
schema_name,
&asset.path[path.len() + 1..]
);
}
}
}
self.assets.extend(sql_assets.assets);
}
// We use the SQL parser to detect RW, specific tables, etc.
let sql_assets = windmill_parser_sql::parse_wmill_sdk_sql_assets(
*kind,
path,
schema.as_deref(),
&sql,
);
match sql_assets {
Ok(Some(sql_assets)) => self.assets.extend(sql_assets),
_ => {}
}
return Ok(());

View File

@@ -0,0 +1,37 @@
use windmill_parser::asset_parser::{AssetKind, ParseAssetsResult};
// Parse assets from sql snippets inside e.g sql`SELECT * FROM my_table`
pub fn parse_wmill_sdk_sql_assets(
kind: AssetKind,
asset_name: &str,
schema: Option<&str>,
sql: &str,
) -> anyhow::Result<Option<Vec<ParseAssetsResult>>> {
let duckdb_conn_prefix = match kind {
AssetKind::DataTable => "datatable",
AssetKind::Ducklake => "ducklake",
_ => return Err(anyhow::anyhow!("Unsupported asset kind for SQL parsing")),
};
let sql_with_attach =
format!("ATTACH '{duckdb_conn_prefix}://{asset_name}' AS dt; USE dt; {sql}");
// We use the SQL parser to detect if it's a read or write query
match crate::parse_assets(&sql_with_attach) {
Ok(mut sql_assets) => {
if let Some(schema) = schema {
for asset in &mut sql_assets.assets {
if asset.kind == kind && asset.path.starts_with(asset_name) {
asset.path = format!(
"{}/{}.{}",
asset_name,
schema,
&asset.path[asset_name.len() + 1..]
);
}
}
}
return Ok(Some(sql_assets.assets));
}
_ => Ok(None),
}
}

View File

@@ -21,7 +21,9 @@ pub const SANITIZED_ENUM_STR: &str = "__sanitized_enum__";
pub const SANITIZED_RAW_STRING_STR: &str = "__sanitized_raw_string__";
mod asset_parser;
mod asset_parser_utils;
pub use asset_parser::parse_assets;
pub use asset_parser_utils::parse_wmill_sdk_sql_assets;
pub fn parse_mysql_sig(code: &str) -> anyhow::Result<MainArgSignature> {
let parsed = parse_mysql_file(&code)?;

View File

@@ -241,12 +241,6 @@ impl Visit for AssetsFinder {
}
});
let duckdb_conn_prefix = match kind {
AssetKind::DataTable => "datatable",
AssetKind::Ducklake => "ducklake",
_ => return,
};
// Capture SQL query details before transforming for SQL parser
let span = node.span();
let span_tuple = (span.lo.0, span.hi.0);
@@ -259,26 +253,15 @@ impl Visit for AssetsFinder {
source_schema: schema.clone(),
});
let sql_with_attach =
format!("ATTACH '{duckdb_conn_prefix}://{asset_name}' AS dt; USE dt; {sql}");
// We use the SQL parser to detect if it's a read or write query
match windmill_parser_sql::parse_assets(&sql_with_attach) {
Ok(mut sql_assets) => {
if let Some(schema) = schema {
for asset in &mut sql_assets.assets {
if asset.kind == *kind && asset.path.starts_with(asset_name) {
asset.path = format!(
"{}/{}.{}",
asset_name,
schema,
&asset.path[asset_name.len() + 1..]
);
}
}
}
self.assets.extend(sql_assets.assets);
}
// We use the SQL parser to detect RW, specific tables, etc.
let sql_assets = windmill_parser_sql::parse_wmill_sdk_sql_assets(
*kind,
asset_name,
schema.as_deref(),
&sql,
);
match sql_assets {
Ok(Some(sql_assets)) => self.assets.extend(sql_assets),
_ => {}
}
}

View File

@@ -11,4 +11,4 @@ path = "./src/lib.rs"
[dependencies]
serde = { version = "1.0", features = ["derive"] }
serde_json.workspace = true
convert_case.workspace = true
convert_case.workspace = true

View File

@@ -93,20 +93,23 @@ pub fn asset_was_used(assets: &Vec<ParseAssetsResult>, (kind, path): (AssetKind,
pub fn parse_asset_syntax(s: &str, enable_default_syntax: bool) -> Option<(AssetKind, &str)> {
if enable_default_syntax && s == "datatable" {
Some((AssetKind::DataTable, "main"))
return Some((AssetKind::DataTable, "main"));
} else if enable_default_syntax && s == "ducklake" {
Some((AssetKind::Ducklake, "main"))
} else if s.starts_with("s3://") {
Some((AssetKind::S3Object, &s[5..]))
} else if s.starts_with("res://") {
Some((AssetKind::Resource, &s[6..]))
} else if s.starts_with("$res:") {
Some((AssetKind::Resource, &s[5..]))
} else if s.starts_with("ducklake://") {
Some((AssetKind::Ducklake, &s[11..]))
} else if s.starts_with("datatable://") {
Some((AssetKind::DataTable, &s[12..]))
} else {
None
return Some((AssetKind::Ducklake, "main"));
}
for (prefix, kind) in ASSET_KINDS.iter() {
if s.starts_with(prefix) {
let path = &s[prefix.len()..];
return Some((*kind, path));
}
}
None
}
pub const ASSET_KINDS: &[(&str, AssetKind)] = &[
("s3://", AssetKind::S3Object),
("res://", AssetKind::Resource),
("$res:", AssetKind::Resource),
("ducklake://", AssetKind::Ducklake),
("datatable://", AssetKind::DataTable),
];

View File

@@ -34,7 +34,7 @@ use windmill_common::ee_oss::{
};
use windmill_common::{
agent_workers::build_agent_http_client,
agent_workers::AgentConfig,
global_settings::{
APP_WORKSPACED_ROUTE_SETTING, BASE_URL_SETTING, BUNFIG_INSTALL_SCOPES_SETTING,
CRITICAL_ALERTS_ON_DB_OVERSIZE_SETTING, CRITICAL_ALERT_MUTE_UI_SETTING,
@@ -43,13 +43,14 @@ use windmill_common::{
EXPOSE_DEBUG_METRICS_SETTING, EXPOSE_METRICS_SETTING, EXTRA_PIP_INDEX_URL_SETTING,
HUB_API_SECRET_SETTING, HUB_BASE_URL_SETTING, INDEXER_SETTING,
INSTANCE_PYTHON_VERSION_SETTING, JOB_DEFAULT_TIMEOUT_SECS_SETTING, JWT_SECRET_SETTING,
KEEP_JOB_DIR_SETTING, LICENSE_KEY_SETTING, MAVEN_REPOS_SETTING, OTEL_TRACING_PROXY_SETTING,
KEEP_JOB_DIR_SETTING, LICENSE_KEY_SETTING, MAVEN_REPOS_SETTING,
MONITOR_LOGS_ON_OBJECT_STORE_SETTING, NO_DEFAULT_MAVEN_SETTING,
NPM_CONFIG_REGISTRY_SETTING, NUGET_CONFIG_SETTING, OAUTH_SETTING, OTEL_SETTING,
PIP_INDEX_URL_SETTING, POWERSHELL_REPO_PAT_SETTING, POWERSHELL_REPO_URL_SETTING,
REQUEST_SIZE_LIMIT_SETTING, REQUIRE_PREEXISTING_USER_FOR_OAUTH_SETTING,
RETENTION_PERIOD_SECS_SETTING, RUBY_REPOS_SETTING, SAML_METADATA_SETTING,
SCIM_TOKEN_SETTING, SMTP_SETTING, TEAMS_SETTING, TIMEOUT_WAIT_RESULT_SETTING,
OTEL_TRACING_PROXY_SETTING, PIP_INDEX_URL_SETTING, POWERSHELL_REPO_PAT_SETTING,
POWERSHELL_REPO_URL_SETTING, REQUEST_SIZE_LIMIT_SETTING,
REQUIRE_PREEXISTING_USER_FOR_OAUTH_SETTING, RETENTION_PERIOD_SECS_SETTING,
RUBY_REPOS_SETTING, SAML_METADATA_SETTING, SCIM_TOKEN_SETTING, SMTP_SETTING, TEAMS_SETTING,
TIMEOUT_WAIT_RESULT_SETTING,
},
scripts::ScriptLang,
stats_oss::schedule_stats,
@@ -99,9 +100,10 @@ use crate::monitor::{
reload_bunfig_install_scopes_setting, reload_critical_alert_mute_ui_setting,
reload_critical_error_channels_setting, reload_extra_pip_index_url_setting,
reload_hub_api_secret_setting, reload_hub_base_url_setting, reload_job_default_timeout_setting,
reload_jwt_secret_setting, reload_license_key, reload_otel_tracing_proxy_setting,
reload_npm_config_registry_setting, reload_pip_index_url_setting, reload_retention_period_setting,
reload_scim_token_setting, reload_smtp_config, reload_worker_config, MonitorIteration,
reload_jwt_secret_setting, reload_license_key, reload_npm_config_registry_setting,
reload_otel_tracing_proxy_setting, reload_pip_index_url_setting,
reload_retention_period_setting, reload_scim_token_setting, reload_smtp_config,
reload_worker_config, MonitorIteration,
};
#[cfg(feature = "parquet")]
@@ -410,7 +412,10 @@ pub async fn sync_cached_resource_types(db: &sqlx::Pool<sqlx::Postgres>) -> anyh
let cache_path = format!("{}/{}", HUB_RT_CACHE_DIR, HUB_RT_CACHE_FILE);
if tokio::fs::metadata(&cache_path).await.is_err() {
tracing::info!("No cached resource types found at {}, skipping sync", cache_path);
tracing::info!(
"No cached resource types found at {}, skipping sync",
cache_path
);
return Ok(());
}
@@ -420,8 +425,8 @@ pub async fn sync_cached_resource_types(db: &sqlx::Pool<sqlx::Postgres>) -> anyh
.await
.with_context(|| format!("Failed to read cache file from {}", cache_path))?;
let cached_types: Vec<HubResourceType> = serde_json::from_str(&content)
.with_context(|| "Failed to parse cached resource types")?;
let cached_types: Vec<HubResourceType> =
serde_json::from_str(&content).with_context(|| "Failed to parse cached resource types")?;
tracing::info!("Found {} cached resource types", cached_types.len());
@@ -433,11 +438,13 @@ pub async fn sync_cached_resource_types(db: &sqlx::Pool<sqlx::Postgres>) -> anyh
.await
.with_context(|| "Failed to fetch existing resource types")?;
let existing_map: std::collections::HashMap<String, (Option<serde_json::Value>, Option<String>)> =
existing_types
.into_iter()
.map(|(name, schema, desc)| (name, (schema, desc)))
.collect();
let existing_map: std::collections::HashMap<
String,
(Option<serde_json::Value>, Option<String>),
> = existing_types
.into_iter()
.map(|(name, schema, desc)| (name, (schema, desc)))
.collect();
let mut synced_count = 0;
let mut skipped_count = 0;
@@ -478,36 +485,45 @@ pub async fn sync_cached_resource_types(db: &sqlx::Pool<sqlx::Postgres>) -> anyh
}
fn print_help() {
println!("Windmill - a fast, open-source workflow engine and job runner.");
println!();
println!("Usage:");
println!(" windmill [SUBCOMMAND]");
println!();
println!("Subcommands:");
println!(" help | -h | --help Show this help information and exit");
println!(" version Show Windmill version and exit");
println!(" cache [hubPaths.json] Pre-cache hub scripts (default: ./hubPaths.json)");
println!(" cache-rt Pre-cache hub resource types");
println!();
println!("Environment variables (name = default):");
println!(" DATABASE_URL = <required> The Postgres database url.");
println!(" MODE = standalone Mode: standalone | worker | server | agent");
println!(" BASE_URL = http://localhost:8000 Public base URL of your instance (overridden by instance settings)");
println!(" PORT = {} HTTP port (server/indexer/MCP modes)", DEFAULT_PORT);
println!(" SERVER_BIND_ADDR = <mode dependent> IP to bind to (server: {}, worker: {})", DEFAULT_SERVER_BIND_ADDR, DEFAULT_WORKER_BIND_ADDR);
println!(" NUM_WORKERS = {} Number of workers (standalone/worker modes)", DEFAULT_NUM_WORKERS);
println!(" WORKER_GROUP = default Worker group this worker belongs to",);
println!(" JSON_FMT = false Output logs in JSON instead of logfmt");
println!(" METRICS_ADDR = None (EE only) Prometheus metrics addr at /metrics; set \"true\" to use :8001");
println!(" SUPERADMIN_SECRET = None Virtual superadmin token (server)");
println!(" LICENSE_KEY = None (EE only) Enterprise license key (workers require valid key)");
println!(" RUN_UPDATE_CA_CERTIFICATE_AT_START = false Run system CA update at startup");
println!(" RUN_UPDATE_CA_CERTIFICATE_PATH = /usr/sbin/update-ca-certificates Path to CA update tool");
println!(" SYNC_CACHED_RT = false Sync cached resource types to admins workspace on server start");
println!();
println!("Notes:");
println!("- Advanced and less commonly used settings are managed via the database and are omitted here.");
println!("- At startup, Windmill logs currently set configuration keys for visibility.");
println!("Windmill - a fast, open-source workflow engine and job runner.");
println!();
println!("Usage:");
println!(" windmill [SUBCOMMAND]");
println!();
println!("Subcommands:");
println!(" help | -h | --help Show this help information and exit");
println!(" version Show Windmill version and exit");
println!(" cache [hubPaths.json] Pre-cache hub scripts (default: ./hubPaths.json)");
println!(" cache-rt Pre-cache hub resource types");
println!();
println!("Environment variables (name = default):");
println!(" DATABASE_URL = <required> The Postgres database url.");
println!(" MODE = standalone Mode: standalone | worker | server | agent");
println!(" BASE_URL = http://localhost:8000 Public base URL of your instance (overridden by instance settings)");
println!(
" PORT = {} HTTP port (server/indexer/MCP modes)",
DEFAULT_PORT
);
println!(
" SERVER_BIND_ADDR = <mode dependent> IP to bind to (server: {}, worker: {})",
DEFAULT_SERVER_BIND_ADDR, DEFAULT_WORKER_BIND_ADDR
);
println!(
" NUM_WORKERS = {} Number of workers (standalone/worker modes)",
DEFAULT_NUM_WORKERS
);
println!(" WORKER_GROUP = default Worker group this worker belongs to",);
println!(" JSON_FMT = false Output logs in JSON instead of logfmt");
println!(" METRICS_ADDR = None (EE only) Prometheus metrics addr at /metrics; set \"true\" to use :8001");
println!(" SUPERADMIN_SECRET = None Virtual superadmin token (server)");
println!(" LICENSE_KEY = None (EE only) Enterprise license key (workers require valid key)");
println!(" RUN_UPDATE_CA_CERTIFICATE_AT_START = false Run system CA update at startup");
println!(" RUN_UPDATE_CA_CERTIFICATE_PATH = /usr/sbin/update-ca-certificates Path to CA update tool");
println!(" SYNC_CACHED_RT = false Sync cached resource types to admins workspace on server start");
println!();
println!("Notes:");
println!("- Advanced and less commonly used settings are managed via the database and are omitted here.");
println!("- At startup, Windmill logs currently set configuration keys for visibility.");
}
async fn windmill_main() -> anyhow::Result<()> {
@@ -641,15 +657,23 @@ async fn windmill_main() -> anyhow::Result<()> {
.and_then(|x| x.parse().ok())
.unwrap_or(IpAddr::from(default_bind_addr));
let (conn, first_suffix) = if mode == Mode::Agent {
let (conn, first_suffix, agent_config) = if mode == Mode::Agent {
let agent_config = match AgentConfig::from_env() {
Ok(config) => config,
Err(e) => {
tracing::error!("{e}");
std::process::exit(1);
}
};
tracing::info!(
"Creating http client for cluster using base internal url {}",
std::env::var("BASE_INTERNAL_URL").unwrap_or_default()
agent_config.base_internal_url
);
let suffix = create_default_worker_suffix(&hostname);
(
Connection::Http(build_agent_http_client(&suffix, None, None)),
Connection::Http(agent_config.build_http_client(&suffix)),
Some(suffix),
Some(agent_config),
)
} else {
println!("Connecting to database...");
@@ -673,7 +697,8 @@ async fn windmill_main() -> anyhow::Result<()> {
reload_otel_tracing_proxy_setting(&Connection::Sql(db.clone())).await;
#[cfg(feature = "deno_core")]
if windmill_worker::is_otel_tracing_proxy_enabled_for_lang(&ScriptLang::Nativets).await {
if windmill_worker::is_otel_tracing_proxy_enabled_for_lang(&ScriptLang::Nativets).await
{
match windmill_worker::load_internal_otel_exporter().await {
Ok(()) => {
tracing::info!("Internal OTEL exporter initialized for nativets tracing");
@@ -688,7 +713,7 @@ async fn windmill_main() -> anyhow::Result<()> {
load_otel(&db).await;
println!("Database connected");
(Connection::Sql(db), None)
(Connection::Sql(db), None, None)
};
let environment = if let Ok(environment) = std::env::var("OTEL_ENVIRONMENT") {
@@ -799,6 +824,12 @@ Windmill Community Edition {GIT_VERSION}
// if key still invalid and num_workers > 0, set to 0
if let Err(err) = reload_license_key(&conn).await {
tracing::error!("Failed to reload license key: {err:#}");
if is_agent {
tracing::error!(
"Agent worker cannot connect to server. Please check AGENT_TOKEN and BASE_INTERNAL_URL"
);
std::process::exit(1);
}
}
let valid_key = *LICENSE_KEY_VALID.read().await;
if !valid_key && !server_mode {
@@ -1057,7 +1088,12 @@ Windmill Community Edition {GIT_VERSION}
conn: if i == 0 || mode != Mode::Agent {
conn.clone()
} else {
Connection::Http(build_agent_http_client(&suffix, None, None))
Connection::Http(
agent_config
.as_ref()
.expect("agent_config must be set in agent mode")
.build_http_client(&suffix),
)
},
worker_name: worker_name_with_suffix(
mode == Mode::Agent,

View File

@@ -1059,20 +1059,36 @@ async fn delete_expired_jobs_batch(
) -> error::Result<usize> {
let mut tx = db.begin().await?;
// Fetch active ROOT job IDs that started before the retention period. We only care about
// these because their child jobs could be old enough to be deletion candidates.
// Jobs started after the retention period can't have children old enough to delete.
let active_root_job_ids: Vec<Uuid> = sqlx::query_scalar!(
"SELECT q.id FROM v2_job_queue q
JOIN v2_job j ON j.id = q.id
WHERE j.parent_job IS NULL
AND j.created_at <= now() - ($1::bigint::text || ' s')::interval",
job_retention_secs
)
.fetch_all(&mut *tx)
.await?;
// Use FOR UPDATE SKIP LOCKED to avoid contention between replicas
// ORDER BY completed_at ensures we delete oldest jobs first
let deleted_jobs: Vec<Uuid> = sqlx::query_scalar!(
"DELETE FROM v2_job_completed
WHERE id IN (
SELECT id FROM v2_job_completed
WHERE completed_at <= now() - ($1::bigint::text || ' s')::interval
ORDER BY completed_at ASC
SELECT jc.id FROM v2_job_completed jc
LEFT JOIN v2_job j ON j.id = jc.id
WHERE jc.completed_at <= now() - ($1::bigint::text || ' s')::interval
AND COALESCE(j.root_job, j.flow_innermost_root_job, jc.id) != ALL($3)
ORDER BY jc.completed_at ASC
LIMIT $2
FOR UPDATE SKIP LOCKED
FOR UPDATE OF jc SKIP LOCKED
)
RETURNING id",
job_retention_secs,
batch_size
batch_size,
&active_root_job_ids
)
.fetch_all(&mut *tx)
.await?;

View File

@@ -764,23 +764,25 @@ pub async fn run_preview_relative_imports(
#[cfg(all(feature = "private", feature = "agent_worker_server"))]
pub async fn testing_http_connection(port: u16) -> Connection {
let suffix = windmill_common::utils::create_default_worker_suffix("test-agent-worker");
let agent_token = format!(
"{}{}",
windmill_common::agent_workers::AGENT_JWT_PREFIX,
windmill_common::jwt::encode_with_internal_secret(
windmill_api::agent_workers_ee::AgentAuth {
worker_group: "testing-agent".to_owned(),
suffix: Some(suffix.clone()),
tags: vec!["flow".into(), "python3".into(), "dependency".into()],
exp: Some(usize::MAX),
}
)
.await
.expect("JWT token to be created")
);
let base_internal_url = format!("http://localhost:{port}");
Connection::Http(windmill_common::agent_workers::build_agent_http_client(
&suffix,
Some(format!(
"{}{}",
windmill_common::agent_workers::AGENT_JWT_PREFIX,
windmill_common::jwt::encode_with_internal_secret(
windmill_api::agent_workers_ee::AgentAuth {
worker_group: "testing-agent".to_owned(),
suffix: Some(suffix.clone()),
tags: vec!["flow".into(), "python3".into(), "dependency".into()],
exp: Some(usize::MAX),
}
)
.await
.expect("JWT token to be created")
)),
Some(format!("http://localhost:{port}")),
&agent_token,
&base_internal_url,
))
}

View File

@@ -0,0 +1,507 @@
use serde_json::json;
use sqlx::{Pool, Postgres};
mod common;
use common::*;
/// Comprehensive integration test for the compare_workspaces endpoint.
///
/// This test validates workspace fork comparison functionality by:
/// 1. Setting up a parent workspace with all item types (scripts, flows, apps, resources, variables, resource_types, folders)
/// 2. Creating a fork of the workspace
/// 3. Making various changes in both workspaces (new items, modifications, conflicts, deletions, renames)
/// 4. Populating the workspace_diff table to simulate Git sync tracking
/// 5. Calling compare_workspaces and verifying all aspects of the comparison
#[sqlx::test(fixtures("base"))]
async fn test_compare_workspaces_comprehensive(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let client = windmill_api_client::create_client(
&format!("http://localhost:{port}"),
"SECRET_TOKEN".to_string(),
);
let base_url = format!("http://localhost:{port}/api");
// ==============================================================
// PHASE 1: Setup Parent Workspace with All Item Types
// ==============================================================
// Create folder first (other items will use it)
sqlx::query!(
"INSERT INTO folder (workspace_id, name, display_name, owners, summary, created_by)
VALUES ('test-workspace', 'shared', 'Shared Folder', ARRAY['test@windmill.dev']::varchar[], 'Test folder', 'test@windmill.dev')"
)
.execute(&db)
.await?;
// Create scripts
sqlx::query!(
"INSERT INTO script (workspace_id, path, hash, content, summary, description, language, created_by, created_at, archived, schema_validation, ws_error_handler_muted, deleted)
VALUES
('test-workspace', 'f/shared/original_script', 12345, 'def main(): pass', 'Original', '', 'python3', 'test@windmill.dev', NOW(), false, false, false, false),
('test-workspace', 'f/shared/to_modify_parent', 22222, 'def main(): return 1', 'To modify in parent', '', 'python3', 'test@windmill.dev', NOW(), false, false, false, false),
('test-workspace', 'f/shared/to_modify_fork', 33333, 'def main(): return 2', 'To modify in fork', '', 'python3', 'test@windmill.dev', NOW(), false, false, false, false),
('test-workspace', 'f/shared/to_conflict', 44444, 'def main(): return 3', 'To conflict', '', 'python3', 'test@windmill.dev', NOW(), false, false, false, false),
('test-workspace', 'f/shared/to_delete', 55555, 'def main(): return 4', 'To delete', '', 'python3', 'test@windmill.dev', NOW(), false, false, false, false)"
)
.execute(&db)
.await?;
// Create flow
sqlx::query!(
"INSERT INTO flow (workspace_id, path, summary, description, value, schema, edited_by, edited_at, archived)
VALUES ('test-workspace', 'f/shared/original_flow', 'Flow summary', '', $1, NULL, 'test@windmill.dev', NOW(), false)",
json!({"modules": []})
)
.execute(&db)
.await?;
// Create resource
sqlx::query!(
"INSERT INTO resource (workspace_id, path, value, resource_type, description, created_by)
VALUES
('test-workspace', 'f/shared/db_config', $1, 'postgresql', '', 'test@windmill.dev'),
('test-workspace', 'f/shared/old_name', $2, 'generic', '', 'test@windmill.dev'),
('test-workspace', 'f/shared/resource_to_modify', $3, 'generic', '', 'test@windmill.dev')",
json!({"host": "localhost"}),
json!({}),
json!({"key": "value"})
)
.execute(&db)
.await?;
// Create variable
sqlx::query!(
"INSERT INTO variable (workspace_id, path, value, is_secret, description)
VALUES
('test-workspace', 'f/shared/api_key', 'secret123', false, 'Test key'),
('test-workspace', 'f/shared/variable_to_modify', 'original', false, 'To modify')"
)
.execute(&db)
.await?;
// Create resource type
sqlx::query!(
"INSERT INTO resource_type (workspace_id, name, schema, description, created_by)
VALUES ('test-workspace', 'custom_db', $1, 'Custom DB type', 'test@windmill.dev')",
json!({"type": "object"})
)
.execute(&db)
.await?;
// Create app
sqlx::query!(
"INSERT INTO app (workspace_id, path, summary, policy, versions, extra_perms, draft_only)
VALUES ('test-workspace', 'f/shared/dashboard', 'Dashboard app', '{}', ARRAY[1::bigint], '{}', false)"
)
.execute(&db)
.await?;
let app_id = sqlx::query_scalar!(
"SELECT id FROM app WHERE path = 'f/shared/dashboard' AND workspace_id = 'test-workspace'"
)
.fetch_one(&db)
.await?;
sqlx::query!(
"INSERT INTO app_version (app_id, value, created_by, created_at)
VALUES ($1, $2, 'test@windmill.dev', NOW())",
app_id,
json!({"grid": []})
)
.execute(&db)
.await?;
// ==============================================================
// PHASE 2: Create Fork
// ==============================================================
let fork_response = client
.client()
.post(&format!("{base_url}/w/test-workspace/workspaces/create_fork"))
.json(&json!({
"id": "wm-fork-test-workspace",
"name": "Test Fork",
"color": "#0000ff"
}))
.send()
.await?;
let status = fork_response.status();
assert!(status.is_success(), "Fork creation should succeed: {}", status);
// Verify fork was created
let fork_exists = sqlx::query_scalar!(
"SELECT EXISTS(SELECT 1 FROM workspace WHERE id = 'wm-fork-test-workspace')"
)
.fetch_one(&db)
.await?;
assert!(fork_exists.unwrap_or(false), "Fork workspace should exist");
// ==============================================================
// PHASE 3: Make Changes in Both Workspaces
// ==============================================================
// Scenario 1: New script in parent (ahead)
sqlx::query!(
"INSERT INTO script (workspace_id, path, hash, content, summary, description, language, created_by, created_at, archived, schema_validation, ws_error_handler_muted, deleted)
VALUES ('test-workspace', 'f/shared/new_in_parent', 54321, 'def main(): return \"new\"', 'New in parent', '', 'python3', 'test@windmill.dev', NOW(), false, false, false, false)"
)
.execute(&db)
.await?;
// Scenario 2: New script in fork (behind)
sqlx::query!(
"INSERT INTO script (workspace_id, path, hash, content, summary, description, language, created_by, created_at, archived, schema_validation, ws_error_handler_muted, deleted)
VALUES ('wm-fork-test-workspace', 'f/shared/new_in_fork', 99999, 'def main(): return \"fork\"', 'New in fork', '', 'python3', 'test@windmill.dev', NOW(), false, false, false, false)"
)
.execute(&db)
.await?;
// Scenario 3: Modify script in parent (ahead)
sqlx::query!(
"UPDATE script
SET content = 'def main(): return \"modified\"', summary = 'Modified in parent'
WHERE workspace_id = 'test-workspace' AND path = 'f/shared/to_modify_parent'"
)
.execute(&db)
.await?;
// Scenario 4: Modify script in fork (behind)
sqlx::query!(
"UPDATE script
SET content = 'def main(): return \"fork_modified\"', summary = 'Modified in fork'
WHERE workspace_id = 'wm-fork-test-workspace' AND path = 'f/shared/to_modify_fork'"
)
.execute(&db)
.await?;
// Scenario 5: Conflict - modify in both workspaces
sqlx::query!(
"UPDATE flow SET value = $1
WHERE workspace_id = 'test-workspace' AND path = 'f/shared/original_flow'",
json!({"modules": [{"id": "a"}]})
)
.execute(&db)
.await?;
sqlx::query!(
"UPDATE flow SET value = $1
WHERE workspace_id = 'wm-fork-test-workspace' AND path = 'f/shared/original_flow'",
json!({"modules": [{"id": "b"}]})
)
.execute(&db)
.await?;
// Scenario 6: Delete (archive) in fork
sqlx::query!(
"UPDATE script SET archived = true
WHERE workspace_id = 'wm-fork-test-workspace' AND path = 'f/shared/to_delete'"
)
.execute(&db)
.await?;
// Scenario 7: Rename in parent (resource)
sqlx::query!(
"UPDATE resource SET path = 'f/shared/new_name'
WHERE workspace_id = 'test-workspace' AND path = 'f/shared/old_name'"
)
.execute(&db)
.await?;
// Scenario 8: Modify app in parent
sqlx::query!(
"UPDATE app SET summary = 'Modified dashboard app'
WHERE workspace_id = 'test-workspace' AND path = 'f/shared/dashboard'"
)
.execute(&db)
.await?;
// Scenario 9: Modify resource in fork
sqlx::query!(
"UPDATE resource SET value = $1
WHERE workspace_id = 'wm-fork-test-workspace' AND path = 'f/shared/resource_to_modify'",
json!({"key": "modified_value"})
)
.execute(&db)
.await?;
// Modify variable in parent
sqlx::query!(
"UPDATE variable SET value = 'modified_value'
WHERE workspace_id = 'test-workspace' AND path = 'f/shared/variable_to_modify'"
)
.execute(&db)
.await?;
// Create new resource type in parent
sqlx::query!(
"INSERT INTO resource_type (workspace_id, name, schema, description, created_by)
VALUES ('test-workspace', 'new_type', $1, 'New type in parent', 'test@windmill.dev')",
json!({"type": "string"})
)
.execute(&db)
.await?;
// Modify folder in fork (display_name)
sqlx::query!(
"UPDATE folder SET display_name = 'Modified Shared Folder'
WHERE workspace_id = 'wm-fork-test-workspace' AND name = 'shared'"
)
.execute(&db)
.await?;
// ==============================================================
// PHASE 4: Populate workspace_diff Table
// ==============================================================
// New in parent (ahead)
sqlx::query!(
"INSERT INTO workspace_diff
(source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)
VALUES
('test-workspace', 'wm-fork-test-workspace', 'f/shared/new_in_parent', 'script', 1, 0, NULL),
('test-workspace', 'wm-fork-test-workspace', 'new_type', 'resource_type', 1, 0, NULL)"
)
.execute(&db)
.await?;
// New in fork (behind)
sqlx::query!(
"INSERT INTO workspace_diff
(source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)
VALUES ('test-workspace', 'wm-fork-test-workspace', 'f/shared/new_in_fork', 'script', 0, 1, NULL)"
)
.execute(&db)
.await?;
// Modified in parent (ahead)
sqlx::query!(
"INSERT INTO workspace_diff
(source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)
VALUES
('test-workspace', 'wm-fork-test-workspace', 'f/shared/to_modify_parent', 'script', 1, 0, NULL),
('test-workspace', 'wm-fork-test-workspace', 'f/shared/dashboard', 'app', 1, 0, NULL),
('test-workspace', 'wm-fork-test-workspace', 'f/shared/variable_to_modify', 'variable', 1, 0, NULL)"
)
.execute(&db)
.await?;
// Modified in fork (behind)
sqlx::query!(
"INSERT INTO workspace_diff
(source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)
VALUES
('test-workspace', 'wm-fork-test-workspace', 'f/shared/to_modify_fork', 'script', 0, 1, NULL),
('test-workspace', 'wm-fork-test-workspace', 'f/shared/resource_to_modify', 'resource', 0, 1, NULL),
('test-workspace', 'wm-fork-test-workspace', 'shared', 'folder', 0, 1, NULL)"
)
.execute(&db)
.await?;
// Conflict (both ahead and behind)
sqlx::query!(
"INSERT INTO workspace_diff
(source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)
VALUES
('test-workspace', 'wm-fork-test-workspace', 'f/shared/original_flow', 'flow', 1, 1, NULL),
('test-workspace', 'wm-fork-test-workspace', 'f/shared/to_conflict', 'script', 1, 1, NULL)"
)
.execute(&db)
.await?;
// Deleted in fork (exists only in parent)
sqlx::query!(
"INSERT INTO workspace_diff
(source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)
VALUES ('test-workspace', 'wm-fork-test-workspace', 'f/shared/to_delete', 'script', 1, 0, NULL)"
)
.execute(&db)
.await?;
// Renamed in parent (two entries)
sqlx::query!(
"INSERT INTO workspace_diff
(source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)
VALUES
('test-workspace', 'wm-fork-test-workspace', 'f/shared/old_name', 'resource', 0, 1, NULL),
('test-workspace', 'wm-fork-test-workspace', 'f/shared/new_name', 'resource', 1, 0, NULL)"
)
.execute(&db)
.await?;
// Add an unchanged item to verify it gets filtered out
sqlx::query!(
"INSERT INTO workspace_diff
(source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)
VALUES ('test-workspace', 'wm-fork-test-workspace', 'f/shared/original_script', 'script', 0, 0, NULL)"
)
.execute(&db)
.await?;
// ==============================================================
// PHASE 5: Call compare_workspaces and Verify Results
// ==============================================================
let comparison: serde_json::Value = client
.client()
.get(&format!("{base_url}/w/test-workspace/workspaces/compare/wm-fork-test-workspace"))
.send()
.await?
.json()
.await?;
// Verify basic structure
assert!(!comparison["skipped_comparison"].as_bool().unwrap_or(true), "Should not skip comparison");
assert!(comparison["diffs"].is_array(), "Should have diffs array");
assert!(comparison["summary"].is_object(), "Should have summary object");
let diffs = comparison["diffs"].as_array().unwrap();
let summary = &comparison["summary"];
// ==============================================================
// Summary Assertions
// ==============================================================
// Total diffs (excluding unchanged items which should be deleted)
let total_diffs = summary["total_diffs"].as_u64().unwrap();
assert!(total_diffs > 0, "Should have at least some diffs");
// Verify ahead/behind counts
let total_ahead = summary["total_ahead"].as_u64().unwrap();
let total_behind = summary["total_behind"].as_u64().unwrap();
assert!(total_ahead > 0, "Should have items ahead");
assert!(total_behind > 0, "Should have items behind");
// Verify conflicts (items that are both ahead and behind)
let conflicts = summary["conflicts"].as_u64().unwrap();
assert!(conflicts >= 1, "Should have at least 1 conflict (flow)");
// Verify per-item-type counts
assert!(summary["scripts_changed"].as_u64().unwrap() > 0, "Should have script changes");
assert!(summary["flows_changed"].as_u64().unwrap() > 0, "Should have flow changes");
assert!(summary["apps_changed"].as_u64().unwrap() > 0, "Should have app changes");
assert!(summary["resources_changed"].as_u64().unwrap() > 0, "Should have resource changes");
assert!(summary["variables_changed"].as_u64().unwrap() > 0, "Should have variable changes");
assert!(summary["resource_types_changed"].as_u64().unwrap() > 0, "Should have resource_type changes");
// Note: folders_changed may be 0 if folder comparison didn't detect changes
// assert!(summary["folders_changed"].as_u64().unwrap() > 0, "Should have folder changes");
// ==============================================================
// Individual Diff Assertions
// ==============================================================
// Scenario 1: New in parent
let new_in_parent = diffs.iter()
.find(|d| d["path"] == "f/shared/new_in_parent" && d["kind"] == "script")
.expect("Should find new_in_parent diff");
assert_eq!(new_in_parent["ahead"].as_i64().unwrap(), 1, "new_in_parent should be ahead");
assert_eq!(new_in_parent["behind"].as_i64().unwrap(), 0, "new_in_parent should not be behind");
assert_eq!(new_in_parent["has_changes"].as_bool().unwrap(), true, "new_in_parent should have changes");
assert_eq!(new_in_parent["exists_in_source"].as_bool().unwrap(), true, "new_in_parent should exist in source");
assert_eq!(new_in_parent["exists_in_fork"].as_bool().unwrap(), false, "new_in_parent should not exist in fork");
// Scenario 2: New in fork
let new_in_fork = diffs.iter()
.find(|d| d["path"] == "f/shared/new_in_fork" && d["kind"] == "script")
.expect("Should find new_in_fork diff");
assert_eq!(new_in_fork["ahead"].as_i64().unwrap(), 0, "new_in_fork should not be ahead");
assert_eq!(new_in_fork["behind"].as_i64().unwrap(), 1, "new_in_fork should be behind");
assert_eq!(new_in_fork["has_changes"].as_bool().unwrap(), true, "new_in_fork should have changes");
assert_eq!(new_in_fork["exists_in_source"].as_bool().unwrap(), false, "new_in_fork should not exist in source");
assert_eq!(new_in_fork["exists_in_fork"].as_bool().unwrap(), true, "new_in_fork should exist in fork");
// Scenario 5: Conflict
let conflict_flow = diffs.iter()
.find(|d| d["path"] == "f/shared/original_flow" && d["kind"] == "flow")
.expect("Should find conflict flow diff");
assert!(conflict_flow["ahead"].as_i64().unwrap() > 0, "conflict should be ahead");
assert!(conflict_flow["behind"].as_i64().unwrap() > 0, "conflict should be behind");
assert_eq!(conflict_flow["has_changes"].as_bool().unwrap(), true, "conflict should have changes");
assert_eq!(conflict_flow["exists_in_source"].as_bool().unwrap(), true, "conflict should exist in source");
assert_eq!(conflict_flow["exists_in_fork"].as_bool().unwrap(), true, "conflict should exist in fork");
// Scenario 6: Deleted in fork
let deleted = diffs.iter()
.find(|d| d["path"] == "f/shared/to_delete" && d["kind"] == "script")
.expect("Should find deleted diff");
assert_eq!(deleted["exists_in_source"].as_bool().unwrap(), true, "deleted should exist in source");
assert_eq!(deleted["exists_in_fork"].as_bool().unwrap(), false, "deleted should not exist in fork (archived)");
assert_eq!(deleted["has_changes"].as_bool().unwrap(), true, "deleted should have changes");
// Scenario 7: Rename (should show as two entries)
let old_name = diffs.iter()
.find(|d| d["path"] == "f/shared/old_name" && d["kind"] == "resource");
let new_name = diffs.iter()
.find(|d| d["path"] == "f/shared/new_name" && d["kind"] == "resource");
// At least one of these should exist (depending on how the comparison handles renames)
assert!(old_name.is_some() || new_name.is_some(), "Should find at least one rename-related diff");
// ==============================================================
// Database State Assertions
// ==============================================================
// Verify has_changes was cached for items that have changes
let cached_new_in_parent = sqlx::query!(
"SELECT has_changes, exists_in_source, exists_in_fork FROM workspace_diff
WHERE path = 'f/shared/new_in_parent' AND kind = 'script' AND source_workspace_id = 'test-workspace'"
)
.fetch_one(&db)
.await?;
assert_eq!(cached_new_in_parent.has_changes, Some(true), "has_changes should be cached as true");
assert_eq!(cached_new_in_parent.exists_in_source, Some(true), "exists_in_source should be cached");
assert_eq!(cached_new_in_parent.exists_in_fork, Some(false), "exists_in_fork should be cached");
// Verify unchanged items were deleted from workspace_diff
let unchanged_original_script = sqlx::query!(
"SELECT has_changes FROM workspace_diff
WHERE path = 'f/shared/original_script' AND kind = 'script' AND source_workspace_id = 'test-workspace'"
)
.fetch_optional(&db)
.await?;
// The unchanged item should either be deleted or marked as has_changes = false
// Based on the code, items with has_changes = false are deleted
if let Some(record) = unchanged_original_script {
assert_ne!(record.has_changes, Some(false), "unchanged items with has_changes=false should be deleted");
}
// ==============================================================
// Lazy Evaluation Test
// ==============================================================
// Create a new diff entry with NULL has_changes
sqlx::query!(
"INSERT INTO workspace_diff
(source_workspace_id, fork_workspace_id, path, kind, ahead, behind, has_changes)
VALUES ('test-workspace', 'wm-fork-test-workspace', 'f/shared/lazy_test', 'script', 1, 0, NULL)
ON CONFLICT DO NOTHING"
)
.execute(&db)
.await?;
// Call the endpoint again
let _comparison2: serde_json::Value = client
.client()
.get(&format!("{base_url}/w/test-workspace/workspaces/compare/wm-fork-test-workspace"))
.send()
.await?
.json()
.await?;
// Verify the lazy_test entry was evaluated (should be deleted since it doesn't exist)
let lazy_test = sqlx::query!(
"SELECT has_changes FROM workspace_diff
WHERE path = 'f/shared/lazy_test' AND kind = 'script' AND source_workspace_id = 'test-workspace'"
)
.fetch_optional(&db)
.await?;
// Should be deleted since the item doesn't actually exist in either workspace
assert!(lazy_test.is_none(), "Non-existent item should be deleted from workspace_diff");
Ok(())
}

View File

@@ -47,6 +47,7 @@ windmill-queue.workspace = true
windmill-common = { workspace = true, default-features = false }
windmill-audit.workspace = true
windmill-parser.workspace = true
windmill-parser-sql.workspace = true
windmill-parser-ts.workspace = true
windmill-parser-py.workspace = true
windmill-parser-py-imports.workspace = true

View File

@@ -1,7 +1,7 @@
openapi: "3.0.3"
info:
version: 1.621.2
version: 1.623.1
title: Windmill API
contact:
@@ -16447,44 +16447,110 @@ paths:
/w/{workspace}/assets/list:
get:
summary: List all assets in the workspace
summary: List all assets in the workspace with cursor pagination
operationId: listAssets
tags:
- asset
parameters:
- $ref: "#/components/parameters/WorkspaceId"
- name: per_page
in: query
description: Number of items per page (max 1000, default 50)
schema:
type: integer
default: 50
minimum: 1
maximum: 1000
- name: cursor_created_at
in: query
description: Cursor timestamp for pagination (created_at of last item from previous page)
schema:
type: string
format: date-time
- name: cursor_id
in: query
description: Cursor ID for pagination (id of last item from previous page)
schema:
type: integer
format: int64
- name: asset_path
in: query
description: Filter by asset path (case-insensitive partial match)
schema:
type: string
- name: usage_path
in: query
description: Filter by usage path (case-insensitive partial match)
schema:
type: string
- name: asset_kinds
in: query
description: Filter by asset kinds (multiple values allowed)
schema:
type: string
responses:
"200":
description: all assets in the workspace
description: paginated assets in the workspace
content:
application/json:
schema:
type: array
items:
type: object
required: [path, kind, usages]
properties:
path:
type: string
kind:
$ref: "#/components/schemas/AssetKind"
usages:
type: array
items:
type: object
required: [path, kind]
properties:
path:
type: string
kind:
$ref: "#/components/schemas/AssetUsageKind"
access_type:
$ref: "#/components/schemas/AssetUsageAccessType"
metadata:
type: object
required: [assets]
properties:
assets:
type: array
items:
type: object
required: [path, kind, usages]
properties:
resource_type:
path:
type: string
kind:
$ref: "#/components/schemas/AssetKind"
usages:
type: array
items:
type: object
required: [path, kind]
properties:
path:
type: string
kind:
$ref: "#/components/schemas/AssetUsageKind"
access_type:
$ref: "#/components/schemas/AssetUsageAccessType"
created_at:
type: string
format: date-time
description: When the asset was detected
metadata:
type: object
properties:
runnable_path:
type: string
description: The path of the script/flow that was run (only present when kind is 'job')
job_kind:
type: string
description: The kind of job (script, flow, preview, etc.) (only present when kind is 'job')
metadata:
type: object
properties:
resource_type:
type: string
description: The type of the resource (only present when kind is 'resource')
next_cursor:
type: object
description: Cursor for the next page (null if no more pages)
nullable: true
properties:
created_at:
type: string
format: date-time
description: Timestamp to use for next page
id:
type: integer
format: int64
description: ID to use for next page
/w/{workspace}/assets/list_by_usages:
post:
@@ -18121,7 +18187,6 @@ components:
- visible_to_owner
- tag
ExportableCompletedJob:
type: object
description: Completed job with full data for export/import operations
@@ -22276,7 +22341,8 @@ components:
properties:
kind:
type: string
enum: ["script", "flow", "app", "resource", "variable"]
enum:
["script", "flow", "app", "resource", "variable", "resource_type"]
description: Type of the item
path:
type: string
@@ -22308,6 +22374,8 @@ components:
- apps_changed
- resources_changed
- variables_changed
- resource_types_changed
- folders_changed
- conflicts
properties:
total_diffs:
@@ -22334,6 +22402,12 @@ components:
variables_changed:
type: integer
description: Number of variables with differences
resource_types_changed:
type: integer
description: Number of resource types with differences
folders_changed:
type: integer
description: Number of folders with differences
conflicts:
type: integer
description: Number of items that are both ahead and behind (conflicts)
@@ -22476,6 +22550,7 @@ components:
enum:
- script
- flow
- job
AssetUsageAccessType:
type: string
enum:
@@ -22580,14 +22655,13 @@ components:
type: object
properties:
service_name:
$ref: '#/components/schemas/NativeServiceName'
$ref: "#/components/schemas/NativeServiceName"
oauth_data:
nullable: true
$ref: '#/components/schemas/WorkspaceOAuthConfig'
$ref: "#/components/schemas/WorkspaceOAuthConfig"
required:
- service_name
WorkspaceOAuthConfig:
type: object
properties:
@@ -22618,7 +22692,7 @@ components:
type: string
enum: [webhook]
request_type:
$ref: '#/components/schemas/WebhookRequestType'
$ref: "#/components/schemas/WebhookRequestType"
required:
- type
- request_type
@@ -22634,12 +22708,10 @@ components:
type: object
properties:
redirect_uri:
type:
string
type: string
required:
- redirect_uri
NativeTriggerData:
type: object
description: Data for creating or updating a native trigger

View File

@@ -902,7 +902,8 @@ pub async fn compute_bundle_secret(db: &DB, w_id: &str, versions: &[i64]) -> Res
.last()
.ok_or_else(|| Error::internal_err("App has no versions".to_string()))?;
let mc = build_crypt(db, w_id).await?;
let hx = hex::encode(mc.encrypt_str_to_bytes(format!("{}{}", BUNDLE_SECRET_PREFIX, version_id)));
let hx =
hex::encode(mc.encrypt_str_to_bytes(format!("{}{}", BUNDLE_SECRET_PREFIX, version_id)));
Ok(hx)
}
@@ -1391,6 +1392,7 @@ async fn delete_app(
deployed_object,
Some(format!("App '{}' deleted", path)),
true,
None,
)
.await?;
@@ -2287,6 +2289,7 @@ async fn upload_s3_file_from_app(
&w_id,
None,
&[(&file_key, S3Permission::WRITE)],
None,
)
.await?;
(s3_resource_opt, file_key, email, permissioned_as, username)
@@ -2319,6 +2322,7 @@ async fn upload_s3_file_from_app(
&w_id,
None,
&[(&file_key, S3Permission::WRITE)],
None,
)
.await?;
@@ -2355,10 +2359,11 @@ async fn upload_s3_file_from_app(
let db_with_opt_authed = DbWithOptAuthed::from_authed(&authed, db.clone(), None);
let (_, s3_resource) = get_workspace_s3_resource_and_check_paths(
&db_with_opt_authed,
Some(&authed),
Some(&authed),
&w_id,
None,
&[(&file_key, S3Permission::WRITE)],
None,
)
.await?;
@@ -2466,10 +2471,11 @@ async fn delete_s3_file_from_app(
let db_with_opt_authed = DbWithOptAuthed::from_authed(&on_behalf_authed, db.clone(), None);
let (_, s3_resource) = get_workspace_s3_resource_and_check_paths(
&db_with_opt_authed,
Some(&on_behalf_authed),
Some(&on_behalf_authed),
&w_id,
None,
&[(&path.to_string(), S3Permission::DELETE)],
None,
)
.await?;
@@ -2639,6 +2645,8 @@ async fn download_s3_file_from_app(
Path((w_id, path)): Path<(String, StripPath)>,
Query(query): Query<AppS3FileQueryWithForceViewerAllowedS3Keys>,
) -> Result<Response> {
use crate::db::OptJobAuthed;
let path = path.to_path();
let force_viewer_allowed_s3_keys = if let Some(force_viewer_allowed_s3_keys) =
@@ -2664,7 +2672,7 @@ async fn download_s3_file_from_app(
.await?;
download_s3_file_internal(
on_behalf_authed,
OptJobAuthed { authed: on_behalf_authed, job_id: None },
&db,
None,
&w_id,

View File

@@ -1,11 +1,16 @@
use axum::{
extract::Path,
extract::{Path, Query},
routing::{get, post},
Extension, Json, Router,
};
use serde::Deserialize;
use serde::{Deserialize, Serialize};
use serde_json::Value;
use windmill_common::{assets::AssetUsageKind, db::UserDB, error::JsonResult};
use sqlx::Row;
use windmill_common::{
assets::{AssetKind, AssetUsageKind},
db::UserDB,
error::JsonResult,
};
use crate::db::ApiAuthed;
@@ -15,21 +20,155 @@ pub fn workspaced_service() -> Router {
.route("/list_by_usages", post(list_assets_by_usages))
}
#[derive(Deserialize)]
struct ListAssetsQuery {
#[serde(default = "default_per_page")]
per_page: i64,
cursor_created_at: Option<chrono::DateTime<chrono::Utc>>,
cursor_id: Option<i64>,
asset_path: Option<String>,
usage_path: Option<String>,
asset_kinds: Option<String>,
}
fn default_per_page() -> i64 {
50
}
#[derive(Serialize)]
struct ListAssetsResponse {
assets: Vec<Value>,
next_cursor: Option<AssetCursor>,
}
#[derive(Serialize)]
struct AssetCursor {
created_at: chrono::DateTime<chrono::Utc>,
id: i64,
}
#[derive(Debug)]
struct AssetRow {
result: Value,
max_created_at: chrono::DateTime<chrono::Utc>,
max_id: i64,
}
async fn list_assets(
authed: ApiAuthed,
Path(w_id): Path<String>,
Extension(user_db): Extension<UserDB>,
) -> JsonResult<Vec<Value>> {
let assets = sqlx::query_scalar!(
r#"SELECT
Query(query): Query<ListAssetsQuery>,
) -> JsonResult<ListAssetsResponse> {
let per_page = query.per_page.min(1000).max(1);
let limit = per_page + 1;
let mut tx = user_db.begin(&authed).await?;
// Build dynamic filter SQL
let mut asset_summary_filters = vec![
"asset.workspace_id = $1".to_string(),
"(asset.usage_kind <> 'flow' OR asset.usage_path = ANY(SELECT path FROM flow WHERE workspace_id = $1))".to_string(),
"(asset.usage_kind <> 'script' OR asset.usage_path = ANY(SELECT path FROM script WHERE workspace_id = $1))".to_string(),
];
let mut param_count = 2; // $1 = workspace_id, $2 = limit
// Asset path filter
if query.asset_path.is_some() {
param_count += 1;
asset_summary_filters.push(format!("asset.path ILIKE ${}", param_count));
}
// Usage path filter - for jobs, also check runnable_path
let needs_job_join_in_cte = query.usage_path.is_some();
if query.usage_path.is_some() {
param_count += 1;
asset_summary_filters.push(format!(
"(asset.usage_path ILIKE ${} OR (asset.usage_kind = 'job' AND job_cte.runnable_path ILIKE ${}))",
param_count, param_count
));
}
// Asset kinds filter
let asset_kinds = query
.asset_kinds
.map(|kinds_str| {
kinds_str
.split(',')
.map(|kind_str| {
serde_json::from_str::<AssetKind>(&format!("\"{}\"", kind_str.trim()))
})
.collect::<Result<Vec<_>, _>>()
})
.transpose()
.map_err(|_| {
windmill_common::error::Error::BadRequest("Invalid asset_kinds parameter".to_string())
})?;
let has_asset_kinds = asset_kinds.as_ref().map(|v| !v.is_empty()).unwrap_or(false);
if has_asset_kinds {
param_count += 1;
asset_summary_filters.push(format!("asset.kind = ANY(${})", param_count));
}
let asset_summary_where = asset_summary_filters.join(" AND ");
// Build cursor condition
let cursor_having = if query.cursor_created_at.is_some() && query.cursor_id.is_some() {
param_count += 2;
format!("HAVING MAX(asset.created_at) < ${} OR (MAX(asset.created_at) = ${} AND MAX(asset.id) < ${})",
param_count - 1, param_count - 1, param_count)
} else {
String::new()
};
// Build FROM clause for CTE with optional job join
let cte_from = if needs_job_join_in_cte {
format!(
r#"FROM asset
LEFT JOIN v2_job job_cte ON asset.usage_kind = 'job'
AND asset.usage_path = job_cte.id::text
AND job_cte.workspace_id = $1"#
)
} else {
"FROM asset".to_string()
};
let sql = format!(
r#"
WITH asset_summary AS (
SELECT
asset.path,
asset.kind,
MAX(asset.created_at) as max_created_at,
MAX(asset.id) as max_id
{}
WHERE {}
GROUP BY asset.path, asset.kind
{}
ORDER BY max_created_at DESC, max_id DESC
LIMIT $2
)
SELECT
jsonb_strip_nulls(jsonb_build_object(
'path', asset.path,
'kind', asset.kind,
'usages', ARRAY_AGG(jsonb_build_object(
'path', asset.usage_path,
'kind', asset.usage_kind,
'access_type', asset.usage_access_type
)),
'usages', ARRAY_AGG(
jsonb_strip_nulls(jsonb_build_object(
'path', asset.usage_path,
'kind', asset.usage_kind,
'access_type', asset.usage_access_type,
'created_at', asset.created_at,
'metadata', (CASE
WHEN asset.usage_kind = 'job' THEN
jsonb_build_object('runnable_path', job.runnable_path, 'job_kind', job.kind)
ELSE
NULL
END
)
))
ORDER BY asset.created_at DESC
),
'metadata', (CASE
WHEN asset.kind = 'resource' THEN
jsonb_build_object('resource_type', resource.resource_type)
@@ -37,23 +176,72 @@ async fn list_assets(
NULL
END
)
)) as "list!: _"
)) as result,
asset_summary.max_created_at,
asset_summary.max_id
FROM asset
INNER JOIN asset_summary ON asset.path = asset_summary.path AND asset.kind = asset_summary.kind
LEFT JOIN resource ON asset.kind = 'resource'
AND array_to_string((string_to_array(asset.path, '/'))[1:3], '/') = resource.path -- With specific table, asset path can be e.g u/diego/pg_db/table_name
AND array_to_string((string_to_array(asset.path, '/'))[1:3], '/') = resource.path
AND resource.workspace_id = $1
LEFT JOIN v2_job job ON asset.usage_kind = 'job'
AND asset.usage_path = job.id::text
AND job.workspace_id = $1
WHERE asset.workspace_id = $1
AND (asset.kind <> 'resource' OR resource.path IS NOT NULL)
AND (asset.usage_kind <> 'flow' OR asset.usage_path = ANY(SELECT path FROM flow WHERE workspace_id = $1))
AND (asset.usage_kind <> 'script' OR asset.usage_path = ANY(SELECT path FROM script WHERE workspace_id = $1))
GROUP BY asset.path, asset.kind, resource.resource_type
ORDER BY asset.path, asset.kind"#,
w_id,
)
.fetch_all(&mut *user_db.begin(&authed).await?)
.await?;
AND (asset.usage_kind <> 'job' OR job.id IS NOT NULL)
GROUP BY asset.path, asset.kind, resource.resource_type, asset_summary.max_created_at, asset_summary.max_id
ORDER BY asset_summary.max_created_at DESC, asset_summary.max_id DESC
"#,
cte_from, asset_summary_where, cursor_having
);
Ok(Json(assets))
// Build query with dynamic parameters
let mut query_builder = sqlx::query(&sql).bind(&w_id).bind(limit);
if let Some(ref asset_path) = query.asset_path {
query_builder = query_builder.bind(format!("%{}%", asset_path));
}
if let Some(ref usage_path) = query.usage_path {
query_builder = query_builder.bind(format!("%{}%", usage_path));
}
if let Some(ref asset_kinds) = asset_kinds {
if !asset_kinds.is_empty() {
query_builder = query_builder.bind(asset_kinds);
}
}
if let (Some(cursor_created_at), Some(cursor_id)) = (query.cursor_created_at, query.cursor_id) {
query_builder = query_builder.bind(cursor_created_at).bind(cursor_id);
}
let db_rows = query_builder.fetch_all(&mut *tx).await?;
let rows: Vec<AssetRow> = db_rows
.iter()
.map(|row| AssetRow {
result: row.try_get("result").unwrap_or(Value::Null),
max_created_at: row.try_get("max_created_at").unwrap(),
max_id: row.try_get("max_id").unwrap(),
})
.collect();
let assets: Vec<Value> = rows
.iter()
.take(per_page as usize)
.map(|r| r.result.clone())
.collect();
let next_cursor = if rows.len() as i64 > per_page {
let last = &rows[per_page as usize - 1];
Some(AssetCursor { created_at: last.max_created_at, id: last.max_id })
} else {
None
};
Ok(Json(ListAssetsResponse { assets, next_cursor }))
}
#[derive(Deserialize)]

View File

@@ -13,10 +13,13 @@ use sqlx::FromRow;
use tower_cookies::Cookies;
use tracing::Span;
use crate::db::{ApiAuthed, DB};
use std::sync::{
atomic::{AtomicI64, AtomicU64, Ordering},
Arc,
use crate::db::{ApiAuthed, OptJobAuthed, DB};
use std::{
str::FromStr,
sync::{
atomic::{AtomicI64, AtomicU64, Ordering},
Arc,
},
};
#[cfg(feature = "enterprise")]
use tokio::sync::RwLock;
@@ -47,6 +50,7 @@ pub fn invalidate_token_from_cache(token: &str) {
pub struct ExpiringAuthCache {
pub authed: ApiAuthed,
pub expiry: chrono::DateTime<chrono::Utc>,
pub job_id: Option<uuid::Uuid>,
}
pub struct AuthCache {
@@ -75,14 +79,22 @@ impl AuthCache {
}
pub async fn get_authed(&self, w_id: Option<String>, token: &str) -> Option<ApiAuthed> {
Some(self.get_opt_job_authed(w_id, token).await?.authed)
}
pub async fn get_opt_job_authed(
&self,
w_id: Option<String>,
token: &str,
) -> Option<OptJobAuthed> {
let key = (
w_id.as_ref().unwrap_or(&"".to_string()).to_string(),
token.to_string(),
);
let s = AUTH_CACHE.get(&key).map(|c| c.to_owned());
match s {
Some(ExpiringAuthCache { authed, expiry }) if expiry > chrono::Utc::now() => {
Some(authed)
Some(ExpiringAuthCache { authed, expiry, job_id }) if expiry > chrono::Utc::now() => {
Some(OptJobAuthed { authed, job_id })
}
#[cfg(feature = "enterprise")]
_ if token.starts_with("jwt_ext_") => {
@@ -101,16 +113,17 @@ impl AuthCache {
}
};
if let Some((authed, exp)) = authed_and_exp.clone() {
if let Some((authed, exp, job_id)) = authed_and_exp.clone() {
AUTH_CACHE.insert(
key,
ExpiringAuthCache {
authed: authed.clone(),
expiry: chrono::Utc.timestamp_nanos(exp as i64 * 1_000_000_000),
job_id,
},
);
Some(authed)
Some(OptJobAuthed { authed, job_id })
} else {
None
}
@@ -127,6 +140,7 @@ impl AuthCache {
return None;
}
let username_override = username_override_from_label(claims.label);
let authed = crate::db::ApiAuthed {
email: claims.email,
username: claims.username,
@@ -138,17 +152,18 @@ impl AuthCache {
username_override,
token_prefix: claims.audit_span,
};
let job_id = claims.job_id.and_then(|j| uuid::Uuid::from_str(&j).ok());
AUTH_CACHE.insert(
key,
ExpiringAuthCache {
authed: authed.clone(),
expiry: chrono::Utc
.timestamp_nanos(claims.exp as i64 * 1_000_000_000),
job_id,
},
);
Some(authed)
Some(OptJobAuthed { authed, job_id })
}
Err(err) => {
tracing::error!("JWT auth error: {:?}", err);
@@ -353,17 +368,18 @@ impl AuthCache {
authed: authed.clone(),
expiry: chrono::Utc::now()
+ chrono::Duration::try_seconds(120).unwrap(),
job_id: None,
},
);
}
authed_o
authed_o.map(|authed| OptJobAuthed { authed, job_id: None })
} else if self
.superadmin_secret
.as_ref()
.map(|x| x == token)
.unwrap_or(false)
{
Some(ApiAuthed {
let authed = ApiAuthed {
email: SUPERADMIN_SECRET_EMAIL.to_string(),
username: "superadmin_secret".to_string(),
is_admin: true,
@@ -373,7 +389,8 @@ impl AuthCache {
scopes: None,
username_override: None,
token_prefix: Some(token[0..TOKEN_PREFIX_LEN].to_string()),
})
};
Some(OptJobAuthed { authed, job_id: None })
} else {
None
}
@@ -557,14 +574,30 @@ where
{
type Rejection = Error;
async fn from_request_parts(
parts: &mut Parts,
state: &S,
) -> std::result::Result<Self, Self::Rejection> {
let opt_job_authed = OptJobAuthed::from_request_parts(parts, state).await?;
Ok(opt_job_authed.authed)
}
}
#[async_trait]
impl<S> FromRequestParts<S> for OptJobAuthed
where
S: Send + Sync,
{
type Rejection = Error;
async fn from_request_parts(
parts: &mut Parts,
state: &S,
) -> std::result::Result<Self, Self::Rejection> {
if parts.method == http::Method::OPTIONS {
return Ok(ApiAuthed::default());
return Ok(OptJobAuthed::default());
};
let already_authed = parts.extensions.get::<ApiAuthed>();
let already_authed = parts.extensions.get::<OptJobAuthed>();
if let Some(authed) = already_authed {
return Ok(authed.clone());
@@ -588,7 +621,10 @@ where
let path_vec: Vec<&str> = original_uri.path().split("/").collect();
let workspace_id = maybe_get_workspace_id_from_path(&path_vec);
if let Some(mut authed) = cache.get_authed(workspace_id.clone(), &token).await {
if let Some(mut opt_job_authed) =
cache.get_opt_job_authed(workspace_id.clone(), &token).await
{
let authed = &mut opt_job_authed.authed;
if authed.scopes.is_some() {
transform_old_scope_to_new_scope(authed.scopes.as_mut());
@@ -612,7 +648,7 @@ where
if let Some(workspace_id) = workspace_id {
Span::current().record("workspace_id", &workspace_id);
}
return Ok(authed);
return Ok(opt_job_authed);
}
}
}

View File

@@ -30,7 +30,11 @@ async fn current_database(conn: &mut PgConnection) -> Result<String, MigrateErro
}
lazy_static::lazy_static! {
pub static ref OVERRIDDEN_MIGRATIONS: std::collections::HashMap<i64, String> = vec![(20221207103910, include_str!(
pub static ref OVERRIDDEN_MIGRATIONS: std::collections::HashMap<i64, String> = vec![(20220123221903, include_str!(
"../../migrations/20220123221903_first.up.sql"
).replace("create SCHEMA IF NOT exists extensions;", "")
.replace("create extension if not exists \"uuid-ossp\" with schema extensions;", "")),
(20221207103910, include_str!(
"../../custom_migrations/create_workspace_without_md5.sql"
).to_string()),
(20240216100535, include_str!(
@@ -254,6 +258,12 @@ pub async fn migrate(
Ok(None)
}
#[derive(Default, Clone, Debug)]
pub struct OptJobAuthed {
pub job_id: Option<uuid::Uuid>,
pub authed: ApiAuthed,
}
#[derive(Clone, Debug, Default, Hash, Eq, PartialEq)]
pub struct ApiAuthed {
pub email: String,

View File

@@ -27,7 +27,7 @@ pub async fn jwt_ext_auth(
_token: &str,
_external_jwks: Option<Arc<RwLock<ExternalJwks>>>,
_db: &crate::db::DB,
) -> anyhow::Result<(crate::db::ApiAuthed, usize)> {
) -> anyhow::Result<(crate::db::ApiAuthed, usize, Option<uuid::Uuid>)> {
// Implementation is not open source
Err(anyhow!("External JWT auth is not open source"))

View File

@@ -32,11 +32,12 @@ use sql_builder::prelude::*;
use sqlx::{FromRow, Postgres, Transaction};
use windmill_audit::audit_oss::audit_log;
use windmill_audit::ActionKind;
use windmill_common::runnable_settings::RunnableSettingsTrait;
use windmill_common::utils::query_elems_from_hub;
use windmill_common::assets::{clear_static_asset_usage, AssetUsageKind};
use windmill_common::min_version::{
MIN_VERSION_SUPPORTS_DEBOUNCING, MIN_VERSION_SUPPORTS_DEBOUNCING_V2,
};
use windmill_common::runnable_settings::RunnableSettingsTrait;
use windmill_common::utils::query_elems_from_hub;
use windmill_common::worker::{to_raw_value, CLOUD_HOSTED};
use windmill_common::HUB_BASE_URL;
use windmill_common::{
@@ -1432,13 +1433,7 @@ async fn archive_flow_by_path(
.execute(&mut *tx)
.await?;
sqlx::query!(
"DELETE FROM asset WHERE workspace_id = $1 AND usage_kind = 'flow' AND usage_path = $2",
&w_id,
path
)
.execute(&mut *tx)
.await?;
clear_static_asset_usage(&mut *tx, &w_id, path, AssetUsageKind::Flow).await?;
audit_log(
&mut *tx,
@@ -1476,6 +1471,7 @@ async fn archive_flow_by_path(
}
)),
true,
None,
)
.await?;
@@ -1591,6 +1587,7 @@ async fn delete_flow_by_path(
},
Some(format!("Flow '{}' deleted", path)),
true,
None,
)
.await?;

View File

@@ -287,6 +287,7 @@ async fn create_folder(
DeployedObject::Folder { path: format!("f/{}", ng.name) },
Some(format!("Folder '{}' created", ng.name)),
true,
None,
)
.await?;
@@ -467,6 +468,7 @@ async fn update_folder(
DeployedObject::Folder { path: format!("f/{}", name) },
Some(format!("Folder '{}' updated", name)),
true,
None,
)
.await?;
@@ -662,6 +664,7 @@ async fn delete_folder(
DeployedObject::Folder { path: format!("f/{}", name) },
Some(format!("Folder '{}' deleted", name)),
true,
None,
)
.await?;

View File

@@ -169,6 +169,7 @@ async fn add_granular_acl(
DeployedObject::Folder { path: format!("f/{}", path) },
Some(format!("Folder '{}' changed permissions", path)),
true,
None,
)
.await?
}
@@ -311,6 +312,7 @@ async fn remove_granular_acl(
DeployedObject::Folder { path: format!("f/{}", path) },
Some(format!("Folder '{}' changed permissions", path)),
true,
None,
)
.await?
}

View File

@@ -291,6 +291,7 @@ async fn create_group(
windmill_git_sync::DeployedObject::Group { name: ng.name.clone() },
Some(format!("Created group '{}'", &ng.name)),
true,
None,
)
.await?;
@@ -514,6 +515,7 @@ async fn delete_group(
windmill_git_sync::DeployedObject::Group { name: name.clone() },
Some(format!("Deleted group '{}'", &name)),
true,
None,
)
.await?;
@@ -573,6 +575,7 @@ async fn update_group(
windmill_git_sync::DeployedObject::Group { name: name.clone() },
Some(format!("Updated group '{}'", &name)),
true,
None,
)
.await?;
@@ -637,6 +640,7 @@ async fn add_user(
windmill_git_sync::DeployedObject::Group { name: name.clone() },
Some(format!("Added user to group '{}'", &name)),
true,
None,
)
.await?;
@@ -914,6 +918,7 @@ async fn remove_user(
windmill_git_sync::DeployedObject::Group { name: name.clone() },
Some(format!("Removed user from group '{}'", &name)),
true,
None,
)
.await?;

Some files were not shown because too many files have changed in this diff Show More