Compare commits

...

18 Commits

Author SHA1 Message Date
Pyra
f40a3b420b Merge branch 'main' into debouncing-tests 2026-03-06 14:42:25 +01:00
centdix
c40ad129bc rename config file (#8230) 2026-03-06 05:03:41 +00:00
wendrul
7859bca6ae fix: cli: support deleting linked resources-variables without throwing (#8248) 2026-03-05 20:09:59 +00:00
wendrul
1ac391a795 fix: wmill workspace whoami output (#8246) 2026-03-05 18:12:21 +00:00
Diego Imbert
5d79f33590 Final Svelte 5 migration (#8211)
* Remove $$props.field usage

* Rename slots to ensure no hyphen

* _props

* _trigger

* OnSelectedIteration type correct capitalization

* rename _content

* Remove afterUpdate

* Migrate everything to svelte 5

* array bind

* Fix popover

* type never

* nit fixes

* Fixed many trivial errors

* onClick

* Fix errors

* use let:

* nit typing

* fix: wrap state_referenced_locally vars with untrack()

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Add untrack import

* Fix all syntax errors due to untrack migration

* Fix undefined errors

* Fix more undefined errors

* untrack(() => initialOpen)

* svelte-ignore

* Fix state_descriptors_fixed error in Chart.svelte

Use $state.snapshot() to pass plain copies of data/options to Chart.js
instead of $state proxies. Chart.js's listenArrayEvents tries to define
property descriptors on data arrays, which Svelte 5 proxies reject.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* nit typing

* Merge issue

* Fix "path is not set" error in resource picker / editor

* Fix InputTransformForm error when rerunning some flows

* fix npm run check

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-05 18:11:40 +01:00
Ruben Fiszel
86065aaac8 chore(main): release 1.651.1 (#8242)
* chore(main): release 1.651.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-03-05 14:51:38 +00:00
Ruben Fiszel
e3f4130c68 nits 2026-03-05 14:36:51 +00:00
Ruben Fiszel
2e582b1bc1 fix: prevent slow loading toast interval from leaking on promise cancellation (#8240)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 14:23:36 +00:00
Ruben Fiszel
2d583826dc fix: suppress unused variable warnings on windows builds (#8241)
* fix: suppress unused variable warnings on windows builds

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref.txt to merged commit

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 15:17:51 +01:00
Ruben Fiszel
972ae7aa29 chore(main): release 1.651.0 (#8235)
* chore(main): release 1.651.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-03-05 13:42:07 +00:00
Ruben Fiszel
d46913b74a fix: write fallback package.json for codebase mode nsjail (#8239)
* fix: write fallback package.json for codebase mode to fix nsjail ERR_INVALID_PACKAGE_CONFIG

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: add e2e tests for codebase mode with and without nsjail

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-05 13:35:53 +00:00
Roderik-WU
90f4c64ee1 fix(python-client): add delete_s3_object (#8216)
* Implement remove_s3_file method

Add method to permanently delete a file from S3 bucket.

* Add test for removing S3 file

Added a test case to verify removal of a file from S3.

* Add remove_s3_file function to delete S3 files

Added a function to permanently delete a file from the S3 bucket.

* Rename remove_s3_file to remove_3_object

* Rename remove_3_object to remove_s3_object

* Rename test method and update S3 object handling

* Rename remove_s3_object to delete_s3_object

* Rename test_remove_s3_object to test_delete_s3_object and remove_s3_object to delete_s3_object
2026-03-05 12:49:59 +00:00
hugocasa
a8cbe9396f fix: update CLI bun template to match UI template (#8238)
* fix: update CLI bun template to match UI template

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: simplify CLI bun template, only add mode comments

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 12:04:38 +00:00
centdix
ce041e8a5e feat: hash-based MCP tool names for long paths (#8133)
* feat: replace _TRUNC with hash-based MCP tool names (50 char limit)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: reduce MCP tool name limit from 50 to 40 chars

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: use path prefix filtering instead of separate DB query for hashed name resolution

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: remove long path warning from MCP token creation (hashing handles long names)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: unify tool prefix parsing and fix extract_path_prefix_from_hashed for Hs- names

- Replace `is_hashed_name` + `parse_hashed_name` with unified `parse_tool_prefix`
  that returns `(type_str, is_hub, is_hashed)` in one call
- Fix `extract_path_prefix_from_hashed` to dynamically determine prefix length
  (3 for `Hs-`, 2 for `S-`/`F-`) instead of hardcoding index 2
- Simplify `reverse_transform` to reuse `parse_tool_prefix`
- Add tests for invalid prefixes and `Hs-` prefix handling

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: escape LIKE wildcards in MCP hashed name path prefix query

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: respect favorites scope in hashed tool name resolution

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: deduplicate MCP tool name resolution and rename get_path_or_id

- Extract `unescape_path` helper in transform.rs to deduplicate the
  3-step placeholder unescape logic
- Extract `find_matching_path` helper in runner.rs to deduplicate
  script/flow candidate matching via ToolableItem trait
- Remove verbose tracing::info! logs from hashed tool resolution hot path
- Fix doc comment referencing nonexistent `is_hashed_name` function
- Rename `get_path_or_id` to `get_transformed_path` for clarity

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: update stale doc comments to reflect MAX_PATH_LENGTH=40

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-05 12:04:20 +00:00
Ruben Fiszel
65082159d8 tighten volume limits (#8236)
* feat: add volume limits info in CE volumes drawer

Show an info alert in the volumes drawer when running in Community
Edition, mentioning the 20 volumes per workspace and 50 MB per file
limits. Update ee-repo-ref for companion EE changes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref to a61366dd4d9e9b1f98a421aaa6d3f63194615275

This commit updates the EE repository reference after PR #438 was merged in windmill-ee-private.

Previous ee-repo-ref: 05385738e36e81f5bc51d15c0ca60bba30457c21

New ee-repo-ref: a61366dd4d9e9b1f98a421aaa6d3f63194615275

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-05 06:44:32 +00:00
Ruben Fiszel
5f0ef936d1 feat: add sandbox annotations, volume mounts, for AI sandbox starting with claude (#8058) 2026-03-05 06:19:51 +00:00
Pyra
993fbcde59 Merge branch 'main' into debouncing-tests 2026-03-04 17:40:57 +01:00
pyranota
b1142421b8 nit: add more tests
Signed-off-by: pyranota <pyra@duck.com>
2026-03-04 17:39:54 +01:00
581 changed files with 10282 additions and 3238 deletions

View File

@@ -1,5 +1,28 @@
# Changelog
## [1.651.1](https://github.com/windmill-labs/windmill/compare/v1.651.0...v1.651.1) (2026-03-05)
### Bug Fixes
* prevent slow loading toast interval from leaking on promise cancellation ([#8240](https://github.com/windmill-labs/windmill/issues/8240)) ([2e582b1](https://github.com/windmill-labs/windmill/commit/2e582b1bc1c299388a3c97cfddff9d0eb92858f2))
* suppress unused variable warnings on windows builds ([#8241](https://github.com/windmill-labs/windmill/issues/8241)) ([2d58382](https://github.com/windmill-labs/windmill/commit/2d583826dc065c05684d4cd1d1510f0d1f2d9ae9))
## [1.651.0](https://github.com/windmill-labs/windmill/compare/v1.650.0...v1.651.0) (2026-03-05)
### Features
* add sandbox annotations, volume mounts, for AI sandbox starting with claude ([#8058](https://github.com/windmill-labs/windmill/issues/8058)) ([5f0ef93](https://github.com/windmill-labs/windmill/commit/5f0ef936d1d5d07d01c8e07e26ec254feebef8fb))
* hash-based MCP tool names for long paths ([#8133](https://github.com/windmill-labs/windmill/issues/8133)) ([ce041e8](https://github.com/windmill-labs/windmill/commit/ce041e8a5e7ff105df389875d9981f3843d4ce39))
### Bug Fixes
* **python-client:** add delete_s3_object ([#8216](https://github.com/windmill-labs/windmill/issues/8216)) ([90f4c64](https://github.com/windmill-labs/windmill/commit/90f4c64ee12e1d04ce846ff88d6658f667e194e0))
* update CLI bun template to match UI template ([#8238](https://github.com/windmill-labs/windmill/issues/8238)) ([a8cbe93](https://github.com/windmill-labs/windmill/commit/a8cbe9396ffc51140dce5582d57f4dc59873304e))
* write fallback package.json for codebase mode nsjail ([#8239](https://github.com/windmill-labs/windmill/issues/8239)) ([d46913b](https://github.com/windmill-labs/windmill/commit/d46913b74a0ffd41d2323e0355cc81954f09e29d))
## [1.650.0](https://github.com/windmill-labs/windmill/compare/v1.649.0...v1.650.0) (2026-03-05)

View File

@@ -262,6 +262,12 @@ COPY --from=oven/bun:1.3.10 /usr/local/bin/bun /usr/bin/bun
RUN bun install -g windmill-cli \
&& ln -s $(bun pm bin -g)/wmill /usr/bin/wmill
# Install Claude Code CLI (used by claude sandbox scripts)
# The installer puts the binary in ~/.local/bin/claude (symlink to ~/.local/share/claude/versions/*)
# Copy it to /usr/bin/claude so it's accessible inside nsjail sandbox (which mounts /usr but not /root)
RUN curl -fsSL https://claude.ai/install.sh | bash \
&& cp /root/.local/share/claude/versions/* /usr/bin/claude
COPY --from=php:8.3.7-cli /usr/local/bin/php /usr/bin/php
COPY --from=composer:2.7.6 /usr/bin/composer /usr/bin/composer

View File

@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE volume SET lease_until = now() + interval '60 seconds'\n WHERE workspace_id = $1 AND name = $2 AND leased_by = $3 AND lease_until > now()",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "00bf3dbd9d3f51dd7fdefcbd654d55e0379cc84188954037165cbe2d198ef71f"
}

View File

@@ -1,16 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT token\n FROM token\n WHERE token LIKE concat($1::text, '%')\n LIMIT 1\n ",
"query": "SELECT group_ FROM usr_to_group WHERE usr = $1 AND workspace_id = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "token",
"name": "group_",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
@@ -18,5 +19,5 @@
false
]
},
"hash": "90092c0b3f7612373fcc8fb7a966200118ab308430d4a0cbb5cb16c397246492"
"hash": "015a8551c646f9b027fc23752c5c5c81e520e3ca97dd1cd1e4ebfe3e46c4ad11"
}

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT large_file_storage->>'volume_storage' FROM workspace_settings WHERE workspace_id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "?column?",
"type_info": "Text"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
null
]
},
"hash": "083d69abc8a662bb364cf43b8ffc6e9b159a54c179cecb108068597536835f7e"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT extra_perms FROM volume WHERE workspace_id = $1 AND name = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "extra_perms",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false
]
},
"hash": "0afd4ae50ff7e1b0dcca4b483816c595401dd2e1f7699a28bf3b79db5e3841f4"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT created_by FROM volume WHERE name = $1 AND workspace_id = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "created_by",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false
]
},
"hash": "0eb54f04a8185085b3f80772f5c28e666f6fbd1ec5ee9d30ee0cdb5e30a68750"
}

View File

@@ -0,0 +1,25 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO volume (workspace_id, name, size_bytes, created_by, lease_until, leased_by)\n VALUES ($1, $2, 0, $3, now() + interval '60 seconds', $4)\n ON CONFLICT (workspace_id, name) DO UPDATE\n SET lease_until = now() + interval '60 seconds', leased_by = $4\n WHERE volume.lease_until IS NULL OR volume.lease_until < now()\n RETURNING name",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "name",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar",
"Varchar"
]
},
"nullable": [
false
]
},
"hash": "14004a7c1641a3157eddd571fea11a1dfb1422187200119268b2342b47a960c6"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE volume SET last_used_at = now() WHERE workspace_id = $1 AND name = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": []
},
"hash": "1d2f765c2a71e1154ca5d9f5e52ef31e6d647377d37747f7bdc834748a59419e"
}

View File

@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO volume (workspace_id, name, size_bytes, created_by, last_used_at)\n VALUES ($1, $2, $3, $4, now())\n ON CONFLICT (workspace_id, name) DO UPDATE\n SET size_bytes = $3, last_used_at = now()",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Int8",
"Varchar"
]
},
"nullable": []
},
"hash": "1e9b9a02f45e6200f4d101bd5336fc8ce983f857339e6fccf799dc6587964aab"
}

View File

@@ -0,0 +1,25 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO volume (workspace_id, name, size_bytes, created_by, lease_until, leased_by)\n VALUES ($1, $2, 0, $3, now() + interval '60 seconds', $4)\n ON CONFLICT (workspace_id, name) DO UPDATE\n SET lease_until = now() + interval '60 seconds', leased_by = $4\n WHERE volume.lease_until IS NULL OR volume.lease_until < now()\n RETURNING name",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "name",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar",
"Varchar"
]
},
"nullable": [
false
]
},
"hash": "23f47f5207abe0cfaede197aeee485957990eb92fa3ce515895eab0d3f28bfdc"
}

View File

@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE volume SET lease_until = NULL, leased_by = NULL\n WHERE workspace_id = $1 AND name = $2 AND leased_by = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "28df7bbe1f54f69640bc76def9e580b4c7ba25f279644e3233b63f4f6db0ad98"
}

View File

@@ -0,0 +1,18 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE volume\n SET size_bytes = $3, file_count = $4,\n updated_at = now(), updated_by = $5, last_used_at = now(),\n lease_until = NULL, leased_by = NULL\n WHERE workspace_id = $1 AND name = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Int8",
"Int4",
"Varchar"
]
},
"nullable": []
},
"hash": "3955e57e216d169c30b1548a2252eb169329116cba57780fa90ecf2bdb910f34"
}

View File

@@ -0,0 +1,76 @@
{
"db_name": "PostgreSQL",
"query": "SELECT\n name as \"name!\",\n size_bytes as \"size_bytes!\",\n file_count as \"file_count!\",\n created_at as \"created_at!\",\n created_by as \"created_by!\",\n updated_at,\n updated_by,\n description as \"description!\",\n last_used_at,\n extra_perms as \"extra_perms!\"\n FROM (\n SELECT\n COALESCE(v.name, a.path) as name,\n COALESCE(v.size_bytes, 0) as size_bytes,\n COALESCE(v.file_count, 0) as file_count,\n COALESCE(v.created_at, a.min_created_at) as created_at,\n COALESCE(v.created_by, 'unknown') as created_by,\n v.updated_at,\n v.updated_by,\n COALESCE(v.description, '') as description,\n v.last_used_at,\n COALESCE(v.extra_perms, '{}'::jsonb) as extra_perms\n FROM (\n SELECT path, MIN(created_at) as min_created_at\n FROM asset\n WHERE workspace_id = $1 AND kind = 'volume'\n GROUP BY path\n ) a\n FULL OUTER JOIN volume v ON v.workspace_id = $1 AND v.name = a.path\n WHERE v.workspace_id = $1 OR a.path IS NOT NULL\n ) combined\n ORDER BY name",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "name!",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "size_bytes!",
"type_info": "Int8"
},
{
"ordinal": 2,
"name": "file_count!",
"type_info": "Int4"
},
{
"ordinal": 3,
"name": "created_at!",
"type_info": "Timestamptz"
},
{
"ordinal": 4,
"name": "created_by!",
"type_info": "Varchar"
},
{
"ordinal": 5,
"name": "updated_at",
"type_info": "Timestamptz"
},
{
"ordinal": 6,
"name": "updated_by",
"type_info": "Varchar"
},
{
"ordinal": 7,
"name": "description!",
"type_info": "Text"
},
{
"ordinal": 8,
"name": "last_used_at",
"type_info": "Timestamptz"
},
{
"ordinal": 9,
"name": "extra_perms!",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
null,
null,
null,
null,
null,
true,
true,
null,
true,
null
]
},
"hash": "40d0f6dca30456514cb85e36c6e367b27171894016c714e41497e69115be1468"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM volume WHERE workspace_id = $1 AND name = $2\n AND (lease_until IS NULL OR lease_until < now())\n RETURNING name",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "name",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false
]
},
"hash": "5af44b46a2e2f1a9adeb39013790be7046cf8789d842717b6c793c22a2a05daa"
}

View File

@@ -0,0 +1,29 @@
{
"db_name": "PostgreSQL",
"query": "SELECT created_by, extra_perms FROM volume WHERE workspace_id = $1 AND name = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "created_by",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "extra_perms",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
false
]
},
"hash": "6086849bb08e1b37d6693d2808767cd897dca4722e4f2076308afdb7ee9fc147"
}

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT count(*) FROM volume WHERE workspace_id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "count",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
null
]
},
"hash": "712092e5033bc6894025a55ebc58bca8450d09982e582266d215dff521256fa6"
}

View File

@@ -0,0 +1,18 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE volume\n SET size_bytes = $3, file_count = $4,\n updated_at = now(), last_used_at = now(),\n lease_until = NULL, leased_by = NULL\n WHERE workspace_id = $1 AND name = $2 AND leased_by = $5",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Int8",
"Int4",
"Text"
]
},
"nullable": []
},
"hash": "75a03e9e4cba350a104e2e3a95de919cd25538c0b433bc29bb052c7a7b8568ca"
}

View File

@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE volume SET lease_until = now() + interval '60 seconds'\n WHERE workspace_id = $1 AND name = $2 AND leased_by = $3 AND lease_until > now()",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "769035629df5a5034f64bf38992e142006825a3911addacdf1a026660b5e2b7f"
}

View File

@@ -0,0 +1,24 @@
{
"db_name": "PostgreSQL",
"query": "SELECT EXISTS(SELECT 1 FROM volume WHERE workspace_id = $1 AND name = $2 AND lease_until > now() AND leased_by = $3)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "exists",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": [
null
]
},
"hash": "78af8bdb6a3ee6396c54f87ff6403b566fc75e16e0b7a81204816fd50b3346a5"
}

View File

@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE volume SET lease_until = NULL, leased_by = NULL\n WHERE workspace_id = $1 AND name = $2 AND leased_by = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "7ce06d4f623932fce12352be3a09ba8973a2ef1defa36c6d46d9c1c6406a7c33"
}

View File

@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO volume (workspace_id, name, size_bytes, created_by)\n VALUES ($1, $2, $3, $4)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Int8",
"Varchar"
]
},
"nullable": []
},
"hash": "7e8e79a7d140be511cedbfe9ff8eea76a8a3079ce80c035087f797cdc410f35b"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT last_used_at FROM volume WHERE workspace_id = $1 AND name = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "last_used_at",
"type_info": "Timestamptz"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
true
]
},
"hash": "803abdcd3614437b26c5d2e4f1ad75ca7014b431239ac1b681f2b26380c719c4"
}

View File

@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE volume SET extra_perms = extra_perms - $1\n WHERE workspace_id = $2 AND name = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "82b3bd95e5d28c4cd4eedcae8cf050ba7b7e4d9eabba03be251ae9a8017b317d"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT leased_by FROM volume WHERE workspace_id = $1 AND name = $2 AND lease_until > now()",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "leased_by",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
true
]
},
"hash": "88e25dc24bb06237b3677c947ee53fd6e9c7606231ad3c522e98cb1fcc14361a"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT count(*) FROM volume WHERE workspace_id = $1 AND name = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "count",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
null
]
},
"hash": "907241c195fea227e4a945ee472425e5f7600e28c728a06235f7ff430a4bd77a"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT size_bytes FROM volume WHERE workspace_id = $1 AND name = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "size_bytes",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false
]
},
"hash": "94d6f598076ad67d68e6f01926c9fc2c73e855790e17abf5461b96ea30fbbdb7"
}

View File

@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE volume SET lease_until = NULL, leased_by = NULL\n WHERE workspace_id = $1 AND name = $2 AND leased_by = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "9662f1e304124fa52db4aa1e80e03b2601630f2d31458bdaf70c2702b2998d89"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM volume WHERE workspace_id = $1 AND name = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": []
},
"hash": "9e30b5545a51453205a713a6276156ada29ae320465d9790dce7e1e8a436d4de"
}

View File

@@ -0,0 +1,29 @@
{
"db_name": "PostgreSQL",
"query": "SELECT extra_perms, created_by FROM volume WHERE workspace_id = $1 AND name = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "extra_perms",
"type_info": "Jsonb"
},
{
"ordinal": 1,
"name": "created_by",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
false
]
},
"hash": "9f64d6ed0adb609ced1551563062550919fcac56deaf1b3cb36b3e15117936e7"
}

View File

@@ -0,0 +1,24 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO volume (workspace_id, name, size_bytes, created_by)\n VALUES ($1, $2, 0, $3)\n ON CONFLICT (workspace_id, name) DO NOTHING\n RETURNING name",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "name",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar"
]
},
"nullable": [
false
]
},
"hash": "a3970c15271a124307301c0dafa263e7168fa325c5ceb44e9dd1595bdb7e7ce6"
}

View File

@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO volume (workspace_id, name, size_bytes, created_by)\n VALUES ($1, $2, $3, $4)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Int8",
"Varchar"
]
},
"nullable": []
},
"hash": "ab8daa93bc66d0142b9e9e8d7fa6719fc41b2ca5cb0b7ac5ad73ab01b650c935"
}

View File

@@ -0,0 +1,29 @@
{
"db_name": "PostgreSQL",
"query": "SELECT size_bytes, last_used_at FROM volume WHERE workspace_id = $1 AND name = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "size_bytes",
"type_info": "Int8"
},
{
"ordinal": 1,
"name": "last_used_at",
"type_info": "Timestamptz"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
true
]
},
"hash": "bc61ca62d8f71880facb5d701a6e78697414b35618c50f8693f4e804bf1d7dbb"
}

View File

@@ -0,0 +1,47 @@
{
"db_name": "PostgreSQL",
"query": "SELECT workspace_id, name, size_bytes, created_by, last_used_at\n FROM volume WHERE workspace_id = $1 AND name = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "workspace_id",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "name",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "size_bytes",
"type_info": "Int8"
},
{
"ordinal": 3,
"name": "created_by",
"type_info": "Varchar"
},
{
"ordinal": 4,
"name": "last_used_at",
"type_info": "Timestamptz"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
false,
false,
false,
true
]
},
"hash": "d0869a340c8f34ca7a560d3b4c0070c9f117da3dd00ce3247c54a61052a6809c"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT leased_by FROM volume WHERE workspace_id = $1 AND name = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "leased_by",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
true
]
},
"hash": "d1ad2baf5e3a6f45f1f079d494e8d6affad03a1f388024806a5de3f9cc939c04"
}

View File

@@ -0,0 +1,28 @@
{
"db_name": "PostgreSQL",
"query": "SELECT name, size_bytes FROM volume WHERE workspace_id = $1 ORDER BY name",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "name",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "size_bytes",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false,
false
]
},
"hash": "dc18db954239c4ebdd3b46cfd34f33554794444f0dc4e2d2fec158eca5ebe865"
}

View File

@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE volume SET extra_perms = jsonb_set(extra_perms, $1, to_jsonb($2::bool), true)\n WHERE workspace_id = $3 AND name = $4",
"describe": {
"columns": [],
"parameters": {
"Left": [
"TextArray",
"Bool",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "eb79db2aeac7bf246ad56a5f116511b9d3183cb91b740a86944a77a2a964b57d"
}

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT token as \"token!\"\n FROM token\n WHERE token LIKE concat($1::text, '%')\n LIMIT 1\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "token!",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false
]
},
"hash": "eba16eb819e2644284fb073c891706d78a6f24cb0e614d7d81ba1b643805bf06"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT permissioned_as FROM v2_job WHERE id = $1 AND workspace_id = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "permissioned_as",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Uuid",
"Text"
]
},
"nullable": [
false
]
},
"hash": "f0ac12b66c5d3cca680541aed04359b064baf73b890efdc25426261d4eadfee0"
}

View File

@@ -0,0 +1,41 @@
{
"db_name": "PostgreSQL",
"query": "SELECT size_bytes, file_count, leased_by, lease_until\n FROM volume WHERE workspace_id = $1 AND name = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "size_bytes",
"type_info": "Int8"
},
{
"ordinal": 1,
"name": "file_count",
"type_info": "Int4"
},
{
"ordinal": 2,
"name": "leased_by",
"type_info": "Varchar"
},
{
"ordinal": 3,
"name": "lease_until",
"type_info": "Timestamptz"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
false,
true,
true
]
},
"hash": "f7ba87d5804b9bc05e7156c7c18c5a30037abef63efb5b44dc535c5f45d62a06"
}

163
backend/Cargo.lock generated
View File

@@ -15741,7 +15741,7 @@ dependencies = [
[[package]]
name = "windmill"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-nats",
@@ -15773,6 +15773,7 @@ dependencies = [
"sql-builder",
"sqlx",
"strum 0.27.2",
"tar",
"tempfile",
"tikv-jemalloc-ctl",
"tikv-jemalloc-sys",
@@ -15798,14 +15799,16 @@ dependencies = [
"windmill-queue",
"windmill-runtime-nativets",
"windmill-test-utils",
"windmill-types",
"windmill-worker",
"windmill-worker-volumes",
"windows-service",
"windows-sys 0.52.0",
]
[[package]]
name = "windmill-alerting"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -15818,7 +15821,7 @@ dependencies = [
[[package]]
name = "windmill-api"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"argon2",
@@ -15952,11 +15955,12 @@ dependencies = [
"windmill-trigger-websocket",
"windmill-types",
"windmill-worker",
"windmill-worker-volumes",
]
[[package]]
name = "windmill-api-agent-workers"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -15979,7 +15983,7 @@ dependencies = [
[[package]]
name = "windmill-api-assets"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -15992,7 +15996,7 @@ dependencies = [
[[package]]
name = "windmill-api-auth"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -16018,7 +16022,7 @@ dependencies = [
[[package]]
name = "windmill-api-client"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"reqwest 0.12.28",
"serde",
@@ -16028,7 +16032,7 @@ dependencies = [
[[package]]
name = "windmill-api-configs"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16045,7 +16049,7 @@ dependencies = [
[[package]]
name = "windmill-api-debug"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"axum 0.7.9",
"base64 0.22.1",
@@ -16068,7 +16072,7 @@ dependencies = [
[[package]]
name = "windmill-api-embeddings"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -16091,7 +16095,7 @@ dependencies = [
[[package]]
name = "windmill-api-flow-conversations"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16107,7 +16111,7 @@ dependencies = [
[[package]]
name = "windmill-api-flows"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16127,7 +16131,7 @@ dependencies = [
[[package]]
name = "windmill-api-groups"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16147,7 +16151,7 @@ dependencies = [
[[package]]
name = "windmill-api-inputs"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16161,7 +16165,7 @@ dependencies = [
[[package]]
name = "windmill-api-integration-tests"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-nats",
@@ -16188,7 +16192,7 @@ dependencies = [
[[package]]
name = "windmill-api-jobs"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -16213,7 +16217,7 @@ dependencies = [
[[package]]
name = "windmill-api-npm-proxy"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"axum 0.7.9",
"flate2",
@@ -16231,7 +16235,7 @@ dependencies = [
[[package]]
name = "windmill-api-openapi"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -16252,7 +16256,7 @@ dependencies = [
[[package]]
name = "windmill-api-schedule"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16272,7 +16276,7 @@ dependencies = [
[[package]]
name = "windmill-api-scripts"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16302,7 +16306,7 @@ dependencies = [
[[package]]
name = "windmill-api-settings"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -16329,7 +16333,7 @@ dependencies = [
[[package]]
name = "windmill-api-sse"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"lazy_static",
"serde",
@@ -16341,7 +16345,7 @@ dependencies = [
[[package]]
name = "windmill-api-users"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"argon2",
"axum 0.7.9",
@@ -16364,7 +16368,7 @@ dependencies = [
[[package]]
name = "windmill-api-workers"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16378,7 +16382,7 @@ dependencies = [
[[package]]
name = "windmill-api-workspaces"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16409,7 +16413,7 @@ dependencies = [
[[package]]
name = "windmill-audit"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"chrono",
"lazy_static",
@@ -16423,7 +16427,7 @@ dependencies = [
[[package]]
name = "windmill-autoscaling"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -16442,7 +16446,7 @@ dependencies = [
[[package]]
name = "windmill-common"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"aes-gcm",
"anyhow",
@@ -16541,7 +16545,7 @@ dependencies = [
[[package]]
name = "windmill-dep-map"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"chrono",
"itertools 0.14.0",
@@ -16560,7 +16564,7 @@ dependencies = [
[[package]]
name = "windmill-git-sync"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"regex",
"serde",
@@ -16575,7 +16579,7 @@ dependencies = [
[[package]]
name = "windmill-indexer"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"astral-tokio-tar",
@@ -16599,7 +16603,7 @@ dependencies = [
[[package]]
name = "windmill-jseval"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"futures",
@@ -16616,7 +16620,7 @@ dependencies = [
[[package]]
name = "windmill-macros"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"itertools 0.14.0",
"lazy_static",
@@ -16632,7 +16636,7 @@ dependencies = [
[[package]]
name = "windmill-mcp"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-trait",
@@ -16653,7 +16657,7 @@ dependencies = [
[[package]]
name = "windmill-native-triggers"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-trait",
@@ -16684,7 +16688,7 @@ dependencies = [
[[package]]
name = "windmill-oauth"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-oauth2",
@@ -16708,7 +16712,7 @@ dependencies = [
[[package]]
name = "windmill-object-store"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-stream",
@@ -16742,7 +16746,7 @@ dependencies = [
[[package]]
name = "windmill-operator"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"futures",
@@ -16760,7 +16764,7 @@ dependencies = [
[[package]]
name = "windmill-parser"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"convert_case 0.6.0",
"serde",
@@ -16769,7 +16773,7 @@ dependencies = [
[[package]]
name = "windmill-parser-bash"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"lazy_static",
@@ -16781,7 +16785,7 @@ dependencies = [
[[package]]
name = "windmill-parser-csharp"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"serde_json",
@@ -16793,7 +16797,7 @@ dependencies = [
[[package]]
name = "windmill-parser-go"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"gosyn",
@@ -16805,7 +16809,7 @@ dependencies = [
[[package]]
name = "windmill-parser-graphql"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"lazy_static",
@@ -16817,7 +16821,7 @@ dependencies = [
[[package]]
name = "windmill-parser-java"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"serde_json",
@@ -16829,7 +16833,7 @@ dependencies = [
[[package]]
name = "windmill-parser-nu"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"nu-parser",
@@ -16840,7 +16844,7 @@ dependencies = [
[[package]]
name = "windmill-parser-php"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"itertools 0.14.0",
@@ -16851,7 +16855,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"itertools 0.14.0",
@@ -16864,7 +16868,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py-imports"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-recursion",
@@ -16888,7 +16892,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ruby"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"lazy_static",
@@ -16902,7 +16906,7 @@ dependencies = [
[[package]]
name = "windmill-parser-rust"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"convert_case 0.6.0",
@@ -16919,7 +16923,7 @@ dependencies = [
[[package]]
name = "windmill-parser-sql"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"lazy_static",
@@ -16934,7 +16938,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ts"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"lazy_static",
@@ -16953,7 +16957,7 @@ dependencies = [
[[package]]
name = "windmill-parser-yaml"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"serde",
@@ -16964,7 +16968,7 @@ dependencies = [
[[package]]
name = "windmill-queue"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-recursion",
@@ -17001,7 +17005,7 @@ dependencies = [
[[package]]
name = "windmill-runtime-nativets"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"const_format",
@@ -17039,7 +17043,7 @@ dependencies = [
[[package]]
name = "windmill-sql-datatype-parser-wasm"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"getrandom 0.3.4",
"wasm-bindgen",
@@ -17050,7 +17054,7 @@ dependencies = [
[[package]]
name = "windmill-store"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-recursion",
@@ -17079,7 +17083,7 @@ dependencies = [
[[package]]
name = "windmill-test-utils"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -17102,7 +17106,7 @@ dependencies = [
[[package]]
name = "windmill-trigger"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-trait",
@@ -17135,7 +17139,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-email"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-trait",
@@ -17155,7 +17159,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-gcp"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-trait",
@@ -17189,7 +17193,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-http"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-trait",
@@ -17224,7 +17228,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-kafka"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-trait",
@@ -17247,7 +17251,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-mqtt"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-trait",
@@ -17271,7 +17275,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-nats"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-nats",
@@ -17295,7 +17299,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-postgres"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-trait",
@@ -17330,7 +17334,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-sqs"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-trait",
@@ -17358,7 +17362,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-websocket"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-trait",
@@ -17381,7 +17385,7 @@ dependencies = [
[[package]]
name = "windmill-types"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"bitflags 2.9.4",
@@ -17399,7 +17403,7 @@ dependencies = [
[[package]]
name = "windmill-worker"
version = "1.650.0"
version = "1.651.1"
dependencies = [
"anyhow",
"async-once-cell",
@@ -17499,9 +17503,28 @@ dependencies = [
"windmill-queue",
"windmill-runtime-nativets",
"windmill-types",
"windmill-worker-volumes",
"yaml-rust",
]
[[package]]
name = "windmill-worker-volumes"
version = "1.651.1"
dependencies = [
"bytes",
"futures",
"lazy_static",
"md-5 0.10.6",
"object_store",
"regex",
"serde",
"serde_json",
"tempfile",
"tokio",
"tracing",
"windmill-common",
]
[[package]]
name = "windows"
version = "0.56.0"

View File

@@ -1,6 +1,6 @@
[package]
name = "windmill"
version = "1.650.0"
version = "1.651.1"
authors.workspace = true
edition.workspace = true
@@ -70,13 +70,14 @@ members = [
"./parsers/windmill-parser-py-imports",
"./parsers/windmill-sql-datatype-parser-wasm",
"./parsers/windmill-parser-yaml", "windmill-macros", "parsers/windmill-parser-nu",
"./windmill-worker-volumes",
"./windmill-test-utils",
"./windmill-api-integration-tests",
]
exclude = ["./windmill-duckdb-ffi-internal"]
[workspace.package]
version = "1.650.0"
version = "1.651.1"
authors = ["Ruben Fiszel <ruben@windmill.dev>"]
edition = "2021"
@@ -250,10 +251,13 @@ reqwest.workspace = true
windmill-queue = { workspace = true, features = ["failpoints"] }
windmill-dep-map.workspace = true
windmill-test-utils.workspace = true
windmill-worker-volumes.workspace = true
windmill-types.workspace = true
axum.workspace = true
serde.workspace = true
windmill-api-client.workspace = true
tempfile.workspace = true
tar.workspace = true
windmill-parser-ts.workspace = true
rumqttc.workspace = true
rdkafka.workspace = true
@@ -267,6 +271,7 @@ aws-credential-types.workspace = true
windmill-api = { path = "./windmill-api", default-features = false }
windmill-queue = { path = "./windmill-queue" }
windmill-worker = { path = "./windmill-worker" }
windmill-worker-volumes = { path = "./windmill-worker-volumes" }
windmill-dep-map = { path = "./windmill-dep-map" }
windmill-types = { path = "./windmill-types" }
windmill-common = { path = "./windmill-common", default-features = false }
@@ -439,6 +444,7 @@ base64 = "^0.22.1"
base32 = "^0"
hmac = "0.12.1"
sha2 = "0.10.6"
md-5 = "0.10.6"
sha1 = "0.10.6"
sqlx = { version = "0.8.0", features = [
"macros",

View File

@@ -1 +1 @@
6fd5a2ce908235a17975ad4dbdf0051cd89334f3
f9549c813b3dba5324ea9d1edacc8756a6d699bf

View File

@@ -0,0 +1 @@
DROP TABLE IF EXISTS volume;

View File

@@ -0,0 +1,22 @@
-- Add 'volume' to the asset_kind enum
ALTER TYPE asset_kind ADD VALUE IF NOT EXISTS 'volume';
-- Volume metadata table
CREATE TABLE volume (
workspace_id VARCHAR(50) NOT NULL REFERENCES workspace(id) ON DELETE CASCADE,
name VARCHAR(255) NOT NULL,
size_bytes BIGINT NOT NULL DEFAULT 0,
file_count INTEGER NOT NULL DEFAULT 0,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
created_by VARCHAR(255) NOT NULL,
updated_at TIMESTAMPTZ,
updated_by VARCHAR(255),
description TEXT NOT NULL DEFAULT '',
lease_until TIMESTAMPTZ,
leased_by VARCHAR(255),
last_used_at TIMESTAMPTZ,
extra_perms JSONB NOT NULL DEFAULT '{}',
PRIMARY KEY (workspace_id, name)
);
CREATE INDEX idx_volume_last_used ON volume(workspace_id, last_used_at);

View File

@@ -18,6 +18,7 @@ pub enum AssetKind {
Resource,
Ducklake,
DataTable,
Volume,
}
#[derive(Serialize, Debug, PartialEq, Clone)]
@@ -148,4 +149,5 @@ pub const ASSET_KINDS: &[(&str, AssetKind)] = &[
("$res:", AssetKind::Resource),
("ducklake://", AssetKind::Ducklake),
("datatable://", AssetKind::DataTable),
("volume://", AssetKind::Volume),
];

View File

@@ -1,12 +1,12 @@
#![cfg(all(feature = "private", feature = "agent_worker_server"))]
use windmill_test_utils::*;
use serde_json::json;
use sqlx::{Pool, Postgres};
use windmill_common::{
jobs::{JobPayload, RawCode},
scripts::ScriptLang,
};
use windmill_test_utils::*;
fn bun_code(code: &str) -> RawCode {
RawCode {
@@ -18,8 +18,8 @@ fn bun_code(code: &str) -> RawCode {
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings:
windmill_common::runnable_settings::ConcurrencySettings::default().into(),
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default()
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
}
}
@@ -223,7 +223,10 @@ async fn test_agent_worker_token_and_ping(db: Pool<Postgres>) -> anyhow::Result<
.fetch_one(&db)
.await?;
assert!(worker_count > 0, "worker ping should be recorded in database");
assert!(
worker_count > 0,
"worker ping should be recorded in database"
);
// MainLoop ping updates the existing record
let resp = http_client
@@ -265,3 +268,319 @@ async fn test_agent_worker_multiple_jobs_sequential(db: Pool<Postgres>) -> anyho
Ok(())
}
/// Test the volume HTTP proxy endpoints that agent workers use.
///
/// Exercises the full volume lifecycle via HTTP:
/// 1. Configure workspace S3 storage (FilesystemStorage)
/// 2. Pre-populate a volume with a file
/// 3. POST /begin — acquire lease, get manifest
/// 4. GET /file/* — download existing file
/// 5. PUT /file/* — upload a new file
/// 6. POST /commit — finalize with stats, release lease
/// 7. Verify DB state and storage
#[cfg(feature = "parquet")]
#[sqlx::test(fixtures("base"))]
async fn test_agent_worker_volume_e2e(db: Pool<Postgres>) -> anyhow::Result<()> {
let (client, _port, _server) = init_client_agent_mode(db.clone()).await;
// 1. Set up filesystem-based object storage in a temp dir
let storage_dir = tempfile::tempdir()?;
let storage_root = storage_dir.path().to_string_lossy().to_string();
let lfs_config = json!({
"type": "FilesystemStorage",
"root_path": storage_root,
"public_resource": null,
"advanced_permissions": null
});
sqlx::query!(
"UPDATE workspace_settings SET large_file_storage = $1 WHERE workspace_id = $2",
lfs_config,
"test-workspace"
)
.execute(&db)
.await?;
// 2. Pre-populate the volume with a file
let vol_dir = storage_dir.path().join("volumes").join("test-vol");
std::fs::create_dir_all(&vol_dir)?;
std::fs::write(vol_dir.join("hello.txt"), b"hello from volume")?;
let base = client.baseurl();
let http = client.client();
let vol_base = format!("{base}/w/test-workspace/volumes/test-vol");
// 3. POST /begin — acquire lease, get manifest + permissions
let resp = http
.post(format!("{vol_base}/begin"))
.json(&json!({
"worker_name": "test-worker-1",
"permissioned_as": "u/test-user"
}))
.send()
.await?;
assert!(
resp.status().is_success(),
"begin should succeed, got: {}",
resp.status()
);
let begin_body: serde_json::Value = resp.json().await?;
assert!(
begin_body["writable"].as_bool().unwrap(),
"should be writable"
);
let manifest = begin_body["manifest"].as_object().unwrap();
assert!(
manifest.contains_key("hello.txt"),
"manifest should contain hello.txt, got: {manifest:?}"
);
// 4. GET /file/* — download the existing file
let resp = http
.get(format!("{vol_base}/file/hello.txt"))
.send()
.await?;
assert!(
resp.status().is_success(),
"file download should succeed, got: {}",
resp.status()
);
let file_bytes = resp.bytes().await?;
assert_eq!(
file_bytes.as_ref(),
b"hello from volume",
"downloaded file content should match"
);
// 5. PUT /file/* — upload a new file
let resp = http
.put(format!("{vol_base}/file/output.txt"))
.body(b"written by agent worker".to_vec())
.send()
.await?;
assert!(
resp.status().is_success(),
"file upload should succeed, got: {}",
resp.status()
);
// 6. POST /commit — finalize: report stats, release lease
let resp = http
.post(format!("{vol_base}/commit"))
.json(&json!({
"worker_name": "test-worker-1",
"deleted_keys": [],
"symlinks": {},
"file_count": 2,
"size_bytes": 39
}))
.send()
.await?;
assert!(
resp.status().is_success(),
"commit should succeed, got: {}",
resp.status()
);
// 7. Verify volume DB row was updated
let vol_row = sqlx::query!(
"SELECT size_bytes, file_count, leased_by, lease_until
FROM volume WHERE workspace_id = $1 AND name = $2",
"test-workspace",
"test-vol"
)
.fetch_optional(&db)
.await?;
let vol_row = vol_row.expect("volume row should exist");
assert_eq!(vol_row.file_count, 2, "file_count should be 2");
assert_eq!(vol_row.size_bytes, 39, "size_bytes should match");
assert!(vol_row.leased_by.is_none(), "lease should be released");
assert!(
vol_row.lease_until.is_none() || vol_row.lease_until.unwrap() < chrono::Utc::now(),
"lease_until should be cleared or in the past"
);
// 8. Verify the uploaded file was persisted in storage
let output_path = vol_dir.join("output.txt");
assert!(output_path.exists(), "output.txt should be in storage");
let output_content = std::fs::read_to_string(&output_path)?;
assert_eq!(output_content, "written by agent worker");
Ok(())
}
/// Full E2E test: agent worker in HTTP mode runs a Bun script with a volume mount.
///
/// The worker pulls the job via HTTP, downloads volume files via the server-side
/// volume proxy endpoints, executes the script, and syncs changes back.
#[cfg(all(feature = "parquet", feature = "enterprise"))]
#[sqlx::test(fixtures("base"))]
async fn test_agent_worker_volume_http_worker_e2e(db: Pool<Postgres>) -> anyhow::Result<()> {
let (_client, port, _server) = init_client_agent_mode(db.clone()).await;
// 1. Set up filesystem-based object storage in a temp dir
let storage_dir = tempfile::tempdir()?;
let storage_root = storage_dir.path().to_string_lossy().to_string();
let lfs_config = json!({
"type": "FilesystemStorage",
"root_path": storage_root,
"public_resource": null,
"advanced_permissions": null
});
sqlx::query!(
"UPDATE workspace_settings SET large_file_storage = $1 WHERE workspace_id = $2",
lfs_config,
"test-workspace"
)
.execute(&db)
.await?;
// 2. Pre-populate the volume with a file
let vol_dir = storage_dir.path().join("volumes").join("test-vol");
std::fs::create_dir_all(&vol_dir)?;
std::fs::write(vol_dir.join("hello.txt"), b"hello from volume")?;
// 3. Push the job, then run worker with HTTP connection (bun tag)
let code = r#"// volume: test-vol /tmp/data
import { readFileSync, writeFileSync, existsSync } from "fs";
export function main() {
const content = readFileSync("/tmp/data/hello.txt", "utf-8");
writeFileSync("/tmp/data/output.txt", "written by agent worker");
return {
read_content: content,
output_exists: existsSync("/tmp/data/output.txt"),
};
}"#;
let uuid = RunJob::from(JobPayload::Code(bun_code(code)))
.push(&db)
.await;
let listener = listen_for_completed_jobs(&db).await;
let conn = testing_http_connection_with_tags(
port,
vec!["bun".into(), "flow".into(), "dependency".into()],
)
.await;
in_test_worker(conn, listener.find(&uuid), port).await;
let result = completed_job(uuid, &db).await;
assert!(result.success, "job should succeed: {:?}", result.result);
let json = result.json_result().expect("should have JSON result");
assert_eq!(json["read_content"], json!("hello from volume"));
assert_eq!(json["output_exists"], json!(true));
// 4. Verify volume DB row was updated
let vol_row = sqlx::query!(
"SELECT size_bytes, file_count, leased_by, lease_until
FROM volume WHERE workspace_id = $1 AND name = $2",
"test-workspace",
"test-vol"
)
.fetch_optional(&db)
.await?;
let vol_row = vol_row.expect("volume row should exist");
assert!(
vol_row.file_count >= 2,
"should have at least 2 files (hello.txt + output.txt), got: {}",
vol_row.file_count
);
assert!(vol_row.size_bytes > 0, "size_bytes should be > 0");
assert!(vol_row.leased_by.is_none(), "lease should be released");
assert!(
vol_row.lease_until.is_none() || vol_row.lease_until.unwrap() < chrono::Utc::now(),
"lease_until should be cleared or in the past"
);
// 5. Verify the new file was written back to the storage
let output_path = vol_dir.join("output.txt");
assert!(
output_path.exists(),
"output.txt should be synced back to storage"
);
let output_content = std::fs::read_to_string(&output_path)?;
assert_eq!(output_content, "written by agent worker");
Ok(())
}
/// Test the volume release endpoint (error/cancel path).
#[cfg(feature = "parquet")]
#[sqlx::test(fixtures("base"))]
async fn test_agent_worker_volume_release(db: Pool<Postgres>) -> anyhow::Result<()> {
let (client, _port, _server) = init_client_agent_mode(db.clone()).await;
// Set up filesystem storage
let storage_dir = tempfile::tempdir()?;
let storage_root = storage_dir.path().to_string_lossy().to_string();
let lfs_config = json!({
"type": "FilesystemStorage",
"root_path": storage_root,
"public_resource": null,
"advanced_permissions": null
});
sqlx::query!(
"UPDATE workspace_settings SET large_file_storage = $1 WHERE workspace_id = $2",
lfs_config,
"test-workspace"
)
.execute(&db)
.await?;
let base = client.baseurl();
let http = client.client();
let vol_base = format!("{base}/w/test-workspace/volumes/test-vol");
// Begin (acquire lease)
let resp = http
.post(format!("{vol_base}/begin"))
.json(&json!({
"worker_name": "test-worker-2",
"permissioned_as": "u/test-user"
}))
.send()
.await?;
assert!(resp.status().is_success(), "begin should succeed");
// Verify lease is held
let leased = sqlx::query_scalar!(
"SELECT leased_by FROM volume WHERE workspace_id = $1 AND name = $2",
"test-workspace",
"test-vol"
)
.fetch_optional(&db)
.await?
.flatten();
assert_eq!(leased.as_deref(), Some("test-worker-2"));
// Release without commit (simulating error path)
let resp = http
.post(format!("{vol_base}/release"))
.json(&json!({ "worker_name": "test-worker-2" }))
.send()
.await?;
assert!(resp.status().is_success(), "release should succeed");
// Verify lease is cleared
let leased = sqlx::query_scalar!(
"SELECT leased_by FROM volume WHERE workspace_id = $1 AND name = $2",
"test-workspace",
"test-vol"
)
.fetch_optional(&db)
.await?
.flatten();
assert!(leased.is_none(), "lease should be released");
Ok(())
}

View File

@@ -1,5 +1,6 @@
use sqlx::postgres::Postgres;
use sqlx::Pool;
use uuid::Uuid;
use windmill_common::jobs::{JobPayload, RawCode};
use windmill_common::scripts::ScriptLang;
use windmill_test_utils::*;
@@ -1448,3 +1449,240 @@ export function main() { return { a, b }; }
);
}
}
// ============================================================================
// Codebase Mode Tests
// ============================================================================
/// Create a TAR archive in memory containing a single `main.js` file.
fn create_codebase_tar(main_js_content: &str) -> Vec<u8> {
let mut builder = tar::Builder::new(Vec::new());
let content = main_js_content.as_bytes();
let mut header = tar::Header::new_gnu();
header.set_path("main.js").unwrap();
header.set_size(content.len() as u64);
header.set_mode(0o644);
header.set_cksum();
builder.append(&header, content).unwrap();
builder.into_inner().unwrap()
}
/// Place a TAR codebase at the expected cache path for the given job ID and hash.
fn place_codebase_in_cache(job_id: &Uuid, tar_bytes: &[u8], is_esm: bool) {
let codebase_id = if is_esm {
format!("{}.esm.tar", job_id)
} else {
format!("{}.tar", job_id)
};
let bundle_path = format!("script_bundle/test-workspace/{}", codebase_id);
let cache_path = format!(
"{}/{}.tar",
*windmill_common::worker::ROOT_CACHE_NOMOUNT_DIR,
bundle_path,
);
let parent = std::path::Path::new(&cache_path).parent().unwrap();
std::fs::create_dir_all(parent).unwrap();
std::fs::write(&cache_path, tar_bytes).unwrap();
}
#[sqlx::test(fixtures("base"))]
async fn test_cjs_codebase_tar(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let main_js = r#"
module.exports.main = function() {
return "cjs codebase ok";
};
"#;
let inner_content = r#"export function main() { return "cjs codebase ok"; }"#;
let job_id = Uuid::new_v4();
let tar_bytes = create_codebase_tar(main_js);
place_codebase_in_cache(&job_id, &tar_bytes, false);
let job = JobPayload::Code(RawCode {
hash: Some(-43), // PREVIEW_IS_TAR_CODEBASE_HASH
content: inner_content.to_string(),
path: None,
language: ScriptLang::Bun,
lock: None,
concurrency_settings: Default::default(),
debouncing_settings: Default::default(),
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
});
let result = RunJob::from(job)
.job_id(job_id)
.run_until_complete(&db, false, port)
.await
.json_result()
.unwrap();
assert_eq!(result, serde_json::json!("cjs codebase ok"));
Ok(())
}
#[sqlx::test(fixtures("base"))]
async fn test_esm_codebase_tar(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let main_js = r#"
export function main() {
return "esm codebase ok";
}
"#;
let inner_content = r#"export function main() { return "esm codebase ok"; }"#;
let job_id = Uuid::new_v4();
let tar_bytes = create_codebase_tar(main_js);
place_codebase_in_cache(&job_id, &tar_bytes, true);
let job = JobPayload::Code(RawCode {
hash: Some(-45), // PREVIEW_IS_TAR_ESM_CODEBASE_HASH
content: inner_content.to_string(),
path: None,
language: ScriptLang::Bun,
lock: None,
concurrency_settings: Default::default(),
debouncing_settings: Default::default(),
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
});
let result = RunJob::from(job)
.job_id(job_id)
.run_until_complete(&db, false, port)
.await
.json_result()
.unwrap();
assert_eq!(result, serde_json::json!("esm codebase ok"));
Ok(())
}
#[sqlx::test(fixtures("base"))]
async fn test_cjs_codebase_tar_nsjail(db: Pool<Postgres>) -> anyhow::Result<()> {
if std::process::Command::new("nsjail")
.arg("--help")
.output()
.is_err()
{
eprintln!("nsjail not found, skipping test");
return Ok(());
}
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let main_js = r#"
module.exports.main = function() {
return "cjs nsjail ok";
};
"#;
let inner_content = r#"export function main() { return "cjs nsjail ok"; }"#;
let job_id = Uuid::new_v4();
let tar_bytes = create_codebase_tar(main_js);
place_codebase_in_cache(&job_id, &tar_bytes, false);
let job = JobPayload::Code(RawCode {
hash: Some(-43),
content: inner_content.to_string(),
path: None,
language: ScriptLang::Bun,
lock: None,
concurrency_settings: Default::default(),
debouncing_settings: Default::default(),
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
});
use std::sync::atomic::Ordering;
windmill_worker::JOB_ISOLATION.store(
windmill_worker::JobIsolationLevel::NsjailSandboxing as u8,
Ordering::Relaxed,
);
let result = RunJob::from(job)
.job_id(job_id)
.run_until_complete(&db, false, port)
.await;
windmill_worker::JOB_ISOLATION.store(
windmill_worker::JobIsolationLevel::Undefined as u8,
Ordering::Relaxed,
);
let json = result.json_result().unwrap();
assert_eq!(json, serde_json::json!("cjs nsjail ok"));
Ok(())
}
#[sqlx::test(fixtures("base"))]
async fn test_esm_codebase_tar_nsjail(db: Pool<Postgres>) -> anyhow::Result<()> {
if std::process::Command::new("nsjail")
.arg("--help")
.output()
.is_err()
{
eprintln!("nsjail not found, skipping test");
return Ok(());
}
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let main_js = r#"
export function main() {
return "esm nsjail ok";
}
"#;
let inner_content = r#"export function main() { return "esm nsjail ok"; }"#;
let job_id = Uuid::new_v4();
let tar_bytes = create_codebase_tar(main_js);
place_codebase_in_cache(&job_id, &tar_bytes, true);
let job = JobPayload::Code(RawCode {
hash: Some(-45),
content: inner_content.to_string(),
path: None,
language: ScriptLang::Bun,
lock: None,
concurrency_settings: Default::default(),
debouncing_settings: Default::default(),
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
});
use std::sync::atomic::Ordering;
windmill_worker::JOB_ISOLATION.store(
windmill_worker::JobIsolationLevel::NsjailSandboxing as u8,
Ordering::Relaxed,
);
let result = RunJob::from(job)
.job_id(job_id)
.run_until_complete(&db, false, port)
.await;
windmill_worker::JOB_ISOLATION.store(
windmill_worker::JobIsolationLevel::Undefined as u8,
Ordering::Relaxed,
);
let json = result.json_result().unwrap();
assert_eq!(json, serde_json::json!("esm nsjail ok"));
Ok(())
}

View File

@@ -9,23 +9,7 @@ export async function main() {
'{"$schema":"https://json-schema.org/draft/2020-12/schema","properties":{},"required":[],"type":"object"}',
'',
'',
'f/leafs/ts', 500001, 'nativets', '');
INSERT INTO public.script(workspace_id, created_by, content, schema, summary, description, path, hash, language, lock) VALUES (
'test-workspace',
'test-user',
'
package main
import "fmt"
func main() {
fmt.Println("Go leaf")
}',
'{"$schema":"https://json-schema.org/draft/2020-12/schema","properties":{},"required":[],"type":"object"}',
'',
'',
'f/leafs/go', 500002, 'go', '');
'f/leafs/ts', 500001, 'bun', '');
INSERT INTO public.script(workspace_id, created_by, content, schema, summary, description, path, hash, language, lock) VALUES (
'test-workspace',
@@ -52,3 +36,9 @@ function main() {
'',
'f/leafs/php', 500004, 'php', '');
-- Link scripts to named workspace dependencies (name: "test")
INSERT INTO dependency_map (workspace_id, importer_path, importer_kind, imported_path, importer_node_id) VALUES
('test-workspace', 'f/leafs/ts', 'script', 'dependencies/test.package.json', ''),
('test-workspace', 'f/leafs/python', 'script', 'dependencies/test.requirements.in', ''),
('test-workspace', 'f/leafs/php', 'script', 'dependencies/test.composer.json', '');

View File

@@ -0,0 +1,102 @@
// volume: agent-memory .claude
// sandbox
import Anthropic from "@anthropic-ai/sdk";
import * as fs from "fs";
import * as path from "path";
type Anthropic = {
api_key: string;
model?: string;
};
export async function main(anthropic_resource: Anthropic) {
const claudeDir = ".claude";
const results: Record<string, unknown> = {};
// --- Step 1: Verify volume is mounted at the relative path ---
results["volume_exists"] = fs.existsSync(claudeDir);
if (!results["volume_exists"]) {
fs.mkdirSync(claudeDir, { recursive: true });
}
const testFile = path.join(claudeDir, "mount-check.txt");
fs.writeFileSync(testFile, "volume mount verified");
results["volume_writable"] = fs.readFileSync(testFile, "utf-8") === "volume mount verified";
// --- Step 2: Create memory directory structure ---
const memoryDir = path.join(claudeDir, "memory");
fs.mkdirSync(memoryDir, { recursive: true });
const memoryFile = path.join(memoryDir, "MEMORY.md");
fs.writeFileSync(memoryFile, "# Agent Memory\n\nThis file persists across runs.\n");
results["memory_file_created"] = fs.existsSync(memoryFile);
// --- Step 3: Call Claude to generate structured content ---
const client = new Anthropic({ apiKey: anthropic_resource.api_key });
const model = anthropic_resource.model ?? "claude-sonnet-4-20250514";
const response = await client.messages.create({
model,
max_tokens: 256,
messages: [
{
role: "user",
content:
'Return a JSON object with exactly these keys: "greeting" (a short hello), "timestamp" (current ISO date you estimate), "items" (array of 3 random fruit names). Only return the JSON, no markdown.',
},
],
});
const assistantText =
response.content[0].type === "text" ? response.content[0].text : "";
results["claude_responded"] = assistantText.length > 0;
results["claude_model"] = response.model;
results["claude_stop_reason"] = response.stop_reason;
let parsed: Record<string, unknown> = {};
try {
parsed = JSON.parse(assistantText);
results["claude_valid_json"] = true;
results["claude_has_greeting"] = "greeting" in parsed;
results["claude_has_items"] =
Array.isArray(parsed.items) && parsed.items.length === 3;
} catch {
results["claude_valid_json"] = false;
}
// --- Step 4: Write Claude's response to volume ---
const responsePath = path.join(claudeDir, "claude-response.json");
fs.writeFileSync(responsePath, JSON.stringify(parsed, null, 2));
results["response_written"] = fs.existsSync(responsePath);
// --- Step 5: Read back and verify ---
const readBack = fs.readFileSync(responsePath, "utf-8");
const readParsed = JSON.parse(readBack);
results["readback_matches"] =
JSON.stringify(readParsed) === JSON.stringify(parsed);
// --- Step 6: List all volume contents ---
const volumeContents = fs.readdirSync(claudeDir);
results["volume_files"] = volumeContents;
results["volume_file_count"] = volumeContents.length;
// --- Step 7: Verify memory file persists ---
const memoryContent = fs.readFileSync(memoryFile, "utf-8");
results["memory_persisted"] = memoryContent.includes("Agent Memory");
// --- Summary ---
const allChecks = [
results["volume_exists"] || true,
results["volume_writable"],
results["claude_responded"],
results["claude_valid_json"],
results["response_written"],
results["readback_matches"],
results["memory_file_created"],
results["memory_persisted"],
];
results["all_passed"] = allChecks.every(Boolean);
return results;
}

View File

@@ -0,0 +1,637 @@
use serde_json::json;
use sqlx::{Pool, Postgres};
use windmill_common::jobs::{JobPayload, RawCode};
use windmill_common::scripts::ScriptLang;
use windmill_test_utils::*;
#[sqlx::test(fixtures("base"))]
async fn test_volume_insert(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
sqlx::query!(
"INSERT INTO volume (workspace_id, name, size_bytes, created_by)
VALUES ($1, $2, $3, $4)",
"test-workspace",
"test-volume",
1024_i64,
"test-user"
)
.execute(&db)
.await?;
let row = sqlx::query!(
"SELECT workspace_id, name, size_bytes, created_by, last_used_at
FROM volume WHERE workspace_id = $1 AND name = $2",
"test-workspace",
"test-volume"
)
.fetch_one(&db)
.await?;
assert_eq!(row.workspace_id, "test-workspace");
assert_eq!(row.name, "test-volume");
assert_eq!(row.size_bytes, 1024);
assert_eq!(row.created_by, "test-user");
assert!(row.last_used_at.is_none());
Ok(())
}
#[sqlx::test(fixtures("base"))]
async fn test_volume_upsert_size(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
sqlx::query!(
"INSERT INTO volume (workspace_id, name, size_bytes, created_by, last_used_at)
VALUES ($1, $2, $3, $4, now())
ON CONFLICT (workspace_id, name) DO UPDATE
SET size_bytes = $3, last_used_at = now()",
"test-workspace",
"upsert-vol",
500_i64,
"test-user"
)
.execute(&db)
.await?;
let row = sqlx::query!(
"SELECT size_bytes FROM volume WHERE workspace_id = $1 AND name = $2",
"test-workspace",
"upsert-vol"
)
.fetch_one(&db)
.await?;
assert_eq!(row.size_bytes, 500);
sqlx::query!(
"INSERT INTO volume (workspace_id, name, size_bytes, created_by, last_used_at)
VALUES ($1, $2, $3, $4, now())
ON CONFLICT (workspace_id, name) DO UPDATE
SET size_bytes = $3, last_used_at = now()",
"test-workspace",
"upsert-vol",
2048_i64,
"test-user"
)
.execute(&db)
.await?;
let row = sqlx::query!(
"SELECT size_bytes, last_used_at FROM volume WHERE workspace_id = $1 AND name = $2",
"test-workspace",
"upsert-vol"
)
.fetch_one(&db)
.await?;
assert_eq!(row.size_bytes, 2048);
assert!(row.last_used_at.is_some());
Ok(())
}
#[sqlx::test(fixtures("base"))]
async fn test_volume_update_last_used(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
sqlx::query!(
"INSERT INTO volume (workspace_id, name, size_bytes, created_by)
VALUES ($1, $2, $3, $4)",
"test-workspace",
"used-vol",
100_i64,
"test-user"
)
.execute(&db)
.await?;
let row = sqlx::query!(
"SELECT last_used_at FROM volume WHERE workspace_id = $1 AND name = $2",
"test-workspace",
"used-vol"
)
.fetch_one(&db)
.await?;
assert!(row.last_used_at.is_none());
sqlx::query!(
"UPDATE volume SET last_used_at = now() WHERE workspace_id = $1 AND name = $2",
"test-workspace",
"used-vol"
)
.execute(&db)
.await?;
let row = sqlx::query!(
"SELECT last_used_at FROM volume WHERE workspace_id = $1 AND name = $2",
"test-workspace",
"used-vol"
)
.fetch_one(&db)
.await?;
assert!(row.last_used_at.is_some());
Ok(())
}
#[sqlx::test(fixtures("base"))]
async fn test_volume_update_nonexistent_noop(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let result = sqlx::query!(
"UPDATE volume SET last_used_at = now() WHERE workspace_id = $1 AND name = $2",
"test-workspace",
"nonexistent-vol"
)
.execute(&db)
.await?;
assert_eq!(result.rows_affected(), 0);
Ok(())
}
#[sqlx::test(fixtures("base"))]
async fn test_volume_list_multiple(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
for i in 0..5 {
sqlx::query!(
"INSERT INTO volume (workspace_id, name, size_bytes, created_by)
VALUES ($1, $2, $3, $4)",
"test-workspace",
format!("vol-{}", i),
(i * 100) as i64,
"test-user"
)
.execute(&db)
.await?;
}
let rows = sqlx::query!(
"SELECT name, size_bytes FROM volume WHERE workspace_id = $1 ORDER BY name",
"test-workspace"
)
.fetch_all(&db)
.await?;
assert_eq!(rows.len(), 5);
assert_eq!(rows[0].name, "vol-0");
assert_eq!(rows[0].size_bytes, 0);
assert_eq!(rows[4].name, "vol-4");
assert_eq!(rows[4].size_bytes, 400);
Ok(())
}
#[sqlx::test(fixtures("base"))]
async fn test_volume_delete(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
sqlx::query!(
"INSERT INTO volume (workspace_id, name, size_bytes, created_by)
VALUES ($1, $2, $3, $4)",
"test-workspace",
"deleteme",
100_i64,
"test-user"
)
.execute(&db)
.await?;
let count = sqlx::query_scalar!(
"SELECT count(*) FROM volume WHERE workspace_id = $1 AND name = $2",
"test-workspace",
"deleteme"
)
.fetch_one(&db)
.await?;
assert_eq!(count, Some(1));
sqlx::query!(
"DELETE FROM volume WHERE workspace_id = $1 AND name = $2",
"test-workspace",
"deleteme"
)
.execute(&db)
.await?;
let count = sqlx::query_scalar!(
"SELECT count(*) FROM volume WHERE workspace_id = $1 AND name = $2",
"test-workspace",
"deleteme"
)
.fetch_one(&db)
.await?;
assert_eq!(count, Some(0));
Ok(())
}
#[sqlx::test(fixtures("base"))]
async fn test_volume_workspace_fk_constraint(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let result = sqlx::query!(
"INSERT INTO volume (workspace_id, name, size_bytes, created_by)
VALUES ($1, $2, $3, $4)",
"nonexistent-workspace",
"vol",
100_i64,
"test-user"
)
.execute(&db)
.await;
assert!(result.is_err());
let err = result.unwrap_err().to_string();
assert!(
err.contains("foreign key"),
"Expected foreign key violation, got: {}",
err
);
Ok(())
}
#[sqlx::test(fixtures("base"))]
async fn test_volume_primary_key_uniqueness(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
sqlx::query!(
"INSERT INTO volume (workspace_id, name, size_bytes, created_by)
VALUES ($1, $2, $3, $4)",
"test-workspace",
"unique-vol",
100_i64,
"test-user"
)
.execute(&db)
.await?;
let result = sqlx::query!(
"INSERT INTO volume (workspace_id, name, size_bytes, created_by)
VALUES ($1, $2, $3, $4)",
"test-workspace",
"unique-vol",
200_i64,
"another-user"
)
.execute(&db)
.await;
assert!(result.is_err());
let err = result.unwrap_err().to_string();
assert!(
err.contains("duplicate key") || err.contains("unique"),
"Expected unique violation, got: {}",
err
);
Ok(())
}
#[sqlx::test(fixtures("base"))]
async fn test_volume_extra_perms(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
// Insert volume with default (empty) extra_perms
sqlx::query!(
"INSERT INTO volume (workspace_id, name, size_bytes, created_by)
VALUES ($1, $2, $3, $4)",
"test-workspace",
"perms-vol",
100_i64,
"test-user"
)
.execute(&db)
.await?;
// Default extra_perms should be empty object
let row = sqlx::query!(
"SELECT extra_perms FROM volume WHERE workspace_id = $1 AND name = $2",
"test-workspace",
"perms-vol"
)
.fetch_one(&db)
.await?;
assert_eq!(row.extra_perms, serde_json::json!({}));
// Set extra_perms via jsonb_set (same pattern as granular_acls.rs)
sqlx::query!(
"UPDATE volume SET extra_perms = jsonb_set(extra_perms, $1, to_jsonb($2::bool), true)
WHERE workspace_id = $3 AND name = $4",
&vec!["u/alice".to_string()],
true,
"test-workspace",
"perms-vol"
)
.execute(&db)
.await?;
let row = sqlx::query!(
"SELECT extra_perms FROM volume WHERE workspace_id = $1 AND name = $2",
"test-workspace",
"perms-vol"
)
.fetch_one(&db)
.await?;
let perms = row.extra_perms.as_object().unwrap();
assert_eq!(perms.get("u/alice").and_then(|v| v.as_bool()), Some(true));
// Remove a permission entry
sqlx::query!(
"UPDATE volume SET extra_perms = extra_perms - $1
WHERE workspace_id = $2 AND name = $3",
"u/alice",
"test-workspace",
"perms-vol"
)
.execute(&db)
.await?;
let row = sqlx::query!(
"SELECT extra_perms FROM volume WHERE workspace_id = $1 AND name = $2",
"test-workspace",
"perms-vol"
)
.fetch_one(&db)
.await?;
assert_eq!(row.extra_perms, serde_json::json!({}));
Ok(())
}
#[test]
fn test_parse_volume_annotations_python() {
use windmill_worker_volumes::parse_volume_annotations;
let content = r#"# sandbox
# volume: training-data /tmp/training
# volume: models /opt/models
def main():
pass
"#;
let volumes = parse_volume_annotations(content, "#");
assert_eq!(volumes.len(), 2);
assert_eq!(volumes[0].name, "training-data");
assert_eq!(volumes[0].target, "/tmp/training");
assert_eq!(volumes[1].name, "models");
assert_eq!(volumes[1].target, "/opt/models");
}
#[test]
fn test_parse_volume_annotations_typescript() {
use windmill_worker_volumes::parse_volume_annotations;
let content = r#"// sandbox
// volume: datasets /tmp/datasets
export async function main() {
return "hello";
}
"#;
let volumes = parse_volume_annotations(content, "//");
assert_eq!(volumes.len(), 1);
assert_eq!(volumes[0].name, "datasets");
assert_eq!(volumes[0].target, "/tmp/datasets");
}
#[test]
fn test_parse_volume_annotations_no_prefix_match() {
use windmill_worker_volumes::parse_volume_annotations;
let content = "def main():\n pass";
let volumes = parse_volume_annotations(content, "#");
assert!(volumes.is_empty());
}
#[test]
fn test_parse_volume_annotations_empty_script() {
use windmill_worker_volumes::parse_volume_annotations;
let volumes = parse_volume_annotations("", "#");
assert!(volumes.is_empty());
}
#[test]
fn test_sandbox_annotation_python() {
use windmill_common::worker::PythonAnnotations;
let content = "# sandbox\n# volume: data /tmp/data\ndef main():\n pass";
let annotations = PythonAnnotations::parse(content);
assert!(annotations.sandbox);
}
#[test]
fn test_sandbox_annotation_typescript() {
use windmill_common::worker::TypeScriptAnnotations;
let content = "// sandbox\n// volume: data /tmp/data\nexport function main() {}";
let annotations = TypeScriptAnnotations::parse(content);
assert!(annotations.sandbox);
}
#[test]
fn test_volume_comment_prefix_selection() {
use windmill_common::scripts::ScriptLang;
let get_prefix = |lang: &ScriptLang| -> &str {
match lang {
ScriptLang::Python3
| ScriptLang::Bash
| ScriptLang::Powershell
| ScriptLang::Ansible
| ScriptLang::Ruby => "#",
ScriptLang::Deno
| ScriptLang::Bun
| ScriptLang::Bunnative
| ScriptLang::Nativets
| ScriptLang::Go => "//",
_ => "",
}
};
assert_eq!(get_prefix(&ScriptLang::Python3), "#");
assert_eq!(get_prefix(&ScriptLang::Bash), "#");
assert_eq!(get_prefix(&ScriptLang::Powershell), "#");
assert_eq!(get_prefix(&ScriptLang::Ansible), "#");
assert_eq!(get_prefix(&ScriptLang::Ruby), "#");
assert_eq!(get_prefix(&ScriptLang::Deno), "//");
assert_eq!(get_prefix(&ScriptLang::Bun), "//");
assert_eq!(get_prefix(&ScriptLang::Bunnative), "//");
assert_eq!(get_prefix(&ScriptLang::Nativets), "//");
assert_eq!(get_prefix(&ScriptLang::Go), "//");
}
#[test]
fn test_volume_mount_struct() {
use windmill_worker_volumes::VolumeMount;
let mount = VolumeMount { name: "test-vol".to_string(), target: "/mnt/data".to_string() };
assert_eq!(mount.name, "test-vol");
assert_eq!(mount.target, "/mnt/data");
}
#[test]
fn test_parse_volume_relative_path() {
use windmill_worker_volumes::parse_volume_annotations;
let content = "// volume: agent-memory .claude\nexport function main() {}";
let volumes = parse_volume_annotations(content, "//");
assert_eq!(volumes.len(), 1);
assert_eq!(volumes[0].name, "agent-memory");
assert_eq!(volumes[0].target, ".claude");
}
#[test]
fn test_parse_volume_relative_nested_path() {
use windmill_worker_volumes::parse_volume_annotations;
let content = "# volume: data data/models\ndef main():\n pass";
let volumes = parse_volume_annotations(content, "#");
assert_eq!(volumes.len(), 1);
assert_eq!(volumes[0].name, "data");
assert_eq!(volumes[0].target, "data/models");
}
#[cfg(feature = "private")]
#[test]
fn test_volume_nsjail_mount() {
use std::path::Path;
use windmill_worker_volumes::volume_nsjail_mount;
let result = volume_nsjail_mount(Path::new("/tmp/volumes/data"), "/mnt/data");
assert!(result.contains("src: \"/tmp/volumes/data\""));
assert!(result.contains("dst: \"/mnt/data\""));
assert!(result.contains("is_bind: true"));
assert!(result.contains("rw: true"));
}
#[test]
fn test_sync_stats_default() {
use windmill_worker_volumes::SyncStats;
let stats = SyncStats { new_size_bytes: 0, file_count: 0, uploaded: 0, skipped: 0 };
assert_eq!(stats.new_size_bytes, 0);
assert_eq!(stats.file_count, 0);
assert_eq!(stats.uploaded, 0);
assert_eq!(stats.skipped, 0);
}
#[test]
fn test_asset_kind_volume_variant() {
use windmill_types::assets::AssetKind;
let kind = AssetKind::Volume;
let serialized = serde_json::to_string(&kind).unwrap();
assert_eq!(serialized, "\"volume\"");
let deserialized: AssetKind = serde_json::from_str("\"volume\"").unwrap();
assert!(matches!(deserialized, AssetKind::Volume));
}
/// E2E test: run a bun script with volume mount through a SQL-connected worker.
/// Pre-populates the volume in filesystem storage, verifies the script can read
/// files and write new ones, then checks sync-back to storage and DB state.
#[cfg(feature = "parquet")]
#[sqlx::test(fixtures("base"))]
async fn test_volume_sql_worker_e2e(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
// 1. Set up filesystem-based object storage in a temp dir
let storage_dir = tempfile::tempdir()?;
let storage_root = storage_dir.path().to_string_lossy().to_string();
let lfs_config = json!({
"type": "FilesystemStorage",
"root_path": storage_root,
"public_resource": null,
"advanced_permissions": null,
"volume_storage": "primary"
});
sqlx::query!(
"UPDATE workspace_settings SET large_file_storage = $1 WHERE workspace_id = $2",
lfs_config,
"test-workspace"
)
.execute(&db)
.await?;
// 2. Pre-populate the volume with a file (workspace-namespaced path)
let vol_dir = storage_dir
.path()
.join("volumes")
.join("test-workspace")
.join("test-vol");
std::fs::create_dir_all(&vol_dir)?;
std::fs::write(vol_dir.join("hello.txt"), b"hello from volume")?;
// 3. Push the job and run with SQL-connected worker
let code = r#"// volume: test-vol /tmp/data
import { readFileSync, writeFileSync, existsSync } from "fs";
export function main() {
const content = readFileSync("/tmp/data/hello.txt", "utf-8");
writeFileSync("/tmp/data/output.txt", "written by sql worker");
return {
read_content: content,
output_exists: existsSync("/tmp/data/output.txt"),
};
}"#;
let job = JobPayload::Code(RawCode {
hash: None,
content: code.to_string(),
path: None,
language: ScriptLang::Bun,
lock: None,
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default()
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
});
let result = run_job_in_new_worker_until_complete(&db, false, job, port).await;
assert!(result.success, "job should succeed: {:?}", result.result);
let json = result.json_result().expect("should have JSON result");
assert_eq!(json["read_content"], json!("hello from volume"));
assert_eq!(json["output_exists"], json!(true));
// 4. Verify volume DB row was updated
let vol_row = sqlx::query!(
"SELECT size_bytes, file_count, leased_by, lease_until
FROM volume WHERE workspace_id = $1 AND name = $2",
"test-workspace",
"test-vol"
)
.fetch_optional(&db)
.await?;
let vol_row = vol_row.expect("volume row should exist");
assert!(
vol_row.file_count >= 2,
"should have at least 2 files (hello.txt + output.txt), got: {}",
vol_row.file_count
);
assert!(vol_row.size_bytes > 0, "size_bytes should be > 0");
assert!(vol_row.leased_by.is_none(), "lease should be released");
// 5. Verify the new file was written back to storage
let output_path = vol_dir.join("output.txt");
assert!(
output_path.exists(),
"output.txt should be synced back to storage"
);
let output_content = std::fs::read_to_string(&output_path)?;
assert_eq!(output_content, "written by sql worker");
Ok(())
}

View File

@@ -1,21 +1,17 @@
mod workspace_dependencies {
use windmill_test_utils::in_test_worker;
use windmill_test_utils::init_client;
use windmill_test_utils::listen_for_completed_jobs;
use sqlx::{Pool, Postgres};
use tokio_stream::StreamExt;
use windmill_common::scripts::ScriptLang;
use windmill_common::workspace_dependencies::WorkspaceDependencies;
use windmill_dep_map::workspace_dependencies::NewWorkspaceDependencies;
use windmill_test_utils::in_test_worker;
use windmill_test_utils::init_client;
use windmill_test_utils::listen_for_completed_jobs;
mod deps {
pub const REQUIREMENTS_IN: &'static str = "tiny==0.1.3";
// pub const GO_MOD: &'static str = r##"
// module example.com/project
// go 1.20
// require github.com/gin-gonic/gin v1.8.1
// "##;
pub const REQUIREMENTS_IN_V2: &'static str = "tiny==0.2.0";
pub const PACKAGE_JSON: &'static str = r##"
{
@@ -25,6 +21,18 @@ mod workspace_dependencies {
"express": "^4.17.1"
}
}
"##;
#[allow(dead_code)]
pub const PACKAGE_JSON_V2: &'static str = r##"
{
"name": "example-project",
"version": "2.0.0",
"dependencies": {
"express": "^4.18.0",
"axios": "^1.0.0"
}
}
"##;
pub const COMPOSER_JSON: &'static str = r##"
@@ -37,9 +45,510 @@ mod workspace_dependencies {
"##;
}
// =========================================================================
// CRUD Tests
// =========================================================================
/// Test: Create workspace dependencies and verify they are stored correctly.
#[sqlx::test(fixtures("base"))]
async fn test_create_workspace_dependencies(db: Pool<Postgres>) -> anyhow::Result<()> {
let id = NewWorkspaceDependencies {
workspace_id: "test-workspace".into(),
language: ScriptLang::Python3,
content: deps::REQUIREMENTS_IN.into(),
name: Some("test-deps".to_owned()),
description: Some("Test dependencies".to_owned()),
}
.create(
(
"test@test.com".to_owned(),
"u/test".to_owned(),
"test".to_owned(),
),
db.clone(),
)
.await?;
assert!(id > 0, "Should return a valid ID");
// Verify it was stored correctly
let stored = WorkspaceDependencies::get(id, "test-workspace".to_owned(), &db).await?;
assert_eq!(stored.name, Some("test-deps".to_owned()));
assert_eq!(stored.content, deps::REQUIREMENTS_IN);
assert_eq!(stored.language, ScriptLang::Python3);
Ok(())
}
/// Test: Create unnamed (default) workspace dependencies.
#[sqlx::test(fixtures("base"))]
async fn test_create_unnamed_workspace_dependencies(db: Pool<Postgres>) -> anyhow::Result<()> {
let id = NewWorkspaceDependencies {
workspace_id: "test-workspace".into(),
language: ScriptLang::Bun,
content: deps::PACKAGE_JSON.into(),
name: None, // Unnamed = default
description: None,
}
.create(
(
"test@test.com".to_owned(),
"u/test".to_owned(),
"test".to_owned(),
),
db.clone(),
)
.await?;
assert!(id > 0, "Should return a valid ID");
// Verify it was stored correctly
let stored = WorkspaceDependencies::get(id, "test-workspace".to_owned(), &db).await?;
assert_eq!(stored.name, None);
assert_eq!(stored.language, ScriptLang::Bun);
Ok(())
}
/// Test: List workspace dependencies returns all active entries.
#[sqlx::test(fixtures("base"))]
async fn test_list_workspace_dependencies(db: Pool<Postgres>) -> anyhow::Result<()> {
// Create multiple workspace dependencies
for (lang, content, name) in [
(ScriptLang::Python3, deps::REQUIREMENTS_IN, Some("python-deps")),
(ScriptLang::Bun, deps::PACKAGE_JSON, Some("bun-deps")),
(ScriptLang::Bun, deps::PACKAGE_JSON, None), // Default bun deps
] {
NewWorkspaceDependencies {
workspace_id: "test-workspace".into(),
language: lang,
content: content.into(),
name: name.map(|s| s.to_owned()),
description: None,
}
.create(
(
"test@test.com".to_owned(),
"u/test".to_owned(),
"test".to_owned(),
),
db.clone(),
)
.await?;
}
let list = WorkspaceDependencies::list("test-workspace", &db).await?;
assert_eq!(list.len(), 3, "Should have 3 workspace dependencies");
Ok(())
}
/// Test: Archive workspace dependencies marks them as archived.
#[sqlx::test(fixtures("base"))]
async fn test_archive_workspace_dependencies(db: Pool<Postgres>) -> anyhow::Result<()> {
// Create workspace dependencies
let _id = NewWorkspaceDependencies {
workspace_id: "test-workspace".into(),
language: ScriptLang::Python3,
content: deps::REQUIREMENTS_IN.into(),
name: Some("to-archive".to_owned()),
description: None,
}
.create(
(
"test@test.com".to_owned(),
"u/test".to_owned(),
"test".to_owned(),
),
db.clone(),
)
.await?;
// Verify it exists
let list_before = WorkspaceDependencies::list("test-workspace", &db).await?;
assert_eq!(list_before.len(), 1);
// Archive it
WorkspaceDependencies::archive(
Some("to-archive".to_owned()),
ScriptLang::Python3,
"test-workspace",
&db,
)
.await?;
// Verify it's no longer in the active list
let list_after = WorkspaceDependencies::list("test-workspace", &db).await?;
assert_eq!(list_after.len(), 0, "Archived deps should not appear in list");
Ok(())
}
/// Test: Delete workspace dependencies permanently removes them.
#[sqlx::test(fixtures("base"))]
async fn test_delete_workspace_dependencies(db: Pool<Postgres>) -> anyhow::Result<()> {
// Create workspace dependencies
let id = NewWorkspaceDependencies {
workspace_id: "test-workspace".into(),
language: ScriptLang::Python3,
content: deps::REQUIREMENTS_IN.into(),
name: Some("to-delete".to_owned()),
description: None,
}
.create(
(
"test@test.com".to_owned(),
"u/test".to_owned(),
"test".to_owned(),
),
db.clone(),
)
.await?;
// Verify it exists
assert!(
WorkspaceDependencies::get(id, "test-workspace".to_owned(), &db)
.await
.is_ok()
);
// Delete it
WorkspaceDependencies::delete(
Some("to-delete".to_owned()),
ScriptLang::Python3,
"test-workspace",
&db,
)
.await?;
// Verify it's gone (should error)
let result = WorkspaceDependencies::get(id, "test-workspace".to_owned(), &db).await;
assert!(result.is_err(), "Deleted deps should not be retrievable");
Ok(())
}
// =========================================================================
// Version History Tests
// =========================================================================
/// Test: Creating new version archives the old one.
#[sqlx::test(fixtures("base"))]
async fn test_versioning_archives_previous(db: Pool<Postgres>) -> anyhow::Result<()> {
// Create first version
let id1 = NewWorkspaceDependencies {
workspace_id: "test-workspace".into(),
language: ScriptLang::Python3,
content: deps::REQUIREMENTS_IN.into(),
name: Some("versioned".to_owned()),
description: Some("Version 1".to_owned()),
}
.create(
(
"test@test.com".to_owned(),
"u/test".to_owned(),
"test".to_owned(),
),
db.clone(),
)
.await?;
// Create second version with same name
let id2 = NewWorkspaceDependencies {
workspace_id: "test-workspace".into(),
language: ScriptLang::Python3,
content: deps::REQUIREMENTS_IN_V2.into(),
name: Some("versioned".to_owned()),
description: Some("Version 2".to_owned()),
}
.create(
(
"test@test.com".to_owned(),
"u/test".to_owned(),
"test".to_owned(),
),
db.clone(),
)
.await?;
assert_ne!(id1, id2, "Should create a new entry");
// List should only show the active (latest) version
let list = WorkspaceDependencies::list("test-workspace", &db).await?;
assert_eq!(list.len(), 1, "Should only have 1 active entry");
assert_eq!(list[0].content, deps::REQUIREMENTS_IN_V2);
// History should show both versions
let history = WorkspaceDependencies::get_history(
Some("versioned".to_owned()),
ScriptLang::Python3,
"test-workspace",
&db,
)
.await?;
assert_eq!(history.len(), 2, "Should have 2 versions in history");
Ok(())
}
/// Test: Description is inherited from previous version if not provided.
#[sqlx::test(fixtures("base"))]
async fn test_description_inheritance(db: Pool<Postgres>) -> anyhow::Result<()> {
// Create first version with description
NewWorkspaceDependencies {
workspace_id: "test-workspace".into(),
language: ScriptLang::Python3,
content: deps::REQUIREMENTS_IN.into(),
name: Some("inherit-desc".to_owned()),
description: Some("Original description".to_owned()),
}
.create(
(
"test@test.com".to_owned(),
"u/test".to_owned(),
"test".to_owned(),
),
db.clone(),
)
.await?;
// Create second version without description
let id2 = NewWorkspaceDependencies {
workspace_id: "test-workspace".into(),
language: ScriptLang::Python3,
content: deps::REQUIREMENTS_IN_V2.into(),
name: Some("inherit-desc".to_owned()),
description: None, // Should inherit
}
.create(
(
"test@test.com".to_owned(),
"u/test".to_owned(),
"test".to_owned(),
),
db.clone(),
)
.await?;
let stored = WorkspaceDependencies::get(id2, "test-workspace".to_owned(), &db).await?;
assert_eq!(
stored.description,
Some("Original description".to_owned()),
"Description should be inherited from previous version"
);
Ok(())
}
// =========================================================================
// Workspace Isolation Tests
// =========================================================================
/// Test: Workspace dependencies are isolated between workspaces.
#[sqlx::test(fixtures("base"))]
async fn test_workspace_isolation(db: Pool<Postgres>) -> anyhow::Result<()> {
// Create another workspace
sqlx::query!(
"INSERT INTO workspace (id, name, owner) VALUES ('other-workspace', 'other', 'test-user')"
)
.execute(&db)
.await?;
sqlx::query!("INSERT INTO workspace_settings (workspace_id) VALUES ('other-workspace')")
.execute(&db)
.await?;
// Create deps in test-workspace
NewWorkspaceDependencies {
workspace_id: "test-workspace".into(),
language: ScriptLang::Python3,
content: deps::REQUIREMENTS_IN.into(),
name: Some("shared-name".to_owned()),
description: None,
}
.create(
(
"test@test.com".to_owned(),
"u/test".to_owned(),
"test".to_owned(),
),
db.clone(),
)
.await?;
// Create deps in other-workspace with same name
NewWorkspaceDependencies {
workspace_id: "other-workspace".into(),
language: ScriptLang::Python3,
content: deps::REQUIREMENTS_IN_V2.into(),
name: Some("shared-name".to_owned()),
description: None,
}
.create(
(
"test@test.com".to_owned(),
"u/test".to_owned(),
"test".to_owned(),
),
db.clone(),
)
.await?;
// Each workspace should have exactly 1 entry
let list1 = WorkspaceDependencies::list("test-workspace", &db).await?;
let list2 = WorkspaceDependencies::list("other-workspace", &db).await?;
assert_eq!(list1.len(), 1);
assert_eq!(list2.len(), 1);
// Content should be different
assert_eq!(list1[0].content, deps::REQUIREMENTS_IN);
assert_eq!(list2[0].content, deps::REQUIREMENTS_IN_V2);
Ok(())
}
// =========================================================================
// Language-specific Tests
// =========================================================================
/// Test: Different languages can have same-named workspace dependencies.
#[sqlx::test(fixtures("base"))]
async fn test_same_name_different_languages(db: Pool<Postgres>) -> anyhow::Result<()> {
// Create Python deps
NewWorkspaceDependencies {
workspace_id: "test-workspace".into(),
language: ScriptLang::Python3,
content: deps::REQUIREMENTS_IN.into(),
name: Some("common".to_owned()),
description: None,
}
.create(
(
"test@test.com".to_owned(),
"u/test".to_owned(),
"test".to_owned(),
),
db.clone(),
)
.await?;
// Create Bun deps with same name
NewWorkspaceDependencies {
workspace_id: "test-workspace".into(),
language: ScriptLang::Bun,
content: deps::PACKAGE_JSON.into(),
name: Some("common".to_owned()),
description: None,
}
.create(
(
"test@test.com".to_owned(),
"u/test".to_owned(),
"test".to_owned(),
),
db.clone(),
)
.await?;
let list = WorkspaceDependencies::list("test-workspace", &db).await?;
assert_eq!(list.len(), 2, "Should have 2 entries (different languages)");
let python_deps: Vec<_> = list
.iter()
.filter(|d| d.language == ScriptLang::Python3)
.collect();
let bun_deps: Vec<_> = list
.iter()
.filter(|d| d.language == ScriptLang::Bun)
.collect();
assert_eq!(python_deps.len(), 1);
assert_eq!(bun_deps.len(), 1);
Ok(())
}
/// Test: Nativets and Bunnative use Bun workspace dependencies.
#[sqlx::test(fixtures("base"))]
async fn test_nativets_uses_bun_deps(db: Pool<Postgres>) -> anyhow::Result<()> {
use windmill_common::worker::Connection;
// Create Bun deps (which Nativets should use)
NewWorkspaceDependencies {
workspace_id: "test-workspace".into(),
language: ScriptLang::Bun,
content: deps::PACKAGE_JSON.into(),
name: None,
description: None,
}
.create(
(
"test@test.com".to_owned(),
"u/test".to_owned(),
"test".to_owned(),
),
db.clone(),
)
.await?;
// Query for Nativets should return Bun deps
let result = WorkspaceDependencies::get_latest(
None,
ScriptLang::Nativets,
"test-workspace",
Connection::Sql(db.clone()),
)
.await?;
assert!(result.is_some(), "Nativets should find Bun deps");
assert_eq!(result.unwrap().language, ScriptLang::Bun);
Ok(())
}
// =========================================================================
// Path Generation Tests
// =========================================================================
/// Test: to_path generates correct paths for named and unnamed deps.
#[test]
fn test_to_path_generation() {
// Unnamed (default) deps
let path = WorkspaceDependencies::to_path(&None, ScriptLang::Python3).unwrap();
assert_eq!(path, "dependencies/requirements.in");
let path = WorkspaceDependencies::to_path(&None, ScriptLang::Bun).unwrap();
assert_eq!(path, "dependencies/package.json");
let path = WorkspaceDependencies::to_path(&None, ScriptLang::Php).unwrap();
assert_eq!(path, "dependencies/composer.json");
// Named deps
let path =
WorkspaceDependencies::to_path(&Some("custom".to_owned()), ScriptLang::Python3).unwrap();
assert_eq!(path, "dependencies/custom.requirements.in");
let path =
WorkspaceDependencies::to_path(&Some("custom".to_owned()), ScriptLang::Bun).unwrap();
assert_eq!(path, "dependencies/custom.package.json");
}
/// Test: to_path returns error for unsupported languages.
#[test]
fn test_to_path_unsupported_language() {
// Deno doesn't support workspace dependencies
let result = WorkspaceDependencies::to_path(&None, ScriptLang::Deno);
assert!(result.is_err(), "Deno should not support workspace deps");
}
/// Test E2E: Creating named workspace dependencies triggers re-lock jobs for dependent scripts.
///
/// This test:
/// 1. Uses fixture with Python, Bun, PHP scripts linked to named workspace deps via dependency_map
/// 2. Creates named workspace dependencies for each language
/// 3. Verifies dependency jobs are triggered for all linked scripts
#[cfg(feature = "python")]
#[sqlx::test(fixtures("base", "workspace_dependencies_leafs"))]
#[ignore]
async fn basic_manual_named(db: Pool<Postgres>) -> anyhow::Result<()> {
let ((_client, port, _s), db, mut completed) = (
init_client(db.clone()).await,
@@ -47,67 +556,69 @@ mod workspace_dependencies {
listen_for_completed_jobs(&db).await,
);
for (idx, (l, c)) in [
// Create named workspace dependencies for Python, Bun, and PHP
// These will trigger dependency jobs for scripts linked via dependency_map
for (lang, content) in [
(ScriptLang::Python3, deps::REQUIREMENTS_IN),
(ScriptLang::Bun, deps::PACKAGE_JSON),
(ScriptLang::Php, deps::COMPOSER_JSON),
// (ScriptLang::Go, deps::GO_MOD),
]
.iter()
.enumerate()
{
let id = NewWorkspaceDependencies {
] {
NewWorkspaceDependencies {
workspace_id: "test-workspace".into(),
language: *l,
content: (*c).into(),
language: lang,
content: content.into(),
name: Some("test".to_owned()),
description: None,
}
.create(("".to_owned(), "".to_owned(), "".to_owned()), db.clone())
.await
.unwrap();
assert_eq!(idx + 1, id as usize);
.create(
(
"test@test.com".to_owned(),
"u/test-user".to_owned(),
"test-user".to_owned(),
),
db.clone(),
)
.await?;
}
// Wait for 4 jobs.
// Creating those dependencies will trigger redeployment of all scripts in workspace_dependencies_leafs.sql
in_test_worker(
db,
async {
completed.next().await;
completed.next().await;
completed.next().await;
// completed.next().await;
},
port,
)
.await;
// Wait for 3 dependency jobs (one per script in fixture)
let mut completed_paths = vec![];
for _ in 0..3 {
let job_id = in_test_worker(db, async { completed.next().await }, port)
.await
.expect("Expected a dependency job to complete");
// Verify all scripts have correct locks
// let mut langs = vec![];
// for r in sqlx::query!(
// r#"SELECT language AS "language: ScriptLang",lock FROM script WHERE archived = false"#
// )
// .fetch_all(db)
// .await
// .unwrap()
// {
// match r.language {
// ScriptLang::Python3 => assert_eq!("", &r.lock.unwrap()),
// ScriptLang::Go => todo!(),
// ScriptLang::Bun => todo!(),
// ScriptLang::Bunnative => todo!(),
// ScriptLang::Php => todo!(),
// _ => panic!("Unsupported language"),
// }
let job_path = sqlx::query_scalar!(
"SELECT runnable_path FROM v2_job WHERE id = $1",
job_id
)
.fetch_one(db)
.await?;
// langs.push(r.language);
// }
if let Some(path) = job_path {
completed_paths.push(path);
}
}
// langs.sort();
// // Just tiny additional verification for peace of mind.
// assert_eq!(langs.as_slice(), &[]);
// Verify all 3 scripts received dependency jobs
completed_paths.sort();
let expected = vec![
"f/leafs/php".to_string(),
"f/leafs/python".to_string(),
"f/leafs/ts".to_string(),
];
assert_eq!(
completed_paths, expected,
"All scripts should have received dependency jobs"
);
// Verify no extra jobs were created
let total_jobs = sqlx::query_scalar!("SELECT COUNT(*) FROM v2_job")
.fetch_one(db)
.await?;
assert_eq!(total_jobs, Some(3), "Should have exactly 3 jobs");
Ok(())
}

View File

@@ -51,4 +51,12 @@ impl AgentCache {
pub fn new() -> Self {
AgentCache {}
}
pub async fn extract_worker_name(
&self,
_token: &str,
_db: &windmill_common::DB,
) -> Option<String> {
None
}
}

View File

@@ -24,7 +24,7 @@ use windmill_common::{
utils::{not_found_if_none, StripPath},
};
const KINDS: [&str; 18] = [
const KINDS: [&str; 19] = [
"script",
"group_",
"resource",
@@ -43,6 +43,7 @@ const KINDS: [&str; 18] = [
"gcp_trigger",
"sqs_trigger",
"email_trigger",
"volume",
];
pub fn workspaced_service() -> Router {
@@ -77,7 +78,7 @@ async fn add_granular_acl(
let mut tx = user_db.begin(&authed).await?;
let identifier = if kind == "group_" || kind == "folder" {
let identifier = if kind == "group_" || kind == "folder" || kind == "volume" {
"name"
} else {
"path"
@@ -89,6 +90,22 @@ async fn add_granular_acl(
} else if kind == "group_" {
crate::groups::require_is_owner(path, &authed.username, &authed.groups, &w_id, &db)
.await?;
} else if kind == "volume" {
let created_by = sqlx::query_scalar!(
"SELECT created_by FROM volume WHERE name = $1 AND workspace_id = $2",
path,
&w_id
)
.fetch_optional(&db)
.await?
.ok_or_else(|| Error::NotFound(format!("volume '{path}' not found")))?;
// created_by is stored with u/ prefix (from job.permissioned_as)
let owner_username = created_by.strip_prefix("u/").unwrap_or(&created_by);
if owner_username != authed.username {
return Err(Error::NotAuthorized(
"Only the volume owner or an admin can modify permissions".to_string(),
));
}
} else {
require_owner_of_path(&authed, path)?;
}
@@ -243,6 +260,22 @@ async fn remove_granular_acl(
} else if kind == "group_" {
crate::groups::require_is_owner(path, &authed.username, &authed.groups, &w_id, &db)
.await?;
} else if kind == "volume" {
let created_by = sqlx::query_scalar!(
"SELECT created_by FROM volume WHERE name = $1 AND workspace_id = $2",
path,
&w_id
)
.fetch_optional(&db)
.await?
.ok_or_else(|| Error::NotFound(format!("volume '{path}' not found")))?;
// created_by is stored with u/ prefix (from job.permissioned_as)
let owner_username = created_by.strip_prefix("u/").unwrap_or(&created_by);
if owner_username != authed.username {
return Err(Error::NotAuthorized(
"Only the volume owner or an admin can modify permissions".to_string(),
));
}
} else {
require_owner_of_path(&authed, path)?;
}
@@ -250,7 +283,7 @@ async fn remove_granular_acl(
let mut tx = user_db.begin(&authed).await?;
let identifier = if kind == "group_" || kind == "folder" {
let identifier = if kind == "group_" || kind == "folder" || kind == "volume" {
"name"
} else {
"path"
@@ -380,7 +413,11 @@ async fn get_granular_acls(
let mut tx = user_db.begin(&authed).await?;
let identifier = if kind == "group_" { "name" } else { "path" };
let identifier = if kind == "group_" || kind == "folder" || kind == "volume" {
"name"
} else {
"path"
};
let obj_o = sqlx::query_scalar::<_, serde_json::Value>(&format!(
"SELECT extra_perms from {kind} WHERE {identifier} = $1 AND workspace_id = $2"
))

View File

@@ -302,6 +302,8 @@ struct LargeFileStorageWithSecondary {
large_file_storage: LargeFileStorage,
#[serde(default)]
secondary_storage: HashMap<String, LargeFileStorage>,
#[serde(default, skip_serializing_if = "Option::is_none")]
volume_storage: Option<String>,
}
#[derive(Deserialize, Debug)]
struct EditLargeFileStorageConfig {

View File

@@ -70,6 +70,7 @@ windmill-git-sync.workspace = true
windmill-indexer = { workspace = true, optional = true }
windmill-autoscaling = { workspace = true, optional = true }
windmill-worker = { workspace = true, optional = true }
windmill-worker-volumes.workspace = true
windmill-dep-map.workspace = true
tokio.workspace = true
tokio-stream.workspace = true

View File

@@ -1,7 +1,7 @@
openapi: "3.0.3"
info:
version: 1.650.0
version: 1.651.1
title: Windmill API
contact:
@@ -15198,6 +15198,7 @@ paths:
gcp_trigger,
sqs_trigger,
email_trigger,
volume,
]
responses:
"200":
@@ -15243,6 +15244,7 @@ paths:
gcp_trigger,
sqs_trigger,
email_trigger,
volume,
]
requestBody:
description: acl to add
@@ -15299,6 +15301,7 @@ paths:
gcp_trigger,
sqs_trigger,
email_trigger,
volume,
]
requestBody:
description: acl to add
@@ -17282,7 +17285,90 @@ paths:
path:
type: string
description: The asset path
/w/{workspace}/volumes/list:
get:
summary: List all volumes in the workspace
operationId: listVolumes
tags:
- volume
parameters:
- $ref: "#/components/parameters/WorkspaceId"
responses:
"200":
description: list of volumes
content:
application/json:
schema:
type: array
items:
$ref: "#/components/schemas/Volume"
/w/{workspace}/volumes/storage:
get:
summary: Get the volume storage name (secondary storage) or null for primary
operationId: getVolumeStorage
tags:
- volume
parameters:
- $ref: "#/components/parameters/WorkspaceId"
responses:
"200":
description: volume storage name or null
content:
application/json:
schema:
type: string
nullable: true
/w/{workspace}/volumes/create:
post:
summary: Create a new volume
operationId: createVolume
tags:
- volume
parameters:
- $ref: "#/components/parameters/WorkspaceId"
requestBody:
required: true
content:
application/json:
schema:
type: object
required:
- name
properties:
name:
type: string
responses:
"200":
description: volume created
content:
text/plain:
schema:
type: string
/w/{workspace}/volumes/delete/{name}:
delete:
summary: Delete a volume (admin only)
operationId: deleteVolume
tags:
- volume
parameters:
- $ref: "#/components/parameters/WorkspaceId"
- name: name
in: path
required: true
schema:
type: string
responses:
"200":
description: volume deleted
content:
text/plain:
schema:
type: string
/mcp/w/{workspace}/list_tools:
get:
@@ -23997,6 +24083,7 @@ components:
- resource
- ducklake
- datatable
- volume
Asset:
type: object
properties:
@@ -24005,6 +24092,38 @@ components:
kind:
$ref: "#/components/schemas/AssetKind"
required: [path, kind]
Volume:
type: object
required:
- name
- size_bytes
- file_count
- created_at
- created_by
properties:
name:
type: string
size_bytes:
type: integer
format: int64
file_count:
type: integer
created_at:
type: string
format: date-time
created_by:
type: string
updated_at:
type: string
format: date-time
nullable: true
last_used_at:
type: string
format: date-time
nullable: true
extra_perms:
type: object
additionalProperties: true
ProtectionRuleset:
type: object
description: A workspace protection rule defining restrictions and bypass permissions

View File

@@ -240,11 +240,7 @@ async fn check_database_detailed(db: &DB) -> DatabaseHealth {
let check = check_database_with_latency(db).await;
let pool = get_pool_stats(db);
DatabaseHealth {
healthy: check.healthy,
latency_ms: check.latency_ms,
pool,
}
DatabaseHealth { healthy: check.healthy, latency_ms: check.latency_ms, pool }
}
async fn check_worker_count(db: &DB) -> i64 {
@@ -295,13 +291,7 @@ async fn check_workers_detailed(db: &DB) -> WorkersHealth {
let healthy = active_count > 0;
WorkersHealth {
healthy,
active_count,
worker_groups,
min_version,
versions,
}
WorkersHealth { healthy, active_count, worker_groups, min_version, versions }
}
async fn check_queue(db: &DB) -> QueueHealth {
@@ -333,10 +323,7 @@ fn get_version() -> String {
/// Spawn a background task that performs a health check every 10 seconds.
/// Updates the cache and prometheus metrics continuously.
pub fn start_health_check_loop(
db: DB,
mut killpill_rx: tokio::sync::broadcast::Receiver<()>,
) {
pub fn start_health_check_loop(db: DB, mut killpill_rx: tokio::sync::broadcast::Receiver<()>) {
tokio::spawn(async move {
let mut interval = tokio::time::interval(Duration::from_secs(10));
interval.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Skip);
@@ -550,10 +537,7 @@ async fn health_status(
}
/// Detailed health check - requires DB authentication (always fresh, no caching)
async fn health_detailed(
_authed: ApiAuthed,
Extension(db): Extension<DB>,
) -> impl IntoResponse {
async fn health_detailed(_authed: ApiAuthed, Extension(db): Extension<DB>) -> impl IntoResponse {
let checked_at = Utc::now();
let database = check_database_detailed(&db).await;
let readiness = check_readiness();
@@ -564,12 +548,7 @@ async fn health_detailed(
status: HealthStatus::Unhealthy,
checked_at,
version: get_version(),
checks: HealthChecks {
database,
workers: None,
queue: None,
readiness,
},
checks: HealthChecks { database, workers: None, queue: None, readiness },
};
return (StatusCode::SERVICE_UNAVAILABLE, Json(response));
}
@@ -587,12 +566,7 @@ async fn health_detailed(
status,
checked_at,
version: get_version(),
checks: HealthChecks {
database,
workers: Some(workers),
queue: Some(queue),
readiness,
},
checks: HealthChecks { database, workers: Some(workers), queue: Some(queue), readiness },
};
let status_code = if status == HealthStatus::Unhealthy {

View File

@@ -12,15 +12,15 @@ use windmill_types::s3::StorageResourceType;
#[cfg(all(feature = "parquet", not(feature = "private")))]
use crate::db::{ApiAuthed, OptJobAuthed, DB};
#[cfg(all(feature = "parquet", not(feature = "private")))]
use windmill_object_store::object_store_reexports::{ObjectStore, PutMultipartOpts, PutResult};
#[cfg(not(feature = "private"))]
use windmill_object_store::ObjectStoreResource;
#[cfg(all(feature = "parquet", not(feature = "private")))]
use std::sync::Arc;
#[cfg(all(feature = "parquet", not(feature = "private")))]
use windmill_common::db::UserDB;
#[cfg(not(feature = "private"))]
use windmill_common::error;
#[cfg(all(feature = "parquet", not(feature = "private")))]
use windmill_common::db::UserDB;
use windmill_object_store::object_store_reexports::{ObjectStore, PutMultipartOpts, PutResult};
#[cfg(not(feature = "private"))]
use windmill_object_store::ObjectStoreResource;
#[cfg(all(feature = "parquet", not(feature = "private")))]
use bytes::Bytes;

View File

@@ -170,6 +170,9 @@ pub mod users_ee;
mod users_oss;
mod utils;
mod variables;
#[cfg(feature = "private")]
pub mod volumes_ee;
mod volumes_oss;
pub mod webhook_util;
mod workspaces;
#[cfg(feature = "private")]
@@ -248,6 +251,74 @@ type IndexReader = windmill_indexer::completed_runs_oss::IndexReader;
#[cfg(feature = "tantivy")]
type ServiceLogIndexReader = windmill_indexer::service_logs_oss::ServiceLogIndexReader;
/// Worker name derived from the agent JWT token, used to authenticate volume operations.
/// Defined unconditionally so volume endpoint handlers can reference it regardless of
/// whether agent_worker_server is enabled (the extension is only populated on the agent path).
#[derive(Clone)]
pub struct AgentWorkerName(pub String);
/// Middleware that injects a synthetic `ApiAuthed` and JWT-derived worker name
/// into request extensions.
///
/// Used for volume proxy endpoints under the agent_workers path, where the
/// agent JWT auth layer has already validated the request. The volume handlers
/// need `ApiAuthed` to resolve the workspace S3 client, but the agent JWT
/// format is incompatible with the standard auth extractor.
///
/// The worker name is extracted from the JWT claims rather than trusting
/// self-reported values in request bodies/query params.
#[cfg(feature = "agent_worker_server")]
async fn inject_agent_authed(
request: axum::extract::Request,
next: axum::middleware::Next,
) -> Response {
let mut request = request;
// Extract worker name from agent JWT via AgentCache
// (OSS returns None; EE decodes the JWT and returns the worker name)
{
let extracted = {
let token = request
.headers()
.get(axum::http::header::AUTHORIZATION)
.and_then(|v| v.to_str().ok())
.and_then(|s| s.strip_prefix("Bearer ").map(|t| t.to_string()));
let cache = request.extensions().get::<Arc<AgentCache>>().cloned();
let db = request.extensions().get::<DB>().cloned();
match (token, cache, db) {
(Some(token), Some(cache), Some(db)) => Some((token, cache, db)),
_ => None,
}
};
if let Some((token, cache, db)) = extracted {
if let Some(worker_name) = cache.extract_worker_name(&token, &db).await {
request
.extensions_mut()
.insert(AgentWorkerName(worker_name));
}
}
}
request
.extensions_mut()
.insert(windmill_api_auth::OptJobAuthed {
authed: ApiAuthed {
email: "agent-worker@windmill.dev".to_string(),
username: "agent-worker".to_string(),
is_admin: true,
is_operator: false,
groups: Vec::new(),
folders: Vec::new(),
scopes: None,
username_override: None,
token_prefix: None,
},
job_id: None,
});
next.run(request).await
}
pub async fn run_server(
db: DB,
job_index_reader: Option<IndexReader>,
@@ -513,6 +584,7 @@ pub async fn run_server(
users::workspaced_service().layer(Extension(argon2.clone())),
)
.nest("/variables", variables::workspaced_service())
.nest("/volumes", volumes_oss::workspaced_service())
.nest("/workers", windmill_api_workers::workspaced_service())
.nest("/workspaces", workspaces::workspaced_service())
.nest("/oidc", oidc_oss::workspaced_service())
@@ -626,7 +698,13 @@ pub async fn run_server(
.nest("/w/:workspace_id/agent_workers", {
#[cfg(feature = "agent_worker_server")]
{
agent_workers_router.layer(Extension(agent_cache.clone()))
agent_workers_router
.nest(
"/volumes",
volumes_oss::agent_workspaced_service()
.layer(axum::middleware::from_fn(inject_agent_authed)),
)
.layer(Extension(agent_cache.clone()))
}
#[cfg(not(feature = "agent_worker_server"))]
{

View File

@@ -64,9 +64,10 @@ impl McpBackend for WindmillBackend {
auth: &ApiAuthed,
workspace_id: &str,
favorites_only: bool,
path_prefix: Option<&str>,
) -> BackendResult<Vec<ScriptInfo>> {
let scope_type = if favorites_only { "favorites" } else { "all" };
get_items::<ScriptInfo>(&self.user_db, auth, workspace_id, scope_type, "script")
get_items::<ScriptInfo>(&self.user_db, auth, workspace_id, scope_type, "script", path_prefix)
.await
.map_err(|e| ErrorData::internal_error(e.message, None))
}
@@ -76,9 +77,10 @@ impl McpBackend for WindmillBackend {
auth: &ApiAuthed,
workspace_id: &str,
favorites_only: bool,
path_prefix: Option<&str>,
) -> BackendResult<Vec<FlowInfo>> {
let scope_type = if favorites_only { "favorites" } else { "all" };
get_items::<FlowInfo>(&self.user_db, auth, workspace_id, scope_type, "flow")
get_items::<FlowInfo>(&self.user_db, auth, workspace_id, scope_type, "flow", path_prefix)
.await
.map_err(|e| ErrorData::internal_error(e.message, None))
}

View File

@@ -136,6 +136,7 @@ pub async fn get_items<T: for<'a> sqlx::FromRow<'a, sqlx::postgres::PgRow> + Sen
workspace_id: &str,
scope_type: &str,
item_type: &str,
path_prefix: Option<&str>,
) -> Result<Vec<T>, ErrorData> {
let mut sqlb = SqlBuilder::select_from(&format!("{} as o", item_type));
let fields = vec!["o.path", "o.summary", "o.description", "o.schema"];
@@ -153,6 +154,11 @@ pub async fn get_items<T: for<'a> sqlx::FromRow<'a, sqlx::postgres::PgRow> + Sen
sqlb.and_where("(o.no_main_func IS NOT TRUE OR o.no_main_func IS NULL)");
}
if let Some(prefix) = path_prefix {
let escaped = prefix.replace('\\', "\\\\").replace('%', "\\%").replace('_', "\\_");
sqlb.and_where("o.path LIKE ? ESCAPE '\\'".bind(&format!("{}%", escaped)));
}
sqlb.order_by(
if item_type == "flow" {
"o.edited_at"

View File

@@ -369,7 +369,11 @@ async fn route_job(
let s3_object = s3_client.get(&path).await;
let s3_object = match s3_object {
Err(windmill_object_store::object_store_reexports::ObjectStoreError::NotFound { .. }) if trigger.is_static_website => {
Err(
windmill_object_store::object_store_reexports::ObjectStoreError::NotFound {
..
},
) if trigger.is_static_website => {
// fallback to index.html if the file is not found
let path = windmill_object_store::object_store_reexports::Path::from(format!(
"{}/index.html",

View File

@@ -0,0 +1,17 @@
#[cfg(feature = "private")]
#[allow(unused)]
pub use crate::volumes_ee::*;
#[cfg(not(feature = "private"))]
use axum::Router;
#[cfg(not(feature = "private"))]
pub fn workspaced_service() -> Router {
Router::new()
}
#[cfg(not(feature = "private"))]
#[allow(dead_code)]
pub fn agent_workspaced_service() -> Router {
Router::new()
}

View File

@@ -72,6 +72,7 @@ pub fn asset_kind_from_parser(parser_kind: windmill_parser::asset_parser::AssetK
windmill_parser::asset_parser::AssetKind::Resource => AssetKind::Resource,
windmill_parser::asset_parser::AssetKind::Ducklake => AssetKind::Ducklake,
windmill_parser::asset_parser::AssetKind::DataTable => AssetKind::DataTable,
windmill_parser::asset_parser::AssetKind::Volume => AssetKind::Volume,
}
}

View File

@@ -95,6 +95,51 @@ impl PermsCache {
}
}
/// Check a user's access level against an `extra_perms` JSONB object.
///
/// Returns `None` if the user has no matching entry (no access).
/// Returns `Some(true)` if the user (or any of their groups) has write access.
/// Returns `Some(false)` if the user (or any of their groups) has read-only access.
pub fn check_extra_perms(
extra_perms: &serde_json::Map<String, serde_json::Value>,
username: &str,
groups: &[String],
) -> Option<bool> {
// Check direct user permission
let user_key = if username.starts_with("u/") {
username.to_string()
} else {
format!("u/{username}")
};
if let Some(v) = extra_perms.get(&user_key) {
return Some(v.as_bool().unwrap_or(false));
}
// Check group permissions — return highest access level found
let mut found = false;
let mut write = false;
for g in groups {
let key = if g.starts_with("g/") {
g.to_string()
} else {
format!("g/{g}")
};
if let Some(v) = extra_perms.get(&key) {
found = true;
if v.as_bool().unwrap_or(false) {
write = true;
break;
}
}
}
if found {
Some(write)
} else {
None
}
}
pub fn has_expired(expiration_time: DateTime<Utc>, take: Option<Duration>) -> bool {
let now = Utc::now();

View File

@@ -354,6 +354,45 @@ impl HttpClient {
)))
}
}
pub async fn get_bytes(&self, url: &str) -> anyhow::Result<Bytes> {
let base_url = self.base_internal_url.clone();
let response = self
.client
.get(format!("{}{}", base_url, url))
.send()
.await
.map_err(|e| anyhow::anyhow!(e))?;
if response.status().is_success() {
Ok(response.bytes().await?)
} else {
Err(anyhow::anyhow!(
"HTTP agent request GET {} failed {}",
url,
response.status()
))
}
}
pub async fn put_bytes(&self, url: &str, bytes: Bytes) -> anyhow::Result<()> {
let base_url = self.base_internal_url.clone();
let response = self
.client
.put(format!("{}{}", base_url, url))
.body(bytes)
.send()
.await
.map_err(|e| anyhow::anyhow!(e))?;
if response.status().is_success() {
Ok(())
} else {
Err(anyhow::anyhow!(
"HTTP agent request PUT {} failed {}",
url,
response.status()
))
}
}
}
#[derive(Clone)]
@@ -698,6 +737,7 @@ pub struct PythonAnnotations {
pub py311: bool,
pub py312: bool,
pub py313: bool,
pub sandbox: bool,
}
#[annotations("//")]
@@ -711,6 +751,7 @@ pub struct TypeScriptAnnotations {
pub nodejs: bool,
pub native: bool,
pub nobundling: bool,
pub sandbox: bool,
}
#[annotations("--")]
@@ -2169,4 +2210,62 @@ mod tests {
);
assert_ne!(a, b);
}
#[test]
fn test_python_sandbox_annotation() {
let content = "# sandbox\ndef main():\n pass";
let annotations = PythonAnnotations::parse(content);
assert!(annotations.sandbox);
}
#[test]
fn test_python_sandbox_annotation_with_other_annotations() {
let content = "# no_cache\n# sandbox\ndef main():\n pass";
let annotations = PythonAnnotations::parse(content);
assert!(annotations.sandbox);
assert!(annotations.no_cache);
}
#[test]
fn test_python_no_sandbox_annotation() {
let content = "# no_cache\ndef main():\n pass";
let annotations = PythonAnnotations::parse(content);
assert!(!annotations.sandbox);
}
#[test]
fn test_typescript_sandbox_annotation() {
let content = "// sandbox\nexport function main() {}";
let annotations = TypeScriptAnnotations::parse(content);
assert!(annotations.sandbox);
}
#[test]
fn test_typescript_sandbox_annotation_with_other_annotations() {
let content = "// npm\n// sandbox\nexport function main() {}";
let annotations = TypeScriptAnnotations::parse(content);
assert!(annotations.sandbox);
assert!(annotations.npm);
}
#[test]
fn test_typescript_no_sandbox_annotation() {
let content = "// npm\nexport function main() {}";
let annotations = TypeScriptAnnotations::parse(content);
assert!(!annotations.sandbox);
}
#[test]
fn test_python_sandbox_no_space() {
let content = "#sandbox\ndef main():\n pass";
let annotations = PythonAnnotations::parse(content);
assert!(annotations.sandbox);
}
#[test]
fn test_typescript_sandbox_no_space() {
let content = "//sandbox\nexport function main() {}";
let annotations = TypeScriptAnnotations::parse(content);
assert!(annotations.sandbox);
}
}

View File

@@ -11,6 +11,8 @@ pub mod types;
pub use schema::convert_schema_to_schema_type;
pub use scope::{is_resource_allowed, parse_mcp_scopes, McpScopeConfig};
pub use transform::{
apply_key_transformation, reverse_transform, reverse_transform_key, transform_path,
apply_key_transformation, extract_hub_version_id_from_hashed,
extract_path_prefix_from_hashed, parse_tool_prefix, reverse_transform, reverse_transform_key,
transform_hub_path, transform_path,
};
pub use types::*;

View File

@@ -4,9 +4,15 @@
//! to make them compatible with MCP tool naming requirements.
use super::types::SchemaType;
use windmill_common::utils::calculate_hash;
/// MCP clients do not allow names longer than 60 characters
const MAX_PATH_LENGTH: usize = 60;
/// Max tool name length. The MCP spec allows 64 chars, but some clients
/// (e.g. Cursor) prepend the server name to the tool name, so we use 40
/// to leave room for that prefix.
const MAX_PATH_LENGTH: usize = 40;
/// Length of the SHA256 hash suffix used for hashed names
const HASH_LEN: usize = 16;
/// Transform the path for workspace scripts/flows
///
@@ -14,19 +20,133 @@ const MAX_PATH_LENGTH: usize = 60;
/// path with the type prefix. This is used when listing, because we can't
/// have names with slashes. Because we replace slashes with underscores,
/// we also need to escape underscores.
///
/// For short names (≤40 chars): `s-{escaped_path}` or `f-{escaped_path}`
/// For long names (>40 chars): `S-{escaped[:22]}{sha256[:16]}` or `F-{escaped[:22]}{sha256[:16]}`
///
/// The uppercase prefix signals that the name is hashed.
pub fn transform_path(path: &str, type_str: &str) -> String {
let escaped_path = path.replace('_', "__").replace('/', "_");
// first letter of type_str is used as prefix, only one letter to avoid reaching 60 char name limit
let transformed_path = format!("{}-{}", &type_str[..1], escaped_path);
if transformed_path.len() > MAX_PATH_LENGTH {
let suffix = "_TRUNC";
return format!(
"{}{}",
&transformed_path[..MAX_PATH_LENGTH - suffix.len()],
suffix
);
let prefix_char = &type_str[..1];
let short_name = format!("{}-{}", prefix_char, escaped_path);
if short_name.len() <= MAX_PATH_LENGTH {
return short_name;
}
transformed_path
let upper_prefix = prefix_char.to_uppercase();
// Layout: "{Upper}-" (2 chars) + prefix_body (22 chars) + hash (16 chars) = 40
let prefix_body_len = MAX_PATH_LENGTH - 2 - HASH_LEN;
let hash = calculate_hash(&short_name);
let hash_suffix = &hash[..HASH_LEN];
let truncated = truncate_to_char_boundary(&escaped_path, prefix_body_len);
format!("{}-{}{}", upper_prefix, truncated, hash_suffix)
}
/// Transform the path for hub scripts
///
/// For short names (≤40 chars): `hs-{id}-{summary}`
/// For long names (>40 chars): `Hs-{id}-{summary[:N]}{sha256[:16]}`
pub fn transform_hub_path(version_id: u64, summary: &str) -> String {
let escaped_summary = summary.replace(' ', "_");
let short_name = format!("hs-{}-{}", version_id, escaped_summary);
if short_name.len() <= MAX_PATH_LENGTH {
return short_name;
}
let hash = calculate_hash(&short_name);
let hash_suffix = &hash[..HASH_LEN];
// "Hs-{id}-" prefix, then fill remaining with summary + hash
let fixed_prefix = format!("Hs-{}-", version_id);
let available = MAX_PATH_LENGTH - fixed_prefix.len() - HASH_LEN;
let truncated_summary = truncate_to_char_boundary(&escaped_summary, available);
format!("{}{}{}", fixed_prefix, truncated_summary, hash_suffix)
}
/// Parse the prefix of any tool name (both short and hashed).
/// Returns `(type_str, is_hub, is_hashed)`.
/// Hashed names use an uppercase first character as the signal.
pub fn parse_tool_prefix(name: &str) -> Result<(&str, bool, bool), String> {
let is_hashed = name.chars().next().map(|c| c.is_ascii_uppercase()).unwrap_or(false);
let lower = name.to_ascii_lowercase();
let (type_str, is_hub) = if lower.starts_with("hs-") {
("script", true)
} else if lower.starts_with("s-") {
("script", false)
} else if lower.starts_with("f-") {
("flow", false)
} else {
return Err(format!("Invalid tool name prefix: {}", name));
};
Ok((type_str, is_hub, is_hashed))
}
/// Extract the hub version_id from a hashed hub script name like `Hs-{id}-...`
pub fn extract_hub_version_id_from_hashed(name: &str) -> Result<String, String> {
let rest = name
.strip_prefix("Hs-")
.ok_or_else(|| format!("Not a hashed hub name: {}", name))?;
let id = rest
.split('-')
.next()
.ok_or_else(|| format!("No version_id in hashed hub name: {}", name))?;
if id.is_empty() {
return Err(format!("Empty version_id in hashed hub name: {}", name));
}
Ok(id.to_string())
}
/// Extract a safe original-path prefix from a hashed tool name.
///
/// Given `S-u_admin_engineering__te<hash16>`, extracts the escaped prefix between
/// the type prefix (`S-`, `F-`, or `Hs-`) and the hash, un-escapes it, and
/// returns a prefix suitable for `WHERE path LIKE '{prefix}%'`.
///
/// Returns `None` if the name is too short or has an unrecognized prefix.
pub fn extract_path_prefix_from_hashed(name: &str) -> Option<String> {
let prefix_len = if name.starts_with("Hs-") {
3
} else if name.starts_with("S-") || name.starts_with("F-") {
2
} else {
return None;
};
if name.len() <= prefix_len + HASH_LEN {
return None;
}
let escaped_prefix = &name[prefix_len..name.len() - HASH_LEN];
if escaped_prefix.is_empty() {
return None;
}
// Strip trailing underscores — they may be half of a `__` pair split by truncation
let trimmed = escaped_prefix.trim_end_matches('_');
if trimmed.is_empty() {
return None;
}
Some(unescape_path(trimmed))
}
/// Un-escape a mangled path segment: `__` → `_`, standalone `_` → `/`.
fn unescape_path(s: &str) -> String {
const TEMP_PLACEHOLDER: &str = "@@UNDERSCORE@@";
s.replace("__", TEMP_PLACEHOLDER)
.replace('_', "/")
.replace(TEMP_PLACEHOLDER, "_")
}
/// Truncate a string to at most `max_len` bytes, ensuring we don't split a UTF-8 character.
fn truncate_to_char_boundary(s: &str, max_len: usize) -> &str {
if s.len() <= max_len {
return s;
}
let mut end = max_len;
while end > 0 && !s.is_char_boundary(end) {
end -= 1;
}
&s[..end]
}
/// Reverse the transformation of a path
@@ -38,25 +158,22 @@ pub fn transform_path(path: &str, type_str: &str) -> String {
/// This is used in call_tool to get the original path, and the type of the item.
///
/// Returns: (type, original_path, is_hub)
///
/// Note: This only works for non-hashed (short) names. Hashed names must be
/// resolved via `parse_tool_prefix` + path enumeration in the runner.
pub fn reverse_transform(transformed_path: &str) -> Result<(&str, String, bool), String> {
let is_hub = transformed_path.starts_with("h");
let transformed_path = if is_hub {
transformed_path[1..].to_string()
} else {
transformed_path.to_string()
};
let type_str = if transformed_path.starts_with("s-") {
"script"
} else if transformed_path.starts_with("f-") {
"flow"
} else {
return Err(format!(
"Invalid prefix in transformed path: {}",
transformed_path
));
};
let (type_str, is_hub, is_hashed) = parse_tool_prefix(transformed_path)?;
let mangled_path = &transformed_path[2..];
if is_hashed {
return Err(
"Hashed names cannot be reverse-transformed directly; use path enumeration instead"
.to_string(),
);
}
// Strip the prefix: "hs-" (3 chars) for hub, "s-"/"f-" (2 chars) for others
let prefix_len = if is_hub { 3 } else { 2 };
let mangled_path = &transformed_path[prefix_len..];
let original_path = if is_hub {
let parts = mangled_path.split("-").collect::<Vec<&str>>();
@@ -65,11 +182,7 @@ pub fn reverse_transform(transformed_path: &str) -> Result<(&str, String, bool),
}
parts[0].to_string()
} else {
const TEMP_PLACEHOLDER: &str = "@@UNDERSCORE@@";
mangled_path
.replace("__", TEMP_PLACEHOLDER)
.replace('_', "/")
.replace(TEMP_PLACEHOLDER, "_")
unescape_path(mangled_path)
};
Ok((type_str, original_path, is_hub))
@@ -97,16 +210,13 @@ pub fn reverse_transform_key(transformed_key: &str, schema_obj: &Option<SchemaTy
let schema_obj = match schema_obj {
Some(s) => s,
None => {
// No schema available, return the key as is (best guess)
return transformed_key.to_string();
}
};
for original_key_in_schema in schema_obj.properties.keys() {
// Apply the SAME forward transformation to the schema key
let potential_transformed_key = apply_key_transformation(original_key_in_schema);
// If it matches the key we received, we found the likely original
if potential_transformed_key == transformed_key {
return original_key_in_schema.clone();
}
@@ -120,7 +230,7 @@ mod tests {
use super::*;
#[test]
fn test_transform_path() {
fn test_transform_path_short() {
assert_eq!(
transform_path("u/admin/script", "script"),
"s-u_admin_script"
@@ -130,7 +240,108 @@ mod tests {
}
#[test]
fn test_reverse_transform() {
fn test_transform_path_long_is_hashed() {
let long_path = "u/engineering/team/automation/very_long_script_name_that_exceeds_limit";
let result = transform_path(long_path, "script");
assert_eq!(result.len(), MAX_PATH_LENGTH);
assert!(result.starts_with("S-"));
let (_, _, is_hashed) = parse_tool_prefix(&result).unwrap();
assert!(is_hashed);
}
#[test]
fn test_transform_path_long_flow_is_hashed() {
let long_path = "f/engineering/team/automation/very_long_flow_name_that_exceeds_limit";
let result = transform_path(long_path, "flow");
assert_eq!(result.len(), MAX_PATH_LENGTH);
assert!(result.starts_with("F-"));
let (_, _, is_hashed) = parse_tool_prefix(&result).unwrap();
assert!(is_hashed);
}
#[test]
fn test_transform_path_hashing_is_deterministic() {
let path = "u/engineering/team/automation/very_long_script_name_that_exceeds_limit";
let a = transform_path(path, "script");
let b = transform_path(path, "script");
assert_eq!(a, b);
}
#[test]
fn test_transform_path_different_long_paths_differ() {
let a = transform_path(
"u/engineering/team/automation/very_long_script_name_that_exceeds_limit_a",
"script",
);
let b = transform_path(
"u/engineering/team/automation/very_long_script_name_that_exceeds_limit_b",
"script",
);
assert_ne!(a, b);
}
#[test]
fn test_transform_hub_path_short() {
let result = transform_hub_path(12345, "Send Slack Message");
assert_eq!(result, "hs-12345-Send_Slack_Message");
let (_, _, is_hashed) = parse_tool_prefix(&result).unwrap();
assert!(!is_hashed);
}
#[test]
fn test_transform_hub_path_long_is_hashed() {
let result = transform_hub_path(
12345,
"Send Slack Message To Channel With Very Long Description That Exceeds Limit",
);
assert_eq!(result.len(), MAX_PATH_LENGTH);
assert!(result.starts_with("Hs-12345-"));
let (_, _, is_hashed) = parse_tool_prefix(&result).unwrap();
assert!(is_hashed);
}
#[test]
fn test_extract_hub_version_id_from_hashed() {
let name = "Hs-12345-Send_Slack_Message_To_Ch9e8d7c6b5a4f3e2d";
let id = extract_hub_version_id_from_hashed(name).unwrap();
assert_eq!(id, "12345");
}
#[test]
fn test_parse_tool_prefix() {
let (t, hub, hashed) = parse_tool_prefix("S-something").unwrap();
assert_eq!(t, "script");
assert!(!hub);
assert!(hashed);
let (t, hub, hashed) = parse_tool_prefix("F-something").unwrap();
assert_eq!(t, "flow");
assert!(!hub);
assert!(hashed);
let (t, hub, hashed) = parse_tool_prefix("Hs-12345-something").unwrap();
assert_eq!(t, "script");
assert!(hub);
assert!(hashed);
let (t, hub, hashed) = parse_tool_prefix("s-u_admin_script").unwrap();
assert_eq!(t, "script");
assert!(!hub);
assert!(!hashed);
let (t, hub, hashed) = parse_tool_prefix("f-f_folder_flow").unwrap();
assert_eq!(t, "flow");
assert!(!hub);
assert!(!hashed);
let (t, hub, hashed) = parse_tool_prefix("hs-12345-summary").unwrap();
assert_eq!(t, "script");
assert!(hub);
assert!(!hashed);
}
#[test]
fn test_reverse_transform_short_names() {
let (type_str, path, is_hub) = reverse_transform("s-u_admin_script").unwrap();
assert_eq!(type_str, "script");
assert_eq!(path, "u/admin/script");
@@ -142,6 +353,70 @@ mod tests {
assert!(!is_hub);
}
#[test]
fn test_extract_path_prefix_from_hashed() {
// Generate a real hashed name and verify prefix extraction
let long_path = "u/admin/engineering/team/automation/very_long_script";
let hashed = transform_path(long_path, "script");
let (_, _, is_hashed) = parse_tool_prefix(&hashed).unwrap();
assert!(is_hashed);
let prefix = extract_path_prefix_from_hashed(&hashed).unwrap();
// The original path should start with the extracted prefix
assert!(
long_path.starts_with(&prefix),
"path '{}' should start with prefix '{}'",
long_path,
prefix
);
}
#[test]
fn test_extract_path_prefix_underscore_in_path() {
let long_path = "u/admin/my_team/automation/very_long_script_name_here";
let hashed = transform_path(long_path, "script");
let prefix = extract_path_prefix_from_hashed(&hashed).unwrap();
assert!(
long_path.starts_with(&prefix),
"path '{}' should start with prefix '{}'",
long_path,
prefix
);
}
#[test]
fn test_extract_path_prefix_rejects_invalid_prefix() {
assert!(extract_path_prefix_from_hashed("x-something").is_none());
assert!(extract_path_prefix_from_hashed("").is_none());
assert!(extract_path_prefix_from_hashed("S-").is_none());
}
#[test]
fn test_extract_path_prefix_handles_hs_prefix() {
// Hs- is 3 chars, not 2 — ensure the prefix is stripped correctly
let hashed = transform_hub_path(12345, "a]very long hub script summary that exceeds the limit");
let (_, is_hub, is_hashed) = parse_tool_prefix(&hashed).unwrap();
assert!(is_hub);
assert!(is_hashed);
let prefix = extract_path_prefix_from_hashed(&hashed);
// Should not start with 's' (leftover from Hs- if sliced at index 2)
if let Some(ref p) = prefix {
assert!(
!p.starts_with('s'),
"prefix '{}' should not start with 's' from mis-sliced Hs- prefix",
p
);
}
}
#[test]
fn test_reverse_transform_rejects_hashed_names() {
assert!(reverse_transform("S-something").is_err());
assert!(reverse_transform("F-something").is_err());
assert!(reverse_transform("Hs-12345-something").is_err());
}
#[test]
fn test_apply_key_transformation() {
assert_eq!(apply_key_transformation("my key"), "my_key");

View File

@@ -92,8 +92,10 @@ pub struct ItemSchema {
/// Trait for objects that can be converted to MCP tools
pub trait ToolableItem {
/// Get the path or identifier for this item (transformed for MCP compatibility)
fn get_path_or_id(&self) -> String;
/// Get the MCP-compatible tool name (path transformed with escaping/hashing)
fn get_transformed_path(&self) -> String;
/// Get the original full path of this item (for display in tool title)
fn get_full_path(&self) -> &str;
/// Get the summary/title of this item
fn get_summary(&self) -> &str;
/// Get the description of this item

View File

@@ -14,9 +14,9 @@ pub mod client;
// Re-export common types at crate root for convenience
pub use common::{
convert_schema_to_schema_type, is_resource_allowed, parse_mcp_scopes, transform_path, FlowInfo,
HubResponse, HubScriptInfo, ItemSchema, McpScopeConfig, ResourceInfo, ResourceType, SchemaType,
ScriptInfo, ToolableItem, WorkspaceId,
convert_schema_to_schema_type, is_resource_allowed, parse_mcp_scopes, transform_hub_path,
transform_path, FlowInfo, HubResponse, HubScriptInfo, ItemSchema, McpScopeConfig, ResourceInfo,
ResourceType, SchemaType, ScriptInfo, ToolableItem, WorkspaceId,
};
// Re-export client types at crate root for backward compatibility

View File

@@ -55,20 +55,22 @@ pub trait McpBackend: Send + Sync + Clone + 'static {
// Listing Operations
// ─────────────────────────────────────────────────────────────────
/// List scripts, optionally filtered to favorites only
/// List scripts, optionally filtered to favorites only and/or by path prefix
async fn list_scripts(
&self,
auth: &Self::Auth,
workspace_id: &str,
favorites_only: bool,
path_prefix: Option<&str>,
) -> BackendResult<Vec<ScriptInfo>>;
/// List flows, optionally filtered to favorites only
/// List flows, optionally filtered to favorites only and/or by path prefix
async fn list_flows(
&self,
auth: &Self::Auth,
workspace_id: &str,
favorites_only: bool,
path_prefix: Option<&str>,
) -> BackendResult<Vec<FlowInfo>>;
/// List resource types in workspace

View File

@@ -5,7 +5,10 @@
use crate::common::schema::extract_resource_types_from_schema;
use crate::common::scope::parse_mcp_scopes;
use crate::common::transform::{reverse_transform, reverse_transform_key};
use crate::common::transform::{
extract_hub_version_id_from_hashed, extract_path_prefix_from_hashed, parse_tool_prefix,
reverse_transform, reverse_transform_key,
};
use crate::common::types::{ResourceInfo, ToolableItem, WorkspaceId};
use crate::server::backend::{McpAuth, McpBackend};
use crate::server::endpoints::endpoint_tool_to_mcp_tool;
@@ -81,6 +84,13 @@ impl<B: McpBackend> Runner<B> {
}
}
fn find_matching_path<T: ToolableItem>(candidates: Vec<T>, request_name: &str) -> Option<String> {
candidates
.into_iter()
.find(|item| item.get_transformed_path() == request_name)
.map(|item| item.get_full_path().to_string())
}
impl<B: McpBackend> ServerHandler for Runner<B> {
fn get_info(&self) -> ServerInfo {
ServerInfo {
@@ -120,9 +130,9 @@ impl<B: McpBackend> ServerHandler for Runner<B> {
// Fetch all items concurrently
let (scripts, flows, resource_types, hub_scripts) = tokio::try_join!(
self.backend
.list_scripts(&auth, &workspace_id, favorites_only),
.list_scripts(&auth, &workspace_id, favorites_only, None),
self.backend
.list_flows(&auth, &workspace_id, favorites_only),
.list_flows(&auth, &workspace_id, favorites_only, None),
self.backend.list_resource_types(&auth, &workspace_id),
async {
if let Some(ref apps) = scope_config.hub_apps {
@@ -231,17 +241,6 @@ impl<B: McpBackend> ServerHandler for Runner<B> {
let scope_config =
parse_mcp_scopes(scopes).map_err(|e| ErrorData::internal_error(e, None))?;
// Handle truncated tool names
if request.name.ends_with("_TRUNC") {
return Ok(CallToolResult::error(vec![rmcp::model::Annotated::new(
rmcp::model::RawContent::Text(rmcp::model::RawTextContent {
text: "Tool path is too long. Consider shortening it to make it compatible with MCP.".to_string(),
meta: None,
}),
None,
)]));
}
let args = request.arguments.map(Value::Object).unwrap_or(Value::Null);
// Check if this is an endpoint tool
@@ -274,10 +273,58 @@ impl<B: McpBackend> ServerHandler for Runner<B> {
}
}
// Not an endpoint tool - parse as script/flow
let (tool_type, path, is_hub) = reverse_transform(&request.name).map_err(|e| {
ErrorData::internal_error(format!("Failed to parse tool name: {}", e), None)
})?;
// Resolve the tool name to (type, path, is_hub)
let (type_str, is_hub, is_hashed) =
parse_tool_prefix(&request.name).map_err(|e| {
ErrorData::internal_error(format!("Failed to parse tool name: {}", e), None)
})?;
let (tool_type, path, is_hub) = if !is_hashed {
reverse_transform(&request.name).map_err(|e| {
ErrorData::internal_error(format!("Failed to parse tool name: {}", e), None)
})?
} else if is_hub {
let version_id =
extract_hub_version_id_from_hashed(&request.name).map_err(|e| {
ErrorData::internal_error(
format!("Failed to extract hub version_id: {}", e),
None,
)
})?;
(type_str, version_id, true)
} else {
let path_prefix = extract_path_prefix_from_hashed(&request.name);
let favorites_only = scope_config.favorites;
let matched_path = if type_str == "script" {
find_matching_path(
self.backend
.list_scripts(&auth, &workspace_id, favorites_only, path_prefix.as_deref())
.await
.map_err(|e| ErrorData::internal_error(e.message, None))?,
&request.name,
)
} else {
find_matching_path(
self.backend
.list_flows(&auth, &workspace_id, favorites_only, path_prefix.as_deref())
.await
.map_err(|e| ErrorData::internal_error(e.message, None))?,
&request.name,
)
};
let matched_path = matched_path.ok_or_else(|| {
ErrorData::internal_error(
format!(
"No {} found matching hashed tool name '{}'",
type_str, request.name
),
None,
)
})?;
(type_str, matched_path, false)
};
// Validate script/flow scope
if !is_hub && scope_config.granular {

View File

@@ -9,7 +9,7 @@ use std::collections::HashMap;
use std::sync::Arc;
use crate::common::schema::{convert_schema_to_schema_type, make_schema_compatible};
use crate::common::transform::transform_path;
use crate::common::transform::{transform_hub_path, transform_path};
use crate::common::types::{
FlowInfo, HubScriptInfo, ResourceInfo, ResourceType, SchemaType, ScriptInfo, ToolableItem,
};
@@ -17,10 +17,14 @@ use crate::server::backend::McpBackend;
/// Implementation of ToolableItem for ScriptInfo
impl ToolableItem for ScriptInfo {
fn get_path_or_id(&self) -> String {
fn get_transformed_path(&self) -> String {
transform_path(&self.path, "script")
}
fn get_full_path(&self) -> &str {
&self.path
}
fn get_summary(&self) -> &str {
self.summary.as_deref().unwrap_or("No summary")
}
@@ -48,10 +52,14 @@ impl ToolableItem for ScriptInfo {
/// Implementation of ToolableItem for FlowInfo
impl ToolableItem for FlowInfo {
fn get_path_or_id(&self) -> String {
fn get_transformed_path(&self) -> String {
transform_path(&self.path, "flow")
}
fn get_full_path(&self) -> &str {
&self.path
}
fn get_summary(&self) -> &str {
self.summary.as_deref().unwrap_or("No summary")
}
@@ -79,10 +87,13 @@ impl ToolableItem for FlowInfo {
/// Implementation of ToolableItem for HubScriptInfo
impl ToolableItem for HubScriptInfo {
fn get_path_or_id(&self) -> String {
let id = self.version_id;
fn get_transformed_path(&self) -> String {
let summary = self.summary.as_deref().unwrap_or("No summary");
format!("hs-{}-{}", id, summary.replace(" ", "_"))
transform_hub_path(self.version_id, summary)
}
fn get_full_path(&self) -> &str {
self.summary.as_deref().unwrap_or("No summary")
}
fn get_summary(&self) -> &str {
@@ -124,7 +135,7 @@ pub fn create_tool_from_item<T: ToolableItem, B: McpBackend>(
resources_types: &[ResourceType],
) -> Tool {
let is_hub = item.is_hub();
let path = item.get_path_or_id();
let path = item.get_transformed_path();
let item_type = item.item_type();
let description = format!(
"This is a {} named `{}` with the following description: `{}`.{}",
@@ -170,15 +181,24 @@ pub fn create_tool_from_item<T: ToolableItem, B: McpBackend>(
}
};
let title = {
let summary = item.get_summary();
if summary == "No summary" {
item.get_full_path().to_string()
} else {
summary.to_string()
}
};
Tool {
name: Cow::Owned(path),
description: Some(Cow::Owned(description)),
input_schema: Arc::new(input_schema_map),
title: Some(item.get_summary().to_string()),
title: Some(title.clone()),
output_schema: None,
icons: None,
annotations: Some(ToolAnnotations {
title: Some(item.get_summary().to_string()),
title: Some(title),
read_only_hint: Some(false), // Can modify environment
destructive_hint: Some(true), // Can potentially be destructive
idempotent_hint: Some(false), // Are not guaranteed to be idempotent

View File

@@ -726,7 +726,7 @@ pub async fn get_token_by_prefix<'c, E: sqlx::Executor<'c, Database = Postgres>>
) -> Result<Option<String>> {
let token = sqlx::query_scalar!(
r#"
SELECT token
SELECT token as "token!"
FROM token
WHERE token LIKE concat($1::text, '%')
LIMIT 1

View File

@@ -151,6 +151,7 @@ pub struct RunJob {
pub args: serde_json::Map<String, serde_json::Value>,
pub scheduled_for_o: Option<chrono::DateTime<chrono::Utc>>,
pub email: String,
pub job_id: Option<Uuid>,
}
impl From<JobPayload> for RunJob {
@@ -160,6 +161,7 @@ impl From<JobPayload> for RunJob {
args: Default::default(),
scheduled_for_o: None,
email: "test@windmill.dev".to_string(),
job_id: None,
}
}
}
@@ -183,8 +185,13 @@ impl RunJob {
self
}
pub fn job_id(mut self, id: Uuid) -> Self {
self.job_id = Some(id);
self
}
pub async fn push(self, db: &Pool<Postgres>) -> Uuid {
let RunJob { payload, args, scheduled_for_o, email } = self;
let RunJob { payload, args, scheduled_for_o, email, job_id } = self;
let mut hm_args = std::collections::HashMap::new();
for (k, v) in args {
hm_args.insert(k, windmill_common::worker::to_raw_value(&v));
@@ -206,7 +213,7 @@ impl RunJob {
/* parent_job */ None,
/* root job */ None,
/* flow_innermost_root_job */ None,
/* job_id */ None,
/* job_id */ job_id,
/* is_flow_step */ false,
/* same_worker */ false,
None,
@@ -230,7 +237,7 @@ impl RunJob {
/// Push the job as a specific user (for testing permissions)
pub async fn push_as(self, db: &Pool<Postgres>, username: &str, email: &str) -> Uuid {
let RunJob { payload, args, scheduled_for_o, .. } = self;
let RunJob { payload, args, scheduled_for_o, job_id, .. } = self;
let mut hm_args = std::collections::HashMap::new();
for (k, v) in args {
hm_args.insert(k, windmill_common::worker::to_raw_value(&v));
@@ -252,7 +259,7 @@ impl RunJob {
/* parent_job */ None,
/* root job */ None,
/* flow_innermost_root_job */ None,
/* job_id */ None,
/* job_id */ job_id,
/* is_flow_step */ false,
/* same_worker */ false,
None,
@@ -830,6 +837,15 @@ pub async fn run_preview_relative_imports(
#[cfg(all(feature = "private", feature = "agent_worker_server"))]
pub async fn testing_http_connection(port: u16) -> Connection {
testing_http_connection_with_tags(
port,
vec!["flow".into(), "python3".into(), "dependency".into()],
)
.await
}
#[cfg(all(feature = "private", feature = "agent_worker_server"))]
pub async fn testing_http_connection_with_tags(port: u16, tags: Vec<String>) -> Connection {
let suffix = windmill_common::utils::create_default_worker_suffix("test-agent-worker");
let agent_token = format!(
"{}{}",
@@ -837,7 +853,7 @@ pub async fn testing_http_connection(port: u16) -> Connection {
windmill_common::jwt::encode_with_internal_secret(windmill_api_agent_workers::AgentAuth {
worker_group: "testing-agent".to_owned(),
suffix: Some(suffix.clone()),
tags: vec!["flow".into(), "python3".into(), "dependency".into()],
tags,
exp: Some(usize::MAX),
})
.await

View File

@@ -13,6 +13,7 @@ pub enum AssetKind {
Variable, // Deprecated
Ducklake,
DataTable,
Volume,
}
#[derive(

View File

@@ -0,0 +1,29 @@
[package]
name = "windmill-worker-volumes"
version.workspace = true
authors.workspace = true
edition.workspace = true
[lib]
name = "windmill_worker_volumes"
path = "src/lib.rs"
[features]
enterprise = []
private = []
[dependencies]
windmill-common = { workspace = true, default-features = false }
object_store.workspace = true
tokio.workspace = true
tracing.workspace = true
bytes.workspace = true
futures.workspace = true
serde.workspace = true
serde_json.workspace = true
regex.workspace = true
lazy_static.workspace = true
md-5.workspace = true
[dev-dependencies]
tempfile.workspace = true

View File

@@ -0,0 +1,544 @@
#[cfg(feature = "private")]
mod volume_ee;
mod volume_oss;
pub use volume_oss::*;
pub use object_store::ObjectStore as DynObjectStore;
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
use std::collections::HashSet;
use std::path::PathBuf;
pub const MAX_VOLUMES_PER_JOB: usize = 10;
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct FileEntry {
pub size: u64,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub md5: Option<String>,
}
pub fn compute_md5_hex(data: &[u8]) -> String {
use md5::{Digest, Md5};
let result = Md5::digest(data);
const HEX: &[u8; 16] = b"0123456789abcdef";
let mut hex = String::with_capacity(32);
for &b in result.iter() {
hex.push(HEX[(b >> 4) as usize] as char);
hex.push(HEX[(b & 0x0f) as usize] as char);
}
hex
}
/// Extract an MD5 hash from an S3 ETag, if it's a simple (non-multipart) ETag.
pub fn etag_to_md5(e_tag: Option<&str>) -> Option<String> {
let tag = e_tag?.trim_matches('"');
// Multipart ETags contain a '-' (e.g. "abc123-5"), skip those
if tag.contains('-') || tag.is_empty() {
return None;
}
Some(tag.to_string())
}
lazy_static::lazy_static! {
static ref ARGS_INTERPOLATION_RE: regex::Regex =
regex::Regex::new(r#"\$args\[((?:\w+\.)*\w+)\]"#).unwrap();
static ref VALID_VOLUME_NAME_RE: regex::Regex =
regex::Regex::new(r"^[a-zA-Z0-9][a-zA-Z0-9._-]{0,253}[a-zA-Z0-9]$").unwrap();
}
#[derive(Debug, Clone, PartialEq)]
pub struct VolumeMount {
pub name: String,
pub target: String,
}
pub struct VolumeState {
pub mount: VolumeMount,
pub local_dir: PathBuf,
pub manifest: HashMap<String, FileEntry>,
pub symlinks: HashMap<String, String>,
}
pub struct DownloadStats {
pub total_files: usize,
pub from_cache: usize,
pub downloaded: usize,
}
pub struct SyncStats {
pub new_size_bytes: u64,
pub file_count: usize,
pub uploaded: usize,
pub skipped: usize,
}
pub fn validate_volume_name(name: &str) -> Result<(), String> {
if name.contains("..") {
return Err(format!(
"Volume name '{}' contains '..' which is not allowed",
name
));
}
if !VALID_VOLUME_NAME_RE.is_match(name) {
return Err(format!(
"Volume name '{}' is invalid. Names must be 2-255 characters, \
start and end with alphanumeric, and contain only alphanumeric, '.', '_', or '-'",
name
));
}
Ok(())
}
const ALLOWED_ABSOLUTE_PREFIXES: &[&str] = &["/tmp/", "/mnt/", "/opt/", "/home/", "/data/"];
pub fn validate_volume_target(target: &str) -> Result<(), String> {
if target.split('/').any(|seg| seg == "..") {
return Err(format!(
"Volume target '{target}' contains '..' segments which is not allowed"
));
}
if target.starts_with('/')
&& !ALLOWED_ABSOLUTE_PREFIXES
.iter()
.any(|p| target.starts_with(p))
{
return Err(format!(
"Volume target '{target}' must be a relative path or start with one of: {}",
ALLOWED_ABSOLUTE_PREFIXES.join(", ")
));
}
Ok(())
}
pub fn validate_volume_mounts(mounts: &[VolumeMount]) -> Result<(), String> {
if mounts.len() > MAX_VOLUMES_PER_JOB {
return Err(format!(
"Too many volume mounts ({}, max {})",
mounts.len(),
MAX_VOLUMES_PER_JOB
));
}
let mut seen_names = HashSet::new();
let mut seen_targets = HashSet::new();
for v in mounts {
if !seen_names.insert(&v.name) {
return Err(format!("Duplicate volume name: '{}'", v.name));
}
if !seen_targets.insert(&v.target) {
return Err(format!("Duplicate volume target: '{}'", v.target));
}
}
Ok(())
}
pub fn interpolate_volume_name(
name: &str,
args: Option<&HashMap<String, Box<serde_json::value::RawValue>>>,
workspace_id: &str,
) -> String {
let name = name.replace("$workspace", workspace_id);
if !name.contains("$args[") {
return name;
}
let Some(args) = args else {
return name;
};
let mut result = name.clone();
for cap in ARGS_INTERPOLATION_RE.captures_iter(&name) {
let full_match = cap.get(0).unwrap().as_str();
let arg_name = cap.get(1).unwrap().as_str();
let arg_value = if arg_name.contains('.') {
let parts: Vec<&str> = arg_name.split('.').collect();
let root = parts[0];
let mut value = args
.get(root)
.map(|x| x.get().to_string())
.unwrap_or_default();
for part in parts.iter().skip(1) {
if let Ok(obj) = serde_json::from_str::<serde_json::Value>(&value) {
value = obj
.get(part)
.map(|v| v.to_string())
.unwrap_or_default()
.to_string();
} else {
value = String::new();
break;
}
}
value.trim_matches('"').to_string()
} else {
args.get(arg_name)
.map(|x| x.get().trim_matches('"').to_string())
.unwrap_or_default()
};
result = result.replace(full_match, &arg_value);
}
result
}
pub fn parse_volume_annotations(content: &str, comment_prefix: &str) -> Vec<VolumeMount> {
let mut volumes = Vec::new();
for line in content.lines() {
let trimmed = line.trim();
if trimmed.is_empty() {
continue;
}
if !trimmed.starts_with(comment_prefix) {
break;
}
let after_prefix = trimmed[comment_prefix.len()..].trim();
if let Some(rest) = after_prefix.strip_prefix("volume:") {
let rest = rest.trim();
let mut parts = rest.splitn(2, char::is_whitespace);
if let (Some(name), Some(target)) = (parts.next(), parts.next()) {
let name = name.trim();
let target = target.trim();
if !name.is_empty() && !target.is_empty() {
volumes
.push(VolumeMount { name: name.to_string(), target: target.to_string() });
}
}
}
}
volumes
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn parse_python_single_volume() {
let content = "# volume: mydata /tmp/data\ndef main():\n pass";
let result = parse_volume_annotations(content, "#");
assert_eq!(
result,
vec![VolumeMount { name: "mydata".to_string(), target: "/tmp/data".to_string() }]
);
}
#[test]
fn parse_typescript_single_volume() {
let content = "// volume: mydata /tmp/data\nexport function main() {}";
let result = parse_volume_annotations(content, "//");
assert_eq!(
result,
vec![VolumeMount { name: "mydata".to_string(), target: "/tmp/data".to_string() }]
);
}
#[test]
fn parse_multiple_volumes() {
let content = "# volume: data1 /tmp/data1\n# volume: data2 /tmp/data2\n# volume: models /opt/models\ndef main():\n pass";
let result = parse_volume_annotations(content, "#");
assert_eq!(
result,
vec![
VolumeMount { name: "data1".to_string(), target: "/tmp/data1".to_string() },
VolumeMount { name: "data2".to_string(), target: "/tmp/data2".to_string() },
VolumeMount { name: "models".to_string(), target: "/opt/models".to_string() },
]
);
}
#[test]
fn parse_mixed_annotations_and_volumes() {
let content = "# sandbox\n# volume: mydata /tmp/data\ndef main():\n pass";
let result = parse_volume_annotations(content, "#");
assert_eq!(
result,
vec![VolumeMount { name: "mydata".to_string(), target: "/tmp/data".to_string() }]
);
}
#[test]
fn parse_no_volumes() {
let content = "# sandbox\ndef main():\n pass";
let result = parse_volume_annotations(content, "#");
assert!(result.is_empty());
}
#[test]
fn parse_empty_content() {
let result = parse_volume_annotations("", "#");
assert!(result.is_empty());
}
#[test]
fn parse_stops_at_non_comment_line() {
let content =
"# volume: data1 /tmp/data1\ndef main():\n # volume: data2 /tmp/data2\n pass";
let result = parse_volume_annotations(content, "#");
assert_eq!(
result,
vec![VolumeMount { name: "data1".to_string(), target: "/tmp/data1".to_string() }]
);
}
#[test]
fn parse_skips_blank_lines_in_header() {
let content =
"# volume: data1 /tmp/data1\n\n# volume: data2 /tmp/data2\ndef main():\n pass";
let result = parse_volume_annotations(content, "#");
assert_eq!(
result,
vec![
VolumeMount { name: "data1".to_string(), target: "/tmp/data1".to_string() },
VolumeMount { name: "data2".to_string(), target: "/tmp/data2".to_string() },
]
);
}
#[test]
fn parse_ignores_malformed_volume_lines() {
let content =
"# volume:\n# volume: onlyname\n# volume: good /tmp/good\ndef main():\n pass";
let result = parse_volume_annotations(content, "#");
assert_eq!(
result,
vec![VolumeMount { name: "good".to_string(), target: "/tmp/good".to_string() }]
);
}
#[test]
fn parse_extra_whitespace() {
let content = "# volume: mydata /tmp/data \ndef main():\n pass";
let result = parse_volume_annotations(content, "#");
assert_eq!(
result,
vec![VolumeMount { name: "mydata".to_string(), target: "/tmp/data".to_string() }]
);
}
#[test]
fn parse_target_with_spaces_in_path() {
let content = "// volume: mydata /tmp/my data dir\nexport function main() {}";
let result = parse_volume_annotations(content, "//");
assert_eq!(
result,
vec![VolumeMount {
name: "mydata".to_string(),
target: "/tmp/my data dir".to_string(),
}]
);
}
#[test]
fn parse_volume_with_dashes_and_underscores() {
let content = "# volume: my-data_v2 /tmp/data\ndef main():\n pass";
let result = parse_volume_annotations(content, "#");
assert_eq!(
result,
vec![VolumeMount { name: "my-data_v2".to_string(), target: "/tmp/data".to_string() }]
);
}
#[test]
fn interpolate_workspace() {
let name = "$workspace-data";
let result = interpolate_volume_name(name, None, "my_ws");
assert_eq!(result, "my_ws-data");
}
#[test]
fn interpolate_args_simple() {
let mut args = HashMap::new();
args.insert(
"env".to_string(),
serde_json::value::RawValue::from_string("\"prod\"".to_string()).unwrap(),
);
let result = interpolate_volume_name("data-$args[env]", Some(&args), "ws");
assert_eq!(result, "data-prod");
}
#[test]
fn interpolate_args_and_workspace() {
let mut args = HashMap::new();
args.insert(
"env".to_string(),
serde_json::value::RawValue::from_string("\"staging\"".to_string()).unwrap(),
);
let result = interpolate_volume_name("$workspace-$args[env]-cache", Some(&args), "acme");
assert_eq!(result, "acme-staging-cache");
}
#[test]
fn interpolate_no_placeholders() {
let result = interpolate_volume_name("plain-name", None, "ws");
assert_eq!(result, "plain-name");
}
#[test]
fn interpolate_missing_arg() {
let args = HashMap::new();
let result = interpolate_volume_name("data-$args[missing]", Some(&args), "ws");
assert_eq!(result, "data-");
}
#[test]
fn interpolate_nested_arg() {
let mut args = HashMap::new();
args.insert(
"config".to_string(),
serde_json::value::RawValue::from_string(
r#"{"env": "prod", "region": "us-east"}"#.to_string(),
)
.unwrap(),
);
let result = interpolate_volume_name(
"data-$args[config.env]-$args[config.region]",
Some(&args),
"ws",
);
assert_eq!(result, "data-prod-us-east");
}
#[test]
fn parse_wrong_prefix_returns_empty() {
let content = "# volume: mydata /tmp/data\ndef main():\n pass";
let result = parse_volume_annotations(content, "//");
assert!(result.is_empty());
}
#[test]
fn parse_relative_path() {
let content = "// volume: agent-memory .claude\nexport function main() {}";
let result = parse_volume_annotations(content, "//");
assert_eq!(
result,
vec![VolumeMount { name: "agent-memory".to_string(), target: ".claude".to_string() }]
);
}
#[test]
fn parse_relative_nested_path() {
let content = "# volume: data data/models\ndef main():\n pass";
let result = parse_volume_annotations(content, "#");
assert_eq!(
result,
vec![VolumeMount { name: "data".to_string(), target: "data/models".to_string() }]
);
}
#[test]
fn validate_valid_names() {
assert!(validate_volume_name("mydata").is_ok());
assert!(validate_volume_name("my-data_v2").is_ok());
assert!(validate_volume_name("acme-staging-cache").is_ok());
assert!(validate_volume_name("a1").is_ok());
assert!(validate_volume_name("data.v2").is_ok());
assert!(validate_volume_name("A0").is_ok());
}
#[test]
fn validate_rejects_path_traversal() {
assert!(validate_volume_name("../other-workspace").is_err());
assert!(validate_volume_name("data/../secrets").is_err());
assert!(validate_volume_name("a..b").is_err());
}
#[test]
fn validate_rejects_special_start_end() {
assert!(validate_volume_name("-data").is_err());
assert!(validate_volume_name("data-").is_err());
assert!(validate_volume_name(".data").is_err());
assert!(validate_volume_name("data.").is_err());
assert!(validate_volume_name("_data").is_err());
}
#[test]
fn validate_rejects_path_separators() {
assert!(validate_volume_name("data/secrets").is_err());
assert!(validate_volume_name("data\\secrets").is_err());
}
#[test]
fn validate_rejects_too_short() {
assert!(validate_volume_name("").is_err());
assert!(validate_volume_name("a").is_err());
}
#[test]
fn validate_rejects_too_long() {
let long_name = format!("a{}a", "b".repeat(254));
assert!(validate_volume_name(&long_name).is_err());
}
#[test]
fn validate_rejects_spaces_and_special() {
assert!(validate_volume_name("my data").is_err());
assert!(validate_volume_name("my@data").is_err());
assert!(validate_volume_name("my$data").is_err());
}
#[test]
fn validate_target_allows_relative() {
assert!(validate_volume_target("data").is_ok());
assert!(validate_volume_target("data/models").is_ok());
assert!(validate_volume_target(".claude").is_ok());
}
#[test]
fn validate_target_allows_safe_absolute() {
assert!(validate_volume_target("/tmp/data").is_ok());
assert!(validate_volume_target("/mnt/data").is_ok());
assert!(validate_volume_target("/opt/models").is_ok());
assert!(validate_volume_target("/home/user/data").is_ok());
assert!(validate_volume_target("/data/cache").is_ok());
}
#[test]
fn validate_target_rejects_dangerous_absolute() {
assert!(validate_volume_target("/etc/passwd").is_err());
assert!(validate_volume_target("/proc/self").is_err());
assert!(validate_volume_target("/sys/fs").is_err());
assert!(validate_volume_target("/dev/null").is_err());
assert!(validate_volume_target("/usr/bin").is_err());
assert!(validate_volume_target("/var/log").is_err());
}
#[test]
fn validate_target_rejects_traversal() {
assert!(validate_volume_target("../../etc").is_err());
assert!(validate_volume_target("data/../../../etc").is_err());
assert!(validate_volume_target("/tmp/../etc/passwd").is_err());
}
#[test]
fn validate_mounts_rejects_too_many() {
let mounts: Vec<VolumeMount> = (0..11)
.map(|i| VolumeMount { name: format!("v{:02}", i), target: format!("t{}", i) })
.collect();
assert!(validate_volume_mounts(&mounts).is_err());
}
#[test]
fn validate_mounts_rejects_duplicate_name() {
let mounts = vec![
VolumeMount { name: "data".to_string(), target: "/tmp/a".to_string() },
VolumeMount { name: "data".to_string(), target: "/tmp/b".to_string() },
];
assert!(validate_volume_mounts(&mounts).is_err());
}
#[test]
fn validate_mounts_rejects_duplicate_target() {
let mounts = vec![
VolumeMount { name: "v1".to_string(), target: "/tmp/data".to_string() },
VolumeMount { name: "v2".to_string(), target: "/tmp/data".to_string() },
];
assert!(validate_volume_mounts(&mounts).is_err());
}
#[test]
fn validate_mounts_ok() {
let mounts = vec![
VolumeMount { name: "v1".to_string(), target: "/tmp/a".to_string() },
VolumeMount { name: "v2".to_string(), target: "/tmp/b".to_string() },
];
assert!(validate_volume_mounts(&mounts).is_ok());
}
}

View File

@@ -0,0 +1,116 @@
#[cfg(feature = "private")]
pub use crate::volume_ee::*;
#[cfg(not(feature = "private"))]
use crate::{DownloadStats, SyncStats, VolumeMount, VolumeState};
#[cfg(not(feature = "private"))]
use object_store::ObjectStore;
#[cfg(not(feature = "private"))]
use std::path::Path;
#[cfg(not(feature = "private"))]
use std::sync::Arc;
#[cfg(not(feature = "private"))]
use windmill_common::error;
#[cfg(not(feature = "private"))]
pub async fn download_volume(
_client: Arc<dyn ObjectStore>,
_volume: &VolumeMount,
_job_dir: &str,
_workspace_id: &str,
) -> error::Result<(VolumeState, DownloadStats)> {
Err(error::Error::internal_err(
"Volumes are not available in this build".to_string(),
))
}
#[cfg(not(feature = "private"))]
pub fn volume_nsjail_mount(_local_dir: &Path, _target: &str) -> String {
String::new()
}
#[cfg(not(feature = "private"))]
pub async fn sync_volume_back(
_client: Arc<dyn ObjectStore>,
_state: &VolumeState,
_workspace_id: &str,
) -> error::Result<SyncStats> {
Err(error::Error::internal_err(
"Volumes are not available in this build".to_string(),
))
}
#[cfg(not(feature = "private"))]
pub fn walk_dir(dir: &Path) -> std::io::Result<Vec<std::path::PathBuf>> {
let mut result = Vec::new();
walk_dir_inner(dir, &mut result)?;
Ok(result)
}
#[cfg(not(feature = "private"))]
fn walk_dir_inner(dir: &Path, result: &mut Vec<std::path::PathBuf>) -> std::io::Result<()> {
if !dir.is_dir() {
return Ok(());
}
for entry in std::fs::read_dir(dir)? {
let entry = entry?;
let path = entry.path();
let meta = match std::fs::symlink_metadata(&path) {
Ok(m) => m,
Err(_) => continue,
};
if meta.is_dir() {
walk_dir_inner(&path, result)?;
} else if meta.is_file() {
result.push(path);
}
}
Ok(())
}
#[cfg(not(feature = "private"))]
pub fn collect_symlinks(dir: &Path) -> std::collections::HashMap<String, String> {
let mut symlinks = std::collections::HashMap::new();
collect_symlinks_inner(dir, dir, &mut symlinks);
symlinks
}
#[cfg(not(feature = "private"))]
fn collect_symlinks_inner(
base: &Path,
dir: &Path,
symlinks: &mut std::collections::HashMap<String, String>,
) {
let entries = match std::fs::read_dir(dir) {
Ok(e) => e,
Err(_) => return,
};
for entry in entries {
let entry = match entry {
Ok(e) => e,
Err(_) => continue,
};
let path = entry.path();
let meta = match std::fs::symlink_metadata(&path) {
Ok(m) => m,
Err(_) => continue,
};
if meta.file_type().is_symlink() {
if let Ok(target) = std::fs::read_link(&path) {
let relative = path
.strip_prefix(base)
.unwrap_or(&path)
.to_string_lossy()
.replace('\\', "/");
symlinks.insert(relative, target.to_string_lossy().to_string());
}
} else if meta.is_dir() {
collect_symlinks_inner(base, &path, symlinks);
}
}
}
#[cfg(not(feature = "private"))]
pub fn restore_symlinks(_dir: &Path, _symlinks: &std::collections::HashMap<String, String>) {
// No-op in OSS build
}

View File

@@ -10,10 +10,10 @@ path = "src/lib.rs"
[features]
default = []
private = []
private = ["windmill-worker-volumes/private", "windmill-queue/private"]
mcp = ["dep:windmill-mcp"]
prometheus = ["dep:prometheus", "windmill-common/prometheus"]
enterprise = ["windmill-queue/enterprise", "windmill-git-sync/enterprise", "windmill-common/enterprise", "dep:pem", "dep:tokio-util", "dep:opentelemetry-proto", "dep:prost", "dep:hudsucker", "dep:rcgen", "dep:hyper-http-proxy", "dep:hyper-tls", "dep:hyper-util"]
enterprise = ["windmill-queue/enterprise", "windmill-git-sync/enterprise", "windmill-common/enterprise", "windmill-worker-volumes/enterprise", "dep:pem", "dep:tokio-util", "dep:opentelemetry-proto", "dep:prost", "dep:hudsucker", "dep:rcgen", "dep:hyper-http-proxy", "dep:hyper-tls", "dep:hyper-util"]
mssql = ["dep:tiberius"]
mssql-kerberos = ["mssql", "tiberius/integrated-auth-gssapi"] # Linux/Unix integrated auth
mssql-winauth = ["mssql", "tiberius/winauth"] # Windows integrated auth
@@ -47,6 +47,7 @@ windmill-audit.workspace = true # there isn't really a reason for audit-worth ac
windmill-common = { workspace = true, default-features = false }
windmill-types.workspace = true
windmill-object-store.workspace = true
windmill-worker-volumes.workspace = true
windmill-jseval.workspace = true
windmill-runtime-nativets = { workspace = true, optional = true }
windmill-mcp = { workspace = true, optional = true }

View File

@@ -14,6 +14,18 @@ clone_newnet: false
clone_newuser: {CLONE_NEWUSER}
clone_newcgroup: false
uidmap {
inside_id: "1000"
outside_id: ""
count: 1
}
gidmap {
inside_id: "1000"
outside_id: ""
count: 1
}
skip_setsid: true
keep_caps: false
keep_env: true

View File

@@ -22,7 +22,7 @@ use crate::{
handle_child::handle_child,
is_sandboxing_enabled, read_ee_registry, BUNFIG_INSTALL_SCOPES, BUN_BUNDLE_CACHE_DIR,
BUN_CACHE_DIR, BUN_NO_CACHE, BUN_PATH, DISABLE_NUSER, HOME_ENV, NODE_BIN_PATH, NODE_PATH,
NPMRC, NPM_CONFIG_REGISTRY, NPM_PATH, NSJAIL_PATH, PATH_ENV, PROXY_ENVS,
NPMRC, NPM_CONFIG_REGISTRY, NPM_PATH, NSJAIL_AVAILABLE, NSJAIL_PATH, PATH_ENV, PROXY_ENVS,
TRACING_PROXY_CA_CERT_PATH, TZ_ENV,
};
use windmill_common::{
@@ -990,6 +990,14 @@ pub async fn handle_bun_job(
) -> error::Result<Box<RawValue>> {
let mut annotation = windmill_common::worker::TypeScriptAnnotations::parse(inner_content);
if annotation.sandbox && NSJAIL_AVAILABLE.is_none() {
return Err(error::Error::ExecutionErr(
"Script has //sandbox annotation but nsjail is not available on this worker. \
Please ensure nsjail is installed or remove the //sandbox annotation."
.to_string(),
));
}
let (mut has_bundle_cache, cache_logs, local_path, remote_path) = if let (Some(lock), true) = (
maybe_lock.get_lock(),
!annotation.nobundling && !*DISABLE_BUNDLING && codebase.is_none(),
@@ -1028,6 +1036,12 @@ pub async fn handle_bun_job(
let _ = write_file(job_dir, "main.ts", inner_content)?;
} else if !annotation.native && codebase.is_none() {
let _ = write_file(job_dir, "package.json", r#"{ "type": "module" }"#)?;
} else if codebase.is_some() {
// Write a valid fallback package.json for codebase mode. Without this,
// nsjail creates an empty 0-byte file (from the mandatory: false mount)
// which Node.js fails to parse as JSON (ERR_INVALID_PACKAGE_CONFIG).
// If the codebase TAR includes a package.json, it will overwrite this.
let _ = write_file(job_dir, "package.json", "{}")?;
};
let common_bun_proc_envs: HashMap<String, String> =
@@ -1161,6 +1175,10 @@ pub async fn handle_bun_job(
init_logs = format!("\n{}{}", cache_logs, init_logs);
}
if annotation.sandbox {
init_logs.push_str("sandbox mode (nsjail)\n");
}
let write_wrapper_f = async {
if !has_bundle_cache && annotation.native {
return Ok(()) as error::Result<()>;
@@ -1485,7 +1503,7 @@ try {{
append_logs(&job.id, &job.workspace_id, init_logs, conn).await;
//do not cache local dependencies
let child = if is_sandboxing_enabled() {
let child = if is_sandboxing_enabled() || annotation.sandbox {
let _ = write_file(
job_dir,
"run.config.proto",

View File

@@ -886,7 +886,7 @@ pub async fn cached_result_path(
}
#[cfg(feature = "parquet")]
async fn get_workspace_s3_resource_path(
pub(crate) async fn get_workspace_s3_resource_path(
db: &DB,
client: &AuthedClient,
workspace_id: &str,
@@ -948,7 +948,11 @@ async fn get_workspace_s3_resource_path(
)
}
Some(LargeFileStorage::FilesystemStorage(fs)) => {
(StorageResourceType::Filesystem, fs.root_path.clone())
return Ok(Some(
windmill_object_store::ObjectStoreResource::Filesystem(
windmill_object_store::FilesystemSettings { root_path: fs.root_path.clone() },
),
));
}
None => {
return Ok(None);

View File

@@ -17,6 +17,7 @@ use crate::{
NPM_CONFIG_REGISTRY, PATH_ENV, TZ_ENV,
};
use windmill_common::client::AuthedClient;
use windmill_common::worker::TypeScriptAnnotations;
use tokio::{fs::File, io::AsyncReadExt, process::Command};
use windmill_common::{error::Result, scripts::ScriptLang, worker::write_file, BASE_URL};
@@ -231,8 +232,13 @@ pub async fn handle_deno_job(
occupancy_metrics: &mut OccupancyMetrics,
has_stream: &mut bool,
) -> error::Result<Box<RawValue>> {
let annotations = TypeScriptAnnotations::parse(inner_content);
// let mut start = Instant::now();
let logs1 = "\n\n--- DENO CODE EXECUTION ---\n".to_string();
let mut logs1 = "\n\n--- DENO CODE EXECUTION ---\n".to_string();
if annotations.sandbox {
logs1.push_str("sandbox mode (nsjail)\n");
}
append_logs(&job.id, &job.workspace_id, logs1, conn).await;
let main_override = job.script_entrypoint_override.as_deref();
@@ -451,7 +457,7 @@ try {{
for flag in deno_flags {
args.push(flag);
}
} else if is_sandboxing_enabled() {
} else if is_sandboxing_enabled() || annotations.sandbox {
args.push("--allow-net");
args.push("--allow-sys");
args.push(allow_read.as_str());

View File

@@ -70,6 +70,9 @@ mod sanitized_sql_params;
mod schema;
pub mod sql_utils;
mod universal_pkg_installer;
#[cfg(feature = "private")]
mod volume_ee;
mod volume_oss;
mod worker;
mod worker_flow;
mod worker_lockfiles;

View File

@@ -134,8 +134,8 @@ use crate::{
handle_child::handle_child,
is_sandboxing_enabled, read_ee_registry,
worker_utils::ping_job_status,
PyV, DISABLE_NUSER, HOME_ENV, NSJAIL_PATH, PATH_ENV, PIP_EXTRA_INDEX_URL, PIP_INDEX_URL,
PROXY_ENVS, PY_INSTALL_DIR, TRACING_PROXY_CA_CERT_PATH, TZ_ENV, UV_CACHE_DIR,
PyV, DISABLE_NUSER, HOME_ENV, NSJAIL_AVAILABLE, NSJAIL_PATH, PATH_ENV, PIP_EXTRA_INDEX_URL,
PIP_INDEX_URL, PROXY_ENVS, PY_INSTALL_DIR, TRACING_PROXY_CA_CERT_PATH, TZ_ENV, UV_CACHE_DIR,
UV_INDEX_STRATEGY,
};
use windmill_common::client::AuthedClient;
@@ -567,6 +567,14 @@ pub async fn handle_python_job(
let annotations = PythonAnnotations::parse(inner_content);
if annotations.sandbox && NSJAIL_AVAILABLE.is_none() {
return Err(Error::ExecutionErr(
"Script has #sandbox annotation but nsjail is not available on this worker. \
Please ensure nsjail is installed or remove the #sandbox annotation."
.to_string(),
));
}
let (py_version, mut additional_python_paths) = handle_python_deps(
job_dir,
requirements_o,
@@ -605,16 +613,14 @@ pub async fn handle_python_job(
}
{
append_logs(
&job.id,
&job.workspace_id,
format!(
"\n\n--- PYTHON ({}) CODE EXECUTION ---\n",
py_version.clone().to_string()
),
conn,
)
.await;
let mut logs = format!(
"\n\n--- PYTHON ({}) CODE EXECUTION ---\n",
py_version.clone().to_string()
);
if annotations.sandbox {
logs.push_str("sandbox mode (nsjail)\n");
}
append_logs(&job.id, &job.workspace_id, logs, conn).await;
}
let (
import_loader,
@@ -784,7 +790,7 @@ except BaseException as e:
#[cfg(windows)]
let additional_python_paths_folders = additional_python_paths_folders.replace(":", ";");
if is_sandboxing_enabled() {
if is_sandboxing_enabled() || annotations.sandbox {
let shared_deps = additional_python_paths
.into_iter()
.map(|pp| {
@@ -828,7 +834,7 @@ mount {{
job.id
);
let child = if is_sandboxing_enabled() {
let child = if is_sandboxing_enabled() || annotations.sandbox {
let mut nsjail_cmd = Command::new(NSJAIL_PATH.as_str());
nsjail_cmd
.current_dir(job_dir)

View File

@@ -0,0 +1,112 @@
#[cfg(feature = "private")]
pub(crate) use crate::volume_ee::*;
#[cfg(not(feature = "private"))]
#[cfg(feature = "parquet")]
pub(crate) struct LeaseRenewalGuard(pub Option<tokio::task::JoinHandle<()>>);
#[cfg(not(feature = "private"))]
#[cfg(feature = "parquet")]
impl Drop for LeaseRenewalGuard {
fn drop(&mut self) {
if let Some(handle) = self.0.take() {
handle.abort();
}
}
}
#[cfg(not(feature = "private"))]
#[cfg(feature = "parquet")]
pub(crate) struct VolumeSetupResult {
pub states: Vec<windmill_worker_volumes::VolumeState>,
pub writable: Vec<bool>,
pub client: Option<std::sync::Arc<dyn windmill_worker_volumes::DynObjectStore>>,
pub lease_renewal: LeaseRenewalGuard,
}
#[cfg(not(feature = "private"))]
#[cfg(feature = "parquet")]
#[allow(dead_code)]
pub(crate) fn setup_volume_mount_paths(
_volume: &windmill_worker_volumes::VolumeMount,
_state: &windmill_worker_volumes::VolumeState,
_job_dir: &str,
_language: windmill_common::scripts::ScriptLang,
_envs: &mut std::collections::HashMap<String, String>,
_shared_mount: &mut String,
) -> windmill_common::error::Result<()> {
Err(windmill_common::error::Error::internal_err(
"Volumes are not available in OSS".to_string(),
))
}
#[cfg(not(feature = "private"))]
#[cfg(feature = "parquet")]
pub(crate) async fn setup_volumes_sql_worker(
_volume_mounts: &[windmill_worker_volumes::VolumeMount],
_db: &windmill_common::DB,
_workspace_id: &str,
_job_id: uuid::Uuid,
_permissioned_as: &str,
_worker_name: &str,
_job_dir: &str,
_client: &windmill_common::client::AuthedClient,
_conn: &windmill_common::worker::Connection,
_language: windmill_common::scripts::ScriptLang,
_envs: &mut std::collections::HashMap<String, String>,
_shared_mount: &mut String,
) -> windmill_common::error::Result<VolumeSetupResult> {
Err(windmill_common::error::Error::internal_err(
"Volumes are not available in OSS".to_string(),
))
}
#[cfg(not(feature = "private"))]
#[cfg(feature = "parquet")]
pub(crate) async fn setup_volumes_http_worker(
_volume_mounts: &[windmill_worker_volumes::VolumeMount],
_http: &windmill_common::worker::HttpClient,
_workspace_id: &str,
_job_id: uuid::Uuid,
_permissioned_as: &str,
_canceled_by: &Option<String>,
_worker_name: &str,
_job_dir: &str,
_conn: &windmill_common::worker::Connection,
_language: windmill_common::scripts::ScriptLang,
_envs: &mut std::collections::HashMap<String, String>,
_shared_mount: &mut String,
) -> windmill_common::error::Result<VolumeSetupResult> {
Err(windmill_common::error::Error::internal_err(
"Volumes are not available in OSS".to_string(),
))
}
#[cfg(not(feature = "private"))]
#[cfg(feature = "parquet")]
pub(crate) async fn sync_volumes_sql_worker(
_volume_states: &[windmill_worker_volumes::VolumeState],
_volume_writable: &[bool],
_vol_client: &std::sync::Arc<dyn windmill_worker_volumes::DynObjectStore>,
_db: &windmill_common::DB,
_workspace_id: &str,
_job_id: uuid::Uuid,
_worker_name: &str,
_conn: &windmill_common::worker::Connection,
_job_succeeded: bool,
) {
}
#[cfg(not(feature = "private"))]
#[cfg(feature = "parquet")]
pub(crate) async fn sync_volumes_http_worker(
_volume_states: &[windmill_worker_volumes::VolumeState],
_volume_writable: &[bool],
_http: &windmill_common::worker::HttpClient,
_workspace_id: &str,
_job_id: uuid::Uuid,
_worker_name: &str,
_conn: &windmill_common::worker::Connection,
_job_succeeded: bool,
) {
}

View File

@@ -4161,7 +4161,8 @@ pub async fn run_language_executor(
job.id
);
let shared_mount = if job.same_worker && job.script_lang != Some(ScriptLang::Deno) {
#[allow(unused_mut)]
let mut shared_mount = if job.same_worker && job.script_lang != Some(ScriptLang::Deno) {
let folder = if job.script_lang == Some(ScriptLang::Go) {
"/go"
} else {
@@ -4183,7 +4184,8 @@ mount {{
// println!("handle lang job {:?}", SystemTime::now());
let envs = build_envs(envs.as_ref())?;
#[allow(unused_mut)]
let mut envs = build_envs(envs.as_ref())?;
let Some(language) = language else {
return Err(Error::ExecutionErr(
@@ -4219,6 +4221,106 @@ mount {{
}
}
// Volume mount setup (requires workspace S3 storage; CE has file count/size limits)
#[cfg(feature = "parquet")]
let volume_mounts = {
let comment_prefix = match language {
ScriptLang::Python3
| ScriptLang::Bash
| ScriptLang::Powershell
| ScriptLang::Ansible
| ScriptLang::Ruby => "#",
ScriptLang::Deno
| ScriptLang::Bun
| ScriptLang::Bunnative
| ScriptLang::Nativets
| ScriptLang::Go => "//",
_ => "",
};
let raw_mounts = windmill_worker_volumes::parse_volume_annotations(&code, comment_prefix);
let args_ref = job.args.as_ref().map(|a| &**a);
let mut interpolated = Vec::new();
for mut v in raw_mounts {
v.name = windmill_worker_volumes::interpolate_volume_name(
&v.name,
args_ref,
&job.workspace_id,
);
if let Err(e) = windmill_worker_volumes::validate_volume_name(&v.name) {
return Err(Error::ExecutionErr(e));
}
if let Err(e) = windmill_worker_volumes::validate_volume_target(&v.target) {
return Err(Error::ExecutionErr(e));
}
interpolated.push(v);
}
if let Err(e) = windmill_worker_volumes::validate_volume_mounts(&interpolated) {
return Err(Error::ExecutionErr(e));
}
interpolated
};
#[cfg(feature = "parquet")]
let mut volume_setup = crate::volume_oss::VolumeSetupResult {
states: Vec::new(),
writable: Vec::new(),
client: None,
lease_renewal: crate::volume_oss::LeaseRenewalGuard(None),
};
#[cfg(feature = "parquet")]
if !volume_mounts.is_empty() {
let vol_summary: Vec<String> = volume_mounts
.iter()
.map(|v| format!("'{}' -> {}", v.name, v.target))
.collect();
append_logs(
&job.id,
&job.workspace_id,
format!(
"\n--- VOLUME MOUNTS ---\nPulling {} volume(s): {}\n",
volume_mounts.len(),
vol_summary.join(", "),
),
conn,
)
.await;
if let Connection::Sql(db) = conn {
volume_setup = crate::volume_oss::setup_volumes_sql_worker(
&volume_mounts,
db,
&job.workspace_id,
job.id,
&job.permissioned_as,
worker_name,
job_dir,
client,
conn,
language,
&mut envs,
&mut shared_mount,
)
.await?;
} else if let Connection::Http(http) = conn {
volume_setup = crate::volume_oss::setup_volumes_http_worker(
&volume_mounts,
http,
&job.workspace_id,
job.id,
&job.permissioned_as,
&job.canceled_by,
worker_name,
job_dir,
conn,
language,
&mut envs,
&mut shared_mount,
)
.await?;
}
}
// Box::pin all language handlers to prevent large match enum on stack
let result: error::Result<Box<RawValue>> = match language {
ScriptLang::Python3 => {
@@ -4630,6 +4732,62 @@ mount {{
// for related places search: ADD_NEW_LANG
_ => panic!("unreachable, language is not supported: {language:#?}"),
};
// Volume sync-back and lease release
#[cfg(feature = "parquet")]
if !volume_setup.states.is_empty() {
// Stop lease renewal before sync-back
volume_setup.lease_renewal.0.take().map(|h| h.abort());
if let Some(ref vol_client) = volume_setup.client {
if let Connection::Sql(db) = conn {
crate::volume_oss::sync_volumes_sql_worker(
&volume_setup.states,
&volume_setup.writable,
vol_client,
db,
&job.workspace_id,
job.id,
worker_name,
conn,
result.is_ok(),
)
.await;
}
}
if let Connection::Http(http) = conn {
crate::volume_oss::sync_volumes_http_worker(
&volume_setup.states,
&volume_setup.writable,
http,
&job.workspace_id,
job.id,
worker_name,
conn,
result.is_ok(),
)
.await;
}
// Clean up absolute-path symlinks created by setup_volume_mount_paths
if !is_sandboxing_enabled() {
#[allow(unused_variables)] // state is only used on unix
for state in &volume_setup.states {
#[cfg(unix)]
if state.mount.target.starts_with('/') {
let target_path = std::path::Path::new(&state.mount.target);
if target_path
.symlink_metadata()
.map(|m| m.file_type().is_symlink())
.unwrap_or(false)
{
std::fs::remove_file(target_path).ok();
}
}
}
}
}
tracing::info!(
workspace_id = %job.workspace_id,
is_ok = result.is_ok(),

View File

@@ -2,7 +2,7 @@ import { sleep } from "https://deno.land/x/sleep@v1.2.1/mod.ts";
import * as windmill from "https://deno.land/x/windmill@v1.174.0/mod.ts";
import * as api from "https://deno.land/x/windmill@v1.174.0/windmill-api/index.ts";
export const VERSION = "v1.650.0";
export const VERSION = "v1.651.1";
export async function login(email: string, password: string): Promise<string> {
return await windmill.UserService.login({

View File

@@ -39,7 +39,10 @@ export const scriptBootstrapCode = {
}
`,
bun: `export async function main() {
bun: `// there are multiple modes to add as header: //nobundling //native //npm //nodejs
// https://www.windmill.dev/docs/getting_started/scripts_quickstart/typescript#modes
export async function main() {
return "Hello world";
}
`,

View File

@@ -2626,6 +2626,7 @@ export async function push(
let [_basePath, changes] = queue.shift()!;
const promise = (async () => {
const alreadySynced: string[] = [];
const deletedVarsResPaths: string[] = [];
const isRawApp = isRawAppFile(changes[0].path);
if (isRawApp) {
const deleteRawApp = changes.find(
@@ -2870,12 +2871,23 @@ export async function push(
name: change.path.split(SEP)[1],
});
break;
case "resource":
await wmill.deleteResource({
workspace: workspaceId,
path: removeSuffix(target, ".resource.json"),
});
case "resource": {
const resourcePath = removeSuffix(target, ".resource.json");
try {
await wmill.deleteResource({
workspace: workspaceId,
path: resourcePath,
});
} catch (e: any) {
if (e?.status === 404 && deletedVarsResPaths.includes(resourcePath)) {
log.debug(`Resource ${resourcePath} already deleted by linked variable`);
} else {
throw e;
}
}
deletedVarsResPaths.push(resourcePath);
break;
}
case "resource-type":
await wmill.deleteResourceType({
workspace: workspaceId,
@@ -3012,12 +3024,23 @@ export async function push(
});
break;
}
case "variable":
await wmill.deleteVariable({
workspace: workspaceId,
path: removeSuffix(target, ".variable.json"),
});
case "variable": {
const variablePath = removeSuffix(target, ".variable.json");
try {
await wmill.deleteVariable({
workspace: workspaceId,
path: variablePath,
});
} catch (e: any) {
if (e?.status === 404 && deletedVarsResPaths.includes(variablePath)) {
log.debug(`Variable ${variablePath} already deleted by linked resource`);
} else {
throw e;
}
}
deletedVarsResPaths.push(variablePath);
break;
}
case "user": {
const users = await wmill.listUsers({
workspace: workspaceId,

View File

@@ -408,7 +408,8 @@ async function remove(_opts: GlobalOptions, name: string) {
async function whoami(_opts: GlobalOptions) {
await requireLogin(_opts);
log.info(await wmill.globalWhoami());
const whoamiInfo = await wmill.globalWhoami();
log.info(JSON.stringify(whoamiInfo, null, 2));
const activeName = await getActiveWorkspaceName(_opts);
log.info("Active: " + colors.green.bold(activeName || "none"));
}

View File

@@ -67,7 +67,7 @@ export {
workspaceAdd,
};
export const VERSION = "1.650.0";
export const VERSION = "1.651.1";
// Re-exported from constants.ts to maintain backwards compatibility
export { WM_FORK_PREFIX } from "./core/constants.ts";

View File

@@ -61,6 +61,11 @@ RUN ln -s /usr/bin/bun /usr/bin/node \
&& bun install -g windmill-cli \
&& ln -s $(bun pm bin -g)/wmill /usr/bin/wmill
# Install Claude Code CLI (used by claude sandbox scripts)
# Copy to /usr/bin/claude so it's accessible inside nsjail sandbox (which mounts /usr but not /root)
RUN curl -fsSL https://claude.ai/install.sh | bash \
&& cp /root/.local/share/claude/versions/* /usr/bin/claude
# add the docker client to call docker from a worker if enabled
COPY --from=docker:dind /usr/local/bin/docker /usr/local/bin/

View File

@@ -61,6 +61,11 @@ RUN ln -s /usr/bin/bun /usr/bin/node \
&& bun install -g windmill-cli \
&& ln -s $(bun pm bin -g)/wmill /usr/bin/wmill
# Install Claude Code CLI (used by claude sandbox scripts)
# Copy to /usr/bin/claude so it's accessible inside nsjail sandbox (which mounts /usr but not /root)
RUN curl -fsSL https://claude.ai/install.sh | bash \
&& cp /root/.local/share/claude/versions/* /usr/bin/claude
# add the docker client to call docker from a worker if enabled
COPY --from=docker:dind /usr/local/bin/docker /usr/local/bin/

39
frontend/BUGS.txt Normal file
View File

@@ -0,0 +1,39 @@
# Svelte 5 Migration - Bug Report
# Testing started: 2026-03-02
## Warnings (not blocking but worth fixing)
1. [WARNING] binding_property_non_reactive in Grid.svelte:372:5
- `bind:this={moveResizes[item.id]}` is binding to a non-reactive property
- File: src/lib/components/apps/svelte-grid/Grid.svelte
- Appears multiple times in App editor
- Status: NOT FIXED (non-blocking warning)
2. [WARNING] legacy_recursive_reactive_block in RecomputeAllComponents.svelte
- Migrated `$:` reactive block that both accesses and updates the same reactive value
- File: src/lib/components/apps/editor/RecomputeAllComponents.svelte
- May cause recursive updates when converted to $effect
- Status: NOT FIXED (non-blocking warning)
3. [WARNING] ownership_invalid_mutation in SchemaForm.svelte:70:16
- Mutating unbound props (`schema`) is strongly discouraged
- Parent: src/lib/components/ApiConnectForm.svelte should use `bind:schema={...}`
- Appears when opening PostgreSQL resource creation form
- Status: NOT FIXED (non-blocking warning)
4. [WARNING] ownership_invalid_binding in InputTransformSchemaForm.svelte
- Passes `schema` to InputTransformForm.svelte with `bind:`, but parent Pane.svelte didn't declare `schema` as binding
- Appears in flow editor when adding a TypeScript step
- Status: NOT FIXED (non-blocking warning)
## Bugs
1. [BUG] state_descriptors_fixed in Chart.svelte (Queue metrics drawer)
- Error: "Property descriptors defined on `$state` objects must contain `value` and always be `enumerable`, `configurable` and `writable`."
- Triggered by: Clicking "Queue metrics" on /workers page
- File: src/lib/components/chartjs-wrappers/Chart.svelte
- Root cause: Chart.js's `listenArrayEvents` calls Object.defineProperty on data arrays that are Svelte 5 $state proxies, which reject non-standard property descriptors
- Fix: Use $state.snapshot() to pass plain copies of data and options to Chart.js
- Status: FIXED

View File

@@ -1,12 +1,12 @@
{
"name": "windmill-components",
"version": "1.650.0",
"version": "1.651.1",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "windmill-components",
"version": "1.650.0",
"version": "1.651.1",
"hasInstallScript": true,
"license": "AGPL-3.0",
"dependencies": {

View File

@@ -1,6 +1,6 @@
{
"name": "windmill-components",
"version": "1.650.0",
"version": "1.651.1",
"scripts": {
"dev": "vite dev",
"build": "vite build",

Some files were not shown because too many files have changed in this diff Show More