Compare commits
51 Commits
v1.668.0
...
suspend-ne
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
65992dde37 | ||
|
|
3e1739905e | ||
|
|
cf30bcf3f9 | ||
|
|
b69d63eeb0 | ||
|
|
26050f96c3 | ||
|
|
c5fccd2f69 | ||
|
|
92b9ac72c5 | ||
|
|
fbb6170650 | ||
|
|
d6957aaf31 | ||
|
|
cd8edcd94f | ||
|
|
9d85768287 | ||
|
|
0520b11d5d | ||
|
|
c41e3adcc6 | ||
|
|
ef1757f5d7 | ||
|
|
bfc2aefdb8 | ||
|
|
3876902a7b | ||
|
|
8e973c892d | ||
|
|
2d27b17a05 | ||
|
|
96d4b3f123 | ||
|
|
4817913f0c | ||
|
|
d532c1d470 | ||
|
|
b5185b0e12 | ||
|
|
abc6b12d68 | ||
|
|
ee6231590e | ||
|
|
ee1afb2415 | ||
|
|
adfb0bd5ff | ||
|
|
98934d59c5 | ||
|
|
33032ed297 | ||
|
|
22da5bd9ea | ||
|
|
f3012ee7cc | ||
|
|
5f2d3e6812 | ||
|
|
9bcda7023f | ||
|
|
9f86c72133 | ||
|
|
754b88a52c | ||
|
|
06bbe7b94b | ||
|
|
970e859a41 | ||
|
|
845db72b73 | ||
|
|
f5fc9f8485 | ||
|
|
be7fbeb8b1 | ||
|
|
8c770a206a | ||
|
|
74fba2abf3 | ||
|
|
b4d1f2aac7 | ||
|
|
8baa7f8a20 | ||
|
|
0549f682fe | ||
|
|
73f649c152 | ||
|
|
c6ce3197a7 | ||
|
|
37799574d8 | ||
|
|
78ac28b4e0 | ||
|
|
f40cdaf434 | ||
|
|
0ea9b945e0 | ||
|
|
38acaa3653 |
@@ -1,3 +1,8 @@
|
||||
---
|
||||
name: native-trigger
|
||||
description: Guidance for adding native trigger services to Windmill. Use when implementing or modifying native trigger integrations across the backend and frontend.
|
||||
---
|
||||
|
||||
# Skill: Adding Native Trigger Services
|
||||
|
||||
This skill provides comprehensive guidance for adding new native trigger services to Windmill. Native triggers allow external services (like Nextcloud, Google Drive, etc.) to trigger Windmill scripts/flows via webhooks or push notifications.
|
||||
|
||||
25
.claude/review-prompt.md
Normal file
25
.claude/review-prompt.md
Normal file
@@ -0,0 +1,25 @@
|
||||
# Code Review Instructions
|
||||
|
||||
Review this pull request and provide comprehensive feedback.
|
||||
|
||||
## Focus Areas
|
||||
|
||||
- **Code quality and best practices** — does the code follow established patterns?
|
||||
- **Potential bugs or issues** — will this code work correctly in all cases?
|
||||
- **Performance considerations** — are there unnecessary allocations, N+1 queries, or bottlenecks?
|
||||
- **Security implications** — injection, auth bypass, data exposure?
|
||||
|
||||
## CLAUDE.md Compliance
|
||||
|
||||
Read all relevant CLAUDE.md files (root and in directories containing changed files). Check each rule against the changed code. Quote the exact rule when flagging a violation.
|
||||
|
||||
## Review Guidelines
|
||||
|
||||
- Provide detailed feedback using inline comments for specific issues
|
||||
- Use top-level comments for general observations or praise
|
||||
- Only flag issues introduced by this PR, not pre-existing problems
|
||||
- Self-validate each finding: "Is this definitely a real issue?" If uncertain, discard it
|
||||
|
||||
## Testing Instructions
|
||||
|
||||
At the end of your review, add complete instructions to reproduce the added changes through the app interface. These instructions will be given to a tester so they can verify the changes. It should be a short descriptive text (not a step-by-step or a list) on how to navigate the app (what page, what action, what input, etc.) to see the changes.
|
||||
@@ -6,53 +6,24 @@ description: Code review a pull request for bugs and CLAUDE.md compliance. MUST
|
||||
|
||||
# Local Code Review Skill
|
||||
|
||||
Review a pull request for real bugs and CLAUDE.md compliance violations. This review targets HIGH SIGNAL issues only.
|
||||
|
||||
## Review Philosophy
|
||||
|
||||
- **Only flag issues you are certain about.** If you are not sure an issue is real, do not flag it. False positives erode trust and waste reviewer time.
|
||||
- Think like a senior engineer doing a final review — flag things that would cause incidents, not things that are merely imperfect.
|
||||
|
||||
## What to Flag
|
||||
|
||||
- Code that won't compile or parse (syntax errors, type errors, missing imports)
|
||||
- Code that will definitely produce wrong results regardless of inputs
|
||||
- Clear, unambiguous CLAUDE.md violations (quote the exact rule being violated)
|
||||
- Security issues in introduced code (injection, auth bypass, data exposure)
|
||||
- Incorrect logic that will fail in production
|
||||
|
||||
## What NOT to Flag
|
||||
|
||||
- Code style or quality concerns
|
||||
- Potential issues that depend on specific inputs or runtime state
|
||||
- Subjective suggestions or improvements
|
||||
- Pre-existing issues not introduced by this PR
|
||||
- Pedantic nitpicks a senior engineer wouldn't flag
|
||||
- Issues a linter or type checker will catch
|
||||
- General quality concerns unless explicitly prohibited in CLAUDE.md
|
||||
- Issues silenced via lint ignore comments
|
||||
Run the same review locally that the GitHub Claude Auto Review action runs on PRs. The shared review instructions live in `.claude/review-prompt.md` — read that file first and follow its instructions.
|
||||
|
||||
## Execution Steps
|
||||
|
||||
1. **Determine the PR scope**:
|
||||
1. **Read `.claude/review-prompt.md`** for the review criteria and focus areas
|
||||
|
||||
2. **Determine the PR scope**:
|
||||
- If an argument is provided, use it as the PR number or branch
|
||||
- Otherwise, detect from the current branch vs main
|
||||
- Run `gh pr view` if a PR exists, or use `git diff main...HEAD`
|
||||
|
||||
2. **Find relevant CLAUDE.md files**:
|
||||
- Read the root `CLAUDE.md`
|
||||
- Check for CLAUDE.md files in directories containing changed files
|
||||
|
||||
3. **Get the diff and metadata**:
|
||||
- `gh pr diff` or `git diff main...HEAD` for the full diff
|
||||
- `gh pr view` or `git log main..HEAD --oneline` for context
|
||||
|
||||
4. **Read changed files** where the diff alone is insufficient to understand context
|
||||
|
||||
5. **Review for**:
|
||||
- CLAUDE.md compliance — check each rule against the changed code
|
||||
- Bugs and logic errors — will this code work correctly?
|
||||
- Security issues — injection, auth, data exposure in new code
|
||||
5. **Apply the review instructions from `.claude/review-prompt.md`**
|
||||
|
||||
6. **Self-validate each finding**: Before reporting, ask yourself:
|
||||
- "Is this definitely a real issue, not a false positive?"
|
||||
|
||||
@@ -1,3 +1,8 @@
|
||||
---
|
||||
name: native-trigger
|
||||
description: Guidance for adding native trigger services to Windmill. Use when implementing or modifying native trigger integrations across the backend and frontend.
|
||||
---
|
||||
|
||||
# Skill: Adding Native Trigger Services
|
||||
|
||||
This skill provides comprehensive guidance for adding new native trigger services to Windmill. Native triggers allow external services (like Nextcloud, Google Drive, etc.) to trigger Windmill scripts/flows via webhooks or push notifications.
|
||||
|
||||
@@ -61,12 +61,13 @@ Generated with [Claude Code](https://claude.com/claude-code)
|
||||
1. Run `git status` to check for uncommitted changes
|
||||
2. Run `git log main..HEAD --oneline` to see all commits in this branch
|
||||
3. Run `git diff main...HEAD` to see the full diff against main
|
||||
4. Check if remote branch exists and is up to date:
|
||||
4. **Run `/local-review`** before creating the PR. If issues are found, fix them and commit before proceeding. Do not skip this step.
|
||||
5. Check if remote branch exists and is up to date:
|
||||
```bash
|
||||
git rev-parse --abbrev-ref --symbolic-full-name @{u} 2>/dev/null || echo "no upstream"
|
||||
```
|
||||
5. Push to remote if needed: `git push -u origin HEAD`
|
||||
6. Create draft PR using gh CLI:
|
||||
6. Push to remote if needed: `git push -u origin HEAD`
|
||||
7. Create draft PR using gh CLI:
|
||||
```bash
|
||||
gh pr create --draft --title "<type>: <description>" --body "$(cat <<'EOF'
|
||||
## Summary
|
||||
@@ -85,7 +86,7 @@ Generated with [Claude Code](https://claude.com/claude-code)
|
||||
EOF
|
||||
)"
|
||||
```
|
||||
7. Return the PR URL to the user
|
||||
8. Return the PR URL to the user
|
||||
|
||||
## EE Companion PR (when `*_ee.rs` files were modified)
|
||||
|
||||
|
||||
22
.github/workflows/pr-ready-review.yml
vendored
22
.github/workflows/pr-ready-review.yml
vendored
@@ -22,6 +22,15 @@ jobs:
|
||||
with:
|
||||
fetch-depth: 1
|
||||
|
||||
- name: Read review prompt
|
||||
id: review-prompt
|
||||
run: |
|
||||
{
|
||||
echo 'REVIEW_PROMPT<<EOF'
|
||||
cat .claude/review-prompt.md
|
||||
echo 'EOF'
|
||||
} >> "$GITHUB_ENV"
|
||||
|
||||
- name: Automatic PR Review
|
||||
uses: anthropics/claude-code-action@v1
|
||||
with:
|
||||
@@ -31,18 +40,7 @@ jobs:
|
||||
REPO: ${{ github.repository }}
|
||||
PR NUMBER: ${{ github.event.pull_request.number }}
|
||||
|
||||
Please review this pull request and provide comprehensive feedback.
|
||||
|
||||
Focus on:
|
||||
- Code quality and best practices
|
||||
- Potential bugs or issues
|
||||
- Performance considerations
|
||||
- Security implications
|
||||
|
||||
Provide detailed feedback using inline comments for specific issues.
|
||||
Use top-level comments for general observations or praise.
|
||||
|
||||
At the end of your review, add complete instructions to reproduce the added changes through the app interface. These instructions will be given to a tester so he can verify the changes. It should be a short descriptive text (not a step by step or a list) on how to navigate the app (what page, what action, what input, etc) to see the changes.
|
||||
${{ env.REVIEW_PROMPT }}
|
||||
claude_args: |
|
||||
--allowedTools "mcp__github_inline_comment__create_inline_comment,Bash(gh pr comment:*),Bash(gh pr diff:*),Bash(gh pr view:*)"
|
||||
--model opus
|
||||
|
||||
68
CHANGELOG.md
68
CHANGELOG.md
@@ -1,5 +1,73 @@
|
||||
# Changelog
|
||||
|
||||
## [1.669.1](https://github.com/windmill-labs/windmill/compare/v1.669.0...v1.669.1) (2026-03-30)
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* avoid doubled /oauth2 path in Okta custom authorization server URLs ([#8620](https://github.com/windmill-labs/windmill/issues/8620)) ([4817913](https://github.com/windmill-labs/windmill/commit/4817913f0cab49980bfeb442089631d7953955ff))
|
||||
* improve db health UI text and prevent label wrapping ([d532c1d](https://github.com/windmill-labs/windmill/commit/d532c1d470fcb0ef02ebc5342ad1cf22e58b1f4d))
|
||||
|
||||
## [1.669.0](https://github.com/windmill-labs/windmill/compare/v1.668.5...v1.669.0) (2026-03-30)
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
* WAC workflow diagram visualization via WASM ([#8604](https://github.com/windmill-labs/windmill/issues/8604)) ([abc6b12](https://github.com/windmill-labs/windmill/commit/abc6b12d6815edc4dda3ddf5f0572ecedcb670dd))
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* add path traversal check in service_logs get_log_file endpoint ([#8605](https://github.com/windmill-labs/windmill/issues/8605)) ([5f2d3e6](https://github.com/windmill-labs/windmill/commit/5f2d3e6812f01fe6194bcfd976970a6e3c4186cc))
|
||||
* cast DuckDB IS_NULLABLE to string in metadata query ([#8607](https://github.com/windmill-labs/windmill/issues/8607)) ([f3012ee](https://github.com/windmill-labs/windmill/commit/f3012ee7ccc7a8947b5f6bd7c7df77984437f91e))
|
||||
* enable S3 bundle cache for PHP previews without lock file ([#8608](https://github.com/windmill-labs/windmill/issues/8608)) ([ee62315](https://github.com/windmill-labs/windmill/commit/ee6231590ed91063f104e6d054b52e88b569986f))
|
||||
* enforce workspace isolation on flow resume endpoint ([#8612](https://github.com/windmill-labs/windmill/issues/8612)) ([33032ed](https://github.com/windmill-labs/windmill/commit/33032ed297cf9ea867388d4ea2ece607c9d36dc7))
|
||||
* handle DuckDB boolean types in ColumnDef deserializers ([#8610](https://github.com/windmill-labs/windmill/issues/8610)) ([22da5bd](https://github.com/windmill-labs/windmill/commit/22da5bd9ea1ca000cfab3eecf1e3fb0fc01200cb))
|
||||
* use route_service instead of fallback_service for MCP router ([#8614](https://github.com/windmill-labs/windmill/issues/8614)) ([98934d5](https://github.com/windmill-labs/windmill/commit/98934d59c552325fcf88c016e31ae977970e8c9a))
|
||||
|
||||
## [1.668.5](https://github.com/windmill-labs/windmill/compare/v1.668.4...v1.668.5) (2026-03-29)
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* add per-IP and per-account brute force protection on login endpoint ([#8601](https://github.com/windmill-labs/windmill/issues/8601)) ([06bbe7b](https://github.com/windmill-labs/windmill/commit/06bbe7b94bfb846bd73aaf6abdc83e4c14e70adc))
|
||||
* add timestamp validation to webhook signature verification ([#8596](https://github.com/windmill-labs/windmill/issues/8596)) ([74fba2a](https://github.com/windmill-labs/windmill/commit/74fba2abf3dc68b682777c01da360258786fded8))
|
||||
* disable workspace webhook events when CLOUD_HOSTED ([#8598](https://github.com/windmill-labs/windmill/issues/8598)) ([be7fbeb](https://github.com/windmill-labs/windmill/commit/be7fbeb8b1f31d15e33b0783b2a504d6a01e532e))
|
||||
* harden login rate limiting with CLOUD_HOSTED gating and memory eviction ([#8602](https://github.com/windmill-labs/windmill/issues/8602)) ([754b88a](https://github.com/windmill-labs/windmill/commit/754b88a52c4e76421cb21c1eed87ad9d8385e9aa))
|
||||
* prevent SSRF and local file read via git repository resource URLs ([#8600](https://github.com/windmill-labs/windmill/issues/8600)) ([845db72](https://github.com/windmill-labs/windmill/commit/845db72b7344fb87ac9c5e24697750549665c7bf))
|
||||
* rename snippet param to avoid svelte compiler shadowing bug in asset usages drawer ([#8595](https://github.com/windmill-labs/windmill/issues/8595)) ([8c770a2](https://github.com/windmill-labs/windmill/commit/8c770a206a3b0704642c0bda2ab2aeb199d8af3f))
|
||||
* require mcp: scope for MCP endpoints instead of blanket bypass ([#8597](https://github.com/windmill-labs/windmill/issues/8597)) ([f5fc9f8](https://github.com/windmill-labs/windmill/commit/f5fc9f8485d2ec3e20f8b451305195446b90e5a3))
|
||||
* use constant-time comparison for API key and basic auth validation ([#8593](https://github.com/windmill-labs/windmill/issues/8593)) ([b4d1f2a](https://github.com/windmill-labs/windmill/commit/b4d1f2aac789306c2e35e123ac93e12c47c26f99))
|
||||
* validate JSON before sql_builder bind to prevent injection via JSONB queries ([#8599](https://github.com/windmill-labs/windmill/issues/8599)) ([970e859](https://github.com/windmill-labs/windmill/commit/970e859a410b0144847a1a30d7059955effdd402))
|
||||
|
||||
## [1.668.4](https://github.com/windmill-labs/windmill/compare/v1.668.3...v1.668.4) (2026-03-29)
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* update git sync version to latest cli ([0549f68](https://github.com/windmill-labs/windmill/commit/0549f682fe14f4d4b2f67941362ed2cc29d974a1))
|
||||
|
||||
## [1.668.3](https://github.com/windmill-labs/windmill/compare/v1.668.2...v1.668.3) (2026-03-28)
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* **cli:** phantom diffs, flow safety, trigger DX, lint watch, error clarity ([#8588](https://github.com/windmill-labs/windmill/issues/8588)) ([c6ce319](https://github.com/windmill-labs/windmill/commit/c6ce3197a72ceeffd702cf2263b1074ecbf1ca33))
|
||||
|
||||
## [1.668.2](https://github.com/windmill-labs/windmill/compare/v1.668.1...v1.668.2) (2026-03-28)
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* **cli:** app push crash, lint path, push --message, run validation, history timestamps ([#8585](https://github.com/windmill-labs/windmill/issues/8585)) ([f40cdaf](https://github.com/windmill-labs/windmill/commit/f40cdaf43453d2643800ed730d6abe6873bbe8e7))
|
||||
|
||||
## [1.668.1](https://github.com/windmill-labs/windmill/compare/v1.668.0...v1.668.1) (2026-03-28)
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* **cli:** fix 13 CLI bugs — exit codes, sync tar fallback, variable encryption, JSON output ([#8582](https://github.com/windmill-labs/windmill/issues/8582)) ([38acaa3](https://github.com/windmill-labs/windmill/commit/38acaa3653728bf9e0ae6f746edf433703b4ab63))
|
||||
|
||||
## [1.668.0](https://github.com/windmill-labs/windmill/compare/v1.667.0...v1.668.0) (2026-03-28)
|
||||
|
||||
|
||||
|
||||
12
backend/.sqlx/query-077467cd813d5af161cb1cc232724f26984822d4c28ba36c0a9331273b10edc0.json
generated
Normal file
12
backend/.sqlx/query-077467cd813d5af161cb1cc232724f26984822d4c28ba36c0a9331273b10edc0.json
generated
Normal file
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "INSERT INTO email_to_igroup (email, igroup) VALUES ('alice@example.com', 'admins') ON CONFLICT DO NOTHING",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": []
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "077467cd813d5af161cb1cc232724f26984822d4c28ba36c0a9331273b10edc0"
|
||||
}
|
||||
@@ -1,11 +1,11 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT COUNT(*) as cnt FROM pg_stat_activity WHERE state = 'active'",
|
||||
"query": "SELECT setting::bigint as \"max!\" FROM pg_settings WHERE name = 'max_connections'",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "cnt",
|
||||
"name": "max!",
|
||||
"type_info": "Int8"
|
||||
}
|
||||
],
|
||||
@@ -16,5 +16,5 @@
|
||||
null
|
||||
]
|
||||
},
|
||||
"hash": "87d07998fe8373f5b89eccf6f0528c02e389bf827d935d867430ad3459104dd9"
|
||||
"hash": "07770a002a49428c4f956cfc7262d6b6792ae5b97ed90b0ee07d17480b2dffe2"
|
||||
}
|
||||
20
backend/.sqlx/query-1721f8b52ea265c0537fd7c742deddf0afbe5cf0d81b15e487c411ae169d3a89.json
generated
Normal file
20
backend/.sqlx/query-1721f8b52ea265c0537fd7c742deddf0afbe5cf0d81b15e487c411ae169d3a89.json
generated
Normal file
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT igroup FROM email_to_igroup WHERE email = 'alice@example.com'",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "igroup",
|
||||
"type_info": "Varchar"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": []
|
||||
},
|
||||
"nullable": [
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "1721f8b52ea265c0537fd7c742deddf0afbe5cf0d81b15e487c411ae169d3a89"
|
||||
}
|
||||
15
backend/.sqlx/query-250a4e3f1a1f95296f7075bf8780e9c7407e89c8f7636484895e99f5a5e71297.json
generated
Normal file
15
backend/.sqlx/query-250a4e3f1a1f95296f7075bf8780e9c7407e89c8f7636484895e99f5a5e71297.json
generated
Normal file
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "INSERT INTO usr (workspace_id, username, email, is_admin, operator, added_via)\n VALUES ($1, 'alice', 'alice@example.com', false, true, $2)",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Varchar",
|
||||
"Jsonb"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "250a4e3f1a1f95296f7075bf8780e9c7407e89c8f7636484895e99f5a5e71297"
|
||||
}
|
||||
15
backend/.sqlx/query-2ba03e555d2e09dbd0e2ae5ddfd9a268a675bdb23615c78904cebe7f1e31f400.json
generated
Normal file
15
backend/.sqlx/query-2ba03e555d2e09dbd0e2ae5ddfd9a268a675bdb23615c78904cebe7f1e31f400.json
generated
Normal file
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "INSERT INTO usr (workspace_id, username, email, is_admin, operator, added_via)\n VALUES ($1, 'alice', 'alice@example.com', true, false, $2)",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Varchar",
|
||||
"Jsonb"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "2ba03e555d2e09dbd0e2ae5ddfd9a268a675bdb23615c78904cebe7f1e31f400"
|
||||
}
|
||||
32
backend/.sqlx/query-2d95191e899d60385b32f36f2e38137e4173a34c54344ee522745640d48b8813.json
generated
Normal file
32
backend/.sqlx/query-2d95191e899d60385b32f36f2e38137e4173a34c54344ee522745640d48b8813.json
generated
Normal file
@@ -0,0 +1,32 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT\n COUNT(*) as \"total!\",\n COUNT(*) FILTER (WHERE state = 'active') as \"active!\",\n COUNT(*) FILTER (WHERE state = 'idle') as \"idle!\"\n FROM pg_stat_activity\n WHERE backend_type = 'client backend'",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "total!",
|
||||
"type_info": "Int8"
|
||||
},
|
||||
{
|
||||
"ordinal": 1,
|
||||
"name": "active!",
|
||||
"type_info": "Int8"
|
||||
},
|
||||
{
|
||||
"ordinal": 2,
|
||||
"name": "idle!",
|
||||
"type_info": "Int8"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": []
|
||||
},
|
||||
"nullable": [
|
||||
null,
|
||||
null,
|
||||
null
|
||||
]
|
||||
},
|
||||
"hash": "2d95191e899d60385b32f36f2e38137e4173a34c54344ee522745640d48b8813"
|
||||
}
|
||||
44
backend/.sqlx/query-30930bfb0513f1a70194a900011b2e890bc4146bb0419210cd76743cacda8bfa.json
generated
Normal file
44
backend/.sqlx/query-30930bfb0513f1a70194a900011b2e890bc4146bb0419210cd76743cacda8bfa.json
generated
Normal file
@@ -0,0 +1,44 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT\n table_name as \"table_name!\",\n SUM(live_tuples)::bigint as \"live_tuples!\",\n SUM(dead_tuples)::bigint as \"dead_tuples!\",\n MAX(last_autovacuum) as \"last_autovacuum\",\n MAX(last_autoanalyze) as \"last_autoanalyze\"\n FROM (\n SELECT\n CASE\n WHEN i.inhparent IS NOT NULL THEN schemaname || '.' || p.relname\n ELSE schemaname || '.' || s.relname\n END as table_name,\n COALESCE(n_live_tup, 0) as live_tuples,\n COALESCE(n_dead_tup, 0) as dead_tuples,\n last_autovacuum,\n last_autoanalyze\n FROM pg_stat_user_tables s\n LEFT JOIN pg_class c ON c.relname = s.relname AND c.relnamespace = (\n SELECT oid FROM pg_namespace WHERE nspname = s.schemaname\n )\n LEFT JOIN pg_inherits i ON i.inhrelid = c.oid\n LEFT JOIN pg_class p ON p.oid = i.inhparent\n ) sub\n GROUP BY table_name\n ORDER BY SUM(dead_tuples) DESC",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "table_name!",
|
||||
"type_info": "Text"
|
||||
},
|
||||
{
|
||||
"ordinal": 1,
|
||||
"name": "live_tuples!",
|
||||
"type_info": "Int8"
|
||||
},
|
||||
{
|
||||
"ordinal": 2,
|
||||
"name": "dead_tuples!",
|
||||
"type_info": "Int8"
|
||||
},
|
||||
{
|
||||
"ordinal": 3,
|
||||
"name": "last_autovacuum",
|
||||
"type_info": "Timestamptz"
|
||||
},
|
||||
{
|
||||
"ordinal": 4,
|
||||
"name": "last_autoanalyze",
|
||||
"type_info": "Timestamptz"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": []
|
||||
},
|
||||
"nullable": [
|
||||
null,
|
||||
null,
|
||||
null,
|
||||
null,
|
||||
null
|
||||
]
|
||||
},
|
||||
"hash": "30930bfb0513f1a70194a900011b2e890bc4146bb0419210cd76743cacda8bfa"
|
||||
}
|
||||
38
backend/.sqlx/query-359cd29f531d263a8cf7205e0869229a610767087f01e0154be8da0620fa114b.json
generated
Normal file
38
backend/.sqlx/query-359cd29f531d263a8cf7205e0869229a610767087f01e0154be8da0620fa114b.json
generated
Normal file
@@ -0,0 +1,38 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "WITH all_audit AS (SELECT username, operation, timestamp FROM audit_partitioned UNION ALL SELECT username, operation, timestamp FROM audit),\n active_users as (SELECT distinct username as email FROM all_audit WHERE timestamp > NOW() - INTERVAL '1 month' AND (operation = 'users.login' OR operation = 'oauth.login' OR operation = 'users.token.refresh')),\n active_authors as (SELECT distinct email FROM usr WHERE usr.operator IS false AND email IN (SELECT email FROM active_users)),\n active_authors_agg as (SELECT array_agg(email) as authors FROM active_authors),\n active_ops_agg as (SELECT array_agg(email) as operators from active_users WHERE email NOT IN (SELECT email FROM active_authors))\n SELECT active_authors_agg.authors, active_ops_agg.operators, array_length(active_authors_agg.authors, 1) as author_count, array_length(active_ops_agg.operators, 1) as operator_count FROM active_authors_agg, active_ops_agg",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "authors",
|
||||
"type_info": "VarcharArray"
|
||||
},
|
||||
{
|
||||
"ordinal": 1,
|
||||
"name": "operators",
|
||||
"type_info": "VarcharArray"
|
||||
},
|
||||
{
|
||||
"ordinal": 2,
|
||||
"name": "author_count",
|
||||
"type_info": "Int4"
|
||||
},
|
||||
{
|
||||
"ordinal": 3,
|
||||
"name": "operator_count",
|
||||
"type_info": "Int4"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": []
|
||||
},
|
||||
"nullable": [
|
||||
null,
|
||||
null,
|
||||
null,
|
||||
null
|
||||
]
|
||||
},
|
||||
"hash": "359cd29f531d263a8cf7205e0869229a610767087f01e0154be8da0620fa114b"
|
||||
}
|
||||
34
backend/.sqlx/query-3bd4f38a1629a69ddda622b6b436198b47c2fe1a507358d49f05031e7beedab6.json
generated
Normal file
34
backend/.sqlx/query-3bd4f38a1629a69ddda622b6b436198b47c2fe1a507358d49f05031e7beedab6.json
generated
Normal file
@@ -0,0 +1,34 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n SELECT workspace_id,\n auto_invite->'instance_groups_roles' as instance_groups_roles,\n auto_invite->'instance_groups' as instance_groups_json\n FROM workspace_settings\n WHERE auto_invite->'instance_groups' ? $1\n ",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "workspace_id",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 1,
|
||||
"name": "instance_groups_roles",
|
||||
"type_info": "Jsonb"
|
||||
},
|
||||
{
|
||||
"ordinal": 2,
|
||||
"name": "instance_groups_json",
|
||||
"type_info": "Jsonb"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
false,
|
||||
null,
|
||||
null
|
||||
]
|
||||
},
|
||||
"hash": "3bd4f38a1629a69ddda622b6b436198b47c2fe1a507358d49f05031e7beedab6"
|
||||
}
|
||||
22
backend/.sqlx/query-5d160ba4958583f1ad42de846c544d8d8e81e1b54925a0c5f2cedc1817d99a1b.json
generated
Normal file
22
backend/.sqlx/query-5d160ba4958583f1ad42de846c544d8d8e81e1b54925a0c5f2cedc1817d99a1b.json
generated
Normal file
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT igroup FROM email_to_igroup WHERE email = $1",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "igroup",
|
||||
"type_info": "Varchar"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "5d160ba4958583f1ad42de846c544d8d8e81e1b54925a0c5f2cedc1817d99a1b"
|
||||
}
|
||||
32
backend/.sqlx/query-62e8e443cf063fcb30799d9c8971c00d761d54811936deb87a0315ca9cdc9769.json
generated
Normal file
32
backend/.sqlx/query-62e8e443cf063fcb30799d9c8971c00d761d54811936deb87a0315ca9cdc9769.json
generated
Normal file
@@ -0,0 +1,32 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT is_admin, operator, added_via FROM usr WHERE workspace_id = 'ws-multi-group' AND email = 'alice@example.com'",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "is_admin",
|
||||
"type_info": "Bool"
|
||||
},
|
||||
{
|
||||
"ordinal": 1,
|
||||
"name": "operator",
|
||||
"type_info": "Bool"
|
||||
},
|
||||
{
|
||||
"ordinal": 2,
|
||||
"name": "added_via",
|
||||
"type_info": "Jsonb"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": []
|
||||
},
|
||||
"nullable": [
|
||||
false,
|
||||
false,
|
||||
true
|
||||
]
|
||||
},
|
||||
"hash": "62e8e443cf063fcb30799d9c8971c00d761d54811936deb87a0315ca9cdc9769"
|
||||
}
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n SELECT\n workspace_id,\n auto_invite->'instance_groups_roles' as instance_groups_roles\n FROM workspace_settings\n WHERE\n auto_invite->'instance_groups' IS NOT NULL\n AND auto_invite->'instance_groups' ? $1\n ",
|
||||
"query": "\n SELECT\n workspace_id,\n auto_invite->'instance_groups_roles' as instance_groups_roles,\n auto_invite->'instance_groups' as instance_groups_json\n FROM workspace_settings\n WHERE\n auto_invite->'instance_groups' IS NOT NULL\n AND auto_invite->'instance_groups' ? $1\n ",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
@@ -12,6 +12,11 @@
|
||||
"ordinal": 1,
|
||||
"name": "instance_groups_roles",
|
||||
"type_info": "Jsonb"
|
||||
},
|
||||
{
|
||||
"ordinal": 2,
|
||||
"name": "instance_groups_json",
|
||||
"type_info": "Jsonb"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
@@ -21,8 +26,9 @@
|
||||
},
|
||||
"nullable": [
|
||||
false,
|
||||
null,
|
||||
null
|
||||
]
|
||||
},
|
||||
"hash": "7e01ef5799168c0fc2779d42ce352827e2fda6711c0a1b104ca6435ddb14b47d"
|
||||
"hash": "66e2f8468ba64f22b7a7caa18639d7c833ac2ec573bd89d878b5c8b1afc74d3a"
|
||||
}
|
||||
30
backend/.sqlx/query-68c19cb0e18b94870bbe81f9aab92ba37da67cd2a56834c9d1378eab7551284d.json
generated
Normal file
30
backend/.sqlx/query-68c19cb0e18b94870bbe81f9aab92ba37da67cd2a56834c9d1378eab7551284d.json
generated
Normal file
@@ -0,0 +1,30 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n UPDATE kafka_trigger\n SET\n kafka_resource_path = $1,\n group_id = $2,\n topics = $3,\n filters = $4,\n filter_logic = $5,\n auto_offset_reset = $6,\n auto_commit = $7,\n script_path = $8,\n path = $9,\n is_flow = $10,\n edited_by = $11,\n permissioned_as = $12,\n edited_at = now(),\n server_id = NULL,\n error = NULL,\n error_handler_path = $15,\n error_handler_args = $16,\n retry = $17\n WHERE\n workspace_id = $13 AND path = $14\n ",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Varchar",
|
||||
"Varchar",
|
||||
"VarcharArray",
|
||||
"JsonbArray",
|
||||
"Varchar",
|
||||
"Varchar",
|
||||
"Bool",
|
||||
"Varchar",
|
||||
"Varchar",
|
||||
"Bool",
|
||||
"Varchar",
|
||||
"Varchar",
|
||||
"Text",
|
||||
"Text",
|
||||
"Varchar",
|
||||
"Jsonb",
|
||||
"Jsonb"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "68c19cb0e18b94870bbe81f9aab92ba37da67cd2a56834c9d1378eab7551284d"
|
||||
}
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n INSERT INTO websocket_trigger (\n workspace_id,\n path,\n url,\n script_path,\n is_flow,\n mode,\n filters,\n initial_messages,\n url_runnable_args,\n edited_by,\n can_return_message,\n can_return_error_result,\n permissioned_as,\n edited_at,\n error_handler_path,\n error_handler_args,\n retry\n ) VALUES (\n $1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, now(), $14, $15, $16\n )\n ",
|
||||
"query": "\n INSERT INTO websocket_trigger (\n workspace_id,\n path,\n url,\n script_path,\n is_flow,\n mode,\n filters,\n filter_logic,\n initial_messages,\n url_runnable_args,\n edited_by,\n can_return_message,\n can_return_error_result,\n permissioned_as,\n edited_at,\n error_handler_path,\n error_handler_args,\n retry\n ) VALUES (\n $1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, now(), $15, $16, $17\n )\n ",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
@@ -23,6 +23,7 @@
|
||||
}
|
||||
},
|
||||
"JsonbArray",
|
||||
"Varchar",
|
||||
"JsonbArray",
|
||||
"Jsonb",
|
||||
"Varchar",
|
||||
@@ -36,5 +37,5 @@
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "942c0abb55c910862fd45d3fa56a4eb6729f1a658101bda2d0b0fca96b3cfee5"
|
||||
"hash": "6948eb5aabf82f2f4a08dd4410eb472080ecab3ed652912397245e5216ae0389"
|
||||
}
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n INSERT INTO kafka_trigger (\n workspace_id,\n path,\n kafka_resource_path,\n group_id,\n topics,\n filters,\n auto_offset_reset,\n auto_commit,\n script_path,\n is_flow,\n mode,\n edited_by,\n permissioned_as,\n edited_at,\n error_handler_path,\n error_handler_args,\n retry\n ) VALUES (\n $1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, now(), $14, $15, $16\n )\n ",
|
||||
"query": "\n INSERT INTO kafka_trigger (\n workspace_id,\n path,\n kafka_resource_path,\n group_id,\n topics,\n filters,\n filter_logic,\n auto_offset_reset,\n auto_commit,\n script_path,\n is_flow,\n mode,\n edited_by,\n permissioned_as,\n edited_at,\n error_handler_path,\n error_handler_args,\n retry\n ) VALUES (\n $1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, now(), $15, $16, $17\n )\n ",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
@@ -12,6 +12,7 @@
|
||||
"VarcharArray",
|
||||
"JsonbArray",
|
||||
"Varchar",
|
||||
"Varchar",
|
||||
"Bool",
|
||||
"Varchar",
|
||||
"Bool",
|
||||
@@ -36,5 +37,5 @@
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "a0a545fda5f3ebea0113d5daaf13358c964d9fb0f41bf2a1c834305b4d2398f2"
|
||||
"hash": "6a8f4ed9946bb2a3c5e90695c90b70aa2e83fcb5aa0c953febdd9bac2d95bbec"
|
||||
}
|
||||
12
backend/.sqlx/query-6f941e4454f736b32eaef80cdfb9582d6e75af3dc159e7c5f12497d3957f1eef.json
generated
Normal file
12
backend/.sqlx/query-6f941e4454f736b32eaef80cdfb9582d6e75af3dc159e7c5f12497d3957f1eef.json
generated
Normal file
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "INSERT INTO usr (workspace_id, username, email, is_admin, operator)\n VALUES ('ws-multi-group', 'alice', 'alice@example.com', true, false)",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": []
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "6f941e4454f736b32eaef80cdfb9582d6e75af3dc159e7c5f12497d3957f1eef"
|
||||
}
|
||||
26
backend/.sqlx/query-88a467f3c943b134a81ac69c3c6686d1ce1ff2f5aafc15ff1b63cfa86c09c4f0.json
generated
Normal file
26
backend/.sqlx/query-88a467f3c943b134a81ac69c3c6686d1ce1ff2f5aafc15ff1b63cfa86c09c4f0.json
generated
Normal file
@@ -0,0 +1,26 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT is_admin, operator FROM usr WHERE workspace_id = 'ws-multi-group' AND email = 'alice@example.com'",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "is_admin",
|
||||
"type_info": "Bool"
|
||||
},
|
||||
{
|
||||
"ordinal": 1,
|
||||
"name": "operator",
|
||||
"type_info": "Bool"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": []
|
||||
},
|
||||
"nullable": [
|
||||
false,
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "88a467f3c943b134a81ac69c3c6686d1ce1ff2f5aafc15ff1b63cfa86c09c4f0"
|
||||
}
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT s.hash as hash, dm.deployment_msg as deployment_msg \n FROM script s LEFT JOIN deployment_metadata dm ON s.hash = dm.script_hash\n WHERE s.workspace_id = $1 AND s.path = $2\n ORDER by s.created_at DESC",
|
||||
"query": "SELECT s.hash as hash, dm.deployment_msg as deployment_msg, s.created_at as created_at\n FROM script s LEFT JOIN deployment_metadata dm ON s.hash = dm.script_hash\n WHERE s.workspace_id = $1 AND s.path = $2\n ORDER by s.created_at DESC",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
@@ -12,6 +12,11 @@
|
||||
"ordinal": 1,
|
||||
"name": "deployment_msg",
|
||||
"type_info": "Text"
|
||||
},
|
||||
{
|
||||
"ordinal": 2,
|
||||
"name": "created_at",
|
||||
"type_info": "Timestamptz"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
@@ -22,8 +27,9 @@
|
||||
},
|
||||
"nullable": [
|
||||
false,
|
||||
true
|
||||
true,
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "726e956cfcd3ac7c07abeecdf92cf0996efe7fa7b671ac2b3b000ead0ea307de"
|
||||
"hash": "9a1483a81f5b086e0765d3d69483e29b09f66090e1f9d394564c16d921d2e66c"
|
||||
}
|
||||
@@ -1,29 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n UPDATE kafka_trigger\n SET\n kafka_resource_path = $1,\n group_id = $2,\n topics = $3,\n filters = $4,\n auto_offset_reset = $5,\n auto_commit = $6,\n script_path = $7,\n path = $8,\n is_flow = $9,\n edited_by = $10,\n permissioned_as = $11,\n edited_at = now(),\n server_id = NULL,\n error = NULL,\n error_handler_path = $14,\n error_handler_args = $15,\n retry = $16\n WHERE\n workspace_id = $12 AND path = $13\n ",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Varchar",
|
||||
"Varchar",
|
||||
"VarcharArray",
|
||||
"JsonbArray",
|
||||
"Varchar",
|
||||
"Bool",
|
||||
"Varchar",
|
||||
"Varchar",
|
||||
"Bool",
|
||||
"Varchar",
|
||||
"Varchar",
|
||||
"Text",
|
||||
"Text",
|
||||
"Varchar",
|
||||
"Jsonb",
|
||||
"Jsonb"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "a37cfc632dd37cf37c06743239b5ebc784e5da5ee25d47af187a75220d8fded7"
|
||||
}
|
||||
15
backend/.sqlx/query-a860dd9722f608184c4b1ef5e609b20cd61f9967a2012fc1c8fe352ee7596358.json
generated
Normal file
15
backend/.sqlx/query-a860dd9722f608184c4b1ef5e609b20cd61f9967a2012fc1c8fe352ee7596358.json
generated
Normal file
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "INSERT INTO pip_resolution_cache (hash, lockfile, expiration) VALUES ($1, $2, now() + ('7 days')::interval) ON CONFLICT (hash) DO UPDATE SET lockfile = EXCLUDED.lockfile, expiration = EXCLUDED.expiration",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Varchar",
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "a860dd9722f608184c4b1ef5e609b20cd61f9967a2012fc1c8fe352ee7596358"
|
||||
}
|
||||
18
backend/.sqlx/query-b38bd869477a729279cac3ccd4825191fb49e17e5f7e7297c0c819f52b486f49.json
generated
Normal file
18
backend/.sqlx/query-b38bd869477a729279cac3ccd4825191fb49e17e5f7e7297c0c819f52b486f49.json
generated
Normal file
@@ -0,0 +1,18 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "UPDATE usr SET is_admin = $1, operator = $2, added_via = $3 WHERE workspace_id = $4 AND email = $5 AND added_via->>'source' = 'instance_group'",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Bool",
|
||||
"Bool",
|
||||
"Jsonb",
|
||||
"Text",
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "b38bd869477a729279cac3ccd4825191fb49e17e5f7e7297c0c819f52b486f49"
|
||||
}
|
||||
@@ -1,44 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT\n schemaname || '.' || relname as \"table_name!\",\n COALESCE(n_live_tup, 0) as \"live_tuples!\",\n COALESCE(n_dead_tup, 0) as \"dead_tuples!\",\n last_autovacuum as \"last_autovacuum\",\n last_autoanalyze as \"last_autoanalyze\"\n FROM pg_stat_user_tables\n ORDER BY n_dead_tup DESC\n LIMIT 15",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "table_name!",
|
||||
"type_info": "Text"
|
||||
},
|
||||
{
|
||||
"ordinal": 1,
|
||||
"name": "live_tuples!",
|
||||
"type_info": "Int8"
|
||||
},
|
||||
{
|
||||
"ordinal": 2,
|
||||
"name": "dead_tuples!",
|
||||
"type_info": "Int8"
|
||||
},
|
||||
{
|
||||
"ordinal": 3,
|
||||
"name": "last_autovacuum",
|
||||
"type_info": "Timestamptz"
|
||||
},
|
||||
{
|
||||
"ordinal": 4,
|
||||
"name": "last_autoanalyze",
|
||||
"type_info": "Timestamptz"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": []
|
||||
},
|
||||
"nullable": [
|
||||
null,
|
||||
null,
|
||||
null,
|
||||
true,
|
||||
true
|
||||
]
|
||||
},
|
||||
"hash": "bc54ea311580a0525c1f36aaa543c5798e6f7aca1e6e564330766d77038ef0e3"
|
||||
}
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT s.hash as hash, dm.deployment_msg as deployment_msg \n FROM script s LEFT JOIN deployment_metadata dm ON s.hash = dm.script_hash\n WHERE s.workspace_id = $1 AND s.path = $2\n ORDER by s.created_at DESC LIMIT 1",
|
||||
"query": "SELECT s.hash as hash, dm.deployment_msg as deployment_msg, s.created_at as created_at\n FROM script s LEFT JOIN deployment_metadata dm ON s.hash = dm.script_hash\n WHERE s.workspace_id = $1 AND s.path = $2\n ORDER by s.created_at DESC LIMIT 1",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
@@ -12,6 +12,11 @@
|
||||
"ordinal": 1,
|
||||
"name": "deployment_msg",
|
||||
"type_info": "Text"
|
||||
},
|
||||
{
|
||||
"ordinal": 2,
|
||||
"name": "created_at",
|
||||
"type_info": "Timestamptz"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
@@ -22,8 +27,9 @@
|
||||
},
|
||||
"nullable": [
|
||||
false,
|
||||
true
|
||||
true,
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "cf2a6ad6471a40b6298775cda9300aeecdd75503bed59d80cd62091d1642d1ec"
|
||||
"hash": "c73e98e5a937f44724a96ee1b74d31fa71a7be3b8ba3dec9f59f54a6c4030462"
|
||||
}
|
||||
@@ -1,29 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n UPDATE\n websocket_trigger\n SET\n url = $1,\n script_path = $2,\n path = $3,\n is_flow = $4,\n filters = $5,\n initial_messages = $6,\n url_runnable_args = $7,\n edited_by = $8,\n permissioned_as = $9,\n can_return_message = $10,\n can_return_error_result = $11,\n edited_at = now(),\n server_id = NULL,\n error = NULL,\n error_handler_path = $14,\n error_handler_args = $15,\n retry = $16\n WHERE\n workspace_id = $12 AND path = $13\n ",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Varchar",
|
||||
"Varchar",
|
||||
"Varchar",
|
||||
"Bool",
|
||||
"JsonbArray",
|
||||
"JsonbArray",
|
||||
"Jsonb",
|
||||
"Varchar",
|
||||
"Varchar",
|
||||
"Bool",
|
||||
"Bool",
|
||||
"Text",
|
||||
"Text",
|
||||
"Varchar",
|
||||
"Jsonb",
|
||||
"Jsonb"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "c7aed7fe3b6774477d403bc3e7fcbce7cdbdd1feb553718cbde60bb8ccff4733"
|
||||
}
|
||||
@@ -1,38 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "WITH active_users as (SELECT distinct username as email FROM audit WHERE timestamp > NOW() - INTERVAL '1 month' AND (operation = 'users.login' OR operation = 'oauth.login' OR operation = 'users.token.refresh')),\n active_authors as (SELECT distinct email FROM usr WHERE usr.operator IS false AND email IN (SELECT email FROM active_users)),\n active_authors_agg as (SELECT array_agg(email) as authors FROM active_authors),\n active_ops_agg as (SELECT array_agg(email) as operators from active_users WHERE email NOT IN (SELECT email FROM active_authors))\n SELECT active_authors_agg.authors, active_ops_agg.operators, array_length(active_authors_agg.authors, 1) as author_count, array_length(active_ops_agg.operators, 1) as operator_count FROM active_authors_agg, active_ops_agg",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "authors",
|
||||
"type_info": "VarcharArray"
|
||||
},
|
||||
{
|
||||
"ordinal": 1,
|
||||
"name": "operators",
|
||||
"type_info": "VarcharArray"
|
||||
},
|
||||
{
|
||||
"ordinal": 2,
|
||||
"name": "author_count",
|
||||
"type_info": "Int4"
|
||||
},
|
||||
{
|
||||
"ordinal": 3,
|
||||
"name": "operator_count",
|
||||
"type_info": "Int4"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": []
|
||||
},
|
||||
"nullable": [
|
||||
null,
|
||||
null,
|
||||
null,
|
||||
null
|
||||
]
|
||||
},
|
||||
"hash": "cb3862634f18160207ee2621ddfca43f00456a27fda32583846497116f92f96c"
|
||||
}
|
||||
20
backend/.sqlx/query-d43a4ff78e48580815fb912c98639a08d45596a9f11a2dcf5b1e0d135844ecda.json
generated
Normal file
20
backend/.sqlx/query-d43a4ff78e48580815fb912c98639a08d45596a9f11a2dcf5b1e0d135844ecda.json
generated
Normal file
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT value FROM global_settings WHERE name = 'plain_emails_telemetry'",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "value",
|
||||
"type_info": "Jsonb"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": []
|
||||
},
|
||||
"nullable": [
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "d43a4ff78e48580815fb912c98639a08d45596a9f11a2dcf5b1e0d135844ecda"
|
||||
}
|
||||
30
backend/.sqlx/query-e3d4f89ce36337af15d237b543eaca47771b480ff194884f9c947dcaf71d6cf9.json
generated
Normal file
30
backend/.sqlx/query-e3d4f89ce36337af15d237b543eaca47771b480ff194884f9c947dcaf71d6cf9.json
generated
Normal file
@@ -0,0 +1,30 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n UPDATE\n websocket_trigger\n SET\n url = $1,\n script_path = $2,\n path = $3,\n is_flow = $4,\n filters = $5,\n filter_logic = $6,\n initial_messages = $7,\n url_runnable_args = $8,\n edited_by = $9,\n permissioned_as = $10,\n can_return_message = $11,\n can_return_error_result = $12,\n edited_at = now(),\n server_id = NULL,\n error = NULL,\n error_handler_path = $15,\n error_handler_args = $16,\n retry = $17\n WHERE\n workspace_id = $13 AND path = $14\n ",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Varchar",
|
||||
"Varchar",
|
||||
"Varchar",
|
||||
"Bool",
|
||||
"JsonbArray",
|
||||
"Varchar",
|
||||
"JsonbArray",
|
||||
"Jsonb",
|
||||
"Varchar",
|
||||
"Varchar",
|
||||
"Bool",
|
||||
"Bool",
|
||||
"Text",
|
||||
"Text",
|
||||
"Varchar",
|
||||
"Jsonb",
|
||||
"Jsonb"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "e3d4f89ce36337af15d237b543eaca47771b480ff194884f9c947dcaf71d6cf9"
|
||||
}
|
||||
26
backend/.sqlx/query-e58ef252b0d2b81e9cd76f394a396abefd791906ada29dd5a7a9148157635ca5.json
generated
Normal file
26
backend/.sqlx/query-e58ef252b0d2b81e9cd76f394a396abefd791906ada29dd5a7a9148157635ca5.json
generated
Normal file
@@ -0,0 +1,26 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n SELECT auto_invite->'instance_groups_roles' as instance_groups_roles,\n auto_invite->'instance_groups' as instance_groups_json\n FROM workspace_settings WHERE workspace_id = 'ws-multi-group'\n ",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "instance_groups_roles",
|
||||
"type_info": "Jsonb"
|
||||
},
|
||||
{
|
||||
"ordinal": 1,
|
||||
"name": "instance_groups_json",
|
||||
"type_info": "Jsonb"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": []
|
||||
},
|
||||
"nullable": [
|
||||
null,
|
||||
null
|
||||
]
|
||||
},
|
||||
"hash": "e58ef252b0d2b81e9cd76f394a396abefd791906ada29dd5a7a9148157635ca5"
|
||||
}
|
||||
408
backend/Cargo.lock
generated
408
backend/Cargo.lock
generated
File diff suppressed because it is too large
Load Diff
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "windmill"
|
||||
version = "1.668.0"
|
||||
version = "1.669.1"
|
||||
authors.workspace = true
|
||||
edition.workspace = true
|
||||
|
||||
@@ -79,10 +79,10 @@ members = [
|
||||
"./windmill-test-utils",
|
||||
"./windmill-api-integration-tests",
|
||||
]
|
||||
exclude = ["./windmill-duckdb-ffi-internal"]
|
||||
exclude = ["./windmill-duckdb-ffi-internal", "./parsers/windmill-parser-wasm"]
|
||||
|
||||
[workspace.package]
|
||||
version = "1.668.0"
|
||||
version = "1.669.1"
|
||||
authors = ["Ruben Fiszel <ruben@windmill.dev>"]
|
||||
edition = "2021"
|
||||
|
||||
|
||||
@@ -1 +1 @@
|
||||
02c0d34e54e71c9293f9cefb56f68652cf0db8a5
|
||||
e08a87450627bef9013498e40ee93a47bedda7ee
|
||||
|
||||
@@ -0,0 +1,2 @@
|
||||
ALTER TABLE kafka_trigger DROP COLUMN filter_logic;
|
||||
ALTER TABLE websocket_trigger DROP COLUMN filter_logic;
|
||||
@@ -0,0 +1,2 @@
|
||||
ALTER TABLE kafka_trigger ADD COLUMN filter_logic VARCHAR(3) NOT NULL DEFAULT 'and';
|
||||
ALTER TABLE websocket_trigger ADD COLUMN filter_logic VARCHAR(3) NOT NULL DEFAULT 'and';
|
||||
@@ -27,11 +27,15 @@ pub struct DagNode {
|
||||
#[serde(tag = "type")]
|
||||
pub enum DagNodeType {
|
||||
Step { name: String, script: String },
|
||||
InlineStep { name: String },
|
||||
Sleep { seconds: String },
|
||||
WaitForApproval,
|
||||
Branch { condition_source: String },
|
||||
ParallelStart,
|
||||
ParallelEnd,
|
||||
LoopStart { iter_source: String },
|
||||
LoopEnd,
|
||||
Merge,
|
||||
Return,
|
||||
}
|
||||
|
||||
|
||||
@@ -37,7 +37,8 @@ impl LineIndex {
|
||||
/// Maps task function name → optional external path (from `@task(path="...")`)
|
||||
type TaskFunctions = HashMap<String, Option<String>>;
|
||||
|
||||
/// First pass: scan top-level `@task async def foo(...)` declarations.
|
||||
/// First pass: scan top-level `@task async def foo(...)` declarations
|
||||
/// and `foo = task_script("path")` / `foo = task_flow("path")` assignments.
|
||||
fn collect_task_functions(stmts: &[Stmt]) -> TaskFunctions {
|
||||
let mut tasks = HashMap::new();
|
||||
for stmt in stmts {
|
||||
@@ -61,6 +62,30 @@ fn collect_task_functions(stmts: &[Stmt]) -> TaskFunctions {
|
||||
}
|
||||
}
|
||||
}
|
||||
// foo = task_script("path") or foo = task_flow("path")
|
||||
if let Stmt::Assign(assign) = stmt {
|
||||
if let Expr::Call(call) = assign.value.as_ref() {
|
||||
if let Expr::Name(ExprName { id, .. }) = call.func.as_ref() {
|
||||
if id.as_str() == "task_script" || id.as_str() == "task_flow" {
|
||||
// Extract the path from the first positional argument
|
||||
let path = call.args.first().and_then(|arg| {
|
||||
if let Expr::Constant(c) = arg {
|
||||
if let rustpython_parser::ast::Constant::Str(s) = &c.value {
|
||||
return Some(s.to_string());
|
||||
}
|
||||
}
|
||||
None
|
||||
});
|
||||
// Extract variable name from target
|
||||
if let Some(Expr::Name(ExprName { id: var_name, .. })) =
|
||||
assign.targets.first()
|
||||
{
|
||||
tasks.insert(var_name.to_string(), path);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
tasks
|
||||
}
|
||||
@@ -88,8 +113,6 @@ struct WacWalker {
|
||||
node_counter: usize,
|
||||
line_index: LineIndex,
|
||||
task_functions: TaskFunctions,
|
||||
in_try: bool,
|
||||
in_while: bool,
|
||||
in_nested_func: bool,
|
||||
in_comprehension: bool,
|
||||
}
|
||||
@@ -103,8 +126,6 @@ impl WacWalker {
|
||||
node_counter: 0,
|
||||
line_index: LineIndex::new(source),
|
||||
task_functions,
|
||||
in_try: false,
|
||||
in_while: false,
|
||||
in_nested_func: false,
|
||||
in_comprehension: false,
|
||||
}
|
||||
@@ -292,6 +313,9 @@ impl WacWalker {
|
||||
if self.is_task_fn_call(expr) {
|
||||
return true;
|
||||
}
|
||||
if Self::is_sdk_call(expr) {
|
||||
return true;
|
||||
}
|
||||
match expr {
|
||||
Expr::Await(ExprAwait { value, .. }) => self.expr_contains_step(value),
|
||||
Expr::Call(call) => {
|
||||
@@ -307,6 +331,17 @@ impl WacWalker {
|
||||
}
|
||||
}
|
||||
|
||||
/// Check if expr is a call to a known SDK function (step, sleep, wait_for_approval)
|
||||
fn is_sdk_call(expr: &Expr) -> bool {
|
||||
if let Expr::Call(call) = expr {
|
||||
if let Expr::Name(ExprName { id, .. }) = call.func.as_ref() {
|
||||
let name = id.as_str();
|
||||
return name == "step" || name == "sleep" || name == "wait_for_approval";
|
||||
}
|
||||
}
|
||||
false
|
||||
}
|
||||
|
||||
/// Walk a list of statements, returning (first_node_id, last_node_id)
|
||||
fn walk_body(&mut self, body: &[Stmt]) -> Option<(String, String)> {
|
||||
let mut first_id: Option<String> = None;
|
||||
@@ -353,13 +388,17 @@ impl WacWalker {
|
||||
}
|
||||
|
||||
fn walk_expr_stmt(&mut self, expr: &Expr) -> Option<(String, String)> {
|
||||
// await task_fn(...)
|
||||
// await task_fn(...) / await step(...) / await sleep(...) / await wait_for_approval(...)
|
||||
if let Expr::Await(ExprAwait { value, .. }) = expr {
|
||||
// await task_fn(...)
|
||||
if let Expr::Call(call) = value.as_ref() {
|
||||
if self.is_task_fn_call(&Expr::Call(call.clone())) {
|
||||
return self.emit_step(call, expr);
|
||||
}
|
||||
// Check for SDK-level calls: step(), sleep(), wait_for_approval()
|
||||
if let Some(result) = self.try_emit_sdk_call(call, expr) {
|
||||
return Some(result);
|
||||
}
|
||||
}
|
||||
// await asyncio.gather(task_fn(...), task_fn(...), ...)
|
||||
if Self::is_asyncio_gather_call(value) {
|
||||
@@ -378,17 +417,69 @@ impl WacWalker {
|
||||
None
|
||||
}
|
||||
|
||||
/// Try to emit a node for SDK-level calls: step(), sleep(), wait_for_approval()
|
||||
fn try_emit_sdk_call(&mut self, call: &ExprCall, expr: &Expr) -> Option<(String, String)> {
|
||||
let callee_name = match call.func.as_ref() {
|
||||
Expr::Name(ExprName { id, .. }) => Some(id.as_str()),
|
||||
_ => None,
|
||||
}?;
|
||||
|
||||
let line = self.line_of_expr(expr);
|
||||
|
||||
match callee_name {
|
||||
"step" => {
|
||||
// step("name", fn) — extract the name from the first string argument
|
||||
let name = call
|
||||
.args
|
||||
.first()
|
||||
.and_then(|arg| {
|
||||
if let Expr::Constant(c) = arg {
|
||||
if let rustpython_parser::ast::Constant::Str(s) = &c.value {
|
||||
return Some(s.to_string());
|
||||
}
|
||||
}
|
||||
None
|
||||
})
|
||||
.unwrap_or_else(|| "step".to_string());
|
||||
let id = self.next_id();
|
||||
let node_id = self.add_node(DagNode {
|
||||
id: id.clone(),
|
||||
node_type: DagNodeType::InlineStep { name: name.clone() },
|
||||
label: name,
|
||||
line,
|
||||
});
|
||||
Some((node_id.clone(), node_id))
|
||||
}
|
||||
"sleep" => {
|
||||
let seconds = call
|
||||
.args
|
||||
.first()
|
||||
.map(|arg| Self::expr_to_source(arg))
|
||||
.unwrap_or_else(|| "?".to_string());
|
||||
let id = self.next_id();
|
||||
let node_id = self.add_node(DagNode {
|
||||
id: id.clone(),
|
||||
node_type: DagNodeType::Sleep { seconds: seconds.clone() },
|
||||
label: format!("sleep({seconds})"),
|
||||
line,
|
||||
});
|
||||
Some((node_id.clone(), node_id))
|
||||
}
|
||||
"wait_for_approval" => {
|
||||
let id = self.next_id();
|
||||
let node_id = self.add_node(DagNode {
|
||||
id: id.clone(),
|
||||
node_type: DagNodeType::WaitForApproval,
|
||||
label: "wait_for_approval".to_string(),
|
||||
line,
|
||||
});
|
||||
Some((node_id.clone(), node_id))
|
||||
}
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
fn emit_step(&mut self, call: &ExprCall, expr: &Expr) -> Option<(String, String)> {
|
||||
if self.in_try {
|
||||
self.errors
|
||||
.push(validation::error_step_in_try(self.line_of_expr(expr)));
|
||||
return None;
|
||||
}
|
||||
if self.in_while {
|
||||
self.errors
|
||||
.push(validation::error_step_in_while(self.line_of_expr(expr)));
|
||||
return None;
|
||||
}
|
||||
if self.in_nested_func {
|
||||
self.errors.push(validation::error_step_in_nested_function(
|
||||
self.line_of_expr(expr),
|
||||
@@ -416,17 +507,6 @@ impl WacWalker {
|
||||
}
|
||||
|
||||
fn emit_parallel(&mut self, gather_call: &ExprCall, expr: &Expr) -> Option<(String, String)> {
|
||||
if self.in_try {
|
||||
self.errors
|
||||
.push(validation::error_step_in_try(self.line_of_expr(expr)));
|
||||
return None;
|
||||
}
|
||||
if self.in_while {
|
||||
self.errors
|
||||
.push(validation::error_step_in_while(self.line_of_expr(expr)));
|
||||
return None;
|
||||
}
|
||||
|
||||
let line = self.line_of_expr(expr);
|
||||
let start_id = self.next_id();
|
||||
let start_node_id = self.add_node(DagNode {
|
||||
@@ -491,8 +571,6 @@ impl WacWalker {
|
||||
line,
|
||||
});
|
||||
|
||||
let merge_id = format!("{branch_id}_merge");
|
||||
|
||||
let mut last_ids = Vec::new();
|
||||
|
||||
if let Some((true_first, true_last)) = self.walk_body(&if_stmt.body) {
|
||||
@@ -514,7 +592,17 @@ impl WacWalker {
|
||||
if last_ids.len() == 1 {
|
||||
Some((branch_node_id, last_ids.into_iter().next().unwrap()))
|
||||
} else {
|
||||
Some((branch_node_id, merge_id))
|
||||
let merge_id = format!("{branch_id}_merge");
|
||||
let merge_node_id = self.add_node(DagNode {
|
||||
id: merge_id,
|
||||
node_type: DagNodeType::Merge,
|
||||
label: "merge".to_string(),
|
||||
line,
|
||||
});
|
||||
for last in last_ids {
|
||||
self.add_edge(&last, &merge_node_id, None);
|
||||
}
|
||||
Some((branch_node_id, merge_node_id))
|
||||
}
|
||||
}
|
||||
|
||||
@@ -552,11 +640,36 @@ impl WacWalker {
|
||||
}
|
||||
|
||||
fn walk_while(&mut self, while_stmt: &StmtWhile) -> Option<(String, String)> {
|
||||
if self.body_contains_step(&while_stmt.body) {
|
||||
let line = self.line_index.line_of(while_stmt.range.start().to_usize());
|
||||
self.errors.push(validation::error_step_in_while(line));
|
||||
if !self.body_contains_step(&while_stmt.body) {
|
||||
return None;
|
||||
}
|
||||
None
|
||||
|
||||
let line = self.line_index.line_of(while_stmt.range.start().to_usize());
|
||||
let condition = Self::expr_to_source(&while_stmt.test);
|
||||
|
||||
let start_id = self.next_id();
|
||||
let start_node_id = self.add_node(DagNode {
|
||||
id: start_id.clone(),
|
||||
node_type: DagNodeType::LoopStart { iter_source: condition },
|
||||
label: "while".to_string(),
|
||||
line,
|
||||
});
|
||||
|
||||
if let Some((body_first, body_last)) = self.walk_body(&while_stmt.body) {
|
||||
self.add_edge(&start_node_id, &body_first, None);
|
||||
self.add_edge(&body_last, &start_node_id, Some("next".to_string()));
|
||||
}
|
||||
|
||||
let end_id = self.next_id();
|
||||
let end_node_id = self.add_node(DagNode {
|
||||
id: end_id.clone(),
|
||||
node_type: DagNodeType::LoopEnd,
|
||||
label: "end while".to_string(),
|
||||
line,
|
||||
});
|
||||
self.add_edge(&start_node_id, &end_node_id, Some("done".to_string()));
|
||||
|
||||
Some((start_node_id, end_node_id))
|
||||
}
|
||||
|
||||
fn walk_try(&mut self, try_stmt: &StmtTry) -> Option<(String, String)> {
|
||||
@@ -569,11 +682,17 @@ impl WacWalker {
|
||||
}
|
||||
});
|
||||
|
||||
if has_steps {
|
||||
let line = self.line_index.line_of(try_stmt.range.start().to_usize());
|
||||
self.errors.push(validation::error_step_in_try(line));
|
||||
if !has_steps {
|
||||
return None;
|
||||
}
|
||||
None
|
||||
|
||||
let line = self.line_index.line_of(try_stmt.range.start().to_usize());
|
||||
self.emit_try_catch_branch(
|
||||
&try_stmt.body,
|
||||
&try_stmt.handlers,
|
||||
&try_stmt.finalbody,
|
||||
line,
|
||||
)
|
||||
}
|
||||
|
||||
fn walk_try_star(&mut self, try_stmt: &StmtTryStar) -> Option<(String, String)> {
|
||||
@@ -586,11 +705,81 @@ impl WacWalker {
|
||||
}
|
||||
});
|
||||
|
||||
if has_steps {
|
||||
let line = self.line_index.line_of(try_stmt.range.start().to_usize());
|
||||
self.errors.push(validation::error_step_in_try(line));
|
||||
if !has_steps {
|
||||
return None;
|
||||
}
|
||||
None
|
||||
|
||||
let line = self.line_index.line_of(try_stmt.range.start().to_usize());
|
||||
self.emit_try_catch_branch(
|
||||
&try_stmt.body,
|
||||
&try_stmt.handlers,
|
||||
&try_stmt.finalbody,
|
||||
line,
|
||||
)
|
||||
}
|
||||
|
||||
fn emit_try_catch_branch(
|
||||
&mut self,
|
||||
try_body: &[Stmt],
|
||||
handlers: &[rustpython_parser::ast::ExceptHandler],
|
||||
finally_body: &[Stmt],
|
||||
line: usize,
|
||||
) -> Option<(String, String)> {
|
||||
let branch_id = self.next_id();
|
||||
let branch_node_id = self.add_node(DagNode {
|
||||
id: branch_id.clone(),
|
||||
node_type: DagNodeType::Branch { condition_source: "try/except".to_string() },
|
||||
label: "try".to_string(),
|
||||
line,
|
||||
});
|
||||
|
||||
let mut last_ids = Vec::new();
|
||||
|
||||
// Try body
|
||||
if let Some((try_first, try_last)) = self.walk_body(try_body) {
|
||||
self.add_edge(&branch_node_id, &try_first, Some("try".to_string()));
|
||||
last_ids.push(try_last);
|
||||
} else {
|
||||
last_ids.push(branch_node_id.clone());
|
||||
}
|
||||
|
||||
// Except handlers
|
||||
for handler in handlers {
|
||||
match handler {
|
||||
rustpython_parser::ast::ExceptHandler::ExceptHandler(eh) => {
|
||||
if let Some((catch_first, catch_last)) = self.walk_body(&eh.body) {
|
||||
self.add_edge(&branch_node_id, &catch_first, Some("except".to_string()));
|
||||
last_ids.push(catch_last);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Finally body — sequential after merge
|
||||
let merge_last = if last_ids.len() == 1 {
|
||||
last_ids.into_iter().next().unwrap()
|
||||
} else {
|
||||
let merge_id = format!("{branch_id}_merge");
|
||||
let merge_node_id = self.add_node(DagNode {
|
||||
id: merge_id,
|
||||
node_type: DagNodeType::Merge,
|
||||
label: "merge".to_string(),
|
||||
line,
|
||||
});
|
||||
for last in last_ids {
|
||||
self.add_edge(&last, &merge_node_id, None);
|
||||
}
|
||||
merge_node_id
|
||||
};
|
||||
|
||||
if !finally_body.is_empty() {
|
||||
if let Some((finally_first, finally_last)) = self.walk_body(finally_body) {
|
||||
self.add_edge(&merge_last, &finally_first, None);
|
||||
return Some((branch_node_id, finally_last));
|
||||
}
|
||||
}
|
||||
|
||||
Some((branch_node_id, merge_last))
|
||||
}
|
||||
|
||||
fn walk_return(&mut self, ret: &StmtReturn) -> Option<(String, String)> {
|
||||
|
||||
@@ -51,22 +51,29 @@ fn extract_var_name(pat: &Pat) -> Option<String> {
|
||||
}
|
||||
}
|
||||
|
||||
/// Check if expr is `task(async fn)` or `task("path", async fn)`.
|
||||
/// Returns Some(optional_path) if it is a task() call.
|
||||
/// Check if expr is `task(async fn)`, `task("path", async fn)`,
|
||||
/// `taskScript("path")`, or `taskFlow("path")`.
|
||||
/// Returns Some(optional_path) if it is a task/taskScript/taskFlow call.
|
||||
fn extract_task_call_info(expr: &Expr) -> Option<Option<String>> {
|
||||
if let Expr::Call(call) = expr {
|
||||
if let Callee::Expr(callee) = &call.callee {
|
||||
if let Expr::Ident(ident) = callee.as_ref() {
|
||||
if ident.sym.as_ref() == "task" {
|
||||
let name = ident.sym.as_ref();
|
||||
if name == "task" {
|
||||
// task("f/path", async fn) or task(async fn)
|
||||
if call.args.len() == 2 {
|
||||
// task("f/path", async fn)
|
||||
let path = extract_string_lit(&call.args[0].expr);
|
||||
return Some(path);
|
||||
} else if call.args.len() == 1 {
|
||||
// task(async fn)
|
||||
return Some(None);
|
||||
}
|
||||
} else if name == "taskScript" || name == "taskFlow" {
|
||||
// taskScript("./helper.ts") or taskFlow("f/my_flow")
|
||||
if let Some(first_arg) = call.args.first() {
|
||||
let path = extract_string_lit(&first_arg.expr);
|
||||
return Some(path);
|
||||
}
|
||||
return Some(None);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -81,8 +88,6 @@ struct TsWacWalker {
|
||||
node_counter: usize,
|
||||
cm: Lrc<SourceMap>,
|
||||
task_functions: TaskFunctions,
|
||||
in_try: bool,
|
||||
in_while: bool,
|
||||
in_nested_func: bool,
|
||||
}
|
||||
|
||||
@@ -95,8 +100,6 @@ impl TsWacWalker {
|
||||
node_counter: 0,
|
||||
cm,
|
||||
task_functions,
|
||||
in_try: false,
|
||||
in_while: false,
|
||||
in_nested_func: false,
|
||||
}
|
||||
}
|
||||
@@ -224,6 +227,9 @@ impl TsWacWalker {
|
||||
if self.is_task_call(expr) {
|
||||
return true;
|
||||
}
|
||||
if Self::is_sdk_call(expr) {
|
||||
return true;
|
||||
}
|
||||
match expr {
|
||||
Expr::Await(await_expr) => self.expr_contains_step(&await_expr.arg),
|
||||
Expr::Call(call) => {
|
||||
@@ -237,6 +243,19 @@ impl TsWacWalker {
|
||||
}
|
||||
}
|
||||
|
||||
/// Check if expr is a call to a known SDK function (step, sleep, waitForApproval)
|
||||
fn is_sdk_call(expr: &Expr) -> bool {
|
||||
if let Expr::Call(call) = expr {
|
||||
if let Callee::Expr(callee) = &call.callee {
|
||||
if let Expr::Ident(ident) = callee.as_ref() {
|
||||
let name = ident.sym.as_ref();
|
||||
return name == "step" || name == "sleep" || name == "waitForApproval";
|
||||
}
|
||||
}
|
||||
}
|
||||
false
|
||||
}
|
||||
|
||||
fn walk_body(&mut self, stmts: &[Stmt]) -> Option<(String, String)> {
|
||||
let mut first_id: Option<String> = None;
|
||||
let mut prev_id: Option<String> = None;
|
||||
@@ -294,12 +313,16 @@ impl TsWacWalker {
|
||||
}
|
||||
|
||||
fn walk_expr_stmt(&mut self, expr: &Expr) -> Option<(String, String)> {
|
||||
// await task_fn(...)
|
||||
// await task_fn(...) / await step(...) / await sleep(...) / await waitForApproval(...)
|
||||
if let Expr::Await(await_expr) = expr {
|
||||
if let Expr::Call(call) = await_expr.arg.as_ref() {
|
||||
if self.is_task_call(&Expr::Call(call.clone())) {
|
||||
return self.emit_step(call, expr);
|
||||
}
|
||||
// Check for SDK-level calls: step(), sleep(), waitForApproval()
|
||||
if let Some(result) = self.try_emit_sdk_call(call, expr) {
|
||||
return Some(result);
|
||||
}
|
||||
}
|
||||
// await Promise.all([task_fn(...), ...])
|
||||
if Self::is_promise_all(&await_expr.arg) {
|
||||
@@ -318,17 +341,70 @@ impl TsWacWalker {
|
||||
None
|
||||
}
|
||||
|
||||
/// Try to emit a node for SDK-level calls: step(), sleep(), waitForApproval()
|
||||
fn try_emit_sdk_call(&mut self, call: &CallExpr, expr: &Expr) -> Option<(String, String)> {
|
||||
let callee_name = match &call.callee {
|
||||
Callee::Expr(callee) => match callee.as_ref() {
|
||||
Expr::Ident(ident) => Some(ident.sym.as_ref().to_string()),
|
||||
_ => None,
|
||||
},
|
||||
_ => None,
|
||||
}?;
|
||||
|
||||
let line = self.span_line(expr.span());
|
||||
|
||||
match callee_name.as_str() {
|
||||
"step" => {
|
||||
// step("name", fn) — extract the name from the first string argument
|
||||
let name = call
|
||||
.args
|
||||
.first()
|
||||
.and_then(|a| extract_string_lit(&a.expr))
|
||||
.unwrap_or_else(|| "step".to_string());
|
||||
let id = self.next_id();
|
||||
let node_id = self.add_node(DagNode {
|
||||
id: id.clone(),
|
||||
node_type: DagNodeType::InlineStep { name: name.clone() },
|
||||
label: name,
|
||||
line,
|
||||
});
|
||||
Some((node_id.clone(), node_id))
|
||||
}
|
||||
"sleep" => {
|
||||
// sleep(N) — extract the duration from the first argument
|
||||
let seconds = call
|
||||
.args
|
||||
.first()
|
||||
.map(|a| {
|
||||
self.cm
|
||||
.span_to_snippet(a.expr.span())
|
||||
.unwrap_or_else(|_| "?".to_string())
|
||||
})
|
||||
.unwrap_or_else(|| "?".to_string());
|
||||
let id = self.next_id();
|
||||
let node_id = self.add_node(DagNode {
|
||||
id: id.clone(),
|
||||
node_type: DagNodeType::Sleep { seconds: seconds.clone() },
|
||||
label: format!("sleep({seconds})"),
|
||||
line,
|
||||
});
|
||||
Some((node_id.clone(), node_id))
|
||||
}
|
||||
"waitForApproval" => {
|
||||
let id = self.next_id();
|
||||
let node_id = self.add_node(DagNode {
|
||||
id: id.clone(),
|
||||
node_type: DagNodeType::WaitForApproval,
|
||||
label: "waitForApproval".to_string(),
|
||||
line,
|
||||
});
|
||||
Some((node_id.clone(), node_id))
|
||||
}
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
fn emit_step(&mut self, call: &CallExpr, expr: &Expr) -> Option<(String, String)> {
|
||||
if self.in_try {
|
||||
self.errors
|
||||
.push(validation::error_step_in_catch(self.span_line(expr.span())));
|
||||
return None;
|
||||
}
|
||||
if self.in_while {
|
||||
self.errors
|
||||
.push(validation::error_step_in_while(self.span_line(expr.span())));
|
||||
return None;
|
||||
}
|
||||
if self.in_nested_func {
|
||||
self.errors.push(validation::error_step_in_nested_function(
|
||||
self.span_line(expr.span()),
|
||||
@@ -350,17 +426,6 @@ impl TsWacWalker {
|
||||
}
|
||||
|
||||
fn emit_parallel(&mut self, promise_call: &CallExpr, expr: &Expr) -> Option<(String, String)> {
|
||||
if self.in_try {
|
||||
self.errors
|
||||
.push(validation::error_step_in_catch(self.span_line(expr.span())));
|
||||
return None;
|
||||
}
|
||||
if self.in_while {
|
||||
self.errors
|
||||
.push(validation::error_step_in_while(self.span_line(expr.span())));
|
||||
return None;
|
||||
}
|
||||
|
||||
let line = self.span_line(expr.span());
|
||||
let start_id = self.next_id();
|
||||
let start_node_id = self.add_node(DagNode {
|
||||
@@ -457,7 +522,16 @@ impl TsWacWalker {
|
||||
Some((branch_node_id, last_ids.into_iter().next().unwrap()))
|
||||
} else {
|
||||
let merge_id = format!("{branch_id}_merge");
|
||||
Some((branch_node_id, merge_id))
|
||||
let merge_node_id = self.add_node(DagNode {
|
||||
id: merge_id,
|
||||
node_type: DagNodeType::Merge,
|
||||
label: "merge".to_string(),
|
||||
line,
|
||||
});
|
||||
for last in last_ids {
|
||||
self.add_edge(&last, &merge_node_id, None);
|
||||
}
|
||||
Some((branch_node_id, merge_node_id))
|
||||
}
|
||||
}
|
||||
|
||||
@@ -473,7 +547,7 @@ impl TsWacWalker {
|
||||
return None;
|
||||
}
|
||||
let iter_source = self.expr_to_source(&for_in.right);
|
||||
self.walk_loop_body_with_iter(&for_in.body, for_in.span, &iter_source)
|
||||
self.walk_loop_body_with_iter(&for_in.body, for_in.span, &iter_source, "for")
|
||||
}
|
||||
|
||||
fn walk_for_of(&mut self, for_of: &ForOfStmt) -> Option<(String, String)> {
|
||||
@@ -481,7 +555,7 @@ impl TsWacWalker {
|
||||
return None;
|
||||
}
|
||||
let iter_source = self.expr_to_source(&for_of.right);
|
||||
self.walk_loop_body_with_iter(&for_of.body, for_of.span, &iter_source)
|
||||
self.walk_loop_body_with_iter(&for_of.body, for_of.span, &iter_source, "for")
|
||||
}
|
||||
|
||||
fn walk_loop_body(
|
||||
@@ -490,7 +564,7 @@ impl TsWacWalker {
|
||||
span: swc_common::Span,
|
||||
_label: &str,
|
||||
) -> Option<(String, String)> {
|
||||
self.walk_loop_body_with_iter(body, span, "...")
|
||||
self.walk_loop_body_with_iter(body, span, "...", "for")
|
||||
}
|
||||
|
||||
fn walk_loop_body_with_iter(
|
||||
@@ -498,13 +572,14 @@ impl TsWacWalker {
|
||||
body: &Stmt,
|
||||
span: swc_common::Span,
|
||||
iter_source: &str,
|
||||
loop_label: &str,
|
||||
) -> Option<(String, String)> {
|
||||
let line = self.span_line(span);
|
||||
let start_id = self.next_id();
|
||||
let start_node_id = self.add_node(DagNode {
|
||||
id: start_id.clone(),
|
||||
node_type: DagNodeType::LoopStart { iter_source: iter_source.to_string() },
|
||||
label: "for".to_string(),
|
||||
label: loop_label.to_string(),
|
||||
line,
|
||||
});
|
||||
|
||||
@@ -526,12 +601,11 @@ impl TsWacWalker {
|
||||
}
|
||||
|
||||
fn walk_while(&mut self, while_stmt: &WhileStmt) -> Option<(String, String)> {
|
||||
if self.stmt_contains_step(&while_stmt.body) {
|
||||
self.errors.push(validation::error_step_in_while(
|
||||
self.span_line(while_stmt.span),
|
||||
));
|
||||
if !self.stmt_contains_step(&while_stmt.body) {
|
||||
return None;
|
||||
}
|
||||
None
|
||||
let condition = self.expr_to_source(&while_stmt.test);
|
||||
self.walk_loop_body_with_iter(&while_stmt.body, while_stmt.span, &condition, "while")
|
||||
}
|
||||
|
||||
fn walk_try(&mut self, try_stmt: &TryStmt) -> Option<(String, String)> {
|
||||
@@ -545,12 +619,62 @@ impl TsWacWalker {
|
||||
.as_ref()
|
||||
.map_or(false, |f| self.body_contains_step(&f.stmts));
|
||||
|
||||
if has_steps {
|
||||
self.errors.push(validation::error_step_in_catch(
|
||||
self.span_line(try_stmt.span),
|
||||
));
|
||||
if !has_steps {
|
||||
return None;
|
||||
}
|
||||
None
|
||||
|
||||
let line = self.span_line(try_stmt.span);
|
||||
let branch_id = self.next_id();
|
||||
let branch_node_id = self.add_node(DagNode {
|
||||
id: branch_id.clone(),
|
||||
node_type: DagNodeType::Branch { condition_source: "try/catch".to_string() },
|
||||
label: "try".to_string(),
|
||||
line,
|
||||
});
|
||||
|
||||
let mut last_ids = Vec::new();
|
||||
|
||||
// Try body
|
||||
if let Some((try_first, try_last)) = self.walk_body(&try_stmt.block.stmts) {
|
||||
self.add_edge(&branch_node_id, &try_first, Some("try".to_string()));
|
||||
last_ids.push(try_last);
|
||||
} else {
|
||||
last_ids.push(branch_node_id.clone());
|
||||
}
|
||||
|
||||
// Catch body
|
||||
if let Some(handler) = &try_stmt.handler {
|
||||
if let Some((catch_first, catch_last)) = self.walk_body(&handler.body.stmts) {
|
||||
self.add_edge(&branch_node_id, &catch_first, Some("catch".to_string()));
|
||||
last_ids.push(catch_last);
|
||||
}
|
||||
}
|
||||
|
||||
// Finally body — sequential after merge
|
||||
let merge_last = if last_ids.len() == 1 {
|
||||
last_ids.into_iter().next().unwrap()
|
||||
} else {
|
||||
let merge_id = format!("{branch_id}_merge");
|
||||
let merge_node_id = self.add_node(DagNode {
|
||||
id: merge_id,
|
||||
node_type: DagNodeType::Merge,
|
||||
label: "merge".to_string(),
|
||||
line,
|
||||
});
|
||||
for last in last_ids {
|
||||
self.add_edge(&last, &merge_node_id, None);
|
||||
}
|
||||
merge_node_id
|
||||
};
|
||||
|
||||
if let Some(finalizer) = &try_stmt.finalizer {
|
||||
if let Some((finally_first, finally_last)) = self.walk_body(&finalizer.stmts) {
|
||||
self.add_edge(&merge_last, &finally_first, None);
|
||||
return Some((branch_node_id, finally_last));
|
||||
}
|
||||
}
|
||||
|
||||
Some((branch_node_id, merge_last))
|
||||
}
|
||||
|
||||
fn walk_return(&mut self, ret: &ReturnStmt) -> Option<(String, String)> {
|
||||
@@ -632,6 +756,19 @@ pub fn parse_ts_workflow(code: &str) -> Result<WorkflowDag, Vec<CompileError>> {
|
||||
}
|
||||
}
|
||||
}
|
||||
// export const main = workflow(async (...) => { ... })
|
||||
if let ModuleItem::ModuleDecl(ModuleDecl::ExportDecl(export)) = item {
|
||||
if let Decl::Var(var_decl) = &export.decl {
|
||||
for decl in &var_decl.decls {
|
||||
if let Some(init) = &decl.init {
|
||||
if let Some(result) = find_workflow_call(init, &cm) {
|
||||
workflow_body = Some(result);
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let (stmts, params) = workflow_body.ok_or_else(|| {
|
||||
|
||||
@@ -12,23 +12,6 @@ impl std::fmt::Display for CompileError {
|
||||
}
|
||||
}
|
||||
|
||||
pub fn error_step_in_try(line: usize) -> CompileError {
|
||||
CompileError {
|
||||
message:
|
||||
"Task calls inside try/except are not allowed. Steps have built-in error handling."
|
||||
.to_string(),
|
||||
line,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn error_step_in_while(line: usize) -> CompileError {
|
||||
CompileError {
|
||||
message: "Task calls inside while loops are not allowed. Use for loops instead."
|
||||
.to_string(),
|
||||
line,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn error_step_in_nested_function(line: usize) -> CompileError {
|
||||
CompileError {
|
||||
message: "Task calls inside nested functions, closures, or lambdas are not allowed."
|
||||
@@ -53,12 +36,3 @@ pub fn error_missing_await(line: usize) -> CompileError {
|
||||
line,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn error_step_in_catch(line: usize) -> CompileError {
|
||||
CompileError {
|
||||
message:
|
||||
"Task calls inside catch blocks are not allowed. Steps have built-in error handling."
|
||||
.to_string(),
|
||||
line,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -147,7 +147,7 @@ async def my_etl(items: list):
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_reject_step_in_try() {
|
||||
fn test_step_in_try_except() {
|
||||
let code = r#"
|
||||
import asyncio
|
||||
from wmill import workflow, task
|
||||
@@ -155,39 +155,52 @@ from wmill import workflow, task
|
||||
@task
|
||||
async def extract_data(): ...
|
||||
|
||||
@task
|
||||
async def handle_error(): ...
|
||||
|
||||
@workflow
|
||||
async def my_etl():
|
||||
try:
|
||||
await extract_data()
|
||||
except Exception:
|
||||
pass
|
||||
await handle_error()
|
||||
"#;
|
||||
|
||||
let result = parse_python_workflow(code);
|
||||
assert!(result.is_err());
|
||||
let errors = result.unwrap_err();
|
||||
assert!(errors[0].message.contains("try/except"));
|
||||
let dag = parse_python_workflow(code).expect("should parse try/except");
|
||||
// Branch(try/except), extract_data, handle_error, merge = 4
|
||||
assert_eq!(dag.nodes.len(), 4);
|
||||
assert!(matches!(dag.nodes[0].node_type, DagNodeType::Branch { .. }));
|
||||
assert_eq!(dag.nodes[0].label, "try");
|
||||
assert!(matches!(dag.nodes[1].node_type, DagNodeType::Step { .. }));
|
||||
assert!(matches!(dag.nodes[2].node_type, DagNodeType::Step { .. }));
|
||||
assert!(matches!(dag.nodes[3].node_type, DagNodeType::Merge));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_reject_step_in_while() {
|
||||
fn test_step_in_while() {
|
||||
let code = r#"
|
||||
import asyncio
|
||||
from wmill import workflow, task
|
||||
|
||||
@task
|
||||
async def extract_data(): ...
|
||||
async def poll_status(): ...
|
||||
|
||||
@workflow
|
||||
async def my_etl():
|
||||
while True:
|
||||
await extract_data()
|
||||
await poll_status()
|
||||
"#;
|
||||
|
||||
let result = parse_python_workflow(code);
|
||||
assert!(result.is_err());
|
||||
let errors = result.unwrap_err();
|
||||
assert!(errors[0].message.contains("while"));
|
||||
let dag = parse_python_workflow(code).expect("should parse while loop");
|
||||
// LoopStart, poll_status, LoopEnd = 3
|
||||
assert_eq!(dag.nodes.len(), 3);
|
||||
assert!(matches!(
|
||||
dag.nodes[0].node_type,
|
||||
DagNodeType::LoopStart { .. }
|
||||
));
|
||||
assert_eq!(dag.nodes[0].label, "while");
|
||||
assert!(matches!(dag.nodes[1].node_type, DagNodeType::Step { .. }));
|
||||
assert!(matches!(dag.nodes[2].node_type, DagNodeType::LoopEnd));
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -264,3 +277,89 @@ async def my_wf(x: int):
|
||||
_ => panic!("expected Step node"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_task_script_and_task_flow_py() {
|
||||
let code = r#"
|
||||
from wmill import workflow, task, task_script, task_flow
|
||||
|
||||
helper = task_script("./helper.py")
|
||||
pipeline = task_flow("f/etl/pipeline")
|
||||
|
||||
@task()
|
||||
async def process(x: str) -> str:
|
||||
return f"processed: {x}"
|
||||
|
||||
@workflow
|
||||
async def main(x: str):
|
||||
a = await process(x=x)
|
||||
b = await helper(a=a)
|
||||
c = await pipeline(b=b)
|
||||
return {"a": a, "b": b, "c": c}
|
||||
"#;
|
||||
|
||||
let dag = parse_python_workflow(code).expect("should parse");
|
||||
assert_eq!(dag.nodes.len(), 4); // 3 steps + 1 return
|
||||
|
||||
match &dag.nodes[1].node_type {
|
||||
DagNodeType::Step { name, script } => {
|
||||
assert_eq!(name, "helper");
|
||||
assert_eq!(script, "./helper.py");
|
||||
}
|
||||
_ => panic!("expected Step node for task_script"),
|
||||
}
|
||||
|
||||
match &dag.nodes[2].node_type {
|
||||
DagNodeType::Step { name, script } => {
|
||||
assert_eq!(name, "pipeline");
|
||||
assert_eq!(script, "f/etl/pipeline");
|
||||
}
|
||||
_ => panic!("expected Step node for task_flow"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_full_template_with_sdk_calls_py() {
|
||||
let code = r#"
|
||||
from wmill import workflow, task, task_script, step, sleep, wait_for_approval, get_resume_urls
|
||||
|
||||
helper = task_script("./helper.py")
|
||||
|
||||
@task()
|
||||
async def process(x: str) -> str:
|
||||
return f"processed: {x}"
|
||||
|
||||
@workflow
|
||||
async def main(x: str):
|
||||
a = await process(x=x)
|
||||
b = await helper(a=a)
|
||||
urls = await step("get_urls", lambda: get_resume_urls())
|
||||
await sleep(1)
|
||||
approval = await wait_for_approval(timeout=3600)
|
||||
return {"processed": a, "helper_result": b, "approval": approval}
|
||||
"#;
|
||||
|
||||
let dag = parse_python_workflow(code).expect("should parse");
|
||||
// process, helper, step("get_urls"), sleep(1), wait_for_approval, return = 6
|
||||
assert_eq!(dag.nodes.len(), 6);
|
||||
|
||||
match &dag.nodes[2].node_type {
|
||||
DagNodeType::InlineStep { name } => {
|
||||
assert_eq!(name, "get_urls");
|
||||
}
|
||||
_ => panic!("expected InlineStep node, got {:?}", dag.nodes[2].node_type),
|
||||
}
|
||||
|
||||
match &dag.nodes[3].node_type {
|
||||
DagNodeType::Sleep { seconds } => {
|
||||
assert_eq!(seconds, "1");
|
||||
}
|
||||
_ => panic!("expected Sleep node, got {:?}", dag.nodes[3].node_type),
|
||||
}
|
||||
|
||||
assert!(matches!(
|
||||
dag.nodes[4].node_type,
|
||||
DagNodeType::WaitForApproval
|
||||
));
|
||||
assert!(matches!(dag.nodes[5].node_type, DagNodeType::Return));
|
||||
}
|
||||
|
||||
@@ -129,45 +129,56 @@ export default workflow(async (items: string[]) => {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_reject_step_in_try_catch() {
|
||||
fn test_step_in_try_catch() {
|
||||
let code = r#"
|
||||
import { workflow, task } from "windmill-client";
|
||||
|
||||
const extract_data = task(async () => {});
|
||||
const handle_error = task(async (e: any) => {});
|
||||
|
||||
export default workflow(async () => {
|
||||
try {
|
||||
await extract_data();
|
||||
} catch (e) {
|
||||
console.log(e);
|
||||
await handle_error(e);
|
||||
}
|
||||
});
|
||||
"#;
|
||||
|
||||
let result = parse_ts_workflow(code);
|
||||
assert!(result.is_err());
|
||||
let errors = result.unwrap_err();
|
||||
assert!(errors[0].message.contains("catch"));
|
||||
let dag = parse_ts_workflow(code).expect("should parse try/catch");
|
||||
// Branch(try/catch), extract_data, handle_error, merge = 4
|
||||
assert_eq!(dag.nodes.len(), 4);
|
||||
assert!(matches!(dag.nodes[0].node_type, DagNodeType::Branch { .. }));
|
||||
assert_eq!(dag.nodes[0].label, "try");
|
||||
assert!(matches!(dag.nodes[1].node_type, DagNodeType::Step { .. }));
|
||||
assert!(matches!(dag.nodes[2].node_type, DagNodeType::Step { .. }));
|
||||
assert!(matches!(dag.nodes[3].node_type, DagNodeType::Merge));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_reject_step_in_while_ts() {
|
||||
fn test_step_in_while_ts() {
|
||||
let code = r#"
|
||||
import { workflow, task } from "windmill-client";
|
||||
|
||||
const extract_data = task(async () => {});
|
||||
const poll_status = task(async () => {});
|
||||
|
||||
export default workflow(async () => {
|
||||
while (true) {
|
||||
await extract_data();
|
||||
await poll_status();
|
||||
}
|
||||
});
|
||||
"#;
|
||||
|
||||
let result = parse_ts_workflow(code);
|
||||
assert!(result.is_err());
|
||||
let errors = result.unwrap_err();
|
||||
assert!(errors[0].message.contains("while"));
|
||||
let dag = parse_ts_workflow(code).expect("should parse while loop");
|
||||
// LoopStart, poll_status, LoopEnd = 3
|
||||
assert_eq!(dag.nodes.len(), 3);
|
||||
assert!(matches!(
|
||||
dag.nodes[0].node_type,
|
||||
DagNodeType::LoopStart { .. }
|
||||
));
|
||||
assert_eq!(dag.nodes[0].label, "while");
|
||||
assert!(matches!(dag.nodes[1].node_type, DagNodeType::Step { .. }));
|
||||
assert!(matches!(dag.nodes[2].node_type, DagNodeType::LoopEnd));
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -243,3 +254,158 @@ export default workflow(async (x: number) => {
|
||||
_ => panic!("expected Step node"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_task_script_and_task_flow() {
|
||||
let code = r#"
|
||||
import { workflow, task, taskScript, taskFlow } from "windmill-client";
|
||||
|
||||
const helper = taskScript("./helper.ts");
|
||||
const pipeline = taskFlow("f/etl/pipeline");
|
||||
const process = task(async (x: string) => {});
|
||||
|
||||
export default workflow(async (x: string) => {
|
||||
const a = await process(x);
|
||||
const b = await helper({ a });
|
||||
const c = await pipeline({ b });
|
||||
return { a, b, c };
|
||||
});
|
||||
"#;
|
||||
|
||||
let dag = parse_ts_workflow(code).expect("should parse");
|
||||
assert_eq!(dag.nodes.len(), 4); // 3 steps + 1 return
|
||||
|
||||
match &dag.nodes[0].node_type {
|
||||
DagNodeType::Step { name, script } => {
|
||||
assert_eq!(name, "process");
|
||||
assert_eq!(script, "process");
|
||||
}
|
||||
_ => panic!("expected Step node"),
|
||||
}
|
||||
|
||||
match &dag.nodes[1].node_type {
|
||||
DagNodeType::Step { name, script } => {
|
||||
assert_eq!(name, "helper");
|
||||
assert_eq!(script, "./helper.ts");
|
||||
}
|
||||
_ => panic!("expected Step node for taskScript"),
|
||||
}
|
||||
|
||||
match &dag.nodes[2].node_type {
|
||||
DagNodeType::Step { name, script } => {
|
||||
assert_eq!(name, "pipeline");
|
||||
assert_eq!(script, "f/etl/pipeline");
|
||||
}
|
||||
_ => panic!("expected Step node for taskFlow"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_full_template_with_sdk_calls() {
|
||||
let code = r#"
|
||||
import { task, taskScript, step, sleep, waitForApproval, getResumeUrls, workflow } from "windmill-client";
|
||||
|
||||
const helper = taskScript("./helper.ts");
|
||||
const process = task(async (x: string): Promise<string> => {
|
||||
return `processed: ${x}`;
|
||||
});
|
||||
|
||||
export const main = workflow(async (x: string) => {
|
||||
const a = await process(x);
|
||||
const b = await helper({ a });
|
||||
const urls = await step("get_urls", () => getResumeUrls());
|
||||
await sleep(1);
|
||||
const approval = await waitForApproval({ timeout: 3600 });
|
||||
return { processed: a, helper_result: b, approval };
|
||||
});
|
||||
"#;
|
||||
|
||||
let dag = parse_ts_workflow(code).expect("should parse");
|
||||
// process, helper, step("get_urls"), sleep(1), waitForApproval, return = 6
|
||||
assert_eq!(dag.nodes.len(), 6);
|
||||
assert_eq!(dag.edges.len(), 5);
|
||||
|
||||
assert!(matches!(dag.nodes[0].node_type, DagNodeType::Step { .. }));
|
||||
|
||||
match &dag.nodes[1].node_type {
|
||||
DagNodeType::Step { name, script } => {
|
||||
assert_eq!(name, "helper");
|
||||
assert_eq!(script, "./helper.ts");
|
||||
}
|
||||
_ => panic!("expected Step node"),
|
||||
}
|
||||
|
||||
match &dag.nodes[2].node_type {
|
||||
DagNodeType::InlineStep { name } => {
|
||||
assert_eq!(name, "get_urls");
|
||||
}
|
||||
_ => panic!("expected InlineStep node, got {:?}", dag.nodes[2].node_type),
|
||||
}
|
||||
|
||||
match &dag.nodes[3].node_type {
|
||||
DagNodeType::Sleep { seconds } => {
|
||||
assert_eq!(seconds, "1");
|
||||
}
|
||||
_ => panic!("expected Sleep node, got {:?}", dag.nodes[3].node_type),
|
||||
}
|
||||
|
||||
assert!(matches!(
|
||||
dag.nodes[4].node_type,
|
||||
DagNodeType::WaitForApproval
|
||||
));
|
||||
assert!(matches!(dag.nodes[5].node_type, DagNodeType::Return));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_complex_mixed_workflow() {
|
||||
let code = r#"
|
||||
import { workflow, task, step, sleep } from "windmill-client";
|
||||
|
||||
const validate = task(async (data: any) => {});
|
||||
const process_csv = task(async (data: any) => {});
|
||||
const process_json = task(async (data: any) => {});
|
||||
const enrich = task(async (item: any) => {});
|
||||
const store = task(async (data: any) => {});
|
||||
|
||||
export default workflow(async (data: any) => {
|
||||
const validated = await validate(data);
|
||||
if (validated.format === "csv") {
|
||||
const parsed = await process_csv(validated);
|
||||
for (const row of parsed.rows) {
|
||||
await enrich(row);
|
||||
}
|
||||
} else {
|
||||
await process_json(validated);
|
||||
}
|
||||
await sleep(5);
|
||||
const ts = await step("timestamp", () => new Date().toISOString());
|
||||
await store(validated);
|
||||
return { done: true };
|
||||
});
|
||||
"#;
|
||||
|
||||
let dag = parse_ts_workflow(code).expect("should parse");
|
||||
|
||||
// validate, Branch, process_csv, LoopStart, enrich, LoopEnd, process_json,
|
||||
// merge, sleep(5), step("timestamp"), store, return = 12
|
||||
assert_eq!(dag.nodes.len(), 12);
|
||||
|
||||
assert!(matches!(dag.nodes[0].node_type, DagNodeType::Step { .. }));
|
||||
assert!(matches!(dag.nodes[1].node_type, DagNodeType::Branch { .. }));
|
||||
assert!(matches!(dag.nodes[2].node_type, DagNodeType::Step { .. })); // process_csv
|
||||
assert!(matches!(
|
||||
dag.nodes[3].node_type,
|
||||
DagNodeType::LoopStart { .. }
|
||||
));
|
||||
assert!(matches!(dag.nodes[4].node_type, DagNodeType::Step { .. })); // enrich
|
||||
assert!(matches!(dag.nodes[5].node_type, DagNodeType::LoopEnd));
|
||||
assert!(matches!(dag.nodes[6].node_type, DagNodeType::Step { .. })); // process_json
|
||||
assert!(matches!(dag.nodes[7].node_type, DagNodeType::Merge));
|
||||
assert!(matches!(dag.nodes[8].node_type, DagNodeType::Sleep { .. }));
|
||||
assert!(matches!(
|
||||
dag.nodes[9].node_type,
|
||||
DagNodeType::InlineStep { .. }
|
||||
)); // timestamp
|
||||
assert!(matches!(dag.nodes[10].node_type, DagNodeType::Step { .. })); // store
|
||||
assert!(matches!(dag.nodes[11].node_type, DagNodeType::Return));
|
||||
}
|
||||
|
||||
@@ -39,3 +39,6 @@ popd
|
||||
|
||||
pushd "pkg-py-imports" && npm publish ${args}
|
||||
popd
|
||||
|
||||
pushd "pkg-wac" && npm publish ${args}
|
||||
popd
|
||||
|
||||
@@ -1906,6 +1906,21 @@ async fn process_notify_event(
|
||||
RESTART_COORDINATION_SETTING => {
|
||||
// Internal coordination key for staggered restarts, no action needed
|
||||
}
|
||||
"plain_emails_telemetry" => {
|
||||
let enabled = sqlx::query_scalar!(
|
||||
"SELECT value FROM global_settings WHERE name = 'plain_emails_telemetry'"
|
||||
)
|
||||
.fetch_optional(db)
|
||||
.await
|
||||
.ok()
|
||||
.flatten()
|
||||
.and_then(|v| v.as_bool())
|
||||
.unwrap_or(false);
|
||||
tracing::info!(
|
||||
"Plain emails telemetry setting changed: enabled={}",
|
||||
enabled
|
||||
);
|
||||
}
|
||||
_ => {
|
||||
tracing::info!("Unrecognized Global Setting Change Payload: {:?}", payload);
|
||||
}
|
||||
|
||||
@@ -442,9 +442,22 @@ pub fn check_route_access(
|
||||
// Find the domain and kind for this route
|
||||
let (required_domain, required_kind, route_suffix) = extract_domain_from_route(route_path)?;
|
||||
|
||||
// Backward compatibility: MCP handlers expect unusual scope actions: all, favorites, hub.
|
||||
// MCP scopes (mcp:all, mcp:favorites, mcp:hub:*, etc.) use a custom format
|
||||
// that doesn't fit the standard domain:action model. Verify the token has at
|
||||
// least one mcp: scope; MCP handlers do their own fine-grained checking.
|
||||
if required_domain == ScopeDomain::Mcp {
|
||||
return Ok(());
|
||||
let is_scoped_token = token_scopes
|
||||
.iter()
|
||||
.any(|s| !s.starts_with("if_jobs:filter_tags:"));
|
||||
if !is_scoped_token {
|
||||
return Ok(());
|
||||
}
|
||||
if token_scopes.iter().any(|s| s.starts_with("mcp:")) {
|
||||
return Ok(());
|
||||
}
|
||||
return Err(Error::NotAuthorized(
|
||||
"Access denied. Required scope: mcp:*".to_string(),
|
||||
));
|
||||
}
|
||||
|
||||
// tracing::error!("Checking route access {:?} {:?} {:?} {:?}", required_action, required_domain, required_kind, route_suffix);
|
||||
@@ -931,4 +944,50 @@ mod tests {
|
||||
ScopeDefinition::new("scripts", "read", None, Some(vec!["u/*".to_string()]));
|
||||
assert!(scope_specific_path.includes(&required_broad));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_mcp_scope_bypass_blocked_without_mcp_scope() {
|
||||
// A token with only jobs:read should NOT be able to access MCP endpoints
|
||||
let scopes = vec!["jobs:read".to_string()];
|
||||
assert!(check_route_access(&scopes, "/api/w/test_workspace/mcp/something", "GET").is_err());
|
||||
assert!(
|
||||
check_route_access(&scopes, "/api/w/test_workspace/mcp/something", "POST").is_err()
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_mcp_scope_allowed_with_mcp_scope() {
|
||||
// A token with mcp:all should access MCP endpoints
|
||||
let scopes = vec!["mcp:all".to_string()];
|
||||
assert!(check_route_access(&scopes, "/api/w/test_workspace/mcp/something", "GET").is_ok());
|
||||
|
||||
// mcp:favorites should also work
|
||||
let scopes = vec!["mcp:favorites".to_string()];
|
||||
assert!(check_route_access(&scopes, "/api/w/test_workspace/mcp/something", "POST").is_ok());
|
||||
|
||||
// mcp:scripts:path should also work
|
||||
let scopes = vec!["mcp:scripts:u/admin/script1".to_string()];
|
||||
assert!(check_route_access(&scopes, "/api/w/test_workspace/mcp/something", "GET").is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_mcp_scope_filter_tags_only_treated_as_unrestricted() {
|
||||
// Token with only filter_tags is not considered scoped — should be allowed
|
||||
let scopes = vec!["if_jobs:filter_tags:tag1".to_string()];
|
||||
assert!(check_route_access(&scopes, "/api/w/test_workspace/mcp/something", "GET").is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_mcp_scope_mixed_scopes_without_mcp() {
|
||||
// Token with multiple non-MCP scopes should be denied
|
||||
let scopes = vec!["jobs:read".to_string(), "scripts:write".to_string()];
|
||||
assert!(check_route_access(&scopes, "/api/w/test_workspace/mcp/something", "GET").is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_mcp_scope_mixed_scopes_with_mcp() {
|
||||
// Token with MCP scope + other scopes should be allowed for MCP
|
||||
let scopes = vec!["jobs:read".to_string(), "mcp:all".to_string()];
|
||||
assert!(check_route_access(&scopes, "/api/w/test_workspace/mcp/something", "GET").is_ok());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -851,9 +851,21 @@ async fn add_user_igroup(
|
||||
#[cfg(all(feature = "private", feature = "enterprise"))]
|
||||
{
|
||||
use windmill_api_workspaces::workspaces_ee::auto_add_user;
|
||||
use windmill_common::users::compute_highest_workspace_role;
|
||||
|
||||
// Find all instance groups this user belongs to (includes the newly added group)
|
||||
let user_igroups: Vec<String> = sqlx::query_scalar!(
|
||||
"SELECT igroup FROM email_to_igroup WHERE email = $1",
|
||||
&email
|
||||
)
|
||||
.fetch_all(&mut *tx)
|
||||
.await?;
|
||||
|
||||
let workspaces = sqlx::query!(
|
||||
r#"
|
||||
SELECT workspace_id, auto_invite->'instance_groups_roles' as instance_groups_roles
|
||||
SELECT workspace_id,
|
||||
auto_invite->'instance_groups_roles' as instance_groups_roles,
|
||||
auto_invite->'instance_groups' as instance_groups_json
|
||||
FROM workspace_settings
|
||||
WHERE auto_invite->'instance_groups' ? $1
|
||||
"#,
|
||||
@@ -861,34 +873,53 @@ async fn add_user_igroup(
|
||||
)
|
||||
.fetch_all(&mut *tx)
|
||||
.await?;
|
||||
|
||||
for ws in workspaces {
|
||||
let role = ws
|
||||
let roles: std::collections::HashMap<String, String> = ws
|
||||
.instance_groups_roles
|
||||
.and_then(|r| r.get(&name).and_then(|v| v.as_str().map(String::from)))
|
||||
.unwrap_or_else(|| "developer".to_string());
|
||||
let (is_admin, is_operator) = match role.as_str() {
|
||||
"admin" => (true, false),
|
||||
"operator" => (false, true),
|
||||
_ => (false, false),
|
||||
};
|
||||
.and_then(|r| serde_json::from_value(r).ok())
|
||||
.unwrap_or_default();
|
||||
|
||||
let ws_configured_groups: Vec<String> = ws
|
||||
.instance_groups_json
|
||||
.and_then(|ig| serde_json::from_value(ig).ok())
|
||||
.unwrap_or_default();
|
||||
|
||||
let (best_group, is_admin, is_operator) =
|
||||
compute_highest_workspace_role(&user_igroups, &ws_configured_groups, &roles);
|
||||
|
||||
let instance_group_source = serde_json::json!({
|
||||
"source": "instance_group",
|
||||
"group": &best_group
|
||||
});
|
||||
|
||||
// auto_add_user creates the user if they don't exist (ON CONFLICT DO NOTHING).
|
||||
// The operator flag here doesn't matter for the final state — the UPDATE below
|
||||
// always sets the correct is_admin/operator based on the highest-precedence role.
|
||||
auto_add_user(
|
||||
&email,
|
||||
&ws.workspace_id,
|
||||
&is_operator,
|
||||
&false,
|
||||
&mut tx,
|
||||
&authed,
|
||||
Some(serde_json::json!({"source": "instance_group", "group": &name})),
|
||||
Some(instance_group_source.clone()),
|
||||
)
|
||||
.await?;
|
||||
if is_admin {
|
||||
sqlx::query!(
|
||||
"UPDATE usr SET is_admin = true WHERE workspace_id = $1 AND email = $2",
|
||||
&ws.workspace_id,
|
||||
&email
|
||||
)
|
||||
.execute(&mut *tx)
|
||||
.await?;
|
||||
}
|
||||
|
||||
// Set the correct role based on highest precedence across all groups.
|
||||
// For new users, auto_add_user already stored added_via with source=instance_group,
|
||||
// so this UPDATE will match. For existing instance_group users, it upgrades/corrects
|
||||
// the role. Manually-added users (added_via is NULL or non-instance_group) are not affected.
|
||||
sqlx::query!(
|
||||
"UPDATE usr SET is_admin = $1, operator = $2, added_via = $3 WHERE workspace_id = $4 AND email = $5 AND added_via->>'source' = 'instance_group'",
|
||||
is_admin,
|
||||
is_operator,
|
||||
&instance_group_source,
|
||||
&ws.workspace_id,
|
||||
&email
|
||||
)
|
||||
.execute(&mut *tx)
|
||||
.await?;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -26,6 +26,7 @@ use windmill_common::{
|
||||
jobs::JobKind,
|
||||
scripts::to_i64,
|
||||
utils::{not_found_if_none, paginate, Pagination},
|
||||
worker::CLOUD_HOSTED,
|
||||
};
|
||||
pub fn workspaced_service() -> Router {
|
||||
Router::new()
|
||||
@@ -134,11 +135,20 @@ async fn get_input_history(
|
||||
Query(g): Query<GetInputHistory>,
|
||||
) -> JsonResult<Vec<Input>> {
|
||||
let (per_page, offset) = paginate(pagination);
|
||||
let per_page = if *CLOUD_HOSTED {
|
||||
per_page.min(100)
|
||||
} else {
|
||||
per_page
|
||||
};
|
||||
|
||||
let mut tx = user_db.begin(&authed).await?;
|
||||
|
||||
let args_query = if let Some(args) = &g.args {
|
||||
sql_builder::bind::Bind::bind(&"and v2_job.args @> ?", &args.replace("'", "''"))
|
||||
if let Ok(v) = serde_json::from_str::<serde_json::Value>(args) {
|
||||
sql_builder::bind::Bind::bind(&"and v2_job.args @> ?", &v.to_string())
|
||||
} else {
|
||||
"AND FALSE".to_string()
|
||||
}
|
||||
} else {
|
||||
"".to_string()
|
||||
};
|
||||
|
||||
@@ -13,7 +13,7 @@ default = []
|
||||
private = ["windmill-test-utils/private", "dep:aws-config", "dep:aws-credential-types", "dep:aws-sdk-sqs", "windmill-git-sync/private"]
|
||||
enterprise = ["windmill-test-utils/enterprise", "dep:base64", "windmill-git-sync/enterprise"]
|
||||
deno_core = ["windmill-test-utils/deno_core"]
|
||||
mcp = []
|
||||
mcp = ["windmill-test-utils/mcp", "dep:rmcp"]
|
||||
run_inline = ["dep:windmill-worker", "windmill-test-utils/run_inline", "windmill-test-utils/duckdb"]
|
||||
|
||||
[dependencies]
|
||||
@@ -41,3 +41,4 @@ aws-credential-types = { workspace = true, optional = true }
|
||||
aws-sdk-sqs = { workspace = true, optional = true }
|
||||
base64 = { workspace = true, optional = true }
|
||||
axum.workspace = true
|
||||
rmcp = { workspace = true, optional = true }
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
use serde_json::json;
|
||||
use sqlx::{Pool, Postgres};
|
||||
use uuid::Uuid;
|
||||
use windmill_common::variables::generate_approval_token;
|
||||
use windmill_test_utils::*;
|
||||
|
||||
fn client() -> reqwest::Client {
|
||||
@@ -248,3 +249,245 @@ async fn test_jobs_unauthed_complex_reachability(db: Pool<Postgres>) -> anyhow::
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Build a minimal FlowValue JSON with a suspend step (with resume_form) followed by an identity
|
||||
/// step. The suspend step is at index 0, the identity step at index 1. After the suspend step
|
||||
/// completes, `flow_status.step` = 1 and `approval_step = 0` points to the form.
|
||||
fn flow_value_with_suspend_form() -> serde_json::Value {
|
||||
json!({
|
||||
"modules": [
|
||||
{
|
||||
"id": "a",
|
||||
"value": {"type": "identity"},
|
||||
"suspend": {
|
||||
"required_events": 1,
|
||||
"resume_form": {
|
||||
"schema": {
|
||||
"properties": {
|
||||
"reason": {"type": "string", "description": "Approval reason"}
|
||||
},
|
||||
"order": ["reason"]
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "b",
|
||||
"value": {"type": "identity"}
|
||||
}
|
||||
],
|
||||
"same_worker": false
|
||||
})
|
||||
}
|
||||
|
||||
/// Build the flow_status JSON for a flow suspended at step 1 (step 0 completed with suspend).
|
||||
fn flow_status_suspended_at_step_1(step_job_id: Uuid) -> serde_json::Value {
|
||||
json!({
|
||||
"step": 1,
|
||||
"modules": [
|
||||
{"type": "Success", "id": "a", "job": step_job_id, "skipped": false},
|
||||
{"type": "WaitingForEvents", "id": "b", "count": 1, "job": step_job_id}
|
||||
],
|
||||
"failure_module": {
|
||||
"parent_module": null,
|
||||
"type": "WaitingForPriorSteps",
|
||||
"id": "failure"
|
||||
},
|
||||
"cleanup_module": {"flow_jobs_to_clean": []}
|
||||
})
|
||||
}
|
||||
|
||||
/// Insert a v2_job_queue + v2_job_status pair (v2_job_status has FK to v2_job_queue).
|
||||
async fn insert_queue_and_status(
|
||||
db: &Pool<Postgres>,
|
||||
flow_id: Uuid,
|
||||
flow_status: &serde_json::Value,
|
||||
) {
|
||||
sqlx::query(
|
||||
"INSERT INTO v2_job_queue (id, workspace_id, scheduled_for)
|
||||
VALUES ($1, 'test-workspace', now())",
|
||||
)
|
||||
.bind(flow_id)
|
||||
.execute(db)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
sqlx::query(
|
||||
"INSERT INTO v2_job_status (id, flow_status)
|
||||
VALUES ($1, $2)",
|
||||
)
|
||||
.bind(flow_id)
|
||||
.bind(flow_status)
|
||||
.execute(db)
|
||||
.await
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
/// Insert a flow job with raw_flow stored in v2_job (RawFlow / FlowPreview path).
|
||||
async fn insert_suspended_flow_with_raw_flow(
|
||||
db: &Pool<Postgres>,
|
||||
raw_flow: &serde_json::Value,
|
||||
) -> Uuid {
|
||||
let flow_id = Uuid::new_v4();
|
||||
let step_job_id = Uuid::new_v4();
|
||||
let flow_status = flow_status_suspended_at_step_1(step_job_id);
|
||||
|
||||
sqlx::query(
|
||||
"INSERT INTO v2_job (id, workspace_id, created_by, permissioned_as, kind, tag, raw_flow)
|
||||
VALUES ($1, 'test-workspace', 'test-user', 'u/test-user', 'flowpreview', 'flow', $2)",
|
||||
)
|
||||
.bind(flow_id)
|
||||
.bind(raw_flow)
|
||||
.execute(db)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
insert_queue_and_status(db, flow_id, &flow_status).await;
|
||||
|
||||
flow_id
|
||||
}
|
||||
|
||||
/// Insert a flow job WITHOUT raw_flow but with a matching flow_node entry (FlowNode path).
|
||||
async fn insert_suspended_flow_node(db: &Pool<Postgres>, flow_value: &serde_json::Value) -> Uuid {
|
||||
let flow_id = Uuid::new_v4();
|
||||
let step_job_id = Uuid::new_v4();
|
||||
let flow_status = flow_status_suspended_at_step_1(step_job_id);
|
||||
|
||||
// Insert the flow_node entry first to get its id
|
||||
let node_id: i64 = sqlx::query_scalar(
|
||||
"INSERT INTO flow_node (workspace_id, hash, path, flow)
|
||||
VALUES ('test-workspace', 12345, 'f/test/flow', $1)
|
||||
RETURNING id",
|
||||
)
|
||||
.bind(flow_value)
|
||||
.fetch_one(db)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
// Insert the job with kind=flownode and runnable_id pointing to the flow_node
|
||||
sqlx::query(
|
||||
"INSERT INTO v2_job (id, workspace_id, created_by, permissioned_as, kind, tag, runnable_id)
|
||||
VALUES ($1, 'test-workspace', 'test-user', 'u/test-user', 'flownode', 'flow', $2)",
|
||||
)
|
||||
.bind(flow_id)
|
||||
.bind(node_id)
|
||||
.execute(db)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
insert_queue_and_status(db, flow_id, &flow_status).await;
|
||||
|
||||
flow_id
|
||||
}
|
||||
|
||||
/// Insert a flow job WITHOUT raw_flow but with runnable_id pointing to flow_node,
|
||||
/// and NO flow_node entry — simulates the broken state before the fix.
|
||||
async fn insert_suspended_flow_node_without_node_entry(db: &Pool<Postgres>) -> Uuid {
|
||||
let flow_id = Uuid::new_v4();
|
||||
let step_job_id = Uuid::new_v4();
|
||||
let flow_status = flow_status_suspended_at_step_1(step_job_id);
|
||||
|
||||
// Use a non-existent runnable_id — simulates FlowNode with raw_flow=NULL and no fallback
|
||||
sqlx::query(
|
||||
"INSERT INTO v2_job (id, workspace_id, created_by, permissioned_as, kind, tag, runnable_id)
|
||||
VALUES ($1, 'test-workspace', 'test-user', 'u/test-user', 'flownode', 'flow', 99999999)",
|
||||
)
|
||||
.bind(flow_id)
|
||||
.execute(db)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
insert_queue_and_status(db, flow_id, &flow_status).await;
|
||||
|
||||
flow_id
|
||||
}
|
||||
|
||||
async fn get_approval_info_response(
|
||||
port: u16,
|
||||
db: &Pool<Postgres>,
|
||||
job_id: Uuid,
|
||||
) -> serde_json::Value {
|
||||
let token = generate_approval_token("test-workspace", job_id, db)
|
||||
.await
|
||||
.unwrap();
|
||||
let base = format!("http://localhost:{port}/api/w/test-workspace/jobs_u");
|
||||
let resp = client()
|
||||
.get(format!("{base}/flow/approval_info/{job_id}?token={token}"))
|
||||
.send()
|
||||
.await
|
||||
.unwrap();
|
||||
let status = resp.status().as_u16();
|
||||
let body = resp.text().await.unwrap();
|
||||
assert!(
|
||||
(200..300).contains(&status),
|
||||
"approval_info returned {status}: {body}",
|
||||
);
|
||||
serde_json::from_str(&body).unwrap()
|
||||
}
|
||||
|
||||
/// Test: approval_info returns form_schema for a top-level suspend (raw_flow stored in v2_job).
|
||||
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
|
||||
async fn test_approval_info_form_schema_from_raw_flow(db: Pool<Postgres>) -> anyhow::Result<()> {
|
||||
initialize_tracing().await;
|
||||
let server = ApiServer::start(db.clone()).await?;
|
||||
let port = server.addr.port();
|
||||
|
||||
let flow_value = flow_value_with_suspend_form();
|
||||
let flow_id = insert_suspended_flow_with_raw_flow(&db, &flow_value).await;
|
||||
|
||||
let info = get_approval_info_response(port, &db, flow_id).await;
|
||||
assert!(
|
||||
info.get("form_schema").is_some(),
|
||||
"form_schema should be present for raw_flow path, got: {info}",
|
||||
);
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Test: approval_info returns form_schema for a FlowNode sub-flow (graph-based branch/loop).
|
||||
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
|
||||
async fn test_approval_info_form_schema_from_flow_node(db: Pool<Postgres>) -> anyhow::Result<()> {
|
||||
initialize_tracing().await;
|
||||
let server = ApiServer::start(db.clone()).await?;
|
||||
let port = server.addr.port();
|
||||
|
||||
// We need a flow path to exist for the flow_node FK
|
||||
sqlx::query(
|
||||
"INSERT INTO flow (workspace_id, path, summary, description, versions, value, edited_by, edited_at, schema)
|
||||
VALUES ('test-workspace', 'f/test/flow', '', '', '{}', '{}'::jsonb, 'test-user', now(), '{}'::jsonb)",
|
||||
)
|
||||
.execute(&db)
|
||||
.await?;
|
||||
|
||||
let flow_value = flow_value_with_suspend_form();
|
||||
let flow_id = insert_suspended_flow_node(&db, &flow_value).await;
|
||||
|
||||
let info = get_approval_info_response(port, &db, flow_id).await;
|
||||
assert!(
|
||||
info.get("form_schema").is_some(),
|
||||
"form_schema should be present for flow_node path, got: {info}",
|
||||
);
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Test: approval_info returns no form_schema when FlowNode has no matching entry
|
||||
/// (simulates the pre-fix behavior where flow_node fallback didn't exist).
|
||||
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
|
||||
async fn test_approval_info_no_form_when_flow_node_missing(
|
||||
db: Pool<Postgres>,
|
||||
) -> anyhow::Result<()> {
|
||||
initialize_tracing().await;
|
||||
let server = ApiServer::start(db.clone()).await?;
|
||||
let port = server.addr.port();
|
||||
|
||||
let flow_id = insert_suspended_flow_node_without_node_entry(&db).await;
|
||||
|
||||
let info = get_approval_info_response(port, &db, flow_id).await;
|
||||
assert!(
|
||||
info.get("form_schema").is_none(),
|
||||
"form_schema should be absent when no flow definition found, got: {info}",
|
||||
);
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
use serde_json::json;
|
||||
use sqlx::{Pool, Postgres};
|
||||
#[cfg(feature = "mcp")]
|
||||
use uuid::Uuid;
|
||||
|
||||
use windmill_test_utils::*;
|
||||
|
||||
@@ -518,3 +520,182 @@ async fn test_mcp_tools(db: Pool<Postgres>) -> anyhow::Result<()> {
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(feature = "mcp")]
|
||||
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
|
||||
async fn test_mcp_endpoint_tools_list(db: Pool<Postgres>) -> anyhow::Result<()> {
|
||||
initialize_tracing().await;
|
||||
let server = ApiServer::start(db.clone()).await?;
|
||||
let port = server.addr.port();
|
||||
|
||||
let resp = authed(client().get(format!(
|
||||
"http://localhost:{port}/api/mcp/w/test-workspace/list_tools"
|
||||
)))
|
||||
.send()
|
||||
.await?;
|
||||
assert_eq!(resp.status(), 200);
|
||||
|
||||
let tools: Vec<serde_json::Value> = resp.json().await?;
|
||||
|
||||
let tool_names: Vec<&str> = tools.iter().filter_map(|t| t["name"].as_str()).collect();
|
||||
|
||||
assert!(
|
||||
tool_names.contains(&"getJob"),
|
||||
"getJob not found in MCP endpoint tools: {tool_names:?}"
|
||||
);
|
||||
assert!(
|
||||
tool_names.contains(&"getJobLogs"),
|
||||
"getJobLogs not found in MCP endpoint tools: {tool_names:?}"
|
||||
);
|
||||
|
||||
// Verify getJob has the expected path and method
|
||||
let get_job_tool = tools.iter().find(|t| t["name"] == "getJob").unwrap();
|
||||
assert_eq!(get_job_tool["path"], "/w/{workspace}/jobs_u/get/{id}");
|
||||
assert_eq!(get_job_tool["method"], "GET");
|
||||
|
||||
// Verify getJobLogs has the expected path and method
|
||||
let get_job_logs_tool = tools.iter().find(|t| t["name"] == "getJobLogs").unwrap();
|
||||
assert_eq!(
|
||||
get_job_logs_tool["path"],
|
||||
"/w/{workspace}/jobs_u/get_logs/{id}"
|
||||
);
|
||||
assert_eq!(get_job_logs_tool["method"], "GET");
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(feature = "mcp")]
|
||||
async fn insert_completed_job_with_logs(db: &Pool<Postgres>) -> Uuid {
|
||||
let id = Uuid::new_v4();
|
||||
sqlx::query(
|
||||
"INSERT INTO v2_job (id, workspace_id, created_by, permissioned_as, kind, tag, args)
|
||||
VALUES ($1, 'test-workspace', 'test-user', 'u/test-user', 'script', 'deno', '{}'::jsonb)",
|
||||
)
|
||||
.bind(id)
|
||||
.execute(db)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
sqlx::query(
|
||||
"INSERT INTO v2_job_completed (id, workspace_id, duration_ms, result, status)
|
||||
VALUES ($1, 'test-workspace', 100, '42'::jsonb, 'success')",
|
||||
)
|
||||
.bind(id)
|
||||
.execute(db)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
sqlx::query(
|
||||
"INSERT INTO job_logs (job_id, workspace_id, logs, log_offset)
|
||||
VALUES ($1, 'test-workspace', 'hello world test log', 0)",
|
||||
)
|
||||
.bind(id)
|
||||
.execute(db)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
id
|
||||
}
|
||||
|
||||
#[cfg(feature = "mcp")]
|
||||
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
|
||||
async fn test_mcp_client_get_job_and_logs(db: Pool<Postgres>) -> anyhow::Result<()> {
|
||||
use rmcp::model::{
|
||||
CallToolRequestParams, ClientCapabilities, ClientInfo, Implementation,
|
||||
InitializeRequestParams,
|
||||
};
|
||||
use rmcp::service::{RoleClient, RunningService};
|
||||
use rmcp::transport::streamable_http_client::{
|
||||
StreamableHttpClientTransport, StreamableHttpClientTransportConfig,
|
||||
};
|
||||
use rmcp::ServiceExt;
|
||||
|
||||
initialize_tracing().await;
|
||||
set_jwt_secret().await;
|
||||
let server = ApiServer::start_mcp(db.clone()).await?;
|
||||
let port = server.addr.port();
|
||||
|
||||
let job_id = insert_completed_job_with_logs(&db).await;
|
||||
|
||||
// Create a token with MCP scopes
|
||||
sqlx::query(
|
||||
"INSERT INTO token (token_hash, token_prefix, token, email, label, super_admin, scopes)
|
||||
VALUES (encode(sha256('MCP_TOKEN'::bytea), 'hex'), 'MCP_TOK', 'MCP_TOKEN', 'test@windmill.dev', 'mcp token', true, ARRAY['mcp:all'])",
|
||||
)
|
||||
.execute(&db)
|
||||
.await?;
|
||||
|
||||
// Connect as MCP client
|
||||
let config = StreamableHttpClientTransportConfig::with_uri(format!(
|
||||
"http://localhost:{port}/api/mcp/w/test-workspace/mcp"
|
||||
))
|
||||
.auth_header("MCP_TOKEN");
|
||||
let transport = StreamableHttpClientTransport::from_config(config);
|
||||
|
||||
let client_info = ClientInfo {
|
||||
protocol_version: Default::default(),
|
||||
capabilities: ClientCapabilities::default(),
|
||||
client_info: Implementation {
|
||||
name: "test-client".to_string(),
|
||||
title: None,
|
||||
version: "0.0.1".to_string(),
|
||||
description: None,
|
||||
website_url: None,
|
||||
icons: None,
|
||||
},
|
||||
meta: None,
|
||||
};
|
||||
|
||||
let client: RunningService<RoleClient, InitializeRequestParams> =
|
||||
client_info.serve(transport).await?;
|
||||
|
||||
// --- Test getJob ---
|
||||
let result = client
|
||||
.call_tool(CallToolRequestParams {
|
||||
name: "getJob".into(),
|
||||
arguments: Some(serde_json::from_value(json!({ "id": job_id.to_string() }))?),
|
||||
task: None,
|
||||
meta: None,
|
||||
})
|
||||
.await?;
|
||||
|
||||
let text = result
|
||||
.content
|
||||
.first()
|
||||
.and_then(|c| c.raw.as_text())
|
||||
.expect("getJob should return text content");
|
||||
let job: serde_json::Value = serde_json::from_str(&text.text)?;
|
||||
assert_eq!(job["id"], job_id.to_string());
|
||||
assert_eq!(job["workspace_id"], "test-workspace");
|
||||
assert_eq!(job["created_by"], "test-user");
|
||||
assert_eq!(job["job_kind"], "script");
|
||||
assert!(
|
||||
job["success"].as_bool().unwrap_or(false),
|
||||
"job should be successful: {job}"
|
||||
);
|
||||
|
||||
// --- Test getJobLogs ---
|
||||
let result = client
|
||||
.call_tool(CallToolRequestParams {
|
||||
name: "getJobLogs".into(),
|
||||
arguments: Some(serde_json::from_value(json!({ "id": job_id.to_string() }))?),
|
||||
task: None,
|
||||
meta: None,
|
||||
})
|
||||
.await?;
|
||||
|
||||
let text = result
|
||||
.content
|
||||
.first()
|
||||
.and_then(|c| c.raw.as_text())
|
||||
.expect("getJobLogs should return text content");
|
||||
// The logs endpoint returns text/plain, which gets wrapped as a JSON string by call_endpoint
|
||||
let logs: String = serde_json::from_str(&text.text)?;
|
||||
assert!(
|
||||
logs.contains("hello world test log"),
|
||||
"expected logs to contain test log, got: {logs}"
|
||||
);
|
||||
|
||||
client.cancel().await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -215,8 +215,8 @@ export async function main() {
|
||||
"scenario 1: secret value leaked in logs\nLogs:\n{logs1}"
|
||||
);
|
||||
assert!(
|
||||
logs1.contains("The secret value is: alp*****"),
|
||||
"scenario 1: expected masked output with first 3 chars\nLogs:\n{logs1}"
|
||||
logs1.contains("The secret value is: alp*****k2m"),
|
||||
"scenario 1: expected masked output with first 3 + last 3 chars\nLogs:\n{logs1}"
|
||||
);
|
||||
assert!(
|
||||
logs1.contains("[windmill] secret value was masked for security reasons, use string transformations to display full value"),
|
||||
@@ -277,11 +277,11 @@ export async function main() {
|
||||
"scenario 3: secret2 leaked\nLogs:\n{logs3}"
|
||||
);
|
||||
assert!(
|
||||
logs3.contains("secret1=alp*****"),
|
||||
logs3.contains("secret1=alp*****k2m"),
|
||||
"scenario 3: secret1 not masked\nLogs:\n{logs3}"
|
||||
);
|
||||
assert!(
|
||||
logs3.contains("secret2=bet*****"),
|
||||
logs3.contains("secret2=bet*****n3p"),
|
||||
"scenario 3: secret2 not masked\nLogs:\n{logs3}"
|
||||
);
|
||||
|
||||
@@ -309,7 +309,7 @@ export async function main() {
|
||||
"scenario 4: secret leaked mid-string\nLogs:\n{logs4}"
|
||||
);
|
||||
assert!(
|
||||
logs4.contains("token=alp*****&user=bob&format=json"),
|
||||
logs4.contains("token=alp*****k2m&user=bob&format=json"),
|
||||
"scenario 4: mid-string masking failed\nLogs:\n{logs4}"
|
||||
);
|
||||
|
||||
@@ -338,7 +338,7 @@ export async function main() {
|
||||
!logs5.contains(secret2),
|
||||
"scenario 5: secret leaked\nLogs:\n{logs5}"
|
||||
);
|
||||
let mask_count = logs5.matches("bet*****").count();
|
||||
let mask_count = logs5.matches("bet*****n3p").count();
|
||||
assert!(
|
||||
mask_count >= 3,
|
||||
"scenario 5: expected >= 3 masked occurrences, found {mask_count}\nLogs:\n{logs5}"
|
||||
@@ -374,7 +374,7 @@ export async function main() {
|
||||
"scenario 6: encrypted password leaked\nLogs:\n{logs6}"
|
||||
);
|
||||
assert!(
|
||||
logs6.contains("password is: enc*****"),
|
||||
logs6.contains("password is: enc*****q5r"),
|
||||
"scenario 6: encrypted password not masked\nLogs:\n{logs6}"
|
||||
);
|
||||
|
||||
@@ -403,7 +403,7 @@ export async function main() {
|
||||
"scenario 7: resource secret leaked\nLogs:\n{logs7}"
|
||||
);
|
||||
assert!(
|
||||
logs7.contains("db password: res*****"),
|
||||
logs7.contains("db password: res*****7t2"),
|
||||
"scenario 7: resource secret not masked\nLogs:\n{logs7}"
|
||||
);
|
||||
// Non-secret field should remain visible
|
||||
|
||||
@@ -8,6 +8,7 @@
|
||||
|
||||
//! Query builders for filtering job lists (queue and completed).
|
||||
|
||||
use serde_json;
|
||||
use sql_builder::prelude::*;
|
||||
use sql_builder::SqlBuilder;
|
||||
use windmill_common::utils::{escape_ilike_pattern, paginate_without_limits, Pagination};
|
||||
@@ -200,7 +201,11 @@ pub fn filter_list_queue_query(
|
||||
}
|
||||
|
||||
if let Some(args) = &lq.args {
|
||||
sqlb.and_where("args @> ?".bind(&args.replace("'", "''")));
|
||||
if let Ok(v) = serde_json::from_str::<serde_json::Value>(args) {
|
||||
sqlb.and_where("args @> ?".bind(&v.to_string()));
|
||||
} else {
|
||||
sqlb.and_where("FALSE");
|
||||
}
|
||||
}
|
||||
|
||||
if lq.scheduled_for_before_now.is_some_and(|x| x) {
|
||||
@@ -499,11 +504,19 @@ pub fn filter_list_completed_query(
|
||||
}
|
||||
|
||||
if let Some(args) = &lq.args {
|
||||
sqlb.and_where("args @> ?".bind(&args.replace("'", "''")));
|
||||
if let Ok(v) = serde_json::from_str::<serde_json::Value>(args) {
|
||||
sqlb.and_where("args @> ?".bind(&v.to_string()));
|
||||
} else {
|
||||
sqlb.and_where("FALSE");
|
||||
}
|
||||
}
|
||||
|
||||
if let Some(result) = &lq.result {
|
||||
sqlb.and_where("result @> ?".bind(&result.replace("'", "''")));
|
||||
if let Ok(v) = serde_json::from_str::<serde_json::Value>(result) {
|
||||
sqlb.and_where("result @> ?".bind(&v.to_string()));
|
||||
} else {
|
||||
sqlb.and_where("FALSE");
|
||||
}
|
||||
}
|
||||
|
||||
if lq.is_not_schedule.unwrap_or(false) {
|
||||
|
||||
@@ -658,7 +658,11 @@ async fn list_schedule(
|
||||
sqlb.and_where_eq("is_flow", "?".bind(&is_flow));
|
||||
}
|
||||
if let Some(args) = &lsq.args {
|
||||
sqlb.and_where("args @> ?".bind(&args.replace("'", "''")));
|
||||
if let Ok(v) = serde_json::from_str::<serde_json::Value>(args) {
|
||||
sqlb.and_where("args @> ?".bind(&v.to_string()));
|
||||
} else {
|
||||
sqlb.and_where("FALSE");
|
||||
}
|
||||
}
|
||||
if let Some(path_start) = &lsq.path_start {
|
||||
sqlb.and_where_like_left("path", path_start);
|
||||
|
||||
@@ -1447,7 +1447,7 @@ async fn get_script_history(
|
||||
check_scopes(&authed, || format!("scripts:read:{}", path))?;
|
||||
let mut tx = user_db.begin(&authed).await?;
|
||||
let query_result = sqlx::query!(
|
||||
"SELECT s.hash as hash, dm.deployment_msg as deployment_msg
|
||||
"SELECT s.hash as hash, dm.deployment_msg as deployment_msg, s.created_at as created_at
|
||||
FROM script s LEFT JOIN deployment_metadata dm ON s.hash = dm.script_hash
|
||||
WHERE s.workspace_id = $1 AND s.path = $2
|
||||
ORDER by s.created_at DESC",
|
||||
@@ -1463,6 +1463,7 @@ async fn get_script_history(
|
||||
.map(|row| ScriptHistory {
|
||||
script_hash: ScriptHash(row.hash),
|
||||
deployment_msg: row.deployment_msg,
|
||||
created_at: Some(row.created_at),
|
||||
})
|
||||
.collect();
|
||||
return Ok(Json(result));
|
||||
@@ -1477,7 +1478,7 @@ async fn get_latest_version(
|
||||
check_scopes(&authed, || format!("scripts:read:{}", path))?;
|
||||
let mut tx = user_db.begin(&authed).await?;
|
||||
let row_o = sqlx::query!(
|
||||
"SELECT s.hash as hash, dm.deployment_msg as deployment_msg
|
||||
"SELECT s.hash as hash, dm.deployment_msg as deployment_msg, s.created_at as created_at
|
||||
FROM script s LEFT JOIN deployment_metadata dm ON s.hash = dm.script_hash
|
||||
WHERE s.workspace_id = $1 AND s.path = $2
|
||||
ORDER by s.created_at DESC LIMIT 1",
|
||||
@@ -1491,7 +1492,8 @@ async fn get_latest_version(
|
||||
if let Some(row) = row_o {
|
||||
let result = ScriptHistory {
|
||||
script_hash: ScriptHash(row.hash),
|
||||
deployment_msg: row.deployment_msg, //
|
||||
deployment_msg: row.deployment_msg,
|
||||
created_at: Some(row.created_at),
|
||||
};
|
||||
return Ok(Json(Some(result)));
|
||||
} else {
|
||||
|
||||
@@ -1746,6 +1746,7 @@ async fn set_login_type(
|
||||
|
||||
#[allow(unreachable_code, unused_variables)]
|
||||
async fn login(
|
||||
headers: axum::http::HeaderMap,
|
||||
cookies: Cookies,
|
||||
Extension(db): Extension<DB>,
|
||||
Extension(argon2): Extension<Arc<Argon2<'_>>>,
|
||||
@@ -1756,8 +1757,10 @@ async fn login(
|
||||
return Ok("no_auth".to_string());
|
||||
}
|
||||
|
||||
let mut tx = db.begin().await?;
|
||||
let email = email.to_lowercase();
|
||||
windmill_common::login_rate_limit::check_and_increment_login_attempt(&headers, &email)?;
|
||||
|
||||
let mut tx = db.begin().await?;
|
||||
let audit_author = AuditAuthor {
|
||||
email: email.clone(),
|
||||
username: email.clone(),
|
||||
@@ -1789,6 +1792,7 @@ async fn login(
|
||||
None,
|
||||
)
|
||||
.await?;
|
||||
windmill_common::login_rate_limit::record_login_failure(&email);
|
||||
Err(Error::BadRequest("Invalid login".to_string()))
|
||||
} else {
|
||||
let token = create_session_token(&email, super_admin, &mut tx, cookies).await?;
|
||||
@@ -1825,6 +1829,7 @@ async fn login(
|
||||
None,
|
||||
)
|
||||
.await?;
|
||||
windmill_common::login_rate_limit::record_login_failure(&email);
|
||||
Err(Error::BadRequest("Invalid login".to_string()))
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1134,6 +1134,12 @@ async fn edit_webhook(
|
||||
) -> Result<String> {
|
||||
require_admin(is_admin, &username)?;
|
||||
|
||||
if *CLOUD_HOSTED {
|
||||
return Err(Error::BadRequest(
|
||||
"Workspace webhooks are not available on cloud-hosted instances".to_string(),
|
||||
));
|
||||
}
|
||||
|
||||
let mut tx = db.begin().await?;
|
||||
|
||||
if let Some(webhook) = &ew.webhook {
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
openapi: "3.0.3"
|
||||
|
||||
info:
|
||||
version: 1.668.0
|
||||
version: 1.669.1
|
||||
title: Windmill API
|
||||
|
||||
contact:
|
||||
@@ -10598,6 +10598,7 @@ paths:
|
||||
get:
|
||||
summary: get job
|
||||
operationId: getJob
|
||||
x-mcp-tool: true
|
||||
tags:
|
||||
- job
|
||||
parameters:
|
||||
@@ -10639,7 +10640,8 @@ paths:
|
||||
/w/{workspace}/jobs_u/get_logs/{id}:
|
||||
get:
|
||||
summary: get job logs
|
||||
operationId: getJob logs
|
||||
operationId: getJobLogs
|
||||
x-mcp-tool: true
|
||||
tags:
|
||||
- job
|
||||
parameters:
|
||||
@@ -21814,6 +21816,13 @@ components:
|
||||
required:
|
||||
- key
|
||||
- value
|
||||
filter_logic:
|
||||
type: string
|
||||
enum:
|
||||
- and
|
||||
- or
|
||||
default: and
|
||||
description: "Logic to apply when evaluating filters. 'and' requires all filters to match, 'or' requires any filter to match."
|
||||
initial_messages:
|
||||
type: array
|
||||
nullable: true
|
||||
@@ -21875,6 +21884,13 @@ components:
|
||||
required:
|
||||
- key
|
||||
- value
|
||||
filter_logic:
|
||||
type: string
|
||||
enum:
|
||||
- and
|
||||
- or
|
||||
default: and
|
||||
description: "Logic to apply when evaluating filters. 'and' requires all filters to match, 'or' requires any filter to match."
|
||||
initial_messages:
|
||||
type: array
|
||||
nullable: true
|
||||
@@ -21943,6 +21959,13 @@ components:
|
||||
required:
|
||||
- key
|
||||
- value
|
||||
filter_logic:
|
||||
type: string
|
||||
enum:
|
||||
- and
|
||||
- or
|
||||
default: and
|
||||
description: "Logic to apply when evaluating filters. 'and' requires all filters to match, 'or' requires any filter to match."
|
||||
initial_messages:
|
||||
type: array
|
||||
nullable: true
|
||||
@@ -22819,6 +22842,13 @@ components:
|
||||
required:
|
||||
- key
|
||||
- value
|
||||
filter_logic:
|
||||
type: string
|
||||
enum:
|
||||
- and
|
||||
- or
|
||||
default: and
|
||||
description: "Logic to apply when evaluating filters. 'and' requires all filters to match, 'or' requires any filter to match."
|
||||
auto_offset_reset:
|
||||
type: string
|
||||
enum:
|
||||
@@ -22890,6 +22920,13 @@ components:
|
||||
required:
|
||||
- key
|
||||
- value
|
||||
filter_logic:
|
||||
type: string
|
||||
enum:
|
||||
- and
|
||||
- or
|
||||
default: and
|
||||
description: "Logic to apply when evaluating filters. 'and' requires all filters to match, 'or' requires any filter to match."
|
||||
auto_offset_reset:
|
||||
type: string
|
||||
enum:
|
||||
@@ -22953,6 +22990,13 @@ components:
|
||||
required:
|
||||
- key
|
||||
- value
|
||||
filter_logic:
|
||||
type: string
|
||||
enum:
|
||||
- and
|
||||
- or
|
||||
default: and
|
||||
description: "Logic to apply when evaluating filters. 'and' requires all filters to match, 'or' requires any filter to match."
|
||||
auto_offset_reset:
|
||||
type: string
|
||||
enum:
|
||||
|
||||
@@ -79,7 +79,7 @@ use windmill_common::{jwt, oauth2::HmacSha256, variables::get_workspace_key};
|
||||
#[cfg(feature = "parquet")]
|
||||
use windmill_types::s3::{S3Object, S3Permission};
|
||||
|
||||
pub fn workspaced_service() -> Router {
|
||||
pub fn workspaced_service(raw_app_body_limit: usize) -> Router {
|
||||
Router::new()
|
||||
.route("/list", get(list_apps))
|
||||
.route("/list_search", get(list_search_apps))
|
||||
@@ -95,10 +95,16 @@ pub fn workspaced_service() -> Router {
|
||||
.route("/get_data/v/{*id}", get(get_raw_app_data))
|
||||
.route("/exists/{*path}", get(exists_app))
|
||||
.route("/update/{*path}", post(update_app))
|
||||
.route("/update_raw/{*path}", post(update_app_raw))
|
||||
.route(
|
||||
"/update_raw/{*path}",
|
||||
post(update_app_raw).layer(axum::extract::DefaultBodyLimit::max(raw_app_body_limit)),
|
||||
)
|
||||
.route("/delete/{*path}", delete(delete_app))
|
||||
.route("/create", post(create_app))
|
||||
.route("/create_raw", post(create_app_raw))
|
||||
.route(
|
||||
"/create_raw",
|
||||
post(create_app_raw).layer(axum::extract::DefaultBodyLimit::max(raw_app_body_limit)),
|
||||
)
|
||||
.route("/history/p/{*path}", get(get_app_history))
|
||||
.route("/get_latest_version/{*path}", get(get_latest_version))
|
||||
.route(
|
||||
@@ -1010,18 +1016,20 @@ macro_rules! process_app_multipart {
|
||||
let mut saved_app = None;
|
||||
let mut uploaded_js = false;
|
||||
|
||||
let request_size_limit_mb = *crate::REQUEST_SIZE_LIMIT.read().await / (1024 * 1024);
|
||||
let raw_app_limit_mb = request_size_limit_mb * 5;
|
||||
let mut multipart = $multipart;
|
||||
while let Some(field) = multipart
|
||||
.next_field()
|
||||
.await
|
||||
.map_err(|e| Error::BadRequest(format!("failed to read multipart field: {e}")))?
|
||||
.map_err(|e| Error::BadRequest(format!("failed to read multipart field: {e}. Could be due to the request size limit for raw app bundles which is {raw_app_limit_mb}MB (adjustable in instance settings)")))?
|
||||
{
|
||||
let name = field
|
||||
.name()
|
||||
.ok_or_else(|| Error::BadRequest("multipart field missing name".to_string()))?
|
||||
.to_string();
|
||||
let data = field.bytes().await.map_err(|e| {
|
||||
Error::BadRequest(format!("failed to read multipart stream: {e}"))
|
||||
Error::BadRequest(format!("failed to read multipart stream: {e}. Could be due to the request size limit for raw app bundles which is {raw_app_limit_mb}MB (adjustable in instance settings)"))
|
||||
})?;
|
||||
if name == "app" {
|
||||
let app = serde_json::from_slice(&data).map_err(to_anyhow)?;
|
||||
|
||||
@@ -12,8 +12,6 @@ use serde::{Deserialize, Serialize};
|
||||
use windmill_common::error::JsonResult;
|
||||
|
||||
use crate::db::{ApiAuthed, DB};
|
||||
use crate::health::get_pool_stats;
|
||||
use crate::health::PoolStats;
|
||||
use crate::utils::require_super_admin;
|
||||
|
||||
pub fn global_service() -> Router {
|
||||
@@ -81,8 +79,10 @@ pub struct LargeResultRow {
|
||||
|
||||
#[derive(Serialize)]
|
||||
pub struct ConnectionPoolInfo {
|
||||
pub pool: PoolStats,
|
||||
pub pg_max_connections: i64,
|
||||
pub pg_total_connections: i64,
|
||||
pub pg_active_connections: i64,
|
||||
pub pg_idle_connections: i64,
|
||||
pub status: HealthLevel,
|
||||
pub message: String,
|
||||
}
|
||||
@@ -307,16 +307,30 @@ async fn fetch_large_results(
|
||||
}
|
||||
|
||||
async fn fetch_connection_pool(db: &DB) -> windmill_common::error::Result<ConnectionPoolInfo> {
|
||||
let pool = get_pool_stats(db);
|
||||
let max_row = sqlx::query_scalar!(
|
||||
r#"SELECT setting::bigint as "max!" FROM pg_settings WHERE name = 'max_connections'"#
|
||||
)
|
||||
.fetch_one(db)
|
||||
.await?;
|
||||
|
||||
let active_row =
|
||||
sqlx::query!("SELECT COUNT(*) as cnt FROM pg_stat_activity WHERE state = 'active'")
|
||||
.fetch_one(db)
|
||||
.await?;
|
||||
let stats_row = sqlx::query!(
|
||||
r#"SELECT
|
||||
COUNT(*) as "total!",
|
||||
COUNT(*) FILTER (WHERE state = 'active') as "active!",
|
||||
COUNT(*) FILTER (WHERE state = 'idle') as "idle!"
|
||||
FROM pg_stat_activity
|
||||
WHERE backend_type = 'client backend'"#
|
||||
)
|
||||
.fetch_one(db)
|
||||
.await?;
|
||||
|
||||
let pg_active = active_row.cnt.unwrap_or(0);
|
||||
let utilization = if pool.max_connections > 0 {
|
||||
pool.size as f64 / pool.max_connections as f64
|
||||
let pg_max = max_row;
|
||||
let pg_total = stats_row.total;
|
||||
let pg_active = stats_row.active;
|
||||
let pg_idle = stats_row.idle;
|
||||
|
||||
let utilization = if pg_max > 0 {
|
||||
pg_total as f64 / pg_max as f64
|
||||
} else {
|
||||
0.0
|
||||
};
|
||||
@@ -325,50 +339,75 @@ async fn fetch_connection_pool(db: &DB) -> windmill_common::error::Result<Connec
|
||||
(
|
||||
HealthLevel::Green,
|
||||
format!(
|
||||
"Pool utilization: {:.0}% ({}/{})",
|
||||
"Connection utilization: {:.0}% ({}/{})",
|
||||
utilization * 100.0,
|
||||
pool.size,
|
||||
pool.max_connections
|
||||
pg_total,
|
||||
pg_max
|
||||
),
|
||||
)
|
||||
} else if utilization < 0.95 {
|
||||
(
|
||||
HealthLevel::Yellow,
|
||||
format!(
|
||||
"Pool utilization is high: {:.0}% ({}/{}). Consider increasing max_connections.",
|
||||
"Connection utilization is high: {:.0}% ({}/{}). Consider increasing max_connections.",
|
||||
utilization * 100.0,
|
||||
pool.size,
|
||||
pool.max_connections
|
||||
pg_total,
|
||||
pg_max
|
||||
),
|
||||
)
|
||||
} else {
|
||||
(
|
||||
HealthLevel::Red,
|
||||
format!(
|
||||
"Pool near exhaustion: {:.0}% ({}/{}). Increase max_connections urgently.",
|
||||
"Connections near exhaustion: {:.0}% ({}/{}). Increase max_connections urgently.",
|
||||
utilization * 100.0,
|
||||
pool.size,
|
||||
pool.max_connections
|
||||
pg_total,
|
||||
pg_max
|
||||
),
|
||||
)
|
||||
};
|
||||
|
||||
Ok(ConnectionPoolInfo { pool, pg_active_connections: pg_active, status, message })
|
||||
Ok(ConnectionPoolInfo {
|
||||
pg_max_connections: pg_max,
|
||||
pg_total_connections: pg_total,
|
||||
pg_active_connections: pg_active,
|
||||
pg_idle_connections: pg_idle,
|
||||
status,
|
||||
message,
|
||||
})
|
||||
}
|
||||
|
||||
async fn fetch_table_maintenance(
|
||||
db: &DB,
|
||||
) -> windmill_common::error::Result<Vec<TableMaintenanceInfo>> {
|
||||
// Aggregate partitioned tables (e.g. audit_YYYYMMDD -> audit_partitioned)
|
||||
// while keeping non-partitioned tables as-is
|
||||
let rows = sqlx::query!(
|
||||
r#"SELECT
|
||||
schemaname || '.' || relname as "table_name!",
|
||||
COALESCE(n_live_tup, 0) as "live_tuples!",
|
||||
COALESCE(n_dead_tup, 0) as "dead_tuples!",
|
||||
last_autovacuum as "last_autovacuum",
|
||||
last_autoanalyze as "last_autoanalyze"
|
||||
FROM pg_stat_user_tables
|
||||
ORDER BY n_dead_tup DESC
|
||||
LIMIT 15"#
|
||||
table_name as "table_name!",
|
||||
SUM(live_tuples)::bigint as "live_tuples!",
|
||||
SUM(dead_tuples)::bigint as "dead_tuples!",
|
||||
MAX(last_autovacuum) as "last_autovacuum",
|
||||
MAX(last_autoanalyze) as "last_autoanalyze"
|
||||
FROM (
|
||||
SELECT
|
||||
CASE
|
||||
WHEN i.inhparent IS NOT NULL THEN schemaname || '.' || p.relname
|
||||
ELSE schemaname || '.' || s.relname
|
||||
END as table_name,
|
||||
COALESCE(n_live_tup, 0) as live_tuples,
|
||||
COALESCE(n_dead_tup, 0) as dead_tuples,
|
||||
last_autovacuum,
|
||||
last_autoanalyze
|
||||
FROM pg_stat_user_tables s
|
||||
LEFT JOIN pg_class c ON c.relname = s.relname AND c.relnamespace = (
|
||||
SELECT oid FROM pg_namespace WHERE nspname = s.schemaname
|
||||
)
|
||||
LEFT JOIN pg_inherits i ON i.inhrelid = c.oid
|
||||
LEFT JOIN pg_class p ON p.oid = i.inhparent
|
||||
) sub
|
||||
GROUP BY table_name
|
||||
ORDER BY SUM(dead_tuples) DESC"#
|
||||
)
|
||||
.fetch_all(db)
|
||||
.await?;
|
||||
|
||||
@@ -2179,13 +2179,25 @@ async fn list_jobs(
|
||||
pub async fn resume_suspended_flow_as_owner(
|
||||
authed: ApiAuthed,
|
||||
Extension(db): Extension<DB>,
|
||||
Path((_w_id, flow_id)): Path<(String, Uuid)>,
|
||||
Path((w_id, flow_id)): Path<(String, Uuid)>,
|
||||
QueryOrBody(value): QueryOrBody<serde_json::Value>,
|
||||
) -> error::Result<StatusCode> {
|
||||
let mut tx = db.begin().await?;
|
||||
|
||||
let (flow, job_id, is_wac) = get_suspended_flow_info(flow_id, &mut tx).await?;
|
||||
|
||||
// Verify the job belongs to this workspace
|
||||
let job_workspace: Option<String> =
|
||||
sqlx::query_scalar("SELECT workspace_id FROM v2_job WHERE id = $1")
|
||||
.bind(&flow.id)
|
||||
.fetch_optional(&mut *tx)
|
||||
.await?;
|
||||
if job_workspace.as_deref() != Some(w_id.as_str()) {
|
||||
return Err(Error::NotFound(
|
||||
"Job not found in this workspace".to_string(),
|
||||
));
|
||||
}
|
||||
|
||||
let flow_path = flow.script_path.as_deref().unwrap_or_else(|| "");
|
||||
require_owner_of_path(&authed, flow_path)?;
|
||||
check_scopes(&authed, || format!("jobs:run:flows:{}", flow_path))?;
|
||||
@@ -2516,7 +2528,8 @@ async fn get_approval_info(
|
||||
let approval_step = fs.as_ref().map(|s| (s.step as usize).saturating_sub(1));
|
||||
|
||||
// Fetch flow definition to get suspend settings (form schema, hide_cancel).
|
||||
// Try raw_flow on the job first, fall back to flow_version for deployed flows.
|
||||
// Try raw_flow on the job first, fall back to flow_version for deployed flows,
|
||||
// then flow_node for graph-based branch/loop sub-flows.
|
||||
let raw_flow: Option<FlowValue> = {
|
||||
let from_job: Option<serde_json::Value> = sqlx::query_scalar(
|
||||
"SELECT raw_flow FROM v2_job WHERE id = $1 AND workspace_id = $2",
|
||||
@@ -2540,7 +2553,23 @@ async fn get_approval_info(
|
||||
.fetch_optional(&db)
|
||||
.await?
|
||||
.flatten();
|
||||
from_version.and_then(|v| serde_json::from_value(v).ok())
|
||||
if let Some(v) = from_version {
|
||||
serde_json::from_value(v).ok()
|
||||
} else {
|
||||
// FlowNode sub-flow (graph-based branch/loop): raw_flow is not stored
|
||||
// in v2_job for newer versions, fetch from flow_node table
|
||||
let from_node: Option<serde_json::Value> = sqlx::query_scalar(
|
||||
"SELECT fn.flow FROM v2_job j \
|
||||
JOIN flow_node fn ON fn.id = j.runnable_id \
|
||||
WHERE j.id = $1 AND j.workspace_id = $2",
|
||||
)
|
||||
.bind(&job_id)
|
||||
.bind(&w_id)
|
||||
.fetch_optional(&db)
|
||||
.await?
|
||||
.flatten();
|
||||
from_node.and_then(|v| serde_json::from_value(v).ok())
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
|
||||
@@ -379,6 +379,8 @@ pub async fn run_server(
|
||||
REQUEST_SIZE_LIMIT.read().await.clone(),
|
||||
));
|
||||
|
||||
let request_size_limit = REQUEST_SIZE_LIMIT.read().await.clone();
|
||||
|
||||
let cors = CorsLayer::new()
|
||||
.allow_methods([http::Method::GET, http::Method::POST, http::Method::DELETE])
|
||||
.allow_headers([http::header::CONTENT_TYPE, http::header::AUTHORIZATION])
|
||||
@@ -475,17 +477,14 @@ pub async fn run_server(
|
||||
let (mcp_router, mcp_cancellation_token) =
|
||||
setup_mcp_server(db.clone(), user_db, _base_internal_url.clone()).await?;
|
||||
// Workspace-scoped MCP router
|
||||
// Use `layer` instead of `route_layer` because the MCP router only has
|
||||
// a fallback_service (no explicit routes), and axum 0.8 panics on
|
||||
// route_layer with no routes.
|
||||
let workspaced_mcp_router = mcp_router
|
||||
.clone()
|
||||
.layer(from_extractor::<ApiAuthed>())
|
||||
.route_layer(from_extractor::<ApiAuthed>())
|
||||
.layer(axum::middleware::from_fn(add_www_authenticate_header))
|
||||
.layer(axum::middleware::from_fn(extract_and_store_workspace_id));
|
||||
// Gateway MCP router — resolves workspace from token
|
||||
let gateway_mcp_router = mcp_router
|
||||
.layer(from_extractor::<ApiAuthed>())
|
||||
.route_layer(from_extractor::<ApiAuthed>())
|
||||
.layer(axum::middleware::from_fn(
|
||||
add_www_authenticate_header_gateway,
|
||||
))
|
||||
@@ -536,7 +535,7 @@ pub async fn run_server(
|
||||
Router::new()
|
||||
// Reordered alphabetically
|
||||
.nest("/acls", granular_acls::workspaced_service())
|
||||
.nest("/apps", apps::workspaced_service())
|
||||
.nest("/apps", apps::workspaced_service(request_size_limit * 5))
|
||||
.nest("/assets", windmill_api_assets::workspaced_service())
|
||||
.nest("/audit", audit::workspaced_service())
|
||||
.nest("/capture", capture::workspaced_service())
|
||||
|
||||
@@ -221,6 +221,22 @@ pub fn all_tools() -> Vec<EndpointTool> {
|
||||
"type": "string",
|
||||
"description": "filter variables by path prefix"
|
||||
},
|
||||
"path": {
|
||||
"type": "string",
|
||||
"description": "exact path match filter"
|
||||
},
|
||||
"description": {
|
||||
"type": "string",
|
||||
"description": "pattern match filter for description field (case-insensitive)"
|
||||
},
|
||||
"value": {
|
||||
"type": "string",
|
||||
"description": "pattern match filter for non-secret variable values (case-insensitive)"
|
||||
},
|
||||
"broad_filter": {
|
||||
"type": "string",
|
||||
"description": "broad search across multiple fields (case-insensitive substring match)"
|
||||
},
|
||||
"page": {
|
||||
"type": "integer",
|
||||
"description": "which page to return (start at 1, default 1)"
|
||||
@@ -405,6 +421,22 @@ pub fn all_tools() -> Vec<EndpointTool> {
|
||||
"path_start": {
|
||||
"type": "string",
|
||||
"description": "filter resources by path prefix"
|
||||
},
|
||||
"path": {
|
||||
"type": "string",
|
||||
"description": "exact path match filter"
|
||||
},
|
||||
"description": {
|
||||
"type": "string",
|
||||
"description": "pattern match filter for description field (case-insensitive)"
|
||||
},
|
||||
"value": {
|
||||
"type": "string",
|
||||
"description": "JSONB subset match filter using base64 encoded JSON"
|
||||
},
|
||||
"broad_filter": {
|
||||
"type": "string",
|
||||
"description": "broad search across multiple fields (case-insensitive substring match)"
|
||||
}
|
||||
},
|
||||
"required": []
|
||||
@@ -451,7 +483,7 @@ pub fn all_tools() -> Vec<EndpointTool> {
|
||||
},
|
||||
"created_by": {
|
||||
"type": "string",
|
||||
"description": "mask to filter exact matching user creator"
|
||||
"description": "filter by exact matching user creator. Supports comma-separated list (e.g. 'alice,bob') and negation by prefixing all values with '!' (e.g. '!alice,!bob')"
|
||||
},
|
||||
"path_start": {
|
||||
"type": "string",
|
||||
@@ -562,7 +594,6 @@ pub fn all_tools() -> Vec<EndpointTool> {
|
||||
"required": [
|
||||
"path",
|
||||
"summary",
|
||||
"description",
|
||||
"content",
|
||||
"language"
|
||||
]
|
||||
@@ -708,7 +739,7 @@ pub fn all_tools() -> Vec<EndpointTool> {
|
||||
},
|
||||
"created_by": {
|
||||
"type": "string",
|
||||
"description": "mask to filter exact matching user creator"
|
||||
"description": "filter by exact matching user creator. Supports comma-separated list (e.g. 'alice,bob') and negation by prefixing all values with '!' (e.g. '!alice,!bob')"
|
||||
},
|
||||
"path_start": {
|
||||
"type": "string",
|
||||
@@ -1078,6 +1109,37 @@ pub fn all_tools() -> Vec<EndpointTool> {
|
||||
},
|
||||
"lock": {
|
||||
"type": "string"
|
||||
},
|
||||
"flow_path": {
|
||||
"type": "string"
|
||||
},
|
||||
"modules": {
|
||||
"type": "object",
|
||||
"nullable": true,
|
||||
"description": "Additional script modules keyed by relative file path",
|
||||
"additionalProperties": {
|
||||
"type": "object",
|
||||
"description": "An additional module file associated with a script",
|
||||
"properties": {
|
||||
"content": {
|
||||
"type": "string",
|
||||
"description": "The source code content of this module"
|
||||
},
|
||||
"language": {
|
||||
"type": "string",
|
||||
"description": "Possible values: python3, deno, go, bash, powershell, postgresql, mysql, bigquery, snowflake, mssql, oracledb, graphql, nativets, bun, php, rust, ansible, csharp, nu, java, ruby, duckdb, bunnative"
|
||||
},
|
||||
"lock": {
|
||||
"type": "string",
|
||||
"nullable": true,
|
||||
"description": "Lock file content for this module's dependencies"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"content",
|
||||
"language"
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
@@ -1106,7 +1168,7 @@ pub fn all_tools() -> Vec<EndpointTool> {
|
||||
},
|
||||
"created_by": {
|
||||
"type": "string",
|
||||
"description": "mask to filter exact matching user creator"
|
||||
"description": "filter by exact matching user creator. Supports comma-separated list (e.g. 'alice,bob') and negation by prefixing all values with '!' (e.g. '!alice,!bob')"
|
||||
},
|
||||
"parent_job": {
|
||||
"type": "string",
|
||||
@@ -1115,15 +1177,15 @@ pub fn all_tools() -> Vec<EndpointTool> {
|
||||
},
|
||||
"worker": {
|
||||
"type": "string",
|
||||
"description": "worker this job was ran on"
|
||||
"description": "filter by worker this job ran on. Supports comma-separated list (e.g. 'worker-1,worker-2') and negation by prefixing all values with '!' (e.g. '!worker-1,!worker-2')"
|
||||
},
|
||||
"script_path_exact": {
|
||||
"type": "string",
|
||||
"description": "mask to filter exact matching path"
|
||||
"description": "filter by exact matching script path. Supports comma-separated list (e.g. 'f/script1,f/script2') and negation by prefixing all values with '!' (e.g. '!f/script1,!f/script2')"
|
||||
},
|
||||
"script_path_start": {
|
||||
"type": "string",
|
||||
"description": "mask to filter matching starting path"
|
||||
"description": "filter by script path prefix. Supports comma-separated list (e.g. 'f/folder1,f/folder2') and negation by prefixing all values with '!' (e.g. '!f/folder1,!f/folder2')"
|
||||
},
|
||||
"schedule_path": {
|
||||
"type": "string",
|
||||
@@ -1131,11 +1193,11 @@ pub fn all_tools() -> Vec<EndpointTool> {
|
||||
},
|
||||
"trigger_path": {
|
||||
"type": "string",
|
||||
"description": "mask to filter by trigger path"
|
||||
"description": "filter by trigger path. Supports comma-separated list (e.g. 'f/trigger1,f/trigger2') and negation by prefixing all values with '!' (e.g. '!f/trigger1,!f/trigger2')"
|
||||
},
|
||||
"trigger_kind": {
|
||||
"description": "trigger kind (schedule, http, websocket...). Possible values: webhook, default_email, email, schedule, http, websocket, postgres, kafka, nats, mqtt, sqs, gcp",
|
||||
"type": "string"
|
||||
"type": "string",
|
||||
"description": "filter by trigger kind. Supports comma-separated list (e.g. 'schedule,webhook') and negation by prefixing all values with '!' (e.g. '!schedule,!webhook')"
|
||||
},
|
||||
"script_hash": {
|
||||
"type": "string",
|
||||
@@ -1161,7 +1223,7 @@ pub fn all_tools() -> Vec<EndpointTool> {
|
||||
},
|
||||
"job_kinds": {
|
||||
"type": "string",
|
||||
"description": "filter on job kind (values 'preview', 'script', 'dependencies', 'flow') separated by,"
|
||||
"description": "filter by job kind. Supports comma-separated list of values ('preview', 'script', 'dependencies', 'flow') and negation by prefixing all values with '!' (e.g. '!preview,!dependencies')"
|
||||
},
|
||||
"suspended": {
|
||||
"type": "boolean",
|
||||
@@ -1185,7 +1247,7 @@ pub fn all_tools() -> Vec<EndpointTool> {
|
||||
},
|
||||
"tag": {
|
||||
"type": "string",
|
||||
"description": "filter on jobs with a given tag/worker group"
|
||||
"description": "filter by tag/worker group. Supports comma-separated list (e.g. 'gpu,highmem') and negation by prefixing all values with '!' (e.g. '!gpu,!highmem')"
|
||||
},
|
||||
"page": {
|
||||
"type": "integer",
|
||||
@@ -1223,15 +1285,15 @@ pub fn all_tools() -> Vec<EndpointTool> {
|
||||
"properties": {
|
||||
"created_by": {
|
||||
"type": "string",
|
||||
"description": "mask to filter exact matching user creator"
|
||||
"description": "filter by exact matching user creator. Supports comma-separated list (e.g. 'alice,bob') and negation by prefixing all values with '!' (e.g. '!alice,!bob')"
|
||||
},
|
||||
"label": {
|
||||
"type": "string",
|
||||
"description": "mask to filter exact matching job's label (job labels are completed jobs with as a result an object containing a string in the array at key 'wm_labels')"
|
||||
"description": "filter by exact matching job label. Supports comma-separated list (e.g. 'deploy,release') and negation by prefixing all values with '!' (e.g. '!deploy,!release')"
|
||||
},
|
||||
"worker": {
|
||||
"type": "string",
|
||||
"description": "worker this job was ran on"
|
||||
"description": "filter by worker this job ran on. Supports comma-separated list (e.g. 'worker-1,worker-2') and negation by prefixing all values with '!' (e.g. '!worker-1,!worker-2')"
|
||||
},
|
||||
"parent_job": {
|
||||
"type": "string",
|
||||
@@ -1240,11 +1302,11 @@ pub fn all_tools() -> Vec<EndpointTool> {
|
||||
},
|
||||
"script_path_exact": {
|
||||
"type": "string",
|
||||
"description": "mask to filter exact matching path"
|
||||
"description": "filter by exact matching script path. Supports comma-separated list (e.g. 'f/script1,f/script2') and negation by prefixing all values with '!' (e.g. '!f/script1,!f/script2')"
|
||||
},
|
||||
"script_path_start": {
|
||||
"type": "string",
|
||||
"description": "mask to filter matching starting path"
|
||||
"description": "filter by script path prefix. Supports comma-separated list (e.g. 'f/folder1,f/folder2') and negation by prefixing all values with '!' (e.g. '!f/folder1,!f/folder2')"
|
||||
},
|
||||
"schedule_path": {
|
||||
"type": "string",
|
||||
@@ -1304,7 +1366,7 @@ pub fn all_tools() -> Vec<EndpointTool> {
|
||||
},
|
||||
"job_kinds": {
|
||||
"type": "string",
|
||||
"description": "filter on job kind (values 'preview', 'script', 'dependencies', 'flow') separated by,"
|
||||
"description": "filter by job kind. Supports comma-separated list of values ('preview', 'script', 'dependencies', 'flow') and negation by prefixing all values with '!' (e.g. '!preview,!dependencies')"
|
||||
},
|
||||
"suspended": {
|
||||
"type": "boolean",
|
||||
@@ -1316,7 +1378,7 @@ pub fn all_tools() -> Vec<EndpointTool> {
|
||||
},
|
||||
"tag": {
|
||||
"type": "string",
|
||||
"description": "filter on jobs with a given tag/worker group"
|
||||
"description": "filter by tag/worker group. Supports comma-separated list (e.g. 'gpu,highmem') and negation by prefixing all values with '!' (e.g. '!gpu,!highmem')"
|
||||
},
|
||||
"result": {
|
||||
"type": "string",
|
||||
@@ -1331,8 +1393,8 @@ pub fn all_tools() -> Vec<EndpointTool> {
|
||||
"description": "number of items to return for a given page (default 30, max 100)"
|
||||
},
|
||||
"trigger_kind": {
|
||||
"description": "trigger kind (schedule, http, websocket...). Possible values: webhook, default_email, email, schedule, http, websocket, postgres, kafka, nats, mqtt, sqs, gcp",
|
||||
"type": "string"
|
||||
"type": "string",
|
||||
"description": "filter by trigger kind. Supports comma-separated list (e.g. 'schedule,webhook') and negation by prefixing all values with '!' (e.g. '!schedule,!webhook')"
|
||||
},
|
||||
"is_skipped": {
|
||||
"type": "boolean",
|
||||
@@ -1357,6 +1419,77 @@ pub fn all_tools() -> Vec<EndpointTool> {
|
||||
"is_not_schedule": {
|
||||
"type": "boolean",
|
||||
"description": "is not a scheduled job"
|
||||
},
|
||||
"broad_filter": {
|
||||
"type": "string",
|
||||
"description": "broad search across multiple fields (case-insensitive substring match on path, tag, schedule path, trigger kind, label)"
|
||||
}
|
||||
},
|
||||
"required": []
|
||||
})),
|
||||
body_schema: None,
|
||||
path_field_renames: None,
|
||||
query_field_renames: None,
|
||||
body_field_renames: None,
|
||||
},
|
||||
EndpointTool {
|
||||
name: Cow::Borrowed("getJob"),
|
||||
description: Cow::Borrowed("get job"),
|
||||
instructions: Cow::Borrowed(""),
|
||||
path: Cow::Borrowed("/w/{workspace}/jobs_u/get/{id}"),
|
||||
method: Cow::Borrowed("GET"),
|
||||
path_params_schema: Some(serde_json::json!({
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "string",
|
||||
"format": "uuid"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"id"
|
||||
]
|
||||
})),
|
||||
query_params_schema: Some(serde_json::json!({
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"no_logs": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"no_code": {
|
||||
"type": "boolean"
|
||||
}
|
||||
},
|
||||
"required": []
|
||||
})),
|
||||
body_schema: None,
|
||||
path_field_renames: None,
|
||||
query_field_renames: None,
|
||||
body_field_renames: None,
|
||||
},
|
||||
EndpointTool {
|
||||
name: Cow::Borrowed("getJobLogs"),
|
||||
description: Cow::Borrowed("get job logs"),
|
||||
instructions: Cow::Borrowed(""),
|
||||
path: Cow::Borrowed("/w/{workspace}/jobs_u/get_logs/{id}"),
|
||||
method: Cow::Borrowed("GET"),
|
||||
path_params_schema: Some(serde_json::json!({
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "string",
|
||||
"format": "uuid"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"id"
|
||||
]
|
||||
})),
|
||||
query_params_schema: Some(serde_json::json!({
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"remove_ansi_warnings": {
|
||||
"type": "boolean"
|
||||
}
|
||||
},
|
||||
"required": []
|
||||
@@ -1411,14 +1544,17 @@ You should get the schema of the script or flow before creating the schedule to
|
||||
},
|
||||
"on_failure": {
|
||||
"type": "string",
|
||||
"nullable": true,
|
||||
"description": "Path to a script or flow to run when the scheduled job fails"
|
||||
},
|
||||
"on_failure_times": {
|
||||
"type": "number",
|
||||
"nullable": true,
|
||||
"description": "Number of consecutive failures before the on_failure handler is triggered (default 1)"
|
||||
},
|
||||
"on_failure_exact": {
|
||||
"type": "boolean",
|
||||
"nullable": true,
|
||||
"description": "If true, trigger on_failure handler only on exactly N failures, not on every failure after N"
|
||||
},
|
||||
"on_failure_extra_args": {
|
||||
@@ -1428,10 +1564,12 @@ You should get the schema of the script or flow before creating the schedule to
|
||||
},
|
||||
"on_recovery": {
|
||||
"type": "string",
|
||||
"nullable": true,
|
||||
"description": "Path to a script or flow to run when the schedule recovers after failures"
|
||||
},
|
||||
"on_recovery_times": {
|
||||
"type": "number",
|
||||
"nullable": true,
|
||||
"description": "Number of consecutive successes before the on_recovery handler is triggered (default 1)"
|
||||
},
|
||||
"on_recovery_extra_args": {
|
||||
@@ -1441,6 +1579,7 @@ You should get the schema of the script or flow before creating the schedule to
|
||||
},
|
||||
"on_success": {
|
||||
"type": "string",
|
||||
"nullable": true,
|
||||
"description": "Path to a script or flow to run after each successful execution"
|
||||
},
|
||||
"on_success_extra_args": {
|
||||
@@ -1516,28 +1655,42 @@ You should get the schema of the script or flow before creating the schedule to
|
||||
},
|
||||
"summary": {
|
||||
"type": "string",
|
||||
"nullable": true,
|
||||
"description": "Short summary describing the purpose of this schedule"
|
||||
},
|
||||
"description": {
|
||||
"type": "string",
|
||||
"nullable": true,
|
||||
"description": "Detailed description of what this schedule does"
|
||||
},
|
||||
"tag": {
|
||||
"type": "string",
|
||||
"nullable": true,
|
||||
"description": "Worker tag to route jobs to specific worker groups"
|
||||
},
|
||||
"paused_until": {
|
||||
"type": "string",
|
||||
"nullable": true,
|
||||
"format": "date-time",
|
||||
"description": "ISO 8601 datetime until which the schedule is paused. Schedule resumes automatically after this time"
|
||||
},
|
||||
"cron_version": {
|
||||
"type": "string",
|
||||
"nullable": true,
|
||||
"description": "Cron parser version. Use 'v2' for extended syntax with additional features"
|
||||
},
|
||||
"dynamic_skip": {
|
||||
"type": "string",
|
||||
"nullable": true,
|
||||
"description": "Path to a script that validates scheduled datetimes. Receives scheduled_for datetime and returns boolean to skip (true) or run (false)"
|
||||
},
|
||||
"permissioned_as": {
|
||||
"type": "string",
|
||||
"description": "The user or group this schedule runs as. Used during deployment to preserve the original schedule owner."
|
||||
},
|
||||
"preserve_permissioned_as": {
|
||||
"type": "boolean",
|
||||
"description": "When true and the caller is a member of the 'wm_deployers' group, preserves the original permissioned_as value instead of overwriting it."
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
@@ -1592,14 +1745,17 @@ You should get the schema of the script or flow before updating the schedule to
|
||||
},
|
||||
"on_failure": {
|
||||
"type": "string",
|
||||
"nullable": true,
|
||||
"description": "Path to a script or flow to run when the scheduled job fails"
|
||||
},
|
||||
"on_failure_times": {
|
||||
"type": "number",
|
||||
"nullable": true,
|
||||
"description": "Number of consecutive failures before the on_failure handler is triggered (default 1)"
|
||||
},
|
||||
"on_failure_exact": {
|
||||
"type": "boolean",
|
||||
"nullable": true,
|
||||
"description": "If true, trigger on_failure handler only on exactly N failures, not on every failure after N"
|
||||
},
|
||||
"on_failure_extra_args": {
|
||||
@@ -1609,10 +1765,12 @@ You should get the schema of the script or flow before updating the schedule to
|
||||
},
|
||||
"on_recovery": {
|
||||
"type": "string",
|
||||
"nullable": true,
|
||||
"description": "Path to a script or flow to run when the schedule recovers after failures"
|
||||
},
|
||||
"on_recovery_times": {
|
||||
"type": "number",
|
||||
"nullable": true,
|
||||
"description": "Number of consecutive successes before the on_recovery handler is triggered (default 1)"
|
||||
},
|
||||
"on_recovery_extra_args": {
|
||||
@@ -1622,6 +1780,7 @@ You should get the schema of the script or flow before updating the schedule to
|
||||
},
|
||||
"on_success": {
|
||||
"type": "string",
|
||||
"nullable": true,
|
||||
"description": "Path to a script or flow to run after each successful execution"
|
||||
},
|
||||
"on_success_extra_args": {
|
||||
@@ -1697,28 +1856,44 @@ You should get the schema of the script or flow before updating the schedule to
|
||||
},
|
||||
"summary": {
|
||||
"type": "string",
|
||||
"nullable": true,
|
||||
"description": "Short summary describing the purpose of this schedule"
|
||||
},
|
||||
"description": {
|
||||
"type": "string",
|
||||
"nullable": true,
|
||||
"description": "Detailed description of what this schedule does"
|
||||
},
|
||||
"tag": {
|
||||
"type": "string",
|
||||
"nullable": true,
|
||||
"description": "Worker tag to route jobs to specific worker groups"
|
||||
},
|
||||
"paused_until": {
|
||||
"type": "string",
|
||||
"nullable": true,
|
||||
"format": "date-time",
|
||||
"description": "ISO 8601 datetime until which the schedule is paused. Schedule resumes automatically after this time"
|
||||
},
|
||||
"cron_version": {
|
||||
"type": "string",
|
||||
"nullable": true,
|
||||
"description": "Cron parser version. Use 'v2' for extended syntax with additional features"
|
||||
},
|
||||
"dynamic_skip": {
|
||||
"type": "string",
|
||||
"nullable": true,
|
||||
"description": "Path to a script that validates scheduled datetimes. Receives scheduled_for datetime and returns boolean to skip (true) or run (false)"
|
||||
},
|
||||
"permissioned_as": {
|
||||
"type": "string",
|
||||
"nullable": true,
|
||||
"description": "The user or group this schedule runs as (e.g., 'u/admin' or 'g/mygroup'). Only admins and wm_deployers can set this via preserve_permissioned_as."
|
||||
},
|
||||
"preserve_permissioned_as": {
|
||||
"type": "boolean",
|
||||
"nullable": true,
|
||||
"description": "If true and user is admin/wm_deployers, preserve the provided permissioned_as instead of using the deploying user's identity"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
@@ -1801,7 +1976,7 @@ You should get the schema of the script or flow before updating the schedule to
|
||||
},
|
||||
"path": {
|
||||
"type": "string",
|
||||
"description": "filter by path"
|
||||
"description": "filter by path (script path)"
|
||||
},
|
||||
"is_flow": {
|
||||
"type": "boolean",
|
||||
@@ -1810,6 +1985,22 @@ You should get the schema of the script or flow before updating the schedule to
|
||||
"path_start": {
|
||||
"type": "string",
|
||||
"description": "filter schedules by path prefix"
|
||||
},
|
||||
"schedule_path": {
|
||||
"type": "string",
|
||||
"description": "exact match on the schedule's path"
|
||||
},
|
||||
"description": {
|
||||
"type": "string",
|
||||
"description": "pattern match filter for description field (case-insensitive)"
|
||||
},
|
||||
"summary": {
|
||||
"type": "string",
|
||||
"description": "pattern match filter for summary field (case-insensitive)"
|
||||
},
|
||||
"broad_filter": {
|
||||
"type": "string",
|
||||
"description": "broad search across multiple fields (case-insensitive substring match)"
|
||||
}
|
||||
},
|
||||
"required": []
|
||||
|
||||
@@ -546,7 +546,7 @@ pub async fn setup_mcp_server(
|
||||
let service =
|
||||
StreamableHttpService::new(move || Ok(runner.clone()), session_manager, service_config);
|
||||
|
||||
let router = Router::new().fallback_service(service);
|
||||
let router = Router::new().route_service("/", service);
|
||||
Ok((router, cancellation_token))
|
||||
}
|
||||
|
||||
|
||||
@@ -97,6 +97,9 @@ async fn get_log_file(
|
||||
|
||||
require_devops_role(&db, &email).await?;
|
||||
let path = path.to_path();
|
||||
if path.contains("..") {
|
||||
return Err(Error::BadRequest("Invalid path".to_string()));
|
||||
}
|
||||
#[cfg(feature = "parquet")]
|
||||
let s3_client = windmill_object_store::get_object_store().await;
|
||||
#[cfg(feature = "parquet")]
|
||||
|
||||
@@ -117,6 +117,7 @@ pin-project-lite.workspace = true
|
||||
futures.workspace = true
|
||||
tempfile.workspace = true
|
||||
globset.workspace = true
|
||||
dashmap.workspace = true
|
||||
|
||||
opentelemetry-semantic-conventions = { workspace = true, optional = true }
|
||||
opentelemetry-otlp = { workspace = true, optional = true }
|
||||
|
||||
@@ -69,6 +69,7 @@ pub mod git_sync_ee;
|
||||
pub mod git_sync_oss;
|
||||
pub mod jobs;
|
||||
pub mod jwt;
|
||||
pub mod login_rate_limit;
|
||||
pub mod more_serde;
|
||||
pub mod oauth2;
|
||||
#[cfg(all(feature = "enterprise", feature = "openidconnect", feature = "private"))]
|
||||
|
||||
206
backend/windmill-common/src/login_rate_limit.rs
Normal file
206
backend/windmill-common/src/login_rate_limit.rs
Normal file
@@ -0,0 +1,206 @@
|
||||
use chrono::Utc;
|
||||
use dashmap::DashMap;
|
||||
use hyper::StatusCode;
|
||||
use std::sync::atomic::{AtomicI32, AtomicI64, AtomicU64, Ordering};
|
||||
use std::sync::LazyLock;
|
||||
|
||||
use crate::error::{Error, Result};
|
||||
use crate::worker::CLOUD_HOSTED;
|
||||
|
||||
const DEFAULT_PER_IP_LIMIT: i32 = 120;
|
||||
const DEFAULT_PER_ACCOUNT_LIMIT: i32 = 30;
|
||||
const DEFAULT_GLOBAL_LIMIT: i32 = 10000;
|
||||
const EVICTION_INTERVAL: u64 = 256;
|
||||
|
||||
struct RateLimitEntry {
|
||||
count: i32,
|
||||
minute_bucket: i64,
|
||||
}
|
||||
|
||||
static IP_RATE_LIMIT: LazyLock<DashMap<String, RateLimitEntry>> = LazyLock::new(DashMap::new);
|
||||
static ACCOUNT_RATE_LIMIT: LazyLock<DashMap<String, RateLimitEntry>> = LazyLock::new(DashMap::new);
|
||||
|
||||
static GLOBAL_COUNT: AtomicI32 = AtomicI32::new(0);
|
||||
static GLOBAL_MINUTE: AtomicI64 = AtomicI64::new(0);
|
||||
|
||||
static EVICTION_COUNTER: AtomicU64 = AtomicU64::new(0);
|
||||
|
||||
static PER_IP_LIMIT: LazyLock<i32> = LazyLock::new(|| {
|
||||
std::env::var("LOGIN_RATE_LIMIT_PER_IP")
|
||||
.ok()
|
||||
.and_then(|v| v.parse().ok())
|
||||
.unwrap_or(DEFAULT_PER_IP_LIMIT)
|
||||
});
|
||||
|
||||
static PER_IP_LIMIT_EXPLICIT: LazyLock<bool> = LazyLock::new(|| {
|
||||
std::env::var("LOGIN_RATE_LIMIT_PER_IP")
|
||||
.ok()
|
||||
.and_then(|v| v.parse::<i32>().ok())
|
||||
.is_some()
|
||||
});
|
||||
|
||||
static PER_ACCOUNT_LIMIT: LazyLock<i32> = LazyLock::new(|| {
|
||||
std::env::var("LOGIN_RATE_LIMIT_PER_ACCOUNT")
|
||||
.ok()
|
||||
.and_then(|v| v.parse().ok())
|
||||
.unwrap_or(DEFAULT_PER_ACCOUNT_LIMIT)
|
||||
});
|
||||
|
||||
static PER_ACCOUNT_LIMIT_EXPLICIT: LazyLock<bool> = LazyLock::new(|| {
|
||||
std::env::var("LOGIN_RATE_LIMIT_PER_ACCOUNT")
|
||||
.ok()
|
||||
.and_then(|v| v.parse::<i32>().ok())
|
||||
.is_some()
|
||||
});
|
||||
|
||||
static GLOBAL_LIMIT: LazyLock<i32> = LazyLock::new(|| {
|
||||
std::env::var("LOGIN_RATE_LIMIT_GLOBAL")
|
||||
.ok()
|
||||
.and_then(|v| v.parse().ok())
|
||||
.unwrap_or(DEFAULT_GLOBAL_LIMIT)
|
||||
});
|
||||
|
||||
/// Extract client IP from proxy headers. Only meaningful when behind a trusted
|
||||
/// reverse proxy (e.g. CLOUD_HOSTED). Returns `None` if no proxy header is present.
|
||||
pub fn extract_client_ip(headers: &axum::http::HeaderMap) -> Option<String> {
|
||||
if let Some(real_ip) = headers.get("x-real-ip") {
|
||||
if let Ok(ip) = real_ip.to_str() {
|
||||
let trimmed = ip.trim();
|
||||
if !trimmed.is_empty() {
|
||||
return Some(trimmed.to_string());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if let Some(forwarded_for) = headers.get("x-forwarded-for") {
|
||||
if let Ok(ips) = forwarded_for.to_str() {
|
||||
if let Some(first_ip) = ips.split(',').next() {
|
||||
let trimmed = first_ip.trim();
|
||||
if !trimmed.is_empty() {
|
||||
return Some(trimmed.to_string());
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
None
|
||||
}
|
||||
|
||||
fn maybe_evict(maps: &[&DashMap<String, RateLimitEntry>], current_minute: i64) {
|
||||
let count = EVICTION_COUNTER.fetch_add(1, Ordering::Relaxed);
|
||||
if count % EVICTION_INTERVAL == 0 {
|
||||
for map in maps {
|
||||
map.retain(|_, v| v.minute_bucket >= current_minute - 1);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Atomically check the rate limit and increment the counter. Follows the
|
||||
/// `public_app_rate_limit.rs` pattern — the DashMap entry lock is held across
|
||||
/// both the check and the increment, preventing TOCTOU races.
|
||||
fn check_and_increment(
|
||||
map: &DashMap<String, RateLimitEntry>,
|
||||
key: &str,
|
||||
limit: i32,
|
||||
current_minute: i64,
|
||||
) -> Result<()> {
|
||||
let mut entry = map
|
||||
.entry(key.to_string())
|
||||
.or_insert(RateLimitEntry { count: 0, minute_bucket: current_minute });
|
||||
|
||||
if entry.minute_bucket != current_minute {
|
||||
entry.count = 0;
|
||||
entry.minute_bucket = current_minute;
|
||||
}
|
||||
|
||||
if entry.count >= limit {
|
||||
return Err(Error::Generic(
|
||||
StatusCode::TOO_MANY_REQUESTS,
|
||||
"Too many login attempts. Please try again later.".to_string(),
|
||||
));
|
||||
}
|
||||
|
||||
entry.count += 1;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn record_failure(map: &DashMap<String, RateLimitEntry>, key: &str) {
|
||||
let current_minute = Utc::now().timestamp() / 60;
|
||||
|
||||
let mut entry = map
|
||||
.entry(key.to_string())
|
||||
.or_insert(RateLimitEntry { count: 0, minute_bucket: current_minute });
|
||||
|
||||
if entry.minute_bucket != current_minute {
|
||||
entry.count = 1;
|
||||
entry.minute_bucket = current_minute;
|
||||
} else {
|
||||
entry.count += 1;
|
||||
}
|
||||
}
|
||||
|
||||
/// Called BEFORE authentication. Checks and increments global + per-IP counters.
|
||||
/// The global counter counts all login attempts (not just failures), so it acts as
|
||||
/// a general throttle on login traffic per server instance.
|
||||
/// Per-IP is only active on CLOUD_HOSTED or when LOGIN_RATE_LIMIT_PER_IP is explicitly set.
|
||||
pub fn check_and_increment_login_attempt(
|
||||
headers: &axum::http::HeaderMap,
|
||||
email: &str,
|
||||
) -> Result<()> {
|
||||
let current_minute = Utc::now().timestamp() / 60;
|
||||
maybe_evict(&[&IP_RATE_LIMIT, &ACCOUNT_RATE_LIMIT], current_minute);
|
||||
|
||||
// Global limit: always on, uses atomics (single key, no need for DashMap)
|
||||
check_and_increment_global(current_minute)?;
|
||||
|
||||
// Per-IP limit: CLOUD_HOSTED or explicit opt-in
|
||||
if *CLOUD_HOSTED || *PER_IP_LIMIT_EXPLICIT {
|
||||
if let Some(ip) = extract_client_ip(headers) {
|
||||
check_and_increment(&IP_RATE_LIMIT, &ip, *PER_IP_LIMIT, current_minute)?;
|
||||
}
|
||||
}
|
||||
|
||||
// Per-account check (read-only, does not increment — failures are recorded separately)
|
||||
if *CLOUD_HOSTED || *PER_ACCOUNT_LIMIT_EXPLICIT {
|
||||
let entry = ACCOUNT_RATE_LIMIT.get(email);
|
||||
if let Some(entry) = entry {
|
||||
if entry.minute_bucket == current_minute && entry.count >= *PER_ACCOUNT_LIMIT {
|
||||
return Err(Error::Generic(
|
||||
StatusCode::TOO_MANY_REQUESTS,
|
||||
"Too many login attempts. Please try again later.".to_string(),
|
||||
));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn check_and_increment_global(current_minute: i64) -> Result<()> {
|
||||
let stored_minute = GLOBAL_MINUTE.load(Ordering::Relaxed);
|
||||
if stored_minute != current_minute {
|
||||
// Minute rolled over — reset. Race here is benign: worst case two threads
|
||||
// both reset, and we lose a few counts at the boundary.
|
||||
GLOBAL_MINUTE.store(current_minute, Ordering::Relaxed);
|
||||
GLOBAL_COUNT.store(1, Ordering::Relaxed);
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
let count = GLOBAL_COUNT.fetch_add(1, Ordering::Relaxed);
|
||||
if count >= *GLOBAL_LIMIT {
|
||||
return Err(Error::Generic(
|
||||
StatusCode::TOO_MANY_REQUESTS,
|
||||
"Too many login attempts. Please try again later.".to_string(),
|
||||
));
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Called AFTER authentication failure. Records per-account failure.
|
||||
/// Per-account is only active on CLOUD_HOSTED or when LOGIN_RATE_LIMIT_PER_ACCOUNT is explicitly set.
|
||||
pub fn record_login_failure(email: &str) {
|
||||
if *CLOUD_HOSTED || *PER_ACCOUNT_LIMIT_EXPLICIT {
|
||||
record_failure(&ACCOUNT_RATE_LIMIT, email);
|
||||
}
|
||||
}
|
||||
@@ -39,7 +39,14 @@ fn deserialize_string_from_null<'de, D>(deserializer: D) -> Result<String, D::Er
|
||||
where
|
||||
D: Deserializer<'de>,
|
||||
{
|
||||
Option::<String>::deserialize(deserializer).map(|v| v.unwrap_or_default())
|
||||
// DuckDB may return booleans for fields that other databases return as strings
|
||||
let v = serde_json::Value::deserialize(deserializer)?;
|
||||
match v {
|
||||
serde_json::Value::Null => Ok(String::new()),
|
||||
serde_json::Value::String(s) => Ok(s),
|
||||
serde_json::Value::Bool(b) => Ok(b.to_string()),
|
||||
other => Ok(other.to_string()),
|
||||
}
|
||||
}
|
||||
|
||||
fn deserialize_column_identity_from_null<'de, D>(
|
||||
@@ -49,15 +56,21 @@ where
|
||||
D: Deserializer<'de>,
|
||||
{
|
||||
// MySQL returns uppercase "YES"/"NO" while the enum expects title case.
|
||||
let v = Option::<String>::deserialize(deserializer)?;
|
||||
match v.as_deref() {
|
||||
None => Ok(ColumnIdentity::default()),
|
||||
Some(s) => match s.to_lowercase().as_str() {
|
||||
// DuckDB returns a boolean false instead of a string.
|
||||
let v = serde_json::Value::deserialize(deserializer)?;
|
||||
match v {
|
||||
serde_json::Value::Null => Ok(ColumnIdentity::default()),
|
||||
serde_json::Value::Bool(_) => Ok(ColumnIdentity::No),
|
||||
serde_json::Value::String(s) => match s.to_lowercase().as_str() {
|
||||
"no" => Ok(ColumnIdentity::No),
|
||||
"yes" | "always" => Ok(ColumnIdentity::Always),
|
||||
"by default" => Ok(ColumnIdentity::ByDefault),
|
||||
_ => Ok(ColumnIdentity::No),
|
||||
},
|
||||
_ => Err(serde::de::Error::custom(format!(
|
||||
"expected string, bool, or null for isidentity, got {}",
|
||||
v
|
||||
))),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2369,7 +2382,7 @@ fn make_load_table_metadata_query(
|
||||
COLUMN_DEFAULT as DefaultValue,
|
||||
false as IsPrimaryKey,
|
||||
false as IsIdentity,
|
||||
IS_NULLABLE as IsNullable,
|
||||
CASE WHEN IS_NULLABLE = true THEN 'YES' ELSE 'NO' END as IsNullable,
|
||||
false as IsEnum,
|
||||
TABLE_NAME as table_name
|
||||
FROM information_schema.columns c
|
||||
|
||||
@@ -88,8 +88,16 @@ pub fn snapshot(job_id: &Uuid) -> Option<MaskSnapshot> {
|
||||
let replacements: Vec<String> = sorted
|
||||
.iter()
|
||||
.map(|s| {
|
||||
let prefix: String = s.chars().take(3).collect();
|
||||
format!("{}*****", prefix)
|
||||
let char_count = s.chars().count();
|
||||
if char_count > 20 {
|
||||
let prefix: String = s.chars().take(3).collect();
|
||||
let suffix: String = s.chars().skip(char_count - 3).collect();
|
||||
format!("{}*****{}", prefix, suffix)
|
||||
} else {
|
||||
let first: String = s.chars().take(1).collect();
|
||||
let last: String = s.chars().skip(char_count - 1).collect();
|
||||
format!("{}*****{}", first, last)
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
|
||||
|
||||
@@ -81,6 +81,46 @@ pub async fn get_email_from_permissioned_as(
|
||||
}
|
||||
}
|
||||
|
||||
/// Compute the highest-precedence workspace role for a user across all their instance groups.
|
||||
///
|
||||
/// Precedence: admin (3) > developer (2) > operator (1).
|
||||
/// Returns `(best_group_name, is_admin, is_operator)`.
|
||||
pub fn compute_highest_workspace_role(
|
||||
user_igroups: &[String],
|
||||
ws_configured_groups: &[String],
|
||||
ws_roles: &std::collections::HashMap<String, String>,
|
||||
) -> (String, bool, bool) {
|
||||
let mut best_group = String::new();
|
||||
let mut best_precedence = 0u8;
|
||||
|
||||
for group in user_igroups {
|
||||
if !ws_configured_groups.contains(group) {
|
||||
continue;
|
||||
}
|
||||
let default_role = "developer".to_string();
|
||||
let role = ws_roles.get(group).unwrap_or(&default_role);
|
||||
let precedence = match role.as_str() {
|
||||
"admin" => 3u8,
|
||||
"operator" => 1,
|
||||
_ => 2,
|
||||
};
|
||||
if precedence > best_precedence {
|
||||
best_precedence = precedence;
|
||||
best_group = group.clone();
|
||||
}
|
||||
}
|
||||
|
||||
let default_role = "developer".to_string();
|
||||
let best_role_str = ws_roles.get(&best_group).unwrap_or(&default_role);
|
||||
let (is_admin, is_operator) = match best_role_str.as_str() {
|
||||
"admin" => (true, false),
|
||||
"operator" => (false, true),
|
||||
_ => (false, false),
|
||||
};
|
||||
|
||||
(best_group, is_admin, is_operator)
|
||||
}
|
||||
|
||||
pub fn truncate_token(token: &str) -> String {
|
||||
if token.len() > 10 {
|
||||
let mut s = token[..10].to_owned();
|
||||
@@ -105,4 +145,63 @@ mod tests {
|
||||
assert_eq!(username_to_permissioned_as("group-all"), "g/all");
|
||||
assert_eq!(username_to_permissioned_as("group-my-team"), "g/my-team");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_compute_highest_workspace_role_admin_wins() {
|
||||
let user_groups = vec!["ops".to_string(), "admins".to_string()];
|
||||
let ws_groups = vec!["ops".to_string(), "admins".to_string()];
|
||||
let mut roles = std::collections::HashMap::new();
|
||||
roles.insert("ops".to_string(), "operator".to_string());
|
||||
roles.insert("admins".to_string(), "admin".to_string());
|
||||
|
||||
let (group, is_admin, is_operator) =
|
||||
compute_highest_workspace_role(&user_groups, &ws_groups, &roles);
|
||||
assert_eq!(group, "admins");
|
||||
assert!(is_admin);
|
||||
assert!(!is_operator);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_compute_highest_workspace_role_developer_over_operator() {
|
||||
let user_groups = vec!["devs".to_string(), "ops".to_string()];
|
||||
let ws_groups = vec!["devs".to_string(), "ops".to_string()];
|
||||
let mut roles = std::collections::HashMap::new();
|
||||
roles.insert("devs".to_string(), "developer".to_string());
|
||||
roles.insert("ops".to_string(), "operator".to_string());
|
||||
|
||||
let (group, is_admin, is_operator) =
|
||||
compute_highest_workspace_role(&user_groups, &ws_groups, &roles);
|
||||
assert_eq!(group, "devs");
|
||||
assert!(!is_admin);
|
||||
assert!(!is_operator);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_compute_highest_workspace_role_skips_unconfigured_groups() {
|
||||
let user_groups = vec!["admins".to_string(), "other".to_string()];
|
||||
let ws_groups = vec!["ops".to_string()]; // admins not configured for this workspace
|
||||
let mut roles = std::collections::HashMap::new();
|
||||
roles.insert("admins".to_string(), "admin".to_string());
|
||||
roles.insert("ops".to_string(), "operator".to_string());
|
||||
|
||||
let (group, is_admin, is_operator) =
|
||||
compute_highest_workspace_role(&user_groups, &ws_groups, &roles);
|
||||
// No user groups match ws_configured_groups, so best_group stays empty
|
||||
assert_eq!(group, "");
|
||||
assert!(!is_admin);
|
||||
assert!(!is_operator);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_compute_highest_workspace_role_defaults_to_developer() {
|
||||
let user_groups = vec!["team".to_string()];
|
||||
let ws_groups = vec!["team".to_string()];
|
||||
let roles = std::collections::HashMap::new(); // no role configured → developer
|
||||
|
||||
let (group, is_admin, is_operator) =
|
||||
compute_highest_workspace_role(&user_groups, &ws_groups, &roles);
|
||||
assert_eq!(group, "team");
|
||||
assert!(!is_admin);
|
||||
assert!(!is_operator);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -233,6 +233,9 @@ impl WebhookShared {
|
||||
}
|
||||
|
||||
pub fn send_message(&self, workspace_id: String, message: WebhookMessage) {
|
||||
if *crate::worker::CLOUD_HOSTED {
|
||||
return;
|
||||
}
|
||||
let _ = self.channel.send(WebhookPayload::WorkspaceEvent(
|
||||
workspace_id.clone(),
|
||||
message,
|
||||
|
||||
@@ -149,7 +149,7 @@ pub enum ObjectType {
|
||||
WorkspaceDependencies,
|
||||
}
|
||||
|
||||
pub const LATEST_GIT_SYNC_SCRIPT_PATH: &str = "hub/28183/sync-script-to-git-repo-windmill";
|
||||
pub const LATEST_GIT_SYNC_SCRIPT_PATH: &str = "hub/28186/sync-script-to-git-repo-windmill";
|
||||
|
||||
#[derive(Serialize, Deserialize, Debug)]
|
||||
pub struct GitRepositorySettings {
|
||||
|
||||
@@ -28,10 +28,14 @@
|
||||
mod tests {
|
||||
use serde_json::json;
|
||||
use sqlx::{Pool, Postgres};
|
||||
use windmill_common::users::compute_highest_workspace_role;
|
||||
|
||||
/// Test that configuring instance groups for a workspace auto-adds existing group members
|
||||
#[ignore = "requires database setup - run with --ignored flag"]
|
||||
#[sqlx::test(migrations = "../migrations", fixtures("base", "instance_group_auto_add"))]
|
||||
#[sqlx::test(
|
||||
migrations = "../migrations",
|
||||
fixtures("base", "instance_group_auto_add")
|
||||
)]
|
||||
async fn test_configure_instance_groups_adds_existing_members(db: Pool<Postgres>) {
|
||||
// Configure workspace to auto-add users from 'engineering' group with 'developer' role
|
||||
let groups = vec!["engineering".to_string()];
|
||||
@@ -112,8 +116,14 @@ mod tests {
|
||||
"Alice should be in the workspace"
|
||||
);
|
||||
let alice = alice_in_workspace.unwrap();
|
||||
assert!(!alice.is_admin, "Alice should not be admin (developer role)");
|
||||
assert!(!alice.operator, "Alice should not be operator (developer role)");
|
||||
assert!(
|
||||
!alice.is_admin,
|
||||
"Alice should not be admin (developer role)"
|
||||
);
|
||||
assert!(
|
||||
!alice.operator,
|
||||
"Alice should not be operator (developer role)"
|
||||
);
|
||||
|
||||
// Check added_via field
|
||||
let added_via = alice.added_via.expect("added_via should be set");
|
||||
@@ -158,7 +168,10 @@ mod tests {
|
||||
|
||||
/// Test role assignment based on instance group configuration
|
||||
#[ignore = "requires database setup - run with --ignored flag"]
|
||||
#[sqlx::test(migrations = "../migrations", fixtures("base", "instance_group_auto_add"))]
|
||||
#[sqlx::test(
|
||||
migrations = "../migrations",
|
||||
fixtures("base", "instance_group_auto_add")
|
||||
)]
|
||||
async fn test_role_assignment_admin(db: Pool<Postgres>) {
|
||||
// Configure workspace with admins group having admin role
|
||||
let groups = vec!["admins".to_string()];
|
||||
@@ -209,7 +222,10 @@ mod tests {
|
||||
|
||||
/// Test role assignment for operator
|
||||
#[ignore = "requires database setup - run with --ignored flag"]
|
||||
#[sqlx::test(migrations = "../migrations", fixtures("base", "instance_group_auto_add"))]
|
||||
#[sqlx::test(
|
||||
migrations = "../migrations",
|
||||
fixtures("base", "instance_group_auto_add")
|
||||
)]
|
||||
async fn test_role_assignment_operator(db: Pool<Postgres>) {
|
||||
// Configure workspace with sales group having operator role
|
||||
let groups = vec!["sales".to_string()];
|
||||
@@ -260,7 +276,10 @@ mod tests {
|
||||
|
||||
/// Test role precedence when user is in multiple instance groups
|
||||
#[ignore = "requires database setup - run with --ignored flag"]
|
||||
#[sqlx::test(migrations = "../migrations", fixtures("base", "instance_group_auto_add"))]
|
||||
#[sqlx::test(
|
||||
migrations = "../migrations",
|
||||
fixtures("base", "instance_group_auto_add")
|
||||
)]
|
||||
async fn test_role_precedence_multiple_groups(db: Pool<Postgres>) {
|
||||
// Configure workspace with multiple groups: engineering (admin), sales (operator)
|
||||
// Bob is in both groups, should get admin role (highest precedence)
|
||||
@@ -306,7 +325,10 @@ mod tests {
|
||||
.await
|
||||
.expect("Failed to query user");
|
||||
|
||||
assert!(bob.is_admin, "Bob should be admin (highest precedence role)");
|
||||
assert!(
|
||||
bob.is_admin,
|
||||
"Bob should be admin (highest precedence role)"
|
||||
);
|
||||
assert!(!bob.operator, "Bob should not be operator");
|
||||
|
||||
// Verify added_via tracks the primary group (engineering, the one with highest precedence)
|
||||
@@ -320,9 +342,305 @@ mod tests {
|
||||
println!("✓ Role precedence works correctly for users in multiple groups");
|
||||
}
|
||||
|
||||
/// Test that adding a user to a second instance group upgrades their workspace role
|
||||
/// if the new group has a higher-precedence role.
|
||||
/// This is a regression test for the bug where only the newly-added group's role was used.
|
||||
#[ignore = "requires database setup - run with --ignored flag"]
|
||||
#[sqlx::test(
|
||||
migrations = "../migrations",
|
||||
fixtures("base", "instance_group_auto_add")
|
||||
)]
|
||||
async fn test_role_upgrade_when_added_to_higher_group(db: Pool<Postgres>) {
|
||||
// Configure workspace: engineering=operator, admins=admin
|
||||
let groups = vec!["engineering".to_string(), "admins".to_string()];
|
||||
let roles = json!({"engineering": "operator", "admins": "admin"});
|
||||
|
||||
sqlx::query!(
|
||||
r#"
|
||||
UPDATE workspace_settings
|
||||
SET auto_invite = jsonb_build_object(
|
||||
'instance_groups', $2::jsonb,
|
||||
'instance_groups_roles', $3::jsonb
|
||||
)
|
||||
WHERE workspace_id = $1
|
||||
"#,
|
||||
"ws-multi-group",
|
||||
serde_json::to_value(&groups).unwrap(),
|
||||
&roles,
|
||||
)
|
||||
.execute(&db)
|
||||
.await
|
||||
.expect("Failed to update workspace settings");
|
||||
|
||||
// Step 1: Alice is added via engineering group (operator)
|
||||
// (alice is already in engineering from fixture)
|
||||
let added_via = json!({"source": "instance_group", "group": "engineering"});
|
||||
sqlx::query!(
|
||||
"INSERT INTO usr (workspace_id, username, email, is_admin, operator, added_via)
|
||||
VALUES ($1, 'alice', 'alice@example.com', false, true, $2)",
|
||||
"ws-multi-group",
|
||||
&added_via,
|
||||
)
|
||||
.execute(&db)
|
||||
.await
|
||||
.expect("Failed to add user");
|
||||
|
||||
sqlx::query!(
|
||||
"INSERT INTO usr_to_group (workspace_id, usr, group_) VALUES ($1, 'alice', 'all')",
|
||||
"ws-multi-group",
|
||||
)
|
||||
.execute(&db)
|
||||
.await
|
||||
.expect("Failed to add user to all group");
|
||||
|
||||
// Verify initial state: alice is operator
|
||||
let alice = sqlx::query!(
|
||||
"SELECT is_admin, operator FROM usr WHERE workspace_id = 'ws-multi-group' AND email = 'alice@example.com'"
|
||||
)
|
||||
.fetch_one(&db)
|
||||
.await
|
||||
.expect("Failed to query user");
|
||||
assert!(!alice.is_admin, "Alice should start as non-admin");
|
||||
assert!(alice.operator, "Alice should start as operator");
|
||||
|
||||
// Step 2: Alice is added to admins group
|
||||
sqlx::query!(
|
||||
"INSERT INTO email_to_igroup (email, igroup) VALUES ('alice@example.com', 'admins') ON CONFLICT DO NOTHING"
|
||||
)
|
||||
.execute(&db)
|
||||
.await
|
||||
.expect("Failed to add to admins group");
|
||||
|
||||
// Step 3: Simulate the fixed logic — find all user's groups, compute highest role, update
|
||||
let user_igroups: Vec<String> = sqlx::query_scalar!(
|
||||
"SELECT igroup FROM email_to_igroup WHERE email = 'alice@example.com'"
|
||||
)
|
||||
.fetch_all(&db)
|
||||
.await
|
||||
.expect("Failed to fetch user groups");
|
||||
|
||||
let ws = sqlx::query!(
|
||||
r#"
|
||||
SELECT auto_invite->'instance_groups_roles' as instance_groups_roles,
|
||||
auto_invite->'instance_groups' as instance_groups_json
|
||||
FROM workspace_settings WHERE workspace_id = 'ws-multi-group'
|
||||
"#,
|
||||
)
|
||||
.fetch_one(&db)
|
||||
.await
|
||||
.expect("Failed to fetch workspace settings");
|
||||
|
||||
let ws_roles: std::collections::HashMap<String, String> = ws
|
||||
.instance_groups_roles
|
||||
.and_then(|r| serde_json::from_value(r).ok())
|
||||
.unwrap_or_default();
|
||||
|
||||
let ws_configured_groups: Vec<String> = ws
|
||||
.instance_groups_json
|
||||
.and_then(|ig| serde_json::from_value(ig).ok())
|
||||
.unwrap_or_default();
|
||||
|
||||
let (best_group, is_admin, is_operator) =
|
||||
compute_highest_workspace_role(&user_igroups, &ws_configured_groups, &ws_roles);
|
||||
|
||||
let instance_group_source = json!({
|
||||
"source": "instance_group",
|
||||
"group": &best_group
|
||||
});
|
||||
|
||||
sqlx::query!(
|
||||
"UPDATE usr SET is_admin = $1, operator = $2, added_via = $3 WHERE workspace_id = $4 AND email = $5 AND added_via->>'source' = 'instance_group'",
|
||||
is_admin,
|
||||
is_operator,
|
||||
&instance_group_source,
|
||||
"ws-multi-group",
|
||||
"alice@example.com"
|
||||
)
|
||||
.execute(&db)
|
||||
.await
|
||||
.expect("Failed to update user role");
|
||||
|
||||
// Verify: alice should now be admin (highest precedence)
|
||||
let alice = sqlx::query!(
|
||||
"SELECT is_admin, operator, added_via FROM usr WHERE workspace_id = 'ws-multi-group' AND email = 'alice@example.com'"
|
||||
)
|
||||
.fetch_one(&db)
|
||||
.await
|
||||
.expect("Failed to query user");
|
||||
|
||||
assert!(alice.is_admin, "Alice should be upgraded to admin");
|
||||
assert!(!alice.operator, "Alice should no longer be operator");
|
||||
|
||||
let added_via = alice.added_via.expect("added_via should be set");
|
||||
assert_eq!(
|
||||
added_via.get("group").and_then(|v| v.as_str()),
|
||||
Some("admins"),
|
||||
"added_via should track the admin group (highest precedence)"
|
||||
);
|
||||
|
||||
println!("✓ Role is upgraded when user is added to a higher-precedence group");
|
||||
}
|
||||
|
||||
/// Test that adding a user to a lower-precedence group does NOT downgrade their role
|
||||
#[ignore = "requires database setup - run with --ignored flag"]
|
||||
#[sqlx::test(
|
||||
migrations = "../migrations",
|
||||
fixtures("base", "instance_group_auto_add")
|
||||
)]
|
||||
async fn test_no_role_downgrade_when_added_to_lower_group(db: Pool<Postgres>) {
|
||||
// Configure workspace: engineering=admin, sales=operator
|
||||
let groups = vec!["engineering".to_string(), "sales".to_string()];
|
||||
let roles = json!({"engineering": "admin", "sales": "operator"});
|
||||
|
||||
sqlx::query!(
|
||||
r#"
|
||||
UPDATE workspace_settings
|
||||
SET auto_invite = jsonb_build_object(
|
||||
'instance_groups', $2::jsonb,
|
||||
'instance_groups_roles', $3::jsonb
|
||||
)
|
||||
WHERE workspace_id = $1
|
||||
"#,
|
||||
"ws-multi-group",
|
||||
serde_json::to_value(&groups).unwrap(),
|
||||
&roles,
|
||||
)
|
||||
.execute(&db)
|
||||
.await
|
||||
.expect("Failed to update workspace settings");
|
||||
|
||||
// Alice starts as admin from engineering
|
||||
let added_via = json!({"source": "instance_group", "group": "engineering"});
|
||||
sqlx::query!(
|
||||
"INSERT INTO usr (workspace_id, username, email, is_admin, operator, added_via)
|
||||
VALUES ($1, 'alice', 'alice@example.com', true, false, $2)",
|
||||
"ws-multi-group",
|
||||
&added_via,
|
||||
)
|
||||
.execute(&db)
|
||||
.await
|
||||
.expect("Failed to add user");
|
||||
|
||||
// Now simulate adding alice to sales group (operator — lower precedence)
|
||||
// The fixed code should keep her as admin
|
||||
let user_igroups = vec!["engineering".to_string(), "sales".to_string()];
|
||||
let ws_configured_groups = vec!["engineering".to_string(), "sales".to_string()];
|
||||
let ws_roles: std::collections::HashMap<String, String> =
|
||||
serde_json::from_value(roles).unwrap();
|
||||
|
||||
let (best_group, is_admin, is_operator) =
|
||||
compute_highest_workspace_role(&user_igroups, &ws_configured_groups, &ws_roles);
|
||||
|
||||
let instance_group_source = json!({"source": "instance_group", "group": &best_group});
|
||||
sqlx::query!(
|
||||
"UPDATE usr SET is_admin = $1, operator = $2, added_via = $3 WHERE workspace_id = $4 AND email = $5 AND added_via->>'source' = 'instance_group'",
|
||||
is_admin, is_operator, &instance_group_source, "ws-multi-group", "alice@example.com"
|
||||
)
|
||||
.execute(&db)
|
||||
.await
|
||||
.expect("Failed to update user role");
|
||||
|
||||
let alice = sqlx::query!(
|
||||
"SELECT is_admin, operator, added_via FROM usr WHERE workspace_id = 'ws-multi-group' AND email = 'alice@example.com'"
|
||||
)
|
||||
.fetch_one(&db)
|
||||
.await
|
||||
.expect("Failed to query user");
|
||||
|
||||
assert!(alice.is_admin, "Alice should remain admin (not downgraded)");
|
||||
assert!(!alice.operator, "Alice should not become operator");
|
||||
assert_eq!(
|
||||
alice
|
||||
.added_via
|
||||
.unwrap()
|
||||
.get("group")
|
||||
.and_then(|v| v.as_str()),
|
||||
Some("engineering"),
|
||||
"added_via should still track engineering (highest precedence)"
|
||||
);
|
||||
|
||||
println!("✓ Role is NOT downgraded when user is added to a lower-precedence group");
|
||||
}
|
||||
|
||||
/// Test that manually-added users are not affected by instance group role updates
|
||||
#[ignore = "requires database setup - run with --ignored flag"]
|
||||
#[sqlx::test(
|
||||
migrations = "../migrations",
|
||||
fixtures("base", "instance_group_auto_add")
|
||||
)]
|
||||
async fn test_manual_users_not_affected_by_group_role_update(db: Pool<Postgres>) {
|
||||
// Configure workspace
|
||||
let groups = vec!["engineering".to_string()];
|
||||
let roles = json!({"engineering": "operator"});
|
||||
|
||||
sqlx::query!(
|
||||
r#"
|
||||
UPDATE workspace_settings
|
||||
SET auto_invite = jsonb_build_object(
|
||||
'instance_groups', $2::jsonb,
|
||||
'instance_groups_roles', $3::jsonb
|
||||
)
|
||||
WHERE workspace_id = $1
|
||||
"#,
|
||||
"ws-multi-group",
|
||||
serde_json::to_value(&groups).unwrap(),
|
||||
&roles,
|
||||
)
|
||||
.execute(&db)
|
||||
.await
|
||||
.expect("Failed to update workspace settings");
|
||||
|
||||
// Alice was manually added as admin (no added_via)
|
||||
sqlx::query!(
|
||||
"INSERT INTO usr (workspace_id, username, email, is_admin, operator)
|
||||
VALUES ('ws-multi-group', 'alice', 'alice@example.com', true, false)",
|
||||
)
|
||||
.execute(&db)
|
||||
.await
|
||||
.expect("Failed to add user");
|
||||
|
||||
// The UPDATE with added_via->>'source' = 'instance_group' filter should NOT match
|
||||
let instance_group_source = json!({"source": "instance_group", "group": "engineering"});
|
||||
let result = sqlx::query!(
|
||||
"UPDATE usr SET is_admin = $1, operator = $2, added_via = $3 WHERE workspace_id = $4 AND email = $5 AND added_via->>'source' = 'instance_group'",
|
||||
false, true, &instance_group_source, "ws-multi-group", "alice@example.com"
|
||||
)
|
||||
.execute(&db)
|
||||
.await
|
||||
.expect("Failed to execute update");
|
||||
|
||||
assert_eq!(
|
||||
result.rows_affected(),
|
||||
0,
|
||||
"UPDATE should not affect manually-added users"
|
||||
);
|
||||
|
||||
let alice = sqlx::query!(
|
||||
"SELECT is_admin, operator, added_via FROM usr WHERE workspace_id = 'ws-multi-group' AND email = 'alice@example.com'"
|
||||
)
|
||||
.fetch_one(&db)
|
||||
.await
|
||||
.expect("Failed to query user");
|
||||
|
||||
assert!(alice.is_admin, "Manually-added admin should remain admin");
|
||||
assert!(
|
||||
!alice.operator,
|
||||
"Manually-added admin should not become operator"
|
||||
);
|
||||
assert!(
|
||||
alice.added_via.is_none(),
|
||||
"added_via should remain NULL for manual users"
|
||||
);
|
||||
|
||||
println!("✓ Manually-added users are not affected by instance group role updates");
|
||||
}
|
||||
|
||||
/// Test removing user from instance group removes them from workspace
|
||||
#[ignore = "requires database setup - run with --ignored flag"]
|
||||
#[sqlx::test(migrations = "../migrations", fixtures("base", "instance_group_auto_add"))]
|
||||
#[sqlx::test(
|
||||
migrations = "../migrations",
|
||||
fixtures("base", "instance_group_auto_add")
|
||||
)]
|
||||
async fn test_remove_user_from_instance_group(db: Pool<Postgres>) {
|
||||
// First, add alice to the workspace via engineering group
|
||||
let added_via = json!({"source": "instance_group", "group": "engineering"});
|
||||
@@ -416,7 +734,10 @@ mod tests {
|
||||
|
||||
/// Test that users added via domain are not affected by instance group removal
|
||||
#[ignore = "requires database setup - run with --ignored flag"]
|
||||
#[sqlx::test(migrations = "../migrations", fixtures("base", "instance_group_auto_add"))]
|
||||
#[sqlx::test(
|
||||
migrations = "../migrations",
|
||||
fixtures("base", "instance_group_auto_add")
|
||||
)]
|
||||
async fn test_domain_added_users_not_affected_by_group_removal(db: Pool<Postgres>) {
|
||||
// Add alice via domain (not instance group)
|
||||
let added_via = json!({"source": "domain", "domain": "example.com"});
|
||||
@@ -469,7 +790,10 @@ mod tests {
|
||||
|
||||
/// Test cleanup when instance group is removed from workspace configuration
|
||||
#[ignore = "requires database setup - run with --ignored flag"]
|
||||
#[sqlx::test(migrations = "../migrations", fixtures("base", "instance_group_auto_add"))]
|
||||
#[sqlx::test(
|
||||
migrations = "../migrations",
|
||||
fixtures("base", "instance_group_auto_add")
|
||||
)]
|
||||
async fn test_cleanup_removed_instance_groups(db: Pool<Postgres>) {
|
||||
// First, add users via engineering group
|
||||
for (username, email) in &[("alice", "alice@example.com"), ("bob", "bob@example.com")] {
|
||||
@@ -513,12 +837,11 @@ mod tests {
|
||||
// This should trigger cleanup of users added via that group
|
||||
|
||||
// Get all users in the engineering group
|
||||
let group_users = sqlx::query_scalar!(
|
||||
"SELECT email FROM email_to_igroup WHERE igroup = 'engineering'"
|
||||
)
|
||||
.fetch_all(&db)
|
||||
.await
|
||||
.expect("Failed to get group users");
|
||||
let group_users =
|
||||
sqlx::query_scalar!("SELECT email FROM email_to_igroup WHERE igroup = 'engineering'")
|
||||
.fetch_all(&db)
|
||||
.await
|
||||
.expect("Failed to get group users");
|
||||
|
||||
// Remove users who were added via engineering group
|
||||
for email in group_users {
|
||||
@@ -588,7 +911,10 @@ mod tests {
|
||||
|
||||
/// Test that users are not duplicated if already in workspace
|
||||
#[ignore = "requires database setup - run with --ignored flag"]
|
||||
#[sqlx::test(migrations = "../migrations", fixtures("base", "instance_group_auto_add"))]
|
||||
#[sqlx::test(
|
||||
migrations = "../migrations",
|
||||
fixtures("base", "instance_group_auto_add")
|
||||
)]
|
||||
async fn test_no_duplicate_users(db: Pool<Postgres>) {
|
||||
// Add alice to workspace first (without instance group tracking)
|
||||
sqlx::query!(
|
||||
@@ -636,7 +962,10 @@ mod tests {
|
||||
|
||||
/// Test workspace without auto-add configured is not affected
|
||||
#[ignore = "requires database setup - run with --ignored flag"]
|
||||
#[sqlx::test(migrations = "../migrations", fixtures("base", "instance_group_auto_add"))]
|
||||
#[sqlx::test(
|
||||
migrations = "../migrations",
|
||||
fixtures("base", "instance_group_auto_add")
|
||||
)]
|
||||
async fn test_workspace_without_auto_add_not_affected(db: Pool<Postgres>) {
|
||||
// ws-no-auto-add has no instance_groups configured
|
||||
|
||||
@@ -672,7 +1001,10 @@ mod tests {
|
||||
|
||||
/// Test querying workspaces configured with a specific instance group
|
||||
#[ignore = "requires database setup - run with --ignored flag"]
|
||||
#[sqlx::test(migrations = "../migrations", fixtures("base", "instance_group_auto_add"))]
|
||||
#[sqlx::test(
|
||||
migrations = "../migrations",
|
||||
fixtures("base", "instance_group_auto_add")
|
||||
)]
|
||||
async fn test_query_workspaces_with_instance_group(db: Pool<Postgres>) {
|
||||
// Configure ws-with-auto-add to use engineering group
|
||||
let groups = vec!["engineering".to_string()];
|
||||
@@ -708,7 +1040,11 @@ mod tests {
|
||||
.await
|
||||
.expect("Failed to query workspaces");
|
||||
|
||||
assert_eq!(workspaces.len(), 1, "Should find 1 workspace with engineering group");
|
||||
assert_eq!(
|
||||
workspaces.len(),
|
||||
1,
|
||||
"Should find 1 workspace with engineering group"
|
||||
);
|
||||
assert_eq!(workspaces[0].workspace_id, "ws-with-auto-add");
|
||||
|
||||
// Verify the role configuration is returned correctly
|
||||
|
||||
@@ -7,6 +7,7 @@
|
||||
*/
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::net::IpAddr;
|
||||
|
||||
use windmill_api_auth::{
|
||||
check_scopes, maybe_refresh_folders, require_owner_of_path, require_super_admin, ApiAuthed,
|
||||
@@ -297,7 +298,11 @@ async fn list_resources(
|
||||
}
|
||||
|
||||
if let Some(value) = &lq.value {
|
||||
sqlb.and_where("resource.value @> ?".bind(&value.replace("'", "''")));
|
||||
if let Ok(v) = serde_json::from_str::<serde_json::Value>(value) {
|
||||
sqlb.and_where("resource.value @> ?".bind(&v.to_string()));
|
||||
} else {
|
||||
sqlb.and_where("FALSE");
|
||||
}
|
||||
}
|
||||
|
||||
if let Some(broad_filter) = &lq.broad_filter {
|
||||
@@ -1767,9 +1772,74 @@ struct GitRepositoryResource {
|
||||
branch: Option<String>,
|
||||
}
|
||||
|
||||
/// Validates a git URL to prevent git option injection attacks.
|
||||
/// Git URLs starting with '-' could be interpreted as command-line options.
|
||||
fn validate_git_url(url: &str) -> Result<()> {
|
||||
/// Checks whether an IP address belongs to a private, loopback, link-local, or
|
||||
/// otherwise reserved range that should not be reachable from git operations.
|
||||
fn is_private_or_reserved_ip(ip: &IpAddr) -> bool {
|
||||
match ip {
|
||||
IpAddr::V4(v4) => {
|
||||
v4.is_loopback()
|
||||
|| v4.is_private()
|
||||
|| v4.is_link_local()
|
||||
|| v4.is_unspecified()
|
||||
|| v4.is_broadcast()
|
||||
// 100.64.0.0/10 (Carrier-grade NAT / CGNAT)
|
||||
|| (v4.octets()[0] == 100 && (v4.octets()[1] & 0xC0) == 64)
|
||||
}
|
||||
IpAddr::V6(v6) => {
|
||||
v6.is_loopback()
|
||||
|| v6.is_unspecified()
|
||||
// IPv4-mapped IPv6 (::ffff:x.x.x.x) — check the inner v4
|
||||
|| v6.to_ipv4_mapped().map_or(false, |v4| {
|
||||
is_private_or_reserved_ip(&IpAddr::V4(v4))
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Extracts the hostname from a git URL.
|
||||
///
|
||||
/// Handles standard URLs (`https://host/path`, `ssh://user@host/path`) and
|
||||
/// SCP-style (`user@host:path`).
|
||||
fn extract_host_from_git_url(url: &str) -> Option<String> {
|
||||
if let Some(after_scheme) = url.split("://").nth(1) {
|
||||
// Standard URL with scheme
|
||||
let host_part = match after_scheme.find('@') {
|
||||
Some(pos) => &after_scheme[pos + 1..],
|
||||
None => after_scheme,
|
||||
};
|
||||
// Handle IPv6 in brackets: [::1]
|
||||
if host_part.starts_with('[') {
|
||||
let end = host_part.find(']')?;
|
||||
let host = &host_part[1..end];
|
||||
return if host.is_empty() {
|
||||
None
|
||||
} else {
|
||||
Some(host.to_lowercase())
|
||||
};
|
||||
}
|
||||
let host_port = host_part.split('/').next()?;
|
||||
let host = host_port.rsplit_once(':').map_or(host_port, |(h, _)| h);
|
||||
if host.is_empty() {
|
||||
return None;
|
||||
}
|
||||
return Some(host.to_lowercase());
|
||||
}
|
||||
|
||||
// SCP-style: user@host:path
|
||||
if let Some(at_pos) = url.find('@') {
|
||||
let after_at = &url[at_pos + 1..];
|
||||
let host = after_at.split(':').next()?;
|
||||
if host.is_empty() {
|
||||
return None;
|
||||
}
|
||||
return Some(host.to_lowercase());
|
||||
}
|
||||
|
||||
None
|
||||
}
|
||||
|
||||
/// Validates a git URL to prevent option injection, SSRF, and local file read.
|
||||
async fn validate_git_url(url: &str) -> Result<()> {
|
||||
let url = url.trim();
|
||||
if url.is_empty() {
|
||||
return Err(Error::BadRequest("Git URL cannot be empty".to_string()));
|
||||
@@ -1779,12 +1849,59 @@ fn validate_git_url(url: &str) -> Result<()> {
|
||||
"Git URL cannot start with '-' (potential option injection)".to_string(),
|
||||
));
|
||||
}
|
||||
// Block other potentially dangerous patterns
|
||||
if url.contains('\0') || url.contains('\n') || url.contains('\r') {
|
||||
return Err(Error::BadRequest(
|
||||
"Git URL contains invalid characters".to_string(),
|
||||
));
|
||||
}
|
||||
|
||||
let lower = url.to_lowercase();
|
||||
|
||||
// Allowlist of URL formats — blocks file://, ftp://, local paths, etc.
|
||||
let has_valid_scheme = lower.starts_with("https://")
|
||||
|| lower.starts_with("http://")
|
||||
|| lower.starts_with("git://")
|
||||
|| lower.starts_with("ssh://");
|
||||
|
||||
// SCP-style: user@host:path (no scheme, has @ before :)
|
||||
let is_scp_style = !url.contains("://") && url.contains('@') && url.contains(':');
|
||||
|
||||
if !has_valid_scheme && !is_scp_style {
|
||||
return Err(Error::BadRequest(
|
||||
"Git URL must use https://, http://, git://, ssh://, or user@host:path format"
|
||||
.to_string(),
|
||||
));
|
||||
}
|
||||
|
||||
let host = extract_host_from_git_url(url)
|
||||
.ok_or_else(|| Error::BadRequest("Could not parse hostname from git URL".to_string()))?;
|
||||
|
||||
if host == "localhost" || host.ends_with(".local") || host == "[::1]" {
|
||||
return Err(Error::BadRequest(
|
||||
"Git URLs targeting localhost or local network are not allowed".to_string(),
|
||||
));
|
||||
}
|
||||
|
||||
// Check literal IP addresses
|
||||
if let Ok(ip) = host.parse::<IpAddr>() {
|
||||
if is_private_or_reserved_ip(&ip) {
|
||||
return Err(Error::BadRequest(
|
||||
"Git URLs targeting private or reserved IP addresses are not allowed".to_string(),
|
||||
));
|
||||
}
|
||||
} else {
|
||||
// Hostname — resolve via DNS and reject if any address is private
|
||||
if let Ok(addrs) = tokio::net::lookup_host(format!("{}:443", host)).await {
|
||||
for addr in addrs {
|
||||
if is_private_or_reserved_ip(&addr.ip()) {
|
||||
return Err(Error::BadRequest(
|
||||
"Git URL hostname resolves to a private or reserved IP address".to_string(),
|
||||
));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -1976,8 +2093,8 @@ async fn get_repo_latest_commit_hash(
|
||||
git_resource: &GitRepositoryResource,
|
||||
git_ssh_command: Option<String>,
|
||||
) -> Result<String> {
|
||||
// Validate URL and branch to prevent option injection attacks
|
||||
validate_git_url(&git_resource.url)?;
|
||||
// Validate URL and branch to prevent option injection and SSRF attacks
|
||||
validate_git_url(&git_resource.url).await?;
|
||||
|
||||
let ref_spec = git_resource
|
||||
.branch
|
||||
@@ -2141,4 +2258,149 @@ mod tests {
|
||||
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_extract_host_from_git_url() {
|
||||
// Standard HTTPS
|
||||
assert_eq!(
|
||||
extract_host_from_git_url("https://github.com/user/repo.git"),
|
||||
Some("github.com".to_string())
|
||||
);
|
||||
// HTTPS with port
|
||||
assert_eq!(
|
||||
extract_host_from_git_url("https://git.example.com:8443/repo.git"),
|
||||
Some("git.example.com".to_string())
|
||||
);
|
||||
// SSH with scheme
|
||||
assert_eq!(
|
||||
extract_host_from_git_url("ssh://git@github.com/user/repo.git"),
|
||||
Some("github.com".to_string())
|
||||
);
|
||||
// SCP-style
|
||||
assert_eq!(
|
||||
extract_host_from_git_url("git@github.com:user/repo.git"),
|
||||
Some("github.com".to_string())
|
||||
);
|
||||
// Git protocol
|
||||
assert_eq!(
|
||||
extract_host_from_git_url("git://example.com/repo.git"),
|
||||
Some("example.com".to_string())
|
||||
);
|
||||
// IPv6 in brackets
|
||||
assert_eq!(
|
||||
extract_host_from_git_url("http://[::1]:8080/repo.git"),
|
||||
Some("::1".to_string())
|
||||
);
|
||||
// No host extractable
|
||||
assert_eq!(extract_host_from_git_url("/local/path"), None);
|
||||
assert_eq!(
|
||||
extract_host_from_git_url("file:///etc/passwd"),
|
||||
Some("".to_string()).filter(|s| !s.is_empty())
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_is_private_or_reserved_ip() {
|
||||
use std::net::IpAddr;
|
||||
// Loopback
|
||||
assert!(is_private_or_reserved_ip(
|
||||
&"127.0.0.1".parse::<IpAddr>().unwrap()
|
||||
));
|
||||
assert!(is_private_or_reserved_ip(
|
||||
&"127.0.0.2".parse::<IpAddr>().unwrap()
|
||||
));
|
||||
// Private ranges
|
||||
assert!(is_private_or_reserved_ip(
|
||||
&"10.0.0.1".parse::<IpAddr>().unwrap()
|
||||
));
|
||||
assert!(is_private_or_reserved_ip(
|
||||
&"172.16.0.1".parse::<IpAddr>().unwrap()
|
||||
));
|
||||
assert!(is_private_or_reserved_ip(
|
||||
&"192.168.1.1".parse::<IpAddr>().unwrap()
|
||||
));
|
||||
// Link-local / cloud metadata
|
||||
assert!(is_private_or_reserved_ip(
|
||||
&"169.254.169.254".parse::<IpAddr>().unwrap()
|
||||
));
|
||||
// CGNAT
|
||||
assert!(is_private_or_reserved_ip(
|
||||
&"100.64.0.1".parse::<IpAddr>().unwrap()
|
||||
));
|
||||
// Unspecified
|
||||
assert!(is_private_or_reserved_ip(
|
||||
&"0.0.0.0".parse::<IpAddr>().unwrap()
|
||||
));
|
||||
// IPv6 loopback
|
||||
assert!(is_private_or_reserved_ip(&"::1".parse::<IpAddr>().unwrap()));
|
||||
// IPv4-mapped IPv6
|
||||
assert!(is_private_or_reserved_ip(
|
||||
&"::ffff:127.0.0.1".parse::<IpAddr>().unwrap()
|
||||
));
|
||||
// Public IPs should pass
|
||||
assert!(!is_private_or_reserved_ip(
|
||||
&"8.8.8.8".parse::<IpAddr>().unwrap()
|
||||
));
|
||||
assert!(!is_private_or_reserved_ip(
|
||||
&"140.82.121.4".parse::<IpAddr>().unwrap()
|
||||
));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_validate_git_url_blocks_file_scheme() {
|
||||
let result = validate_git_url("file:///etc/passwd").await;
|
||||
assert!(result.is_err());
|
||||
assert!(result.unwrap_err().to_string().contains("https://"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_validate_git_url_blocks_private_ips() {
|
||||
assert!(validate_git_url("http://127.0.0.1/repo.git").await.is_err());
|
||||
assert!(validate_git_url("http://169.254.169.254/latest/meta-data/")
|
||||
.await
|
||||
.is_err());
|
||||
assert!(validate_git_url("http://10.0.0.1/repo.git").await.is_err());
|
||||
assert!(validate_git_url("http://172.16.0.1/repo.git")
|
||||
.await
|
||||
.is_err());
|
||||
assert!(validate_git_url("http://192.168.1.1/repo.git")
|
||||
.await
|
||||
.is_err());
|
||||
assert!(validate_git_url("git://0.0.0.0/repo.git").await.is_err());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_validate_git_url_blocks_localhost() {
|
||||
assert!(validate_git_url("http://localhost/repo.git").await.is_err());
|
||||
assert!(validate_git_url("http://myhost.local/repo.git")
|
||||
.await
|
||||
.is_err());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_validate_git_url_blocks_local_paths() {
|
||||
assert!(validate_git_url("/etc/passwd").await.is_err());
|
||||
assert!(validate_git_url("../relative/path").await.is_err());
|
||||
assert!(validate_git_url("./local/repo").await.is_err());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_validate_git_url_allows_valid_urls() {
|
||||
// These should succeed (host resolution may fail but validation passes)
|
||||
assert!(validate_git_url("https://github.com/user/repo.git")
|
||||
.await
|
||||
.is_ok());
|
||||
assert!(validate_git_url("git@github.com:user/repo.git")
|
||||
.await
|
||||
.is_ok());
|
||||
assert!(validate_git_url("ssh://git@github.com/user/repo.git")
|
||||
.await
|
||||
.is_ok());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_validate_git_url_blocks_option_injection() {
|
||||
assert!(validate_git_url("-evil").await.is_err());
|
||||
assert!(validate_git_url("--upload-pack=evil").await.is_err());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -14,6 +14,7 @@ private = ["windmill-api/private"]
|
||||
enterprise = ["windmill-api/enterprise"]
|
||||
python = ["windmill-common/python"]
|
||||
deno_core = ["dep:windmill-runtime-nativets"]
|
||||
mcp = ["windmill-api/mcp"]
|
||||
agent_worker_server = ["dep:windmill-api-agent-workers"]
|
||||
run_inline = ["windmill-api/run_inline"]
|
||||
duckdb = ["windmill-worker/duckdb"]
|
||||
|
||||
@@ -81,20 +81,29 @@ pub struct ApiServer {
|
||||
|
||||
impl ApiServer {
|
||||
pub async fn start(db: Pool<Postgres>) -> anyhow::Result<Self> {
|
||||
Self::start_inner(db, false).await
|
||||
Self::start_inner(db, false, false).await
|
||||
}
|
||||
|
||||
pub async fn start_agent_mode(db: Pool<Postgres>) -> anyhow::Result<Self> {
|
||||
Self::start_inner(db, true).await
|
||||
Self::start_inner(db, true, false).await
|
||||
}
|
||||
|
||||
/// Start the API server with server_mode=true so trigger listeners are active.
|
||||
/// Alias for `start_agent_mode` with a clearer name for trigger e2e tests.
|
||||
pub async fn start_with_listeners(db: Pool<Postgres>) -> anyhow::Result<Self> {
|
||||
Self::start_inner(db, true).await
|
||||
Self::start_inner(db, true, false).await
|
||||
}
|
||||
|
||||
async fn start_inner(db: Pool<Postgres>, agent_mode: bool) -> anyhow::Result<Self> {
|
||||
/// Start the API server with mcp_mode=true so MCP routes are active.
|
||||
pub async fn start_mcp(db: Pool<Postgres>) -> anyhow::Result<Self> {
|
||||
Self::start_inner(db, false, true).await
|
||||
}
|
||||
|
||||
async fn start_inner(
|
||||
db: Pool<Postgres>,
|
||||
server_mode: bool,
|
||||
mcp_mode: bool,
|
||||
) -> anyhow::Result<Self> {
|
||||
let (tx, rx) = tokio::sync::broadcast::channel::<()>(1);
|
||||
|
||||
let listener = tokio::net::TcpListener::bind("127.0.0.1:0")
|
||||
@@ -114,8 +123,8 @@ impl ApiServer {
|
||||
listener,
|
||||
rx,
|
||||
port_tx,
|
||||
agent_mode,
|
||||
false,
|
||||
server_mode,
|
||||
mcp_mode,
|
||||
format!("http://localhost:{}", addr.port()),
|
||||
Some(name.clone()),
|
||||
));
|
||||
|
||||
@@ -45,3 +45,4 @@ itertools.workspace = true
|
||||
thiserror.workspace = true
|
||||
anyhow.workspace = true
|
||||
hex.workspace = true
|
||||
chrono.workspace = true
|
||||
|
||||
@@ -12,6 +12,23 @@ use sha1::Sha1;
|
||||
use sha2::{Sha256, Sha512};
|
||||
use std::{borrow::Cow, collections::HashMap};
|
||||
|
||||
const MAX_TIMESTAMP_AGE_SECS: i64 = 300; // 5 minutes
|
||||
|
||||
fn validate_unix_timestamp(timestamp_str: &str) -> Result<(), AuthenticationError> {
|
||||
let ts: i64 = timestamp_str
|
||||
.parse()
|
||||
.map_err(|_| AuthenticationError::InvalidTimestamp)?;
|
||||
let now = chrono::Utc::now().timestamp();
|
||||
let diff = now - ts;
|
||||
if diff > MAX_TIMESTAMP_AGE_SECS {
|
||||
return Err(AuthenticationError::TimestampTooOldError);
|
||||
}
|
||||
if diff < -MAX_TIMESTAMP_AGE_SECS {
|
||||
return Err(AuthenticationError::FutureTimestampError);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub type HmacSha256 = Hmac<Sha256>;
|
||||
pub type HmacSha512 = Hmac<Sha512>;
|
||||
pub type HmacSha1 = Hmac<Sha1>;
|
||||
@@ -82,6 +99,11 @@ mod slack {
|
||||
SignatureAuthenticationDetails::new(HmacAlgorithm::Sha256, Encoding::Hex),
|
||||
))
|
||||
}
|
||||
|
||||
fn validate_timestamp(&self, headers: &HeaderMap) -> Result<(), AuthenticationError> {
|
||||
let ts = headers.try_get_webhook_header("X-Slack-Request-Timestamp")?;
|
||||
validate_unix_timestamp(ts)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -126,6 +148,13 @@ mod stripe {
|
||||
SignatureAuthenticationDetails::new(HmacAlgorithm::Sha256, Encoding::Hex),
|
||||
))
|
||||
}
|
||||
|
||||
fn validate_timestamp(&self, headers: &HeaderMap) -> Result<(), AuthenticationError> {
|
||||
let sig_header = headers.try_get_webhook_header("STRIPE-SIGNATURE")?;
|
||||
let sig = parse_signature(sig_header, (",", "="));
|
||||
let ts = *sig.get("t").ok_or(AuthenticationError::InvalidTimestamp)?;
|
||||
validate_unix_timestamp(ts)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -170,6 +199,13 @@ mod tiktok {
|
||||
SignatureAuthenticationDetails::new(HmacAlgorithm::Sha256, Encoding::Hex),
|
||||
))
|
||||
}
|
||||
|
||||
fn validate_timestamp(&self, headers: &HeaderMap) -> Result<(), AuthenticationError> {
|
||||
let sig_header = headers.try_get_webhook_header("TikTok-Signature")?;
|
||||
let sig = parse_signature(sig_header, (",", "="));
|
||||
let ts = *sig.get("t").ok_or(AuthenticationError::InvalidTimestamp)?;
|
||||
validate_unix_timestamp(ts)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -244,6 +280,23 @@ mod twitch {
|
||||
|
||||
Ok(Some(response.into_response()))
|
||||
}
|
||||
|
||||
fn validate_timestamp(&self, headers: &HeaderMap) -> Result<(), AuthenticationError> {
|
||||
let ts_str = headers.try_get_webhook_header("Twitch-Eventsub-Message-Timestamp")?;
|
||||
let ts: chrono::DateTime<chrono::Utc> = chrono::DateTime::parse_from_rfc3339(ts_str)
|
||||
.map_err(|_| AuthenticationError::InvalidTimestamp)?
|
||||
.into();
|
||||
let now = chrono::Utc::now();
|
||||
let diff = (now - ts).num_seconds();
|
||||
// Twitch recommends 10 minutes tolerance
|
||||
if diff > 600 {
|
||||
return Err(AuthenticationError::TimestampTooOldError);
|
||||
}
|
||||
if diff < -600 {
|
||||
return Err(AuthenticationError::FutureTimestampError);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -321,6 +374,11 @@ mod zoom {
|
||||
SignatureAuthenticationDetails::new(HmacAlgorithm::Sha256, Encoding::Hex),
|
||||
))
|
||||
}
|
||||
|
||||
fn validate_timestamp(&self, headers: &HeaderMap) -> Result<(), AuthenticationError> {
|
||||
let ts = headers.try_get_webhook_header("x-zm-request-timestamp")?;
|
||||
validate_unix_timestamp(ts)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -397,6 +455,10 @@ pub trait WebhookHandler {
|
||||
headers: &'header HeaderMap,
|
||||
raw_payload: &'payload str,
|
||||
) -> Result<SignatureAuthenticationData<'payload, 'header, 'prefix>, AuthenticationError>;
|
||||
|
||||
fn validate_timestamp(&self, _headers: &HeaderMap) -> Result<(), AuthenticationError> {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, Debug, Serialize, Deserialize)]
|
||||
@@ -592,6 +654,10 @@ impl AuthenticationMethod {
|
||||
return Ok(Some(challenge_response));
|
||||
}
|
||||
|
||||
if let Some(handler) = handler {
|
||||
handler.validate_timestamp(headers)?;
|
||||
}
|
||||
|
||||
let authentication_data = match handler {
|
||||
Some(handler) => handler.get_hmac_authentication_data(headers, raw_payload)?,
|
||||
None => {
|
||||
@@ -621,7 +687,7 @@ impl AuthenticationMethod {
|
||||
let api_key_to_cmp = headers
|
||||
.try_get_webhook_header(&api_key_header)
|
||||
.map_err(|_| AuthenticationError::InvalidApiKey)?;
|
||||
if api_key_to_cmp != api_key_secret {
|
||||
if !constant_time_eq(api_key_to_cmp.as_bytes(), api_key_secret.as_bytes()) {
|
||||
return Err(AuthenticationError::InvalidApiKey);
|
||||
}
|
||||
}
|
||||
@@ -654,8 +720,11 @@ impl AuthenticationMethod {
|
||||
return Err(AuthenticationError::UnauthorizedBasicHttpAuth);
|
||||
}
|
||||
|
||||
if credentials.get(0).unwrap() != username
|
||||
|| credentials.get(1).unwrap() != password
|
||||
if !constant_time_eq(credentials.get(0).unwrap().as_bytes(), username.as_bytes())
|
||||
|| !constant_time_eq(
|
||||
credentials.get(1).unwrap().as_bytes(),
|
||||
password.as_bytes(),
|
||||
)
|
||||
{
|
||||
return Err(AuthenticationError::UnauthorizedBasicHttpAuth);
|
||||
}
|
||||
@@ -667,7 +736,6 @@ impl AuthenticationMethod {
|
||||
}
|
||||
|
||||
#[derive(thiserror::Error, Debug)]
|
||||
#[allow(unused)]
|
||||
pub enum AuthenticationError {
|
||||
#[error("failed to parse timestamp")]
|
||||
InvalidTimestamp,
|
||||
@@ -1207,11 +1275,15 @@ mod tests {
|
||||
headers
|
||||
}
|
||||
|
||||
fn current_timestamp() -> String {
|
||||
chrono::Utc::now().timestamp().to_string()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_slack_authenticate_valid() {
|
||||
let secret = "slack_signing_secret";
|
||||
let payload = "token=xxx&command=%2Ftest".to_string();
|
||||
let timestamp = "1531420618";
|
||||
let timestamp = ¤t_timestamp();
|
||||
let headers = slack_headers(secret, &payload, timestamp);
|
||||
|
||||
let method = AuthenticationMethod::Signature(SignatureAuthentication {
|
||||
@@ -1225,7 +1297,7 @@ mod tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_slack_authenticate_wrong_timestamp() {
|
||||
fn test_slack_authenticate_stale_timestamp_rejected() {
|
||||
let secret = "slack_secret";
|
||||
let payload = "data".to_string();
|
||||
let headers = slack_headers(secret, &payload, "1000000000");
|
||||
@@ -1235,10 +1307,10 @@ mod tests {
|
||||
secret_key: secret.to_string(),
|
||||
authentication_config: None,
|
||||
});
|
||||
// Constructed with timestamp "1000000000" but that's valid - it just needs to match
|
||||
assert!(method
|
||||
.authenticate_http_request(&headers, Some(&payload))
|
||||
.is_ok());
|
||||
assert!(matches!(
|
||||
method.authenticate_http_request(&headers, Some(&payload)),
|
||||
Err(AuthenticationError::TimestampTooOldError)
|
||||
));
|
||||
}
|
||||
|
||||
// --- Stripe webhook end-to-end ---
|
||||
@@ -1259,7 +1331,7 @@ mod tests {
|
||||
fn test_stripe_authenticate_valid() {
|
||||
let secret = "whsec_stripe_secret";
|
||||
let payload = r#"{"id":"evt_123"}"#.to_string();
|
||||
let timestamp = "1614556800";
|
||||
let timestamp = ¤t_timestamp();
|
||||
let headers = stripe_headers(secret, &payload, timestamp);
|
||||
|
||||
let method = AuthenticationMethod::Signature(SignatureAuthentication {
|
||||
@@ -1305,7 +1377,7 @@ mod tests {
|
||||
fn test_tiktok_authenticate_valid() {
|
||||
let secret = "tiktok_secret";
|
||||
let payload = r#"{"event":"video.upload"}"#.to_string();
|
||||
let timestamp = "1700000000";
|
||||
let timestamp = ¤t_timestamp();
|
||||
let headers = tiktok_headers(secret, &payload, timestamp);
|
||||
|
||||
let method = AuthenticationMethod::Signature(SignatureAuthentication {
|
||||
@@ -1350,17 +1422,16 @@ mod tests {
|
||||
headers
|
||||
}
|
||||
|
||||
fn current_rfc3339_timestamp() -> String {
|
||||
chrono::Utc::now().to_rfc3339()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_twitch_authenticate_valid_notification() {
|
||||
let secret = "twitch_secret";
|
||||
let payload = r#"{"subscription":{},"event":{"user_id":"123"}}"#.to_string();
|
||||
let headers = twitch_headers(
|
||||
secret,
|
||||
&payload,
|
||||
"msg-123",
|
||||
"2024-01-01T00:00:00Z",
|
||||
"notification",
|
||||
);
|
||||
let ts = current_rfc3339_timestamp();
|
||||
let headers = twitch_headers(secret, &payload, "msg-123", &ts, "notification");
|
||||
|
||||
let method = AuthenticationMethod::Signature(SignatureAuthentication {
|
||||
signature_provider: WebhookType::Twitch,
|
||||
@@ -1376,11 +1447,12 @@ mod tests {
|
||||
fn test_twitch_challenge_response() {
|
||||
let secret = "twitch_secret";
|
||||
let payload = r#"{"challenge":"test_challenge_string","subscription":{"id":"sub-123"}}"#;
|
||||
let ts = current_rfc3339_timestamp();
|
||||
let headers = twitch_headers(
|
||||
secret,
|
||||
payload,
|
||||
"msg-456",
|
||||
"2024-01-01T00:00:00Z",
|
||||
&ts,
|
||||
"webhook_callback_verification",
|
||||
);
|
||||
|
||||
@@ -1396,13 +1468,8 @@ mod tests {
|
||||
fn test_twitch_non_challenge_returns_none() {
|
||||
let secret = "twitch_secret";
|
||||
let payload = r#"{"subscription":{},"event":{}}"#;
|
||||
let headers = twitch_headers(
|
||||
secret,
|
||||
payload,
|
||||
"msg-789",
|
||||
"2024-01-01T00:00:00Z",
|
||||
"notification",
|
||||
);
|
||||
let ts = current_rfc3339_timestamp();
|
||||
let headers = twitch_headers(secret, payload, "msg-789", &ts, "notification");
|
||||
|
||||
let handler = WebhookType::Twitch.get_webhook_handler().unwrap();
|
||||
let config_data = SignatureConfigData { secret_key: secret };
|
||||
@@ -1434,7 +1501,7 @@ mod tests {
|
||||
fn test_zoom_authenticate_valid() {
|
||||
let secret = "zoom_secret";
|
||||
let payload = r#"{"event":"meeting.started"}"#.to_string();
|
||||
let timestamp = "1700000000";
|
||||
let timestamp = ¤t_timestamp();
|
||||
let headers = zoom_headers(secret, &payload, timestamp);
|
||||
|
||||
let method = AuthenticationMethod::Signature(SignatureAuthentication {
|
||||
@@ -1789,4 +1856,84 @@ mod tests {
|
||||
let response = AuthenticationError::InvalidTimestamp.into_response();
|
||||
assert_eq!(response.status(), StatusCode::BAD_REQUEST);
|
||||
}
|
||||
|
||||
// --- validate_unix_timestamp ---
|
||||
|
||||
#[test]
|
||||
fn test_validate_unix_timestamp_current() {
|
||||
let now = chrono::Utc::now().timestamp().to_string();
|
||||
assert!(validate_unix_timestamp(&now).is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_validate_unix_timestamp_recent() {
|
||||
let ts = (chrono::Utc::now().timestamp() - 60).to_string();
|
||||
assert!(validate_unix_timestamp(&ts).is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_validate_unix_timestamp_too_old() {
|
||||
let ts = (chrono::Utc::now().timestamp() - 600).to_string();
|
||||
assert!(matches!(
|
||||
validate_unix_timestamp(&ts),
|
||||
Err(AuthenticationError::TimestampTooOldError)
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_validate_unix_timestamp_future() {
|
||||
let ts = (chrono::Utc::now().timestamp() + 600).to_string();
|
||||
assert!(matches!(
|
||||
validate_unix_timestamp(&ts),
|
||||
Err(AuthenticationError::FutureTimestampError)
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_validate_unix_timestamp_invalid() {
|
||||
assert!(matches!(
|
||||
validate_unix_timestamp("not-a-number"),
|
||||
Err(AuthenticationError::InvalidTimestamp)
|
||||
));
|
||||
}
|
||||
|
||||
// --- Slack timestamp validation ---
|
||||
|
||||
#[test]
|
||||
fn test_slack_validate_timestamp_current() {
|
||||
let handler = slack::Slack;
|
||||
let mut headers = HeaderMap::new();
|
||||
let now = chrono::Utc::now().timestamp().to_string();
|
||||
headers.insert("X-Slack-Request-Timestamp", now.parse().unwrap());
|
||||
assert!(handler.validate_timestamp(&headers).is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_slack_validate_timestamp_stale() {
|
||||
let handler = slack::Slack;
|
||||
let mut headers = HeaderMap::new();
|
||||
let old = (chrono::Utc::now().timestamp() - 600).to_string();
|
||||
headers.insert("X-Slack-Request-Timestamp", old.parse().unwrap());
|
||||
assert!(handler.validate_timestamp(&headers).is_err());
|
||||
}
|
||||
|
||||
// --- Twitch timestamp validation (ISO 8601) ---
|
||||
|
||||
#[test]
|
||||
fn test_twitch_validate_timestamp_current() {
|
||||
let handler = twitch::Twitch;
|
||||
let mut headers = HeaderMap::new();
|
||||
let now = chrono::Utc::now().to_rfc3339();
|
||||
headers.insert("Twitch-Eventsub-Message-Timestamp", now.parse().unwrap());
|
||||
assert!(handler.validate_timestamp(&headers).is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_twitch_validate_timestamp_stale() {
|
||||
let handler = twitch::Twitch;
|
||||
let mut headers = HeaderMap::new();
|
||||
let old = (chrono::Utc::now() - chrono::TimeDelta::seconds(1200)).to_rfc3339();
|
||||
headers.insert("Twitch-Eventsub-Message-Timestamp", old.parse().unwrap());
|
||||
assert!(handler.validate_timestamp(&headers).is_err());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -36,6 +36,7 @@ impl TriggerCrud for WebsocketTrigger {
|
||||
const ADDITIONAL_SELECT_FIELDS: &[&'static str] = &[
|
||||
"url",
|
||||
"filters",
|
||||
"filter_logic",
|
||||
"initial_messages",
|
||||
"url_runnable_args",
|
||||
"can_return_message",
|
||||
@@ -103,6 +104,7 @@ impl TriggerCrud for WebsocketTrigger {
|
||||
is_flow,
|
||||
mode,
|
||||
filters,
|
||||
filter_logic,
|
||||
initial_messages,
|
||||
url_runnable_args,
|
||||
edited_by,
|
||||
@@ -114,7 +116,7 @@ impl TriggerCrud for WebsocketTrigger {
|
||||
error_handler_args,
|
||||
retry
|
||||
) VALUES (
|
||||
$1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, now(), $14, $15, $16
|
||||
$1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, now(), $15, $16, $17
|
||||
)
|
||||
"#,
|
||||
w_id,
|
||||
@@ -124,6 +126,7 @@ impl TriggerCrud for WebsocketTrigger {
|
||||
trigger.base.is_flow,
|
||||
trigger.base.mode() as _,
|
||||
&filters as _,
|
||||
trigger.config.filter_logic,
|
||||
&initial_messages as _,
|
||||
trigger
|
||||
.config
|
||||
@@ -178,26 +181,28 @@ impl TriggerCrud for WebsocketTrigger {
|
||||
path = $3,
|
||||
is_flow = $4,
|
||||
filters = $5,
|
||||
initial_messages = $6,
|
||||
url_runnable_args = $7,
|
||||
edited_by = $8,
|
||||
permissioned_as = $9,
|
||||
can_return_message = $10,
|
||||
can_return_error_result = $11,
|
||||
filter_logic = $6,
|
||||
initial_messages = $7,
|
||||
url_runnable_args = $8,
|
||||
edited_by = $9,
|
||||
permissioned_as = $10,
|
||||
can_return_message = $11,
|
||||
can_return_error_result = $12,
|
||||
edited_at = now(),
|
||||
server_id = NULL,
|
||||
error = NULL,
|
||||
error_handler_path = $14,
|
||||
error_handler_args = $15,
|
||||
retry = $16
|
||||
error_handler_path = $15,
|
||||
error_handler_args = $16,
|
||||
retry = $17
|
||||
WHERE
|
||||
workspace_id = $12 AND path = $13
|
||||
workspace_id = $13 AND path = $14
|
||||
",
|
||||
trigger.config.url,
|
||||
trigger.base.script_path,
|
||||
trigger.base.path,
|
||||
trigger.base.is_flow,
|
||||
filters.as_slice() as &[SqlxJson<Box<RawValue>>],
|
||||
trigger.config.filter_logic,
|
||||
initial_messages.as_slice() as &[SqlxJson<Box<RawValue>>],
|
||||
trigger
|
||||
.config
|
||||
|
||||
@@ -1,12 +1,9 @@
|
||||
use std::collections::HashMap;
|
||||
|
||||
use windmill_api_auth::ApiAuthed;
|
||||
use windmill_trigger::trigger_helpers::{
|
||||
trigger_runnable_and_wait_for_raw_result_with_error_ctx, TriggerJobArgs,
|
||||
};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use serde_json::value::RawValue;
|
||||
use sqlx::{types::Json as SqlxJson, FromRow};
|
||||
use windmill_api_auth::ApiAuthed;
|
||||
use windmill_common::{
|
||||
error::{Error, Result},
|
||||
jobs::JobTriggerKind,
|
||||
@@ -15,6 +12,9 @@ use windmill_common::{
|
||||
DB,
|
||||
};
|
||||
use windmill_queue::PushArgsOwned;
|
||||
use windmill_trigger::trigger_helpers::{
|
||||
trigger_runnable_and_wait_for_raw_result_with_error_ctx, TriggerJobArgs,
|
||||
};
|
||||
|
||||
pub mod handler;
|
||||
pub mod listener;
|
||||
@@ -30,11 +30,17 @@ impl TriggerJobArgs for WebsocketTrigger {
|
||||
}
|
||||
}
|
||||
|
||||
fn default_filter_logic() -> String {
|
||||
"and".to_string()
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, FromRow, Serialize, Deserialize)]
|
||||
pub struct WebsocketConfig {
|
||||
pub url: String,
|
||||
#[serde(default)]
|
||||
pub filters: Vec<SqlxJson<Box<RawValue>>>,
|
||||
#[serde(default = "default_filter_logic")]
|
||||
pub filter_logic: String,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub initial_messages: Option<Vec<SqlxJson<Box<RawValue>>>>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
@@ -49,6 +55,8 @@ pub struct WebsocketConfig {
|
||||
pub struct WebsocketConfigRequest {
|
||||
url: String,
|
||||
filters: Vec<serde_json::Value>,
|
||||
#[serde(default = "default_filter_logic")]
|
||||
filter_logic: String,
|
||||
initial_messages: Option<Vec<serde_json::Value>>,
|
||||
url_runnable_args: Option<serde_json::Value>,
|
||||
can_return_message: bool,
|
||||
|
||||
@@ -18,7 +18,7 @@ use windmill_common::{
|
||||
DB,
|
||||
};
|
||||
use windmill_queue::PushArgsOwned;
|
||||
use windmill_trigger::filter::{is_value_superset, Filter, JsonFilter};
|
||||
use windmill_trigger::filter::{check_filters, Filter};
|
||||
use windmill_trigger::listener::ListeningTrigger;
|
||||
use windmill_trigger::trigger_helpers::{
|
||||
trigger_runnable, trigger_runnable_and_wait_for_raw_result,
|
||||
@@ -267,26 +267,8 @@ impl Listener for WebsocketTrigger {
|
||||
match msg {
|
||||
tokio_tungstenite::tungstenite::Message::Text(text) => {
|
||||
tracing::debug!("Received text message from WebSocket {}: {}", url, text);
|
||||
let mut should_handle = true;
|
||||
for filter in &filters {
|
||||
match filter {
|
||||
Filter::JsonFilter(JsonFilter { key, value }) => {
|
||||
let mut deserializer = serde_json::Deserializer::from_str(text.as_str());
|
||||
should_handle = match is_value_superset(&mut deserializer, key, &value) {
|
||||
Ok(filter_match) => {
|
||||
filter_match
|
||||
},
|
||||
Err(err) => {
|
||||
tracing::warn!("Error deserializing filter for WebSocket {}: {:?}", url, err);
|
||||
false
|
||||
}
|
||||
};
|
||||
}
|
||||
}
|
||||
if !should_handle {
|
||||
break;
|
||||
}
|
||||
}
|
||||
let use_or = listening_trigger.trigger_config.filter_logic == "or";
|
||||
let should_handle = check_filters(&text, &filters, use_or);
|
||||
if should_handle {
|
||||
let trigger_info = HashMap::from([
|
||||
("url".to_string(), to_raw_value(&listening_trigger.trigger_config.url)),
|
||||
|
||||
@@ -80,6 +80,27 @@ where
|
||||
deserializer.deserialize_map(SupersetVisitor { key, value_to_check })
|
||||
}
|
||||
|
||||
pub fn check_filters(text: &str, filters: &[Filter], use_or_logic: bool) -> bool {
|
||||
if filters.is_empty() {
|
||||
return true;
|
||||
}
|
||||
|
||||
let check = |filter: &Filter| -> bool {
|
||||
match filter {
|
||||
Filter::JsonFilter(JsonFilter { key, value }) => {
|
||||
let mut deserializer = serde_json::Deserializer::from_str(text);
|
||||
is_value_superset(&mut deserializer, key, value).unwrap_or(false)
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
if use_or_logic {
|
||||
filters.iter().any(check)
|
||||
} else {
|
||||
filters.iter().all(check)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
@@ -450,6 +450,8 @@ pub struct ScriptHistory {
|
||||
pub script_hash: ScriptHash,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub deployment_msg: Option<String>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub created_at: Option<chrono::DateTime<chrono::Utc>>,
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
|
||||
@@ -84,8 +84,27 @@ pub async fn composer_install(
|
||||
) -> Result<String> {
|
||||
check_executor_binary_exists("php", PHP_PATH.as_str(), "php")?;
|
||||
|
||||
// When a lock file is available the dependency set is fully pinned, so we
|
||||
// can cache the installed vendor/ directory and reuse it across executions.
|
||||
// When no lock is provided (previews), try to reuse a previously resolved
|
||||
// lockfile from the DB so we can hit the same vendor cache as deployed scripts.
|
||||
let lock = if lock.is_none() && !*COMPOSER_VENDOR_CACHE_DISABLED {
|
||||
let req_hash = format!("composer-{}", calculate_hash(&requirements));
|
||||
if let Some(db) = conn.as_sql() {
|
||||
sqlx::query_scalar!(
|
||||
"SELECT lockfile FROM pip_resolution_cache WHERE hash = $1",
|
||||
req_hash
|
||||
)
|
||||
.fetch_optional(db)
|
||||
.await
|
||||
.ok()
|
||||
.flatten()
|
||||
} else {
|
||||
None
|
||||
}
|
||||
} else {
|
||||
lock
|
||||
};
|
||||
|
||||
// Cache the installed vendor/ directory keyed by requirements + lock content.
|
||||
// Set COMPOSER_VENDOR_CACHE_DISABLED=1 to opt out.
|
||||
let vendor_cache_hit = if !*COMPOSER_VENDOR_CACHE_DISABLED {
|
||||
if let Some(ref lock_content) = lock {
|
||||
@@ -171,6 +190,9 @@ pub async fn composer_install(
|
||||
)
|
||||
.await?;
|
||||
|
||||
// lock was `None` means composer resolved deps from scratch (no lock from
|
||||
// caller or DB). This is the only case where we should update the DB cache.
|
||||
let freshly_resolved = lock.is_none();
|
||||
let resolved_lock = match lock {
|
||||
Some(l) => l,
|
||||
None => {
|
||||
@@ -195,6 +217,27 @@ pub async fn composer_install(
|
||||
{
|
||||
tracing::warn!("Could not save composer vendor dir to cache: {e:?}");
|
||||
}
|
||||
|
||||
// Cache the resolved lockfile in the DB so future previews (which lack a
|
||||
// lock file) can look it up by requirements hash and hit the same vendor
|
||||
// cache. TTL of 7 days keeps previews reasonably fresh.
|
||||
// Only write when composer resolved from scratch (no lock from caller or
|
||||
// DB) to avoid endlessly refreshing the TTL on stale resolutions.
|
||||
if freshly_resolved {
|
||||
let req_hash = format!("composer-{}", calculate_hash(&requirements));
|
||||
if let Some(db) = conn.as_sql() {
|
||||
if let Err(e) = sqlx::query!(
|
||||
"INSERT INTO pip_resolution_cache (hash, lockfile, expiration) VALUES ($1, $2, now() + ('7 days')::interval) ON CONFLICT (hash) DO UPDATE SET lockfile = EXCLUDED.lockfile, expiration = EXCLUDED.expiration",
|
||||
req_hash,
|
||||
&resolved_lock
|
||||
)
|
||||
.execute(db)
|
||||
.await
|
||||
{
|
||||
tracing::warn!("Could not cache composer lockfile resolution: {e:?}");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Ok(format!(
|
||||
|
||||
@@ -2,7 +2,7 @@ import { sleep } from "https://deno.land/x/sleep@v1.2.1/mod.ts";
|
||||
import * as windmill from "https://deno.land/x/windmill@v1.174.0/mod.ts";
|
||||
import * as api from "https://deno.land/x/windmill@v1.174.0/windmill-api/index.ts";
|
||||
|
||||
export const VERSION = "v1.668.0";
|
||||
export const VERSION = "v1.669.1";
|
||||
|
||||
export async function login(email: string, password: string): Promise<string> {
|
||||
return await windmill.UserService.login({
|
||||
|
||||
@@ -166,8 +166,10 @@ export async function createBundle(
|
||||
// Dynamically import esbuild
|
||||
const esbuild = await import("esbuild");
|
||||
|
||||
// Detect frameworks to determine default entry point
|
||||
const frameworks = detectFrameworks(process.cwd());
|
||||
// Detect frameworks to determine default entry point.
|
||||
// Use the entryPoint's directory if provided, otherwise fall back to cwd.
|
||||
const appDir = options.entryPoint ? path.dirname(options.entryPoint) : process.cwd();
|
||||
const frameworks = detectFrameworks(appDir);
|
||||
const defaultEntry = (frameworks.svelte || frameworks.vue) ? "index.ts" : "index.tsx";
|
||||
|
||||
const entryPoint = options.entryPoint ?? defaultEntry;
|
||||
@@ -184,7 +186,6 @@ export async function createBundle(
|
||||
}
|
||||
|
||||
// Ensure node_modules exists in the app directory
|
||||
const appDir = path.dirname(entryPoint) || process.cwd();
|
||||
await ensureNodeModules(appDir);
|
||||
|
||||
// Load framework-specific plugins (svelte, vue) based on package.json
|
||||
|
||||
@@ -6,7 +6,7 @@ import { colors } from "@cliffy/ansi/colors";
|
||||
import * as log from "../../core/log.ts";
|
||||
import { yamlParseFile } from "../../utils/yaml.ts";
|
||||
import { GlobalOptions } from "../../types.ts";
|
||||
import { createBundle } from "./bundle.ts";
|
||||
import { createBundle, detectFrameworks } from "./bundle.ts";
|
||||
import { APP_BACKEND_FOLDER } from "./app_metadata.ts";
|
||||
import { loadRunnablesFromBackend } from "./raw_apps.ts";
|
||||
import {
|
||||
@@ -113,7 +113,11 @@ async function validateBuild(
|
||||
log.info(colors.blue("🔨 Testing build..."));
|
||||
|
||||
// Try to create a bundle - this will validate that all dependencies are in place
|
||||
const frameworks = detectFrameworks(appDir);
|
||||
const entryFile = frameworks.svelte || frameworks.vue ? "index.ts" : "index.tsx";
|
||||
const entryPoint = path.join(appDir, entryFile);
|
||||
await createBundle({
|
||||
entryPoint,
|
||||
production: true,
|
||||
minify: false,
|
||||
});
|
||||
|
||||
@@ -42,6 +42,13 @@ async function list(
|
||||
log.info("No audit logs found.");
|
||||
return;
|
||||
}
|
||||
if (logs.every((l) => l.operation === "redacted")) {
|
||||
log.info(colors.yellow(
|
||||
"Audit log details are not available on the Community Edition.\n" +
|
||||
"Upgrade to the Enterprise Edition for full audit logging with operation details."
|
||||
));
|
||||
return;
|
||||
}
|
||||
new Table()
|
||||
.header(["ID", "Timestamp", "Username", "Operation", "Action", "Resource"])
|
||||
.padding(2)
|
||||
|
||||
@@ -43,70 +43,66 @@ export async function pushWorkspaceDependencies(
|
||||
_befObj: any,
|
||||
newDependenciesContent: string,
|
||||
): Promise<void> {
|
||||
try {
|
||||
const res = workspaceDependenciesPathToLanguageAndFilename(path);
|
||||
if (!res) {
|
||||
throw new Error(`Unknown workspace dependencies file format: ${path}`);
|
||||
}
|
||||
|
||||
const { language, name } = res;
|
||||
|
||||
const displayName = name
|
||||
? `named dependencies "${name}"`
|
||||
: `workspace default dependencies`;
|
||||
|
||||
// Fetch remote workspace dependencies and compare content directly
|
||||
try {
|
||||
const remoteDeps = await wmill.getLatestWorkspaceDependencies({
|
||||
workspace,
|
||||
language,
|
||||
name,
|
||||
});
|
||||
|
||||
if (remoteDeps && remoteDeps.content === newDependenciesContent) {
|
||||
log.info(
|
||||
colors.green(
|
||||
`${displayName} for ${language} are up-to-date, skipping push`,
|
||||
),
|
||||
);
|
||||
return;
|
||||
}
|
||||
} catch (e: any) {
|
||||
// If 404 or not found, the dependency doesn't exist remotely yet - proceed with push
|
||||
if (e.status !== 404 && !e.message?.includes("not found")) {
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
|
||||
log.info(
|
||||
colors.yellow(
|
||||
`Pushing ${
|
||||
name ? "named" : "workspace default"
|
||||
} dependencies for ${language}...`,
|
||||
),
|
||||
const res = workspaceDependenciesPathToLanguageAndFilename(path);
|
||||
if (!res) {
|
||||
throw new Error(
|
||||
`Unknown workspace dependencies file format: ${path}. ` +
|
||||
`Valid files: package.json, requirements.in, composer.json, go.mod, modules.json`
|
||||
);
|
||||
}
|
||||
|
||||
await wmill.createWorkspaceDependencies({
|
||||
const { language, name } = res;
|
||||
|
||||
const displayName = name
|
||||
? `named dependencies "${name}"`
|
||||
: `workspace default dependencies`;
|
||||
|
||||
// Fetch remote workspace dependencies and compare content directly
|
||||
try {
|
||||
const remoteDeps = await wmill.getLatestWorkspaceDependencies({
|
||||
workspace,
|
||||
requestBody: {
|
||||
name,
|
||||
content: newDependenciesContent,
|
||||
language,
|
||||
workspace_id: workspace,
|
||||
// Description is not supported in cli, it will use old description
|
||||
description: undefined,
|
||||
},
|
||||
language,
|
||||
name,
|
||||
});
|
||||
|
||||
log.info(
|
||||
colors.green(`Successfully pushed ${displayName} for ${language}`),
|
||||
);
|
||||
} catch (error: any) {
|
||||
log.error(
|
||||
colors.red(`Failed to push workspace dependencies: ${error.message}`),
|
||||
);
|
||||
throw error;
|
||||
if (remoteDeps && remoteDeps.content === newDependenciesContent) {
|
||||
log.info(
|
||||
colors.green(
|
||||
`${displayName} for ${language} are up-to-date, skipping push`,
|
||||
),
|
||||
);
|
||||
return;
|
||||
}
|
||||
} catch (e: any) {
|
||||
// If 404 or not found, the dependency doesn't exist remotely yet - proceed with push
|
||||
if (e.status !== 404 && !e.message?.includes("not found")) {
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
|
||||
log.info(
|
||||
colors.yellow(
|
||||
`Pushing ${
|
||||
name ? "named" : "workspace default"
|
||||
} dependencies for ${language}...`,
|
||||
),
|
||||
);
|
||||
|
||||
await wmill.createWorkspaceDependencies({
|
||||
workspace,
|
||||
requestBody: {
|
||||
name,
|
||||
content: newDependenciesContent,
|
||||
language,
|
||||
workspace_id: workspace,
|
||||
// Description is not supported in cli, it will use old description
|
||||
description: undefined,
|
||||
},
|
||||
});
|
||||
|
||||
log.info(
|
||||
colors.green(`Successfully pushed ${displayName} for ${language}`),
|
||||
);
|
||||
}
|
||||
|
||||
export default command;
|
||||
|
||||
@@ -7,6 +7,7 @@ import * as log from "../../core/log.ts";
|
||||
import { sep as SEP } from "node:path";
|
||||
import { stringify as yamlStringify } from "yaml";
|
||||
import { yamlParseFile } from "../../utils/yaml.ts";
|
||||
import { validateRequiredArgs } from "../../utils/utils.ts";
|
||||
import * as wmill from "../../../gen/services.gen.ts";
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { mkdirSync, writeFileSync } from "node:fs";
|
||||
@@ -153,18 +154,27 @@ export async function pushFlow(
|
||||
const localFlow = (await yamlParseFile(localPath + "flow.yaml")) as FlowFile;
|
||||
|
||||
const fileReader = async (path: string) => await readFile(localPath + path, "utf-8");
|
||||
const missingFiles: string[] = [];
|
||||
await replaceInlineScripts(
|
||||
localFlow.value.modules,
|
||||
fileReader,
|
||||
log,
|
||||
localPath,
|
||||
SEP
|
||||
SEP,
|
||||
undefined,
|
||||
missingFiles
|
||||
);
|
||||
if (localFlow.value.failure_module) {
|
||||
await replaceInlineScripts([localFlow.value.failure_module], fileReader, log, localPath, SEP);
|
||||
await replaceInlineScripts([localFlow.value.failure_module], fileReader, log, localPath, SEP, undefined, missingFiles);
|
||||
}
|
||||
if (localFlow.value.preprocessor_module) {
|
||||
await replaceInlineScripts([localFlow.value.preprocessor_module], fileReader, log, localPath, SEP);
|
||||
await replaceInlineScripts([localFlow.value.preprocessor_module], fileReader, log, localPath, SEP, undefined, missingFiles);
|
||||
}
|
||||
if (missingFiles.length > 0) {
|
||||
log.warn(colors.yellow(
|
||||
`Warning: missing inline script file(s): ${missingFiles.join(", ")}. ` +
|
||||
`The flow will be pushed with unresolved !inline references.`
|
||||
));
|
||||
}
|
||||
|
||||
if (flow) {
|
||||
@@ -204,14 +214,14 @@ export async function pushFlow(
|
||||
|
||||
type Options = GlobalOptions;
|
||||
|
||||
async function push(opts: Options, filePath: string, remotePath: string) {
|
||||
async function push(opts: Options & { message?: string }, filePath: string, remotePath: string) {
|
||||
if (!validatePath(remotePath)) {
|
||||
return;
|
||||
}
|
||||
const workspace = await resolveWorkspace(opts);
|
||||
await requireLogin(opts);
|
||||
|
||||
await pushFlow(workspace.workspaceId, remotePath, filePath);
|
||||
await pushFlow(workspace.workspaceId, remotePath, filePath, opts.message);
|
||||
log.info(colors.bold.underline.green("Flow pushed"));
|
||||
}
|
||||
|
||||
@@ -252,6 +262,7 @@ async function list(
|
||||
}
|
||||
}
|
||||
async function get(opts: GlobalOptions & { json?: boolean }, path: string) {
|
||||
if (opts.json) log.setSilent(true);
|
||||
const workspace = await resolveWorkspace(opts);
|
||||
await requireLogin(opts);
|
||||
const f = await wmill.getFlowByPath({
|
||||
@@ -270,11 +281,26 @@ async function get(opts: GlobalOptions & { json?: boolean }, path: string) {
|
||||
const modules = (f as any).value?.modules;
|
||||
if (modules && Array.isArray(modules) && modules.length > 0) {
|
||||
console.log(colors.bold("Steps:"));
|
||||
for (const mod of modules) {
|
||||
const type = mod.value?.type ?? "unknown";
|
||||
const detail = mod.value?.language ?? mod.value?.path ?? "";
|
||||
console.log(` ${mod.id}: ${type}${detail ? " (" + detail + ")" : ""}`);
|
||||
function printModules(mods: any[], indent: string = " ") {
|
||||
for (const mod of mods) {
|
||||
const type = mod.value?.type ?? "unknown";
|
||||
const detail = mod.value?.language ?? mod.value?.path ?? "";
|
||||
console.log(`${indent}${mod.id}: ${type}${detail ? " (" + detail + ")" : ""}`);
|
||||
if (type === "branchall" || type === "branchone") {
|
||||
for (const branch of mod.value?.branches ?? []) {
|
||||
console.log(`${indent} Branch: ${branch.summary || "(default)"}`);
|
||||
if (branch.modules) printModules(branch.modules, indent + " ");
|
||||
}
|
||||
if (type === "branchone" && mod.value?.default) {
|
||||
console.log(`${indent} Default:`);
|
||||
printModules(mod.value.default, indent + " ");
|
||||
}
|
||||
} else if (type === "forloopflow" || type === "whileloopflow") {
|
||||
if (mod.value?.modules) printModules(mod.value.modules, indent + " ");
|
||||
}
|
||||
}
|
||||
}
|
||||
printModules(modules);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -294,6 +320,20 @@ async function run(
|
||||
|
||||
const input = opts.data ? await resolve(opts.data) : {};
|
||||
|
||||
// Validate required args against schema when no data provided
|
||||
if (!opts.data) {
|
||||
try {
|
||||
const flow = await wmill.getFlowByPath({
|
||||
workspace: workspace.workspaceId,
|
||||
path,
|
||||
});
|
||||
validateRequiredArgs(flow.schema as Record<string, unknown>);
|
||||
} catch (e: any) {
|
||||
if (e.message?.startsWith("Missing required")) throw e;
|
||||
log.warn(`Could not fetch schema to validate args: ${e.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
const id = await wmill.runFlowByPath({
|
||||
workspace: workspace.workspaceId,
|
||||
path,
|
||||
@@ -328,18 +368,39 @@ async function run(
|
||||
i++;
|
||||
}
|
||||
|
||||
if (!opts.silent) {
|
||||
log.info(colors.green.underline.bold("Flow ran to completion"));
|
||||
log.info("\n");
|
||||
// Wait for flow completion with retry (handles race when --silent skips module tracking)
|
||||
const MAX_RETRIES = 600; // ~60 seconds at 100ms intervals
|
||||
let retries = 0;
|
||||
while (retries < MAX_RETRIES) {
|
||||
try {
|
||||
const jobInfo = await wmill.getCompletedJob({
|
||||
workspace: workspace.workspaceId,
|
||||
id,
|
||||
});
|
||||
|
||||
if (!opts.silent) {
|
||||
log.info(colors.green.underline.bold("Flow ran to completion"));
|
||||
log.info("\n");
|
||||
}
|
||||
|
||||
if (jobInfo.success === false) {
|
||||
process.exitCode = 1;
|
||||
}
|
||||
|
||||
if (opts.silent) {
|
||||
console.log(JSON.stringify(jobInfo.result ?? {}));
|
||||
} else {
|
||||
log.info(JSON.stringify(jobInfo.result ?? {}, null, 2));
|
||||
}
|
||||
|
||||
break;
|
||||
} catch {
|
||||
retries++;
|
||||
await new Promise((resolve) => setTimeout(resolve, 100));
|
||||
}
|
||||
}
|
||||
const jobInfo = await wmill.getCompletedJob({
|
||||
workspace: workspace.workspaceId,
|
||||
id,
|
||||
});
|
||||
if (opts.silent) {
|
||||
console.log(JSON.stringify(jobInfo.result ?? {}));
|
||||
} else {
|
||||
log.info(JSON.stringify(jobInfo.result ?? {}, null, 2));
|
||||
if (retries >= MAX_RETRIES) {
|
||||
throw new Error(`Timed out waiting for flow ${id} to complete`);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -445,7 +506,17 @@ async function preview(
|
||||
});
|
||||
} catch (e: any) {
|
||||
if (e.body) {
|
||||
log.error(`Flow preview failed: ${JSON.stringify(e.body)}`);
|
||||
// If a failure_module ran, the body contains its result — not an error
|
||||
if (e.body.result !== undefined) {
|
||||
if (opts.silent) {
|
||||
console.log(JSON.stringify(e.body.result));
|
||||
} else {
|
||||
log.info(colors.yellow.bold("Flow failed, error handler result:"));
|
||||
log.info(JSON.stringify(e.body.result, null, 2));
|
||||
}
|
||||
process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
}
|
||||
throw e;
|
||||
}
|
||||
@@ -551,7 +622,7 @@ export async function bootstrap(
|
||||
await loadNonDottedPathsSetting();
|
||||
|
||||
const flowDirFullPath = buildFolderPath(flowPath, "flow");
|
||||
mkdirSync(flowDirFullPath, { recursive: false });
|
||||
mkdirSync(flowDirFullPath, { recursive: true });
|
||||
|
||||
const newFlowDefinition = defaultFlowDefinition();
|
||||
if (opts.summary !== undefined) {
|
||||
@@ -653,6 +724,7 @@ const command = new Command()
|
||||
"push a local flow spec. This overrides any remote versions."
|
||||
)
|
||||
.arguments("<file_path:string> <remote_path:string>")
|
||||
.option("--message <message:string>", "Deployment message")
|
||||
.action(push as any)
|
||||
.command("run", "run a flow by path.")
|
||||
.arguments("<path:string>")
|
||||
|
||||
@@ -43,7 +43,7 @@ export interface InitOptions {
|
||||
*/
|
||||
async function initAction(opts: InitOptions) {
|
||||
if (await stat("wmill.yaml").catch(() => null)) {
|
||||
log.error(colors.red("wmill.yaml already exists"));
|
||||
log.info("wmill.yaml already exists, skipping config generation");
|
||||
} else {
|
||||
// Detect current git branch for template
|
||||
const { isGitRepository, getCurrentGitBranch } = await import(
|
||||
|
||||
@@ -67,17 +67,20 @@ async function list(
|
||||
let successFilter = opts.success;
|
||||
if (opts.failed) successFilter = false;
|
||||
|
||||
const jobs = await wmill.listJobs({
|
||||
const limit = Math.min(opts.limit ?? 30, 100);
|
||||
const allJobs = await wmill.listJobs({
|
||||
workspace: workspace.workspaceId,
|
||||
scriptPathExact: opts.scriptPath,
|
||||
createdBy: opts.createdBy,
|
||||
running: opts.running,
|
||||
success: successFilter,
|
||||
perPage: Math.min(opts.limit ?? 30, 100),
|
||||
perPage: limit,
|
||||
jobKinds: opts.jobKinds ?? "script,flow,singlestepflow",
|
||||
label: opts.label,
|
||||
hasNullParent: opts.all ? undefined : true,
|
||||
});
|
||||
// API may return more than perPage — enforce limit client-side
|
||||
const jobs = allJobs.slice(0, limit);
|
||||
|
||||
if (opts.json) {
|
||||
console.log(JSON.stringify(jobs));
|
||||
@@ -92,7 +95,7 @@ async function list(
|
||||
.border(true)
|
||||
.body(
|
||||
jobs.map((j: any) => [
|
||||
j.id.substring(0, 8),
|
||||
j.id,
|
||||
getJobStatus(j),
|
||||
j.script_path ?? j.raw_code?.substring(0, 30) ?? "-",
|
||||
j.created_by ?? j.email ?? "-",
|
||||
@@ -170,12 +173,35 @@ async function logs(
|
||||
const workspace = await resolveWorkspace(opts);
|
||||
await requireLogin(opts);
|
||||
|
||||
// Check if this is a flow job (flows don't have top-level logs)
|
||||
try {
|
||||
const job = await wmill.getJob({
|
||||
workspace: workspace.workspaceId,
|
||||
id,
|
||||
});
|
||||
const jobKind = (job as any).job_kind; // job_kind not in generated types yet
|
||||
if (jobKind === "flow" || jobKind === "flowpreview") {
|
||||
log.info(colors.yellow(
|
||||
"Flow jobs don't have direct logs. Each step runs as a separate job.\n" +
|
||||
"Use 'wmill job list --all' to see sub-jobs, then 'wmill job logs <sub-job-id>' for individual step logs."
|
||||
));
|
||||
return;
|
||||
}
|
||||
} catch {
|
||||
// If we can't get the job info, proceed with trying to get logs anyway
|
||||
}
|
||||
|
||||
const jobLogs = await wmill.getJobLogs({
|
||||
workspace: workspace.workspaceId,
|
||||
id,
|
||||
});
|
||||
|
||||
console.log(jobLogs);
|
||||
if (jobLogs == null || jobLogs === "") {
|
||||
log.info("No logs available for this job.");
|
||||
} else {
|
||||
console.error("to remove ansi colors, use: | sed 's/\\x1B\\[[0-9;]\\{1,\\}[A-Za-z]//g'");
|
||||
console.log(jobLogs);
|
||||
}
|
||||
}
|
||||
|
||||
async function cancel(
|
||||
|
||||
@@ -625,7 +625,13 @@ export async function runLint(
|
||||
throw new Error(`Path is not a directory: ${targetDirectory}`);
|
||||
}
|
||||
|
||||
const ignore = await ignoreF(mergedOpts);
|
||||
// When the user specifies a subdirectory (that doesn't contain wmill.yaml),
|
||||
// skip include/exclude filters since they're relative to the project root.
|
||||
const isSubdirectory = explicitTargetDirectory &&
|
||||
!(await stat(path.join(targetDirectory, "wmill.yaml")).catch(() => null));
|
||||
const ignore = isSubdirectory
|
||||
? (_p: string, _isDir: boolean) => false
|
||||
: await ignoreF(mergedOpts);
|
||||
const root = await FSFSElement(targetDirectory, [], false);
|
||||
const validator = new WindmillYamlValidator();
|
||||
|
||||
@@ -640,9 +646,10 @@ export async function runLint(
|
||||
if (entry.isDirectory || entry.ignored) {
|
||||
continue;
|
||||
}
|
||||
scannedFiles += 1;
|
||||
|
||||
const normalizedPath = normalizePath(entry.path);
|
||||
|
||||
scannedFiles += 1;
|
||||
if (!YAML_FILE_REGEX.test(normalizedPath)) {
|
||||
continue;
|
||||
}
|
||||
@@ -742,7 +749,11 @@ export function printReport(report: LintReport, jsonOutput: boolean) {
|
||||
}
|
||||
}
|
||||
|
||||
async function lint(opts: LintOptions, directory?: string) {
|
||||
async function lint(opts: LintOptions & { watch?: boolean }, directory?: string) {
|
||||
if (opts.watch) {
|
||||
await lintWatch(opts, directory);
|
||||
return;
|
||||
}
|
||||
try {
|
||||
const report = await runLint(opts, directory);
|
||||
printReport(report, !!opts.json);
|
||||
@@ -770,6 +781,37 @@ async function lint(opts: LintOptions, directory?: string) {
|
||||
}
|
||||
}
|
||||
|
||||
async function lintWatch(opts: LintOptions, directory?: string) {
|
||||
const { watch } = await import("node:fs");
|
||||
const targetDir = directory ? path.resolve(process.cwd(), directory) : process.cwd();
|
||||
|
||||
log.info(colors.blue(`Watching ${targetDir} for changes... (Ctrl+C to stop)`));
|
||||
|
||||
async function runAndReport() {
|
||||
try {
|
||||
const report = await runLint(opts, directory);
|
||||
// Clear screen for readability
|
||||
process.stdout.write("\x1Bc");
|
||||
log.info(colors.gray(`[${new Date().toLocaleTimeString()}] Lint results:\n`));
|
||||
printReport(report, false);
|
||||
} catch (error) {
|
||||
log.error(error instanceof Error ? error.message : String(error));
|
||||
}
|
||||
}
|
||||
|
||||
await runAndReport();
|
||||
|
||||
let debounce: ReturnType<typeof setTimeout> | null = null;
|
||||
watch(targetDir, { recursive: true }, (_event, filename) => {
|
||||
if (!filename || !filename.toString().endsWith(".yaml") && !filename.toString().endsWith(".yml")) return;
|
||||
if (debounce) clearTimeout(debounce);
|
||||
debounce = setTimeout(runAndReport, 300);
|
||||
});
|
||||
|
||||
// Keep the process alive
|
||||
await new Promise(() => {});
|
||||
}
|
||||
|
||||
const command = new Command()
|
||||
.description(
|
||||
"Validate Windmill flow, schedule, and trigger YAML files in a directory",
|
||||
@@ -781,6 +823,7 @@ const command = new Command()
|
||||
"--locks-required",
|
||||
"Fail if scripts or flow inline scripts that need locks have no locks",
|
||||
)
|
||||
.option("-w, --watch", "Watch for file changes and re-lint automatically")
|
||||
.action(lint as any);
|
||||
|
||||
export default command;
|
||||
|
||||
@@ -97,6 +97,10 @@ async function list(opts: GlobalOptions & { schema?: boolean; json?: boolean })
|
||||
|
||||
if (opts.json) {
|
||||
console.log(JSON.stringify(res));
|
||||
} else if (res.length === 0) {
|
||||
log.info("No custom resource types found in this workspace.");
|
||||
log.info("Built-in types like 'postgresql', 'slack', 'mysql', etc. are available from the Windmill Hub.");
|
||||
return;
|
||||
} else if (opts.schema) {
|
||||
new Table()
|
||||
.header(["Workspace", "Name", "Schema"])
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { stat, writeFile, readdir, readFile } from "node:fs/promises";
|
||||
import { mkdir, stat, writeFile, readdir, readFile } from "node:fs/promises";
|
||||
import { stringify as yamlStringify } from "yaml";
|
||||
import nodePath from "node:path";
|
||||
|
||||
@@ -203,6 +203,7 @@ async function newResource(opts: GlobalOptions, path: string) {
|
||||
resource_type: "",
|
||||
description: "",
|
||||
};
|
||||
await mkdir(nodePath.dirname(filePath), { recursive: true });
|
||||
await writeFile(filePath, yamlStringify(template as Record<string, any>), {
|
||||
flag: "wx",
|
||||
encoding: "utf-8",
|
||||
@@ -211,6 +212,7 @@ async function newResource(opts: GlobalOptions, path: string) {
|
||||
}
|
||||
|
||||
async function get(opts: GlobalOptions & { json?: boolean }, path: string) {
|
||||
if (opts.json) log.setSilent(true);
|
||||
const workspace = await resolveWorkspace(opts);
|
||||
await requireLogin(opts);
|
||||
const r = await wmill.getResource({
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { stat, writeFile } from "node:fs/promises";
|
||||
import { mkdir, stat, writeFile } from "node:fs/promises";
|
||||
import { dirname } from "node:path";
|
||||
import { stringify as yamlStringify } from "yaml";
|
||||
|
||||
import { Command } from "@cliffy/command";
|
||||
@@ -70,6 +71,7 @@ async function newSchedule(opts: GlobalOptions, path: string) {
|
||||
is_flow: false,
|
||||
enabled: false,
|
||||
};
|
||||
await mkdir(dirname(filePath), { recursive: true });
|
||||
await writeFile(filePath, yamlStringify(template as Record<string, any>), {
|
||||
flag: "wx",
|
||||
encoding: "utf-8",
|
||||
@@ -78,6 +80,7 @@ async function newSchedule(opts: GlobalOptions, path: string) {
|
||||
}
|
||||
|
||||
async function get(opts: GlobalOptions & { json?: boolean }, path: string) {
|
||||
if (opts.json) log.setSilent(true);
|
||||
const workspace = await resolveWorkspace(opts);
|
||||
await requireLogin(opts);
|
||||
const s = await wmill.getSchedule({
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user