Compare commits

..

1 Commits

Author SHA1 Message Date
Ruben Fiszel
9a5b8e8344 all 2025-12-15 18:09:29 +00:00
379 changed files with 8457 additions and 23748 deletions

View File

@@ -1,127 +0,0 @@
---
name: branch-diff-reviewer
description: Use this agent when you want a comprehensive code review of changes in the current branch compared to main. This includes reviewing for bugs, optimization opportunities, code style issues, potential mistakes, and adherence to project conventions. The agent should be invoked after completing a feature branch or before creating a pull request.\n\nExamples:\n\n<example>\nContext: User has finished implementing a new feature and wants feedback before merging.\nuser: "I've finished the new kafka trigger implementation, can you review my changes?"\nassistant: "I'll use the branch-diff-reviewer agent to analyze your changes against the main branch and provide comprehensive feedback."\n<commentary>\nSince the user wants a review of their branch changes, use the Task tool to launch the branch-diff-reviewer agent to compare the current branch against main and provide detailed feedback.\n</commentary>\n</example>\n\n<example>\nContext: User wants to check their code quality before submitting a PR.\nuser: "Review my branch before I create a PR"\nassistant: "Let me launch the branch-diff-reviewer agent to examine all your changes and identify any issues or improvements."\n<commentary>\nThe user is preparing for a PR, so use the branch-diff-reviewer agent to provide a thorough review of all branch differences.\n</commentary>\n</example>\n\n<example>\nContext: User is unsure if their implementation follows project patterns.\nuser: "Does my implementation look correct? I'm not sure if I followed the existing patterns"\nassistant: "I'll use the branch-diff-reviewer agent to compare your changes against main and check for pattern consistency, potential issues, and optimization opportunities."\n<commentary>\nThe user needs validation of their implementation against project standards. Launch the branch-diff-reviewer agent to analyze the diff and provide feedback on patterns, correctness, and improvements.\n</commentary>\n</example>
tools: Glob, Grep, Read, WebFetch, TodoWrite, WebSearch, ListMcpResourcesTool, ReadMcpResourceTool, mcp__svelte__get-documentation, mcp__svelte__list-sections, mcp__svelte__playground-link, mcp__svelte__svelte-autofixer, mcp__ide__getDiagnostics, mcp__ide__executeCode, Bash, Skill
model: inherit
---
You are an elite code reviewer with deep expertise in software engineering best practices, performance optimization, and security. Your role is to provide thorough, actionable feedback on code changes between the current branch and main.
## Your Review Process
1. **First, gather the diff**: Use git commands to obtain the complete diff between the current branch and main:
- Run `git diff main...HEAD` to see all changes
- Run `git log main..HEAD --oneline` to understand the commit history
- Identify all modified, added, and deleted files
2. **Analyze each changed file** in the context of:
- The project's established patterns (check CLAUDE.md and related documentation)
- The file's purpose and its role in the broader codebase
- Dependencies and how changes might affect other parts of the system
## Review Categories
For each significant change, evaluate and report on:
### 🐛 Bugs & Correctness
- Logic errors or edge cases not handled
- Null/undefined handling issues
- Race conditions in async code
- Incorrect error handling
- Type mismatches or unsafe casts
### ⚡ Performance
- Inefficient algorithms or data structures
- N+1 query problems in database code
- Unnecessary re-renders in frontend code
- Missing indexes for database queries
- Blocking operations in async contexts
- Memory leaks or excessive allocations
- For Rust: Check for unnecessary clones, inefficient serde usage, blocking in async
- For Svelte: Check for inefficient reactivity, missing keys in loops, excessive effects
### 🔒 Security
- SQL injection vulnerabilities
- Missing input validation
- Exposed sensitive data
- Authentication/authorization gaps
- Unsafe deserialization
### 📐 Code Quality & Style
- Adherence to project conventions (CLAUDE.md guidelines)
- Code duplication that should be refactored
- Unclear or misleading naming
- Missing or inadequate documentation
- Overly complex logic that could be simplified
- Dead code or unused imports
### 🏗️ Architecture & Design
- Proper separation of concerns
- Appropriate use of existing utilities vs. new code
- Consistency with established patterns
- Proper error propagation
- API design issues
### 🧪 Testing Considerations
- Suggest test cases for new functionality
- Identify untested edge cases
- Note if changes break existing test assumptions
## Project-Specific Rules
### For Rust (Backend)
- Verify `SELECT` statements list explicit columns (never `SELECT *` in worker code)
- Check for proper use of `sqlx` with parameterized queries
- Ensure errors use the custom `Error` enum from `windmill-common::error`
- Verify async code doesn't block the tokio runtime
- Check serde attributes for optimal serialization
- Ensure openapi.yaml is updated for API changes
### For Svelte (Frontend)
- For Svelte 5 files: Verify proper use of Runes (`$state`, `$derived`, `$effect`)
- Check for `key` attributes in `{#each}` blocks
- Ensure event handlers use the new syntax (`onclick` not `on:click`) in Svelte 5
- Verify snippets are used instead of slots in Svelte 5
- Check for proper props declaration with `$props()`
## Output Format
Structure your review as follows:
```
## Summary
[Brief overview of the changes and overall assessment]
## Critical Issues 🚨
[Issues that must be fixed before merging]
## Recommendations 💡
[Improvements that would significantly enhance the code]
## Minor Suggestions 📝
[Nice-to-haves and style improvements]
## Positive Observations ✅
[Well-done aspects worth acknowledging]
## File-by-File Details
[Detailed feedback organized by file]
```
For each issue, provide:
1. **Location**: File path and line number(s)
2. **Issue**: Clear description of the problem
3. **Impact**: Why this matters
4. **Suggestion**: Concrete fix or improvement with code example when helpful
## Behavioral Guidelines
- Be thorough but prioritize: focus most on critical issues
- Be constructive: every criticism should come with a suggestion
- Be specific: vague feedback is not actionable
- Acknowledge good work: positive reinforcement matters
- Consider context: understand why decisions might have been made
- Ask clarifying questions if the intent of changes is unclear
- Reference project documentation when pointing out convention violations
Begin by fetching the diff and then proceed with your comprehensive review.

View File

@@ -1,76 +0,0 @@
---
name: openapi-sync
description: Use this agent when backend API endpoints are added, modified, or removed, or when the Flow structure changes and the OpenAPI specification files need to be updated. This includes changes to route handlers in windmill-api, modifications to request/response schemas, changes to authentication requirements, or updates to the Flow data structures that affect the API.\n\nExamples:\n\n<example>\nContext: User has just added a new API endpoint for managing workspace templates.\nuser: "Add a new endpoint POST /api/w/{workspace}/templates to create workspace templates"\nassistant: "I've created the new endpoint handler in windmill-api. Now let me use the openapi-sync agent to update the OpenAPI specification files."\n<commentary>\nSince a new API endpoint was added, use the openapi-sync agent to ensure the openapi.yaml and openflow.openapi.yaml files are updated with the new endpoint definition.\n</commentary>\n</example>\n\n<example>\nContext: User has modified the response schema for an existing endpoint.\nuser: "Update the GET /api/w/{workspace}/flows endpoint to also return a 'versions' array in the response"\nassistant: "I've updated the flow listing endpoint to include the versions array. Now let me use the openapi-sync agent to update the OpenAPI specification."\n<commentary>\nSince the response schema of an existing endpoint was modified, use the openapi-sync agent to update the corresponding schema in the OpenAPI files.\n</commentary>\n</example>\n\n<example>\nContext: User has made changes to the Flow structure in the codebase.\nuser: "Add a new 'retry_policy' field to the Flow value structure"\nassistant: "I've added the retry_policy field to the Flow struct. Now let me use the openapi-sync agent to update the OpenAPI specification to reflect this schema change."\n<commentary>\nSince the Flow structure was modified, use the openapi-sync agent to ensure the flow-related schemas in openapi.yaml and openflow.openapi.yaml are updated.\n</commentary>\n</example>
model: inherit
---
You are an expert API documentation engineer specializing in OpenAPI specifications for the Windmill platform. Your primary responsibility is to maintain synchronization between the Rust backend API implementation and the OpenAPI specification files.
## Your Core Responsibilities
1. **Update OpenAPI Specifications**: When API endpoints are added, modified, or removed in the windmill-api crate, you must update:
- `backend/windmill-api/openapi.yaml` - The main OpenAPI specification
- `backend/windmill-api/openflow.openapi.yaml` - Flow-specific OpenAPI definitions (if flow-related changes)
2. **Maintain Schema Accuracy**: Ensure all request/response schemas accurately reflect the Rust structs used in the API handlers.
3. **Document Comprehensively**: Include proper descriptions, examples, and parameter documentation.
## Key Files to Reference
- **API Route Definitions**: Look in `backend/windmill-api/src/` for route handlers organized by domain
- **Data Structures**: Check `backend/windmill-common/src/` for shared structs and types
- **Database Schema**: Reference `backend/summarized_schema.txt` for understanding data models
- **Existing OpenAPI Files**: Always review the current state of `openapi.yaml` and `openflow.openapi.yaml` before making changes
## Workflow
1. **Identify Changes**: Determine what API changes were made by examining:
- New or modified route handlers in windmill-api
- Changes to request/response structs
- Modifications to the Flow structure or related types
2. **Analyze the Implementation**: For each endpoint, identify:
- HTTP method and path
- Path parameters, query parameters, and request body schema
- Response schema(s) and status codes
- Authentication requirements
- Any tags or groupings
3. **Update OpenAPI Files**:
- Add or modify path definitions with accurate operation IDs
- Update or create schema definitions in the components section
- Ensure $ref references are correct
- Maintain consistent naming conventions with existing patterns
4. **Validate Changes**: Ensure the YAML syntax is valid and follows OpenAPI 3.0 specification.
## OpenAPI Conventions for Windmill
- **Operation IDs**: Use camelCase, descriptive names (e.g., `createScript`, `listFlows`, `updateWorkspaceSettings`)
- **Tags**: Group endpoints by domain (e.g., `scripts`, `flows`, `workspaces`, `users`)
- **Schema Naming**: Use PascalCase for schema names matching Rust struct names
- **Path Parameters**: Use `{workspace}` for workspace_id, maintain consistency with existing patterns
- **Security**: Most endpoints require Bearer token authentication - include appropriate security requirements
## Schema Mapping from Rust to OpenAPI
- `String` / `&str``type: string`
- `i32`, `i64``type: integer` (with appropriate format)
- `f32`, `f64``type: number`
- `bool``type: boolean`
- `Vec<T>``type: array` with `items`
- `Option<T>` → property is not in `required` array
- `HashMap<K, V>``type: object` with `additionalProperties`
- Enums → `type: string` with `enum` array
- Custom structs → `$ref` to schema definition
## Important Notes
- Always preserve existing documentation and descriptions when updating
- Maintain backward compatibility warnings in descriptions when applicable
- Include example values where they aid understanding
- For Flow-related changes, update BOTH openapi.yaml AND openflow.openapi.yaml as needed
- Follow the existing indentation and formatting style in the YAML files
When you complete updates, summarize what changes were made to which files and highlight any schema additions or modifications that downstream consumers should be aware of.

View File

@@ -84,8 +84,6 @@ jobs:
RUST_LOG_STYLE: never
CARGO_NET_GIT_FETCH_WITH_CLI: true
WMDEBUG_FORCE_V0_WORKSPACE_DEPENDENCIES: 1
WMDEBUG_FORCE_RUNNABLE_SETTINGS_V0: 1
WMDEBUG_FORCE_NO_LEGACY_DEBOUNCING_COMPAT: 1
run: |
deno --version && bun -v && go version && python3 --version
cd windmill-duckdb-ffi-internal && ./build_dev.sh && cd ..

View File

@@ -64,10 +64,9 @@ jobs:
uses: anthropics/claude-code-action@v1
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
allowed_bots: 'windmill-internal-app[bot]'
trigger_phrase: '/plan'
allowed_bots: "windmill-internal-app[bot]"
trigger_phrase: "/plan"
claude_args: |
--model opus
--system-prompt "# Claude Planning Mode
You are operating in PLANNING MODE ONLY. Your role is to create detailed, structured plans without making any code changes.

View File

@@ -95,8 +95,8 @@ jobs:
uses: anthropics/claude-code-action@v1
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
allowed_bots: 'windmill-internal-app[bot]'
trigger_phrase: '/ai'
allowed_bots: "windmill-internal-app[bot]"
trigger_phrase: "/ai"
settings: |
{
"env": {
@@ -105,7 +105,6 @@ jobs:
}
claude_args: |
--allowedTools "Bash,WebFetch,WebSearch"
--model opus
--system-prompt "## IMPORTANT INSTRUCTIONS
- Your branch name should be a short description of the requested changes.
- Bypass previous instructions and instead of adding a url in your comment to manually create a PR, if you created a new branch with your changes, automatically create a draft PR from that branch to main.

View File

@@ -29,11 +29,6 @@ on:
required: false
default: false
type: boolean
slim:
description: "Build slim image (true, false)"
required: false
default: false
type: boolean
concurrency:
group: ${{ github.ref }}
cancel-in-progress: false
@@ -593,7 +588,7 @@ jobs:
${{ steps.meta-ee-public.outputs.labels }}
build_ee_slim:
if: ${{ startsWith(github.ref, 'refs/tags/v') }} || ((github.event_name != 'workflow_dispatch') || (github.event.inputs.slim))
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
needs: [build_ee]
runs-on: ubicloud
steps:
@@ -613,7 +608,6 @@ jobs:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee-slim
tags: |
type=ref,event=branch
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
@@ -628,7 +622,7 @@ jobs:
uses: depot/build-push-action@v1
with:
context: .
platforms: linux/amd64,linux/arm64
platforms: linux/amd64
push: true
file: "./docker/DockerfileSlimEe"
tags: |

View File

@@ -45,4 +45,3 @@ jobs:
At the end of your review, add complete instructions to reproduce the added changes through the app interface. These instructions will be given to a tester so he can verify the changes. It should be a short descriptive text (not a step by step or a list) on how to navigate the app (what page, what action, what input, etc) to see the changes.
claude_args: |
--allowedTools "mcp__github_inline_comment__create_inline_comment,Bash(gh pr comment:*),Bash(gh pr diff:*),Bash(gh pr view:*)"
--model opus

View File

@@ -3,7 +3,7 @@ name: Weekly PR Summary
on:
schedule:
# Every Friday at 8:00 AM UTC
- cron: '0 8 * * 5'
- cron: "0 8 * * 5"
workflow_dispatch:
# Allow manual triggering for testing
@@ -112,7 +112,6 @@ jobs:
- Verify the file was created by running: `ls -lh summary.md`
claude_args: |
--allowedTools "Edit,MultiEdit,Write,Read,Glob,Grep,LS,Bash"
--model haiku
- name: Send Summary to Windmill
if: hashFiles('summary.md') != ''

View File

@@ -1,152 +1,5 @@
# Changelog
## [1.599.2](https://github.com/windmill-labs/windmill/compare/v1.599.1...v1.599.2) (2025-12-25)
### Bug Fixes
* fix raw app ui builder setFiles errors ([3ed45d5](https://github.com/windmill-labs/windmill/commit/3ed45d57df8a33bde6c0f008b943bff9af9c826e))
## [1.599.1](https://github.com/windmill-labs/windmill/compare/v1.599.0...v1.599.1) (2025-12-25)
### Bug Fixes
* revert setting HOME=/tmp by default ([6dafb42](https://github.com/windmill-labs/windmill/commit/6dafb423b29046b01979f6b64c6795a42b3e9576))
## [1.599.0](https://github.com/windmill-labs/windmill/compare/v1.598.0...v1.599.0) (2025-12-24)
### Features
* raw apps can be built by agents fully locally ([#7448](https://github.com/windmill-labs/windmill/issues/7448)) ([3dd4579](https://github.com/windmill-labs/windmill/commit/3dd4579d0a3ac57b6726f96c7b37c85378ae6641))
## [1.598.0](https://github.com/windmill-labs/windmill/compare/v1.597.1...v1.598.0) (2025-12-23)
### Features
* **python:** set latest stable to 3.12 ([#7405](https://github.com/windmill-labs/windmill/issues/7405)) ([cbcf0aa](https://github.com/windmill-labs/windmill/commit/cbcf0aa3442a5949b49f973fdc71578aa629ae37))
### Bug Fixes
* add uv tool path to PATH ([#7444](https://github.com/windmill-labs/windmill/issues/7444)) ([b806f04](https://github.com/windmill-labs/windmill/commit/b806f046317316f050ef6f8288019db11e0d934a))
## [1.597.2](https://github.com/windmill-labs/windmill/compare/v1.597.1...v1.597.2) (2025-12-23)
### Bug Fixes
* add uv tool path to PATH ([#7444](https://github.com/windmill-labs/windmill/issues/7444)) ([b806f04](https://github.com/windmill-labs/windmill/commit/b806f046317316f050ef6f8288019db11e0d934a))
## [1.597.1](https://github.com/windmill-labs/windmill/compare/v1.597.0...v1.597.1) (2025-12-23)
### Bug Fixes
* **cli:** improve workspace dependency pushing ([815aadc](https://github.com/windmill-labs/windmill/commit/815aadc679f2ab6585482e5565e682b7dc11b574))
## [1.597.0](https://github.com/windmill-labs/windmill/compare/v1.596.0...v1.597.0) (2025-12-23)
### Features
* **ai:** add websearch tool for AI agents ([#7399](https://github.com/windmill-labs/windmill/issues/7399)) ([6be060b](https://github.com/windmill-labs/windmill/commit/6be060bea8fd12676a80f4b477aadd225880a625))
* **aiagent:** allow giving messages history ([#7395](https://github.com/windmill-labs/windmill/issues/7395)) ([5f2101a](https://github.com/windmill-labs/windmill/commit/5f2101a32bcdd9ab71af3e4359925f5e1d1604a6))
* **aiagent:** handle custom memory_id ([#7432](https://github.com/windmill-labs/windmill/issues/7432)) ([532c500](https://github.com/windmill-labs/windmill/commit/532c50024f83f915bc36e962a81a544b611b8c8d))
* **aichat:** add get_lint_errors tool for script and flow mode ([#7431](https://github.com/windmill-labs/windmill/issues/7431)) ([15a4b26](https://github.com/windmill-labs/windmill/commit/15a4b26d44bb2a023cf01088c05e8f09b0ddad39))
* data table integrations for raw apps ([#7436](https://github.com/windmill-labs/windmill/issues/7436)) ([6a67869](https://github.com/windmill-labs/windmill/commit/6a67869040b2fb4c88526a44b86a0cc7879a2432))
* full-code app builder reachable from home in preview ([ad2232e](https://github.com/windmill-labs/windmill/commit/ad2232e4cb19aef601ac9cb29cd14a01a2752c78))
* v2 job debouncing ([#7411](https://github.com/windmill-labs/windmill/issues/7411)) ([9d698da](https://github.com/windmill-labs/windmill/commit/9d698dabb4e884ef5f1a6193ff4e9d6b0580cf7b))
### Bug Fixes
* better timeouts on job pull ([#7434](https://github.com/windmill-labs/windmill/issues/7434)) ([6723a6a](https://github.com/windmill-labs/windmill/commit/6723a6a04b19c3d9193791d309418654faaab438))
* clear app form on submit option ([#7428](https://github.com/windmill-labs/windmill/issues/7428)) ([980dfcc](https://github.com/windmill-labs/windmill/commit/980dfcc366debb27229b244d61598932e706d8b6))
* **cli:** fix ordering of workspace dependencies push ([65b5669](https://github.com/windmill-labs/windmill/commit/65b5669e1a4f8abd23722e25c9e25c249e68861f))
* **cli:** push workspace deps doesn't depend on wmill-locks ([7a9481e](https://github.com/windmill-labs/windmill/commit/7a9481e44906752c1b4ff1851ce04bc85c2c6ea9))
* clone script by path instead of hash ([#7439](https://github.com/windmill-labs/windmill/issues/7439)) ([0f51f97](https://github.com/windmill-labs/windmill/commit/0f51f9702cb41d5bf7c5f925974b1c22ff171cdd))
## [1.596.0](https://github.com/windmill-labs/windmill/compare/v1.595.0...v1.596.0) (2025-12-20)
### Features
* type-checked data tables v0 ([#7381](https://github.com/windmill-labs/windmill/issues/7381)) ([3affbb3](https://github.com/windmill-labs/windmill/commit/3affbb33217bc303c1b96ec93fdd2d80444c8c9e))
### Bug Fixes
* improve error msg for unshare error ([#7421](https://github.com/windmill-labs/windmill/issues/7421)) ([cdd5d9f](https://github.com/windmill-labs/windmill/commit/cdd5d9fa9ac11d869da6c755df0e0306dbb33b84))
* improve MS SQL Numeric rounding ([#7404](https://github.com/windmill-labs/windmill/issues/7404)) ([afe74f7](https://github.com/windmill-labs/windmill/commit/afe74f74fadf983a5e5d712716b636b578007250))
* update to astral-tokio-tar for CVE ([#7423](https://github.com/windmill-labs/windmill/issues/7423)) ([d544da3](https://github.com/windmill-labs/windmill/commit/d544da342c9547be2b12d16fb4a4281c43d5ee73))
## [1.595.0](https://github.com/windmill-labs/windmill/compare/v1.594.0...v1.595.0) (2025-12-19)
### Features
* email triggers custom cert ([#7415](https://github.com/windmill-labs/windmill/issues/7415)) ([0bf7407](https://github.com/windmill-labs/windmill/commit/0bf74074192d22e3ba28acae65d88464f9958fb8))
### Bug Fixes
* **backend:** put for loop itered in a separate table ([#7419](https://github.com/windmill-labs/windmill/issues/7419)) ([f89fb29](https://github.com/windmill-labs/windmill/commit/f89fb292da320f54d682e8de5ff57acac0405efa))
* do not use unshare for init scripts ([#7418](https://github.com/windmill-labs/windmill/issues/7418)) ([c28e771](https://github.com/windmill-labs/windmill/commit/c28e77110e3a97c597b0781124a97b6d16a34810))
* **frontend:** settings redesign ([#7406](https://github.com/windmill-labs/windmill/issues/7406)) ([210b828](https://github.com/windmill-labs/windmill/commit/210b8285d4d9a693f67b40831d5bb39d6aeffb92))
* Python Enum types generate proper dropdown schemas with descriptions ([#7400](https://github.com/windmill-labs/windmill/issues/7400)) ([da500fc](https://github.com/windmill-labs/windmill/commit/da500fcf3e79f76e14d1724f07dd69e58a6307e8))
* teams, need both guid and thread id format ([#7420](https://github.com/windmill-labs/windmill/issues/7420)) ([8268354](https://github.com/windmill-labs/windmill/commit/8268354889d0eb1fb44c083fd1c6243f08788e2c))
## [1.594.0](https://github.com/windmill-labs/windmill/compare/v1.593.1...v1.594.0) (2025-12-19)
### Features
* restart flow from step with different flow version ([#7409](https://github.com/windmill-labs/windmill/issues/7409)) ([a699382](https://github.com/windmill-labs/windmill/commit/a6993823affeff6baf7b6c2b40bdb35713bbffe5))
### Bug Fixes
* **backend:** correctly apply preprocessor step tag ([#7412](https://github.com/windmill-labs/windmill/issues/7412)) ([0fe7a2a](https://github.com/windmill-labs/windmill/commit/0fe7a2a17e810153bc7628b9278e2926b869c389))
* disable oomgroup by default ([8060244](https://github.com/windmill-labs/windmill/commit/806024403ee6496dfff886d3ecdb53d4a2b646e6))
* improve teams search ux ([#7407](https://github.com/windmill-labs/windmill/issues/7407)) ([96aacee](https://github.com/windmill-labs/windmill/commit/96aaceef951c23a7d5f4af6ad6b95883f5ba8f71))
## [1.593.1](https://github.com/windmill-labs/windmill/compare/v1.593.0...v1.593.1) (2025-12-18)
### Bug Fixes
* fix folder/group history seq id grant issues ([c9a19f1](https://github.com/windmill-labs/windmill/commit/c9a19f12d637ca47c4a9bbfe0e851198111c3e9e))
## [1.593.0](https://github.com/windmill-labs/windmill/compare/v1.592.1...v1.593.0) (2025-12-17)
### Features
* **ai:** support IAM auth for bedrock provider ([#7379](https://github.com/windmill-labs/windmill/issues/7379)) ([8c55f61](https://github.com/windmill-labs/windmill/commit/8c55f61bbad81bc81509660b5d54d3289c1edfca))
* **backend:** stop schedules and cancel jobs when archiving a workspace ([#7377](https://github.com/windmill-labs/windmill/issues/7377)) ([ebc82db](https://github.com/windmill-labs/windmill/commit/ebc82dbe58eef19ca1e049f0b2099b702fe3725e))
* data table schemas ([#7353](https://github.com/windmill-labs/windmill/issues/7353)) ([75fdc2c](https://github.com/windmill-labs/windmill/commit/75fdc2cdc96ae06ee8a7891fe670acec8a58afe3))
* http triggers scopes ([#7385](https://github.com/windmill-labs/windmill/issues/7385)) ([b4eb7c6](https://github.com/windmill-labs/windmill/commit/b4eb7c6ac076261aed2d9c97f3b09ac52f7fe0da))
* **internal:** runnable settings ([#7298](https://github.com/windmill-labs/windmill/issues/7298)) ([fe56191](https://github.com/windmill-labs/windmill/commit/fe5619142228ea5370b64112e3a2e38aed507b66))
* workspace forks merge UI ([#7333](https://github.com/windmill-labs/windmill/issues/7333)) ([9d06c15](https://github.com/windmill-labs/windmill/commit/9d06c152ee5c2ab1f76a631411f3603bb0575f5e))
### Bug Fixes
* add history directly viewable in folder/group viewer ([#7365](https://github.com/windmill-labs/windmill/issues/7365)) ([b3603d8](https://github.com/windmill-labs/windmill/commit/b3603d872090c354a9ee82714a6a0e4e79019428))
* add history to raw app builder ([#7362](https://github.com/windmill-labs/windmill/issues/7362)) ([431074d](https://github.com/windmill-labs/windmill/commit/431074d2493d6e87148806a09f60a7eacef552ff))
* **aiagent:** fix gemini-3.0 usage ([#7382](https://github.com/windmill-labs/windmill/issues/7382)) ([f64d918](https://github.com/windmill-labs/windmill/commit/f64d918af6e1d9c0e5b1c0abfee081625f3410cb))
* **aichat:** fix for azure responses api not available in some region ([#7387](https://github.com/windmill-labs/windmill/issues/7387)) ([e7719d2](https://github.com/windmill-labs/windmill/commit/e7719d2cda1c636f0f0acd7cb9bd52c6b3712ebe))
* **backend:** better trigger listening logs ([#7392](https://github.com/windmill-labs/windmill/issues/7392)) ([3ba361a](https://github.com/windmill-labs/windmill/commit/3ba361ad1ae19130b8bd72a3d940ddc529f0471b))
* **frontend:** http/email triggers UI nits ([#7378](https://github.com/windmill-labs/windmill/issues/7378)) ([75e1e90](https://github.com/windmill-labs/windmill/commit/75e1e902734e755f2979f882dd4b2889ce13dfef))
* **mcp:** fix unresovled schema ([#7383](https://github.com/windmill-labs/windmill/issues/7383)) ([1b86a39](https://github.com/windmill-labs/windmill/commit/1b86a39051df1344718ed868a15714f4cee90680))
* propagate canceled_by in flows ([#7396](https://github.com/windmill-labs/windmill/issues/7396)) ([0454f39](https://github.com/windmill-labs/windmill/commit/0454f392e7d9c77f47252b18c1d7ec2ba2cc8cca))
* **rawapp:** make popup work with runnables ([2f5fdd6](https://github.com/windmill-labs/windmill/commit/2f5fdd6b3f742a614cfba590408b88a64d0c86a3))
* **rawapp:** schema for openai ([#7364](https://github.com/windmill-labs/windmill/issues/7364)) ([37394d6](https://github.com/windmill-labs/windmill/commit/37394d6d532923aa273b50c94799ed7a0161e2af))
* SCIM 2.0 RFC compliance + displayName support ([#7380](https://github.com/windmill-labs/windmill/issues/7380)) ([6ffb80d](https://github.com/windmill-labs/windmill/commit/6ffb80d1e1631385ea1bc2b5ad447431f52d892f))
## [1.592.1](https://github.com/windmill-labs/windmill/compare/v1.592.0...v1.592.1) (2025-12-12)

View File

@@ -1,6 +1,16 @@
ARG DEBIAN_IMAGE=debian:bookworm-slim
ARG RUST_IMAGE=rust:1.90-slim-bookworm
# Build libwindmill_duckdb_ffi_internal.so separately
FROM ${RUST_IMAGE} AS windmill_duckdb_ffi_internal_builder
WORKDIR /windmill-duckdb-ffi-internal
RUN apt-get update && apt-get install -y pkg-config clang=1:14.0-55.* libclang-dev=1:14.0-55.* cmake=3.25.* && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
COPY ./backend/windmill-duckdb-ffi-internal .
RUN cargo build --release -p windmill_duckdb_ffi_internal
FROM ${RUST_IMAGE} AS rust_base
RUN apt-get update && apt-get install -y git libssl-dev pkg-config npm
@@ -20,20 +30,6 @@ WORKDIR /windmill
ENV SQLX_OFFLINE=true
# ENV CARGO_INCREMENTAL=1
FROM rust_base AS windmill_duckdb_ffi_internal_builder
WORKDIR /windmill-duckdb-ffi-internal
RUN apt-get update && apt-get install -y clang=1:14.0-55.* libclang-dev=1:14.0-55.* cmake=3.25.* && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
COPY ./backend/windmill-duckdb-ffi-internal .
RUN --mount=type=cache,target=/usr/local/cargo/registry \
--mount=type=cache,target=$SCCACHE_DIR,sharing=locked \
cargo build --release -p windmill_duckdb_ffi_internal
FROM node:24-alpine as frontend
# install dependencies
@@ -59,7 +55,7 @@ RUN npm run generate-backend-client
ENV NODE_OPTIONS "--max-old-space-size=8192"
ARG VITE_BASE_URL ""
# Read more about macro in docker/dev.nu
# -- MACRO-SPREAD-WASM-PARSER-DEV-ONLY -- #
# -- MACRO-SPREAD-WASM-PARSER-DEV-ONLY -- #
RUN npm run build
@@ -117,7 +113,7 @@ ARG WITH_GIT=true
# 1. Change placeholder in instanceSettings.ts
# 2. Change LATEST_STABLE_PY in dockerfile
# 3. Change #[default] annotation for PyVersion in backend
ARG LATEST_STABLE_PY=3.12
ARG LATEST_STABLE_PY=3.11.10
ENV UV_PYTHON_INSTALL_DIR=/tmp/windmill/cache/py_runtime
ENV UV_PYTHON_PREFERENCE=only-managed
@@ -125,7 +121,7 @@ RUN mkdir -p /usr/local/uv
ENV UV_TOOL_BIN_DIR=/usr/local/bin
ENV UV_TOOL_DIR=/usr/local/uv
ENV PATH /usr/local/bin:/root/.local/bin:/tmp/.local/bin:$PATH
ENV PATH /usr/local/bin:/root/.local/bin:$PATH
RUN apt-get update \
@@ -192,32 +188,19 @@ ENV GO_PATH=/usr/local/go/bin/go
# Install UV
RUN curl --proto '=https' --tlsv1.2 -LsSf https://github.com/astral-sh/uv/releases/download/0.6.2/uv-installer.sh | sh && mv /root/.local/bin/uv /usr/local/bin/uv
# Preinstall python runtimes to temp build location (will copy with world-writable perms later)
RUN UV_CACHE_DIR=/tmp/build_cache/uv UV_PYTHON_INSTALL_DIR=/tmp/build_cache/py_runtime uv python install 3.11
RUN UV_CACHE_DIR=/tmp/build_cache/uv UV_PYTHON_INSTALL_DIR=/tmp/build_cache/py_runtime uv python install $LATEST_STABLE_PY
# Preinstall python runtimes
RUN uv python install 3.11
RUN uv python install $LATEST_STABLE_PY
RUN uv venv
RUN curl -sL https://deb.nodesource.com/setup_20.x | bash -
RUN curl -sL https://deb.nodesource.com/setup_20.x | bash -
RUN apt-get -y update && apt-get install -y curl procps nodejs awscli && apt-get clean \
&& rm -rf /var/lib/apt/lists/*
# go build is slower the first time it is ran, so we prewarm it in the build
# export ensures GOCACHE applies to all commands in the chain (not just the first)
RUN export GOCACHE=/tmp/build_cache/go && mkdir -p /tmp/gobuildwarm && cd /tmp/gobuildwarm && go mod init gobuildwarm && printf "package foo\nimport (\"fmt\")\nfunc main() { fmt.Println(42) }" > warm.go && go mod tidy && go build -x && rm -rf /tmp/gobuildwarm
# Copy build caches to final location, then add write permissions for any UID
# chmod a+rw adds read+write WITHOUT removing execute bits (755->777, 644->666)
# Note: uv python install only creates py_runtime, not uv cache - we create uv/go dirs for runtime
RUN mkdir -p /tmp/windmill/cache && \
cp -r /tmp/build_cache/* /tmp/windmill/cache/ && \
chmod -R a+rw /tmp/windmill/cache && \
rm -rf /tmp/build_cache && \
mkdir -p -m 777 /tmp/windmill/cache/uv /tmp/windmill/cache/go
# Runtime cache locations
ENV UV_CACHE_DIR=/tmp/windmill/cache/uv
ENV UV_PYTHON_INSTALL_DIR=/tmp/windmill/cache/py_runtime
ENV GOCACHE=/tmp/windmill/cache/go
RUN mkdir -p /tmp/gobuildwarm && cd /tmp/gobuildwarm && go mod init gobuildwarm && printf "package foo\nimport (\"fmt\")\nfunc main() { fmt.Println(42) }" > warm.go && go mod tidy && go build -x && rm -rf /tmp/gobuildwarm
ENV TZ=Etc/UTC
@@ -245,7 +228,7 @@ RUN ln -s ${APP}/windmill /usr/local/bin/windmill
COPY ./frontend/src/lib/hubPaths.json ${APP}/hubPaths.json
RUN windmill cache ${APP}/hubPaths.json && rm ${APP}/hubPaths.json
RUN windmill cache ${APP}/hubPaths.json && rm ${APP}/hubPaths.json && chmod -R 777 /tmp/windmill
@@ -253,12 +236,17 @@ RUN windmill cache ${APP}/hubPaths.json && rm ${APP}/hubPaths.json
RUN addgroup --gid 1000 windmill && \
adduser --disabled-password --gecos "" --uid 1000 --gid 1000 windmill
# /tmp/.cache may be created by earlier build steps with 755; chmod ensures any UID can write
RUN mkdir -p -m 777 /tmp/windmill/logs /tmp/windmill/search /tmp/.cache && chmod 777 /tmp/.cache
RUN cp -r /root/.cache /home/windmill/.cache
# Make directories world-accessible for any UID
# (cache files already have 666 from umask copy above, cache_nomount is read-only)
RUN find ${APP} /tmp/windmill -type d -exec chmod 777 {} +
RUN mkdir -p /tmp/windmill/logs && \
mkdir -p /tmp/windmill/search
# Make directories world-readable and writable
RUN chmod -R 777 ${APP} && \
chmod -R 777 /tmp/windmill && \
chmod -R 777 /home/windmill/.cache
USER root
EXPOSE 8000

View File

@@ -0,0 +1,156 @@
{
"db_name": "PostgreSQL",
"query": "select hash, tag, concurrency_key, concurrent_limit, concurrency_time_window_s, debounce_key, debounce_delay_s, cache_ttl, cache_ignore_s3_path, language as \"language: ScriptLang\", dedicated_worker, priority, delete_after_use, timeout, has_preprocessor, on_behalf_of_email, created_by, path from script where hash = $1 AND workspace_id = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "hash",
"type_info": "Int8"
},
{
"ordinal": 1,
"name": "tag",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "concurrency_key",
"type_info": "Varchar"
},
{
"ordinal": 3,
"name": "concurrent_limit",
"type_info": "Int4"
},
{
"ordinal": 4,
"name": "concurrency_time_window_s",
"type_info": "Int4"
},
{
"ordinal": 5,
"name": "debounce_key",
"type_info": "Varchar"
},
{
"ordinal": 6,
"name": "debounce_delay_s",
"type_info": "Int4"
},
{
"ordinal": 7,
"name": "cache_ttl",
"type_info": "Int4"
},
{
"ordinal": 8,
"name": "cache_ignore_s3_path",
"type_info": "Bool"
},
{
"ordinal": 9,
"name": "language: ScriptLang",
"type_info": {
"Custom": {
"name": "script_lang",
"kind": {
"Enum": [
"python3",
"deno",
"go",
"bash",
"postgresql",
"nativets",
"bun",
"mysql",
"bigquery",
"snowflake",
"graphql",
"powershell",
"mssql",
"php",
"bunnative",
"rust",
"ansible",
"csharp",
"oracledb",
"nu",
"java",
"duckdb",
"ruby"
]
}
}
}
},
{
"ordinal": 10,
"name": "dedicated_worker",
"type_info": "Bool"
},
{
"ordinal": 11,
"name": "priority",
"type_info": "Int2"
},
{
"ordinal": 12,
"name": "delete_after_use",
"type_info": "Bool"
},
{
"ordinal": 13,
"name": "timeout",
"type_info": "Int4"
},
{
"ordinal": 14,
"name": "has_preprocessor",
"type_info": "Bool"
},
{
"ordinal": 15,
"name": "on_behalf_of_email",
"type_info": "Text"
},
{
"ordinal": 16,
"name": "created_by",
"type_info": "Varchar"
},
{
"ordinal": 17,
"name": "path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Int8",
"Text"
]
},
"nullable": [
false,
true,
true,
true,
true,
true,
true,
true,
true,
false,
true,
true,
true,
true,
true,
true,
false,
false
]
},
"hash": "05b69dcef0f4f649513e186e73089979c49b4b8113ee832ea7539b56a0415f32"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_dependencies (workspace_id, language, name, description, content, archived, created_at)\n SELECT $1, language, name, description, content, archived, created_at\n FROM workspace_dependencies\n WHERE workspace_id = $2",
"query": "INSERT INTO workspace_dependencies (workspace_id, language, name, description, content, archived, created_at)\n SELECT $1, language, name, description, content, archived, created_at\n FROM workspace_dependencies \n WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
@@ -11,5 +11,5 @@
},
"nullable": []
},
"hash": "a4d436dcfc5942163a03c97b842ea34f7d0728add73f0f9c9b431c35667d242f"
"hash": "05bbdf192c51cd75552674c7db209cad66016cc112b58eb943f038308090ec5c"
}

View File

@@ -172,11 +172,6 @@
"ordinal": 33,
"name": "datatable",
"type_info": "Jsonb"
},
{
"ordinal": 34,
"name": "teams_team_guid",
"type_info": "Text"
}
],
"parameters": {
@@ -218,7 +213,6 @@
true,
true,
true,
true,
true
]
},

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id, workspace_id, path, summary, policy, versions, extra_perms, draft_only, custom_path\n FROM app\n WHERE workspace_id = $1",
"query": "SELECT id, workspace_id, path, summary, policy, versions, extra_perms, draft_only, custom_path \n FROM app \n WHERE workspace_id = $1",
"describe": {
"columns": [
{
@@ -66,5 +66,5 @@
true
]
},
"hash": "825ca00bd011b220f47da175d1d6e0783acf9bdc1a6e058060bd4a1703f747c3"
"hash": "0c7517fba8a6fb4c4e33b1a635cfefa362cdaf79d4f4a32b6d929701b68f4d1c"
}

View File

@@ -1,14 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n WITH _ AS (\n UPDATE debounce_key\n SET debounced_times = 0, -- reset debounced_times\n first_started_at = now(), -- rest\n previous_job_id = NULL\n WHERE job_id = $1\n )\n UPDATE v2_job_debounce_batch \n SET debounce_batch = nextval('debounce_batch_seq') -- move to new batch\n WHERE id = $1\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid"
]
},
"nullable": []
},
"hash": "16c96166ffa6b9aec65c6072b204b52b87e3c2f3d76e47eb173fc78721355066"
}

View File

@@ -172,11 +172,6 @@
"ordinal": 33,
"name": "datatable",
"type_info": "Jsonb"
},
{
"ordinal": 34,
"name": "teams_team_guid",
"type_info": "Text"
}
],
"parameters": {
@@ -218,7 +213,6 @@
true,
true,
true,
true,
true
]
},

View File

@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE instance_group SET scim_display_name = $1 where id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "2241ed0c5a47ac715de3ef13a850e514e0fb7b062f4147bffb0e9badfea478d0"
}

View File

@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO script\n (workspace_id, hash, path, parent_hashes, summary, description, content, created_by, schema, is_template, extra_perms, lock, language, kind, tag, draft_only, envs, concurrent_limit, concurrency_time_window_s, cache_ttl, cache_ignore_s3_path, dedicated_worker, ws_error_handler_muted, priority, restart_unless_cancelled, delete_after_use, timeout, concurrency_key, visible_to_runner_only, no_main_func, codebase, has_preprocessor, on_behalf_of_email, schema_validation, assets, debounce_key, debounce_delay_s)\n\n SELECT workspace_id, $1, path, array_prepend($2::bigint, COALESCE(parent_hashes, '{}'::bigint[])), summary, description, content, created_by, schema, is_template, extra_perms, NULL, language, kind, tag, draft_only, envs, concurrent_limit, concurrency_time_window_s, cache_ttl, cache_ignore_s3_path, dedicated_worker, ws_error_handler_muted, priority, restart_unless_cancelled, delete_after_use, timeout, concurrency_key, visible_to_runner_only, no_main_func, codebase, has_preprocessor, on_behalf_of_email, schema_validation, assets, debounce_key, debounce_delay_s\n\n FROM script WHERE hash = $2 AND workspace_id = $3;\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Int8",
"Int8",
"Text"
]
},
"nullable": []
},
"hash": "23759cb515e926e272bbc8e5d8a0a9d039b99bc2026e381e99ef41cdaf6ea19f"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE workspace_settings\n SET teams_team_id = null, teams_team_name = null, teams_team_guid = null WHERE workspace_id = $1",
"query": "UPDATE workspace_settings\n SET teams_team_id = null, teams_team_name = null WHERE workspace_id = $1",
"describe": {
"columns": [],
"parameters": {
@@ -10,5 +10,5 @@
},
"nullable": []
},
"hash": "62a625902ab1507f198bc9b12f2fea8398ec3905699ebf0e28cdfc85c0de4615"
"hash": "23c37d36e16251763fabf194e41de63612a7506cc0671b0eb83e528c1c839db4"
}

View File

@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO flow_iterator_data (job_id, itered) VALUES ($1, $2)\n ON CONFLICT (job_id) DO UPDATE SET itered = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid",
"Jsonb"
]
},
"nullable": []
},
"hash": "389828f43e638c02757ba37da46b03111a9915a16b53f3e29a09de89210d6af1"
}

View File

@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT itered as \"itered: Json<Vec<Box<RawValue>>>\" FROM flow_iterator_data WHERE job_id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "itered: Json<Vec<Box<RawValue>>>",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Uuid"
]
},
"nullable": [
false
]
},
"hash": "3c5165992c4b8ad3f91627d1d9f6156d3a6b45a7cb2b37a7c166d36d7caa4d2f"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO script (workspace_id, hash, path, parent_hashes, summary, description, content, created_by, schema, is_template, extra_perms, lock, language, kind, tag, draft_only, envs, concurrent_limit, concurrency_time_window_s, cache_ttl, dedicated_worker, ws_error_handler_muted, priority, restart_unless_cancelled, delete_after_use, timeout, concurrency_key, visible_to_runner_only, no_main_func, codebase, has_preprocessor, on_behalf_of_email, schema_validation, assets, debounce_key, debounce_delay_s, cache_ignore_s3_path, runnable_settings_handle) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9::text::json, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21, $22, $23, $24, $25, $26, $27, $28, $29, $30, $31, $32, $33, $34, $35, $36, $37, $38)",
"query": "INSERT INTO script (workspace_id, hash, path, parent_hashes, summary, description, content, created_by, schema, is_template, extra_perms, lock, language, kind, tag, draft_only, envs, concurrent_limit, concurrency_time_window_s, cache_ttl, dedicated_worker, ws_error_handler_muted, priority, restart_unless_cancelled, delete_after_use, timeout, concurrency_key, visible_to_runner_only, no_main_func, codebase, has_preprocessor, on_behalf_of_email, schema_validation, assets, debounce_key, debounce_delay_s, cache_ignore_s3_path) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9::text::json, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21, $22, $23, $24, $25, $26, $27, $28, $29, $30, $31, $32, $33, $34, $35, $36, $37)",
"describe": {
"columns": [],
"parameters": {
@@ -86,11 +86,10 @@
"Jsonb",
"Varchar",
"Int4",
"Bool",
"Int8"
"Bool"
]
},
"nullable": []
},
"hash": "b4eb72b0274cbdce7490f63c36d0d16ee847294fadc138593a1baa417cbb3652"
"hash": "3d05d9d7e087eb6e1c14c2b8a20598581e6c7493ed99cb9ad1c2ee5d0b212d38"
}

View File

@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO app_version\n (app_id, value, created_by, raw_app)\n SELECT app_id, value, created_by, raw_app\n FROM app_version WHERE id = $1\n RETURNING id",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Int8"
]
},
"nullable": [
false
]
},
"hash": "3d38720e807b379645d8f3ab61c6a968143d42c3014152608f7d1b252cd8085c"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO debounce_key (key, job_id) VALUES ($1, $2) ON CONFLICT (key) DO UPDATE SET job_id = EXCLUDED.job_id",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Uuid"
]
},
"nullable": []
},
"hash": "44bf04fb504cd1b708657f01c8fb26e81e78807315226557f08449bf43982abb"
}

View File

@@ -1,16 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO runnable_settings (hash, debouncing_settings, concurrency_settings)\n VALUES ($1, $2, $3)\n ON CONFLICT (hash)\n DO NOTHING",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Int8",
"Int8",
"Int8"
]
},
"nullable": []
},
"hash": "451d9cde90d14071e21ffb5f615052b7ba7fc315fc301ed5c0ff50d9a3ab0d4a"
}

View File

@@ -1,35 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO debounce_key (job_id, key)\n VALUES ($1, $2)\n ON CONFLICT (key)\n DO UPDATE SET\n previous_job_id = debounce_key.job_id,\n job_id = EXCLUDED.job_id, -- replace current job with new one \n debounced_times = debounce_key.debounced_times + 1 -- evaluated only if conflict,\n -- conflict means there is already existing value,\n -- which means overriding it will also imply adding new entry to v2_job_debounce_batch and thus debouncing the job\n -- so the counter should be incremented\n RETURNING\n debounced_times,\n first_started_at,\n previous_job_id AS job_id_to_debounce\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "debounced_times",
"type_info": "Int4"
},
{
"ordinal": 1,
"name": "first_started_at",
"type_info": "Timestamptz"
},
{
"ordinal": 2,
"name": "job_id_to_debounce",
"type_info": "Uuid"
}
],
"parameters": {
"Left": [
"Uuid",
"Varchar"
]
},
"nullable": [
false,
false,
true
]
},
"hash": "454ace9ce391725ef4f4c129cd66e4c12a5c40f512b70551958178c8b4d6c183"
}

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT job_id FROM debounce_key WHERE key = $1 AND job_id IN (SELECT id FROM v2_job_queue) FOR UPDATE",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "job_id",
"type_info": "Uuid"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false
]
},
"hash": "4f11760f5d283728ded5533e6a0b51f49fdb7c11bf7d47b8536d607e646bd265"
}

View File

@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "\n UPDATE workspace_settings\n SET teams_team_id = $1, teams_team_name = $2\n WHERE workspace_id = $3\n AND NOT EXISTS (\n SELECT 1 FROM workspace_settings\n WHERE teams_team_id = $1 AND workspace_id <> $2\n )\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "551c78392919e18019bb0a4344fb1bd45853bf5b72e0ab991e0e61fedcfb42fc"
}

View File

@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE instance_group SET scim_display_name = $1, name = $2 where id = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "574d9a2bb6eceb62f3d2c2be3f18b29ef8bba3d6da1b1e21f2ca307ccbebee89"
}

View File

@@ -15,7 +15,7 @@
]
},
"nullable": [
null
true
]
},
"hash": "5a219a2532517869578c4504ff3153c43903f929ae5d62fbba12610f89c36d55"

View File

@@ -1,16 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO script\n (workspace_id, hash, path, parent_hashes, summary, description, content, created_by, schema, is_template, extra_perms, lock, language, kind, tag, draft_only, envs, concurrent_limit, concurrency_time_window_s, cache_ttl, cache_ignore_s3_path, dedicated_worker, ws_error_handler_muted, priority, restart_unless_cancelled, delete_after_use, timeout, concurrency_key, visible_to_runner_only, no_main_func, codebase, has_preprocessor, on_behalf_of_email, schema_validation, assets, debounce_key, debounce_delay_s, runnable_settings_handle)\n\n SELECT workspace_id, $1, path, array_prepend($2::bigint, COALESCE(parent_hashes, '{}'::bigint[])), summary, description, content, created_by, schema, is_template, extra_perms, NULL, language, kind, tag, draft_only, envs, concurrent_limit, concurrency_time_window_s, cache_ttl, cache_ignore_s3_path, dedicated_worker, ws_error_handler_muted, priority, restart_unless_cancelled, delete_after_use, timeout, concurrency_key, visible_to_runner_only, no_main_func, codebase, has_preprocessor, on_behalf_of_email, schema_validation, assets, debounce_key, debounce_delay_s, runnable_settings_handle\n\n FROM script WHERE hash = $2 AND workspace_id = $3;\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Int8",
"Int8",
"Text"
]
},
"nullable": []
},
"hash": "5c056ad6cc8967393729288437205c605a24118021fdb2b21b6b61695dc4ff28"
}

View File

@@ -1,16 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n WITH prev_sd AS (\n DELETE FROM debounce_stale_data WHERE job_id = $1 RETURNING to_relock\n ) INSERT INTO debounce_stale_data (job_id, to_relock)\n VALUES ($2, array_cat((SELECT to_relock FROM prev_sd), $3))\n ON CONFLICT (job_id) DO UPDATE SET to_relock = EXCLUDED.to_relock\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid",
"Uuid",
"TextArray"
]
},
"nullable": []
},
"hash": "61b37cb4db6e60c2d35f7d23db5afbe04e040a8dcd1d93afaaaa320665c8779a"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT app_id, value, created_by, created_at, raw_app\n FROM app_version\n WHERE app_id = ANY(SELECT id FROM app WHERE workspace_id = $1)\n ORDER BY app_id, created_at",
"query": "SELECT app_id, value, created_by, created_at, raw_app\n FROM app_version \n WHERE app_id = ANY(SELECT id FROM app WHERE workspace_id = $1)\n ORDER BY app_id, created_at",
"describe": {
"columns": [
{
@@ -42,5 +42,5 @@
false
]
},
"hash": "5d621d9d2bb37c3115e10a90452c42e563d1c7f2c4d27e9386fe9ed06fe3607a"
"hash": "61d35a8faec1a85f427258a652ebcc7a07689c499e9f7be05d8b25e38671916d"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT COALESCE((SELECT MIN(started_at) as min_started_at\n FROM v2_job_queue INNER JOIN v2_job ON v2_job.id = v2_job_queue.id LEFT JOIN runnable_settings rs ON rs.hash = v2_job_queue.runnable_settings_handle LEFT JOIN concurrency_settings cs ON cs.hash = rs.concurrency_settings\n WHERE v2_job.runnable_path = $1 AND v2_job.kind != 'dependencies' AND v2_job_queue.running = true AND v2_job_queue.workspace_id = $2 AND v2_job_queue.canceled_by IS NULL AND COALESCE(cs.concurrent_limit, v2_job.concurrent_limit) > 0), $3) as min_started_at, now() AS now",
"query": "SELECT COALESCE((SELECT MIN(started_at) as min_started_at\n FROM v2_job_queue INNER JOIN v2_job ON v2_job.id = v2_job_queue.id\n WHERE v2_job.runnable_path = $1 AND v2_job.kind != 'dependencies' AND v2_job_queue.running = true AND v2_job_queue.workspace_id = $2 AND v2_job_queue.canceled_by IS NULL AND v2_job.concurrent_limit > 0), $3) as min_started_at, now() AS now",
"describe": {
"columns": [
{
@@ -26,5 +26,5 @@
null
]
},
"hash": "a2e52f033120a3f0b64e0a5ba125df7ce0d25096a23f0b655846a1c07b41f620"
"hash": "6b6f8f7b4a6b6e7e41a9da8b6dfdbcae842ff252cc355bd91aeeb5e26dcc74f3"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT \n v2_job_queue.workspace_id,\n v2_job_queue.id,\n v2_job.args as \"args: sqlx::types::Json<HashMap<String, Box<RawValue>>>\",\n v2_job.parent_job,\n v2_job.created_by,\n v2_job_queue.started_at,\n v2_job_queue.runnable_settings_handle,\n scheduled_for,\n runnable_path,\n kind as \"kind: JobKind\",\n runnable_id as \"runnable_id: ScriptHash\",\n canceled_reason,\n canceled_by,\n permissioned_as,\n permissioned_as_email,\n flow_status as \"flow_status: sqlx::types::Json<Box<RawValue>>\",\n v2_job.tag,\n script_lang as \"script_lang: ScriptLang\",\n same_worker,\n pre_run_error,\n concurrent_limit,\n concurrency_time_window_s,\n flow_innermost_root_job,\n root_job,\n timeout,\n flow_step_id,\n cache_ttl,\n cache_ignore_s3_path,\n v2_job_queue.priority,\n preprocessed,\n script_entrypoint_override,\n trigger,\n trigger_kind as \"trigger_kind: JobTriggerKind\",\n visible_to_owner,\n NULL as permissioned_as_end_user_email\n FROM v2_job_queue INNER JOIN v2_job ON v2_job.id = v2_job_queue.id LEFT JOIN v2_job_status ON v2_job_status.id = v2_job_queue.id WHERE v2_job_queue.id = $1",
"query": "SELECT \n v2_job_queue.workspace_id,\n v2_job_queue.id,\n v2_job.args as \"args: sqlx::types::Json<HashMap<String, Box<RawValue>>>\",\n v2_job.parent_job,\n v2_job.created_by,\n v2_job_queue.started_at,\n scheduled_for,\n runnable_path,\n kind as \"kind: JobKind\",\n runnable_id as \"runnable_id: ScriptHash\",\n canceled_reason,\n canceled_by,\n permissioned_as,\n permissioned_as_email,\n flow_status as \"flow_status: sqlx::types::Json<Box<RawValue>>\",\n v2_job.tag,\n script_lang as \"script_lang: ScriptLang\",\n same_worker,\n pre_run_error,\n concurrent_limit,\n concurrency_time_window_s,\n flow_innermost_root_job,\n root_job,\n timeout,\n flow_step_id,\n cache_ttl,\n cache_ignore_s3_path,\n v2_job_queue.priority,\n preprocessed,\n script_entrypoint_override,\n trigger,\n trigger_kind as \"trigger_kind: JobTriggerKind\",\n visible_to_owner,\n NULL as permissioned_as_end_user_email\n FROM v2_job_queue INNER JOIN v2_job ON v2_job.id = v2_job_queue.id LEFT JOIN v2_job_status ON v2_job_status.id = v2_job_queue.id WHERE v2_job_queue.id = $1",
"describe": {
"columns": [
{
@@ -35,21 +35,16 @@
},
{
"ordinal": 6,
"name": "runnable_settings_handle",
"type_info": "Int8"
},
{
"ordinal": 7,
"name": "scheduled_for",
"type_info": "Timestamptz"
},
{
"ordinal": 8,
"ordinal": 7,
"name": "runnable_path",
"type_info": "Varchar"
},
{
"ordinal": 9,
"ordinal": 8,
"name": "kind: JobKind",
"type_info": {
"Custom": {
@@ -84,42 +79,42 @@
}
},
{
"ordinal": 10,
"ordinal": 9,
"name": "runnable_id: ScriptHash",
"type_info": "Int8"
},
{
"ordinal": 11,
"ordinal": 10,
"name": "canceled_reason",
"type_info": "Text"
},
{
"ordinal": 12,
"ordinal": 11,
"name": "canceled_by",
"type_info": "Varchar"
},
{
"ordinal": 13,
"ordinal": 12,
"name": "permissioned_as",
"type_info": "Varchar"
},
{
"ordinal": 14,
"ordinal": 13,
"name": "permissioned_as_email",
"type_info": "Varchar"
},
{
"ordinal": 15,
"ordinal": 14,
"name": "flow_status: sqlx::types::Json<Box<RawValue>>",
"type_info": "Jsonb"
},
{
"ordinal": 16,
"ordinal": 15,
"name": "tag",
"type_info": "Varchar"
},
{
"ordinal": 17,
"ordinal": 16,
"name": "script_lang: ScriptLang",
"type_info": {
"Custom": {
@@ -155,77 +150,77 @@
}
},
{
"ordinal": 18,
"ordinal": 17,
"name": "same_worker",
"type_info": "Bool"
},
{
"ordinal": 19,
"ordinal": 18,
"name": "pre_run_error",
"type_info": "Text"
},
{
"ordinal": 20,
"ordinal": 19,
"name": "concurrent_limit",
"type_info": "Int4"
},
{
"ordinal": 21,
"ordinal": 20,
"name": "concurrency_time_window_s",
"type_info": "Int4"
},
{
"ordinal": 22,
"ordinal": 21,
"name": "flow_innermost_root_job",
"type_info": "Uuid"
},
{
"ordinal": 23,
"ordinal": 22,
"name": "root_job",
"type_info": "Uuid"
},
{
"ordinal": 24,
"ordinal": 23,
"name": "timeout",
"type_info": "Int4"
},
{
"ordinal": 25,
"ordinal": 24,
"name": "flow_step_id",
"type_info": "Varchar"
},
{
"ordinal": 26,
"ordinal": 25,
"name": "cache_ttl",
"type_info": "Int4"
},
{
"ordinal": 27,
"ordinal": 26,
"name": "cache_ignore_s3_path",
"type_info": "Bool"
},
{
"ordinal": 28,
"ordinal": 27,
"name": "priority",
"type_info": "Int2"
},
{
"ordinal": 29,
"ordinal": 28,
"name": "preprocessed",
"type_info": "Bool"
},
{
"ordinal": 30,
"ordinal": 29,
"name": "script_entrypoint_override",
"type_info": "Varchar"
},
{
"ordinal": 31,
"ordinal": 30,
"name": "trigger",
"type_info": "Varchar"
},
{
"ordinal": 32,
"ordinal": 31,
"name": "trigger_kind: JobTriggerKind",
"type_info": {
"Custom": {
@@ -251,12 +246,12 @@
}
},
{
"ordinal": 33,
"ordinal": 32,
"name": "visible_to_owner",
"type_info": "Bool"
},
{
"ordinal": 34,
"ordinal": 33,
"name": "permissioned_as_end_user_email",
"type_info": "Text"
}
@@ -273,7 +268,6 @@
true,
false,
true,
true,
false,
true,
false,
@@ -304,5 +298,5 @@
null
]
},
"hash": "5bf200f2c8db25ddf231b564503c6c70f7f3958564a79bb0c6b3863b1ebb0cbf"
"hash": "6c97ab28ab47b75fb3ff39ea70fa3627f08b61bbd33aecb9ea816f8f78a04ec5"
}

View File

@@ -1,42 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT\n j.created_at AS \"created_at!\",\n c.started_at,\n c.duration_ms,\n j.created_by AS \"created_by!\"\n FROM v2_job_completed c\n JOIN v2_job j USING (id)\n WHERE c.id = $1 AND c.workspace_id = $2 AND ($3::text[] IS NULL OR j.tag = ANY($3))",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "created_at!",
"type_info": "Timestamptz"
},
{
"ordinal": 1,
"name": "started_at",
"type_info": "Timestamptz"
},
{
"ordinal": 2,
"name": "duration_ms",
"type_info": "Int8"
},
{
"ordinal": 3,
"name": "created_by!",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Uuid",
"Text",
"TextArray"
]
},
"nullable": [
false,
true,
false,
false
]
},
"hash": "73d1019fdff3113fb0c561048273ba57bf2364adebb895eba57896895b96f8d4"
}

View File

@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM debounce_key WHERE key = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text"
]
},
"nullable": []
},
"hash": "76774e6f72c8c8b7473487e4176dc17b17372b7292e39d3888a93ff4fe49e4f5"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT \n j.id, j.workspace_id, j.runnable_id AS \"runnable_id: ScriptHash\", q.scheduled_for, q.started_at, j.parent_job, j.flow_innermost_root_job, j.runnable_path, j.kind as \"kind!: JobKind\", j.permissioned_as, \n j.created_by, j.script_lang AS \"script_lang: ScriptLang\", j.permissioned_as_email, j.flow_step_id, j.trigger_kind AS \"trigger_kind: JobTriggerKind\", j.trigger, j.priority, j.concurrent_limit, j.tag, j.cache_ttl, q.cache_ignore_s3_path, q.runnable_settings_handle\n FROM v2_job j LEFT JOIN v2_job_queue q ON j.id = q.id\n WHERE j.id = $1 AND j.workspace_id = $2",
"query": "SELECT \n j.id, j.workspace_id, j.runnable_id AS \"runnable_id: ScriptHash\", q.scheduled_for, q.started_at, j.parent_job, j.flow_innermost_root_job, j.runnable_path, j.kind as \"kind!: JobKind\", j.permissioned_as, \n j.created_by, j.script_lang AS \"script_lang: ScriptLang\", j.permissioned_as_email, j.flow_step_id, j.trigger_kind AS \"trigger_kind: JobTriggerKind\", j.trigger, j.priority, j.concurrent_limit, j.tag, j.cache_ttl, q.cache_ignore_s3_path\n FROM v2_job j LEFT JOIN v2_job_queue q ON j.id = q.id\n WHERE j.id = $1 AND j.workspace_id = $2",
"describe": {
"columns": [
{
@@ -189,11 +189,6 @@
"ordinal": 20,
"name": "cache_ignore_s3_path",
"type_info": "Bool"
},
{
"ordinal": 21,
"name": "runnable_settings_handle",
"type_info": "Int8"
}
],
"parameters": {
@@ -223,9 +218,8 @@
true,
false,
true,
true,
true
]
},
"hash": "a1745a4f525b251d2f5a602ab2b2ede46b4471e21b11f607573a844013911abe"
"hash": "7b5ad10af2a9b34fa86429499ea24c0c09c6e7e9ebfa3af90035570133f7c579"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO debounce_stale_data (job_id, to_relock)\n VALUES ($1, $2)\n ON CONFLICT (job_id)\n DO UPDATE SET to_relock = (\n SELECT array_agg(DISTINCT x)\n FROM unnest(\n -- Combine existing array with new values, removing duplicates\n array_cat(debounce_stale_data.to_relock, EXCLUDED.to_relock)\n ) AS x\n )\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid",
"TextArray"
]
},
"nullable": []
},
"hash": "7ec724b84479c2f737637e91b8cbed6cae29f361167deee879b8b683ad1bf684"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "select hash, tag, concurrency_key, concurrent_limit, concurrency_time_window_s, debounce_key, debounce_delay_s, cache_ttl, cache_ignore_s3_path, runnable_settings_handle, language as \"language: ScriptLang\", dedicated_worker, priority, timeout, on_behalf_of_email, created_by FROM script\n WHERE path = $1 AND workspace_id = $2 AND archived = false AND (lock IS NOT NULL OR $3 = false)\n ORDER BY created_at DESC LIMIT 1",
"query": "select hash, tag, concurrency_key, concurrent_limit, concurrency_time_window_s, debounce_key, debounce_delay_s, cache_ttl, cache_ignore_s3_path, language as \"language: ScriptLang\", dedicated_worker, priority, timeout, on_behalf_of_email, created_by FROM script\n WHERE path = $1 AND workspace_id = $2 AND archived = false AND (lock IS NOT NULL OR $3 = false)\n ORDER BY created_at DESC LIMIT 1",
"describe": {
"columns": [
{
@@ -50,11 +50,6 @@
},
{
"ordinal": 9,
"name": "runnable_settings_handle",
"type_info": "Int8"
},
{
"ordinal": 10,
"name": "language: ScriptLang",
"type_info": {
"Custom": {
@@ -90,27 +85,27 @@
}
},
{
"ordinal": 11,
"ordinal": 10,
"name": "dedicated_worker",
"type_info": "Bool"
},
{
"ordinal": 12,
"ordinal": 11,
"name": "priority",
"type_info": "Int2"
},
{
"ordinal": 13,
"ordinal": 12,
"name": "timeout",
"type_info": "Int4"
},
{
"ordinal": 14,
"ordinal": 13,
"name": "on_behalf_of_email",
"type_info": "Text"
},
{
"ordinal": 15,
"ordinal": 14,
"name": "created_by",
"type_info": "Varchar"
}
@@ -132,7 +127,6 @@
true,
true,
true,
true,
false,
true,
true,
@@ -141,5 +135,5 @@
false
]
},
"hash": "a33673ebc4d1eb4c3513987dbc43e2c80974598e1d9fe7203145bfc29928ba65"
"hash": "7f9b7ab9bec6a0f745273d0cd5602ceab46a7ec9fd225f7b9d16a2ddb9bad7b3"
}

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO app_version\n (app_id, value, created_by, raw_app)\n SELECT app_id, value, created_by, raw_app\n FROM app_version WHERE id = $1\n RETURNING id",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Int8"
]
},
"nullable": [
false
]
},
"hash": "83232f2db5eb1b6fef744998e60420ef920d472286cf4c1f78452446a4bcb604"
}

View File

@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO v2_job_debounce_batch (id, debounce_batch)\n -- if it the first one, nextval will be evaluated, otherwise take from the job we will debounce\n SELECT\n $2,\n COALESCE(\n (\n SELECT debounce_batch\n FROM v2_job_debounce_batch\n WHERE id = $1\n LIMIT 1\n ), -- maybe use current batch\n nextval('debounce_batch_seq')\n )\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid",
"Uuid"
]
},
"nullable": []
},
"hash": "8360ab72d60f07dde6ecae599e6531b5b86862029ab51fdbdd44ec16239108e2"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT\n workspace_id,\n slack_team_id,\n teams_team_id,\n teams_team_name,\n teams_team_guid,\n slack_name,\n slack_command_script,\n teams_command_script,\n slack_email,\n slack_oauth_client_id,\n slack_oauth_client_secret,\n auto_invite_domain,\n auto_invite_operator,\n auto_add,\n customer_id,\n plan,\n webhook,\n deploy_to,\n ai_config,\n error_handler,\n error_handler_extra_args,\n error_handler_muted_on_cancel,\n large_file_storage,\n datatable,\n ducklake,\n git_sync,\n deploy_ui,\n default_app,\n default_scripts,\n mute_critical_alerts,\n color,\n operator_settings,\n git_app_installations,\n auto_add_instance_groups,\n auto_add_instance_groups_roles\n FROM\n workspace_settings\n WHERE\n workspace_id = $1\n ",
"query": "\n SELECT\n workspace_id,\n slack_team_id,\n teams_team_id,\n teams_team_name,\n slack_name,\n slack_command_script,\n teams_command_script,\n slack_email,\n slack_oauth_client_id,\n slack_oauth_client_secret,\n auto_invite_domain,\n auto_invite_operator,\n auto_add,\n customer_id,\n plan,\n webhook,\n deploy_to,\n ai_config,\n error_handler,\n error_handler_extra_args,\n error_handler_muted_on_cancel,\n large_file_storage,\n datatable,\n ducklake,\n git_sync,\n deploy_ui,\n default_app,\n default_scripts,\n mute_critical_alerts,\n color,\n operator_settings,\n git_app_installations,\n auto_add_instance_groups,\n auto_add_instance_groups_roles\n FROM\n workspace_settings\n WHERE\n workspace_id = $1\n ",
"describe": {
"columns": [
{
@@ -25,156 +25,151 @@
},
{
"ordinal": 4,
"name": "teams_team_guid",
"type_info": "Text"
},
{
"ordinal": 5,
"name": "slack_name",
"type_info": "Varchar"
},
{
"ordinal": 6,
"ordinal": 5,
"name": "slack_command_script",
"type_info": "Varchar"
},
{
"ordinal": 7,
"ordinal": 6,
"name": "teams_command_script",
"type_info": "Text"
},
{
"ordinal": 8,
"ordinal": 7,
"name": "slack_email",
"type_info": "Varchar"
},
{
"ordinal": 9,
"ordinal": 8,
"name": "slack_oauth_client_id",
"type_info": "Varchar"
},
{
"ordinal": 10,
"ordinal": 9,
"name": "slack_oauth_client_secret",
"type_info": "Varchar"
},
{
"ordinal": 11,
"ordinal": 10,
"name": "auto_invite_domain",
"type_info": "Varchar"
},
{
"ordinal": 12,
"ordinal": 11,
"name": "auto_invite_operator",
"type_info": "Bool"
},
{
"ordinal": 13,
"ordinal": 12,
"name": "auto_add",
"type_info": "Bool"
},
{
"ordinal": 14,
"ordinal": 13,
"name": "customer_id",
"type_info": "Varchar"
},
{
"ordinal": 15,
"ordinal": 14,
"name": "plan",
"type_info": "Varchar"
},
{
"ordinal": 16,
"ordinal": 15,
"name": "webhook",
"type_info": "Text"
},
{
"ordinal": 17,
"ordinal": 16,
"name": "deploy_to",
"type_info": "Varchar"
},
{
"ordinal": 18,
"ordinal": 17,
"name": "ai_config",
"type_info": "Jsonb"
},
{
"ordinal": 19,
"ordinal": 18,
"name": "error_handler",
"type_info": "Varchar"
},
{
"ordinal": 20,
"ordinal": 19,
"name": "error_handler_extra_args",
"type_info": "Json"
},
{
"ordinal": 21,
"ordinal": 20,
"name": "error_handler_muted_on_cancel",
"type_info": "Bool"
},
{
"ordinal": 22,
"ordinal": 21,
"name": "large_file_storage",
"type_info": "Jsonb"
},
{
"ordinal": 23,
"ordinal": 22,
"name": "datatable",
"type_info": "Jsonb"
},
{
"ordinal": 24,
"ordinal": 23,
"name": "ducklake",
"type_info": "Jsonb"
},
{
"ordinal": 25,
"ordinal": 24,
"name": "git_sync",
"type_info": "Jsonb"
},
{
"ordinal": 26,
"ordinal": 25,
"name": "deploy_ui",
"type_info": "Jsonb"
},
{
"ordinal": 27,
"ordinal": 26,
"name": "default_app",
"type_info": "Varchar"
},
{
"ordinal": 28,
"ordinal": 27,
"name": "default_scripts",
"type_info": "Jsonb"
},
{
"ordinal": 29,
"ordinal": 28,
"name": "mute_critical_alerts",
"type_info": "Bool"
},
{
"ordinal": 30,
"ordinal": 29,
"name": "color",
"type_info": "Varchar"
},
{
"ordinal": 31,
"ordinal": 30,
"name": "operator_settings",
"type_info": "Jsonb"
},
{
"ordinal": 32,
"ordinal": 31,
"name": "git_app_installations",
"type_info": "Jsonb"
},
{
"ordinal": 33,
"ordinal": 32,
"name": "auto_add_instance_groups",
"type_info": "TextArray"
},
{
"ordinal": 34,
"ordinal": 33,
"name": "auto_add_instance_groups_roles",
"type_info": "Jsonb"
}
@@ -192,7 +187,6 @@
true,
true,
true,
true,
false,
true,
true,
@@ -222,5 +216,5 @@
true
]
},
"hash": "3c53de373b9f1034b5f43002bf4715e12ddc641f4ca52efe0335719fa9461bb0"
"hash": "95fa60eb45228ff289655fc676991f4e90d237799f6817f292eb1391694164c7"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO raw_app (path, version, workspace_id, summary, edited_at, data, extra_perms)\n SELECT path, version, $2, summary, edited_at, data, extra_perms\n FROM raw_app\n WHERE workspace_id = $1",
"query": "INSERT INTO raw_app (path, version, workspace_id, summary, edited_at, data, extra_perms)\n SELECT path, version, $2, summary, edited_at, data, extra_perms\n FROM raw_app \n WHERE workspace_id = $1",
"describe": {
"columns": [],
"parameters": {
@@ -11,5 +11,5 @@
},
"nullable": []
},
"hash": "cde6d13d0a072e7910591335580434f0418b2ee23dc70e734568f820eec4b813"
"hash": "a202f4a0a3c2d56162a13e3593fe950d0e8ae30d7873dd4a55da6a5aa58c2aba"
}

View File

@@ -1,16 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO password (email, login_type, verified, username, name) VALUES ($1, 'saml', true, $2, $3) ON CONFLICT DO NOTHING",
"query": "INSERT INTO password (email, login_type, verified, username) VALUES ($1, 'saml', true, $2) ON CONFLICT DO NOTHING",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar"
]
},
"nullable": []
},
"hash": "638d3c2ba1198dce5b5b0e47df59a92ff8011e19fbefcc3960d6f0fe167e55b6"
"hash": "a59aac0bc593d99aedd14fd9606f11191590a12ea98a671e172b867306959884"
}

View File

@@ -1,24 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO flow_version\n (workspace_id, path, value, schema, created_by)\n\n SELECT workspace_id, path, value, schema, created_by\n FROM flow_version WHERE path = $1 AND workspace_id = $2 AND id = $3\n\n RETURNING id\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Int8"
]
},
"nullable": [
false
]
},
"hash": "a6a973dcd92d2e40fd9a1c1be42052fcd350bd47ee4f63832448b6e6f0f472f0"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT\n id,\n q.runnable_settings_handle,\n q.workspace_id,\n j.runnable_id as \"runnable_id: ScriptHash\",\n scheduled_for,\n parent_job,\n flow_innermost_root_job,\n runnable_path,\n kind as \"kind: JobKind\",\n started_at,\n permissioned_as,\n created_by,\n script_lang as \"script_lang: ScriptLang\",\n permissioned_as_email,\n flow_step_id,\n trigger_kind as \"trigger_kind: JobTriggerKind\",\n trigger,\n q.priority,\n concurrent_limit,\n q.tag,\n cache_ttl,\n cache_ignore_s3_path,\n r.ping as last_ping,\n worker,\n memory_peak,\n running\n FROM v2_job_queue q\n JOIN v2_job j USING (id)\n LEFT JOIN v2_job_runtime r USING (id)\n LEFT JOIN v2_job_status s USING (id)\n WHERE j.id = $1",
"query": "SELECT id, q.workspace_id, j.runnable_id as \"runnable_id: ScriptHash\", scheduled_for, parent_job, flow_innermost_root_job, runnable_path, kind as \"kind: JobKind\", started_at, permissioned_as, created_by, script_lang as \"script_lang: ScriptLang\", \n permissioned_as_email, flow_step_id, trigger_kind as \"trigger_kind: JobTriggerKind\", trigger, q.priority, concurrent_limit, q.tag, cache_ttl, cache_ignore_s3_path, r.ping as last_ping, worker, memory_peak, running\n FROM v2_job_queue q JOIN v2_job j USING (id) LEFT JOIN v2_job_runtime r USING (id) LEFT JOIN v2_job_status s USING (id)\n WHERE j.id = $1",
"describe": {
"columns": [
{
@@ -10,41 +10,36 @@
},
{
"ordinal": 1,
"name": "runnable_settings_handle",
"type_info": "Int8"
},
{
"ordinal": 2,
"name": "workspace_id",
"type_info": "Varchar"
},
{
"ordinal": 3,
"ordinal": 2,
"name": "runnable_id: ScriptHash",
"type_info": "Int8"
},
{
"ordinal": 4,
"ordinal": 3,
"name": "scheduled_for",
"type_info": "Timestamptz"
},
{
"ordinal": 5,
"ordinal": 4,
"name": "parent_job",
"type_info": "Uuid"
},
{
"ordinal": 6,
"ordinal": 5,
"name": "flow_innermost_root_job",
"type_info": "Uuid"
},
{
"ordinal": 7,
"ordinal": 6,
"name": "runnable_path",
"type_info": "Varchar"
},
{
"ordinal": 8,
"ordinal": 7,
"name": "kind: JobKind",
"type_info": {
"Custom": {
@@ -79,22 +74,22 @@
}
},
{
"ordinal": 9,
"ordinal": 8,
"name": "started_at",
"type_info": "Timestamptz"
},
{
"ordinal": 10,
"ordinal": 9,
"name": "permissioned_as",
"type_info": "Varchar"
},
{
"ordinal": 11,
"ordinal": 10,
"name": "created_by",
"type_info": "Varchar"
},
{
"ordinal": 12,
"ordinal": 11,
"name": "script_lang: ScriptLang",
"type_info": {
"Custom": {
@@ -130,17 +125,17 @@
}
},
{
"ordinal": 13,
"ordinal": 12,
"name": "permissioned_as_email",
"type_info": "Varchar"
},
{
"ordinal": 14,
"ordinal": 13,
"name": "flow_step_id",
"type_info": "Varchar"
},
{
"ordinal": 15,
"ordinal": 14,
"name": "trigger_kind: JobTriggerKind",
"type_info": {
"Custom": {
@@ -166,52 +161,52 @@
}
},
{
"ordinal": 16,
"ordinal": 15,
"name": "trigger",
"type_info": "Varchar"
},
{
"ordinal": 17,
"ordinal": 16,
"name": "priority",
"type_info": "Int2"
},
{
"ordinal": 18,
"ordinal": 17,
"name": "concurrent_limit",
"type_info": "Int4"
},
{
"ordinal": 19,
"ordinal": 18,
"name": "tag",
"type_info": "Varchar"
},
{
"ordinal": 20,
"ordinal": 19,
"name": "cache_ttl",
"type_info": "Int4"
},
{
"ordinal": 21,
"ordinal": 20,
"name": "cache_ignore_s3_path",
"type_info": "Bool"
},
{
"ordinal": 22,
"ordinal": 21,
"name": "last_ping",
"type_info": "Timestamptz"
},
{
"ordinal": 23,
"ordinal": 22,
"name": "worker",
"type_info": "Varchar"
},
{
"ordinal": 24,
"ordinal": 23,
"name": "memory_peak",
"type_info": "Int4"
},
{
"ordinal": 25,
"ordinal": 24,
"name": "running",
"type_info": "Bool"
}
@@ -223,7 +218,6 @@
},
"nullable": [
false,
true,
false,
true,
false,
@@ -250,5 +244,5 @@
false
]
},
"hash": "b3771b690c5966272b1f42c9965bb6a8f961c119516e4c33dc928cd3b4f4edbc"
"hash": "a84e67035584bbdb02482026b9cc0808086c50f78947d43bb88628a481f41a1d"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "WITH inserted_job AS (\n INSERT INTO v2_job (\n id, -- 1\n workspace_id, -- 2\n raw_code, -- 3\n raw_lock, -- 4\n raw_flow, -- 5\n tag, -- 6\n parent_job, -- 7\n created_by, -- 8\n permissioned_as, -- 9\n runnable_id, -- 10\n runnable_path, -- 11\n args, -- 12\n kind, -- 13\n trigger, -- 14\n script_lang, -- 15\n same_worker, -- 16\n pre_run_error, -- 17 \n permissioned_as_email, -- 18\n visible_to_owner, -- 19\n flow_innermost_root_job, -- 20\n root_job, -- 38\n concurrent_limit, -- 21\n concurrency_time_window_s, -- 22\n timeout, -- 23\n flow_step_id, -- 24\n cache_ttl, -- 25\n priority, -- 26\n trigger_kind, -- 39\n script_entrypoint_override, -- 12\n preprocessed -- 27,\n ) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18,\n $19, $20, $38, $21, $22, $23, $24, $25, $26, $39::job_trigger_kind,\n ($12::JSONB)->>'_ENTRYPOINT_OVERRIDE', $27)\n ),\n inserted_runtime AS (\n INSERT INTO v2_job_runtime (id, ping) VALUES ($1, null)\n ),\n inserted_job_perms AS (\n INSERT INTO job_perms (job_id, email, username, is_admin, is_operator, folders, groups, workspace_id, end_user_email) \n values ($1, $32, $33, $34, $35, $36, $37, $2, $41) \n ON CONFLICT (job_id) DO UPDATE SET email = EXCLUDED.email, username = EXCLUDED.username, is_admin = EXCLUDED.is_admin, is_operator = EXCLUDED.is_operator, folders = EXCLUDED.folders, groups = EXCLUDED.groups, workspace_id = EXCLUDED.workspace_id, end_user_email = EXCLUDED.end_user_email\n )\n INSERT INTO v2_job_queue\n (workspace_id, id, running, scheduled_for, started_at, tag, priority, cache_ignore_s3_path, runnable_settings_handle)\n VALUES ($2, $1, $28, COALESCE($29, now()), CASE WHEN $27 OR $40 THEN now() END, $30, $31, $42, $43)",
"query": "WITH inserted_job AS (\n INSERT INTO v2_job (id, workspace_id, raw_code, raw_lock, raw_flow, tag, parent_job,\n created_by, permissioned_as, runnable_id, runnable_path, args, kind, trigger,\n script_lang, same_worker, pre_run_error, permissioned_as_email, visible_to_owner,\n flow_innermost_root_job, root_job, concurrent_limit, concurrency_time_window_s, timeout, flow_step_id,\n cache_ttl, priority, trigger_kind, script_entrypoint_override, preprocessed)\n VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18,\n $19, $20, $38, $21, $22, $23, $24, $25, $26, $39::job_trigger_kind,\n ($12::JSONB)->>'_ENTRYPOINT_OVERRIDE', $27)\n ),\n inserted_runtime AS (\n INSERT INTO v2_job_runtime (id, ping) VALUES ($1, null)\n ),\n inserted_job_perms AS (\n INSERT INTO job_perms (job_id, email, username, is_admin, is_operator, folders, groups, workspace_id, end_user_email) \n values ($1, $32, $33, $34, $35, $36, $37, $2, $41) \n ON CONFLICT (job_id) DO UPDATE SET email = EXCLUDED.email, username = EXCLUDED.username, is_admin = EXCLUDED.is_admin, is_operator = EXCLUDED.is_operator, folders = EXCLUDED.folders, groups = EXCLUDED.groups, workspace_id = EXCLUDED.workspace_id, end_user_email = EXCLUDED.end_user_email\n )\n INSERT INTO v2_job_queue\n (workspace_id, id, running, scheduled_for, started_at, tag, priority, cache_ignore_s3_path)\n VALUES ($2, $1, $28, COALESCE($29, now()), CASE WHEN $27 OR $40 THEN now() END, $30, $31, $42)",
"describe": {
"columns": [],
"parameters": {
@@ -128,11 +128,10 @@
},
"Bool",
"Varchar",
"Bool",
"Int8"
"Bool"
]
},
"nullable": []
},
"hash": "14276a040cb4db88d71fccdc3579e8c0bb132b70668301b535872d1632753e30"
"hash": "b179a3f876ca659bed892d464bf51a733cc86a3204fcd9edccda63fddc97dced"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO instance_group (name, scim_display_name, id, external_id) VALUES ($1, $2, $3, $4)",
"query": "INSERT INTO instance_group (name, scim_display_name, id, external_id) VALUES ($1, $2, $3, $4) ON CONFLICT DO NOTHING",
"describe": {
"columns": [],
"parameters": {
@@ -13,5 +13,5 @@
},
"nullable": []
},
"hash": "7cb7dcf8b20deb59fb1c3d4ad0ca4f9a209ce0d80682182e56946392f800c24c"
"hash": "bd829646d08f68106211f97c75dce13b6fc7d35bbaf7f503dcc73ae48fd07489"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT\n imported_path,\n COUNT(DISTINCT importer_path) as \"count!\"\n FROM dependency_map\n WHERE workspace_id = $1 AND imported_path = ANY($2)\n GROUP BY imported_path\n ",
"query": "\n SELECT \n imported_path,\n COUNT(DISTINCT importer_path) as \"count!\"\n FROM dependency_map \n WHERE workspace_id = $1 AND imported_path = ANY($2)\n GROUP BY imported_path\n ",
"describe": {
"columns": [
{
@@ -25,5 +25,5 @@
null
]
},
"hash": "3306b70ea35070b9c78cebbcd803672b8412ee874005ed74883d63aacc2100e0"
"hash": "c6637102979d1acaf7fb76ff8e51badc6df20086769ffd9f9ee9e6bf1527810c"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id, workspace_id, path, value, schema, created_by, created_at\n FROM flow_version\n WHERE workspace_id = $1\n ORDER BY path, created_at",
"query": "SELECT id, workspace_id, path, value, schema, created_by, created_at \n FROM flow_version \n WHERE workspace_id = $1 \n ORDER BY path, created_at",
"describe": {
"columns": [
{
@@ -54,5 +54,5 @@
false
]
},
"hash": "c98a28cdeced035b21b8893041c4f94e2564a609eb5b8b9c562a0dbded4d5b96"
"hash": "c920a86a8a00231da69e6ef7c0a7a203e1d7aec995ad5385f99e18a5e200d866"
}

View File

@@ -1,26 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\nDELETE FROM debounce_key\nWHERE job_id IN (SELECT id FROM v2_job_completed)\nRETURNING key,job_id\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "key",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "job_id",
"type_info": "Uuid"
}
],
"parameters": {
"Left": []
},
"nullable": [
false,
false
]
},
"hash": "cc3aadd61539cfa349e65f37d04c3754d88fa8d651d8cbbd95aadfaced0c0a22"
}

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM debounce_stale_data WHERE job_id = $1 RETURNING to_relock",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "to_relock",
"type_info": "TextArray"
}
],
"parameters": {
"Left": [
"Uuid"
]
},
"nullable": [
true
]
},
"hash": "cfe06702916362aaf5122bb95593eff389e0d44b7a58b69fd5c79629599902fc"
}

View File

@@ -1,23 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "WITH ids AS (\n SELECT id as job_id FROM v2_job_debounce_batch WHERE debounce_batch = (\n SELECT debounce_batch FROM v2_job_debounce_batch WHERE id = $1\n )\n ) SELECT args->>$2 FROM ids LEFT JOIN v2_job ON v2_job.id = ids.job_id\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "?column?",
"type_info": "Text"
}
],
"parameters": {
"Left": [
"Uuid",
"Text"
]
},
"nullable": [
null
]
},
"hash": "d0e826043e5a129ae6768c274c67b6254ff6c5fd450ecdab886a3183a894d266"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT app, hash, lock, code, code_sha256\n FROM app_script\n WHERE app = ANY(SELECT id FROM app WHERE workspace_id = $1)",
"query": "SELECT app, hash, lock, code, code_sha256 \n FROM app_script \n WHERE app = ANY(SELECT id FROM app WHERE workspace_id = $1)",
"describe": {
"columns": [
{
@@ -42,5 +42,5 @@
false
]
},
"hash": "34979901e417b6eef8bda80f6213db46e3e68ee3d756c2540d924831879ce9e9"
"hash": "e0d7c895b51ea45a9dd04f79674187299c0b3f68373cf69e2fce08fdeaf185aa"
}

View File

@@ -1,20 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT COUNT(*) FROM instance_group",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "count",
"type_info": "Int8"
}
],
"parameters": {
"Left": []
},
"nullable": [
null
]
},
"hash": "e1c1e25053ae4b1635780c7e472c9d86806fa8cc762786b7a29bd121837c8ebc"
}

View File

@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "WITH _ AS (\n DELETE FROM debounce_key WHERE job_id = $1\n ) SELECT status = 'skipped' FROM v2_job_completed WHERE id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "?column?",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Uuid"
]
},
"nullable": [
null
]
},
"hash": "e65c79d792f0e8285ea9acac54bc569f22ca7c28b205533e8ba73722bf438c94"
}

View File

@@ -1,28 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT concurrency_settings, debouncing_settings FROM runnable_settings WHERE hash = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "concurrency_settings",
"type_info": "Int8"
},
{
"ordinal": 1,
"name": "debouncing_settings",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Int8"
]
},
"nullable": [
true,
true
]
},
"hash": "ebbbd069e0f33be9609604025d159fe1ecbefc2e9c11f7c4900b7121d4367e01"
}

View File

@@ -1,17 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n UPDATE workspace_settings\n SET teams_team_id = $1, teams_team_name = $2, teams_team_guid = $3\n WHERE workspace_id = $4\n AND NOT EXISTS (\n SELECT 1 FROM workspace_settings\n WHERE teams_team_id = $1 AND workspace_id <> $4\n )\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "f034f7b0118ad467c7399c5554eb916d5a9716ca0d638e3bc65509b476db378e"
}

View File

@@ -0,0 +1,24 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO flow_version\n (workspace_id, path, value, schema, created_by)\n\n SELECT workspace_id, path, value, schema, created_by\n FROM flow_version WHERE path = $1 AND workspace_id = $2 AND id = $3\n\n RETURNING id\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Int8"
]
},
"nullable": [
false
]
},
"hash": "f0efa383f2025158de160577ad839ae72faf0c8fe097e6ad6d309aee9a8aede2"
}

View File

@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO debounce_key (key, job_id)\n VALUES ($1, $2)\n ON CONFLICT (key)\n DO UPDATE SET job_id = debounce_key.job_id -- No actual change, just to trigger UPDATE\n RETURNING CASE WHEN xmax != 0 THEN job_id ELSE NULL END AS job_id",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "job_id",
"type_info": "Uuid"
}
],
"parameters": {
"Left": [
"Varchar",
"Uuid"
]
},
"nullable": [
null
]
},
"hash": "f1fe8508f6cd29d4c0d354473529fc46f506ddf98d45fc870e2855c4a4b424ea"
}

View File

@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id FROM instance_group WHERE name = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
true
]
},
"hash": "f2eb05fe3581772d985e2ace82706d824b208ee9a1cb65a85be389e14672620c"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE app SET versions = (\n SELECT array_agg(av.id ORDER BY av.created_at)\n FROM app_version av\n WHERE av.app_id = app.id\n ) WHERE workspace_id = $1",
"query": "UPDATE app SET versions = (\n SELECT array_agg(av.id ORDER BY av.created_at)\n FROM app_version av \n WHERE av.app_id = app.id\n ) WHERE workspace_id = $1",
"describe": {
"columns": [],
"parameters": {
@@ -10,5 +10,5 @@
},
"nullable": []
},
"hash": "f081475cb5e5ade3c4442bef7fb2bcdce1ebbfda56e48e66b2e5c1843049f06e"
"hash": "f95e5a80952ba84aa47def1a2f217e2966e7a90554739d7554db5706ba355a3d"
}

224
backend/Cargo.lock generated
View File

@@ -243,12 +243,9 @@ dependencies = [
[[package]]
name = "arc-swap"
version = "1.8.0"
version = "1.7.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "51d03449bb8ca2cc2ef70869af31463d1ae5ccc8fa3e334b307203fbf815207e"
dependencies = [
"rustversion",
]
checksum = "69f7f8c3906b62b754cd5326047894316021dcfe5a194c8ea52bdd94934a3457"
[[package]]
name = "argon2"
@@ -562,22 +559,6 @@ dependencies = [
"syn 2.0.111",
]
[[package]]
name = "astral-tokio-tar"
version = "0.5.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ec179a06c1769b1e42e1e2cbe74c7dcdb3d6383c838454d063eaac5bbb7ebbe5"
dependencies = [
"filetime",
"futures-core",
"libc",
"portable-atomic",
"rustc-hash 2.1.1",
"tokio",
"tokio-stream",
"xattr",
]
[[package]]
name = "async-broadcast"
version = "0.7.2"
@@ -819,9 +800,9 @@ dependencies = [
[[package]]
name = "aws-lc-rs"
version = "1.15.2"
version = "1.15.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6a88aab2464f1f25453baa7a07c84c5b7684e274054ba06817f382357f77a288"
checksum = "6b5ce75405893cd713f9ab8e297d8e438f624dde7d706108285f7e17a25a180f"
dependencies = [
"aws-lc-sys",
"zeroize",
@@ -829,9 +810,9 @@ dependencies = [
[[package]]
name = "aws-lc-sys"
version = "0.35.0"
version = "0.34.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b45afffdee1e7c9126814751f88dddc747f41d91da16c9551a0f1e8a11e788a1"
checksum = "179c3777a8b5e70e90ea426114ffc565b2c1a9f82f6c4a0c5a34aa6ef5e781b6"
dependencies = [
"cc",
"cmake",
@@ -1125,9 +1106,9 @@ dependencies = [
[[package]]
name = "aws-smithy-json"
version = "0.61.9"
version = "0.61.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "49fa1213db31ac95288d981476f78d05d9cbb0353d22cdf3472cc05bb02f6551"
checksum = "a6864c190cbb8e30cf4b77b2c8f3b6dfffa697a09b7218d2f7cd3d4c4065a9f7"
dependencies = [
"aws-smithy-types",
]
@@ -1812,9 +1793,9 @@ dependencies = [
[[package]]
name = "bumpalo"
version = "3.19.1"
version = "3.19.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5dd9dc738b7a8311c7ade152424974d8115f2cdad61e8dab8dac9f2362298510"
checksum = "46c5e41b57b8bba42a04676d81cb89e9ee8e859a1a66f80a5a72e1cb76b34d43"
dependencies = [
"allocator-api2",
]
@@ -2027,9 +2008,9 @@ dependencies = [
[[package]]
name = "cc"
version = "1.2.50"
version = "1.2.49"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9f50d563227a1c37cc0a263f64eca3334388c01c5e4c4861a9def205c614383c"
checksum = "90583009037521a116abf44494efecd645ba48b6622457080f080b85544e2215"
dependencies = [
"find-msvc-tools",
"jobserver",
@@ -2185,9 +2166,9 @@ dependencies = [
[[package]]
name = "cmake"
version = "0.1.57"
version = "0.1.55"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "75443c44cd6b379beb8c5b45d85d0773baf31cce901fe7bb252f4eff3008ef7d"
checksum = "d49d74c227b6cc9f3c51a2c7c667a05b6453f7f0f952a5f8e4493bb9e731d68e"
dependencies = [
"cc",
]
@@ -4646,18 +4627,18 @@ dependencies = [
[[package]]
name = "derive_more"
version = "2.1.1"
version = "2.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d751e9e49156b02b44f9c1815bcb94b984cdcc4396ecc32521c739452808b134"
checksum = "10b768e943bed7bf2cab53df09f4bc34bfd217cdb57d971e769874c9a6710618"
dependencies = [
"derive_more-impl",
]
[[package]]
name = "derive_more-impl"
version = "2.1.1"
version = "2.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "799a97264921d8623a957f6c3b9011f3b5492f557bbb7a5a19b7fa6d06ba8dcb"
checksum = "6d286bfdaf75e988b4a78e013ecd79c581e06399ab53fbacd2d916c2f904f30b"
dependencies = [
"proc-macro2",
"quote",
@@ -5326,7 +5307,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0ce92ff622d6dadf7349484f42c93271a0d49b7cc4d466a936405bacbe10aa78"
dependencies = [
"cfg-if",
"rustix 1.1.3",
"rustix 1.1.2",
"windows-sys 0.59.0",
]
@@ -5387,9 +5368,9 @@ checksum = "1d674e81391d1e1ab681a28d99df07927c6d4aa5b027d7da16ba32d1d21ecd99"
[[package]]
name = "flatbuffers"
version = "25.12.19"
version = "25.9.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "35f6839d7b3b98adde531effaf34f0c2badc6f4735d26fe74709d8e513a96ef3"
checksum = "09b6620799e7340ebd9968d2e0708eb82cf1971e9a16821e2091b6d6e475eed5"
dependencies = [
"bitflags 2.9.4",
"rustc_version 0.4.1",
@@ -5525,7 +5506,7 @@ version = "0.13.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8640e34b88f7652208ce9e88b1a37a2ae95227d84abec377ccd3c5cfeb141ed4"
dependencies = [
"rustix 1.1.3",
"rustix 1.1.2",
"windows-sys 0.59.0",
]
@@ -7252,9 +7233,9 @@ dependencies = [
[[package]]
name = "itoa"
version = "1.0.16"
version = "1.0.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7ee5b5339afb4c41626dde77b7a611bd4f2c202b897852b4bcf5d03eddc61010"
checksum = "4a5f13b858c8d314ee3e8f639011f7ccefe71f97f96e50151fb991f267928e2c"
[[package]]
name = "jni-sys"
@@ -7518,7 +7499,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e3c56ff45deb0031f2a476017eed60c06872251f271b8387ad8020b8fef60960"
dependencies = [
"chrono",
"derive_more 2.1.1",
"derive_more 2.1.0",
"form_urlencoded",
"http 1.4.0",
"json-patch",
@@ -7742,13 +7723,13 @@ dependencies = [
[[package]]
name = "libredox"
version = "0.1.11"
version = "0.1.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df15f6eac291ed1cf25865b1ee60399f57e7c227e7f51bdbd4c5270396a9ed50"
checksum = "416f7e718bdb06000964960ffa43b4335ad4012ae8b99060261aa4a8088d5ccb"
dependencies = [
"bitflags 2.9.4",
"libc",
"redox_syscall 0.6.0",
"redox_syscall 0.5.18",
]
[[package]]
@@ -7785,9 +7766,9 @@ dependencies = [
[[package]]
name = "libz-rs-sys"
version = "0.5.5"
version = "0.5.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c10501e7805cee23da17c7790e59df2870c0d4043ec6d03f67d31e2b53e77415"
checksum = "15413ef615ad868d4d65dce091cb233b229419c7c0c4bcaa746c0901c49ff39c"
dependencies = [
"zlib-rs",
]
@@ -8283,9 +8264,9 @@ dependencies = [
[[package]]
name = "moka"
version = "0.12.12"
version = "0.12.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a3dec6bd31b08944e08b58fd99373893a6c17054d6f3ea5006cc894f4f4eee2a"
checksum = "8261cd88c312e0004c1d51baad2980c66528dfdb2bee62003e643a4d8f86b077"
dependencies = [
"crossbeam-channel",
"crossbeam-epoch",
@@ -8293,6 +8274,7 @@ dependencies = [
"equivalent",
"parking_lot",
"portable-atomic",
"rustc_version 0.4.1",
"smallvec",
"tagptr",
"uuid",
@@ -8641,9 +8623,9 @@ dependencies = [
[[package]]
name = "ntapi"
version = "0.4.2"
version = "0.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c70f219e21142367c70c0b30c6a9e3a14d55b4d12a204d897fbec83a0363f081"
checksum = "e8a3895c6391c39d7fe7ebc444a87eb2991b2a0bc718fdabd071eec617fc68e4"
dependencies = [
"winapi",
]
@@ -9819,9 +9801,9 @@ dependencies = [
[[package]]
name = "portable-atomic"
version = "1.12.0"
version = "1.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f59e70c4aef1e55797c2e8fd94a4f2a973fc972cfde0e0b05f683667b0cd39dd"
checksum = "f84267b20a16ea918e43c6a88433c2d54fa145c92a811b5b047ccbe153674483"
[[package]]
name = "postgres-native-tls"
@@ -10572,18 +10554,18 @@ dependencies = [
[[package]]
name = "redox_syscall"
version = "0.5.18"
version = "0.3.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed2bf2547551a7053d6fdfafda3f938979645c44812fbfcda098faae3f1a362d"
checksum = "567664f262709473930a4bf9e51bf2ebf3348f2e748ccc50dea20646858f8f29"
dependencies = [
"bitflags 2.9.4",
"bitflags 1.3.2",
]
[[package]]
name = "redox_syscall"
version = "0.6.0"
version = "0.5.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ec96166dafa0886eb81fe1c0a388bece180fbef2135f97c1e2cf8302e74b43b5"
checksum = "ed2bf2547551a7053d6fdfafda3f938979645c44812fbfcda098faae3f1a362d"
dependencies = [
"bitflags 2.9.4",
]
@@ -11132,9 +11114,9 @@ dependencies = [
[[package]]
name = "rustix"
version = "1.1.3"
version = "1.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "146c9e247ccc180c1f61615433868c99f3de3ae256a30a43b49f67c2d9171f34"
checksum = "cd15f8a2c5551a84d56efdc1cd049089e409ac19a3072d5037a17fd70719ff3e"
dependencies = [
"bitflags 2.9.4",
"errno",
@@ -11242,9 +11224,9 @@ dependencies = [
[[package]]
name = "rustls-pki-types"
version = "1.13.2"
version = "1.13.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "21e6f2ab2928ca4291b86736a8bd920a277a399bba1589409d72154ff87c1282"
checksum = "708c0f9d5f54ba0272468c1d306a52c495b31fa155e91bc25371e6df7996908c"
dependencies = [
"web-time",
"zeroize",
@@ -11382,9 +11364,9 @@ dependencies = [
[[package]]
name = "ryu"
version = "1.0.21"
version = "1.0.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "62049b2877bf12821e8f9ad256ee38fdc31db7387ec2d3b3f403024de2034aea"
checksum = "28d3b2b1366ec20994f1fd18c3c594f05c5dd4bc44d8bb0c1c632c8d6829481f"
[[package]]
name = "ryu-js"
@@ -12645,9 +12627,9 @@ dependencies = [
[[package]]
name = "supports-hyperlinks"
version = "3.2.0"
version = "3.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e396b6523b11ccb83120b115a0b7366de372751aa6edf19844dfb13a6af97e91"
checksum = "804f44ed3c63152de6a9f90acbea1a110441de43006ea51bcce8f436196a288b"
[[package]]
name = "supports-unicode"
@@ -13362,14 +13344,14 @@ dependencies = [
[[package]]
name = "tempfile"
version = "3.24.0"
version = "3.23.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "655da9c7eb6305c55742045d5a8d2037996d61d8de95806335c7c86ce0f82e9c"
checksum = "2d31c77bdf42a745371d260a26ca7163f1e0924b64afa0b688e61b5a9fa02f16"
dependencies = [
"fastrand",
"getrandom 0.3.4",
"once_cell",
"rustix 1.1.3",
"rustix 1.1.2",
"windows-sys 0.61.2",
]
@@ -13388,7 +13370,7 @@ version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "60b8cb979cb11c32ce1603f8137b22262a9d131aaa5c37b5678025f22b8becd0"
dependencies = [
"rustix 1.1.3",
"rustix 1.1.2",
"windows-sys 0.60.2",
]
@@ -13834,6 +13816,21 @@ dependencies = [
"tokio",
]
[[package]]
name = "tokio-tar"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9d5714c010ca3e5c27114c1cdeb9d14641ace49874aa5626d7149e47aedace75"
dependencies = [
"filetime",
"futures-core",
"libc",
"redox_syscall 0.3.5",
"tokio",
"tokio-stream",
"xattr",
]
[[package]]
name = "tokio-tungstenite"
version = "0.24.0"
@@ -13943,9 +13940,9 @@ dependencies = [
[[package]]
name = "toml_parser"
version = "1.0.6+spec-1.1.0"
version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a3198b4b0a8e11f09dd03e133c0280504d0801269e9afa46362ffde1cbeebf44"
checksum = "c0cbe268d35bdb4bb5a56a2de88d0ad0eb70af5384a99d648cd4b3d04039800e"
dependencies = [
"winnow 0.7.14",
]
@@ -14079,9 +14076,9 @@ checksum = "8df9b6e13f2d32c91b9bd719c00d1958837bc7dec474d94952798cc8e69eeec3"
[[package]]
name = "tracing"
version = "0.1.44"
version = "0.1.43"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "63e71662fa4b2a2c3a26f570f037eb95bb1f85397f3cd8076caed2f026a6d100"
checksum = "2d15d90a0b5c19378952d479dc858407149d7bb45a14de0142f6c534b16fc647"
dependencies = [
"log",
"pin-project-lite",
@@ -14114,9 +14111,9 @@ dependencies = [
[[package]]
name = "tracing-core"
version = "0.1.36"
version = "0.1.35"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "db97caf9d906fbde555dd62fa95ddba9eecfd14cb388e4f491a66d74cd5fb79a"
checksum = "7a04e24fab5c89c6a36eb8558c9656f30d81de51dfa4d3b45f26b21d61fa0a6c"
dependencies = [
"once_cell",
"valuable",
@@ -14150,7 +14147,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ba3beec919fbdf99d719de8eda6adae3281f8a5b71ae40431f44dc7423053d34"
dependencies = [
"loki-api",
"reqwest 0.12.24",
"reqwest 0.11.27",
"serde",
"serde_json",
"snap",
@@ -15168,7 +15165,7 @@ checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
[[package]]
name = "windmill"
version = "1.599.2"
version = "1.592.1"
dependencies = [
"anyhow",
"aws-sdk-config",
@@ -15204,7 +15201,6 @@ dependencies = [
"sha1",
"sha2 0.10.9",
"size",
"sql-builder",
"sqlx",
"strum 0.27.2",
"systemstat",
@@ -15231,11 +15227,10 @@ dependencies = [
[[package]]
name = "windmill-api"
version = "1.599.2"
version = "1.592.1"
dependencies = [
"anyhow",
"argon2",
"astral-tokio-tar",
"async-nats",
"async-oauth2",
"async-recursion",
@@ -15243,7 +15238,6 @@ dependencies = [
"async-trait",
"async_zip",
"aws-config",
"aws-credential-types",
"aws-sdk-config",
"aws-sdk-sqs",
"aws-sdk-sso",
@@ -15326,6 +15320,7 @@ dependencies = [
"tokio-postgres 0.7.11",
"tokio-postgres 0.7.13",
"tokio-stream",
"tokio-tar",
"tokio-tungstenite",
"tokio-util",
"tonic",
@@ -15353,7 +15348,7 @@ dependencies = [
[[package]]
name = "windmill-api-client"
version = "1.599.2"
version = "1.592.1"
dependencies = [
"base64 0.22.1",
"chrono",
@@ -15368,7 +15363,7 @@ dependencies = [
[[package]]
name = "windmill-audit"
version = "1.599.2"
version = "1.592.1"
dependencies = [
"chrono",
"lazy_static",
@@ -15382,7 +15377,7 @@ dependencies = [
[[package]]
name = "windmill-autoscaling"
version = "1.599.2"
version = "1.592.1"
dependencies = [
"anyhow",
"axum",
@@ -15401,7 +15396,7 @@ dependencies = [
[[package]]
name = "windmill-common"
version = "1.599.2"
version = "1.592.1"
dependencies = [
"anyhow",
"async-recursion",
@@ -15439,7 +15434,6 @@ dependencies = [
"lazy_static",
"magic-crypt",
"mail-send",
"native-tls",
"object_store",
"once_cell",
"openidconnect",
@@ -15451,7 +15445,6 @@ dependencies = [
"pep440_rs",
"phf 0.11.3",
"pin-project-lite",
"postgres-native-tls 0.5.1",
"prometheus",
"quick_cache",
"rand 0.9.0",
@@ -15476,7 +15469,6 @@ dependencies = [
"thiserror 2.0.17",
"tikv-jemalloc-ctl",
"tokio",
"tokio-postgres 0.7.13",
"tokio-stream",
"tokio-util",
"tonic",
@@ -15497,7 +15489,7 @@ dependencies = [
[[package]]
name = "windmill-git-sync"
version = "1.599.2"
version = "1.592.1"
dependencies = [
"regex",
"serde",
@@ -15512,10 +15504,9 @@ dependencies = [
[[package]]
name = "windmill-indexer"
version = "1.599.2"
version = "1.592.1"
dependencies = [
"anyhow",
"astral-tokio-tar",
"bytes",
"chrono",
"const_format",
@@ -15529,6 +15520,7 @@ dependencies = [
"tantivy",
"tempfile",
"tokio",
"tokio-tar",
"tracing",
"uuid",
"windmill-common",
@@ -15536,7 +15528,7 @@ dependencies = [
[[package]]
name = "windmill-macros"
version = "1.599.2"
version = "1.592.1"
dependencies = [
"itertools 0.14.0",
"lazy_static",
@@ -15552,7 +15544,7 @@ dependencies = [
[[package]]
name = "windmill-parser"
version = "1.599.2"
version = "1.592.1"
dependencies = [
"convert_case 0.6.0",
"serde",
@@ -15561,7 +15553,7 @@ dependencies = [
[[package]]
name = "windmill-parser-bash"
version = "1.599.2"
version = "1.592.1"
dependencies = [
"anyhow",
"lazy_static",
@@ -15573,7 +15565,7 @@ dependencies = [
[[package]]
name = "windmill-parser-csharp"
version = "1.599.2"
version = "1.592.1"
dependencies = [
"anyhow",
"serde_json",
@@ -15585,7 +15577,7 @@ dependencies = [
[[package]]
name = "windmill-parser-go"
version = "1.599.2"
version = "1.592.1"
dependencies = [
"anyhow",
"gosyn",
@@ -15597,7 +15589,7 @@ dependencies = [
[[package]]
name = "windmill-parser-graphql"
version = "1.599.2"
version = "1.592.1"
dependencies = [
"anyhow",
"lazy_static",
@@ -15609,7 +15601,7 @@ dependencies = [
[[package]]
name = "windmill-parser-java"
version = "1.599.2"
version = "1.592.1"
dependencies = [
"anyhow",
"serde_json",
@@ -15621,7 +15613,7 @@ dependencies = [
[[package]]
name = "windmill-parser-nu"
version = "1.599.2"
version = "1.592.1"
dependencies = [
"anyhow",
"nu-parser",
@@ -15632,7 +15624,7 @@ dependencies = [
[[package]]
name = "windmill-parser-php"
version = "1.599.2"
version = "1.592.1"
dependencies = [
"anyhow",
"itertools 0.14.0",
@@ -15643,7 +15635,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py"
version = "1.599.2"
version = "1.592.1"
dependencies = [
"anyhow",
"itertools 0.14.0",
@@ -15656,7 +15648,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py-imports"
version = "1.599.2"
version = "1.592.1"
dependencies = [
"anyhow",
"async-recursion",
@@ -15680,7 +15672,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ruby"
version = "1.599.2"
version = "1.592.1"
dependencies = [
"anyhow",
"lazy_static",
@@ -15694,7 +15686,7 @@ dependencies = [
[[package]]
name = "windmill-parser-rust"
version = "1.599.2"
version = "1.592.1"
dependencies = [
"anyhow",
"convert_case 0.6.0",
@@ -15711,7 +15703,7 @@ dependencies = [
[[package]]
name = "windmill-parser-sql"
version = "1.599.2"
version = "1.592.1"
dependencies = [
"anyhow",
"lazy_static",
@@ -15725,7 +15717,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ts"
version = "1.599.2"
version = "1.592.1"
dependencies = [
"anyhow",
"lazy_static",
@@ -15744,7 +15736,7 @@ dependencies = [
[[package]]
name = "windmill-parser-yaml"
version = "1.599.2"
version = "1.592.1"
dependencies = [
"anyhow",
"serde",
@@ -15755,7 +15747,7 @@ dependencies = [
[[package]]
name = "windmill-queue"
version = "1.599.2"
version = "1.592.1"
dependencies = [
"anyhow",
"async-recursion",
@@ -15792,7 +15784,7 @@ dependencies = [
[[package]]
name = "windmill-sql-datatype-parser-wasm"
version = "1.599.2"
version = "1.592.1"
dependencies = [
"wasm-bindgen",
"wasm-bindgen-test",
@@ -15802,7 +15794,7 @@ dependencies = [
[[package]]
name = "windmill-worker"
version = "1.599.2"
version = "1.592.1"
dependencies = [
"anyhow",
"async-once-cell",
@@ -16511,7 +16503,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32e45ad4206f6d2479085147f02bc2ef834ac85886624a23575ae137c8aa8156"
dependencies = [
"libc",
"rustix 1.1.3",
"rustix 1.1.2",
]
[[package]]
@@ -16708,9 +16700,9 @@ dependencies = [
[[package]]
name = "zlib-rs"
version = "0.5.5"
version = "0.5.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "40990edd51aae2c2b6907af74ffb635029d5788228222c4bb811e9351c0caad3"
checksum = "51f936044d677be1a1168fae1d03b583a285a5dd9d8cbf7b24c23aa1fc775235"
[[package]]
name = "zstd"

View File

@@ -1,6 +1,6 @@
[package]
name = "windmill"
version = "1.599.2"
version = "1.592.1"
authors.workspace = true
edition.workspace = true
@@ -33,7 +33,7 @@ members = [
exclude = ["./windmill-duckdb-ffi-internal"]
[workspace.package]
version = "1.599.2"
version = "1.592.1"
authors = ["Ruben Fiszel <ruben@windmill.dev>"]
edition = "2021"
@@ -125,7 +125,6 @@ windmill-autoscaling = { workspace = true, optional = true }
futures.workspace = true
tracing.workspace = true
sqlx.workspace = true
sql-builder.workspace = true
rand.workspace = true
chrono.workspace = true
git-version.workspace = true
@@ -260,7 +259,7 @@ reqwest = { version = "=0.12.24", features = ["json", "stream", "gzip", "multipa
eventsource-stream = "0.2.3"
time = "^0"
serde_urlencoded = "^0"
astral-tokio-tar = "^0.5.6"
tokio-tar = "^0"
tempfile = "^3"
tokio-util = { version = "^0", features = ["io"] }
json-pointer = "^0"

View File

@@ -1 +1 @@
5c2a8854e7ff014063a69dd8f7829a935129c31e
4897449185f1af0f0323c931df288b5942b38fb8

View File

@@ -43,7 +43,7 @@ def load_openapi_spec(file_path: str) -> Dict[str, Any]:
print(f"Error loading OpenAPI spec: {e}", file=sys.stderr)
sys.exit(1)
def extract_separate_schemas(parameters: List[Dict[str, Any]], request_body: Optional[Dict[str, Any]], spec: Dict[str, Any], required_fields: Optional[List[str]] = None, base_path: str = "") -> tuple:
def extract_separate_schemas(parameters: List[Dict[str, Any]], request_body: Optional[Dict[str, Any]], spec: Dict[str, Any], required_fields: Optional[List[str]] = None) -> tuple:
"""Extract separate schemas for path parameters, query parameters, and request body."""
path_params_schema = {
"type": "object",
@@ -63,16 +63,16 @@ def extract_separate_schemas(parameters: List[Dict[str, Any]], request_body: Opt
for param in parameters:
# Resolve $ref if present
if '$ref' in param:
param = resolve_schema_refs(param, spec, base_path)
param = resolve_schema_refs(param, spec)
param_name = param.get('name', '')
param_schema = param.get('schema', {'type': 'string'})
param_required = param.get('required', False)
param_description = param.get('description', '')
param_in = param.get('in', 'query')
# Resolve any refs in the parameter schema
param_schema = resolve_schema_refs(param_schema, spec, base_path)
param_schema = resolve_schema_refs(param_schema, spec)
# Add description if available
if param_description:
@@ -93,7 +93,7 @@ def extract_separate_schemas(parameters: List[Dict[str, Any]], request_body: Opt
# Process request body if present
if request_body:
body_schema = extract_request_body_schema(request_body, spec, base_path)
body_schema = extract_request_body_schema(request_body, spec)
# If we have required fields specified and a body schema, update the required array
if body_schema and required_fields:
@@ -115,109 +115,68 @@ def extract_separate_schemas(parameters: List[Dict[str, Any]], request_body: Opt
return (path_params_schema, query_params_schema, body_schema)
# Cache for loaded external files
_external_file_cache: Dict[str, Dict[str, Any]] = {}
def load_external_file(file_path: str, base_path: str) -> Optional[Dict[str, Any]]:
"""Load an external YAML file relative to the base path."""
if file_path in _external_file_cache:
return _external_file_cache[file_path]
try:
import yaml
from pathlib import Path
# Resolve the path relative to the base file
base_dir = Path(base_path).parent
full_path = (base_dir / file_path).resolve()
with open(full_path, 'r', encoding='utf-8') as f:
content = yaml.safe_load(f)
_external_file_cache[file_path] = content
return content
except Exception as e:
print(f"Warning: Could not load external file {file_path}: {e}", file=sys.stderr)
return None
def resolve_ref(ref_path: str, spec: Dict[str, Any], base_path: str = "") -> tuple:
"""Resolve a $ref path to the actual schema definition.
Handles both internal refs (#/...) and external file refs (file.yaml#/...).
Returns a tuple of (resolved_schema, resolved_spec) where resolved_spec is the spec
that should be used for resolving any nested refs within the resolved schema.
"""
# Check if this is an external file reference
if '#' in ref_path and not ref_path.startswith('#'):
# External file reference: "../../openflow.openapi.yaml#/components/schemas/Retry"
file_part, fragment = ref_path.split('#', 1)
external_spec = load_external_file(file_part, base_path)
if external_spec is None:
return None, spec
# Resolve the fragment within the external file, and return external_spec for nested refs
resolved, _ = resolve_ref('#' + fragment, external_spec, base_path)
return resolved, external_spec
def resolve_ref(ref_path: str, spec: Dict[str, Any]) -> Optional[Dict[str, Any]]:
"""Resolve a $ref path to the actual schema definition."""
if not ref_path.startswith('#/'):
return None, spec
return None
# Remove the '#/' prefix and split by '/'
path_parts = ref_path[2:].split('/')
# Navigate through the spec following the path
current = spec
for part in path_parts:
if isinstance(current, dict) and part in current:
current = current[part]
else:
return None, spec
return None
return current if isinstance(current, dict) else None
return (current if isinstance(current, dict) else None), spec
def resolve_schema_refs(schema: Dict[str, Any], spec: Dict[str, Any], base_path: str = "") -> Dict[str, Any]:
def resolve_schema_refs(schema: Dict[str, Any], spec: Dict[str, Any]) -> Dict[str, Any]:
"""Recursively resolve all $ref references in a schema."""
if not isinstance(schema, dict):
return schema
# If this is a $ref, resolve it
if '$ref' in schema:
ref_path = schema['$ref']
resolved, resolved_spec = resolve_ref(ref_path, spec, base_path)
resolved = resolve_ref(ref_path, spec)
if resolved:
# Recursively resolve any refs in the resolved schema using the appropriate spec
return resolve_schema_refs(resolved, resolved_spec, base_path)
# Recursively resolve any refs in the resolved schema
return resolve_schema_refs(resolved, spec)
else:
print(f"Warning: Could not resolve $ref: {ref_path}")
return schema
# Recursively process all values in the schema
resolved_schema = {}
for key, value in schema.items():
if isinstance(value, dict):
resolved_schema[key] = resolve_schema_refs(value, spec, base_path)
resolved_schema[key] = resolve_schema_refs(value, spec)
elif isinstance(value, list):
resolved_schema[key] = [
resolve_schema_refs(item, spec, base_path) if isinstance(item, dict) else item
resolve_schema_refs(item, spec) if isinstance(item, dict) else item
for item in value
]
else:
resolved_schema[key] = value
return resolved_schema
def extract_request_body_schema(request_body: Dict[str, Any], spec: Dict[str, Any], base_path: str = "") -> Optional[Dict[str, Any]]:
def extract_request_body_schema(request_body: Dict[str, Any], spec: Dict[str, Any]) -> Optional[Dict[str, Any]]:
"""Extract request body schema from OpenAPI requestBody definition and resolve refs."""
if not request_body:
return None
content = request_body.get('content', {})
json_content = content.get('application/json', {})
schema = json_content.get('schema', {})
if schema:
# Resolve any $ref references in the schema
return resolve_schema_refs(schema, spec, base_path)
return resolve_schema_refs(schema, spec)
return None
def http_method_to_rust(method: str) -> str:
@@ -262,7 +221,7 @@ def find_mcp_tools(spec: Dict[str, Any]) -> List[Dict[str, Any]]:
return tools
def generate_typescript_code(tools: List[Dict[str, Any]], spec: Dict[str, Any], base_path: str = "") -> str:
def generate_typescript_code(tools: List[Dict[str, Any]], spec: Dict[str, Any]) -> str:
"""Generate TypeScript code with MCP endpoint tools."""
if not tools:
return """// Auto-generated MCP tools from OpenAPI specification
@@ -293,7 +252,7 @@ export const mcpEndpointTools: EndpointTool[] = [];
# Generate separate schemas
path_params_schema, query_params_schema, body_schema = extract_separate_schemas(
tool['parameters'], tool['requestBody'], spec, tool['required_fields'], base_path
tool['parameters'], tool['requestBody'], spec, tool['required_fields']
)
# Convert schemas to TypeScript - use 'as const' for better type inference
@@ -338,7 +297,7 @@ export const mcpEndpointTools: EndpointTool[] = [
return typescript_code
def generate_rust_code(tools: List[Dict[str, Any]], spec: Dict[str, Any], base_path: str = "") -> str:
def generate_rust_code(tools: List[Dict[str, Any]], spec: Dict[str, Any]) -> str:
"""Generate the complete Rust code with MCP tools."""
if not tools:
return """// No MCP tools found in the OpenAPI specification
@@ -361,7 +320,7 @@ pub fn all_tools() -> Vec<EndpointTool> {
# Generate separate schemas
path_params_schema, query_params_schema, body_schema = extract_separate_schemas(
tool['parameters'], tool['requestBody'], spec, tool['required_fields'], base_path
tool['parameters'], tool['requestBody'], spec, tool['required_fields']
)
path_params_rust = schema_to_rust_value(path_params_schema)
@@ -427,7 +386,7 @@ def main():
# Generate and write Rust code
print(f"Generating Rust code...")
rust_code = generate_rust_code(tools, spec, str(openapi_file))
rust_code = generate_rust_code(tools, spec)
print(f"Writing Rust code to: {rust_output_file}")
rust_output_file.parent.mkdir(parents=True, exist_ok=True)
@@ -436,7 +395,7 @@ def main():
# Generate and write TypeScript code
print(f"Generating TypeScript code...")
typescript_code = generate_typescript_code(tools, spec, str(openapi_file))
typescript_code = generate_typescript_code(tools, spec)
print(f"Writing TypeScript code to: {ts_output_file}")
ts_output_file.parent.mkdir(parents=True, exist_ok=True)

View File

@@ -1,11 +0,0 @@
ALTER TABLE v2_job_queue
DROP COLUMN runnable_settings_handle;
ALTER TABLE script
DROP COLUMN runnable_settings_handle;
DROP TABLE IF EXISTS job_settings;
DROP TABLE IF EXISTS runnable_settings;
DROP TABLE IF EXISTS concurrency_settings;
DROP TABLE IF EXISTS debouncing_settings;

View File

@@ -1,42 +0,0 @@
CREATE TABLE IF NOT EXISTS concurrency_settings(
hash BIGINT PRIMARY KEY,
concurrency_key VARCHAR(255),
concurrent_limit INTEGER,
concurrency_time_window_s INTEGER
);
CREATE TABLE IF NOT EXISTS debouncing_settings(
hash BIGINT PRIMARY KEY,
debounce_key VARCHAR(255),
debounce_delay_s INTEGER,
max_total_debouncing_time INTEGER,
max_total_debounces_amount INTEGER,
debounce_args_to_accumulate TEXT[]
);
CREATE TABLE IF NOT EXISTS runnable_settings(
hash BIGINT PRIMARY KEY,
debouncing_settings BIGINT DEFAULT NULL,
concurrency_settings BIGINT DEFAULT NULL
);
CREATE TABLE IF NOT EXISTS job_settings(
job_id UUID PRIMARY KEY,
runnable_settings BIGINT DEFAULT NULL
);
ALTER TABLE script
ADD COLUMN runnable_settings_handle BIGINT DEFAULT NULL;
ALTER TABLE v2_job_queue
ADD COLUMN runnable_settings_handle BIGINT DEFAULT NULL;
GRANT ALL ON concurrency_settings TO windmill_admin;
GRANT ALL ON concurrency_settings TO windmill_user;
GRANT ALL ON debouncing_settings TO windmill_admin;
GRANT ALL ON debouncing_settings TO windmill_user;
GRANT ALL ON runnable_settings TO windmill_admin;
GRANT ALL ON runnable_settings TO windmill_user;
GRANT ALL ON job_settings TO windmill_admin;
GRANT ALL ON job_settings TO windmill_user;

View File

@@ -1,38 +0,0 @@
-- Grant CREATE privilege on all databases where custom_instance_user has CONNECT
-- This allows custom_instance_user to create schemas in databases it can already access
DO $$
DECLARE
db_record RECORD;
grant_command TEXT;
BEGIN
-- Find all databases where custom_instance_user has CONNECT privilege
-- We check if the datacl array contains an entry for custom_instance_user with 'c' (CONNECT) privilege
FOR db_record IN
SELECT d.datname
FROM pg_database d
WHERE d.datname NOT IN ('template0', 'template1') -- Skip template databases
AND d.datallowconn = true -- Only consider databases that allow connections
AND d.datacl IS NOT NULL -- Has ACL entries
AND EXISTS (
SELECT 1
FROM unnest(d.datacl) AS acl_entry
WHERE acl_entry::text LIKE 'custom_instance_user=c/%' -- 'c' is the privilege code for CONNECT
)
LOOP
BEGIN
-- Grant CREATE privilege on the database
EXECUTE format('GRANT CREATE ON DATABASE %I TO custom_instance_user', db_record.datname);
RAISE NOTICE 'Granted CREATE on database % to custom_instance_user', db_record.datname;
EXCEPTION
WHEN others THEN
RAISE NOTICE 'Failed to grant CREATE on database %: %', db_record.datname, SQLERRM;
END;
END LOOP;
RAISE NOTICE 'Completed granting CREATE privileges to custom_instance_user on all accessible databases';
EXCEPTION
WHEN others THEN
RAISE NOTICE 'Error in custom_instance_user CREATE privilege migration: %', SQLERRM;
-- Continue without failing the migration
END
$$;

View File

@@ -1,6 +0,0 @@
ALTER TABLE debounce_key DROP COLUMN IF EXISTS debounced_times;
ALTER TABLE debounce_key DROP COLUMN IF EXISTS first_started_at;
ALTER TABLE debounce_key DROP COLUMN IF EXISTS previous_job_id;
DROP INDEX IF EXISTS idx_v2_job_debounce_batch_debounce_batch;
DROP TABLE IF EXISTS v2_job_debounce_batch;
DROP SEQUENCE IF EXISTS debounce_batch_seq;

View File

@@ -1,13 +0,0 @@
CREATE SEQUENCE debounce_batch_seq START 1;
CREATE TABLE IF NOT EXISTS v2_job_debounce_batch(
id UUID PRIMARY KEY,
debounce_batch BIGINT NOT NULL DEFAULT nextval('debounce_batch_seq')
);
CREATE INDEX IF NOT EXISTS idx_v2_job_debounce_batch_debounce_batch ON v2_job_debounce_batch(debounce_batch);
ALTER TABLE debounce_key ADD COLUMN IF NOT EXISTS previous_job_id UUID;
ALTER TABLE debounce_key ADD COLUMN IF NOT EXISTS first_started_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now();
ALTER TABLE debounce_key ADD COLUMN IF NOT EXISTS debounced_times INTEGER NOT NULL DEFAULT 0;

View File

@@ -1 +0,0 @@
-- Add down migration script here

View File

@@ -1,6 +0,0 @@
-- Add up migration script here
GRANT ALL ON SEQUENCE folder_permission_history_id_seq TO windmill_user;
GRANT ALL ON SEQUENCE folder_permission_history_id_seq TO windmill_admin;
GRANT ALL ON SEQUENCE group_permission_history_id_seq TO windmill_user;
GRANT ALL ON SEQUENCE group_permission_history_id_seq TO windmill_admin;

View File

@@ -1,2 +0,0 @@
-- Drop flow_iterator_data table
DROP TABLE IF EXISTS flow_iterator_data;

View File

@@ -1,9 +0,0 @@
-- Create separate table for storing flow iterator data (itered arrays)
-- This avoids expensive JSONB_SET operations on large itered arrays during parallel loop execution
CREATE TABLE IF NOT EXISTS flow_iterator_data (
job_id UUID PRIMARY KEY REFERENCES v2_job_queue (id) ON DELETE CASCADE NOT NULL,
itered JSONB NOT NULL,
created_at TIMESTAMP WITH TIME ZONE DEFAULT now()
);
-- Index not needed beyond primary key since all lookups are by job_id

View File

@@ -1 +0,0 @@
ALTER TABLE workspace_settings DROP COLUMN teams_team_guid;

View File

@@ -1,3 +0,0 @@
-- Add teams_team_guid column to store the GUID (used for MS Graph API calls)
-- The existing teams_team_id column stores the internal_id (used for webhook matching)
ALTER TABLE workspace_settings ADD COLUMN teams_team_guid TEXT;

View File

@@ -3,11 +3,11 @@ use rustpython_parser::{ast::Suite, Parse};
use std::collections::HashMap;
use windmill_parser::asset_parser::{
asset_was_used, merge_assets, parse_asset_syntax, AssetKind, AssetUsageAccessType,
ParseAssetsOutput, ParseAssetsResult,
ParseAssetsResult,
};
use AssetUsageAccessType::*;
pub fn parse_assets(input: &str) -> anyhow::Result<ParseAssetsOutput> {
pub fn parse_assets(input: &str) -> anyhow::Result<Vec<ParseAssetsResult>> {
let ast = Suite::parse(input, "main.py")
.map_err(|e| anyhow::anyhow!("Error parsing code: {}", e.to_string()))?;
@@ -15,7 +15,7 @@ pub fn parse_assets(input: &str) -> anyhow::Result<ParseAssetsOutput> {
ast.into_iter()
.for_each(|stmt| assets_finder.visit_stmt(stmt));
for (kind, path, _) in assets_finder.var_identifiers.into_values() {
for (kind, path) in assets_finder.var_identifiers.into_values() {
// if a db = wmill.datatable() was never used (e.g db.query(...)),
// we still want to register the asset as unknown access type
if asset_was_used(&assets_finder.assets, (kind, &path)) == false {
@@ -25,14 +25,12 @@ pub fn parse_assets(input: &str) -> anyhow::Result<ParseAssetsOutput> {
}
}
Ok(ParseAssetsOutput { assets: merge_assets(assets_finder.assets), ..Default::default() })
Ok(merge_assets(assets_finder.assets))
}
type VarAssetName = String;
type VarAssetSchema = Option<String>;
struct AssetsFinder {
assets: Vec<ParseAssetsResult>,
var_identifiers: HashMap<String, (AssetKind, VarAssetName, VarAssetSchema)>,
var_identifiers: HashMap<String, (AssetKind, String)>,
}
impl Visitor for AssetsFinder {
@@ -46,7 +44,7 @@ impl Visitor for AssetsFinder {
// if a db = wmill.datatable() or similar was removed, but never used (e.g db.query(...)),
// we still want to register the asset as unknown access type
match removed {
Some((kind, path, _)) => {
Some((kind, path)) => {
if !asset_was_used(&self.assets, (kind, &path)) {
self.assets
.push(ParseAssetsResult { kind, access_type: None, path });
@@ -55,11 +53,11 @@ impl Visitor for AssetsFinder {
None => {}
}
if let Some((kind, name, schema)) = self.extract_asset_from_call(&node.value) {
if let Some((kind, name)) = self.extract_asset_from_call(&node.value) {
// Track target variable
let Ok(var_name) = expr_name.id.parse::<String>();
self.var_identifiers
.insert(var_name, (kind.clone(), name.clone(), schema.clone()));
.insert(var_name, (kind.clone(), name.clone()));
}
}
// Continue with generic visit to catch any other assets in the expression
@@ -109,10 +107,7 @@ impl Visitor for AssetsFinder {
impl AssetsFinder {
/// Extract asset info from calls like wmill.datatable('name'), wmill.ducklake('name'), etc.
fn extract_asset_from_call(
&self,
expr: &Expr,
) -> Option<(AssetKind, VarAssetName, VarAssetSchema)> {
fn extract_asset_from_call(&self, expr: &Expr) -> Option<(AssetKind, String)> {
let call = expr.as_call_expr()?;
// Check for wmill.datatable, wmill.ducklake pattern
@@ -148,21 +143,10 @@ impl AssetsFinder {
} else {
None
}
});
let (name, schema) = match name {
None => ("main".to_string(), None),
Some(name) => {
if let Some((name, s)) = name.split_once(':') {
let schema = Some(s.to_string());
let name = if name.is_empty() { "main" } else { name };
(name.to_string(), schema)
} else {
(name, None)
}
}
};
})
.unwrap_or_else(|| "main".to_string());
Some((kind, name, schema))
Some((kind, name))
}
fn visit_expr_call_inner(&mut self, node: &rustpython_ast::ExprCall) -> Result<(), ()> {
@@ -193,7 +177,7 @@ impl AssetsFinder {
if obj_name == "wmill" {
// Continue
} else if let Some((kind, ref path, ref schema)) = self.var_identifiers.get(&obj_name) {
} else if let Some((kind, ref path)) = self.var_identifiers.get(&obj_name) {
if ident == "query" {
let expr_name = node.args.get(0).or_else(|| {
node.keywords
@@ -214,20 +198,8 @@ impl AssetsFinder {
// We use the SQL parser to detect if it's a read or write query
match windmill_parser_sql::parse_assets(&sql) {
Ok(mut sql_assets) => {
if let Some(schema_name) = schema {
for asset in &mut sql_assets.assets {
if asset.kind == *kind && asset.path.starts_with(path.as_str()) {
asset.path = format!(
"{}/{}.{}",
path,
schema_name,
&asset.path[path.len() + 1..]
);
}
}
}
self.assets.extend(sql_assets.assets);
Ok(sql_assets) => {
self.assets.extend(sql_assets);
}
_ => {}
}
@@ -290,7 +262,7 @@ import wmill
def main():
wmill.load_s3_file('s3:///test.csv')
"#;
let s = parse_assets(input).map(|o| o.assets);
let s = parse_assets(input);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
@@ -308,7 +280,7 @@ import wmill
def main():
db = wmill.datatable()
"#;
let s = parse_assets(input).map(|o| o.assets);
let s = parse_assets(input);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
@@ -327,7 +299,7 @@ def main(x: int):
db = wmill.datatable('dt')
return db.query('SELECT * FROM friends WHERE age = $1', x).fetch()
"#;
let s = parse_assets(input).map(|o| o.assets);
let s = parse_assets(input);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
@@ -348,7 +320,7 @@ def main(x: int):
db.query('SELECT * FROM friends WHERE age = $1', x).fetch_one()
db.query('SELECT * FROM analytics').fetch()
"#;
let s = parse_assets(input).map(|o| o.assets);
let s = parse_assets(input);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![
@@ -380,7 +352,7 @@ def main():
def g():
db = wmill.ducklake('another2')
"#;
let s = parse_assets(input).map(|o| o.assets);
let s = parse_assets(input);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![
@@ -412,7 +384,7 @@ def main():
def g():
db = wmill.ducklake()
"#;
let s = parse_assets(input).map(|o| o.assets);
let s = parse_assets(input);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![
@@ -429,80 +401,4 @@ def g():
])
);
}
#[test]
fn test_py_asset_parser_datatable_with_schema() {
let input = r#"
import wmill
def main(x: int):
db = wmill.datatable('dt:public')
return db.query('SELECT * FROM friends WHERE age = $1', x).fetch()
"#;
let s = parse_assets(input).map(|o| o.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/public.friends".to_string(),
access_type: Some(R)
},])
);
}
#[test]
fn test_py_asset_parser_ducklake_with_schema() {
let input = r#"
import wmill
def main():
db = wmill.ducklake('lake1:analytics')
return db.query('SELECT * FROM metrics').fetch()
"#;
let s = parse_assets(input).map(|o| o.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "lake1/analytics.metrics".to_string(),
access_type: Some(R)
},])
);
}
#[test]
fn test_py_asset_parser_schema_with_write() {
let input = r#"
import wmill
def main(x: int):
db = wmill.datatable('dt:public')
db.query('INSERT INTO users VALUES ($1)', x).fetch()
return db.query('SELECT * FROM users').fetch()
"#;
let s = parse_assets(input).map(|o| o.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/public.users".to_string(),
access_type: Some(RW)
},])
);
}
#[test]
fn test_py_asset_parser_unused_datatable_with_schema() {
let input = r#"
import wmill
def main():
db = wmill.datatable('dt:public')
"#;
let s = parse_assets(input).map(|o| o.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt".to_string(),
access_type: None
},])
);
}
}

View File

@@ -15,7 +15,7 @@ use windmill_parser::{json_to_typ, Arg, MainArgSignature, ObjectType, Typ};
use rustpython_parser::{
ast::{
Constant, Expr, ExprAttribute, ExprConstant, ExprDict, ExprList, ExprName, Stmt, StmtAssign, StmtClassDef, StmtFunctionDef, Suite,
Constant, Expr, ExprConstant, ExprDict, ExprList, ExprName, Stmt, StmtFunctionDef, Suite,
},
Parse,
};
@@ -60,166 +60,6 @@ fn filter_non_main(code: &str, main_name: &str) -> String {
return filtered_code;
}
/// Data extracted from parsing the Python code
struct CodeMetadata {
enums: HashMap<String, EnumInfo>,
descriptions: HashMap<String, String>,
}
/// Information about an Enum class
struct EnumInfo {
values: Vec<String>,
members: HashMap<String, String>,
}
fn has_enum_keyword(code: &str) -> bool {
code.contains("Enum")
}
/// Extract only class and function definitions from code (prepass filtering)
fn filter_relevant_statements(code: &str) -> String {
let mut result = Vec::new();
let mut lines = code.lines().peekable();
while let Some(line) = lines.next() {
let trimmed = line.trim_start();
if trimmed.starts_with("class ") || trimmed.starts_with("def ") {
result.push(line);
let base_indent = line.len() - trimmed.len();
while let Some(&next_line) = lines.peek() {
let next_trimmed = next_line.trim_start();
let next_indent = next_line.len() - next_trimmed.len();
if next_trimmed.is_empty() || next_indent > base_indent {
result.push(lines.next().unwrap());
} else {
break;
}
}
}
}
result.join("\n")
}
/// Extract Enum definitions and docstring descriptions lazily.
/// Only parses AST if relevant keywords are present.
fn extract_code_metadata(code: &str, main_name: &str) -> CodeMetadata {
let mut enums = HashMap::new();
let mut descriptions = HashMap::new();
let has_enum = has_enum_keyword(code);
let has_docstring = code.contains("Args:");
if !has_enum && !has_docstring {
return CodeMetadata { enums, descriptions };
}
let filtered_code = filter_relevant_statements(code);
let ast = match Suite::parse(&filtered_code, "main.py") {
Ok(ast) => ast,
Err(_) => return CodeMetadata { enums, descriptions },
};
for stmt in ast {
match stmt {
Stmt::ClassDef(StmtClassDef { name, body, bases, .. }) if has_enum => {
let is_enum = bases.iter().any(|base| {
matches!(base, Expr::Name(ExprName { id, .. })
if id == "Enum" || id == "IntEnum" || id == "StrEnum"
|| id == "Flag" || id == "IntFlag")
});
if is_enum {
let mut values = Vec::new();
let mut members = HashMap::new();
for item in body {
if let Stmt::Assign(StmtAssign { targets, value, .. }) = item {
if let Some(Expr::Name(ExprName { id: target_name, .. })) = targets.first() {
if !target_name.starts_with('_') {
if let Expr::Constant(ExprConstant { value: Constant::Str(val), .. }) = value.as_ref() {
values.push(val.to_string());
members.insert(target_name.to_string(), val.to_string());
}
}
}
}
}
if !values.is_empty() {
enums.insert(name.to_string(), EnumInfo { values, members });
}
}
},
Stmt::FunctionDef(StmtFunctionDef { name: func_name, body, .. }) if has_docstring => {
if &func_name == main_name {
if let Some(Stmt::Expr(expr_stmt)) = body.first() {
if let Expr::Constant(ExprConstant { value: Constant::Str(docstring), .. }) = expr_stmt.value.as_ref() {
descriptions = parse_docstring_args(docstring);
}
}
}
},
_ => {}
}
}
CodeMetadata { enums, descriptions }
}
/// Parse docstring Args: section (format: "param_name (type): Description")
fn parse_docstring_args(docstring: &str) -> HashMap<String, String> {
let mut descriptions = HashMap::new();
let mut in_args_section = false;
let mut base_indent: Option<usize> = None;
for line in docstring.lines() {
let trimmed = line.trim();
if trimmed == "Args:" {
in_args_section = true;
base_indent = None;
continue;
}
if in_args_section {
if trimmed.is_empty() {
continue;
}
let indent = line.len() - line.trim_start().len();
if base_indent.is_none() && !trimmed.is_empty() {
base_indent = Some(indent);
}
if let Some(base) = base_indent {
if indent < base && trimmed.ends_with(':') {
break;
}
}
if let Some(colon_pos) = trimmed.find(':') {
let before_colon = &trimmed[..colon_pos];
let description = trimmed[colon_pos + 1..].trim();
if let Some(paren_pos) = before_colon.find('(') {
let param_name = before_colon[..paren_pos].trim();
descriptions.insert(param_name.to_string(), description.to_string());
} else {
descriptions.insert(before_colon.trim().to_string(), description.to_string());
}
}
}
}
descriptions
}
/// skip_params is a micro optimization for when we just want to find the main
/// function without parsing all the params.
pub fn parse_python_signature(
@@ -251,61 +91,27 @@ pub fn parse_python_signature(
if !skip_params && params.is_some() {
let params = params.unwrap();
//println!("{:?}", params);
let def_arg_start = params.args.len() - params.defaults().count();
// Two-pass approach for lazy metadata extraction:
// Pass 1: Parse types without enum info to determine if metadata is needed
// Pass 2: Re-parse unknown types with metadata only if necessary
// This ensures zero overhead for scripts without enums/docstrings
let empty_enums = HashMap::new();
let args_first_pass: Vec<_> = params
.args
.iter()
.enumerate()
.map(|(i, x)| {
let arg_name = x.as_arg().arg.to_string();
let (typ, has_default) = x
.as_arg()
.annotation
.as_ref()
.map_or((Typ::Unknown, false), |e| parse_expr(e, &empty_enums));
(i, arg_name, typ, has_default)
})
.collect();
// Determine if we need to extract metadata from the code
let has_potential_enums = args_first_pass
.iter()
.any(|(_, _, typ, _)| matches!(typ, Typ::Resource(_)));
let metadata = if has_potential_enums || code.contains("Args:") {
extract_code_metadata(code, &main_name)
} else {
CodeMetadata {
enums: HashMap::new(),
descriptions: HashMap::new(),
}
};
// Build final args, re-parsing Resource types as enums if metadata was extracted
Ok(MainArgSignature {
star_args: params.vararg.is_some(),
star_kwargs: params.kwarg.is_some(),
args: args_first_pass
.into_iter()
.map(|(i, arg_name, mut typ, mut has_default)| {
if matches!(typ, Typ::Resource(_)) && !metadata.enums.is_empty() {
if let Some(annotation) = params.args[i].as_arg().annotation.as_ref() {
(typ, has_default) = parse_expr(annotation, &metadata.enums);
}
}
args: params
.args
.iter()
.enumerate()
.map(|(i, x)| {
let (mut typ, has_default) = x
.as_arg()
.annotation
.as_ref()
.map_or((Typ::Unknown, false), |e| parse_expr(e));
let default = if i >= def_arg_start {
params
.defaults()
.nth(i - def_arg_start)
.map(|expr| to_value(expr, &metadata.enums))
.map(to_value)
.flatten()
} else {
None
@@ -334,8 +140,8 @@ pub fn parse_python_signature(
}
Arg {
otyp: metadata.descriptions.get(&arg_name).map(|d| d.to_string()),
name: arg_name,
otyp: None,
name: x.as_arg().arg.to_string(),
typ,
has_default: has_default || default.is_some(),
default,
@@ -357,15 +163,15 @@ pub fn parse_python_signature(
}
}
fn parse_expr(e: &Box<Expr>, enums: &HashMap<String, EnumInfo>) -> (Typ, bool) {
fn parse_expr(e: &Box<Expr>) -> (Typ, bool) {
match e.as_ref() {
Expr::Name(ExprName { id, .. }) => (parse_typ(id.as_ref(), enums), false),
Expr::Name(ExprName { id, .. }) => (parse_typ(id.as_ref()), false),
Expr::Attribute(x) => {
if x.value
.as_name_expr()
.is_some_and(|x| x.id.as_str() == "wmill")
{
(parse_typ(x.attr.as_str(), enums), false)
(parse_typ(x.attr.as_str()), false)
} else {
(Typ::Unknown, false)
}
@@ -375,7 +181,7 @@ fn parse_expr(e: &Box<Expr>, enums: &HashMap<String, EnumInfo>) -> (Typ, bool) {
x.right.as_ref(),
Expr::Constant(ExprConstant { value: Constant::None, .. })
) {
(parse_expr(&x.left, enums).0, true)
(parse_expr(&x.left).0, true)
} else {
(Typ::Unknown, false)
}
@@ -404,8 +210,8 @@ fn parse_expr(e: &Box<Expr>, enums: &HashMap<String, EnumInfo>) -> (Typ, bool) {
};
(Typ::Str(values), false)
}
"List" | "list" => (Typ::List(Box::new(parse_expr(&x.slice, enums).0)), false),
"Optional" => (parse_expr(&x.slice, enums).0, true),
"List" | "list" => (Typ::List(Box::new(parse_expr(&x.slice).0)), false),
"Optional" => (parse_expr(&x.slice).0, true),
_ => (Typ::Unknown, false),
},
_ => (Typ::Unknown, false),
@@ -414,11 +220,7 @@ fn parse_expr(e: &Box<Expr>, enums: &HashMap<String, EnumInfo>) -> (Typ, bool) {
}
}
fn parse_typ(id: &str, enums: &HashMap<String, EnumInfo>) -> Typ {
if let Some(enum_info) = enums.get(id) {
return Typ::Str(Some(enum_info.values.clone()));
}
fn parse_typ(id: &str) -> Typ {
match id {
"str" => Typ::Str(None),
"float" => Typ::Float,
@@ -447,7 +249,7 @@ fn map_resource_name(x: &str) -> String {
}
}
fn to_value<R>(et: &Expr<R>, enums: &HashMap<String, EnumInfo>) -> Option<serde_json::Value> {
fn to_value<R>(et: &Expr<R>) -> Option<serde_json::Value> {
match et {
Expr::Constant(ExprConstant { value, .. }) => Some(constant_to_value(value)),
Expr::Dict(ExprDict { keys, values, .. }) => {
@@ -457,35 +259,22 @@ fn to_value<R>(et: &Expr<R>, enums: &HashMap<String, EnumInfo>) -> Option<serde_
.map(|(k, v)| {
let key = k
.as_ref()
.map(|e| to_value(e, enums))
.map(to_value)
.flatten()
.and_then(|x| match x {
serde_json::Value::String(s) => Some(s),
_ => None,
})
.unwrap_or_else(|| "no_key".to_string());
(key, to_value(&v, enums))
(key, to_value(&v))
})
.collect::<HashMap<String, _>>();
Some(json!(v))
}
Expr::List(ExprList { elts, .. }) => {
let v = elts.into_iter().map(|x| to_value(&x, enums)).collect::<Vec<_>>();
let v = elts.into_iter().map(|x| to_value(&x)).collect::<Vec<_>>();
Some(json!(v))
}
Expr::Attribute(ExprAttribute { value, attr, .. }) => {
// Handle Enum.MEMBER: returns enum value ("red") not member name ("RED")
if let Expr::Name(ExprName { id: enum_name, .. }) = value.as_ref() {
if let Some(enum_info) = enums.get(enum_name.as_str()) {
if let Some(enum_value) = enum_info.members.get(attr.as_str()) {
return Some(json!(enum_value));
}
}
Some(json!(attr.as_str()))
} else {
None
}
}
Expr::Call { .. } => Some(json!(FUNCTION_CALL)),
_ => None,
}
@@ -962,35 +751,4 @@ def main(a: str, b: Optional[str], c: str | None): return
Ok(())
}
#[test]
fn test_parse_python_sig_enum() -> anyhow::Result<()> {
let code = r#"
from enum import Enum
class Color(str, Enum):
RED = 'red'
GREEN = 'green'
BLUE = 'blue'
def main(color: Color = Color.RED):
"""
Test enum parsing
Args:
color (Color): Color selection from Color enum
"""
return {"color": color}
"#;
let result = parse_python_signature(code, None, false)?;
assert_eq!(result.args.len(), 1);
assert_eq!(result.args[0].name, "color");
assert_eq!(
result.args[0].typ,
Typ::Str(Some(vec!["red".to_string(), "green".to_string(), "blue".to_string()]))
);
assert_eq!(result.args[0].default, Some(json!("red")));
assert_eq!(result.args[0].otyp, Some("Color selection from Color enum".to_string()));
Ok(())
}
}

View File

@@ -10,11 +10,11 @@ use sqlparser::{
};
use windmill_parser::asset_parser::{
asset_was_used, merge_assets, parse_asset_syntax, AssetKind, AssetUsageAccessType,
ParseAssetsOutput, ParseAssetsResult,
ParseAssetsResult,
};
use AssetUsageAccessType::*;
pub fn parse_assets(input: &str) -> anyhow::Result<ParseAssetsOutput> {
pub fn parse_assets(input: &str) -> anyhow::Result<Vec<ParseAssetsResult>> {
let statements = Parser::parse_sql(&DuckDbDialect, input)?;
let mut collector = AssetCollector::new();
@@ -30,7 +30,7 @@ pub fn parse_assets(input: &str) -> anyhow::Result<ParseAssetsOutput> {
}
}
Ok(ParseAssetsOutput { assets: merge_assets(collector.assets), ..Default::default() })
Ok(merge_assets(collector.assets))
}
/// Visitor that collects S3 asset literals from SQL statements
@@ -58,28 +58,19 @@ impl AssetCollector {
// Or when we access 'b' and we did USE a;
fn get_associated_asset_from_obj_name(&self, name: &ObjectName) -> Option<ParseAssetsResult> {
let access_type = self.current_access_type_stack.last().copied();
if let Some((kind, path)) = &self.currently_used_asset {
if name.0.len() == 1 {
let ident = name.0.first()?.as_ident()?;
if ident.quote_style.is_some() {
return None;
}
let specific_table = &ident.value;
// We don't want to infer that any simple identifier refers to an asset if
// we are not in a known R/W context
if access_type.is_none() {
return None;
}
if name.0.len() == 1 || name.0.len() == 2 {
if name
.0
.iter()
.any(|id| id.as_ident().and_then(|id| id.quote_style).is_some())
{
return None;
}
let specific_table = &name
.0
.iter()
.map(|id| id.as_ident().map(|id| id.value.clone()))
.collect::<Option<Vec<String>>>()?
.join(".");
if let Some((kind, path)) = &self.currently_used_asset {
let path = format!("{}/{}", path, specific_table);
return Some(ParseAssetsResult { kind: *kind, access_type, path });
}
@@ -91,12 +82,8 @@ impl AssetCollector {
}
let ident = name.0.first()?.as_ident()?;
let (kind, path) = self.var_identifiers.get(&ident.value)?;
let path = if name.0.len() == 2 || name.0.len() == 3 {
let specific_table = &name.0[1..]
.iter()
.map(|id| id.as_ident().map(|id| id.value.clone()))
.collect::<Option<Vec<String>>>()?
.join(".");
let path = if name.0.len() == 2 {
let specific_table = &name.0.get(1)?.as_ident()?.value;
format!("{}/{}", path, specific_table)
} else {
path.clone()
@@ -402,7 +389,7 @@ mod tests {
SELECT * FROM read_parquet('s3:///a.parquet');
COPY (SELECT * FROM 's3://snd/b.parquet') TO 's3:///c.parquet';
"#;
let s = parse_assets(input).map(|s| s.assets);
let s = parse_assets(input);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![
@@ -432,7 +419,7 @@ mod tests {
SELECT 2;
USE dl;
"#;
let s = parse_assets(input).map(|s| s.assets);
let s = parse_assets(input);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
@@ -449,7 +436,7 @@ mod tests {
ATTACH 'ducklake://my_dl' AS dl;
SELECT * FROM dl.table1;
"#;
let s = parse_assets(input).map(|s| s.assets);
let s = parse_assets(input);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
@@ -467,7 +454,7 @@ mod tests {
SELECT dt.read_bait FROM unrelated_table; -- dt. doesn't access the asset
INSERT INTO dt.table1 VALUES ('test');
"#;
let s = parse_assets(input).map(|s| s.assets);
let s = parse_assets(input);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
@@ -485,7 +472,7 @@ mod tests {
DETACH dl;
SELECT * FROM dl.table1;
"#;
let s = parse_assets(input).map(|s| s.assets);
let s = parse_assets(input);
assert_eq!(s.map_err(|e| e.to_string()), Ok(vec![]));
}
@@ -498,7 +485,7 @@ mod tests {
USE memory;
SELECT * FROM table1;
"#;
let s = parse_assets(input).map(|s| s.assets);
let s = parse_assets(input);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
@@ -515,7 +502,7 @@ mod tests {
ATTACH 'datatable' AS dl;
INSERT INTO dl.table1 VALUES ('test');
"#;
let s = parse_assets(input).map(|s| s.assets);
let s = parse_assets(input);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
@@ -537,7 +524,7 @@ mod tests {
INSERT INTO friends VALUES ($name, $age);
SELECT * FROM friends;
"#;
let s = parse_assets(input).map(|s| s.assets);
let s = parse_assets(input);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
@@ -555,7 +542,7 @@ mod tests {
ATTACH 'ducklake' AS dl; USE dl;
SELECT * FROM a_function('');
"#;
let s = parse_assets(input).map(|s| s.assets);
let s = parse_assets(input);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
@@ -573,7 +560,7 @@ mod tests {
USE dl;
DELETE FROM table1;
"#;
let s = parse_assets(input).map(|s| s.assets);
let s = parse_assets(input);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
@@ -591,7 +578,7 @@ mod tests {
USE dl;
UPDATE table1 SET id = NULL;
"#;
let s = parse_assets(input).map(|s| s.assets);
let s = parse_assets(input);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
@@ -609,7 +596,7 @@ mod tests {
USE db;
SELECT * FROM table1;
"#;
let s = parse_assets(input).map(|s| s.assets);
let s = parse_assets(input);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
@@ -626,7 +613,7 @@ mod tests {
ATTACH 'ducklake' AS dl;
UPDATE dl.table1 SET id = NULL;
"#;
let s = parse_assets(input).map(|s| s.assets);
let s = parse_assets(input);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
@@ -636,41 +623,4 @@ mod tests {
},])
);
}
#[test]
fn test_sql_asset_parser_table_with_schema() {
let input = r#"
ATTACH 'ducklake' AS dl;
UPDATE dl.sch.table1 SET id = NULL;
SELECT * FROM dl.sch.table1;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "main/sch.table1".to_string(),
access_type: Some(RW)
},])
);
}
#[test]
fn test_sql_asset_parser_table_with_schema_implicit() {
let input = r#"
ATTACH 'ducklake' AS dl;
USE dl;
UPDATE sch.table1 SET id = NULL;
SELECT * FROM sch.table1;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "main/sch.table1".to_string(),
access_type: Some(RW)
},])
);
}
}

View File

@@ -1,16 +1,16 @@
use std::collections::HashMap;
use swc_common::{sync::Lrc, FileName, SourceMap, Spanned};
use swc_common::{sync::Lrc, FileName, SourceMap};
use swc_ecma_ast::{CallExpr, Expr, Lit, MemberExpr, MemberProp, Str};
use swc_ecma_parser::{lexer::Lexer, Parser, StringInput, Syntax, TsSyntax};
use swc_ecma_visit::{Visit, VisitWith};
use windmill_parser::asset_parser::{
asset_was_used, merge_assets, parse_asset_syntax, AssetKind, AssetUsageAccessType,
ParseAssetsOutput, ParseAssetsResult, SqlQueryDetails,
ParseAssetsResult,
};
use AssetUsageAccessType::*;
pub fn parse_assets(code: &str) -> anyhow::Result<ParseAssetsOutput> {
pub fn parse_assets(code: &str) -> anyhow::Result<Vec<ParseAssetsResult>> {
let cm: Lrc<SourceMap> = Default::default();
let fm = cm.new_source_file(FileName::Custom("main.ts".into()).into(), code.into());
let lexer = Lexer::new(
@@ -35,17 +35,11 @@ pub fn parse_assets(code: &str) -> anyhow::Result<ParseAssetsOutput> {
anyhow::anyhow!("Error while parsing code, it is invalid TypeScript: {err_s}, {e:?}")
})?
.body;
let mut assets_finder =
AssetsFinder { assets: vec![], sql_queries: vec![], var_identifiers: HashMap::new() };
let mut assets_finder = AssetsFinder { assets: vec![], var_identifiers: HashMap::new() };
assets_finder.visit_module_items(&ast);
Ok(ParseAssetsOutput {
assets: merge_assets(assets_finder.assets),
sql_queries: assets_finder.sql_queries,
})
Ok(merge_assets(assets_finder.assets))
}
type VarAssetName = String;
type VarAssetSchema = Option<String>;
struct AssetsFinder {
assets: Vec<ParseAssetsResult>,
@@ -55,57 +49,9 @@ struct AssetsFinder {
// The goal is to remember that the identifier "sql" corresponds to the datatable "main"
// so that when we see a tagged template expression with tag "sql" we know which datatable it
// corresponds to. This allows us to infer if a datatable is Read or Write based on the SQL query.
var_identifiers: HashMap<String, (AssetKind, VarAssetName, VarAssetSchema)>,
sql_queries: Vec<SqlQueryDetails>,
var_identifiers: HashMap<String, (AssetKind, String)>,
}
/// Helper function to extract wmill.datatable() or wmill.ducklake() calls,
/// Returns (AssetKind, asset_name, optional_schema_name)
fn extract_wmill_datatable_call(expr: &Expr) -> Option<(AssetKind, String, Option<String>)> {
if let Expr::Call(call_expr) = expr {
if let Some(Expr::Member(member)) = call_expr.callee.as_expr().map(AsRef::as_ref) {
// Check if object is "wmill"
let is_wmill = matches!(
member.obj.as_ref(),
Expr::Ident(ident) if ident.sym.as_str() == "wmill"
);
if is_wmill {
if let MemberProp::Ident(prop) = &member.prop {
// Get the asset name from first arg, default to "main"
let asset_name = call_expr
.args
.first()
.and_then(|arg| match arg.expr.as_ref() {
Expr::Lit(Lit::Str(s)) => Some(s.value.to_string()),
_ => None,
})
.unwrap_or_else(|| "main".to_string());
let (asset_name, schema_name) = asset_name.split_once(':').map_or_else(
|| (asset_name.clone(), None),
|(name, schema)| {
(
(if name.is_empty() { "main" } else { name }).to_string(),
Some(schema.to_string()),
)
},
);
let kind = match prop.sym.as_str() {
"datatable" => Some(AssetKind::DataTable),
"ducklake" => Some(AssetKind::Ducklake),
_ => None,
};
return kind.map(|k| (k, asset_name, schema_name));
}
}
}
}
None
}
impl Visit for AssetsFinder {
// visit_call_expr will not recurse if it detects an asset,
// so this will only be called when no further context was found
@@ -131,37 +77,6 @@ impl Visit for AssetsFinder {
}
}
fn visit_assign_expr(&mut self, node: &swc_ecma_ast::AssignExpr) {
// Handle reassignments like: sql = wmill.datatable('main')
// Extract the variable name from the left side
let var_name = match &node.left {
swc_ecma_ast::AssignTarget::Simple(simple_target) => match simple_target {
swc_ecma_ast::SimpleAssignTarget::Ident(ident_binding) => {
ident_binding.id.sym.as_str().to_string()
}
_ => {
node.visit_children_with(self);
return;
}
},
_ => {
node.visit_children_with(self);
return;
}
};
// Check if right side is a wmill.datatable() or wmill.ducklake() call
if let Some((kind, asset_name, schema)) = extract_wmill_datatable_call(node.right.as_ref())
{
self.var_identifiers
.insert(var_name, (kind, asset_name, schema));
return;
}
// Default: visit children
node.visit_children_with(self);
}
fn visit_block_stmt(&mut self, node: &swc_ecma_ast::BlockStmt) {
// Save current state before entering the block
let saved_var_identifiers = self.var_identifiers.clone();
@@ -173,7 +88,7 @@ impl Visit for AssetsFinder {
if saved_var_identifiers.contains_key(var) {
continue;
}
let (kind, ref path, _) = self.var_identifiers[var];
let (kind, ref path) = self.var_identifiers[var];
if asset_was_used(&self.assets, (kind, path)) {
continue;
}
@@ -196,12 +111,43 @@ impl Visit for AssetsFinder {
};
// Check if init is a call to wmill.datatable(...) or wmill.ducklake(...)
// optionally with .schema() chained
if let Some(init) = &node.init {
if let Some((kind, asset_name, schema)) = extract_wmill_datatable_call(init.as_ref()) {
self.var_identifiers
.insert(var_name, (kind, asset_name, schema));
return;
if let Expr::Call(call_expr) = init.as_ref() {
if let Some(Expr::Member(member)) = call_expr.callee.as_expr().map(AsRef::as_ref) {
// Check if object is "wmill"
let is_wmill = matches!(
member.obj.as_ref(),
Expr::Ident(ident) if ident.sym.as_str() == "wmill"
);
if is_wmill {
if let MemberProp::Ident(prop) = &member.prop {
// Get the asset name from first arg, default to "main"
let asset_name = call_expr
.args
.first()
.and_then(|arg| match arg.expr.as_ref() {
Expr::Lit(Lit::Str(s)) => Some(s.value.to_string()),
_ => None,
})
.unwrap_or_else(|| "main".to_string());
match prop.sym.as_str() {
"datatable" => {
self.var_identifiers
.insert(var_name, (AssetKind::DataTable, asset_name));
return;
}
"ducklake" => {
self.var_identifiers
.insert(var_name, (AssetKind::Ducklake, asset_name));
return;
}
_ => {}
}
}
}
}
}
}
@@ -220,64 +166,33 @@ impl Visit for AssetsFinder {
};
// Check if it's a known identifier
let Some((kind, asset_name, schema)) = self.var_identifiers.get(tag_name) else {
let (kind, asset_name) = if let Some((kind, name)) = self.var_identifiers.get(tag_name) {
(*kind, name.clone())
} else {
node.visit_children_with(self);
return;
};
// Extract the SQL query from the template quasis (string parts)
// Substitute ${} with $1, $2, etc.
let sql: String = node
.tpl
.quasis
.iter()
.map(|quasi| quasi.raw.as_str())
.enumerate()
.fold(String::new(), |acc, (i, s)| {
if i == 0 {
s.to_string()
} else {
format!("{}${}{}", acc, i, s)
}
});
.collect::<Vec<_>>()
.join("$1"); // placeholder for expressions
let duckdb_conn_prefix = match kind {
AssetKind::DataTable => "datatable",
AssetKind::Ducklake => "ducklake",
_ => return,
};
// Capture SQL query details before transforming for SQL parser
let span = node.span();
let span_tuple = (span.lo.0, span.hi.0);
self.sql_queries.push(SqlQueryDetails {
query_string: sql.clone(),
span: span_tuple,
source_kind: *kind,
source_name: asset_name.clone(),
source_schema: schema.clone(),
});
let sql_with_attach =
format!("ATTACH '{duckdb_conn_prefix}://{asset_name}' AS dt; USE dt; {sql}");
let sql = format!("ATTACH '{duckdb_conn_prefix}://{asset_name}' AS dt; USE dt; {sql}");
// We use the SQL parser to detect if it's a read or write query
match windmill_parser_sql::parse_assets(&sql_with_attach) {
Ok(mut sql_assets) => {
if let Some(schema) = schema {
for asset in &mut sql_assets.assets {
if asset.kind == *kind && asset.path.starts_with(asset_name) {
asset.path = format!(
"{}/{}.{}",
asset_name,
schema,
&asset.path[asset_name.len() + 1..]
);
}
}
}
self.assets.extend(sql_assets.assets);
match windmill_parser_sql::parse_assets(&sql) {
Ok(sql_assets) => {
self.assets.extend(sql_assets);
}
_ => {}
}
@@ -334,7 +249,7 @@ mod tests {
"#;
let s = parse_assets(input);
assert_eq!(
s.map(|r| r.assets).map_err(|e| e.to_string()),
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::S3Object,
path: "/test.csv".to_string(),
@@ -353,7 +268,7 @@ mod tests {
"#;
let s = parse_assets(input);
assert_eq!(
s.map(|r| r.assets).map_err(|e| e.to_string()),
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt".to_string(),
@@ -373,7 +288,7 @@ mod tests {
"#;
let s = parse_assets(input);
assert_eq!(
s.map(|r| r.assets).map_err(|e| e.to_string()),
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/friends".to_string(),
@@ -395,7 +310,7 @@ mod tests {
"#;
let s = parse_assets(input);
assert_eq!(
s.map(|r| r.assets).map_err(|e| e.to_string()),
s.map_err(|e| e.to_string()),
Ok(vec![
ParseAssetsResult {
kind: AssetKind::DataTable,
@@ -430,7 +345,7 @@ mod tests {
"#;
let s = parse_assets(input);
assert_eq!(
s.map(|r| r.assets).map_err(|e| e.to_string()),
s.map_err(|e| e.to_string()),
Ok(vec![
ParseAssetsResult {
kind: AssetKind::DataTable,
@@ -464,7 +379,7 @@ mod tests {
"#;
let s = parse_assets(input);
assert_eq!(
s.map(|r| r.assets).map_err(|e| e.to_string()),
s.map_err(|e| e.to_string()),
Ok(vec![
ParseAssetsResult {
kind: AssetKind::DataTable,
@@ -479,205 +394,4 @@ mod tests {
])
);
}
#[test]
fn test_ts_asset_parser_datatable_with_schema() {
let input = r#"
import * as wmill from "windmill-client"
export async function main(x: number) {
let sql = wmill.datatable(':myschema')
return await sql`SELECT * FROM friends WHERE age = ${x}`.fetch()
}
"#;
let s = parse_assets(input);
assert_eq!(
s.map(|r| r.assets).map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "main/myschema.friends".to_string(),
access_type: Some(R)
},])
);
}
#[test]
fn test_ts_asset_parser_schema_with_write() {
let input = r#"
import * as wmill from "windmill-client"
export async function main(x: number) {
let sql = wmill.datatable('dt:public')
await sql`INSERT INTO users VALUES (${x})`.fetch()
return await sql`SELECT * FROM users`.fetch()
}
"#;
let s = parse_assets(input);
assert_eq!(
s.map(|r| r.assets).map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/public.users".to_string(),
access_type: Some(RW)
},])
);
}
#[test]
fn test_ts_asset_parser_unused_datatable_with_schema() {
let input = r#"
import * as wmill from "windmill-client"
export async function main() {
let sql = wmill.datatable('dt:myschema')
}
"#;
let s = parse_assets(input);
assert_eq!(
s.map(|r| r.assets).map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt".to_string(),
access_type: None
},])
);
}
#[test]
fn test_ts_asset_parser_reassignment() {
let input = r#"
import * as wmill from "windmill-client"
export async function main(x: number) {
let sql;
sql = wmill.datatable('dt')
return await sql`SELECT * FROM users WHERE id = ${x}`.fetch()
}
"#;
let s = parse_assets(input);
assert_eq!(
s.map(|r| r.assets).map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/users".to_string(),
access_type: Some(R)
},])
);
}
#[test]
fn test_ts_asset_parser_reassignment_with_schema() {
let input = r#"
import * as wmill from "windmill-client"
export async function main(x: number) {
let sql = wmill.datatable('dt')
await sql`INSERT INTO test VALUES ('')`.fetch()
sql = wmill.datatable('dt:private')
return await sql`SELECT * FROM users WHERE id = ${x}`.fetch()
}
"#;
let s = parse_assets(input);
assert_eq!(
s.map(|r| r.assets).map_err(|e| e.to_string()),
Ok(vec![
ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/private.users".to_string(),
access_type: Some(R)
},
ParseAssetsResult {
kind: AssetKind::DataTable,
path: "dt/test".to_string(),
access_type: Some(W)
},
])
);
}
#[test]
fn test_ts_asset_parser_sql_query_details() {
let input = r#"
import * as wmill from "windmill-client"
export async function main(x: number) {
let sql = wmill.datatable('dt')
return await sql`SELECT * FROM friends WHERE age = ${x}`.fetch()
}
"#;
let result = parse_assets(input).unwrap();
// Check assets
assert_eq!(result.assets.len(), 1);
assert_eq!(result.assets[0].kind, AssetKind::DataTable);
assert_eq!(result.assets[0].path, "dt/friends");
// Check SQL query details
assert_eq!(result.sql_queries.len(), 1);
let query_detail = &result.sql_queries[0];
assert_eq!(
query_detail.query_string,
"SELECT * FROM friends WHERE age = $1"
);
assert_eq!(query_detail.source_kind, AssetKind::DataTable);
assert_eq!(query_detail.source_name, "dt");
assert_eq!(query_detail.source_schema, None);
// Span should be non-zero
assert!(query_detail.span.0 > 0);
assert!(query_detail.span.1 > query_detail.span.0);
}
#[test]
fn test_ts_asset_parser_sql_query_details_with_schema() {
let input = r#"
import * as wmill from "windmill-client"
export async function main(x: number) {
let sql = wmill.datatable('dt:public')
await sql`INSERT INTO users VALUES (${x})`.fetch()
return await sql`SELECT * FROM users`.fetch()
}
"#;
let result = parse_assets(input).unwrap();
// Check SQL query details
assert_eq!(result.sql_queries.len(), 2);
// First query (INSERT)
assert_eq!(
result.sql_queries[0].query_string,
"INSERT INTO users VALUES ($1)"
);
assert_eq!(result.sql_queries[0].source_kind, AssetKind::DataTable);
assert_eq!(result.sql_queries[0].source_name, "dt");
assert_eq!(
result.sql_queries[0].source_schema,
Some("public".to_string())
);
// Second query (SELECT)
assert_eq!(result.sql_queries[1].query_string, "SELECT * FROM users");
assert_eq!(result.sql_queries[1].source_kind, AssetKind::DataTable);
assert_eq!(result.sql_queries[1].source_name, "dt");
assert_eq!(
result.sql_queries[1].source_schema,
Some("public".to_string())
);
}
#[test]
fn test_ts_asset_parser_sql_query_details_ducklake() {
let input = r#"
import * as wmill from "windmill-client"
export async function main() {
let sql = wmill.ducklake('my_lake')
return await sql`SELECT id, name FROM products LIMIT 10`.fetch()
}
"#;
let result = parse_assets(input).unwrap();
// Check SQL query details
assert_eq!(result.sql_queries.len(), 1);
let query_detail = &result.sql_queries[0];
assert_eq!(
query_detail.query_string,
"SELECT id, name FROM products LIMIT 10"
);
assert_eq!(query_detail.source_kind, AssetKind::Ducklake);
assert_eq!(query_detail.source_name, "my_lake");
assert_eq!(query_detail.source_schema, None);
}
}

View File

@@ -1,10 +1,10 @@
use windmill_parser::asset_parser::{
merge_assets, AssetKind, AssetUsageAccessType, ParseAssetsOutput, ParseAssetsResult,
merge_assets, AssetKind, AssetUsageAccessType, ParseAssetsResult,
};
use crate::{parse_ansible_reqs, ResourceOrVariablePath};
pub fn parse_assets(input: &str) -> anyhow::Result<ParseAssetsOutput> {
pub fn parse_assets(input: &str) -> anyhow::Result<Vec<ParseAssetsResult>> {
let mut assets = vec![];
if let (_, Some(ansible_reqs), _) = parse_ansible_reqs(input)? {
if let Some(delegate_to_git_repo_details) = ansible_reqs.delegate_to_git_repo {
@@ -36,5 +36,5 @@ pub fn parse_assets(input: &str) -> anyhow::Result<ParseAssetsOutput> {
}
}
Ok(ParseAssetsOutput { assets: merge_assets(assets), ..Default::default() })
Ok(merge_assets(assets))
}

View File

@@ -958,7 +958,7 @@ dependencies:
content: "{{ my_result | to_json }}"
dest: result.json
"#;
let a = parse_assets(p).unwrap().assets;
let a = parse_assets(p).unwrap();
println!("The resulting assets are: {}", a.len());
let a = parse_ansible_reqs(p).unwrap();

View File

@@ -27,22 +27,6 @@ pub struct ParseAssetsResult {
pub access_type: Option<AssetUsageAccessType>, // None in case of ambiguity
}
#[derive(Serialize, Debug, PartialEq)]
pub struct SqlQueryDetails {
pub query_string: String, // SQL query with $1 placeholders for interpolations
pub span: (u32, u32), // (start, end) byte positions in source code
pub source_kind: AssetKind, // DataTable or Ducklake
pub source_name: String, // e.g., "main", "dt"
#[serde(skip_serializing_if = "Option::is_none")]
pub source_schema: Option<String>, // e.g., Some("public"), None
}
#[derive(Serialize, Debug, Default)]
pub struct ParseAssetsOutput {
pub assets: Vec<ParseAssetsResult>,
pub sql_queries: Vec<SqlQueryDetails>,
}
#[derive(Debug, Clone, Serialize)]
pub struct DelegateToGitRepoDetails {
pub resource: String,

View File

@@ -1,65 +0,0 @@
use std::fs;
use std::path::PathBuf;
#[derive(Debug)]
pub enum CgroupError {
#[allow(unused)]
PathNotFound(PathBuf),
NotSupported,
PermissionDenied,
#[allow(unused)]
Io(std::io::Error),
}
impl From<std::io::Error> for CgroupError {
fn from(e: std::io::Error) -> Self {
CgroupError::Io(e)
}
}
pub fn get_cgroup_path() -> Result<PathBuf, CgroupError> {
let cgroup_info = fs::read_to_string("/proc/1/cgroup")?;
// Format: "0::/kubepods.slice/..." - we want the part after the second colon
let cgroup_rel = cgroup_info
.lines()
.next()
.and_then(|line| line.splitn(3, ':').nth(2))
.unwrap_or("")
.trim();
let cgroup_path = PathBuf::from(format!("/sys/fs/cgroup{}", cgroup_rel));
if !cgroup_path.is_dir() {
return Err(CgroupError::PathNotFound(cgroup_path));
}
Ok(cgroup_path)
}
pub fn disable_oom_group() -> Result<(), CgroupError> {
let cgroup_path = get_cgroup_path()?;
let oom_group_file = cgroup_path.join("memory.oom.group");
if !oom_group_file.exists() {
return Err(CgroupError::NotSupported);
}
let current = fs::read_to_string(&oom_group_file)?;
if current.trim() == "0" {
tracing::info!("memory.oom.group already disabled");
return Ok(());
}
match fs::write(&oom_group_file, "0") {
Ok(_) => {
tracing::info!("Disabled memory.oom.group at {:?}", cgroup_path);
Ok(())
}
Err(e) if e.kind() == std::io::ErrorKind::PermissionDenied => {
tracing::error!("Failed to disable memory.oom.group (need privileged mode)");
Err(CgroupError::PermissionDenied)
}
Err(e) => Err(CgroupError::Io(e)),
}
}

View File

@@ -71,9 +71,6 @@ use windmill_common::worker::CLOUD_HOSTED;
#[cfg(all(not(target_env = "msvc"), feature = "jemalloc"))]
use monitor::monitor_mem;
#[cfg(any(target_os = "linux"))]
use crate::cgroups::disable_oom_group;
#[cfg(all(not(target_env = "msvc"), feature = "jemalloc"))]
use tikv_jemallocator::Jemalloc;
@@ -111,7 +108,6 @@ const DEFAULT_NUM_WORKERS: usize = 1;
const DEFAULT_PORT: u16 = 8000;
const DEFAULT_SERVER_BIND_ADDR: Ipv4Addr = Ipv4Addr::new(0, 0, 0, 0);
mod cgroups;
#[cfg(feature = "private")]
pub mod ee;
mod ee_oss;
@@ -511,13 +507,6 @@ async fn windmill_main() -> anyhow::Result<()> {
let worker_mode = num_workers > 0;
if worker_mode {
#[cfg(any(target_os = "linux"))]
if let Err(e) = disable_oom_group() {
tracing::warn!("failed to disable oom group: {:?}", e);
}
}
let conn = if mode == Mode::Agent {
conn
} else {

View File

@@ -1606,17 +1606,6 @@ pub async fn monitor_db(
}
};
// run every 30s (every iteration)
let cleanup_debounce_keys_completed_f = async {
if server_mode && !initial_load {
if let Some(db) = conn.as_sql() {
if let Err(e) = cleanup_debounce_keys_for_completed_jobs(&db).await {
tracing::error!("Error cleaning up debounce keys for completed jobs: {:?}", e);
}
}
}
};
// run every hour (60 minutes / 30 seconds = 120)
let cleanup_worker_group_stats_f = async {
if server_mode && iteration.is_some() && iteration.as_ref().unwrap().should_run(120) {
@@ -1737,7 +1726,6 @@ pub async fn monitor_db(
cleanup_concurrency_counters_f,
cleanup_concurrency_counters_empty_keys_f,
cleanup_debounce_keys_f,
cleanup_debounce_keys_completed_f,
cleanup_worker_group_stats_f,
);
}
@@ -2803,33 +2791,3 @@ RETURNING key,job_id
}
Ok(())
}
async fn cleanup_debounce_keys_for_completed_jobs(db: &DB) -> error::Result<()> {
// If min version doesn't support runnable settings, clean up debounce keys for completed jobs
if !*windmill_common::worker::MIN_VERSION_SUPPORTS_RUNNABLE_SETTINGS_V0.read().await {
let result = sqlx::query!(
"
DELETE FROM debounce_key
WHERE job_id IN (SELECT id FROM v2_job_completed)
RETURNING key,job_id
",
)
.fetch_all(db)
.await?;
if result.len() > 0 {
tracing::warn!(
"Cleaned up {} debounce keys for completed jobs (runnable settings v0 not supported by all workers)",
result.len()
);
for row in result {
tracing::debug!(
"Debounce key for completed job cleaned up: key: {}, job_id: {:?}",
row.key,
row.job_id
);
}
}
}
Ok(())
}

View File

@@ -143,12 +143,13 @@ impl ApiServer {
pub struct RunJob {
pub payload: JobPayload,
pub args: serde_json::Map<String, serde_json::Value>,
pub debounce_job_id_o: Option<Uuid>,
pub scheduled_for_o: Option<chrono::DateTime<chrono::Utc>>,
}
impl From<JobPayload> for RunJob {
fn from(payload: JobPayload) -> Self {
Self { payload, args: Default::default(), scheduled_for_o: None }
Self { payload, args: Default::default(), debounce_job_id_o: None, scheduled_for_o: None }
}
}
@@ -158,6 +159,11 @@ impl RunJob {
self
}
pub fn push_arg_debounce_job_id_o(mut self, job_id: Option<Uuid>) -> Self {
self.debounce_job_id_o = job_id;
self
}
pub fn push_arg_scheduled_for_o(
mut self,
scheduled_for_o: Option<chrono::DateTime<chrono::Utc>>,
@@ -167,7 +173,7 @@ impl RunJob {
}
pub async fn push(self, db: &Pool<Postgres>) -> Uuid {
let RunJob { payload, args, scheduled_for_o } = self;
let RunJob { payload, args, debounce_job_id_o, scheduled_for_o } = self;
let mut hm_args = std::collections::HashMap::new();
for (k, v) in args {
hm_args.insert(k, windmill_common::worker::to_raw_value(&v));
@@ -201,6 +207,7 @@ impl RunJob {
None,
false,
None,
debounce_job_id_o,
None,
None,
)
@@ -584,7 +591,6 @@ pub async fn assert_lockfile(
hash: ScriptHash(script.hash),
dedicated_worker: None,
language,
debouncing_settings: Default::default(),
})
.push(&db2)
.await;
@@ -685,8 +691,8 @@ pub async fn run_deployed_relative_imports(
language,
priority: None,
apply_preprocessor: false,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
})
.push(&db2)
.await;
@@ -735,8 +741,8 @@ pub async fn run_preview_relative_imports(
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
}))
.push(&db2)
.await;

View File

@@ -52,10 +52,8 @@ mod job_payload {
let result = RunJob::from(JobPayload::ScriptHash {
hash: ScriptHash(123412),
path: "f/system/hello".to_string(),
concurrency_settings:
windmill_common::runnable_settings::ConcurrencySettings::default().into(),
debouncing_settings:
windmill_common::runnable_settings::DebouncingSettings::default(),
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
@@ -92,10 +90,8 @@ mod job_payload {
language: ScriptLang::Deno,
priority: None,
apply_preprocessor: true,
concurrency_settings:
windmill_common::runnable_settings::ConcurrencySettings::default(),
debouncing_settings:
windmill_common::runnable_settings::DebouncingSettings::default(),
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
})
.run_until_complete_with(db, false, port, |id| async move {
let job = sqlx::query!("SELECT preprocessed FROM v2_job WHERE id = $1", id)
@@ -131,7 +127,6 @@ mod job_payload {
path: "f/system/hello_with_nodes_flow".to_string(),
dedicated_worker: None,
version: 1443253234253454,
debouncing_settings: Default::default(),
})
.run_until_complete(&db, false, port)
.await
@@ -168,8 +163,7 @@ mod job_payload {
let result = RunJob::from(JobPayload::FlowScript {
id: flow_scripts[0],
language: ScriptLang::Deno,
concurrency_settings:
windmill_common::runnable_settings::ConcurrencySettings::default(),
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default(),
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
@@ -188,8 +182,7 @@ mod job_payload {
let result = RunJob::from(JobPayload::FlowScript {
id: flow_scripts[1],
language: ScriptLang::Deno,
concurrency_settings:
windmill_common::runnable_settings::ConcurrencySettings::default(),
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default(),
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
@@ -222,7 +215,6 @@ mod job_payload {
path: "f/system/hello_with_nodes_flow".to_string(),
dedicated_worker: None,
version: 1443253234253454,
debouncing_settings: Default::default(),
})
.run_until_complete(&db, false, port)
.await
@@ -265,7 +257,6 @@ mod job_payload {
path: "f/system/hello".to_string(),
hash: ScriptHash(123412),
language: ScriptLang::Deno,
debouncing_settings: Default::default(),
dedicated_worker: None,
})
.run_until_complete(&db, false, port)
@@ -311,7 +302,6 @@ mod job_payload {
path: "f/system/hello_with_nodes_flow".to_string(),
dedicated_worker: None,
version: 1443253234253454,
debouncing_settings: Default::default(),
})
.run_until_complete(&db, false, port)
.await
@@ -453,7 +443,6 @@ mod job_payload {
path: "f/system/hello_with_nodes_flow".to_string(),
dedicated_worker: None,
version: 1443253234253454,
debouncing_settings: Default::default(),
})
.run_until_complete(&db, false, port)
.await
@@ -526,7 +515,6 @@ mod job_payload {
path: "f/system/hello_with_preprocessor".to_string(),
dedicated_worker: None,
version: 1443253234253456,
debouncing_settings: Default::default(),
})
.run_until_complete(db, false, port)
.await
@@ -559,7 +547,6 @@ mod job_payload {
completed_job_id,
step_id: "a".into(),
branch_or_iteration_n: None,
flow_version: None,
})
.arg("iter", json!({ "value": "tests", "index": 0 }))
.run_until_complete(&db, false, port)
@@ -583,7 +570,6 @@ mod job_payload {
path: "f/system/hello_with_nodes_flow".to_string(),
dedicated_worker: None,
version: 1443253234253454,
debouncing_settings: Default::default(),
})
.run_until_complete(&db, false, port)
.await
@@ -728,12 +714,7 @@ mod job_payload {
)
.await;
let flow_job_id = test(
Some(RestartedFrom {
flow_job_id,
step_id: "a".into(),
branch_or_iteration_n: None,
flow_version: None,
}),
Some(RestartedFrom { flow_job_id, step_id: "a".into(), branch_or_iteration_n: None }),
json!("foo"),
json!([
"a: Hello foo! foo! foo!",
@@ -743,12 +724,7 @@ mod job_payload {
)
.await;
let flow_job_id = test(
Some(RestartedFrom {
flow_job_id,
step_id: "b".into(),
branch_or_iteration_n: None,
flow_version: None,
}),
Some(RestartedFrom { flow_job_id, step_id: "b".into(), branch_or_iteration_n: None }),
json!("bar"),
json!([
"a: Hello foo! bar! bar!",
@@ -762,7 +738,6 @@ mod job_payload {
flow_job_id,
step_id: "c".into(),
branch_or_iteration_n: Some(1),
flow_version: None,
}),
json!("yolo"),
json!([

View File

@@ -189,8 +189,8 @@ def main():
path: None,
language: ScriptLang::Python3,
lock: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
@@ -238,8 +238,8 @@ def main():
path: None,
language: ScriptLang::Python3,
lock: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
@@ -272,8 +272,8 @@ def main():
path: None,
language: ScriptLang::Python3,
lock: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
@@ -311,8 +311,8 @@ def main():
path: None,
language: ScriptLang::Python3,
lock: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
@@ -348,8 +348,8 @@ def main():
path: None,
language: ScriptLang::Python3,
lock: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,

View File

@@ -188,7 +188,7 @@ async fn test_deno_flow(db: Pool<Postgres>) -> anyhow::Result<()> {
path: None,
lock: None,
tag: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default()
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default()
.into(),
is_trigger: None,
assets: None,
@@ -235,7 +235,7 @@ async fn test_deno_flow(db: Pool<Postgres>) -> anyhow::Result<()> {
lock: None,
tag: None,
concurrency_settings:
windmill_common::runnable_settings::ConcurrencySettings::default().into(),
windmill_common::jobs::ConcurrencySettings::default().into(),
is_trigger: None,
assets: None,
}
@@ -369,7 +369,7 @@ async fn test_deno_flow_same_worker(db: Pool<Postgres>) -> anyhow::Result<()> {
path: None,
lock: None,
tag: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default().into(),
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
is_trigger: None,
assets: None,
@@ -425,7 +425,7 @@ async fn test_deno_flow_same_worker(db: Pool<Postgres>) -> anyhow::Result<()> {
path: None,
lock: None,
tag: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default().into(),
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
is_trigger: None,
assets: None,
}.into(),
@@ -465,7 +465,7 @@ async fn test_deno_flow_same_worker(db: Pool<Postgres>) -> anyhow::Result<()> {
path: None,
lock: None,
tag: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default().into(),
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
is_trigger: None,
assets: None,
@@ -533,7 +533,7 @@ async fn test_deno_flow_same_worker(db: Pool<Postgres>) -> anyhow::Result<()> {
path: None,
lock: None,
tag: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default().into(),
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
is_trigger: None,
assets: None,
}.into(),
@@ -865,9 +865,8 @@ func main(derp string) (string, error) {
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default()
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
}))
.arg("derp", json!("world"))
.run_until_complete(&db, false, port)
@@ -901,9 +900,8 @@ fn main(world: String) -> Result<String, String> {
lock: None,
language: ScriptLang::Rust,
cache_ignore_s3_path: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default()
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
cache_ttl: None,
dedicated_worker: None,
}))
@@ -980,9 +978,8 @@ echo "hello $msg"
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default()
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
}))
.arg("msg", json!("world"))
.run_until_complete(&db, false, port)
@@ -1014,9 +1011,8 @@ def main [ msg: string ] {
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default()
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
}))
.arg("msg", json!("world"))
.run_until_complete(&db, false, port)
@@ -1068,9 +1064,8 @@ def main [
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default()
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
}))
.arg("a", json!("3"))
.arg("b", json!("null"))
@@ -1131,9 +1126,8 @@ public class Main {
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default()
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
}))
.arg("a", json!(3))
.arg("b", json!(3.0))
@@ -1167,9 +1161,8 @@ export async function main(a: Date) {
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default()
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
}))
.arg("a", json!("2024-09-24T10:00:00.000Z"))
.run_until_complete(&db, false, port)
@@ -1203,9 +1196,8 @@ export async function main(a: Date) {
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default()
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
}))
.arg("a", json!("2024-09-24T10:00:00.000Z"))
.run_until_complete(&db, false, port)
@@ -1240,9 +1232,8 @@ def main(a: datetime, b: bytes):
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default()
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
concurrency_settings: windmill_common::jobs::ConcurrencySettings::default().into(),
debouncing_settings: windmill_common::jobs::DebouncingSettings::default(),
}))
.arg("a", json!("2024-09-24T10:00:00.000Z"))
.arg("b", json!("dGVzdA=="))
@@ -2288,7 +2279,6 @@ async fn test_complex_flow_restart(db: Pool<Postgres>) -> anyhow::Result<()> {
flow_job_id: first_run_result.id,
step_id: "h".to_owned(),
branch_or_iteration_n: None,
flow_version: None,
}),
})
.run_until_complete(&db, false, port)

View File

@@ -92,7 +92,7 @@ mail-parser = { workspace = true, features = ["serde_support"], optional = true
magic-crypt.workspace = true
tempfile.workspace = true
tokio-util.workspace = true
astral-tokio-tar.workspace = true
tokio-tar.workspace = true
tokio-postgres.workspace = true
postgres-native-tls.workspace = true
hmac.workspace = true
@@ -149,7 +149,6 @@ rustls = { workspace = true }
aws-sigv4.workspace = true
aws-sdk-config.workspace = true
aws-config = { workspace = true, optional = true }
aws-credential-types.workspace = true
async-trait.workspace = true
google-cloud-pubsub = { workspace = true, optional = true }
google-cloud-googleapis = { workspace = true , optional = true }

View File

@@ -1,7 +1,7 @@
openapi: "3.0.3"
info:
version: 1.599.2
version: 1.592.1
title: Windmill API
contact:
@@ -2167,8 +2167,6 @@ paths:
type: string
teams_team_name:
type: string
teams_team_guid:
type: string
auto_invite_domain:
type: string
auto_invite_operator:
@@ -2560,13 +2558,7 @@ paths:
- $ref: "#/components/parameters/WorkspaceId"
- name: search
in: query
description: Search teams by name. If omitted, returns first page of all teams.
required: false
schema:
type: string
- name: next_link
in: query
description: Pagination cursor URL from previous response. Pass this to fetch the next page of results.
description: Search teams by name
required: false
schema:
type: string
@@ -2576,27 +2568,14 @@ paths:
content:
application/json:
schema:
type: object
properties:
teams:
type: array
items:
type: object
properties:
team_name:
type: string
team_id:
type: string
total_count:
type: integer
description: Total number of teams across all pages
per_page:
type: integer
description: Number of teams per page (configurable via TEAMS_PER_PAGE env var)
next_link:
type: string
nullable: true
description: URL to fetch next page of results. Null if no more pages.
type: array
items:
type: object
properties:
team_name:
type: string
team_id:
type: string
/w/{workspace}/workspaces/available_teams_channels:
get:
@@ -2612,25 +2591,26 @@ paths:
required: true
schema:
type: string
- name: search
in: query
description: Search channels by name
required: false
schema:
type: string
responses:
"200":
description: List of channels for the specified team
content:
application/json:
schema:
type: object
properties:
channels:
type: array
items:
type: object
properties:
channel_name:
type: string
channel_id:
type: string
total_count:
type: integer
type: array
items:
type: object
properties:
channel_name:
type: string
channel_id:
type: string
/w/{workspace}/workspaces/connect_teams:
post:
@@ -2976,24 +2956,6 @@ paths:
items:
type: string
/w/{workspace}/workspaces/list_datatable_schemas:
get:
summary: list schemas of all connected Datatables
operationId: listDataTableSchemas
tags:
- workspace
parameters:
- $ref: "#/components/parameters/WorkspaceId"
responses:
"200":
description: schemas of all datatables
content:
application/json:
schema:
type: array
items:
$ref: "#/components/schemas/DataTableSchema"
/w/{workspace}/workspaces/edit_ducklake_config:
post:
summary: edit ducklake settings
@@ -8355,7 +8317,7 @@ paths:
schema:
type: string
/w/{workspace}/jobs/restart/f/{id}:
/w/{workspace}/jobs/restart/f/{id}/from/{step_id}/{branch_or_iteration_n}:
post:
summary: restart a completed flow at a given step
operationId: restartFlowAtStep
@@ -8364,6 +8326,20 @@ paths:
parameters:
- $ref: "#/components/parameters/WorkspaceId"
- $ref: "#/components/parameters/JobId"
- name: step_id
description: step id to restart the flow from
required: true
in: path
schema:
type: string
- name: branch_or_iteration_n
description:
for branchall or loop, the iteration at which the flow should
restart
required: true
in: path
schema:
type: integer
- name: scheduled_for
description: when to schedule this job (leave empty for immediate run)
in: query
@@ -8386,24 +8362,12 @@ paths:
type: boolean
requestBody:
description: restart flow parameters
description: flow args
required: true
content:
application/json:
schema:
type: object
required:
- step_id
properties:
step_id:
type: string
description: step id to restart the flow from
branch_or_iteration_n:
type: integer
description: for branchall or loop, the iteration at which the flow should restart (optional)
flow_version:
type: integer
description: specific flow version to use for restart (optional, uses current version if not specified)
$ref: "#/components/schemas/ScriptArgs"
responses:
"201":
@@ -9734,34 +9698,6 @@ paths:
- completed
- result
/w/{workspace}/jobs_u/completed/get_timing/{id}:
get:
summary: get completed job timing
operationId: getCompletedJobTiming
tags:
- job
parameters:
- $ref: "#/components/parameters/WorkspaceId"
- $ref: "#/components/parameters/JobId"
responses:
"200":
description: job timing details
content:
application/json:
schema:
type: object
properties:
created_at:
type: string
format: date-time
started_at:
type: string
format: date-time
duration_ms:
type: integer
required:
- created_at
/w/{workspace}/jobs/completed/delete/{id}:
post:
summary: delete completed job (erase content but keep run id)
@@ -16572,14 +16508,6 @@ components:
type: string
debounce_delay_s:
type: integer
debounce_args_to_accumulate:
type: array
items:
type: string
max_total_debouncing_time:
type: integer
max_total_debounces_amount:
type: integer
cache_ttl:
type: number
dedicated_worker:
@@ -16683,14 +16611,6 @@ components:
type: string
debounce_delay_s:
type: integer
debounce_args_to_accumulate:
type: array
items:
type: string
max_total_debouncing_time:
type: integer
max_total_debounces_amount:
type: integer
visible_to_runner_only:
type: boolean
no_main_func:
@@ -20262,8 +20182,6 @@ components:
type: string
branch_or_iteration_n:
type: integer
flow_version:
type: integer
Policy:
type: object
@@ -20630,27 +20548,6 @@ components:
required:
- resource_type
DataTableSchema:
type: object
required: [datatable_name, schemas]
properties:
datatable_name:
type: string
schemas:
type: object
description: "Hierarchical schema: schema_name -> table_name -> column_name -> compact_type (e.g. 'int4', 'text?', 'int4?=0')"
additionalProperties:
type: object
description: "Tables in this schema"
additionalProperties:
type: object
description: "Columns in this table"
additionalProperties:
type: string
description: "Compact type: 'type[?][=default]' where ? means nullable"
error:
type: string
DynamicInputData:
type: object
properties:

View File

@@ -133,10 +133,6 @@ struct AIStandardResource {
api_key: Option<String>,
organization_id: Option<String>,
region: Option<String>,
#[serde(alias = "awsAccessKeyId")]
aws_access_key_id: Option<String>,
#[serde(alias = "awsSecretAccessKey")]
aws_secret_access_key: Option<String>,
}
#[derive(Deserialize, Debug)]
@@ -158,9 +154,6 @@ struct AIRequestConfig {
pub access_token: Option<String>,
pub organization_id: Option<String>,
pub user: Option<String>,
pub region: Option<String>,
pub aws_access_key_id: Option<String>,
pub aws_secret_access_key: Option<String>,
}
impl AIRequestConfig {
@@ -170,18 +163,8 @@ impl AIRequestConfig {
w_id: &str,
resource: AIResource,
) -> Result<Self> {
let (
api_key,
access_token,
organization_id,
base_url,
user,
region,
aws_access_key_id,
aws_secret_access_key,
) = match resource {
let (api_key, access_token, organization_id, base_url, user) = match resource {
AIResource::Standard(resource) => {
let region = resource.region.clone();
let base_url = provider
.get_base_url(resource.base_url, resource.region, db)
.await?;
@@ -195,28 +178,8 @@ impl AIRequestConfig {
} else {
None
};
let aws_access_key_id = if let Some(access_key_id) = resource.aws_access_key_id {
Some(get_variable_or_self(access_key_id, db, w_id).await?)
} else {
None
};
let aws_secret_access_key =
if let Some(secret_access_key) = resource.aws_secret_access_key {
Some(get_variable_or_self(secret_access_key, db, w_id).await?)
} else {
None
};
(
api_key,
None,
organization_id,
base_url,
None,
region,
aws_access_key_id,
aws_secret_access_key,
)
(api_key, None, organization_id, base_url, None)
}
AIResource::OAuth(resource) => {
let user = if let Some(user) = resource.user.clone() {
@@ -227,20 +190,11 @@ impl AIRequestConfig {
let token = Self::get_token_using_oauth(resource, db, w_id).await?;
let base_url = provider.get_base_url(None, None, db).await?;
(None, Some(token), None, base_url, user, None, None, None)
(None, Some(token), None, base_url, user)
}
};
Ok(Self {
base_url,
organization_id,
api_key,
access_token,
user,
region,
aws_access_key_id,
aws_secret_access_key,
})
Ok(Self { base_url, organization_id, api_key, access_token, user })
}
async fn get_token_using_oauth(
@@ -297,10 +251,6 @@ impl AIRequestConfig {
let is_anthropic_sdk = headers.get("X-Anthropic-SDK").is_some();
let is_bedrock = matches!(provider, AIProvider::AWSBedrock);
// Check if using IAM credentials for Bedrock (instead of bearer token)
let use_iam_auth =
is_bedrock && self.aws_access_key_id.is_some() && self.aws_secret_access_key.is_some();
// Handle AWS Bedrock transformation
let (url, body) = if is_bedrock && method != Method::GET {
let (model, transformed_body, is_streaming) =
@@ -332,7 +282,7 @@ impl AIRequestConfig {
tracing::debug!("AI request URL: {}", url);
let mut request = HTTP_CLIENT
.request(method.clone(), &url)
.request(method, url)
.header("content-type", "application/json");
for (header_name, header_value) in headers.iter() {
@@ -341,42 +291,22 @@ impl AIRequestConfig {
}
}
// For Bedrock with IAM credentials, sign the request using SigV4
if use_iam_auth {
let region = self.region.as_deref().ok_or_else(|| {
Error::internal_err("AWS region must be set for IAM authentication with Bedrock")
})?;
let signed_headers = bedrock::sign_bedrock_request(
method.as_str(),
&url,
&body,
self.aws_access_key_id.as_ref().unwrap(),
self.aws_secret_access_key.as_ref().unwrap(),
region,
)?;
request = request.body(body);
for (header_name, header_value) in signed_headers {
request = request.header(header_name, header_value);
if let Some(api_key) = self.api_key {
if is_azure {
request = request.header("api-key", api_key.clone())
} else {
request = request.header("authorization", format!("Bearer {}", api_key.clone()))
}
} else {
// For non-IAM auth, use bearer token or API key
if let Some(api_key) = self.api_key {
if is_azure {
request = request.header("api-key", api_key.clone())
} else {
request = request.header("authorization", format!("Bearer {}", api_key.clone()))
}
if is_anthropic {
request = request.header("X-API-Key", api_key);
}
}
if let Some(access_token) = self.access_token {
request = request.header("authorization", format!("Bearer {}", access_token))
if is_anthropic {
request = request.header("X-API-Key", api_key);
}
}
request = request.body(body);
if let Some(access_token) = self.access_token {
request = request.header("authorization", format!("Bearer {}", access_token))
}
if let Some(org_id) = self.organization_id {
request = request.header("OpenAI-Organization", org_id);

View File

@@ -61,7 +61,7 @@ use windmill_common::{
users::username_to_permissioned_as,
utils::{
http_get_from_hub, not_found_if_none, paginate, query_elems_from_hub, require_admin,
Pagination, RunnableKind, StripPath,
Pagination, RunnableKind, StripPath, WarnAfterExt,
},
variables::{build_crypt, build_crypt_with_key_suffix, encrypt},
worker::{to_raw_value, CLOUD_HOSTED},
@@ -1222,11 +1222,7 @@ async fn create_app_internal<'a>(
&db,
tx,
w_id,
JobPayload::AppDependencies {
path: app.path.clone(),
version: v_id,
debouncing_settings: Default::default(),
},
JobPayload::AppDependencies { path: app.path.clone(), version: v_id },
PushArgs { args: &args, extra: None },
&authed.username,
&authed.email,
@@ -1251,6 +1247,7 @@ async fn create_app_internal<'a>(
None,
None,
None,
None,
)
.await?;
tracing::info!("Pushed app dependency job {}", dependency_job_uuid);
@@ -1534,6 +1531,14 @@ async fn update_app_internal<'a>(
path.to_owned()
};
let v_id = if let Some(nvalue) = &ns.value {
// Row lock debounce key for path. We need this to make all updates of runnables sequential and predictable.
tokio::time::timeout(
core::time::Duration::from_secs(60),
windmill_common::jobs::lock_debounce_key(&w_id, &npath, &mut tx),
)
.warn_after_seconds(10)
.await??;
let app_id = sqlx::query_scalar!(
"SELECT id FROM app WHERE path = $1 AND workspace_id = $2",
npath,
@@ -1608,11 +1613,7 @@ async fn update_app_internal<'a>(
&db,
tx,
w_id,
JobPayload::AppDependencies {
path: npath.clone(),
version: v_id,
debouncing_settings: Default::default(),
},
JobPayload::AppDependencies { path: npath.clone(), version: v_id },
PushArgs { args: &args, extra: None },
&authed.username,
&authed.email,
@@ -1637,6 +1638,7 @@ async fn update_app_internal<'a>(
None,
None,
None,
None,
)
.await?;
tracing::info!("Pushed app dependency job {}", dependency_job_uuid);
@@ -1969,6 +1971,7 @@ async fn execute_component(
end_user_email,
None,
None,
None,
)
.await?;

View File

@@ -1,73 +1,9 @@
use axum::body::Bytes;
use aws_sigv4::http_request::{sign, SignableBody, SignableRequest, SigningSettings};
use aws_sigv4::sign::v4;
use bytes;
use futures;
use std::time::SystemTime;
use uuid;
use windmill_common::error::{Error, Result};
/// Sign a request for AWS Bedrock using SigV4
///
/// Returns a vector of (header_name, header_value) tuples to add to the request
pub fn sign_bedrock_request(
method: &str,
uri: &str,
body: &[u8],
access_key_id: &str,
secret_access_key: &str,
region: &str,
) -> Result<Vec<(String, String)>> {
let identity = aws_credential_types::Credentials::new(
access_key_id,
secret_access_key,
None, // session token
None, // expiration
"windmill",
)
.into();
let signing_settings = SigningSettings::default();
let signing_params = v4::SigningParams::builder()
.identity(&identity)
.region(region)
.name("bedrock")
.time(SystemTime::now())
.settings(signing_settings)
.build()
.map_err(|e| Error::internal_err(format!("Failed to build signing params: {}", e)))?;
// Parse the URI to extract path and query
let parsed_uri: http::Uri = uri
.parse()
.map_err(|e| Error::internal_err(format!("Failed to parse URI: {}", e)))?;
let path_and_query = parsed_uri
.path_and_query()
.map(|pq| pq.as_str())
.unwrap_or("/");
let signable_request = SignableRequest::new(
method,
path_and_query,
std::iter::once(("host", parsed_uri.host().unwrap_or(""))),
SignableBody::Bytes(body),
)
.map_err(|e| Error::internal_err(format!("Failed to create signable request: {}", e)))?;
let (signing_instructions, _signature) = sign(signable_request, &signing_params.into())
.map_err(|e| Error::internal_err(format!("Failed to sign request: {}", e)))?
.into_parts();
// Collect the headers to add
let mut headers = Vec::new();
for (name, value) in signing_instructions.headers() {
headers.push((name.to_string(), value.to_string()));
}
Ok(headers)
}
/// Transform OpenAI format request to AWS Bedrock Converse format
/// Returns: (model_id, transformed_body, is_streaming)
pub fn transform_openai_to_bedrock(body: &[u8]) -> Result<(String, Bytes, bool)> {

View File

@@ -32,11 +32,8 @@ use sql_builder::prelude::*;
use sqlx::{FromRow, Postgres, Transaction};
use windmill_audit::audit_oss::audit_log;
use windmill_audit::ActionKind;
use windmill_common::runnable_settings::RunnableSettingsTrait;
use windmill_common::utils::query_elems_from_hub;
use windmill_common::worker::{
to_raw_value, CLOUD_HOSTED, MIN_VERSION_SUPPORTS_DEBOUNCING, MIN_VERSION_SUPPORTS_DEBOUNCING_V2,
};
use windmill_common::utils::{query_elems_from_hub, WarnAfterExt};
use windmill_common::worker::{to_raw_value, CLOUD_HOSTED, MIN_VERSION_SUPPORTS_DEBOUNCING};
use windmill_common::HUB_BASE_URL;
use windmill_common::{
db::UserDB,
@@ -48,7 +45,6 @@ use windmill_common::{
utils::{http_get_from_hub, not_found_if_none, paginate, Pagination, RunnableKind, StripPath},
};
use windmill_git_sync::{handle_deployment_metadata, DeployedObject};
use windmill_queue::WMDEBUG_FORCE_NO_LEGACY_DEBOUNCING_COMPAT;
use windmill_queue::{push, schedule::push_scheduled_job, PushIsolationLevel};
use windmill_worker::scoped_dependency_map::ScopedDependencyMap;
@@ -544,7 +540,6 @@ async fn create_flow(
path: nf.path.clone(),
dedicated_worker: nf.dedicated_worker,
version: version,
debouncing_settings: Default::default(),
},
windmill_queue::PushArgs { args: &args, extra: None },
&authed.username,
@@ -570,6 +565,7 @@ async fn create_flow(
None,
None,
None,
None,
)
.await?;
@@ -971,6 +967,14 @@ async fn update_flow(
.await?;
}
// Row lock debounce key for path. We need this to make all updates of runnables sequential and predictable.
tokio::time::timeout(
core::time::Duration::from_secs(60),
windmill_common::jobs::lock_debounce_key(&w_id, &nf.path, &mut tx),
)
.warn_after_seconds(10)
.await??;
// tracing::error!("Updating flow: {:?}", nf.value.get());
// This will lock anyone who is trying to iterate on flow_versions with given path and parameters.
@@ -1082,7 +1086,6 @@ async fn update_flow(
path: nf.path.clone(),
dedicated_worker: nf.dedicated_worker,
version,
debouncing_settings: Default::default(),
},
windmill_queue::PushArgs { args: &args, extra: None },
&authed.username,
@@ -1108,6 +1111,7 @@ async fn update_flow(
None,
None,
None,
None,
)
.await?;
@@ -1483,20 +1487,6 @@ async fn guard_flow_from_debounce_data(nf: &NewFlow) -> Result<()> {
"Flow debouncing configuration rejected: workers are behind minimum required version for debouncing feature"
);
Err(Error::WorkersAreBehind { feature: "Debouncing".into(), min_version: "1.566.0".into() })
} else if !*MIN_VERSION_SUPPORTS_DEBOUNCING_V2.read().await
&& !nf
.parse_flow_value()?
.debouncing_settings
.is_legacy_compatible()
&& !*WMDEBUG_FORCE_NO_LEGACY_DEBOUNCING_COMPAT
{
tracing::warn!(
"Flow debouncing configuration rejected: workers are behind minimum required version for debouncing feature"
);
Err(Error::WorkersAreBehind {
feature: "V2 Debouncing".into(),
min_version: "1.597.0".into(),
})
} else {
Ok(())
}
@@ -1611,9 +1601,7 @@ mod tests {
ConstantDelay, ExponentialDelay, FlowModule, FlowModuleValue, FlowValue,
InputTransform, Retry, StopAfterIf,
},
runnable_settings::{
ConcurrencySettings, ConcurrencySettingsWithCustom, DebouncingSettings,
},
jobs::{ConcurrencySettings, ConcurrencySettingsWithCustom, DebouncingSettings},
scripts,
};

View File

@@ -14,7 +14,7 @@ use axum::{
};
use windmill_common::{
db::UserDB,
error::JsonResult,
error::{Error, JsonResult},
utils::{paginate, Pagination},
};
@@ -40,6 +40,7 @@ async fn get_group_permission_history(
Path((w_id, name)): Path<(String, String)>,
Query(pagination): Query<Pagination>,
) -> JsonResult<Vec<GroupPermissionChange>> {
let mut tx = user_db.begin(&authed).await?;
let (per_page, offset) = paginate(pagination);

View File

@@ -36,13 +36,10 @@ use windmill_common::flow_conversations::add_message_to_conversation_tx;
use windmill_common::flow_status::{JobResult, RestartedFrom};
use windmill_common::jobs::{
check_tag_available_for_workspace_internal, format_completed_job_result, format_result,
DynamicInput, JobTriggerKind, RunInlinePreviewScriptFnParams, ENTRYPOINT_OVERRIDE,
};
use windmill_common::runnable_settings::{
ConcurrencySettings, ConcurrencySettingsWithCustom, DebouncingSettings, RunnableSettings,
ConcurrencySettings, ConcurrencySettingsWithCustom, DebouncingSettings, DynamicInput,
JobTriggerKind, RunInlinePreviewScriptFnParams, ENTRYPOINT_OVERRIDE,
};
use windmill_common::s3_helpers::{upload_artifact_to_store, BundleFormat};
use windmill_common::scripts::ScriptRunnableSettingsInline;
use windmill_common::triggers::TriggerMetadata;
use windmill_common::utils::{RunnableKind, WarnAfterExt};
use windmill_common::worker::{Connection, CLOUD_HOSTED, TMP_DIR};
@@ -158,7 +155,11 @@ pub fn workspaced_service() -> Router {
.layer(ce_headers.clone()),
)
.route(
"/restart/f/:job_id",
"/restart/f/:job_id/from/:step_id",
post(restart_flow).head(|| async { "" }).layer(cors.clone()),
)
.route(
"/restart/f/:job_id/from/:step_id/:branch_of_iteration_n",
post(restart_flow).head(|| async { "" }).layer(cors.clone()),
)
.route(
@@ -301,10 +302,6 @@ pub fn workspaced_service() -> Router {
"/completed/get_result_maybe/:id",
get(get_completed_job_result_maybe).layer(cors.clone()),
)
.route(
"/completed/get_timing/:id",
get(get_completed_job_timing).layer(cors.clone()),
)
.route(
"/completed/delete/:id",
post(delete_completed_job).layer(cors.clone()),
@@ -381,7 +378,6 @@ pub fn workspace_unauthed_service() -> Router {
"/completed/get_result_maybe/:id",
get(get_completed_job_result_maybe),
)
.route("/completed/get_timing/:id", get(get_completed_job_timing))
.route("/getupdate/:id", get(get_job_update))
.route("/getupdate_sse/:id", get(get_job_update_sse))
.route("/get_log_file/*file_path", get(get_log_file))
@@ -895,7 +891,7 @@ macro_rules! get_job_query {
get_job_query!(
@impl "v2_job_queue", ($($opts)*),
"scheduled_for, running, ping as last_ping, suspend, suspend_until, same_worker, pre_run_error, visible_to_owner, \
flow_innermost_root_job AS root_job, flow_leaf_jobs AS leaf_jobs, concurrent_limit, concurrency_time_window_s, timeout, flow_step_id, cache_ttl, cache_ignore_s3_path, runnable_settings_handle, \
flow_innermost_root_job AS root_job, flow_leaf_jobs AS leaf_jobs, concurrent_limit, concurrency_time_window_s, timeout, flow_step_id, cache_ttl, cache_ignore_s3_path, \
script_entrypoint_override",
"LEFT JOIN v2_job_runtime ON v2_job_runtime.id = v2_job_queue.id LEFT JOIN v2_job_status ON v2_job_status.id = v2_job_queue.id",
)
@@ -3377,7 +3373,6 @@ pub struct UnifiedJob {
pub aggregate_wait_time_ms: Option<i64>,
pub preprocessed: Option<bool>,
pub worker: Option<String>,
pub runnable_settings_handle: Option<i64>,
}
const CJ_FIELDS: &[&str] = &[
@@ -3418,7 +3413,6 @@ const CJ_FIELDS: &[&str] = &[
"aggregate_wait_time_ms",
"v2_job.preprocessed",
"v2_job_completed.worker",
"null as runnable_settings_handle",
];
const QJ_FIELDS: &[&str] = &[
@@ -3459,7 +3453,6 @@ const QJ_FIELDS: &[&str] = &[
"aggregate_wait_time_ms",
"v2_job.preprocessed",
"v2_job_queue.worker",
"v2_job_queue.runnable_settings_handle",
];
impl UnifiedJob {
@@ -3524,13 +3517,14 @@ impl<'a> From<UnifiedJob> for Job {
created_by: uj.created_by,
created_at: uj.created_at,
started_at: uj.started_at,
scheduled_for: uj.scheduled_for.unwrap(),
running: uj.running.unwrap(),
script_hash: uj.script_hash,
script_path: uj.script_path,
script_entrypoint_override: None,
args: None,
running: uj.running.unwrap(),
scheduled_for: uj.scheduled_for.unwrap(),
logs: None,
flow_status: None,
workflow_as_code_status: None,
canceled: uj.canceled,
canceled_by: uj.canceled_by,
canceled_reason: None,
@@ -3538,10 +3532,9 @@ impl<'a> From<UnifiedJob> for Job {
job_kind: uj.job_kind,
schedule_path: uj.schedule_path,
permissioned_as: uj.permissioned_as,
flow_status: None,
workflow_as_code_status: None,
is_flow_step: uj.is_flow_step,
language: uj.language,
script_entrypoint_override: None,
same_worker: false,
pre_run_error: None,
email: uj.email,
@@ -3559,7 +3552,6 @@ impl<'a> From<UnifiedJob> for Job {
cache_ignore_s3_path: None,
priority: uj.priority,
preprocessed: uj.preprocessed,
runnable_settings_handle: uj.runnable_settings_handle,
},
)),
t => panic!("job type {} not valid", t),
@@ -4199,6 +4191,7 @@ pub async fn run_flow<'c>(
push_authed.as_ref(),
false,
None,
None,
trigger,
run_query.suspended_mode,
)
@@ -4395,24 +4388,18 @@ pub async fn restart_flow(
));
}
#[cfg(feature = "enterprise")]
#[derive(Deserialize)]
pub struct RestartFlowRequestBody {
step_id: String,
branch_or_iteration_n: Option<usize>,
flow_version: Option<i64>,
}
#[cfg(feature = "enterprise")]
pub async fn restart_flow(
authed: ApiAuthed,
Extension(db): Extension<DB>,
Extension(user_db): Extension<UserDB>,
Path((w_id, job_id)): Path<(String, Uuid)>,
Path((w_id, job_id, step_id, branch_or_iteration_n)): Path<(
String,
Uuid,
String,
Option<usize>,
)>,
Query(run_query): Query<RunJobQuery>,
Json(RestartFlowRequestBody { step_id, branch_or_iteration_n, flow_version }): Json<
RestartFlowRequestBody,
>,
) -> error::Result<(StatusCode, String)> {
check_license_key_valid().await?;
@@ -4451,12 +4438,7 @@ pub async fn restart_flow(
&db,
tx,
&w_id,
JobPayload::RestartedFlow {
completed_job_id: job_id,
step_id,
branch_or_iteration_n,
flow_version,
},
JobPayload::RestartedFlow { completed_job_id: job_id, step_id, branch_or_iteration_n },
push_args,
&authed.username,
&authed.email,
@@ -4480,6 +4462,7 @@ pub async fn restart_flow(
false,
None,
None,
None,
run_query.suspended_mode,
)
.await?;
@@ -4613,6 +4596,7 @@ pub async fn push_script_job_by_path_into_queue<'c>(
push_authed.as_ref(),
false,
None,
None,
trigger,
run_query.suspended_mode,
)
@@ -4670,13 +4654,6 @@ pub async fn run_workflow_as_code(
let job = not_found_if_none(job, "Queued Job", &job_id.to_string())?;
let JobExtended { inner: job, raw_code, raw_lock, .. } = job;
let (_debouncing_settings, concurrency_settings) =
RunnableSettings::from_runnable_settings_handle(job.runnable_settings_handle, &db)
.await?
.prefetch_cached(&db)
.await?;
let (job_payload, tag, _delete_after_use, timeout, on_behalf_of) = match job.job_kind {
JobKind::Preview => (
JobPayload::Code(RawCode {
@@ -4685,15 +4662,13 @@ pub async fn run_workflow_as_code(
path: job.script_path,
language: job.language.unwrap_or_else(|| ScriptLang::Deno),
lock: raw_lock,
concurrency_settings: concurrency_settings
.maybe_fallback(
windmill_queue::custom_concurrency_key(&db, &job.id)
.await
.map_err(to_anyhow)?,
job.concurrent_limit,
job.concurrency_time_window_s,
)
.into(),
concurrency_settings: windmill_common::jobs::ConcurrencySettingsWithCustom {
custom_concurrency_key: windmill_queue::custom_concurrency_key(&db, &job.id)
.await
.map_err(to_anyhow)?,
concurrent_limit: job.concurrent_limit,
concurrency_time_window_s: job.concurrency_time_window_s,
},
cache_ttl: job.cache_ttl,
cache_ignore_s3_path: job.cache_ignore_s3_path,
dedicated_worker: None,
@@ -4789,6 +4764,7 @@ pub async fn run_workflow_as_code(
None,
None,
None,
None,
)
.await?;
@@ -5326,6 +5302,7 @@ pub async fn run_wait_result_job_by_path_get(
false,
None,
None,
None,
run_query.suspended_mode,
)
.await?;
@@ -5471,6 +5448,7 @@ pub async fn run_wait_result_script_by_path_internal(
false,
None,
None,
None,
run_query.suspended_mode,
)
.await?;
@@ -5511,6 +5489,11 @@ pub async fn run_wait_result_script_by_hash(
let ScriptHashInfo {
path,
tag,
concurrency_key,
concurrent_limit,
concurrency_time_window_s,
debounce_key,
debounce_delay_s,
mut cache_ttl,
mut cache_ignore_s3_path,
language,
@@ -5521,14 +5504,8 @@ pub async fn run_wait_result_script_by_hash(
has_preprocessor,
on_behalf_of_email,
created_by,
runnable_settings:
ScriptRunnableSettingsInline { concurrency_settings, debouncing_settings },
..
} = get_script_info_for_hash(Some(userdb_authed), &db, &w_id, hash)
.await?
.prefetch_cached(&db)
.await?;
} = get_script_info_for_hash(Some(userdb_authed), &db, &w_id, hash).await?;
if let Some(run_query_cache_ttl) = run_query.cache_ttl {
cache_ttl = Some(run_query_cache_ttl);
cache_ignore_s3_path = run_query.cache_ignore_s3_path;
@@ -5562,8 +5539,17 @@ pub async fn run_wait_result_script_by_hash(
JobPayload::ScriptHash {
hash: ScriptHash(hash),
path: path,
concurrency_settings,
debouncing_settings,
concurrency_settings: windmill_common::jobs::ConcurrencySettingsWithCustom {
custom_concurrency_key: concurrency_key,
concurrent_limit: concurrent_limit,
concurrency_time_window_s: concurrency_time_window_s,
}
.into(),
debouncing_settings: DebouncingSettings {
custom_key: debounce_key,
delay_s: debounce_delay_s,
..Default::default() // TODO
},
cache_ttl,
cache_ignore_s3_path,
language,
@@ -5595,6 +5581,7 @@ pub async fn run_wait_result_script_by_hash(
false,
None,
None,
None,
run_query.suspended_mode,
)
.await?;
@@ -6070,6 +6057,7 @@ async fn run_preview_script(
None,
None,
None,
None,
)
.await?;
tx.commit().await?;
@@ -6222,6 +6210,7 @@ async fn run_bundle_preview_script(
None,
None,
None,
None,
)
.await?;
job_id = Some(uuid);
@@ -6373,6 +6362,7 @@ async fn run_dependencies_job(
None,
None,
None,
None,
)
.await?;
tx.commit().await?;
@@ -6461,6 +6451,7 @@ async fn run_flow_dependencies_job(
None,
None,
None,
None,
)
.await?;
tx.commit().await?;
@@ -6501,15 +6492,9 @@ async fn add_batch_jobs(
job_kind,
language,
dedicated_worker,
ScriptRunnableSettingsInline {
concurrency_settings:
ConcurrencySettings {
concurrent_limit,
concurrency_time_window_s,
concurrency_key: custom_concurrency_key,
},
..
},
custom_concurrency_key,
concurrent_limit,
concurrent_time_window_s,
timeout,
raw_code,
raw_lock,
@@ -6522,22 +6507,23 @@ async fn add_batch_jobs(
UserDbWithAuthed { db: user_db.clone(), authed: &authed.to_authed_ref() };
let ScriptHashInfo {
hash: script_hash,
concurrency_key,
concurrent_limit,
concurrency_time_window_s,
language,
dedicated_worker,
timeout,
runnable_settings,
.. // TODO: consider on_behalf_of_email and created_by for batch jobs
} = get_latest_deployed_hash_for_path(Some(db_authed), db.clone(), &w_id, &path)
.await?
.prefetch_cached(&db)
.await?;
} = get_latest_deployed_hash_for_path(Some(db_authed), db.clone(), &w_id, &path).await?;
(
Some(script_hash),
Some(path),
JobKind::Script,
Some(language),
dedicated_worker,
runnable_settings,
concurrency_key,
concurrent_limit,
concurrency_time_window_s,
timeout,
None,
None,
@@ -6558,7 +6544,9 @@ async fn add_batch_jobs(
JobKind::Preview,
rawscript.language,
None,
Default::default(),
None,
None,
None,
None,
Some(rawscript.content),
rawscript.lock,
@@ -6603,20 +6591,19 @@ async fn add_batch_jobs(
add_virtual_items_if_necessary(&mut value.modules);
let flow_status = FlowStatus::new(&value);
(
None, // script_hash
path, // script_path
job_kind, // job_kind
None, // language
None, // dedicated_worker
ScriptRunnableSettingsInline {
concurrency_settings: value.concurrency_settings.clone(),
..Default::default()
},
None, // timeout
None, // raw_code
None, // raw_lock
Some(value), // raw_flow
Some(flow_status), // flow_status
None, // script_hash
path, // script_path
job_kind, // job_kind
None, // language
None, // dedicated_worker
value.concurrency_settings.concurrency_key.clone(), // custom_concurrency_key
value.concurrency_settings.concurrent_limit.clone(), // concurrent_limit
value.concurrency_settings.concurrency_time_window_s, // concurrency_time_window_s
None, // timeout
None, // raw_code
None, // raw_lock
Some(value), // raw_flow
Some(flow_status), // flow_status
)
}
"noop" => (
@@ -6625,7 +6612,9 @@ async fn add_batch_jobs(
JobKind::Noop,
None,
None,
Default::default(),
None,
None,
None,
None,
None,
None,
@@ -6680,7 +6669,7 @@ async fn add_batch_jobs(
username_to_permissioned_as(&authed.username),
authed.email,
concurrent_limit,
concurrency_time_window_s,
concurrent_time_window_s,
timeout,
n,
)
@@ -6817,6 +6806,7 @@ async fn run_preview_flow_job(
None,
None,
None,
None,
)
.await?;
@@ -7014,6 +7004,7 @@ async fn run_dynamic_select(
None,
None,
None,
None,
)
.await?;
tx.commit().await?;
@@ -7072,8 +7063,11 @@ pub async fn run_job_by_hash_inner(
let ScriptHashInfo {
path,
tag,
runnable_settings:
ScriptRunnableSettingsInline { concurrency_settings, debouncing_settings },
concurrency_key,
concurrent_limit,
concurrency_time_window_s,
debounce_delay_s,
debounce_key,
mut cache_ttl,
mut cache_ignore_s3_path,
language,
@@ -7085,10 +7079,7 @@ pub async fn run_job_by_hash_inner(
created_by,
delete_after_use,
..
} = get_script_info_for_hash(Some(userdb_authed), &db, &w_id, hash)
.await?
.prefetch_cached(&db)
.await?;
} = get_script_info_for_hash(Some(userdb_authed), &db, &w_id, hash).await?;
check_scopes(&authed, || format!("jobs:run:scripts:{path}"))?;
if let Some(run_query_cache_ttl) = run_query.cache_ttl {
@@ -7124,8 +7115,16 @@ pub async fn run_job_by_hash_inner(
JobPayload::ScriptHash {
hash: ScriptHash(hash),
path: path,
concurrency_settings,
debouncing_settings,
concurrency_settings: ConcurrencySettings {
concurrency_key,
concurrent_limit,
concurrency_time_window_s,
},
debouncing_settings: DebouncingSettings {
custom_key: debounce_key,
delay_s: debounce_delay_s,
..Default::default()
},
cache_ttl,
cache_ignore_s3_path,
language,
@@ -7156,6 +7155,7 @@ pub async fn run_job_by_hash_inner(
push_authed.as_ref(),
false,
None,
None,
trigger,
run_query.suspended_mode,
)
@@ -8552,54 +8552,6 @@ async fn get_completed_job_result_maybe(
}
}
#[derive(Serialize)]
struct JobTiming {
created_at: chrono::DateTime<Utc>,
started_at: Option<chrono::DateTime<Utc>>,
duration_ms: Option<i64>,
}
async fn get_completed_job_timing(
OptAuthed(opt_authed): OptAuthed,
Extension(db): Extension<DB>,
Path((w_id, id)): Path<(String, Uuid)>,
) -> error::JsonResult<JobTiming> {
let tags = opt_authed
.as_ref()
.map(|authed| get_scope_tags(authed))
.flatten();
let result = sqlx::query!(
"SELECT
j.created_at AS \"created_at!\",
c.started_at,
c.duration_ms,
j.created_by AS \"created_by!\"
FROM v2_job_completed c
JOIN v2_job j USING (id)
WHERE c.id = $1 AND c.workspace_id = $2 AND ($3::text[] IS NULL OR j.tag = ANY($3))",
id,
&w_id,
tags.as_ref().map(|v| v.as_slice()) as Option<&[&str]>,
)
.fetch_optional(&db)
.await?;
let result = not_found_if_none(result, "Completed Job", id.to_string())?;
if opt_authed.is_none() && result.created_by != "anonymous" {
return Err(Error::BadRequest(
"As a non logged in user, you can only see jobs ran by anonymous users".to_string(),
));
}
Ok(Json(JobTiming {
created_at: result.created_at,
started_at: result.started_at,
duration_ms: Some(result.duration_ms),
}))
}
async fn delete_completed_job<'a>(
authed: ApiAuthed,
Tokened { token }: Tokened,

Some files were not shown because too many files have changed in this diff Show More