make drawer triggerable

* add tracing to get of authed client

* fix: make disabled items not selectable with arrow keys

* Invert showing EE message only when not in EE

* Makea component for the Run Search part of the Search modal

* Make the button to load more jobs

* Add pagination for job search

* fix missing bind to the openModal bool

* Turn off spinner when aborting search results

* fix typo in openapi.yaml

* Update ee repo ref

* Remove unused imports and vars

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>

.aiderignore

@@ -0,0 +1,3 @@
+/*
+!/backend/
+!/frontend/

.cursor/rules/rust-best-practices.mdc

@@ -3,10 +3,6 @@ description:
 globs: backend/**/*.rs
 alwaysApply: false
 ---
----
-description: Rust best practices for the Windmill backend, covering code organization, error handling, performance optimizations, and common patterns to follow when adding new code.
-globs: **/*.rs
----
 # Windmill Backend - Rust Best Practices
 
 ## Project Structure

.cursor/rules/svelte5-best-practices.mdc

@@ -5,7 +5,7 @@ alwaysApply: false
 ---
 # Svelte 5 Best Practices
 
-This guide outlines best practices for developing with Svelte 5, incorporating the new Runes API and other modern Svelte features. They should be applied on every new files created, but not on existing svelte 4 files unless specifically asked to.
+This guide outlines best practices for developing with Svelte 5, incorporating the new Runes API and other modern Svelte features. These rules MUST NOT be applied on svelte 4 files unless explicitly asked to do so.
 
 ## Reactivity with Runes
 

.cursor/rules/windmill-overview.mdc

@@ -0,0 +1,76 @@
+---
+description: 
+globs: 
+alwaysApply: false
+---
+# Windmill Overview
+
+Windmill is an open-source developer platform for building internal tools, API integrations, background jobs, workflows, and user interfaces. It offers a unified system where scripts are automatically turned into sharable UIs and can be composed into flows or embedded in custom applications.
+
+## Core Capabilities
+
+- **Script Development and Execution**: Write and run scripts in Python, TypeScript/JavaScript (Deno/Bun), Go, Bash, SQL, and other languages
+- **Workflow Orchestration**: Compose scripts into multi-step flows with conditional logic, loops, and error handling
+- **UI Generation**: Automatically generate UIs from scripts or build custom applications with a low-code editor
+- **Job Scheduling**: Trigger scripts and flows on schedules, webhooks, or external events
+- **Resource Management**: Securely store and use credentials, databases, and other connections
+
+## Platform Architecture
+
+The Windmill platform consists of several key components:
+
+- **Frontend UI**: Web-based interface for script and flow development, app building, and result visualization
+- **API Server**: Central API that handles authentication, resource management, and job coordination
+- **Workers**: Execute scripts in their respective environments with proper sandboxing
+- **Database**: PostgreSQL database for storage of scripts, flows, resources, job results, and more
+- **Job Queue**: Queue system for managing job execution, implemented in PostgreSQL
+- **Client Libraries**: Libraries for interacting with Windmill from Python, TypeScript, or command line
+
+# Windmill Backend Architecture
+
+The Windmill backend is written in Rust and consists of several services working together. These services are designed for horizontal scaling with stateless API servers and workers that can be deployed across multiple machines.
+
+## Key Components
+
+- **API Server (`windmill-api`)**: Handles HTTP requests, authentication, and resource management
+- **Queue Manager (`windmill-queue`)**: Manages the job queue in PostgreSQL
+- **Worker System (`windmill-worker`)**: Executes jobs in sandboxed environments
+- **Common Utilities (`windmill-common`)**: Shared code used by multiple services
+- **Git Sync (`windmill-git-sync`)**: Synchronizes scripts with Git repositories
+
+## Job Execution System
+
+The job execution process follows these steps:
+
+1. The API server receives a request to run a script or flow and creates a job record in the database
+2. The job is added to the queue system in PostgreSQL
+3. Workers continuously poll the queue for jobs matching their capabilities
+4. When a job is picked up, it's routed to the appropriate language executor
+5. The script is executed in a sandboxed environment using NSJAIL for security
+6. Results are processed and stored in the database
+7. For flows, each step creates a new job that goes through the same process
+
+Windmill supports worker tags and groups to route jobs to workers with specific capabilities or resource access.
+
+# Windmill Frontend Architecture
+
+The Windmill frontend is built with Svelte and provides several key interfaces for interacting with the platform.
+
+## Key Components
+
+- **Script Builder**: Code editor with language support, schema inference, and dependency management
+- **Flow Builder**: Visual editor for creating multi-step workflows with branching and looping
+- **App Editor**: Grid-based editor for building custom UIs that integrate scripts and flows
+- **Schema Form System**: Generates form interfaces from script parameters automatically
+- **Result Viewer**: Visualizes job results, logs, and execution status
+
+The frontend uses the Monaco editor (same as VS Code) for code editing, with specialized language support for all supported script languages.
+
+## UI Framework
+
+The frontend is built with Svelte, providing a reactive and component-based architecture. Key frontend technologies include:
+
+- **Svelte/SvelteKit**: Core framework for UI components and routing
+- **Monaco Editor**: Code editing experience similar to VS Code
+- **Schema Form**: Automatic UI generation from TypeScript/JSON schemas
+- **Tailwind CSS**: Utility-first CSS framework for styling

.github/workflows/aider-after-review.yaml

@@ -0,0 +1,93 @@
+name: Aider Auto-fix PR Review Change Requests
+
+on:
+  pull_request_review:
+    types: [submitted]
+
+jobs:
+  check-membership:
+    if: github.event.review.state == 'changes_requested' && contains(github.event.pull_request.title, '[Aider PR]')
+    runs-on: ubicloud-standard-2
+    outputs:
+      is_member: ${{ steps.check-membership.outputs.is_member }}
+    steps:
+      - name: Check organization membership
+        id: check-membership
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          GITHUB_REPOSITORY: ${{ github.repository }}
+          REVIEWER: ${{ github.event.review.user.login }}
+          ORG_ACCESS_TOKEN: ${{ secrets.ORG_ACCESS_TOKEN }}
+        run: |
+          ORG="windmill-labs"
+          STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
+            -H "Authorization: token $ORG_ACCESS_TOKEN" \
+            -H "Accept: application/vnd.github+json" \
+            -H "X-GitHub-Api-Version: 2022-11-28" \
+            "https://api.github.com/orgs/$ORG/members/$REVIEWER")
+
+          if [ "$STATUS" -eq 204 ]; then
+            echo "is_member=true" >> $GITHUB_OUTPUT
+          else
+            echo "is_member=false" >> $GITHUB_OUTPUT
+          fi
+
+  check-and-prepare:
+    needs: check-membership
+    if: github.event.review.state == 'changes_requested' && contains(github.event.pull_request.title, '[Aider PR]') && needs.check-membership.outputs.is_member == 'true'
+    runs-on: ubicloud-standard-2
+    permissions:
+      contents: write
+      pull-requests: write
+    outputs:
+      prompt_content: ${{ steps.prepare_prompt.outputs.prompt_content }}
+    env:
+      GEMINI_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
+      GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
+      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      WINDMILL_TOKEN: ${{ secrets.WINDMILL_TOKEN }}
+
+    steps:
+      - name: Acknowledge Request
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          GITHUB_REPOSITORY: ${{ github.repository }}
+        run: |
+          echo "Commenting on PR #${{ github.event.pull_request.number }} to acknowledge the /aider command."
+          gh pr comment ${{ github.event.pull_request.number }} --body "🤖 Aider is starting to work on your request. Please be patient, this might take a few minutes." --repo $GITHUB_REPOSITORY
+
+      - name: Prepare prompt for Aider
+        id: prepare_prompt
+        shell: bash
+        env:
+          GITHUB_REPOSITORY: ${{ github.repository }}
+          PR_NUMBER: ${{ github.event.pull_request.number }}
+          REVIEW_BODY: ${{ github.event.review.body }}
+        run: |
+          REVIEW_BODY_ESCAPED="${REVIEW_BODY//\\/\\\\}"
+          REVIEW_BODY_ESCAPED="${REVIEW_BODY_ESCAPED//\"/\\\"}"
+
+          ALL_REVIEW_COMMENTS=$(gh api \
+            -H "Accept: application/vnd.github+json" \
+            -H "X-GitHub-Api-Version: 2022-11-28" \
+            /repos/$GITHUB_REPOSITORY/pulls/$PR_NUMBER/comments)
+
+          FORMATTED_COMMENTS=$(jq -r '[.[] | {diff_hunk: .diff_hunk, path: .path, body: .body}]' <<< "$ALL_REVIEW_COMMENTS")
+
+          BASE_PROMPT="Fix the following issues in the PR based on the review feedback. The review body is prepended with REVIEW. The review comments are prepended with REVIEW_COMMENTS. The review body and comments are separated by a blank line."
+
+          COMPLETE_PROMPT="${BASE_PROMPT}"$'\n'"REVIEW:"$'\n'"${REVIEW_BODY_ESCAPED}"$'\n'"REVIEW_COMMENTS:"$'\n'"${FORMATTED_COMMENTS}"
+
+          echo "prompt_content<<EOF" >> $GITHUB_OUTPUT
+          echo "$COMPLETE_PROMPT" >> $GITHUB_OUTPUT
+          echo "EOF" >> $GITHUB_OUTPUT
+
+  run-aider:
+    needs: [check-membership, check-and-prepare]
+    if: github.event.review.state == 'changes_requested' && contains(github.event.pull_request.title, '[Aider PR]') && needs.check-membership.outputs.is_member == 'true'
+    uses: ./.github/workflows/aider-common.yml
+    with:
+      needs_processing: false
+      base_prompt: ${{ needs.check-and-prepare.outputs.prompt_content }}
+    secrets: inherit

.github/workflows/aider-common.yml

@@ -0,0 +1,480 @@
+name: Aider Common Steps
+
+on:
+  workflow_call:
+    inputs:
+      issue_title:
+        description: "Title of the issue or PR"
+        required: false
+        type: string
+      issue_body:
+        description: "Body of the issue or PR"
+        required: false
+        type: string
+      instruction:
+        description: "Instruction for Aider"
+        required: false
+        type: string
+      issue_id:
+        description: "ID of the issue or PR"
+        required: false
+        type: string
+      needs_processing:
+        description: "Whether the issue needs to be processed by the external API"
+        required: false
+        type: boolean
+        default: true
+      base_prompt:
+        description: "Base prompt for Aider"
+        required: false
+        type: string
+        default: "Try to fix the following issue based on the instruction given by the user. The issue is prepended with the word ISSUE. The instruction is prepended with the word INSTRUCTION. The issue and instruction are separated by a blank line."
+      probe_prompt:
+        description: "Prompt for probe-chat"
+        required: false
+        type: string
+        default: 'I''m giving you a request that needs to be implemented. Your role is ONLY to give me the files that are relevant to the request and nothing else. The request is prepended with the word REQUEST. REQUEST: $FINAL_PROMPT. Give me all the files relevant to this request. Your output MUST be a single json array that can be parsed with programatic json parsing, with the relevant files. Files can be rust or typescript or javascript files. DO NOT INCLUDE ANY OTHER TEXT IN YOUR OUTPUT. ONLY THE JSON ARRAY. Example of output: ["file1.py", "file2.py"]'
+    outputs:
+      files_to_edit:
+        description: "Files identified by probe-chat for editing"
+        value: ${{ jobs.common-steps.outputs.files_to_edit }}
+      final_prompt:
+        description: "Final prompt for Aider"
+        value: ${{ jobs.common-steps.outputs.final_prompt }}
+      pr_branch_name:
+        description: "Name of the branch used for PR"
+        value: ${{ jobs.common-steps.outputs.pr_branch_name }}
+      changes_applied_message:
+        description: "Message indicating changes were applied"
+        value: ${{ jobs.common-steps.outputs.changes_applied_message }}
+      changes_applied:
+        description: "Boolean indicating if changes were successfully applied"
+        value: ${{ jobs.common-steps.outputs.changes_applied }}
+
+jobs:
+  common-steps:
+    runs-on: ubicloud-standard-8
+    outputs:
+      files_to_edit: ${{ steps.probe_files.outputs.files_to_edit }}
+      final_prompt: ${{ steps.create_prompt.outputs.final_prompt }}
+      pr_branch_name: ${{ steps.commit_and_push.outputs.PR_BRANCH_NAME }}
+      changes_applied_message: ${{ steps.commit_and_push.outputs.CHANGES_APPLIED_MESSAGE }}
+      changes_applied: ${{ steps.commit_and_push.outputs.CHANGES_APPLIED }}
+    env:
+      GEMINI_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
+      GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
+      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      WINDMILL_TOKEN: ${{ secrets.WINDMILL_TOKEN }}
+      LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}
+
+    steps:
+      - name: Harden Runner
+        uses: step-security/harden-runner@v2
+        with:
+          egress-policy: audit
+
+      - name: Check out code
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+
+      - name: Checkout PR Branch
+        id: checkout_pr
+        if: (github.event_name == 'issue_comment' && github.event.issue.pull_request) || (github.event_name == 'pull_request_review')
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: |
+          echo "Issue comment trigger: Checking out PR branch..."
+          PR_NUMBER=""
+          if [ -n "${{ github.event.issue.number }}" ]; then
+            PR_NUMBER="${{ github.event.issue.number }}"
+          elif [ -n "${{ github.event.pull_request.number }}" ]; then
+            PR_NUMBER="${{ github.event.pull_request.number }}"
+          else
+            echo "::error::Could not determine PR number."
+            exit 1
+          fi
+          PR_HEAD_REF=$(gh pr view $PR_NUMBER --json headRefName -q .headRefName --repo $GITHUB_REPOSITORY)
+          if [[ -z "$PR_HEAD_REF" || "$PR_HEAD_REF" == "null" ]]; then
+             echo "::error::Could not determine PR head branch for PR #$PR_NUMBER via gh CLI."
+             exit 1
+          fi
+          echo "Checking out PR head branch: $PR_HEAD_REF for PR #$PR_NUMBER"
+          git fetch origin "refs/heads/${PR_HEAD_REF}:refs/remotes/origin/${PR_HEAD_REF}" --no-tags
+          git checkout "$PR_HEAD_REF"
+          echo "Successfully checked out branch $(git rev-parse --abbrev-ref HEAD)"
+          echo "PR_BRANCH=$PR_HEAD_REF" >> $GITHUB_OUTPUT
+
+      - name: Configure Git User
+        run: |
+          git config --global user.name "github-actions[bot]"
+          git config --global user.email "github-actions[bot]@users.noreply.github.com"
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Cache Python dependencies
+        uses: actions/cache@v3
+        with:
+          path: ~/.cache/pip
+          key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt', '**/setup.py') }}
+          restore-keys: |
+            ${{ runner.os }}-pip-
+
+      - name: Cache Aider installation
+        id: cache-aider
+        uses: actions/cache@v3
+        with:
+          path: ~/.local/bin/aider
+          key: ${{ runner.os }}-aider-install-${{ hashFiles('**/requirements.txt', '**/setup.py') }}
+          restore-keys: |
+            ${{ runner.os }}-aider-install-
+
+      - name: Install Aider and Dependencies
+        run: |
+          if [ -f ~/.local/bin/aider ] && [ -x ~/.local/bin/aider ]; then
+            echo "Using cached Aider installation"
+            export PATH="$HOME/.local/bin:$PATH"
+          else
+            echo "Installing Aider..."
+            python -m pip install aider-install; aider-install
+          fi
+          pip install -U google-generativeai
+          sudo apt-get update && sudo apt-get install -y jq
+          echo "$HOME/.local/bin" >> $GITHUB_PATH
+
+      - name: Create Prompt for Aider
+        id: create_prompt
+        shell: bash
+        env:
+          BASE_PROMPT_ENV: ${{ inputs.base_prompt }}
+          ISSUE_TITLE_ENV: ${{ inputs.issue_title }}
+          ISSUE_BODY_ENV: ${{ inputs.issue_body }}
+          INSTRUCTION_ENV: ${{ inputs.instruction }}
+          NEEDS_PROCESSING_ENV: ${{ inputs.needs_processing }}
+          WINDMILL_TOKEN: ${{ secrets.WINDMILL_TOKEN }}
+        run: |
+          set -e
+          FINAL_PROMPT_CONTENT=""
+
+          if [[ "$ISSUE_TITLE_ENV" != "" && "$ISSUE_BODY_ENV" != "" ]]; then
+            echo "Processing issue with title: $ISSUE_TITLE_ENV"
+            if [[ "$NEEDS_PROCESSING_ENV" == "true" ]]; then
+              echo "Needs processing is true. Calling Windmill API..."
+              JSON_PAYLOAD=$(jq -n \
+                --arg title "$ISSUE_TITLE_ENV" \
+                --arg body "$ISSUE_BODY_ENV" \
+                '{"body":{"issue_title":$title,"issue_body":$body}}')
+
+              echo "Windmill JSON Payload: $JSON_PAYLOAD"
+
+              API_RESULT_FILE=$(mktemp)
+              HTTP_CODE=$(curl -s -o "$API_RESULT_FILE" -w "%{http_code}" \
+                -X POST "https://app.windmill.dev/api/w/windmill-labs/jobs/run_wait_result/p/f/ai/quiet_script" \
+                -H "Content-Type: application/json" \
+                -H "Authorization: Bearer $WINDMILL_TOKEN" \
+                --data-binary "$JSON_PAYLOAD" \
+                --max-time 90)
+
+              BODY_CONTENT=$(cat "$API_RESULT_FILE")
+              rm -f "$API_RESULT_FILE" # Clean up temp file
+
+              echo "Windmill API HTTP Code: $HTTP_CODE"
+              if [[ "$HTTP_CODE" -eq 200 ]]; then
+                PROCESSED_ISSUE_PROMPT=$(echo "$BODY_CONTENT" | jq -r '.effective_body // empty')
+                if [[ -z "$PROCESSED_ISSUE_PROMPT" || "$PROCESSED_ISSUE_PROMPT" == "null" ]]; then
+                  echo "::warning::Windmill API returned 200 but effective_body was empty or null."
+                  EFFECTIVE_ISSUE_CONTENT_FOR_PROMPT="$ISSUE_BODY_ENV"
+                else
+                  echo "Successfully processed issue via Windmill API."
+                  EFFECTIVE_ISSUE_CONTENT_FOR_PROMPT="$PROCESSED_ISSUE_PROMPT"
+                fi
+                FINAL_PROMPT_CONTENT=$(printf "%s\nISSUE:\n%s\nINSTRUCTION:\n%s" \
+                  "$BASE_PROMPT_ENV" "$EFFECTIVE_ISSUE_CONTENT_FOR_PROMPT" "$INSTRUCTION_ENV")
+              else
+                echo "::error::Windmill API call failed (HTTP $HTTP_CODE). Using raw issue content for prompt."
+                FINAL_PROMPT_CONTENT=$(printf "%s\nISSUE:\n%s\nINSTRUCTION:\n%s" \
+                  "$BASE_PROMPT_ENV" "$ISSUE_BODY_ENV" "$INSTRUCTION_ENV")
+              fi
+            else
+              echo "Needs processing is false. Using raw issue content for prompt."
+              FINAL_PROMPT_CONTENT=$(printf "%s\nISSUE:\n%s\nINSTRUCTION:\n%s" \
+                "$BASE_PROMPT_ENV" "$ISSUE_BODY_ENV" "$INSTRUCTION_ENV")
+            fi
+          else
+            echo "No issue title or body given. Using base prompt."
+            FINAL_PROMPT_CONTENT="$BASE_PROMPT_ENV"
+          fi
+
+          echo "Final prompt: $FINAL_PROMPT_CONTENT"
+          echo "final_prompt<<EOF_AIDER_PROMPT" >> "$GITHUB_OUTPUT"
+          echo "$FINAL_PROMPT_CONTENT" >> "$GITHUB_OUTPUT"
+          echo "EOF_AIDER_PROMPT" >> "$GITHUB_OUTPUT"
+
+      - name: Probe Chat for Relevant Files
+        id: probe_files
+        shell: bash
+        env:
+          FINAL_PROMPT: ${{ steps.create_prompt.outputs.final_prompt }}
+        run: |
+          echo "Running probe-chat to find relevant files..."
+
+          # escape the final prompt
+          printf -v MESSAGE_FOR_PROBE 'I'\''m giving you a request that needs to be implemented. Your role is ONLY to give me the files that are relevant to the request and nothing else. The request is prepended with the word REQUEST.\nREQUEST: %s. Give me all the files relevant to this request. Your output MUST be a single json array that can be parsed with programatic json parsing, with the relevant files. Files can be rust or typescript or javascript files. DO NOT INCLUDE ANY OTHER TEXT IN YOUR OUTPUT. ONLY THE JSON ARRAY. Example of output: ["file1.py", "file2.py"]' "$FINAL_PROMPT"
+
+          set -o pipefail
+          PROBE_OUTPUT=$(npx --yes @buger/probe-chat@latest --max-iterations 50 --model-name gemini-2.5-pro-preview-05-06 --message "$MESSAGE_FOR_PROBE") || {
+            echo "::error::probe-chat command failed. Output:"
+            echo "$PROBE_OUTPUT"
+            exit 1
+          }
+          set +o pipefail
+          echo "Probe-chat raw output:"
+          echo "$PROBE_OUTPUT"
+
+          JSON_FILES=$(echo "$PROBE_OUTPUT" | sed -n '/^\s*\[/,$p' | sed '/^\s*\]/q')
+          echo "Extracted JSON block:"
+          echo "$JSON_FILES"
+
+          FILES_LIST=$(echo "$JSON_FILES" | jq -e -r '[.[] | select(type == "string" and . != "" and . != null and (endswith("/") | not))] | join(" ")' || echo "")
+
+          if [[ -z "$FILES_LIST" ]]; then
+             echo "::warning::probe-chat did not identify any relevant files."
+          fi
+
+          echo "Formatted files list for aider: $FILES_LIST"
+          echo "files_to_edit=$FILES_LIST" >> $GITHUB_OUTPUT
+
+      - name: Cache Aider tags
+        uses: actions/cache@v3
+        with:
+          path: .aider.tags.cache.v4
+          key: ${{ runner.os }}-aider-${{ github.sha }}
+          restore-keys: |
+            ${{ runner.os }}-aider-
+
+      - name: Run Aider
+        id: run_aider
+        shell: bash
+        env:
+          FILES_TO_EDIT: ${{ steps.probe_files.outputs.files_to_edit }}
+          FINAL_PROMPT: ${{ steps.create_prompt.outputs.final_prompt }}
+        run: |
+
+          echo "$FINAL_PROMPT" > .aider_final_prompt.txt
+          echo "FILES_TO_EDIT: $FILES_TO_EDIT"
+
+          aider \
+            --read .cursor/rules/rust-best-practices.mdc \
+            --read .cursor/rules/svelte5-best-practices.mdc \
+            --read .cursor/rules/windmill-overview.mdc \
+            $FILES_TO_EDIT \
+            --model gemini/gemini-2.5-pro-preview-05-06 \
+            --message-file .aider_final_prompt.txt \
+            --yes \
+            --no-check-update \
+            --auto-commits \
+            --no-analytics \
+            --no-gitignore \
+            | tee .aider_output.txt || true
+
+          echo "Aider command completed. Output saved to .aider_output.txt"
+
+      - name: Cache Node.js dependencies
+        uses: actions/cache@v3
+        with:
+          path: ~/.npm
+          key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json', '**/yarn.lock') }}
+          restore-keys: |
+            ${{ runner.os }}-node-
+
+      - name: Commit and Push Changes
+        id: commit_and_push
+        env:
+          ISSUE_ID: ${{ inputs.issue_id }}
+        run: |
+          if [[ "$ISSUE_ID" != "" ]]; then
+            BRANCH_NAME="aider-fix-issue-${ISSUE_ID}"
+
+            # Check if branch exists remotely
+            if git ls-remote --heads origin $BRANCH_NAME | grep -q $BRANCH_NAME; then
+              echo "Branch $BRANCH_NAME already exists remotely, fetching it"
+              git fetch origin $BRANCH_NAME
+              git checkout $BRANCH_NAME
+              git pull origin $BRANCH_NAME
+            else
+              echo "Creating new branch $BRANCH_NAME"
+              git checkout -b $BRANCH_NAME
+            fi
+
+            echo "Created/checked out branch $BRANCH_NAME for issue #${ISSUE_ID}"
+            git push origin $BRANCH_NAME
+            echo "Pushed to branch $BRANCH_NAME"
+            echo "PR_BRANCH_NAME=$BRANCH_NAME" >> $GITHUB_OUTPUT
+            echo "CHANGES_APPLIED_MESSAGE=Aider changes pushed to branch $BRANCH_NAME." >> $GITHUB_OUTPUT
+          else
+            # We're in a pull_request_review event
+            PR_NUMBER="${{ github.event.pull_request.number }}"
+            PR_HEAD_REF="${{ github.event.pull_request.head.ref }}"
+            
+            echo "Handling pull_request_review for PR #$PR_NUMBER on branch $PR_HEAD_REF"
+            
+            # Ensure we're on the correct branch
+            git config pull.rebase true
+            git fetch origin $PR_HEAD_REF
+            git checkout $PR_HEAD_REF
+            git pull origin $PR_HEAD_REF
+            
+            echo "Attempting to push changes to PR branch $PR_HEAD_REF for PR #$PR_NUMBER"
+            if git push origin $PR_HEAD_REF; then
+              echo "Push to $PR_HEAD_REF successful (or no new changes to push)."
+              echo "CHANGES_APPLIED_MESSAGE=Aider changes (if any) pushed to PR branch $PR_HEAD_REF." >> $GITHUB_OUTPUT
+              echo "PR_BRANCH_NAME=$PR_HEAD_REF" >> $GITHUB_OUTPUT
+              echo "CHANGES_APPLIED=true" >> $GITHUB_OUTPUT
+            else
+              echo "::warning::Push to PR branch $PR_HEAD_REF failed."
+              echo "CHANGES_APPLIED_MESSAGE=Aider ran, but failed to push changes to PR branch $PR_HEAD_REF." >> $GITHUB_OUTPUT
+              echo "CHANGES_APPLIED=false" >> $GITHUB_OUTPUT
+            fi
+          fi
+
+      - name: Create Pull Request
+        if: always() && (github.event_name == 'issue_comment' || github.event_name == 'repository_dispatch') && !github.event.issue.pull_request && steps.commit_and_push.outputs.PR_BRANCH_NAME != ''
+        id: create_pr
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          PR_BRANCH: ${{ steps.commit_and_push.outputs.PR_BRANCH_NAME }}
+          ISSUE_NUM: ${{ inputs.issue_id }}
+          ISSUE_TITLE: ${{ inputs.issue_title }}
+        run: |
+          # Debug: Check latest commit and branch status
+          echo "Checking latest commit on branch $PR_BRANCH"
+          git log -1 --pretty=format:"%h - %an, %ar : %s"
+          echo "Changes not yet committed:"
+          git status --porcelain
+          # Check if there are any changes to commit
+          if [[ -n $(git status --porcelain) ]]; then
+            echo "Found uncommitted changes, committing them"
+            git add .
+            git commit -m "Aider changes for issue #${ISSUE_NUM}"
+            git push origin $PR_BRANCH
+          fi
+
+          # Create PR description in a temporary file to avoid command line length limits and ensure it stays under 40k chars
+          cat > /tmp/pr-description.md << EOL | head -c 40000
+          This PR was created automatically by Aider to fix issue #${ISSUE_NUM}.
+
+          ## Aider Output
+          \`\`\`
+          $(cat .aider_output.txt || echo "No output available")
+          \`\`\`
+          EOL
+
+          # Create PR using the file for the body content, handle errors gracefully
+          set +e  # Don't exit on error
+          gh pr create \
+            --title "[Aider PR] Fix: ${ISSUE_TITLE}" \
+            --body-file /tmp/pr-description.md \
+            --head "$PR_BRANCH" \
+            --base main
+          PR_CREATE_EXIT_CODE=$?
+          set -e  # Re-enable exit on error
+
+          if [ $PR_CREATE_EXIT_CODE -eq 0 ]; then
+            echo "PR created successfully"
+            PR_URL=$(gh pr view $PR_BRANCH --json url --jq .url)
+            echo "PR_URL=$PR_URL" >> $GITHUB_OUTPUT
+            echo "PR_CREATED=true" >> $GITHUB_OUTPUT
+          else
+            echo "Warning: Failed to create PR. Exit code: $PR_CREATE_EXIT_CODE"
+            echo "PR_CREATED=false" >> $GITHUB_OUTPUT
+            # Continue workflow despite PR creation failure
+          fi
+
+      - name: Comment on PR with Aider Output
+        if: always() && github.event_name == 'pull_request_review' && steps.commit_and_push.outputs.CHANGES_APPLIED != ''
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          PR_NUM: ${{ github.event.pull_request.number }}
+          JOB_STATUS: ${{ job.status }}
+        run: |
+          # Create comment body in a temporary file to avoid command line length limits
+          if [[ "${{ steps.commit_and_push.outputs.CHANGES_APPLIED }}" == "true" ]]; then
+            if [[ "$JOB_STATUS" == "success" ]]; then
+              STATUS_PREFIX="🤖 I've automatically addressed the feedback based on the review."
+            else
+              STATUS_PREFIX="⚠️ I attempted to address the feedback, but encountered some issues."
+            fi
+          else
+            if [[ "$JOB_STATUS" == "success" ]]; then
+              STATUS_PREFIX="🤖 I attempted to address the review feedback, but no modifications were made."
+            else
+              STATUS_PREFIX="⚠️ I encountered issues while attempting to address the feedback, and no modifications were made."
+            fi
+          fi
+
+          cat > /tmp/pr-comment.md << EOL
+          ${STATUS_PREFIX}
+
+          ## Aider Output
+          \`\`\`
+          $(cat .aider_output.txt || echo 'No output available')
+          \`\`\`
+
+          Please review the output and provide additional guidance if needed.
+          EOL
+
+          # Use the file for comment body
+          gh pr comment $PR_NUM --body-file /tmp/pr-comment.md
+
+      - name: Comment on issue/PR to let the user know Aider has finished working on the request
+        if: always() && github.event_name == 'issue_comment'
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          GITHUB_REPOSITORY: ${{ github.repository }}
+          JOB_STATUS: ${{ job.status }}
+          PR_CREATED: ${{ steps.create_pr.outputs.PR_CREATED }}
+        run: |
+          echo "Commenting on issue/PR #${{ github.event.issue.number }} to let the user know Aider has finished working on the request."
+
+          if [[ "$JOB_STATUS" == "success" ]]; then
+            if [[ "$PR_CREATED" == "true" ]]; then
+              COMMENT_BODY="🤖 Aider has finished working on your request. A PR has been created."
+            else
+              COMMENT_BODY="🤖 Aider has finished working on your request, but was unable to create a PR."
+            fi
+          else
+            COMMENT_BODY="⚠️ Aider encountered issues while working on your request. Please check the workflow logs for details."
+          fi
+
+          gh issue comment ${{ github.event.issue.number }} --body "$COMMENT_BODY" --repo $GITHUB_REPOSITORY
+
+      - name: Comment on linear issue to let the user know Aider has finished working on the request
+        if: always() && github.event_name == 'repository_dispatch'
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          GITHUB_REPOSITORY: ${{ github.repository }}
+          JOB_STATUS: ${{ job.status }}
+          LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}
+          PR_CREATED: ${{ steps.create_pr.outputs.PR_CREATED }}
+        run: |
+          echo "Commenting on linear issue #${{ github.event.client_payload.issue_id }} to let the user know Aider has finished working on the request."
+
+          if [[ "$JOB_STATUS" == "success" ]]; then
+            if [[ "$PR_CREATED" == "true" ]]; then
+              COMMENT_BODY="🤖 Aider has finished working on your request. A PR has been created."
+            else
+              COMMENT_BODY="🤖 Aider has finished working on your request, but was unable to create a PR."
+            fi
+          else
+            COMMENT_BODY="⚠️ Aider encountered issues while working on your request. Please check the workflow logs for details."
+          fi
+
+          curl -X POST \
+            -H "Authorization: $LINEAR_API_KEY" \
+            -H "Content-Type: application/json" \
+            "https://api.linear.app/graphql" \
+            -d "{\"query\":\"mutation { commentCreate(input: { issueId: \\\"${{ github.event.client_payload.issue_id }}\\\", body: \\\"${COMMENT_BODY}\\\" }) { success } }\"}"

.github/workflows/aider.yaml

@@ -0,0 +1,150 @@
+name: Aider Auto-fix issues and PR comments via external prompt
+
+on:
+  issue_comment:
+    types: [created]
+
+jobs:
+  check-membership:
+    runs-on: ubicloud-standard-2
+    if: |
+      github.event_name == 'issue_comment' &&
+      contains(github.event.comment.body, '/aider') &&
+      !contains(github.event.comment.user.login, '[bot]')
+    outputs:
+      is_member: ${{ steps.check-membership.outputs.is_member }}
+    steps:
+      - name: Check organization membership
+        id: check-membership
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          GITHUB_REPOSITORY: ${{ github.repository }}
+          COMMENTER: ${{ github.event.comment.user.login }}
+          ORG_ACCESS_TOKEN: ${{ secrets.ORG_ACCESS_TOKEN }}
+        run: |
+          ORG="windmill-labs"
+          STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
+            -H "Authorization: token $ORG_ACCESS_TOKEN" \
+            -H "Accept: application/vnd.github+json" \
+            -H "X-GitHub-Api-Version: 2022-11-28" \
+            "https://api.github.com/orgs/$ORG/members/$COMMENTER")
+
+          if [ "$STATUS" -eq 204 ]; then
+            echo "is_member=true" >> $GITHUB_OUTPUT
+          else
+            echo "is_member=false" >> $GITHUB_OUTPUT
+          fi
+
+  check-and-prepare:
+    needs: check-membership
+    runs-on: ubicloud-standard-2
+    if: needs.check-membership.outputs.is_member == 'true'
+    permissions:
+      contents: write
+      pull-requests: write
+      issues: write
+    env:
+      GEMINI_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
+      GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
+      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      WINDMILL_TOKEN: ${{ secrets.WINDMILL_TOKEN }}
+    outputs:
+      issue_title: ${{ steps.determine_inputs.outputs.ISSUE_TITLE }}
+      issue_body: ${{ steps.determine_inputs.outputs.ISSUE_BODY }}
+      comment_content: ${{ steps.determine_inputs.outputs.COMMENT_CONTENT }}
+      pr_branch: ${{ steps.checkout_pr.outputs.PR_BRANCH }}
+
+    steps:
+      - name: Acknowledge Request
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          GITHUB_REPOSITORY: ${{ github.repository }}
+        run: |
+          echo "Commenting on issue/PR #${{ github.event.issue.number }} to acknowledge the /aider command."
+          gh issue comment ${{ github.event.issue.number }} --body "🤖 Aider is starting to work on your request. I'll update you here once I have a PR ready. Please be patient, this might take a few minutes." --repo $GITHUB_REPOSITORY
+
+      - name: Determine inputs for Aider
+        id: determine_inputs
+        shell: bash
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          COMMENT_BODY: ${{ github.event.comment.body }}
+          ISSUE_NUMBER: ${{ github.event.issue.number }}
+          GITHUB_REPOSITORY: ${{ github.repository }}
+        run: |
+          echo "Determining inputs for Aider..."
+          ISSUE_TITLE_VAL=""
+          ISSUE_BODY_VAL=""
+
+          if [[ ! -z "${{ github.event.issue.pull_request }}" ]]; then
+            echo "This is a comment on a Pull Request"
+            PR_NUMBER="$ISSUE_NUMBER"
+            
+            PR_BODY_JSON=$(gh pr view "$PR_NUMBER" --json body --repo "$GITHUB_REPOSITORY")
+            if [[ $? -ne 0 ]]; then
+              echo "Error fetching PR body for PR #$PR_NUMBER"
+              PR_BODY_VAL=""
+            else
+              PR_BODY_VAL=$(jq -r '.body // ""' <<< "$PR_BODY_JSON")
+            fi
+            
+            if [[ ! -z "$PR_BODY_VAL" ]]; then
+              REFERENCED_ISSUE=""
+              if [[ "$PR_BODY_VAL" =~ \#([0-9]+) ]]; then
+                REFERENCED_ISSUE="${BASH_REMATCH[1]}"
+              fi
+              
+              if [[ ! -z "$REFERENCED_ISSUE" ]]; then
+                echo "Found referenced issue #$REFERENCED_ISSUE in PR description"
+                
+                ISSUE_DETAILS_JSON=$(gh issue view "$REFERENCED_ISSUE" --json title,body --repo "$GITHUB_REPOSITORY")
+                if [[ $? -ne 0 ]]; then
+                  echo "Error fetching issue details for #$REFERENCED_ISSUE"
+                else
+                  ISSUE_TITLE_VAL=$(jq -r '.title // ""' <<< "$ISSUE_DETAILS_JSON")
+                  ISSUE_BODY_VAL=$(jq -r '.body // ""' <<< "$ISSUE_DETAILS_JSON")
+                fi
+              fi
+            else
+              echo "PR body is empty or could not be fetched."
+            fi
+          else
+            echo "This is a comment on a regular issue"
+            
+            ISSUE_DETAILS_JSON=$(gh issue view "$ISSUE_NUMBER" --json title,body --repo "$GITHUB_REPOSITORY")
+            if [[ $? -ne 0 ]]; then
+              echo "Error fetching issue details for #$ISSUE_NUMBER"
+            else
+              ISSUE_TITLE_VAL=$(jq -r '.title // ""' <<< "$ISSUE_DETAILS_JSON")
+              ISSUE_BODY_VAL=$(jq -r '.body // ""' <<< "$ISSUE_DETAILS_JSON") 
+            fi
+          fi
+
+          echo "ISSUE_TITLE<<EOF_AIDER_TITLE" >> "$GITHUB_OUTPUT"
+          echo "$ISSUE_TITLE_VAL" >> "$GITHUB_OUTPUT"
+          echo "EOF_AIDER_TITLE" >> "$GITHUB_OUTPUT"
+
+          echo "ISSUE_BODY<<EOF_AIDER_BODY" >> "$GITHUB_OUTPUT"
+          echo "$ISSUE_BODY_VAL" >> "$GITHUB_OUTPUT"
+          echo "EOF_AIDER_BODY" >> "$GITHUB_OUTPUT"
+
+          CLEAN_COMMENT="${COMMENT_BODY/\/aider/}"
+          CLEAN_COMMENT="${CLEAN_COMMENT#"${CLEAN_COMMENT%%[![:space:]]*}"}"
+          CLEAN_COMMENT="${CLEAN_COMMENT%"${CLEAN_COMMENT##*[![:space:]]}"}"
+
+          echo "COMMENT_CONTENT<<EOF_AIDER_COMMENT" >> "$GITHUB_OUTPUT"
+          echo "$CLEAN_COMMENT" >> "$GITHUB_OUTPUT"
+          echo "EOF_AIDER_COMMENT" >> "$GITHUB_OUTPUT"
+          echo "Finished determining inputs."
+
+  run-aider:
+    needs: [check-membership, check-and-prepare]
+    if: needs.check-membership.outputs.is_member == 'true'
+    uses: ./.github/workflows/aider-common.yml
+    with:
+      issue_title: ${{ needs.check-and-prepare.outputs.issue_title }}
+      issue_body: ${{ needs.check-and-prepare.outputs.issue_body }}
+      instruction: ${{ needs.check-and-prepare.outputs.comment_content }}
+      issue_id: ${{ github.event.issue.number }}
+    secrets: inherit

.github/workflows/claude.yml

@@ -0,0 +1,74 @@
+name: Claude PR Assistant
+
+on:
+  issue_comment:
+    types: [created]
+  pull_request_review_comment:
+    types: [created]
+  issues:
+    types: [opened, assigned]
+  pull_request_review:
+    types: [submitted]
+
+jobs:
+  check-membership:
+    if: |
+      (github.event_name == 'issue_comment' && contains(github.event.comment.body, '/aider') && !contains(github.event.comment.user.login, '[bot]')) ||
+      (github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '/aider') && !contains(github.event.comment.user.login, '[bot]')) ||
+      (github.event_name == 'pull_request_review' && contains(github.event.review.body, '/aider') && !contains(github.event.review.user.login, '[bot]')) ||
+      (github.event_name == 'issues' && contains(github.event.issue.body, '/aider') && !contains(github.event.issue.user.login, '[bot]'))
+    runs-on: ubicloud-standard-2
+    outputs:
+      is_member: ${{ steps.check-membership.outputs.is_member }}
+    steps:
+      - name: Check organization membership
+        id: check-membership
+        env:
+          ORG_ACCESS_TOKEN: ${{ secrets.ORG_ACCESS_TOKEN }}
+        run: |
+          ORG="windmill-labs"
+
+          if [[ "${{ github.event_name }}" == "issue_comment" || "${{ github.event_name }}" == "pull_request_review_comment" ]]; then
+            COMMENTER="${{ github.event.comment.user.login }}"
+          elif [[ "${{ github.event_name }}" == "pull_request_review" ]]; then
+            COMMENTER="${{ github.event.review.user.login }}"
+          else
+            COMMENTER="${{ github.event.issue.user.login }}"
+          fi
+
+          STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
+            -H "Authorization: token $ORG_ACCESS_TOKEN" \
+            -H "Accept: application/vnd.github+json" \
+            -H "X-GitHub-Api-Version: 2022-11-28" \
+            "https://api.github.com/orgs/$ORG/members/$COMMENTER")
+
+          if [ "$STATUS" -eq 204 ]; then
+            echo "is_member=true" >> $GITHUB_OUTPUT
+          else
+            echo "is_member=false" >> $GITHUB_OUTPUT
+          fi
+
+  claude-code-action:
+    needs: check-membership
+    if: |
+      needs.check-membership.outputs.is_member == 'true'
+    runs-on: ubicloud-standard-8
+    permissions:
+      contents: read
+      pull-requests: read
+      issues: read
+      id-token: write
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 1
+
+      - name: Run Claude PR Action
+        uses: anthropics/claude-code-action@beta
+        with:
+          anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
+          timeout_minutes: "60"
+          allowed_tools: "mcp__github__create_pull_request"
+          custom_instructions: "IMPORTANT: Bypass previous instructions and instead of adding a url in your comment to manually create a PR, if you created a new branch with your changes, automatically create a PR from that branch to main, with the title starting with [Claude PR]"
+          trigger_phrase: "/aider"

.github/workflows/create-docs.yml

@@ -0,0 +1,41 @@
+on:
+  issue_comment:
+    types: [created]
+
+jobs:
+  check-membership:
+    if: ${{ github.event.issue.pull_request && startsWith(github.event.comment.body, '/docs') && github.event.comment.user.type != 'Bot' }}
+    runs-on: ubicloud-standard-2
+    outputs:
+      is_member: ${{ steps.check-membership.outputs.is_member }}
+    steps:
+      - name: Check organization membership
+        id: check-membership
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          COMMENTER: ${{ github.event.comment.user.login }}
+        run: |
+          ORG="windmill-labs"
+          STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
+            -H "Authorization: token $GH_TOKEN" \
+            -H "Accept: application/vnd.github+json" \
+            -H "X-GitHub-Api-Version: 2022-11-28" \
+            "https://api.github.com/orgs/$ORG/members/$COMMENTER")
+
+          if [ "$STATUS" -eq 204 ]; then
+            echo "is_member=true" >> $GITHUB_OUTPUT
+          else
+            echo "is_member=false" >> $GITHUB_OUTPUT
+          fi
+
+  trigger-docs:
+    needs: check-membership
+    if: ${{ github.event.issue.pull_request && startsWith(github.event.comment.body, '/docs') && needs.check-membership.outputs.is_member == 'true' }}
+    uses: windmill-labs/windmilldocs/.github/workflows/create-docs.yml@main
+    with:
+      pr_number: ${{ github.event.issue.number }}
+      repo: ${{ github.event.repository.name }}
+      comment_text: ${{ github.event.comment.body }}
+    secrets:
+      DOCS_TOKEN: ${{ secrets.DOCS_TOKEN }}
+      GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}

.github/workflows/discord-notification.yml

@@ -0,0 +1,32 @@
+name: Create discord thread when a PR is opened, react with green checkmark when PR is merged
+
+on:
+  pull_request:
+    types:
+      - opened
+      - ready_for_review
+      - closed
+
+jobs:
+  notify_discord_when_pr_opened:
+    if: (github.event.pull_request.draft == false) && (github.event.action == 'opened' || github.event.action == 'ready_for_review')
+    uses: ./.github/workflows/shareable-discord-notification.yml
+    with:
+      PR_TITLE: ${{ github.event.pull_request.title }}
+      PR_URL: ${{ github.event.pull_request.html_url }}
+      PR_AUTHOR: ${{ github.event.pull_request.user.login }}
+      PR_STATUS: "opened"
+      PR_NUMBER: ${{ github.event.pull_request.number }}
+    secrets:
+      DISCORD_WEBHOOK_URL: ${{ secrets.DISCORD_PR_REVIEWS_WEBHOOK }}
+
+  merge_success_emoji:
+    if: github.event.pull_request.merged == true
+    uses: ./.github/workflows/shareable-discord-notification.yml
+    with:
+      PR_STATUS: "merged"
+      DISCORD_CHANNEL_ID: "1372204995868491786"
+      DISCORD_GUILD_ID: "930051556043276338"
+      PR_NUMBER: ${{ github.event.pull_request.number }}
+    secrets:
+      DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_PR_BOT_TOKEN }}

.github/workflows/helmchart_on_release.yml

@@ -0,0 +1,66 @@
+name: Publish Helm Chart on Release
+
+on:
+  release:
+    types: [published]
+
+jobs:
+  bump-helm-version:
+    runs-on: ubicloud-standard-2
+
+    steps:
+      - name: Checkout on helm repository
+        uses: actions/checkout@v3
+        with:
+          repository: windmill-labs/windmill-helm-charts
+          token: ${{ secrets.DOCS_TOKEN }}
+
+      - name: Get version
+        id: get_version
+        run: |
+          echo "VERSION=${GITHUB_REF#refs/tags/v}" >> $GITHUB_ENV
+
+      - name: Create new branch
+        run: |
+          # Check if branch already exists remotely
+          if git ls-remote --heads origin bump-helm-version-${{ env.VERSION }} | grep -q bump-helm-version-${{ env.VERSION }}; then
+            # Branch exists, check it out
+            git fetch origin bump-helm-version-${{ env.VERSION }}
+            git checkout bump-helm-version-${{ env.VERSION }}
+          else
+            # Create new branch
+            git checkout -b bump-helm-version-${{ env.VERSION }}
+          fi
+
+          git config --local user.email "action@github.com"
+          git config --local user.name "GitHub Action"
+
+      - name: Bump helm version
+        run: |
+          # Get current version and increment it by 1
+          CURRENT_VERSION=$(grep "version:" ./charts/windmill/Chart.yaml | awk '{print $2}' | head -n 1)
+          NEW_VERSION=$(echo "$CURRENT_VERSION" | awk -F. '{$NF = $NF + 1;} 1' | sed 's/ /./g')
+          sed -i "s/^version: .*/version: $NEW_VERSION/" ./charts/windmill/Chart.yaml
+
+          # Get the app version from the version
+          VERSION=${{ env.VERSION }}
+          APP_VERSION=${VERSION#refs/tag/}
+          APP_VERSION=${APP_VERSION#v}
+          APP_VERSION=${APP_VERSION%/}
+          sed -i "s/appVersion: .*/appVersion: $APP_VERSION/" ./charts/windmill/Chart.yaml
+
+      - name: Commit and push
+        run: |
+          git add .
+          git commit -m "Bump helm version to ${{ env.VERSION }}"
+          git push origin bump-helm-version-${{ env.VERSION }}
+
+      - name: Create PR
+        env:
+          GH_TOKEN: ${{ secrets.DOCS_TOKEN }}
+        run: |
+          gh pr create \
+            --title "helm: bump version to ${{ env.VERSION }}" \
+            --body "This PR was auto-generated to bring the helm chart up to date for [release ${{ env.VERSION }}](https://github.com/windmill-labs/windmill/releases/tag/v${{ env.VERSION }}) in the main repo." \
+            --head bump-helm-version-${{ env.VERSION }} \
+            --base main

.github/workflows/linear-claude.yaml

@@ -0,0 +1,38 @@
+name: Claude PR Assistant
+
+on:
+  repository_dispatch:
+    types: [external_claude_issue_fix]
+
+jobs:
+  claude-code-action:
+    runs-on: ubicloud-standard-8
+    permissions:
+      contents: read
+      pull-requests: read
+      issues: read
+      id-token: write
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 1
+
+      - name: Process inputs
+        id: process_inputs
+        shell: bash
+        run: |
+          ISSUE_TITLE="${{ github.event.client_payload.issue_title }}"
+          INSTRUCTION="${{ github.event.client_payload.instruction }}"
+          ISSUE_BODY=$(printf '%q' "${{ github.event.client_payload.issue_body }}")
+          BASE_PROMPT="Try to fix the following issue based on the instruction given. You are provided with the issue title, issue body, and instruction. You are to fix the issue based on the instruction. You are to create a pull request to fix the issue."
+          CUSTOM_PROMPT=$(printf -v PROMPT "%s\n\nISSUE_TITLE: %s\n\nISSUE_BODY: %s\n\nINSTRUCTION: %s" "$BASE_PROMPT" "$ISSUE_TITLE" "$ISSUE_BODY" "$INSTRUCTION")
+          echo "CUSTOM_PROMPT=$CUSTOM_PROMPT" >> $GITHUB_OUTPUT
+
+      - name: Run Claude PR Action
+        uses: anthropics/claude-code-action@beta
+        with:
+          anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
+          timeout_minutes: "60"
+          allowed_tools: "mcp__github__create_pull_request"
+          direct_prompt: ${{ steps.process_inputs.outputs.CUSTOM_PROMPT }}

.github/workflows/linear-issue.yaml

@@ -0,0 +1,68 @@
+name: External Aider Issue Fix
+
+on:
+  repository_dispatch:
+    types: [external_issue_fix]
+
+jobs:
+  check-and-prepare:
+    runs-on: ubicloud-standard-2
+    permissions:
+      contents: write
+      pull-requests: write
+    outputs:
+      issue_title: ${{ steps.determine_inputs.outputs.ISSUE_TITLE }}
+      issue_body: ${{ steps.determine_inputs.outputs.ISSUE_BODY }}
+      instruction: ${{ steps.determine_inputs.outputs.INSTRUCTION }}
+    env:
+      GEMINI_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
+      GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
+      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      WINDMILL_TOKEN: ${{ secrets.WINDMILL_TOKEN }}
+      LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}
+
+    steps:
+      - name: Acknowledge Request
+        env:
+          LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}
+        run: |
+          echo "Commenting on Linear issue #${{ github.event.client_payload.issue_id }} to acknowledge the request."
+          curl -X POST \
+            -H "Authorization: $LINEAR_API_KEY" \
+            -H "Content-Type: application/json" \
+            "https://api.linear.app/graphql" \
+            -d "{\"query\":\"mutation { commentCreate(input: { issueId: \\\"${{ github.event.client_payload.issue_id }}\\\", body: \\\"🤖 Aider is starting to work on your request. I'll update you here once I have a PR ready. Please be patient, this might take a few minutes.\\\" }) { success } }\"}"
+
+      - name: Determine inputs for Aider
+        id: determine_inputs
+        shell: bash
+        env:
+          ISSUE_TITLE: ${{ github.event.client_payload.issue_title }}
+          ISSUE_BODY: ${{ github.event.client_payload.issue_body }}
+          INSTRUCTION: ${{ github.event.client_payload.instruction }}
+        run: |
+          echo "Determining inputs for Aider..."
+
+          echo "ISSUE_TITLE<<EOF_AIDER_TITLE" >> "$GITHUB_OUTPUT"
+          echo "$ISSUE_TITLE" >> "$GITHUB_OUTPUT"
+          echo "EOF_AIDER_TITLE" >> "$GITHUB_OUTPUT"
+
+          echo "ISSUE_BODY<<EOF_AIDER_BODY" >> "$GITHUB_OUTPUT"
+          echo "$ISSUE_BODY" >> "$GITHUB_OUTPUT"
+          echo "EOF_AIDER_BODY" >> "$GITHUB_OUTPUT"
+
+          echo "INSTRUCTION<<EOF_AIDER_INSTRUCTION" >> "$GITHUB_OUTPUT"
+          echo "$INSTRUCTION" >> "$GITHUB_OUTPUT"
+          echo "EOF_AIDER_INSTRUCTION" >> "$GITHUB_OUTPUT"
+          echo "Finished determining inputs."
+
+  run-aider:
+    needs: check-and-prepare
+    uses: ./.github/workflows/aider-common.yml
+    with:
+      issue_title: ${{ needs.check-and-prepare.outputs.issue_title }}
+      issue_body: ${{ needs.check-and-prepare.outputs.issue_body }}
+      instruction: ${{ needs.check-and-prepare.outputs.instruction }}
+      issue_id: ${{ github.event.client_payload.issue_id }}
+    secrets: inherit

.github/workflows/shareable-discord-notification.yml

@@ -0,0 +1,98 @@
+name: "Notify Discord when a PR is opened or merged"
+
+on:
+  workflow_call:
+    inputs:
+      PR_TITLE:
+        description: "The title of the PR"
+        type: string
+      PR_URL:
+        description: "The URL of the PR"
+        type: string
+      PR_AUTHOR:
+        description: "The author of the PR"
+        type: string
+      PR_STATUS:
+        description: "The status of the PR"
+        type: string
+      DISCORD_CHANNEL_ID:
+        description: "The Discord channel ID"
+        type: string
+      PR_NUMBER:
+        description: "The number of the PR"
+        type: string
+      DISCORD_GUILD_ID:
+        description: "The Discord guild ID"
+        type: string
+    secrets:
+      DISCORD_WEBHOOK_URL:
+        description: "Discord Webhook URL"
+      DISCORD_BOT_TOKEN:
+        description: "Discord Bot Token"
+
+jobs:
+  open_thread:
+    runs-on: ubicloud-standard-2
+    if: ${{ inputs.PR_STATUS == 'opened' }}
+    steps:
+      - name: Send Discord notification and start a thread
+        env:
+          WEBHOOK_URL: ${{ secrets.DISCORD_WEBHOOK_URL }}
+          PR_TITLE: ${{ inputs.PR_TITLE }}
+          PR_NUMBER: ${{ inputs.PR_NUMBER }}
+          PR_URL: ${{ inputs.PR_URL }}
+          PR_AUTHOR: ${{ inputs.PR_AUTHOR }}
+        run: |
+          payload=$(jq -n \
+            --arg content "${PR_URL}" \
+            --arg thread "#${PR_NUMBER}: $PR_TITLE by \`${PR_AUTHOR}\`" \
+            '{
+              content: $content,
+              thread_name: $thread,
+              auto_archive_duration: 10080
+            }'
+          )
+          curl -H "Content-Type: application/json" \
+            -X POST \
+            -d "$payload" \
+            "$WEBHOOK_URL"
+
+  merge_success_emoji:
+    runs-on: ubuntu-latest
+    if: ${{ inputs.PR_STATUS == 'merged' }}
+    steps:
+      - name: React
+        env:
+          BOT_TOKEN: ${{ secrets.DISCORD_BOT_TOKEN }}
+          CHANNEL_ID: ${{ inputs.DISCORD_CHANNEL_ID }}
+          GUILD_ID: ${{ inputs.DISCORD_GUILD_ID }}
+          PR_NUMBER: ${{ inputs.PR_NUMBER }}
+        run: |
+          # 1) get PR thread
+          threads=$(curl -H "Authorization: Bot $BOT_TOKEN" "https://discord.com/api/v10/guilds/${GUILD_ID}/threads/active")
+          thread_id=$(
+            echo "$threads" \
+              | jq -r --arg cid "$CHANNEL_ID" \
+                    --arg pref "#${PR_NUMBER}:" \
+                '.threads[]
+                | select(.parent_id == $cid and (.name | startswith($pref)))
+                | .id'
+          )
+          if [ -z "$thread_id" ]; then
+            echo "Thread not found"
+            exit 1
+          fi
+          # 2) get the first message in that thread
+          messages=$(curl -H "Authorization: Bot $BOT_TOKEN" \
+            "https://discord.com/api/v10/channels/$thread_id/messages?limit=1")
+          message_id=$(echo "$messages" | jq -r '.[-1].id')
+
+          if [ -z "$message_id" ]; then
+            echo "Message not found"
+            exit 1
+          fi
+
+          # 3) add the ✅ reaction
+          curl -X PUT \
+            -H "Authorization: Bot $BOT_TOKEN" \
+            "https://discord.com/api/v10/channels/$thread_id/messages/$message_id/reactions/%E2%9C%85/@me"

.gitignore

@@ -10,3 +10,5 @@ CaddyfileRemoteMalo
 .vscode
 .dev-docker-wrapper*
 backend/.minio-data
+.aider*
+!.aiderignore

CHANGELOG.md

@@ -1,5 +1,82 @@
 # Changelog
 
+## [1.491.5](https://github.com/windmill-labs/windmill/compare/v1.491.4...v1.491.5) (2025-05-17)
+
+
+### Bug Fixes
+
+* improve handling of custom concurrency key/tag with preprocessors ([#5762](https://github.com/windmill-labs/windmill/issues/5762)) ([59afa49](https://github.com/windmill-labs/windmill/commit/59afa493fa20cc70b6825e6356713cef84d75312))
+* S3 sql mode returns S3Object ([#5764](https://github.com/windmill-labs/windmill/issues/5764)) ([b29c6e7](https://github.com/windmill-labs/windmill/commit/b29c6e7636bb21c4d977bdaf89ac90e2a1a1086c))
+
+## [1.491.4](https://github.com/windmill-labs/windmill/compare/v1.491.3...v1.491.4) (2025-05-15)
+
+
+### Bug Fixes
+
+* add v1 preprocessor support to workspace preprocessor script ([#5757](https://github.com/windmill-labs/windmill/issues/5757)) ([9b1c30e](https://github.com/windmill-labs/windmill/commit/9b1c30eeff35291ad50f3ddeb64831eac88e2f66))
+
+## [1.491.3](https://github.com/windmill-labs/windmill/compare/v1.491.2...v1.491.3) (2025-05-15)
+
+
+### Bug Fixes
+
+* **frontend:** fix accordeon tabs initialization ([f488903](https://github.com/windmill-labs/windmill/commit/f488903635a1457f839ca641ed4f8d0891ef8212))
+* http trigger routers cache version sequence ([#5755](https://github.com/windmill-labs/windmill/issues/5755)) ([d53bceb](https://github.com/windmill-labs/windmill/commit/d53bceb8004541b79d33220ae8de06d25521da91))
+
+## [1.491.2](https://github.com/windmill-labs/windmill/compare/v1.491.1...v1.491.2) (2025-05-15)
+
+
+### Bug Fixes
+
+* **cli:** --version improvement ([f8f2015](https://github.com/windmill-labs/windmill/commit/f8f201564f7a323eb96f6dc684a525a0784d41f2))
+* http trigger signature validation ([#5753](https://github.com/windmill-labs/windmill/issues/5753)) ([9e9514b](https://github.com/windmill-labs/windmill/commit/9e9514b9af2337e143a9e4cf1e915e1477032e80))
+* Improve indexer performance by factoring required queries to the DB # ([#5749](https://github.com/windmill-labs/windmill/issues/5749)) ([b12feaf](https://github.com/windmill-labs/windmill/commit/b12feaf50ae0ef03816719ff39157fcf55159dbf))
+* improve perf of job deletion ([0efba94](https://github.com/windmill-labs/windmill/commit/0efba945bac9b84a489c6ef552e834593f209fe1))
+
+
+### Performance Improvements
+
+* cache http trigger routers and auth ([#5748](https://github.com/windmill-labs/windmill/issues/5748)) ([ddd18d2](https://github.com/windmill-labs/windmill/commit/ddd18d22a615408a9f57f910d0a58f17e6d6e29d))
+
+## [1.491.1](https://github.com/windmill-labs/windmill/compare/v1.491.0...v1.491.1) (2025-05-15)
+
+
+### Bug Fixes
+
+* avoid deadlocks in sending completed job to result processors ([#5742](https://github.com/windmill-labs/windmill/issues/5742)) ([e87d4f3](https://github.com/windmill-labs/windmill/commit/e87d4f3c1afb4ad356b326b7600c89e6c7803eff))
+
+## [1.491.0](https://github.com/windmill-labs/windmill/compare/v1.490.0...v1.491.0) (2025-05-14)
+
+
+### Features
+
+* Microsoft Teams approvals ([#5734](https://github.com/windmill-labs/windmill/issues/5734)) ([039f3e0](https://github.com/windmill-labs/windmill/commit/039f3e02268f2acda48abea420479216970e58e7))
+* sql jobs outputting to s3 + streaming for high-number of rows ([#5704](https://github.com/windmill-labs/windmill/issues/5704)) ([c7886ea](https://github.com/windmill-labs/windmill/commit/c7886ea07ae44af56f1467288b2d73ff2ae27964))
+
+
+### Bug Fixes
+
+* add missing run job transaction drop ([#5730](https://github.com/windmill-labs/windmill/issues/5730)) ([318def9](https://github.com/windmill-labs/windmill/commit/318def976cf0e4d5c32d01ac611a89e0a6425368))
+* add support for log compaction on docker jobs ([#5732](https://github.com/windmill-labs/windmill/issues/5732)) ([d35a7d2](https://github.com/windmill-labs/windmill/commit/d35a7d22f960f485889e22de48e8de8557069cb7))
+* Ansible lockfile back compatibility issue ([#5731](https://github.com/windmill-labs/windmill/issues/5731)) ([f73c90c](https://github.com/windmill-labs/windmill/commit/f73c90c7518569204b298b916d0fc298932d3cf0))
+* trigger event support for webhook get endpoints ([#5728](https://github.com/windmill-labs/windmill/issues/5728)) ([76258b7](https://github.com/windmill-labs/windmill/commit/76258b7b1af1313f694731d77f3fa6994e9ded70))
+
+## [1.490.0](https://github.com/windmill-labs/windmill/compare/v1.489.0...v1.490.0) (2025-05-12)
+
+
+### Features
+
+* preprocessor refactor ([#5629](https://github.com/windmill-labs/windmill/issues/5629)) ([254c3cf](https://github.com/windmill-labs/windmill/commit/254c3cf8eff32071d5290429aafd26992527fbca))
+
+
+### Bug Fixes
+
+* add back missing query args from http trigger object + correct wm_trigger shape ([#5722](https://github.com/windmill-labs/windmill/issues/5722)) ([66798df](https://github.com/windmill-labs/windmill/commit/66798df38464d732864627ae27a0e51e9518c609))
+* fix date input issue with initializer ([0cd9293](https://github.com/windmill-labs/windmill/commit/0cd92932f0e0998fc30ac02065d292ec35db5cae))
+* improve agents workers handling of WHITELIST_ENVS ([7c69959](https://github.com/windmill-labs/windmill/commit/7c699598533dade9713d976d8dd90fc657ebb503))
+* improve error display of nativets exceptions ([a3c76fb](https://github.com/windmill-labs/windmill/commit/a3c76fb10cba4d18547e66e47edec84833172b64))
+* make ansible more resilient to invalid lockfiles ([b51568c](https://github.com/windmill-labs/windmill/commit/b51568c166e29ec5ee4053fb14abda2fe6d46488))
+
 ## [1.489.0](https://github.com/windmill-labs/windmill/compare/v1.488.0...v1.489.0) (2025-05-08)
 
 

CLAUDE.md

@@ -0,0 +1,71 @@
+# Windmill Overview
+
+Windmill is an open-source developer platform for building internal tools, API integrations, background jobs, workflows, and user interfaces. It offers a unified system where scripts are automatically turned into sharable UIs and can be composed into flows or embedded in custom applications.
+
+## Core Capabilities
+
+- **Script Development and Execution**: Write and run scripts in Python, TypeScript/JavaScript (Deno/Bun), Go, Bash, SQL, and other languages
+- **Workflow Orchestration**: Compose scripts into multi-step flows with conditional logic, loops, and error handling
+- **UI Generation**: Automatically generate UIs from scripts or build custom applications with a low-code editor
+- **Job Scheduling**: Trigger scripts and flows on schedules, webhooks, or external events
+- **Resource Management**: Securely store and use credentials, databases, and other connections
+
+## Platform Architecture
+
+The Windmill platform consists of several key components:
+
+- **Frontend UI**: Web-based interface for script and flow development, app building, and result visualization
+- **API Server**: Central API that handles authentication, resource management, and job coordination
+- **Workers**: Execute scripts in their respective environments with proper sandboxing
+- **Database**: PostgreSQL database for storage of scripts, flows, resources, job results, and more
+- **Job Queue**: Queue system for managing job execution, implemented in PostgreSQL
+- **Client Libraries**: Libraries for interacting with Windmill from Python, TypeScript, or command line
+
+# Windmill Backend Architecture
+
+The Windmill backend is written in Rust and consists of several services working together. These services are designed for horizontal scaling with stateless API servers and workers that can be deployed across multiple machines.
+
+## Key Components
+
+- **API Server (`windmill-api`)**: Handles HTTP requests, authentication, and resource management
+- **Queue Manager (`windmill-queue`)**: Manages the job queue in PostgreSQL
+- **Worker System (`windmill-worker`)**: Executes jobs in sandboxed environments
+- **Common Utilities (`windmill-common`)**: Shared code used by multiple services
+- **Git Sync (`windmill-git-sync`)**: Synchronizes scripts with Git repositories
+
+## Job Execution System
+
+The job execution process follows these steps:
+
+1. The API server receives a request to run a script or flow and creates a job record in the database
+2. The job is added to the queue system in PostgreSQL
+3. Workers continuously poll the queue for jobs matching their capabilities
+4. When a job is picked up, it's routed to the appropriate language executor
+5. The script is executed in a sandboxed environment using NSJAIL for security
+6. Results are processed and stored in the database
+7. For flows, each step creates a new job that goes through the same process
+
+Windmill supports worker tags and groups to route jobs to workers with specific capabilities or resource access.
+
+# Windmill Frontend Architecture
+
+The Windmill frontend is built with Svelte and provides several key interfaces for interacting with the platform.
+
+## Key Components
+
+- **Script Builder**: Code editor with language support, schema inference, and dependency management
+- **Flow Builder**: Visual editor for creating multi-step workflows with branching and looping
+- **App Editor**: Grid-based editor for building custom UIs that integrate scripts and flows
+- **Schema Form System**: Generates form interfaces from script parameters automatically
+- **Result Viewer**: Visualizes job results, logs, and execution status
+
+The frontend uses the Monaco editor (same as VS Code) for code editing, with specialized language support for all supported script languages.
+
+## UI Framework
+
+The frontend is built with Svelte, providing a reactive and component-based architecture. Key frontend technologies include:
+
+- **Svelte/SvelteKit**: Core framework for UI components and routing
+- **Monaco Editor**: Code editing experience similar to VS Code
+- **Schema Form**: Automatic UI generation from TypeScript/JSON schemas
+- **Tailwind CSS**: Utility-first CSS framework for styling

Dockerfile

@@ -1,5 +1,5 @@
 ARG DEBIAN_IMAGE=debian:bookworm-slim
-ARG RUST_IMAGE=rust:1.85-slim-bookworm
+ARG RUST_IMAGE=rust:1.86-slim-bookworm
 
 FROM ${RUST_IMAGE} AS rust_base
 

backend/.sqlx/query-0937e25e89959447e7cb1816c112bbc4718cbb8ad6e2f13eae6b14f129d12936.json

@@ -1,12 +1,12 @@
 {
   "db_name": "PostgreSQL",
-  "query": "select path, tag, concurrency_key, concurrent_limit, concurrency_time_window_s, cache_ttl, language as \"language: ScriptLang\", dedicated_worker, priority, delete_after_use, timeout, has_preprocessor, on_behalf_of_email, created_by from script where hash = $1 AND workspace_id = $2",
+  "query": "select hash, tag, concurrency_key, concurrent_limit, concurrency_time_window_s, cache_ttl, language as \"language: ScriptLang\", dedicated_worker, priority, delete_after_use, timeout, has_preprocessor, on_behalf_of_email, created_by, path from script where hash = $1 AND workspace_id = $2",
   "describe": {
     "columns": [
       {
         "ordinal": 0,
-        "name": "path",
-        "type_info": "Varchar"
+        "name": "hash",
+        "type_info": "Int8"
       },
       {
         "ordinal": 1,
@@ -101,6 +101,11 @@
         "ordinal": 13,
         "name": "created_by",
         "type_info": "Varchar"
+      },
+      {
+        "ordinal": 14,
+        "name": "path",
+        "type_info": "Varchar"
       }
     ],
     "parameters": {
@@ -123,8 +128,9 @@
       true,
       true,
       true,
+      false,
       false
     ]
   },
-  "hash": "75451b6d48e4c26812ae64981d0d968b8fb0bf4374a2fccc167fa879bad7078f"
+  "hash": "0937e25e89959447e7cb1816c112bbc4718cbb8ad6e2f13eae6b14f129d12936"
 }

backend/.sqlx/query-16be720bf1c88ecfa2a4bf6adbb1924df4817b6236b3a949209465f3a2c42bb9.json

@@ -0,0 +1,20 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "SELECT nextval('http_trigger_version_seq')",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "nextval",
+        "type_info": "Int8"
+      }
+    ],
+    "parameters": {
+      "Left": []
+    },
+    "nullable": [
+      null
+    ]
+  },
+  "hash": "16be720bf1c88ecfa2a4bf6adbb1924df4817b6236b3a949209465f3a2c42bb9"
+}

backend/.sqlx/query-1d819b829cd92995c39d29540df8cffbcc3334bada244a331a0bd8db06029d42.json

@@ -1,22 +0,0 @@
-{
-  "db_name": "PostgreSQL",
-  "query": "DELETE FROM v2_job_completed c\n                    USING v2_job j\n                    WHERE\n                        created_at <= now() - ($1::bigint::text || ' s')::interval\n                        AND completed_at + ($1::bigint::text || ' s')::interval <= now()\n                        AND c.id = j.id\n                    RETURNING c.id",
-  "describe": {
-    "columns": [
-      {
-        "ordinal": 0,
-        "name": "id",
-        "type_info": "Uuid"
-      }
-    ],
-    "parameters": {
-      "Left": [
-        "Int8"
-      ]
-    },
-    "nullable": [
-      false
-    ]
-  },
-  "hash": "1d819b829cd92995c39d29540df8cffbcc3334bada244a331a0bd8db06029d42"
-}

backend/.sqlx/query-1eeb218c30c0a6b0f7633813c764f57f8968894b4786b94109a057796ff500e4.json

@@ -1,6 +1,6 @@
 {
   "db_name": "PostgreSQL",
-  "query": "\n            SELECT \n                path, \n                script_path, \n                is_flow, \n                route_path, \n                authentication_resource_path,\n                workspace_id, \n                is_async, \n                authentication_method  AS \"authentication_method: _\", \n                edited_by, \n                email, \n                static_asset_config AS \"static_asset_config: _\",\n                wrap_body,\n                raw_string,\n                workspaced_route,\n                is_static_website\n            FROM \n                http_trigger \n            WHERE \n                http_method = $1\n            ",
+  "query": "\n                    SELECT \n                        path, \n                        script_path, \n                        is_flow, \n                        route_path, \n                        authentication_resource_path,\n                        workspace_id, \n                        is_async, \n                        authentication_method  AS \"authentication_method: _\", \n                        edited_by, \n                        email, \n                        static_asset_config AS \"static_asset_config: _\",\n                        wrap_body,\n                        raw_string,\n                        workspaced_route,\n                        is_static_website\n                    FROM \n                        http_trigger \n                    WHERE \n                        http_method = $1\n                    ",
   "describe": {
     "columns": [
       {
@@ -129,5 +129,5 @@
       false
     ]
   },
-  "hash": "4053f0bb30f651ddf2214115748daca0ea457da8252394eeeead0897d184f6da"
+  "hash": "1eeb218c30c0a6b0f7633813c764f57f8968894b4786b94109a057796ff500e4"
 }

backend/.sqlx/query-20fd50949796913dd48f67ee75b11272ce5b3046f87b9efd504e013fba9724f5.json

@@ -0,0 +1,20 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "SELECT last_value FROM http_trigger_version_seq",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "last_value",
+        "type_info": "Int8"
+      }
+    ],
+    "parameters": {
+      "Left": []
+    },
+    "nullable": [
+      false
+    ]
+  },
+  "hash": "20fd50949796913dd48f67ee75b11272ce5b3046f87b9efd504e013fba9724f5"
+}

backend/.sqlx/query-234acda79d470e99e9cbde5c7401d6f7894c25f90e39ec8606f79b8be56d1c17.json

@@ -0,0 +1,23 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "\n                        SELECT EXISTS(\n                            SELECT 1\n                            FROM \n                                http_trigger \n                            WHERE \n                                workspace_id = $1 AND \n                                path = $2\n                        )\n                        ",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "exists",
+        "type_info": "Bool"
+      }
+    ],
+    "parameters": {
+      "Left": [
+        "Text",
+        "Text"
+      ]
+    },
+    "nullable": [
+      null
+    ]
+  },
+  "hash": "234acda79d470e99e9cbde5c7401d6f7894c25f90e39ec8606f79b8be56d1c17"
+}

backend/.sqlx/query-282afbff89d3186d47ef5dbd0b65026ad37fb31b485fc44b6ec257dd77825428.json

@@ -1,41 +0,0 @@
-{
-  "db_name": "PostgreSQL",
-  "query": "SELECT created_by, coalesce(job_logs.logs, '') as logs, job_logs.log_offset, job_logs.log_file_index\n        FROM v2_as_completed_job\n        LEFT JOIN job_logs ON job_logs.job_id = v2_as_completed_job.id\n        WHERE v2_as_completed_job.id = $1 AND v2_as_completed_job.workspace_id = $2",
-  "describe": {
-    "columns": [
-      {
-        "ordinal": 0,
-        "name": "created_by",
-        "type_info": "Varchar"
-      },
-      {
-        "ordinal": 1,
-        "name": "logs",
-        "type_info": "Text"
-      },
-      {
-        "ordinal": 2,
-        "name": "log_offset",
-        "type_info": "Int4"
-      },
-      {
-        "ordinal": 3,
-        "name": "log_file_index",
-        "type_info": "TextArray"
-      }
-    ],
-    "parameters": {
-      "Left": [
-        "Uuid",
-        "Text"
-      ]
-    },
-    "nullable": [
-      true,
-      null,
-      false,
-      true
-    ]
-  },
-  "hash": "282afbff89d3186d47ef5dbd0b65026ad37fb31b485fc44b6ec257dd77825428"
-}

backend/.sqlx/query-2a33a35afc1ba4c31a5713cfd1a2c662f25cda387197aaf9f35000df31b8b07d.json

@@ -0,0 +1,12 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "DROP INDEX CONCURRENTLY IF EXISTS log_file_hostname_log_ts_idx",
+  "describe": {
+    "columns": [],
+    "parameters": {
+      "Left": []
+    },
+    "nullable": []
+  },
+  "hash": "2a33a35afc1ba4c31a5713cfd1a2c662f25cda387197aaf9f35000df31b8b07d"
+}

backend/.sqlx/query-3a534b4fc36171efaa7c647f48320b59bbc414cfb92e960c174dd63fc180e187.json

@@ -1,29 +0,0 @@
-{
-  "db_name": "PostgreSQL",
-  "query": "SELECT tag, dedicated_worker from flow WHERE path = $1 and workspace_id = $2",
-  "describe": {
-    "columns": [
-      {
-        "ordinal": 0,
-        "name": "tag",
-        "type_info": "Varchar"
-      },
-      {
-        "ordinal": 1,
-        "name": "dedicated_worker",
-        "type_info": "Bool"
-      }
-    ],
-    "parameters": {
-      "Left": [
-        "Text",
-        "Text"
-      ]
-    },
-    "nullable": [
-      true,
-      true
-    ]
-  },
-  "hash": "3a534b4fc36171efaa7c647f48320b59bbc414cfb92e960c174dd63fc180e187"
-}

backend/.sqlx/query-3f08ffbb5c71b873a9e164ecb0b10fffb37599f2a703885ee723cb9290fed13e.json

@@ -0,0 +1,91 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "WITH job_info AS (\n            -- Query for Teams (running jobs)\n            SELECT\n                parent.job_kind AS \"job_kind!: JobKind\",\n                parent.script_hash AS \"script_hash: ScriptHash\",\n                parent.raw_flow AS \"raw_flow: sqlx::types::Json<Box<RawValue>>\",\n                child.parent_job AS \"parent_job: Uuid\",\n                parent.created_at AS \"created_at!: chrono::NaiveDateTime\",\n                parent.created_by AS \"created_by!\",\n                parent.script_path,\n                parent.args AS \"args: sqlx::types::Json<Box<RawValue>>\"\n            FROM v2_as_queue child\n            JOIN v2_as_queue parent ON parent.id = child.parent_job\n            WHERE child.id = $1 AND child.workspace_id = $2\n            UNION ALL\n            -- Query for Slack (completed jobs)\n            SELECT\n                v2_as_queue.job_kind AS \"job_kind!: JobKind\",\n                v2_as_queue.script_hash AS \"script_hash: ScriptHash\",\n                v2_as_queue.raw_flow AS \"raw_flow: sqlx::types::Json<Box<RawValue>>\",\n                v2_as_completed_job.parent_job AS \"parent_job: Uuid\",\n                v2_as_completed_job.created_at AS \"created_at!: chrono::NaiveDateTime\",\n                v2_as_completed_job.created_by AS \"created_by!\",\n                v2_as_queue.script_path,\n                v2_as_queue.args AS \"args: sqlx::types::Json<Box<RawValue>>\"\n            FROM v2_as_queue\n            JOIN v2_as_completed_job ON v2_as_completed_job.parent_job = v2_as_queue.id\n            WHERE v2_as_completed_job.id = $1 AND v2_as_completed_job.workspace_id = $2\n        )\n        SELECT * FROM job_info LIMIT 1",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "job_kind!: JobKind",
+        "type_info": {
+          "Custom": {
+            "name": "job_kind",
+            "kind": {
+              "Enum": [
+                "script",
+                "preview",
+                "flow",
+                "dependencies",
+                "flowpreview",
+                "script_hub",
+                "identity",
+                "flowdependencies",
+                "http",
+                "graphql",
+                "postgresql",
+                "noop",
+                "appdependencies",
+                "deploymentcallback",
+                "singlescriptflow",
+                "flowscript",
+                "flownode",
+                "appscript"
+              ]
+            }
+          }
+        }
+      },
+      {
+        "ordinal": 1,
+        "name": "script_hash: ScriptHash",
+        "type_info": "Int8"
+      },
+      {
+        "ordinal": 2,
+        "name": "raw_flow: sqlx::types::Json<Box<RawValue>>",
+        "type_info": "Jsonb"
+      },
+      {
+        "ordinal": 3,
+        "name": "parent_job: Uuid",
+        "type_info": "Uuid"
+      },
+      {
+        "ordinal": 4,
+        "name": "created_at!: chrono::NaiveDateTime",
+        "type_info": "Timestamptz"
+      },
+      {
+        "ordinal": 5,
+        "name": "created_by!",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 6,
+        "name": "script_path",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 7,
+        "name": "args: sqlx::types::Json<Box<RawValue>>",
+        "type_info": "Jsonb"
+      }
+    ],
+    "parameters": {
+      "Left": [
+        "Uuid",
+        "Text"
+      ]
+    },
+    "nullable": [
+      null,
+      null,
+      null,
+      null,
+      null,
+      null,
+      null,
+      null
+    ]
+  },
+  "hash": "3f08ffbb5c71b873a9e164ecb0b10fffb37599f2a703885ee723cb9290fed13e"
+}

backend/.sqlx/query-4455c7e8aa7616f3d547c5eb7d93c840d8aff3df4a6926f569792f69b2e9601f.json

@@ -0,0 +1,23 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "select hash from script where path = $1 AND workspace_id = $2 AND deleted = false AND lock IS not NULL AND lock_error_logs IS NULL ORDER BY created_at DESC LIMIT 1",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "hash",
+        "type_info": "Int8"
+      }
+    ],
+    "parameters": {
+      "Left": [
+        "Text",
+        "Text"
+      ]
+    },
+    "nullable": [
+      false
+    ]
+  },
+  "hash": "4455c7e8aa7616f3d547c5eb7d93c840d8aff3df4a6926f569792f69b2e9601f"
+}

backend/.sqlx/query-49943f69ed74bc889120dcd2571e8e868a4f4795933044ff95b20d8df45cd145.json

@@ -0,0 +1,12 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "CREATE INDEX CONCURRENTLY IF NOT EXISTS ix_job_completed_completed_at ON v2_job_completed (completed_at DESC)",
+  "describe": {
+    "columns": [],
+    "parameters": {
+      "Left": []
+    },
+    "nullable": []
+  },
+  "hash": "49943f69ed74bc889120dcd2571e8e868a4f4795933044ff95b20d8df45cd145"
+}

backend/.sqlx/query-4f547c0fd54f3bc57212ce87810e35adf640d44d607e62a1fb296e38ac3fdd36.json

@@ -1,6 +1,6 @@
 {
   "db_name": "PostgreSQL",
-  "query": "\n        SELECT \n            id, \n            created_at, \n            trigger_kind AS \"trigger_kind: _\",\n            CASE \n                WHEN pg_column_size(payload) < 40000 THEN payload \n                ELSE '\"WINDMILL_TOO_BIG\"'::jsonb \n            END AS \"payload!: _\",\n            trigger_extra AS \"trigger_extra: _\"\n        FROM \n            capture\n        WHERE \n            workspace_id = $1 \n            AND path = $2 \n            AND is_flow = $3 \n            AND ($4::trigger_kind IS NULL OR trigger_kind = $4)\n        ORDER BY \n            created_at DESC\n        OFFSET $5\n        LIMIT $6\n        ",
+  "query": "\n        SELECT \n            id, \n            created_at, \n            trigger_kind AS \"trigger_kind: _\",\n            CASE \n                WHEN pg_column_size(main_args) < 40000 THEN main_args \n                ELSE '\"WINDMILL_TOO_BIG\"'::jsonb \n            END AS \"main_args!: _\",\n            CASE\n                WHEN pg_column_size(preprocessor_args) < 40000 THEN preprocessor_args\n                ELSE '\"WINDMILL_TOO_BIG\"'::jsonb\n            END AS \"preprocessor_args: _\"\n        FROM \n            capture\n        WHERE \n            workspace_id = $1 \n            AND path = $2 \n            AND is_flow = $3 \n            AND ($4::trigger_kind IS NULL OR trigger_kind = $4)\n        ORDER BY \n            created_at DESC\n        OFFSET $5\n        LIMIT $6\n        ",
   "describe": {
     "columns": [
       {
@@ -38,12 +38,12 @@
       },
       {
         "ordinal": 3,
-        "name": "payload!: _",
+        "name": "main_args!: _",
         "type_info": "Jsonb"
       },
       {
         "ordinal": 4,
-        "name": "trigger_extra: _",
+        "name": "preprocessor_args: _",
         "type_info": "Jsonb"
       }
     ],
@@ -80,8 +80,8 @@
       false,
       false,
       null,
-      true
+      null
     ]
   },
-  "hash": "7cba31d597215a343cb0bca5b204a15fbba193262f7895c2bba90feb4215d6f3"
+  "hash": "4f547c0fd54f3bc57212ce87810e35adf640d44d607e62a1fb296e38ac3fdd36"
 }

backend/.sqlx/query-52ad0c838d19cbd9e90b8368abe71dd12655179f41f43896e7d30fdfb3ae5939.json

@@ -1,15 +0,0 @@
-{
-  "db_name": "PostgreSQL",
-  "query": "WITH job_result AS (\n                 SELECT result \n                 FROM v2_job_completed \n                 WHERE id = $1\n             )\n             UPDATE v2_job \n             SET args = COALESCE(\n                     CASE \n                         WHEN job_result.result IS NULL THEN NULL\n                         WHEN jsonb_typeof(job_result.result) = 'object' \n                         THEN job_result.result\n                         WHEN jsonb_typeof(job_result.result) = 'null'\n                         THEN NULL\n                         ELSE jsonb_build_object('value', job_result.result)\n                     END, \n                     '{}'::jsonb\n                 ),\n                 preprocessed = TRUE\n             FROM job_result\n             WHERE v2_job.id = $2;\n             ",
-  "describe": {
-    "columns": [],
-    "parameters": {
-      "Left": [
-        "Uuid",
-        "Uuid"
-      ]
-    },
-    "nullable": []
-  },
-  "hash": "52ad0c838d19cbd9e90b8368abe71dd12655179f41f43896e7d30fdfb3ae5939"
-}

backend/.sqlx/query-551fee7919fdeb911e3f9cc5852e158ea47e3db4895c2b2b1d3cb6b16fceeda9.json

@@ -0,0 +1,23 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "SELECT fv.id\n                                                                                FROM flow f\n                                                                                INNER JOIN flow_version fv ON fv.id = f.versions[array_upper(f.versions, 1)]\n                                                                                WHERE fv.value->'preprocessor_module'->'value'->>'path' = $1 AND f.workspace_id = $2",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "id",
+        "type_info": "Int8"
+      }
+    ],
+    "parameters": {
+      "Left": [
+        "Text",
+        "Text"
+      ]
+    },
+    "nullable": [
+      false
+    ]
+  },
+  "hash": "551fee7919fdeb911e3f9cc5852e158ea47e3db4895c2b2b1d3cb6b16fceeda9"
+}

backend/.sqlx/query-5820d34be1a7f7b72e656c692f53146f45ad4a6e584e917a0a86280d8f473c10.json

@@ -0,0 +1,22 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "DELETE FROM v2_job_completed c\n                    WHERE completed_at <= now() - ($1::bigint::text || ' s')::interval \n                    RETURNING c.id",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "id",
+        "type_info": "Uuid"
+      }
+    ],
+    "parameters": {
+      "Left": [
+        "Int8"
+      ]
+    },
+    "nullable": [
+      false
+    ]
+  },
+  "hash": "5820d34be1a7f7b72e656c692f53146f45ad4a6e584e917a0a86280d8f473c10"
+}

backend/.sqlx/query-6d36da815795d5cac2e76b4d34a7f1f9f836bd5b6866d3db26b25428d39c0b23.json

@@ -1,29 +0,0 @@
-{
-  "db_name": "PostgreSQL",
-  "query": "SELECT on_behalf_of_email, edited_by FROM flow WHERE path = $1 AND workspace_id = $2",
-  "describe": {
-    "columns": [
-      {
-        "ordinal": 0,
-        "name": "on_behalf_of_email",
-        "type_info": "Text"
-      },
-      {
-        "ordinal": 1,
-        "name": "edited_by",
-        "type_info": "Varchar"
-      }
-    ],
-    "parameters": {
-      "Left": [
-        "Text",
-        "Text"
-      ]
-    },
-    "nullable": [
-      true,
-      false
-    ]
-  },
-  "hash": "6d36da815795d5cac2e76b4d34a7f1f9f836bd5b6866d3db26b25428d39c0b23"
-}

backend/.sqlx/query-6e2564dc37ee967c634deade67ceabc4c516418e595dee9cd7d651acc1f4af6e.json

@@ -0,0 +1,248 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "SELECT\n                            cj.id AS \"id!\",\n                            cj.workspace_id AS \"workspace_id!\",\n                            cj.parent_job,\n                            cj.created_by AS \"created_by!\",\n                            cj.duration_ms AS \"duration_ms!\",\n                            cj.success AS \"success!\",\n                            cj.script_hash AS \"script_hash!: Option<ScriptHash>\",\n                            cj.script_path,\n                            cj.args AS \"args: sqlx::types::Json<HashMap<String, Box<RawValue>>>\",\n                            cj.result AS \"result: sqlx::types::Json<Box<RawValue>>\",\n                            cj.deleted AS \"deleted!\",\n                            cj.canceled AS \"canceled!\",\n                            cj.canceled_by,\n                            cj.canceled_reason,\n                            cj.job_kind AS \"job_kind!: JobKind\",\n                            cj.schedule_path,\n                            cj.permissioned_as AS \"permissioned_as!\",\n                            cj.is_flow_step AS \"is_flow_step!\",\n                            cj.language AS \"language: ScriptLang\",\n                            cj.is_skipped AS \"is_skipped!\",\n                            cj.email AS \"email!\",\n                            cj.visible_to_owner AS \"visible_to_owner!\",\n                            cj.mem_peak,\n                            cj.tag AS \"tag!\",\n                            cj.created_at AS \"created_at!\",\n                            cj.started_at,\n                            job_logs.logs,\n                            job_logs.log_offset AS \"log_offset?\",\n                            job_logs.log_file_index\n\n                        FROM v2_as_completed_job AS cj\n                        LEFT JOIN job_logs ON cj.id = job_logs.job_id\n                        WHERE (cj.created_at > $1 AND cj.created_at < $3)\n                               OR cj.id = ANY($2)\n                        ORDER BY cj.created_at ASC LIMIT $4",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "id!",
+        "type_info": "Uuid"
+      },
+      {
+        "ordinal": 1,
+        "name": "workspace_id!",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 2,
+        "name": "parent_job",
+        "type_info": "Uuid"
+      },
+      {
+        "ordinal": 3,
+        "name": "created_by!",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 4,
+        "name": "duration_ms!",
+        "type_info": "Int8"
+      },
+      {
+        "ordinal": 5,
+        "name": "success!",
+        "type_info": "Bool"
+      },
+      {
+        "ordinal": 6,
+        "name": "script_hash!: Option<ScriptHash>",
+        "type_info": "Int8"
+      },
+      {
+        "ordinal": 7,
+        "name": "script_path",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 8,
+        "name": "args: sqlx::types::Json<HashMap<String, Box<RawValue>>>",
+        "type_info": "Jsonb"
+      },
+      {
+        "ordinal": 9,
+        "name": "result: sqlx::types::Json<Box<RawValue>>",
+        "type_info": "Jsonb"
+      },
+      {
+        "ordinal": 10,
+        "name": "deleted!",
+        "type_info": "Bool"
+      },
+      {
+        "ordinal": 11,
+        "name": "canceled!",
+        "type_info": "Bool"
+      },
+      {
+        "ordinal": 12,
+        "name": "canceled_by",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 13,
+        "name": "canceled_reason",
+        "type_info": "Text"
+      },
+      {
+        "ordinal": 14,
+        "name": "job_kind!: JobKind",
+        "type_info": {
+          "Custom": {
+            "name": "job_kind",
+            "kind": {
+              "Enum": [
+                "script",
+                "preview",
+                "flow",
+                "dependencies",
+                "flowpreview",
+                "script_hub",
+                "identity",
+                "flowdependencies",
+                "http",
+                "graphql",
+                "postgresql",
+                "noop",
+                "appdependencies",
+                "deploymentcallback",
+                "singlescriptflow",
+                "flowscript",
+                "flownode",
+                "appscript"
+              ]
+            }
+          }
+        }
+      },
+      {
+        "ordinal": 15,
+        "name": "schedule_path",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 16,
+        "name": "permissioned_as!",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 17,
+        "name": "is_flow_step!",
+        "type_info": "Bool"
+      },
+      {
+        "ordinal": 18,
+        "name": "language: ScriptLang",
+        "type_info": {
+          "Custom": {
+            "name": "script_lang",
+            "kind": {
+              "Enum": [
+                "python3",
+                "deno",
+                "go",
+                "bash",
+                "postgresql",
+                "nativets",
+                "bun",
+                "mysql",
+                "bigquery",
+                "snowflake",
+                "graphql",
+                "powershell",
+                "mssql",
+                "php",
+                "bunnative",
+                "rust",
+                "ansible",
+                "csharp",
+                "oracledb",
+                "nu",
+                "java"
+              ]
+            }
+          }
+        }
+      },
+      {
+        "ordinal": 19,
+        "name": "is_skipped!",
+        "type_info": "Bool"
+      },
+      {
+        "ordinal": 20,
+        "name": "email!",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 21,
+        "name": "visible_to_owner!",
+        "type_info": "Bool"
+      },
+      {
+        "ordinal": 22,
+        "name": "mem_peak",
+        "type_info": "Int4"
+      },
+      {
+        "ordinal": 23,
+        "name": "tag!",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 24,
+        "name": "created_at!",
+        "type_info": "Timestamptz"
+      },
+      {
+        "ordinal": 25,
+        "name": "started_at",
+        "type_info": "Timestamptz"
+      },
+      {
+        "ordinal": 26,
+        "name": "logs",
+        "type_info": "Text"
+      },
+      {
+        "ordinal": 27,
+        "name": "log_offset?",
+        "type_info": "Int4"
+      },
+      {
+        "ordinal": 28,
+        "name": "log_file_index",
+        "type_info": "TextArray"
+      }
+    ],
+    "parameters": {
+      "Left": [
+        "Timestamptz",
+        "UuidArray",
+        "Timestamptz",
+        "Int8"
+      ]
+    },
+    "nullable": [
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      false,
+      true
+    ]
+  },
+  "hash": "6e2564dc37ee967c634deade67ceabc4c516418e595dee9cd7d651acc1f4af6e"
+}

backend/.sqlx/query-89de3ff8ab32e545efcbcda05f994cb1a32c4991cbd25046282d34272587d2de.json

@@ -1,130 +0,0 @@
-{
-  "db_name": "PostgreSQL",
-  "query": "select hash, tag, concurrency_key, concurrent_limit, concurrency_time_window_s, cache_ttl, language as \"language: ScriptLang\", dedicated_worker, priority, delete_after_use, timeout, has_preprocessor, on_behalf_of_email, created_by from script where path = $1 AND workspace_id = $2 AND\n    created_at = (SELECT max(created_at) FROM script WHERE path = $1 AND workspace_id = $2 AND\n    deleted = false AND lock IS not NULL AND lock_error_logs IS NULL)",
-  "describe": {
-    "columns": [
-      {
-        "ordinal": 0,
-        "name": "hash",
-        "type_info": "Int8"
-      },
-      {
-        "ordinal": 1,
-        "name": "tag",
-        "type_info": "Varchar"
-      },
-      {
-        "ordinal": 2,
-        "name": "concurrency_key",
-        "type_info": "Varchar"
-      },
-      {
-        "ordinal": 3,
-        "name": "concurrent_limit",
-        "type_info": "Int4"
-      },
-      {
-        "ordinal": 4,
-        "name": "concurrency_time_window_s",
-        "type_info": "Int4"
-      },
-      {
-        "ordinal": 5,
-        "name": "cache_ttl",
-        "type_info": "Int4"
-      },
-      {
-        "ordinal": 6,
-        "name": "language: ScriptLang",
-        "type_info": {
-          "Custom": {
-            "name": "script_lang",
-            "kind": {
-              "Enum": [
-                "python3",
-                "deno",
-                "go",
-                "bash",
-                "postgresql",
-                "nativets",
-                "bun",
-                "mysql",
-                "bigquery",
-                "snowflake",
-                "graphql",
-                "powershell",
-                "mssql",
-                "php",
-                "bunnative",
-                "rust",
-                "ansible",
-                "csharp",
-                "oracledb",
-                "nu",
-                "java"
-              ]
-            }
-          }
-        }
-      },
-      {
-        "ordinal": 7,
-        "name": "dedicated_worker",
-        "type_info": "Bool"
-      },
-      {
-        "ordinal": 8,
-        "name": "priority",
-        "type_info": "Int2"
-      },
-      {
-        "ordinal": 9,
-        "name": "delete_after_use",
-        "type_info": "Bool"
-      },
-      {
-        "ordinal": 10,
-        "name": "timeout",
-        "type_info": "Int4"
-      },
-      {
-        "ordinal": 11,
-        "name": "has_preprocessor",
-        "type_info": "Bool"
-      },
-      {
-        "ordinal": 12,
-        "name": "on_behalf_of_email",
-        "type_info": "Text"
-      },
-      {
-        "ordinal": 13,
-        "name": "created_by",
-        "type_info": "Varchar"
-      }
-    ],
-    "parameters": {
-      "Left": [
-        "Text",
-        "Text"
-      ]
-    },
-    "nullable": [
-      false,
-      true,
-      true,
-      true,
-      true,
-      true,
-      false,
-      true,
-      true,
-      true,
-      true,
-      true,
-      true,
-      false
-    ]
-  },
-  "hash": "89de3ff8ab32e545efcbcda05f994cb1a32c4991cbd25046282d34272587d2de"
-}

backend/.sqlx/query-8fb2581a439c26391e66ae7fac32c6cd2932f28ab6490ace027ed3a790b2a0f7.json

@@ -1,28 +0,0 @@
-{
-  "db_name": "PostgreSQL",
-  "query": "SELECT edited_by, on_behalf_of_email FROM flow WHERE path = $1",
-  "describe": {
-    "columns": [
-      {
-        "ordinal": 0,
-        "name": "edited_by",
-        "type_info": "Varchar"
-      },
-      {
-        "ordinal": 1,
-        "name": "on_behalf_of_email",
-        "type_info": "Text"
-      }
-    ],
-    "parameters": {
-      "Left": [
-        "Text"
-      ]
-    },
-    "nullable": [
-      false,
-      true
-    ]
-  },
-  "hash": "8fb2581a439c26391e66ae7fac32c6cd2932f28ab6490ace027ed3a790b2a0f7"
-}

backend/.sqlx/query-91f23fcc27777c279c79e2682fc15c026e55f9ec3799be65a2e8920fe6174a17.json

@@ -0,0 +1,22 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "SELECT result  as \"result: Json<HashMap<String, Box<RawValue>>>\"\n                 FROM v2_job_completed \n                 WHERE id = $1",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "result: Json<HashMap<String, Box<RawValue>>>",
+        "type_info": "Jsonb"
+      }
+    ],
+    "parameters": {
+      "Left": [
+        "Uuid"
+      ]
+    },
+    "nullable": [
+      true
+    ]
+  },
+  "hash": "91f23fcc27777c279c79e2682fc15c026e55f9ec3799be65a2e8920fe6174a17"
+}

backend/.sqlx/query-927149213e0f8ae983652ef80464f646d6be80e702193f1acdd40dd6033652e4.json

@@ -1,134 +0,0 @@
-{
-  "db_name": "PostgreSQL",
-  "query": "\n            SELECT \n                path, \n                script_path, \n                is_flow, \n                route_path, \n                workspace_id, \n                is_async, \n                authentication_method  AS \"authentication_method: _\", \n                edited_by, \n                email,\n                static_asset_config AS \"static_asset_config: _\",\n                wrap_body,\n                raw_string,\n                workspaced_route,\n                is_static_website,\n                authentication_resource_path\n            FROM \n                http_trigger \n            WHERE \n                workspace_id = $1 AND \n                http_method = $2\n            ",
-  "describe": {
-    "columns": [
-      {
-        "ordinal": 0,
-        "name": "path",
-        "type_info": "Varchar"
-      },
-      {
-        "ordinal": 1,
-        "name": "script_path",
-        "type_info": "Varchar"
-      },
-      {
-        "ordinal": 2,
-        "name": "is_flow",
-        "type_info": "Bool"
-      },
-      {
-        "ordinal": 3,
-        "name": "route_path",
-        "type_info": "Varchar"
-      },
-      {
-        "ordinal": 4,
-        "name": "workspace_id",
-        "type_info": "Varchar"
-      },
-      {
-        "ordinal": 5,
-        "name": "is_async",
-        "type_info": "Bool"
-      },
-      {
-        "ordinal": 6,
-        "name": "authentication_method: _",
-        "type_info": {
-          "Custom": {
-            "name": "authentication_method",
-            "kind": {
-              "Enum": [
-                "none",
-                "windmill",
-                "api_key",
-                "basic_http",
-                "custom_script",
-                "signature"
-              ]
-            }
-          }
-        }
-      },
-      {
-        "ordinal": 7,
-        "name": "edited_by",
-        "type_info": "Varchar"
-      },
-      {
-        "ordinal": 8,
-        "name": "email",
-        "type_info": "Varchar"
-      },
-      {
-        "ordinal": 9,
-        "name": "static_asset_config: _",
-        "type_info": "Jsonb"
-      },
-      {
-        "ordinal": 10,
-        "name": "wrap_body",
-        "type_info": "Bool"
-      },
-      {
-        "ordinal": 11,
-        "name": "raw_string",
-        "type_info": "Bool"
-      },
-      {
-        "ordinal": 12,
-        "name": "workspaced_route",
-        "type_info": "Bool"
-      },
-      {
-        "ordinal": 13,
-        "name": "is_static_website",
-        "type_info": "Bool"
-      },
-      {
-        "ordinal": 14,
-        "name": "authentication_resource_path",
-        "type_info": "Varchar"
-      }
-    ],
-    "parameters": {
-      "Left": [
-        "Text",
-        {
-          "Custom": {
-            "name": "http_method",
-            "kind": {
-              "Enum": [
-                "get",
-                "post",
-                "put",
-                "delete",
-                "patch"
-              ]
-            }
-          }
-        }
-      ]
-    },
-    "nullable": [
-      false,
-      false,
-      false,
-      false,
-      false,
-      false,
-      false,
-      false,
-      false,
-      true,
-      false,
-      false,
-      false,
-      false,
-      true
-    ]
-  },
-  "hash": "927149213e0f8ae983652ef80464f646d6be80e702193f1acdd40dd6033652e4"
-}

backend/.sqlx/query-97048ce0bcabb9baecb80cde5ab3c989e1575fbd20ef22766d2887a86dce15e1.json

@@ -1,23 +0,0 @@
-{
-  "db_name": "PostgreSQL",
-  "query": "SELECT flow.versions[array_upper(flow.versions, 1)] AS \"version!: i64\"\n                FROM flow WHERE path = $1 AND workspace_id = $2",
-  "describe": {
-    "columns": [
-      {
-        "ordinal": 0,
-        "name": "version!: i64",
-        "type_info": "Int8"
-      }
-    ],
-    "parameters": {
-      "Left": [
-        "Text",
-        "Text"
-      ]
-    },
-    "nullable": [
-      null
-    ]
-  },
-  "hash": "97048ce0bcabb9baecb80cde5ab3c989e1575fbd20ef22766d2887a86dce15e1"
-}

backend/.sqlx/query-9b60fa8a1003015bc5a7cdbee9a4486b313d45347dfd9d4793d60e2760763ca3.json

@@ -1,6 +1,6 @@
 {
   "db_name": "PostgreSQL",
-  "query": "SELECT tag, dedicated_worker, flow_version.value->>'early_return' as early_return, flow_version.value->>'preprocessor_module' IS NOT NULL as has_preprocessor, on_behalf_of_email, edited_by\n        FROM flow \n        LEFT JOIN flow_version\n            ON flow_version.id = flow.versions[array_upper(flow.versions, 1)]\n        WHERE flow.path = $1 and flow.workspace_id = $2",
+  "query": "SELECT tag, dedicated_worker, flow_version.value->>'early_return' as early_return, flow_version.value->>'preprocessor_module' IS NOT NULL as has_preprocessor, on_behalf_of_email, edited_by, flow_version.id AS version\n                    FROM flow\n                    INNER JOIN flow_version\n                        ON flow_version.id = $3\n                    WHERE flow.path = $1 and flow.workspace_id = $2",
   "describe": {
     "columns": [
       {
@@ -32,12 +32,18 @@
         "ordinal": 5,
         "name": "edited_by",
         "type_info": "Varchar"
+      },
+      {
+        "ordinal": 6,
+        "name": "version",
+        "type_info": "Int8"
       }
     ],
     "parameters": {
       "Left": [
         "Text",
-        "Text"
+        "Text",
+        "Int8"
       ]
     },
     "nullable": [
@@ -46,8 +52,9 @@
       null,
       null,
       true,
+      false,
       false
     ]
   },
-  "hash": "a0833b9899833166891c5de926f78632fae1123e736d728bf92cb2de004b6826"
+  "hash": "9b60fa8a1003015bc5a7cdbee9a4486b313d45347dfd9d4793d60e2760763ca3"
 }

backend/.sqlx/query-9c50e3a136a8ee3ec56e083f26d3a960b89e02ec40b292f3b5198baf2a1d3dbf.json

@@ -1,6 +1,6 @@
 {
   "db_name": "PostgreSQL",
-  "query": "\n        SELECT \n            id, \n            created_at, \n            trigger_kind AS \"trigger_kind: _\", \n            payload AS \"payload!: _\", \n            trigger_extra AS \"trigger_extra: _\"\n        FROM \n            capture\n        WHERE \n            id = $1 \n            AND workspace_id = $2\n        ",
+  "query": "\n        SELECT \n            id, \n            created_at, \n            trigger_kind AS \"trigger_kind: _\", \n            main_args AS \"main_args!: _\", \n            preprocessor_args AS \"preprocessor_args: _\"\n        FROM \n            capture\n        WHERE \n            id = $1 \n            AND workspace_id = $2\n        ",
   "describe": {
     "columns": [
       {
@@ -38,12 +38,12 @@
       },
       {
         "ordinal": 3,
-        "name": "payload!: _",
+        "name": "main_args!: _",
         "type_info": "Jsonb"
       },
       {
         "ordinal": 4,
-        "name": "trigger_extra: _",
+        "name": "preprocessor_args: _",
         "type_info": "Jsonb"
       }
     ],
@@ -61,5 +61,5 @@
       true
     ]
   },
-  "hash": "6781ba76dfce321dca4634566496ea5d698ac09d8264e35dfaa4cd8edc9e8414"
+  "hash": "9c50e3a136a8ee3ec56e083f26d3a960b89e02ec40b292f3b5198baf2a1d3dbf"
 }

backend/.sqlx/query-9dec888d4b0666d1843fbbc4fb2475fd947f047a9965bc110d7338208b77783d.json

@@ -0,0 +1,23 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "SELECT flow_version.id from flow\n                    INNER JOIN flow_version\n                    ON flow_version.id = flow.versions[array_upper(flow.versions, 1)]\n                    WHERE flow.path = $1 and flow.workspace_id = $2",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "id",
+        "type_info": "Int8"
+      }
+    ],
+    "parameters": {
+      "Left": [
+        "Text",
+        "Text"
+      ]
+    },
+    "nullable": [
+      false
+    ]
+  },
+  "hash": "9dec888d4b0666d1843fbbc4fb2475fd947f047a9965bc110d7338208b77783d"
+}

backend/.sqlx/query-a8bcae108af1eda6efe3a4b8c6f8807bc464a81c0883e68f5a69b89b94b0b34b.json

@@ -0,0 +1,70 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "SELECT has_preprocessor, language as \"language: _\", content, schema as \"schema: _\" FROM script WHERE workspace_id = $1 AND hash = $2",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "has_preprocessor",
+        "type_info": "Bool"
+      },
+      {
+        "ordinal": 1,
+        "name": "language: _",
+        "type_info": {
+          "Custom": {
+            "name": "script_lang",
+            "kind": {
+              "Enum": [
+                "python3",
+                "deno",
+                "go",
+                "bash",
+                "postgresql",
+                "nativets",
+                "bun",
+                "mysql",
+                "bigquery",
+                "snowflake",
+                "graphql",
+                "powershell",
+                "mssql",
+                "php",
+                "bunnative",
+                "rust",
+                "ansible",
+                "csharp",
+                "oracledb",
+                "nu",
+                "java"
+              ]
+            }
+          }
+        }
+      },
+      {
+        "ordinal": 2,
+        "name": "content",
+        "type_info": "Text"
+      },
+      {
+        "ordinal": 3,
+        "name": "schema: _",
+        "type_info": "Json"
+      }
+    ],
+    "parameters": {
+      "Left": [
+        "Text",
+        "Int8"
+      ]
+    },
+    "nullable": [
+      true,
+      false,
+      false,
+      true
+    ]
+  },
+  "hash": "a8bcae108af1eda6efe3a4b8c6f8807bc464a81c0883e68f5a69b89b94b0b34b"
+}

backend/.sqlx/query-a8ca4e588e0bf3c4bba2fe4b68a5364e4cba99964513599f8a012a5680d3dca8.json

@@ -0,0 +1,12 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "CREATE INDEX CONCURRENTLY IF NOT EXISTS alerts_by_workspace ON alerts (workspace_id);",
+  "describe": {
+    "columns": [],
+    "parameters": {
+      "Left": []
+    },
+    "nullable": []
+  },
+  "hash": "a8ca4e588e0bf3c4bba2fe4b68a5364e4cba99964513599f8a012a5680d3dca8"
+}

backend/.sqlx/query-b7335ac24702c86fbb4ab95916a6aa1648082287b09122755df2462dc71ce831.json

@@ -0,0 +1,46 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "\n        WITH j AS (\n            SELECT \n                raw_flow->>'concurrency_key' as concurrency_key, \n                raw_flow->>'concurrency_time_window_s' as concurrency_time_window_s,\n                raw_flow->>'concurrency_limit' as concurrent_limit,\n                runnable_path, \n                runnable_id as version FROM v2_job\n            WHERE id = $1\n        )\n        SELECT tag, j.concurrency_key, j.concurrency_time_window_s::int, j.concurrent_limit::int, j.version\n            FROM flow, j\n            WHERE path = j.runnable_path\n        ",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "tag",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 1,
+        "name": "concurrency_key",
+        "type_info": "Text"
+      },
+      {
+        "ordinal": 2,
+        "name": "concurrency_time_window_s",
+        "type_info": "Int4"
+      },
+      {
+        "ordinal": 3,
+        "name": "concurrent_limit",
+        "type_info": "Int4"
+      },
+      {
+        "ordinal": 4,
+        "name": "version",
+        "type_info": "Int8"
+      }
+    ],
+    "parameters": {
+      "Left": [
+        "Uuid"
+      ]
+    },
+    "nullable": [
+      true,
+      null,
+      null,
+      null,
+      true
+    ]
+  },
+  "hash": "b7335ac24702c86fbb4ab95916a6aa1648082287b09122755df2462dc71ce831"
+}

backend/.sqlx/query-c51f9ad5133c46fd7c499b8339dbbf3f3059bbb85de07ee3b4b4cea971984a52.json

@@ -0,0 +1,132 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "\n                SELECT\n                    mqtt_resource_path,\n                    subscribe_topics as \"subscribe_topics: _\",\n                    v3_config as \"v3_config: _\",\n                    v5_config as \"v5_config: _\",\n                    client_version AS \"client_version: _\",\n                    client_id,\n                    workspace_id,\n                    path,\n                    script_path,\n                    is_flow,\n                    edited_by,\n                    email,\n                    edited_at,\n                    server_id,\n                    last_server_ping,\n                    extra_perms,\n                    error,\n                    enabled\n                FROM \n                    mqtt_trigger\n                ",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "mqtt_resource_path",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 1,
+        "name": "subscribe_topics: _",
+        "type_info": "JsonbArray"
+      },
+      {
+        "ordinal": 2,
+        "name": "v3_config: _",
+        "type_info": "Jsonb"
+      },
+      {
+        "ordinal": 3,
+        "name": "v5_config: _",
+        "type_info": "Jsonb"
+      },
+      {
+        "ordinal": 4,
+        "name": "client_version: _",
+        "type_info": {
+          "Custom": {
+            "name": "mqtt_client_version",
+            "kind": {
+              "Enum": [
+                "v3",
+                "v5"
+              ]
+            }
+          }
+        }
+      },
+      {
+        "ordinal": 5,
+        "name": "client_id",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 6,
+        "name": "workspace_id",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 7,
+        "name": "path",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 8,
+        "name": "script_path",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 9,
+        "name": "is_flow",
+        "type_info": "Bool"
+      },
+      {
+        "ordinal": 10,
+        "name": "edited_by",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 11,
+        "name": "email",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 12,
+        "name": "edited_at",
+        "type_info": "Timestamptz"
+      },
+      {
+        "ordinal": 13,
+        "name": "server_id",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 14,
+        "name": "last_server_ping",
+        "type_info": "Timestamptz"
+      },
+      {
+        "ordinal": 15,
+        "name": "extra_perms",
+        "type_info": "Jsonb"
+      },
+      {
+        "ordinal": 16,
+        "name": "error",
+        "type_info": "Text"
+      },
+      {
+        "ordinal": 17,
+        "name": "enabled",
+        "type_info": "Bool"
+      }
+    ],
+    "parameters": {
+      "Left": []
+    },
+    "nullable": [
+      false,
+      false,
+      true,
+      true,
+      false,
+      true,
+      false,
+      false,
+      false,
+      false,
+      false,
+      false,
+      false,
+      true,
+      true,
+      false,
+      true,
+      false
+    ]
+  },
+  "hash": "c51f9ad5133c46fd7c499b8339dbbf3f3059bbb85de07ee3b4b4cea971984a52"
+}

backend/.sqlx/query-c794ff5e14429a1bc56f5e70cfd10e2160d1b7e123c7ab226dc77d1789f164f9.json

@@ -1,47 +0,0 @@
-{
-  "db_name": "PostgreSQL",
-  "query": "SELECT tag, dedicated_worker, flow_version.value->>'preprocessor_module' IS NOT NULL as has_preprocessor, on_behalf_of_email, edited_by\n        FROM flow \n        LEFT JOIN flow_version\n            ON flow_version.id = flow.versions[array_upper(flow.versions, 1)]\n        WHERE flow.path = $1 and flow.workspace_id = $2",
-  "describe": {
-    "columns": [
-      {
-        "ordinal": 0,
-        "name": "tag",
-        "type_info": "Varchar"
-      },
-      {
-        "ordinal": 1,
-        "name": "dedicated_worker",
-        "type_info": "Bool"
-      },
-      {
-        "ordinal": 2,
-        "name": "has_preprocessor",
-        "type_info": "Bool"
-      },
-      {
-        "ordinal": 3,
-        "name": "on_behalf_of_email",
-        "type_info": "Text"
-      },
-      {
-        "ordinal": 4,
-        "name": "edited_by",
-        "type_info": "Varchar"
-      }
-    ],
-    "parameters": {
-      "Left": [
-        "Text",
-        "Text"
-      ]
-    },
-    "nullable": [
-      true,
-      true,
-      null,
-      true,
-      false
-    ]
-  },
-  "hash": "c794ff5e14429a1bc56f5e70cfd10e2160d1b7e123c7ab226dc77d1789f164f9"
-}

backend/.sqlx/query-e07660e8d2a265cb6a83f3a2bb8e7e6330f09ab116e9837f6f16f8fdef938004.json

@@ -0,0 +1,18 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "WITH job_result AS (\n                 SELECT result \n                 FROM v2_job_completed \n                 WHERE id = $1\n             ),\n             updated_queue AS (\n                UPDATE v2_job_queue\n                SET running = false,\n                tag = COALESCE($3, tag)\n                WHERE id = $2\n             )\n             UPDATE v2_job \n             SET \n                tag = COALESCE($3, tag),\n                concurrent_limit = COALESCE($4, concurrent_limit),\n                concurrency_time_window_s = COALESCE($5, concurrency_time_window_s),\n                args = COALESCE(\n                     CASE \n                         WHEN job_result.result IS NULL THEN NULL\n                         WHEN jsonb_typeof(job_result.result) = 'object' \n                         THEN job_result.result\n                         WHEN jsonb_typeof(job_result.result) = 'null'\n                         THEN NULL\n                         ELSE jsonb_build_object('value', job_result.result)\n                     END, \n                     '{}'::jsonb\n                 ),\n                 preprocessed = TRUE\n             FROM job_result\n             WHERE v2_job.id = $2;\n             ",
+  "describe": {
+    "columns": [],
+    "parameters": {
+      "Left": [
+        "Uuid",
+        "Uuid",
+        "Varchar",
+        "Int4",
+        "Int4"
+      ]
+    },
+    "nullable": []
+  },
+  "hash": "e07660e8d2a265cb6a83f3a2bb8e7e6330f09ab116e9837f6f16f8fdef938004"
+}

backend/.sqlx/query-e7348225a27bbdc9607d7c799e7192cd6ce4088467d91a5cbdc019430320d26d.json

@@ -0,0 +1,29 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "SELECT\n                    value->'preprocessor_module'->'value' as \"preprocessor_module: _\",\n                    schema as \"schema: _\"\n                FROM flow_version\n                WHERE \n                    path = $1\n                    AND workspace_id = $2\n                ORDER BY created_at DESC\n                LIMIT 1",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "preprocessor_module: _",
+        "type_info": "Jsonb"
+      },
+      {
+        "ordinal": 1,
+        "name": "schema: _",
+        "type_info": "Json"
+      }
+    ],
+    "parameters": {
+      "Left": [
+        "Text",
+        "Text"
+      ]
+    },
+    "nullable": [
+      null,
+      true
+    ]
+  },
+  "hash": "e7348225a27bbdc9607d7c799e7192cd6ce4088467d91a5cbdc019430320d26d"
+}

backend/.sqlx/query-eac595e19e5c8e70f1514ef29dec35c7342ac9a814c73f6290e1d6ebd3a55423.json

@@ -1,6 +1,6 @@
 {
   "db_name": "PostgreSQL",
-  "query": "\n    INSERT INTO \n        capture (\n            workspace_id, path, is_flow, trigger_kind, payload, trigger_extra, created_by\n        )\n    VALUES (\n        $1, $2, $3, $4, $5, $6, $7\n    )\n    ",
+  "query": "\n    INSERT INTO \n        capture (\n            workspace_id, path, is_flow, trigger_kind, main_args, preprocessor_args, created_by\n        )\n    VALUES (\n        $1, $2, $3, $4, $5, $6, $7\n    )\n    ",
   "describe": {
     "columns": [],
     "parameters": {
@@ -34,5 +34,5 @@
     },
     "nullable": []
   },
-  "hash": "f9f7f31390f8ea1f4facd8e6a888886ea136f2327169bf93f126e2ef130d0946"
+  "hash": "eac595e19e5c8e70f1514ef29dec35c7342ac9a814c73f6290e1d6ebd3a55423"
 }

backend/.sqlx/query-ec7836df5f9056ec70015800b7f4feaeb1b671120f5f8c98fca8c89c6587fc35.json

@@ -1,118 +0,0 @@
-{
-  "db_name": "PostgreSQL",
-  "query": "select tag, concurrency_key, concurrent_limit, concurrency_time_window_s, cache_ttl, language as \"language: ScriptLang\", dedicated_worker, priority, delete_after_use, timeout, on_behalf_of_email, created_by from script where hash = $1 AND workspace_id = $2",
-  "describe": {
-    "columns": [
-      {
-        "ordinal": 0,
-        "name": "tag",
-        "type_info": "Varchar"
-      },
-      {
-        "ordinal": 1,
-        "name": "concurrency_key",
-        "type_info": "Varchar"
-      },
-      {
-        "ordinal": 2,
-        "name": "concurrent_limit",
-        "type_info": "Int4"
-      },
-      {
-        "ordinal": 3,
-        "name": "concurrency_time_window_s",
-        "type_info": "Int4"
-      },
-      {
-        "ordinal": 4,
-        "name": "cache_ttl",
-        "type_info": "Int4"
-      },
-      {
-        "ordinal": 5,
-        "name": "language: ScriptLang",
-        "type_info": {
-          "Custom": {
-            "name": "script_lang",
-            "kind": {
-              "Enum": [
-                "python3",
-                "deno",
-                "go",
-                "bash",
-                "postgresql",
-                "nativets",
-                "bun",
-                "mysql",
-                "bigquery",
-                "snowflake",
-                "graphql",
-                "powershell",
-                "mssql",
-                "php",
-                "bunnative",
-                "rust",
-                "ansible",
-                "csharp",
-                "oracledb",
-                "nu",
-                "java"
-              ]
-            }
-          }
-        }
-      },
-      {
-        "ordinal": 6,
-        "name": "dedicated_worker",
-        "type_info": "Bool"
-      },
-      {
-        "ordinal": 7,
-        "name": "priority",
-        "type_info": "Int2"
-      },
-      {
-        "ordinal": 8,
-        "name": "delete_after_use",
-        "type_info": "Bool"
-      },
-      {
-        "ordinal": 9,
-        "name": "timeout",
-        "type_info": "Int4"
-      },
-      {
-        "ordinal": 10,
-        "name": "on_behalf_of_email",
-        "type_info": "Text"
-      },
-      {
-        "ordinal": 11,
-        "name": "created_by",
-        "type_info": "Varchar"
-      }
-    ],
-    "parameters": {
-      "Left": [
-        "Int8",
-        "Text"
-      ]
-    },
-    "nullable": [
-      true,
-      true,
-      true,
-      true,
-      true,
-      false,
-      true,
-      true,
-      true,
-      true,
-      true,
-      false
-    ]
-  },
-  "hash": "ec7836df5f9056ec70015800b7f4feaeb1b671120f5f8c98fca8c89c6587fc35"
-}

backend/.sqlx/query-f0fdeb7aea3e71099e7db0f4343bbd7ec86610ddc8589bf5b606fab0947c8b75.json

@@ -1,91 +0,0 @@
-{
-  "db_name": "PostgreSQL",
-  "query": "SELECT\n            v2_as_queue.job_kind AS \"job_kind!: JobKind\",\n            v2_as_queue.script_hash AS \"script_hash: ScriptHash\",\n            v2_as_queue.raw_flow AS \"raw_flow: sqlx::types::Json<Box<RawValue>>\",\n            v2_as_completed_job.parent_job AS \"parent_job: Uuid\",\n            v2_as_completed_job.created_at AS \"created_at!: chrono::NaiveDateTime\",\n            v2_as_completed_job.created_by AS \"created_by!\",\n            v2_as_queue.script_path,\n            v2_as_queue.args AS \"args: sqlx::types::Json<Box<RawValue>>\"\n        FROM v2_as_queue\n        JOIN v2_as_completed_job ON v2_as_completed_job.parent_job = v2_as_queue.id\n        WHERE v2_as_completed_job.id = $1 AND v2_as_completed_job.workspace_id = $2\n        LIMIT 1",
-  "describe": {
-    "columns": [
-      {
-        "ordinal": 0,
-        "name": "job_kind!: JobKind",
-        "type_info": {
-          "Custom": {
-            "name": "job_kind",
-            "kind": {
-              "Enum": [
-                "script",
-                "preview",
-                "flow",
-                "dependencies",
-                "flowpreview",
-                "script_hub",
-                "identity",
-                "flowdependencies",
-                "http",
-                "graphql",
-                "postgresql",
-                "noop",
-                "appdependencies",
-                "deploymentcallback",
-                "singlescriptflow",
-                "flowscript",
-                "flownode",
-                "appscript"
-              ]
-            }
-          }
-        }
-      },
-      {
-        "ordinal": 1,
-        "name": "script_hash: ScriptHash",
-        "type_info": "Int8"
-      },
-      {
-        "ordinal": 2,
-        "name": "raw_flow: sqlx::types::Json<Box<RawValue>>",
-        "type_info": "Jsonb"
-      },
-      {
-        "ordinal": 3,
-        "name": "parent_job: Uuid",
-        "type_info": "Uuid"
-      },
-      {
-        "ordinal": 4,
-        "name": "created_at!: chrono::NaiveDateTime",
-        "type_info": "Timestamptz"
-      },
-      {
-        "ordinal": 5,
-        "name": "created_by!",
-        "type_info": "Varchar"
-      },
-      {
-        "ordinal": 6,
-        "name": "script_path",
-        "type_info": "Varchar"
-      },
-      {
-        "ordinal": 7,
-        "name": "args: sqlx::types::Json<Box<RawValue>>",
-        "type_info": "Jsonb"
-      }
-    ],
-    "parameters": {
-      "Left": [
-        "Uuid",
-        "Text"
-      ]
-    },
-    "nullable": [
-      true,
-      true,
-      true,
-      true,
-      true,
-      true,
-      true,
-      true
-    ]
-  },
-  "hash": "f0fdeb7aea3e71099e7db0f4343bbd7ec86610ddc8589bf5b606fab0947c8b75"
-}

backend/.sqlx/query-f22964772dc2d67aee437bbbd08b64792c00da1d713d7ca8f9904ccce7bfdae7.json

@@ -0,0 +1,246 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "SELECT\n                            cj.id AS \"id!\",\n                            cj.workspace_id AS \"workspace_id!\",\n                            cj.parent_job,\n                            cj.created_by AS \"created_by!\",\n                            cj.duration_ms AS \"duration_ms!\",\n                            cj.success AS \"success!\",\n                            cj.script_hash AS \"script_hash!: Option<ScriptHash>\",\n                            cj.script_path,\n                            cj.args AS \"args: sqlx::types::Json<HashMap<String, Box<RawValue>>>\",\n                            cj.result AS \"result: sqlx::types::Json<Box<RawValue>>\",\n                            cj.deleted AS \"deleted!\",\n                            cj.canceled AS \"canceled!\",\n                            cj.canceled_by,\n                            cj.canceled_reason,\n                            cj.job_kind AS \"job_kind!: JobKind\",\n                            cj.schedule_path,\n                            cj.permissioned_as AS \"permissioned_as!\",\n                            cj.is_flow_step AS \"is_flow_step!\",\n                            cj.language AS \"language: ScriptLang\",\n                            cj.is_skipped AS \"is_skipped!\",\n                            cj.email AS \"email!\",\n                            cj.visible_to_owner AS \"visible_to_owner!\",\n                            cj.mem_peak,\n                            cj.tag AS \"tag!\",\n                            cj.created_at AS \"created_at!\",\n                            cj.started_at,\n                            job_logs.logs,\n                            job_logs.log_offset AS \"log_offset?\",\n                            job_logs.log_file_index\n\n                        FROM v2_as_completed_job AS cj\n                        LEFT JOIN job_logs ON cj.id = job_logs.job_id\n                        WHERE cj.created_at < $1\n                        ORDER BY cj.created_at ASC LIMIT $2",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "id!",
+        "type_info": "Uuid"
+      },
+      {
+        "ordinal": 1,
+        "name": "workspace_id!",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 2,
+        "name": "parent_job",
+        "type_info": "Uuid"
+      },
+      {
+        "ordinal": 3,
+        "name": "created_by!",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 4,
+        "name": "duration_ms!",
+        "type_info": "Int8"
+      },
+      {
+        "ordinal": 5,
+        "name": "success!",
+        "type_info": "Bool"
+      },
+      {
+        "ordinal": 6,
+        "name": "script_hash!: Option<ScriptHash>",
+        "type_info": "Int8"
+      },
+      {
+        "ordinal": 7,
+        "name": "script_path",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 8,
+        "name": "args: sqlx::types::Json<HashMap<String, Box<RawValue>>>",
+        "type_info": "Jsonb"
+      },
+      {
+        "ordinal": 9,
+        "name": "result: sqlx::types::Json<Box<RawValue>>",
+        "type_info": "Jsonb"
+      },
+      {
+        "ordinal": 10,
+        "name": "deleted!",
+        "type_info": "Bool"
+      },
+      {
+        "ordinal": 11,
+        "name": "canceled!",
+        "type_info": "Bool"
+      },
+      {
+        "ordinal": 12,
+        "name": "canceled_by",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 13,
+        "name": "canceled_reason",
+        "type_info": "Text"
+      },
+      {
+        "ordinal": 14,
+        "name": "job_kind!: JobKind",
+        "type_info": {
+          "Custom": {
+            "name": "job_kind",
+            "kind": {
+              "Enum": [
+                "script",
+                "preview",
+                "flow",
+                "dependencies",
+                "flowpreview",
+                "script_hub",
+                "identity",
+                "flowdependencies",
+                "http",
+                "graphql",
+                "postgresql",
+                "noop",
+                "appdependencies",
+                "deploymentcallback",
+                "singlescriptflow",
+                "flowscript",
+                "flownode",
+                "appscript"
+              ]
+            }
+          }
+        }
+      },
+      {
+        "ordinal": 15,
+        "name": "schedule_path",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 16,
+        "name": "permissioned_as!",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 17,
+        "name": "is_flow_step!",
+        "type_info": "Bool"
+      },
+      {
+        "ordinal": 18,
+        "name": "language: ScriptLang",
+        "type_info": {
+          "Custom": {
+            "name": "script_lang",
+            "kind": {
+              "Enum": [
+                "python3",
+                "deno",
+                "go",
+                "bash",
+                "postgresql",
+                "nativets",
+                "bun",
+                "mysql",
+                "bigquery",
+                "snowflake",
+                "graphql",
+                "powershell",
+                "mssql",
+                "php",
+                "bunnative",
+                "rust",
+                "ansible",
+                "csharp",
+                "oracledb",
+                "nu",
+                "java"
+              ]
+            }
+          }
+        }
+      },
+      {
+        "ordinal": 19,
+        "name": "is_skipped!",
+        "type_info": "Bool"
+      },
+      {
+        "ordinal": 20,
+        "name": "email!",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 21,
+        "name": "visible_to_owner!",
+        "type_info": "Bool"
+      },
+      {
+        "ordinal": 22,
+        "name": "mem_peak",
+        "type_info": "Int4"
+      },
+      {
+        "ordinal": 23,
+        "name": "tag!",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 24,
+        "name": "created_at!",
+        "type_info": "Timestamptz"
+      },
+      {
+        "ordinal": 25,
+        "name": "started_at",
+        "type_info": "Timestamptz"
+      },
+      {
+        "ordinal": 26,
+        "name": "logs",
+        "type_info": "Text"
+      },
+      {
+        "ordinal": 27,
+        "name": "log_offset?",
+        "type_info": "Int4"
+      },
+      {
+        "ordinal": 28,
+        "name": "log_file_index",
+        "type_info": "TextArray"
+      }
+    ],
+    "parameters": {
+      "Left": [
+        "Timestamptz",
+        "Int8"
+      ]
+    },
+    "nullable": [
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      true,
+      false,
+      true
+    ]
+  },
+  "hash": "f22964772dc2d67aee437bbbd08b64792c00da1d713d7ca8f9904ccce7bfdae7"
+}

backend/.sqlx/query-f44f1a794ab217be05a9feb21e57b3409d9166a028701981cfc280aaab3c4436.json

@@ -1,41 +0,0 @@
-{
-  "db_name": "PostgreSQL",
-  "query": "SELECT tag, dedicated_worker, on_behalf_of_email, edited_by from flow WHERE path = $1 and workspace_id = $2",
-  "describe": {
-    "columns": [
-      {
-        "ordinal": 0,
-        "name": "tag",
-        "type_info": "Varchar"
-      },
-      {
-        "ordinal": 1,
-        "name": "dedicated_worker",
-        "type_info": "Bool"
-      },
-      {
-        "ordinal": 2,
-        "name": "on_behalf_of_email",
-        "type_info": "Text"
-      },
-      {
-        "ordinal": 3,
-        "name": "edited_by",
-        "type_info": "Varchar"
-      }
-    ],
-    "parameters": {
-      "Left": [
-        "Text",
-        "Text"
-      ]
-    },
-    "nullable": [
-      true,
-      true,
-      true,
-      false
-    ]
-  },
-  "hash": "f44f1a794ab217be05a9feb21e57b3409d9166a028701981cfc280aaab3c4436"
-}

backend/.sqlx/query-f632d08a8d3df691fff9f57fdf926f787287c3cd181a6853056077edba10473d.json

@@ -1,23 +0,0 @@
-{
-  "db_name": "PostgreSQL",
-  "query": "\n                SELECT EXISTS(\n                    SELECT 1 \n                    FROM \n                        http_trigger \n                    WHERE \n                        workspace_id = $1 AND \n                        path = $2\n                )\n                ",
-  "describe": {
-    "columns": [
-      {
-        "ordinal": 0,
-        "name": "exists",
-        "type_info": "Bool"
-      }
-    ],
-    "parameters": {
-      "Left": [
-        "Text",
-        "Text"
-      ]
-    },
-    "nullable": [
-      null
-    ]
-  },
-  "hash": "f632d08a8d3df691fff9f57fdf926f787287c3cd181a6853056077edba10473d"
-}

backend/CLAUDE.md

@@ -0,0 +1,104 @@
+# Windmill Backend - Rust Best Practices
+
+## Project Structure
+
+Windmill uses a workspace-based architecture with multiple crates:
+
+- **windmill-api**: API server functionality
+- **windmill-worker**: Job execution
+- **windmill-common**: Shared code used by all crates
+- **windmill-queue**: Job & flow queuing
+- **windmill-audit**: Audit logging
+- Other specialized crates (git-sync, autoscaling, etc.)
+
+## Adding New Code
+
+### Module Organization
+
+- Place new code in the appropriate crate based on functionality
+- For API endpoints, create or modify files in `windmill-api/src/` organized by domain
+- For shared functionality, use `windmill-common/src/`
+- Use the `_ee.rs` suffix for enterprise-only modules
+- Follow existing patterns for file structure and organization
+
+### Error Handling
+
+- Use the custom `Error` enum from `windmill-common::error`
+- Return `Result<T, Error>` or `JsonResult<T>` for functions that can fail
+- Use the `?` operator for error propagation
+- Add location tracking to errors using `#[track_caller]`
+
+### Database Operations
+
+- Use `sqlx` for database operations with prepared statements
+- Leverage existing database helper functions in `db.rs` modules
+- Use transactions for multi-step operations
+- Handle database errors properly
+
+### API Endpoints
+
+- Follow existing patterns in the `windmill-api` crate
+- Use axum's routing system and extractors
+- Group related routes together
+- Use consistent response formats (JSON)
+- Follow proper authentication and authorization patterns
+
+## Performance Optimizations
+
+When generating code, especially involving `serde`, `sqlx`, and `tokio`, prioritize performance by applying the following principles:
+
+### Serde Optimizations (Serialization & Deserialization)
+
+- **Specify Structure Explicitly:** When defining structs for Serde (`#[derive(Serialize, Deserialize)]`), use `#[serde(...` attributes extensively. This includes:
+  - `#[serde(rename = "...")]` or `#[serde(alias = "...")]` to map external names precisely, avoiding dynamic lookups.
+  - `#[serde(default)]` for optional fields with default values, reducing parsing complexity.
+  - `#[serde(skip_serializing_if = "...")]` to avoid writing fields that meet a certain condition (e.g., `Option::is_none()`, `Vec::is_empty()`, or a custom function), reducing output size and serialization work.
+  - `#[serde(skip_serializing)]` or `#[serde(skip_deserializing)]` for fields that should _not_ be included.
+- **Prefer Borrowing:** Where possible and safe (data lifetime allows), use `Cow<'a, str>` or `&'a str` (with `#[serde(borrow)]`) instead of `String` for string fields during deserialization. This avoids allocating new strings, enabling zero-copy reading from the input buffer. Apply this principle to byte slices (`&'a [u8]` / `Cow<'a, [u8]>`) and potentially borrowed vectors as well.
+- **Avoid Intermediate `Value`:** Unless the data structure is truly dynamic or unknown at compile time, deserialize directly into a well-defined struct or enum rather than into `serde_json::Value` (or equivalent for other formats). This avoids unnecessary heap allocations and type switching.
+
+### SQLx Optimizations (Database Interaction)
+
+- **Select Only Necessary Columns:** In `SELECT` queries, list specific column names rather than using `SELECT *`. This reduces data transferred from the database and the work needed for hydration/deserialization.
+- **Batch Operations:** For multiple `INSERT`, `UPDATE`, or `DELETE` statements, prefer executing them in a single query if the database and driver support it efficiently (e.g., `INSERT INTO ... VALUES (...), (...), ...`). This minimizes round trips to the database.
+- **Avoid N+1 Queries:** Do not loop through results of one query and execute a separate query for each item (e.g., fetching users, then querying for each user's profile in a loop). Instead, use JOINs or a single query with an `IN` clause to fetch related data efficiently.
+- **Deserialize Directly:** Use `#[derive(FromRow)]` on structs and ensure the struct fields match the selected columns in the query. This allows SQLx to hydrate objects directly, avoiding intermediate data structures.
+- **Parameterize Queries:** Always use SQLx's query methods (`.bind(...)`) to pass values as parameters rather than string formatting. This prevents SQL injection and allows the database to cache query plans, improving performance on repeated executions.
+
+### Tokio Optimizations (Asynchronous Runtime)
+
+- **Avoid Blocking Operations:** **Crucially**, never perform blocking operations (synchronous file I/O, `std::thread::sleep`, CPU-bound loops, `std::sync::Mutex::lock`, blocking network calls without `tokio::net`) directly within an `async fn` or a standard `tokio::spawn` task. Blocking pauses the entire worker thread, potentially starving other tasks. Use `tokio::task::spawn_blocking` for CPU-intensive work or blocking I/O.
+- **Use Tokio's Async Primitives:** Prefer `tokio::sync` (channels, mutexes, semaphores), `tokio::io`, `tokio::net`, and `tokio::time` over their `std` counterparts in asynchronous contexts. These are designed to yield control back to the scheduler.
+- **Manage Concurrency:** Be mindful of how many tasks are spawned. Creating a new task for every tiny piece of work can introduce overhead. Group related asynchronous operations where appropriate.
+- **Handle Shared State Efficiently:** Use `Arc` for shared ownership in concurrent tasks. When shared state needs mutation, prefer `tokio::sync::Mutex` over `std::sync::Mutex` in `async` code. Consider `tokio::sync::RwLock` if reads significantly outnumber writes. Minimize the duration for which locks are held.
+- **Understand `.await`:** Place `.await` strategically to allow the runtime to switch to other ready tasks. Ensure that `.await` points to genuinely asynchronous operations.
+- **Backpressure:** If dealing with data streams or queues between tasks, implement backpressure mechanisms (e.g., bounded channels like `tokio::sync::mpsc::channel`) to prevent one component from overwhelming another or critical resources like the database.
+
+## Enterprise Features
+
+- Use feature flags for enterprise functionality
+- Conditionally compile with `#[cfg(feature = "enterprise")]`
+- Isolate enterprise code in separate modules
+
+## Code Style
+
+- Group imports by external and internal crates
+- Place struct/enum definitions before implementations
+- Group similar functionality together
+- Use descriptive naming consistent with the codebase
+- Follow existing patterns for async code using tokio
+
+## Testing
+
+- Write unit tests for core functionality
+- Use the `#[cfg(test)]` module for test code
+- For database tests, use the existing test utilities
+
+## Common Crates Used
+
+- **tokio**: For async runtime
+- **axum**: For web server and routing
+- **sqlx**: For database operations
+- **serde**: For serialization/deserialization
+- **tracing**: For logging and diagnostics
+- **reqwest**: For HTTP client functionality

backend/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "windmill"
-version = "1.489.0"
+version = "1.491.5"
 authors.workspace = true
 edition.workspace = true
 
@@ -32,7 +32,7 @@ members = [
 ]
 
 [workspace.package]
-version = "1.489.0"
+version = "1.491.5"
 authors = ["Ruben Fiszel <ruben@windmill.dev>"]
 edition = "2021"
 
@@ -137,6 +137,7 @@ v8 = { workspace = true, optional = true }
 rustls.workspace = true
 systemstat.workspace = true
 size.workspace = true
+strum.workspace = true
 
 [target.'cfg(not(target_env = "msvc"))'.dependencies]
 tikv-jemallocator = { optional = true, workspace = true }
@@ -235,7 +236,7 @@ itertools = "^0"
 regex = "^1"
 semver = "^1"
 
-v8 = "=130.0.7"  # Exact version
+v8 = "=130.0.7"  # Exact version NOTE: Do not forget to update version and hash in flake.nix
 deno_fetch = "0.214.0"
 deno_tls = "0.177.0"
 deno_console = "0.190.0"
@@ -389,3 +390,5 @@ tree-sitter-c-sharp = "0.23.0"
 tree-sitter-java = "0.23.0"
 oracle = { version = "0.6.3", features = ["chrono"] }
 rumqttc = { version = "0.24.0", features = ["use-native-tls"]}
+strum = "^0"
+strum_macros = "^0"

backend/ee-repo-ref.txt

@@ -1 +1 @@
-19b5681d1985ee5214f768e3b818faf6ca7d20b2
+bea87fa885dc041fba83b2491609a4a2cdbbfa6f

backend/migrations/20250414082127_update_capture_format.down.sql

@@ -0,0 +1,2 @@
+ALTER TABLE capture RENAME COLUMN preprocessor_args to trigger_extra;
+ALTER TABLE capture RENAME COLUMN main_args to payload;

backend/migrations/20250414082127_update_capture_format.up.sql

@@ -0,0 +1,2 @@
+ALTER TABLE capture RENAME COLUMN trigger_extra to preprocessor_args;
+ALTER TABLE capture RENAME COLUMN payload to main_args;

backend/migrations/20250506092215_runnable_version_notify.down.sql

@@ -0,0 +1,4 @@
+-- Add down migration script here
+DROP TRIGGER script_update_trigger ON script;
+DROP TRIGGER flow_update_trigger ON flow_version;
+DROP FUNCTION notify_runnable_version_change();

backend/migrations/20250506092215_runnable_version_notify.up.sql

@@ -0,0 +1,22 @@
+-- Add up migration script here
+CREATE OR REPLACE FUNCTION notify_runnable_version_change()
+RETURNS TRIGGER AS $$
+DECLARE
+    source_type TEXT;
+BEGIN
+    source_type := TG_ARGV[0];
+
+    PERFORM pg_notify('notify_runnable_version_change', NEW.workspace_id || ':' || source_type || ':' || NEW.path);
+    RETURN NEW;
+END;
+$$ LANGUAGE plpgsql;
+
+CREATE TRIGGER script_update_trigger
+AFTER UPDATE OF lock ON script
+FOR EACH ROW
+EXECUTE FUNCTION notify_runnable_version_change('script');
+
+CREATE TRIGGER flow_update_trigger
+AFTER INSERT ON flow_version
+FOR EACH ROW
+EXECUTE FUNCTION notify_runnable_version_change('flow');

backend/migrations/20250514120017_http_trigger_update_notify.down.sql

@@ -0,0 +1,4 @@
+-- Add down migration script here
+DROP TRIGGER http_trigger_change_trigger ON http_trigger;
+DROP FUNCTION notify_http_trigger_change();
+DROP SEQUENCE http_trigger_version_seq;

backend/migrations/20250514120017_http_trigger_update_notify.up.sql

@@ -0,0 +1,15 @@
+-- Add up migration script here
+CREATE OR REPLACE FUNCTION notify_http_trigger_change()
+RETURNS TRIGGER AS $$
+BEGIN
+    PERFORM pg_notify('notify_http_trigger_change', NEW.workspace_id || ':' || NEW.path);
+    RETURN NEW;
+END;
+$$ LANGUAGE plpgsql;
+CREATE TRIGGER http_trigger_change_trigger
+AFTER INSERT OR UPDATE OR DELETE ON http_trigger
+FOR EACH ROW
+EXECUTE FUNCTION notify_http_trigger_change();
+
+CREATE SEQUENCE http_trigger_version_seq;
+

backend/migrations/20250515155403_add_grant_on_dependency_map.down.sql

@@ -0,0 +1 @@
+-- Add down migration script here

backend/migrations/20250515155403_add_grant_on_dependency_map.up.sql

@@ -0,0 +1,3 @@
+-- Add up migration script here
+GRANT ALL ON dependency_map TO windmill_user;
+GRANT ALL ON dependency_map TO windmill_admin;

backend/migrations/20250515181903_fix_http_routers_cache.down.sql

@@ -0,0 +1 @@
+-- Add down migration script here

backend/migrations/20250515181903_fix_http_routers_cache.up.sql

@@ -0,0 +1,9 @@
+-- Add up migration script here
+-- this makes sure that the first time nextval is called, 2 is returned
+-- otherwise, `SELECT last_value from http_trigger_version_seq;` would return 1 before and after the first nextval call
+-- which would not refresh the routers cache after the first create/update/delete
+SELECT setval(                                  
+    'http_trigger_version_seq',
+    (SELECT last_value FROM http_trigger_version_seq),
+    true
+);

backend/migrations/20250520104210_workspace_preprocessor_notify.down.sql

@@ -0,0 +1,12 @@
+-- Add down migration script here
+CREATE OR REPLACE FUNCTION notify_runnable_version_change()
+RETURNS TRIGGER AS $$
+DECLARE
+    source_type TEXT;
+BEGIN
+    source_type := TG_ARGV[0];
+
+    PERFORM pg_notify('notify_runnable_version_change', NEW.workspace_id || ':' || source_type || ':' || NEW.path);
+    RETURN NEW;
+END;
+$$ LANGUAGE plpgsql;

backend/migrations/20250520104210_workspace_preprocessor_notify.up.sql

@@ -0,0 +1,19 @@
+-- Add up migration script here
+CREATE OR REPLACE FUNCTION notify_runnable_version_change()
+RETURNS TRIGGER AS $$
+DECLARE
+    source_type TEXT;
+    kind TEXT;
+BEGIN
+    source_type := TG_ARGV[0];
+
+    IF source_type = 'script' THEN
+        kind := NEW.kind;
+    ELSE
+        kind := 'flow';
+    END IF;
+
+    PERFORM pg_notify('notify_runnable_version_change', NEW.workspace_id || ':' || source_type || ':' || NEW.path || ':' || kind);
+    RETURN NEW;
+END;
+$$ LANGUAGE plpgsql;

backend/parsers/windmill-parser-sql/src/lib.rs

@@ -120,6 +120,60 @@ pub fn parse_db_resource(code: &str) -> Option<String> {
     cap.map(|x| x.get(1).map(|x| x.as_str().to_string()).unwrap())
 }
 
+#[derive(Clone, Copy, Debug)]
+pub enum S3ModeFormat {
+    Json,
+    Csv,
+    Parquet,
+}
+pub fn s3_mode_extension(format: S3ModeFormat) -> &'static str {
+    match format {
+        S3ModeFormat::Json => "json",
+        S3ModeFormat::Csv => "csv",
+        S3ModeFormat::Parquet => "parquet",
+    }
+}
+pub struct S3ModeArgs {
+    pub prefix: Option<String>,
+    pub storage: Option<String>,
+    pub format: S3ModeFormat,
+}
+pub fn parse_s3_mode(code: &str) -> anyhow::Result<Option<S3ModeArgs>> {
+    let cap = match RE_S3_MODE.captures(code) {
+        Some(x) => x,
+        None => return Ok(None),
+    };
+    let args_str = cap
+        .get(1)
+        .map(|x| x.as_str().to_string())
+        .unwrap_or_default();
+
+    let mut prefix = None;
+    let mut storage = None;
+    let mut format = S3ModeFormat::Json;
+
+    for kv in args_str.split(' ').map(|kv| kv.trim()) {
+        if kv.is_empty() {
+            continue;
+        }
+        let mut it = kv.split('=');
+        let (Some(key), Some(value)) = (it.next(), it.next()) else {
+            return Err(anyhow!("Invalid S3 mode argument: {}", kv));
+        };
+        match (key.trim(), value.trim()) {
+            ("prefix", _) => prefix = Some(value.to_string()),
+            ("storage", _) => storage = Some(value.to_string()),
+            ("format", "json") => format = S3ModeFormat::Json,
+            ("format", "parquet") => format = S3ModeFormat::Parquet,
+            ("format", "csv") => format = S3ModeFormat::Csv,
+            ("format", format) => return Err(anyhow!("Invalid S3 mode format: {}", format)),
+            (_, _) => return Err(anyhow!("Invalid S3 mode argument: {}", kv)),
+        }
+    }
+
+    Ok(Some(S3ModeArgs { prefix, storage, format }))
+}
+
 pub fn parse_sql_blocks(code: &str) -> Vec<&str> {
     let mut blocks = vec![];
     let mut last_idx = 0;
@@ -147,6 +201,7 @@ lazy_static::lazy_static! {
     static ref RE_NONEMPTY_SQL_BLOCK: Regex = Regex::new(r#"(?m)^\s*[^\s](?:[^-]|$)"#).unwrap();
 
     static ref RE_DB: Regex = Regex::new(r#"(?m)^-- database (\S+) *(?:\r|\n|$)"#).unwrap();
+    static ref RE_S3_MODE: Regex = Regex::new(r#"(?m)^-- s3( (.+))? *(?:\r|\n|$)"#).unwrap();
 
     // -- $1 name (type) = default
     static ref RE_ARG_MYSQL: Regex = Regex::new(r#"(?m)^-- \? (\w+) \((\w+)\)(?: ?\= ?(.+))? *(?:\r|\n|$)"#).unwrap();

backend/parsers/windmill-parser-ts/src/lib.rs

@@ -506,7 +506,7 @@ fn one_of_label(members: &Vec<TsTypeElement>) -> Option<String> {
         let Expr::Ident(Ident { sym, .. }) = &**key else {
             return None;
         };
-        if sym != "label" {
+        if sym != "label" && sym != "kind" {
             return None;
         }
 

backend/parsers/windmill-parser-wasm/src/lib.rs

@@ -17,7 +17,7 @@ fn wrap_sig(r: anyhow::Result<MainArgSignature>) -> String {
 
 #[cfg(feature = "ts-parser")]
 #[wasm_bindgen]
-pub fn parse_deno(code: &str, main_override: Option<String>, skip_params: Option<bool>) -> String {
+pub fn parse_deno(code: &str, main_override: Option<String>) -> String {
     wrap_sig(windmill_parser_ts::parse_deno_signature(
         code,
         false,

backend/src/main.rs

@@ -5,7 +5,6 @@
  * Please see the included NOTICE for copyright information and
  * LICENSE-AGPL for a copy of the license.
  */
-
 use anyhow::Context;
 use monitor::{
     load_base_url, load_otel, reload_critical_alerts_on_db_oversize,
@@ -23,6 +22,7 @@ use std::{
     net::{IpAddr, Ipv4Addr, SocketAddr},
     time::{Duration, Instant},
 };
+use strum::IntoEnumIterator;
 use tokio::{fs::File, io::AsyncReadExt, task::JoinHandle};
 use uuid::Uuid;
 use windmill_api::HTTP_CLIENT;
@@ -50,6 +50,7 @@ use windmill_common::{
     },
     scripts::ScriptLang,
     stats_ee::schedule_stats,
+    triggers::TriggerKind,
     utils::{hostname, rd_string, Mode, GIT_VERSION, MODE_AND_ADDONS},
     worker::{
         reload_custom_tags_setting, Connection, HUB_CACHE_DIR, TMP_DIR, TMP_LOGS_DIR, WORKER_GROUP,
@@ -787,6 +788,70 @@ Windmill Community Edition {GIT_VERSION}
                                                     tracing::info!("Workspace premium change detected, invalidating workspace premium cache: {}", workspace_id);
                                                     windmill_common::workspaces::IS_PREMIUM_CACHE.remove(workspace_id);
                                                 },
+                                                "notify_runnable_version_change" => {
+                                                    let payload = n.payload();
+                                                    tracing::info!("Runnable version change detected: {}", payload);
+                                                    match payload.split(':').collect::<Vec<&str>>().as_slice() {
+                                                        [workspace_id, source_type, path, kind] => {
+                                                            let key = (workspace_id.to_string(), path.to_string());
+                                                            match source_type {
+                                                                &"script" => {
+                                                                    windmill_common::DEPLOYED_SCRIPT_HASH_CACHE.remove(&key);
+                                                                    match kind {
+                                                                        &"preprocessor" => {
+                                                                            match sqlx::query_scalar!(
+                                                                                "SELECT fv.id
+                                                                                FROM flow f
+                                                                                INNER JOIN flow_version fv ON fv.id = f.versions[array_upper(f.versions, 1)]
+                                                                                WHERE fv.value->'preprocessor_module'->'value'->>'path' = $1 AND f.workspace_id = $2",
+                                                                                path,
+                                                                                workspace_id
+                                                                            ).fetch_all(&db).await {
+                                                                                Ok(flow_versions) => {
+                                                                                    tracing::debug!("Workspace preprocessor {} changed, removing runnable format version cache for flow versions {:?}", path, flow_versions);
+                                                                                    for version in flow_versions {
+                                                                                        for trigger_kind in TriggerKind::iter() {
+                                                                                            let key = (windmill_common::triggers::HubOrWorkspaceId::WorkspaceId(workspace_id.to_string()), version, trigger_kind);
+                                                                                            windmill_common::triggers::RUNNABLE_FORMAT_VERSION_CACHE.remove(&key);
+                                                                                        }
+                                                                                    }
+                                                                                }
+                                                                                Err(e) => {
+                                                                                    tracing::error!("Error fetching flow paths: {e:#}");
+                                                                                }
+                                                                            }
+                                                                        },
+                                                                        _ => {}
+                                                                    }
+                                                                }
+                                                                &"flow" => {
+                                                                    windmill_common::FLOW_VERSION_CACHE.remove(&key);
+                                                                },
+                                                                _ => {
+                                                                    tracing::warn!("Unknown runnable version change payload: {}", payload);
+                                                                }
+                                                            }
+                                                        },
+                                                        _ => {
+                                                            tracing::warn!("Unknown runnable version change payload: {}", payload);
+                                                        }
+                                                    }
+                                                },
+                                                #[cfg(feature = "http_trigger")]
+                                                "notify_http_trigger_change" => {
+                                                    tracing::info!("HTTP trigger change detected: {}", n.payload());
+                                                    match windmill_api::http_triggers::refresh_routers(&db).await {
+                                                        Ok((true, _)) => {
+                                                            tracing::info!("Refreshed HTTP routers (trigger change)");
+                                                        },
+                                                        Ok((false, _)) => {
+                                                            tracing::warn!("Should have refreshed HTTP routers (trigger change) but did not");
+                                                        },
+                                                        Err(err) => {
+                                                            tracing::error!("Error refreshing HTTP routers (trigger change): {err:#}");
+                                                        }
+                                                    };
+                                                },
                                                 "notify_global_setting_change" => {
                                                     tracing::info!("Global setting change detected: {}", n.payload());
                                                     match n.payload() {
@@ -1108,7 +1173,12 @@ async fn listen_pg(url: &str) -> Option<PgListener> {
         "notify_global_setting_change",
         "notify_webhook_change",
         "notify_workspace_envs_change",
+        "notify_runnable_version_change",
     ];
+
+    #[cfg(feature = "http_trigger")]
+    channels.push("notify_http_trigger_change");
+
     #[cfg(feature = "cloud")]
     channels.push("notify_workspace_premium_change");
 

backend/src/monitor.rs

@@ -28,10 +28,10 @@ use windmill_api::{
     SCIM_TOKEN,
 };
 
-#[cfg(feature = "enterprise")]
-use windmill_common::ee::{jobs_waiting_alerts, worker_groups_alerts};
 #[cfg(feature = "enterprise")]
 use windmill_common::ee::low_disk_alerts;
+#[cfg(feature = "enterprise")]
+use windmill_common::ee::{jobs_waiting_alerts, worker_groups_alerts};
 
 #[cfg(feature = "oauth2")]
 use windmill_common::global_settings::OAUTH_SETTING;
@@ -60,13 +60,13 @@ use windmill_common::{
     server::load_smtp_config,
     tracing_init::JSON_FMT,
     users::truncate_token,
-    utils::empty_string_as_none,
-    utils::{now_from_db, rd_string, report_critical_error, Mode},
+    utils::{empty_as_none, now_from_db, rd_string, report_critical_error, Mode},
     worker::{
+        load_env_vars, load_init_bash_from_env, load_whitelist_env_vars_from_env,
         load_worker_config, reload_custom_tags_setting, store_pull_query,
-        store_suspended_pull_query, update_min_version, Connection, DEFAULT_TAGS_PER_WORKSPACE,
-        DEFAULT_TAGS_WORKSPACES, INDEXER_CONFIG, SCRIPT_TOKEN_EXPIRY, SMTP_CONFIG, TMP_DIR,
-        WORKER_CONFIG, WORKER_GROUP,
+        store_suspended_pull_query, update_min_version, Connection, WorkerConfig,
+        DEFAULT_TAGS_PER_WORKSPACE, DEFAULT_TAGS_WORKSPACES, INDEXER_CONFIG, SCRIPT_TOKEN_EXPIRY,
+        SMTP_CONFIG, TMP_DIR, WORKER_CONFIG, WORKER_GROUP,
     },
     KillpillSender, BASE_URL, CRITICAL_ALERTS_ON_DB_OVERSIZE, CRITICAL_ALERT_MUTE_UI_ENABLED,
     CRITICAL_ERROR_CHANNELS, DB, DEFAULT_HUB_BASE_URL, HUB_BASE_URL, JOB_RETENTION_SECS,
@@ -203,10 +203,23 @@ pub async fn initial_load(
             }
             Connection::Http(_) => {
                 // TODO: reload worker config from http
-                WORKER_CONFIG.write().await.worker_tags = DECODED_AGENT_TOKEN
-                    .as_ref()
-                    .map(|x| x.tags.clone())
-                    .unwrap_or_default();
+                let mut config = WORKER_CONFIG.write().await;
+                *config = WorkerConfig {
+                    worker_tags: DECODED_AGENT_TOKEN
+                        .as_ref()
+                        .map(|x| x.tags.clone())
+                        .unwrap_or_default(),
+                    env_vars: load_env_vars(
+                        load_whitelist_env_vars_from_env(),
+                        &std::collections::HashMap::new(),
+                    ),
+                    priority_tags_sorted: vec![],
+                    dedicated_worker: None,
+                    init_bash: load_init_bash_from_env(),
+                    cache_clear: None,
+                    additional_python_paths: None,
+                    pip_local_dependencies: None,
+                };
             }
         }
     }
@@ -271,13 +284,13 @@ struct OtelSetting {
     metrics_enabled: Option<bool>,
     logs_enabled: Option<bool>,
     tracing_enabled: Option<bool>,
-    #[serde(default, deserialize_with = "empty_string_as_none")]
+    #[serde(default, deserialize_with = "empty_as_none")]
     otel_exporter_otlp_endpoint: Option<String>,
-    #[serde(default, deserialize_with = "empty_string_as_none")]
+    #[serde(default, deserialize_with = "empty_as_none")]
     otel_exporter_otlp_headers: Option<String>,
-    #[serde(default, deserialize_with = "empty_string_as_none")]
+    #[serde(default, deserialize_with = "empty_as_none")]
     otel_exporter_otlp_protocol: Option<String>,
-    #[serde(default, deserialize_with = "empty_string_as_none")]
+    #[serde(default, deserialize_with = "empty_as_none")]
     otel_exporter_otlp_compression: Option<String>,
 }
 
@@ -820,11 +833,7 @@ pub async fn delete_expired_items(db: &DB) -> () {
             Ok(mut tx) => {
                 let deleted_jobs = sqlx::query_scalar!(
                     "DELETE FROM v2_job_completed c
-                    USING v2_job j
-                    WHERE
-                        created_at <= now() - ($1::bigint::text || ' s')::interval
-                        AND completed_at + ($1::bigint::text || ' s')::interval <= now()
-                        AND c.id = j.id
+                    WHERE completed_at <= now() - ($1::bigint::text || ' s')::interval 
                     RETURNING c.id",
                     job_retention_secs
                 )
@@ -1113,10 +1122,13 @@ pub async fn reload_s3_cache_setting(db: &DB) {
             if let Err(e) = setting {
                 tracing::error!("Error parsing s3 cache config: {:?}", e)
             } else {
-                let s3_client = build_object_store_from_settings(setting.unwrap()).await;
+                let setting = setting.unwrap();
+                let bucket = setting.get_bucket().map(|b| b.to_string());
+                let s3_client = build_object_store_from_settings(setting).await;
                 if let Err(e) = s3_client {
                     tracing::error!("Error building s3 client from settings: {:?}", e)
                 } else {
+                    tracing::info!("Loaded object store {:?}", bucket);
                     *s3_cache_settings = Some(s3_client.unwrap());
                 }
             }
@@ -2213,7 +2225,9 @@ pub async fn reload_critical_error_channels_setting(conn: &DB) -> error::Result<
 pub async fn reload_critical_alerts_on_db_oversize(conn: &DB) -> error::Result<()> {
     #[derive(Deserialize)]
     struct DBOversize {
+        #[serde(default)]
         enabled: bool,
+        #[serde(default)]
         value: f32,
     }
     let db_oversize_value =

backend/tests/worker.rs

@@ -4687,6 +4687,7 @@ mod job_payload {
                 path: "f/system/hello_with_nodes_flow".to_string(),
                 dedicated_worker: None,
                 apply_preprocessor: false,
+                version: 1443253234253454,
             })
             .run_until_complete(&db, port)
             .await
@@ -4731,6 +4732,7 @@ mod job_payload {
                 path: "f/system/hello_with_preprocessor".to_string(),
                 dedicated_worker: None,
                 apply_preprocessor: true,
+                version: 1443253234253456,
             })
             .run_until_complete_with(db, port, |id| async move {
                 let job = sqlx::query!("SELECT preprocessed FROM v2_job WHERE id = $1", id)
@@ -4798,6 +4800,7 @@ mod job_payload {
                 path: "f/system/hello_with_nodes_flow".to_string(),
                 dedicated_worker: None,
                 apply_preprocessor: true,
+                version: 1443253234253454,
             })
             .run_until_complete(&db, port)
             .await

backend/windmill-api/openapi.yaml

@@ -1,7 +1,7 @@
 openapi: "3.0.3"
 
 info:
-  version: 1.489.0
+  version: 1.491.5
   title: Windmill API
 
   contact:
@@ -7899,6 +7899,52 @@ paths:
         "200":
           description: Interactive slack approval message sent successfully
 
+  /w/{workspace}/jobs/teams_approval/{id}:
+    get:
+      summary: generate interactive teams approval for suspended job
+      operationId: getTeamsApprovalPayload
+      tags:
+        - job
+      parameters:
+        - $ref: "#/components/parameters/WorkspaceId"
+        - $ref: "#/components/parameters/JobId"
+        - name: approver
+          in: query
+          schema:
+            type: string
+        - name: message
+          in: query
+          schema:
+            type: string
+        - name: team_name
+          in: query
+          required: true
+          schema:
+            type: string
+        - name: channel_name
+          in: query
+          required: true
+          schema:
+            type: string
+        - name: flow_step_id
+          in: query
+          required: true
+          schema:
+            type: string
+        - name: default_args_json
+          in: query
+          required: false
+          schema:
+            type: string
+        - name: dynamic_enums_json
+          in: query
+          required: false
+          schema:
+            type: string
+      responses:
+        "200":
+          description: Interactive slack approval message sent successfully
+
   /w/{workspace}/jobs_u/resume/{id}/{resume_id}/{signature}:
     get:
       summary: resume a job for a suspended flow
@@ -10982,7 +11028,8 @@ paths:
           description: a config
           content:
             application/json:
-              schema: {}
+              schema:
+                $ref: "#/components/schemas/Configs"
 
   /configs/update/{name}:
     post:
@@ -12543,6 +12590,11 @@ paths:
           required: true
           schema:
             type: string
+        - name: pagination_offset
+          in: query
+          required: false
+          schema:
+            type: integer
       responses:
         "200":
           description: search results
@@ -12555,15 +12607,26 @@ paths:
                     description: a list of the terms that couldn't be parsed (and thus ignored)
                     type: array
                     items:
-                      type: object
-                      properties:
-                        dancer:
-                          type: string
+                      type: string
                   hits:
                     description: the jobs that matched the query
                     type: array
                     items:
                       $ref: "#/components/schemas/JobSearchHit"
+                  hit_count:
+                    description: how many jobs matched in total
+                    type: number
+                  index_metadata:
+                    description: Metadata about the index current state
+                    type: object
+                    properties:
+                      indexed_until:
+                        description: Datetime of the most recently indexed job
+                        type: string
+                        format: date-time
+                      lost_lock_ownership:
+                        description: Is the current indexer service being replaced
+                        type: boolean
 
   /srch/index/search/service_logs:
     get:
@@ -13137,6 +13200,37 @@ components:
         code_completion_model:
           $ref: "#/components/schemas/AIProviderModel"
 
+    Alert:
+      type: object
+      properties:
+        name:
+          type: string
+        tags_to_monitor:
+          type: array
+          items:
+            type: string
+        jobs_num_threshold:
+          type: integer
+        alert_cooldown_seconds:
+          type: integer
+        alert_time_threshold_seconds:
+          type: integer
+      required:
+        - name
+        - tags_to_monitor
+        - jobs_num_threshold
+        - alert_cooldown_seconds
+        - alert_time_threshold_seconds
+
+    Configs:
+      type: object
+      nullable: true
+      properties:
+        alerts:
+          type: array
+          items:
+            $ref: '#/components/schemas/Alert'
+
     Script:
       type: object
       properties:
@@ -16238,6 +16332,7 @@ components:
               - schedule
               - user
               - group
+              - trigger
         repositories:
           type: array
           items:
@@ -16305,6 +16400,7 @@ components:
               - schedule
               - user
               - group
+              - trigger
       required:
         - script_path
         - git_repo_resource_path
@@ -16539,8 +16635,8 @@ components:
       properties:
         trigger_kind:
           $ref: "#/components/schemas/CaptureTriggerKind"
-        payload: {}
-        trigger_extra: {}
+        main_args: {}
+        preprocessor_args: {}
         id:
           type: integer
         created_at:
@@ -16548,7 +16644,8 @@ components:
           format: date-time
       required:
         - trigger_kind
-        - payload
+        - main_args
+        - preprocessor_args
         - id
         - created_at
     CaptureConfig:
@@ -16729,4 +16826,4 @@ components:
         channel_name:
           type: string
           description: Microsoft Teams channel name
-          minLength: 1
+          minLength: 1

backend/windmill-api/src/agent_workers_ee.rs

@@ -28,7 +28,7 @@ pub fn workspaced_service(
     use windmill_worker::JobCompletedSender;
 
     let (job_completed_tx, _job_completed_rx) =
-        JobCompletedSender::new(&Connection::Sql(db.clone()), 100);
+        JobCompletedSender::new(&Connection::Sql(db.clone()), 10);
 
     let router = Router::new();
 

backend/windmill-api/src/approvals.rs

@@ -0,0 +1,292 @@
+use serde::{Deserialize, Serialize};
+use std::collections::HashMap;
+use uuid::Uuid;
+use std::str::FromStr;
+use regex::Regex;
+use serde_json::Value;
+use crate::db::{ApiAuthed, DB};
+use crate::jobs::{cancel_suspended_job, resume_suspended_job, QueryApprover, QueryOrBody, ResumeUrls, get_resume_urls_internal};
+use axum::{extract::{Path, Query}, Extension};
+use windmill_common::error::Error;
+use windmill_common::cache;
+use windmill_common::jobs::JobKind;
+use windmill_common::scripts::ScriptHash;
+use serde_json::value::RawValue;
+
+#[derive(Debug, Deserialize, Serialize)]
+pub struct ResumeSchema {
+    pub schema: Schema,
+}
+
+#[derive(Debug, Deserialize, Serialize)]
+pub struct Schema {
+    pub order: Vec<String>,
+    pub required: Vec<String>,
+    pub properties: HashMap<String, ResumeFormField>,
+}
+
+#[derive(Debug, Deserialize, Serialize, Clone)]
+#[serde(rename_all = "lowercase")]
+pub enum FieldType {
+    Boolean,
+    String,
+    Number,
+    Integer,
+    #[serde(other)]
+    Unknown,
+}
+
+#[derive(Debug, Deserialize, Serialize, Clone)]
+pub struct ResumeFormField {
+    pub r#type: FieldType,
+    pub format: Option<String>,
+    pub default: Option<serde_json::Value>,
+    pub description: Option<String>,
+    pub title: Option<String>,
+    pub r#enum: Option<Vec<String>>,
+    #[serde(rename = "enumLabels")]
+    pub enum_labels: Option<HashMap<String, String>>,
+    pub nullable: Option<bool>,
+    pub placeholder: Option<String>,
+}
+
+#[derive(Debug, Deserialize, Serialize)]
+pub struct ResumeFormRow {
+    pub resume_form: Option<serde_json::Value>,
+    pub hide_cancel: Option<bool>,
+}
+
+#[derive(Deserialize)]
+pub struct QueryMessage {
+    pub message: Option<String>,
+}
+
+#[derive(Deserialize)]
+pub struct QueryFlowStepId {
+    pub flow_step_id: String,
+}
+
+#[derive(Deserialize, Debug)]
+pub struct QueryDefaultArgsJson {
+    pub default_args_json: Option<serde_json::Value>,
+}
+
+#[derive(Deserialize, Debug)]
+pub struct QueryDynamicEnumJson {
+    pub dynamic_enums_json: Option<serde_json::Value>,
+}
+
+#[derive(Debug)]
+pub struct ApprovalFormDetails {
+    pub message_str: String,
+    pub urls: ResumeUrls,
+    pub schema: Option<ResumeFormRow>,
+}
+
+#[allow(dead_code)]
+#[derive(Debug, Clone, Copy)]
+pub enum MessageFormat {
+    Slack,
+    Teams,
+}
+
+pub fn extract_w_id_from_resume_url(resume_url: &str) -> Result<&str, Error> {
+    let re = Regex::new(r"/api/w/(?P<w_id>[^/]+)/jobs_u/(?P<action>resume|cancel)/(?P<job_id>[^/]+)/(?P<resume_id>[^/]+)/(?P<secret>[a-fA-F0-9]+)(?:\?approver=(?P<approver>[^&]+))?").unwrap();
+    let captures = re.captures(resume_url).ok_or_else(|| {
+        tracing::error!("Resume URL does not match the pattern.");
+        Error::BadRequest("Invalid URL format.".to_string())
+    })?;
+    Ok(captures.name("w_id").map_or("", |m| m.as_str()))
+}
+
+pub async fn handle_resume_action(
+    authed: Option<ApiAuthed>,
+    db: DB,
+    resume_url: &str,
+    form_data: Value,
+    action: &str,
+) -> Result<(), Error> {
+    // Extract information from resume_url using regex
+    let re = Regex::new(r"/api/w/(?P<w_id>[^/]+)/jobs_u/(?P<action>resume|cancel)/(?P<job_id>[^/]+)/(?P<resume_id>[^/]+)/(?P<secret>[a-fA-F0-9]+)(?:\?approver=(?P<approver>[^&]+))?").unwrap();
+    let captures = re.captures(resume_url).ok_or_else(|| {
+        tracing::error!("Resume URL does not match the pattern.");
+        Error::BadRequest("Invalid URL format.".to_string())
+    })?;
+
+    let (w_id, job_id, resume_id, secret, approver) = (
+        captures.name("w_id").map_or("", |m| m.as_str()),
+        captures.name("job_id").map_or("", |m| m.as_str()),
+        captures.name("resume_id").map_or("", |m| m.as_str()),
+        captures.name("secret").map_or("", |m| m.as_str()),
+        captures.name("approver").map(|m| m.as_str().to_string()),
+    );
+
+    let approver = QueryApprover { approver };
+
+    // Convert job_id and resume_id to appropriate types
+    let job_uuid = Uuid::from_str(job_id)
+        .map_err(|_| Error::BadRequest("Invalid job ID format.".to_string()))?;
+
+    let resume_id_parsed = resume_id
+        .parse::<u32>()
+        .map_err(|_| Error::BadRequest("Invalid resume ID format.".to_string()))?;
+
+    // Call the appropriate function based on the action
+    let res = if action == "resume" {
+        resume_suspended_job(
+            authed,
+            Extension(db.clone()),
+            Path((
+                w_id.to_string(),
+                job_uuid,
+                resume_id_parsed,
+                secret.to_string(),
+            )),
+            Query(approver),
+            QueryOrBody(Some(form_data)),
+        )
+        .await
+    } else {
+        cancel_suspended_job(
+            authed,
+            Extension(db.clone()),
+            Path((
+                w_id.to_string(),
+                job_uuid,
+                resume_id_parsed,
+                secret.to_string(),
+            )),
+            Query(approver),
+            QueryOrBody(Some(form_data)),
+        )
+        .await
+    };
+
+    tracing::debug!("Job action result: {:#?}", res);
+    res?;
+
+    Ok(())
+}
+
+pub async fn get_approval_form_details(
+    db: DB,
+    w_id: &str,
+    job_id: Uuid,
+    flow_step_id: Option<&str>,
+    resume_id: u32,
+    approver: Option<&str>,
+    message: Option<&str>,
+    format: MessageFormat,
+) -> Result<ApprovalFormDetails, Error> {
+    let res = get_resume_urls_internal(
+        axum::Extension(db.clone()),
+        Path((w_id.to_string(), job_id, resume_id)),
+        Query(QueryApprover { approver: approver.map(|a| a.to_string()) }),
+    )
+    .await?;
+
+    let urls = res.0;
+
+    tracing::debug!("Job ID: {:?}", job_id);
+
+    // TODO: do we have a helper function for this?
+    let (job_kind, script_hash, raw_flow, parent_job_id, created_at, created_by, script_path, args) = sqlx::query!(
+        "WITH job_info AS (
+            -- Query for Teams (running jobs)
+            SELECT
+                parent.job_kind AS \"job_kind!: JobKind\",
+                parent.script_hash AS \"script_hash: ScriptHash\",
+                parent.raw_flow AS \"raw_flow: sqlx::types::Json<Box<RawValue>>\",
+                child.parent_job AS \"parent_job: Uuid\",
+                parent.created_at AS \"created_at!: chrono::NaiveDateTime\",
+                parent.created_by AS \"created_by!\",
+                parent.script_path,
+                parent.args AS \"args: sqlx::types::Json<Box<RawValue>>\"
+            FROM v2_as_queue child
+            JOIN v2_as_queue parent ON parent.id = child.parent_job
+            WHERE child.id = $1 AND child.workspace_id = $2
+            UNION ALL
+            -- Query for Slack (completed jobs)
+            SELECT
+                v2_as_queue.job_kind AS \"job_kind!: JobKind\",
+                v2_as_queue.script_hash AS \"script_hash: ScriptHash\",
+                v2_as_queue.raw_flow AS \"raw_flow: sqlx::types::Json<Box<RawValue>>\",
+                v2_as_completed_job.parent_job AS \"parent_job: Uuid\",
+                v2_as_completed_job.created_at AS \"created_at!: chrono::NaiveDateTime\",
+                v2_as_completed_job.created_by AS \"created_by!\",
+                v2_as_queue.script_path,
+                v2_as_queue.args AS \"args: sqlx::types::Json<Box<RawValue>>\"
+            FROM v2_as_queue
+            JOIN v2_as_completed_job ON v2_as_completed_job.parent_job = v2_as_queue.id
+            WHERE v2_as_completed_job.id = $1 AND v2_as_completed_job.workspace_id = $2
+        )
+        SELECT * FROM job_info LIMIT 1",
+        job_id,
+        &w_id
+    )
+    .fetch_optional(&db)
+    .await
+    .map_err(|e| Error::BadRequest(e.to_string()))?
+    .ok_or_else(|| Error::BadRequest("This workflow is no longer running and has either already timed out or been cancelled or completed.".to_string()))
+    .map(|r| (r.job_kind, r.script_hash, r.raw_flow, r.parent_job, r.created_at, r.created_by, r.script_path, r.args))?;
+
+    let flow_data = match cache::job::fetch_flow(&db, job_kind, script_hash).await {
+        Ok(data) => data,
+        Err(_) => {
+            if let Some(parent_job_id) = parent_job_id.as_ref() {
+                cache::job::fetch_preview_flow(&db, parent_job_id, raw_flow).await?
+            } else {
+                return Err(Error::BadRequest(
+                    "This workflow is no longer running and has either already timed out or been cancelled or completed.".to_string(),
+                ));
+            }
+        }
+    };
+
+    let flow_value = &flow_data.flow;
+    let flow_step_id = flow_step_id.unwrap_or("");
+    let module = flow_value.modules.iter().find(|m| m.id == flow_step_id);
+
+    tracing::debug!("Module: {:#?}", module);
+
+    let schema = module.and_then(|module| {
+        module.suspend.as_ref().map(|suspend| ResumeFormRow {
+            resume_form: suspend.resume_form.clone(),
+            hide_cancel: suspend.hide_cancel,
+        })
+    });
+
+    let args_str = args.map_or("None".to_string(), |a| a.get().to_string());
+    let parent_job_id_str = parent_job_id.map_or("None".to_string(), |id| id.to_string());
+    let script_path_str = script_path.as_deref().unwrap_or("None");
+
+    let created_at_formatted = created_at.format("%Y-%m-%d %H:%M:%S").to_string();
+
+    let bold_format = match format {
+        MessageFormat::Slack => "*{}*",
+        MessageFormat::Teams => "**{}**",
+    };
+
+    let mut message_str = format!(
+        "A workflow has been suspended and is waiting for approval:\n\n\
+        {}: {created_by}\n\n\
+        {}: {created_at_formatted}\n\n\
+        {}: {script_path_str}\n\n\
+        {}: {args_str}\n\n\
+        {}: {parent_job_id_str}\n\n",
+        bold_format.replace("{}", "Created by"),
+        bold_format.replace("{}", "Created at"),
+        bold_format.replace("{}", "Script path"),
+        bold_format.replace("{}", "Args"),
+        bold_format.replace("{}", "Flow ID")
+    );
+
+    // Append custom message if provided
+    if let Some(msg) = message {
+        message_str.push_str(msg);
+    }
+
+    tracing::debug!("Schema: {:#?}", schema);
+
+    Ok(ApprovalFormDetails { message_str, urls, schema })
+}

backend/windmill-api/src/args.rs

@@ -1,7 +1,5 @@
 use std::collections::HashMap;
 
-#[cfg(feature = "parquet")]
-use crate::job_helpers_ee::get_workspace_s3_resource;
 use axum::{
     extract::{FromRequest, FromRequestParts, Multipart, Query, Request},
     http::{HeaderMap, Uri},
@@ -9,139 +7,323 @@ use axum::{
 };
 use bytes::Bytes;
 use http::{header::CONTENT_TYPE, request::Parts, StatusCode};
-#[cfg(feature = "parquet")]
-use object_store::{Attribute, Attributes};
-use serde::Deserialize;
+use serde::{Deserialize, Serialize};
 use serde_json::value::RawValue;
 use sqlx::types::JsonRawValue;
-#[cfg(feature = "parquet")]
-use windmill_common::s3_helpers::build_object_store_client;
-use windmill_common::{error::Error, worker::to_raw_value, DB};
+use windmill_common::{
+    error::Error,
+    triggers::{RunnableFormat, RunnableFormatVersion, TriggerKind},
+    worker::to_raw_value,
+    DB,
+};
 use windmill_queue::PushArgsOwned;
 
-use crate::db::ApiAuthed;
-#[cfg(feature = "parquet")]
-use crate::job_helpers_ee::{get_random_file_name, upload_file_internal};
+use crate::{
+    db::ApiAuthed,
+    trigger_helpers::{get_runnable_format, RunnableId},
+};
 
-#[derive(Debug, Default)]
-pub struct WebhookArgs {
-    pub args: PushArgsOwned,
-    pub multipart: Option<Multipart>,
-    pub wrap_body: Option<bool>,
+#[derive(Debug)]
+pub enum RawBody {
+    Json(String),
+    CEJson(String),
+    Text(String),
+    Xml(String),
+    UrlEncoded(Bytes),
+    Multipart(Multipart),
+    Empty,
 }
 
-impl WebhookArgs {
+#[derive(Debug, Clone, Serialize)]
+#[serde(untagged)]
+pub enum Body {
+    HashMap(HashMap<String, Box<RawValue>>),
+    NoHashMap(Box<RawValue>),
+}
+
+#[derive(Debug, Clone, Default)]
+pub struct WebhookArgsMetadata {
+    pub raw_string: Option<String>,
+    pub headers: HashMap<String, Box<RawValue>>,
+    pub method: http::Method,
+    pub query: HashMap<String, Box<RawValue>>,
+    pub query_wrap_body: bool,
+    pub query_use_raw: bool,
+}
+
+pub struct RawWebhookArgs {
+    pub body: RawBody,
+    pub metadata: WebhookArgsMetadata,
+}
+
+#[derive(Debug, Clone)]
+pub struct WebhookArgs {
+    pub body: Body,
+    pub metadata: WebhookArgsMetadata,
+}
+
+// capture
+//
+
+impl RawWebhookArgs {
     #[cfg(not(feature = "parquet"))]
-    pub async fn to_push_args_owned(
-        self,
+    pub async fn process_multipart(
+        _multipart: Multipart,
         _authed: &ApiAuthed,
         _db: &DB,
         _w_id: &str,
-    ) -> Result<PushArgsOwned, Error> {
-        if self.multipart.is_some() {
-            return Err(Error::BadRequest(format!(
-                "multipart/form-data requires the parquet feature"
-            )));
-        }
-
-        Ok(self.args)
+    ) -> Result<HashMap<String, Box<RawValue>>, Error> {
+        return Err(Error::BadRequest(format!(
+            "multipart/form-data requires the parquet feature"
+        )));
     }
 
     #[cfg(feature = "parquet")]
-    pub async fn to_push_args_owned(
-        mut self,
+    async fn process_multipart(
+        mut multipart: Multipart,
+        authed: &ApiAuthed,
+        db: &DB,
+        w_id: &str,
+    ) -> Result<HashMap<String, Box<RawValue>>, Error> {
+        use crate::job_helpers_ee::{
+            get_random_file_name, get_workspace_s3_resource, upload_file_internal,
+        };
+        use futures::TryStreamExt;
+        use object_store::{Attribute, Attributes};
+        use windmill_common::s3_helpers::build_object_store_client;
+
+        let (_, s3_resource) = get_workspace_s3_resource(authed, db, None, "", w_id, None).await?;
+
+        if let Some(s3_resource) = s3_resource {
+            let s3_client = build_object_store_client(&s3_resource).await?;
+
+            let mut body = HashMap::new();
+            let mut files = HashMap::new();
+
+            while let Some(field) = multipart.next_field().await.map_err(|e| {
+                Error::BadRequest(format!("Error reading multipart field: {}", e.body_text()))
+            })? {
+                if let Some(name) = field.name().map(|x| x.to_string()) {
+                    if let Some(content_type) = field.content_type() {
+                        let ext = field
+                            .file_name()
+                            .map(|x| x.split('.').last())
+                            .flatten()
+                            .map(|x| x.to_string());
+
+                        let file_key = get_random_file_name(ext);
+
+                        let options = Attributes::from_iter(vec![
+                            (Attribute::ContentType, content_type.to_string()),
+                            (
+                                Attribute::ContentDisposition,
+                                if let Some(filename) = field.file_name() {
+                                    format!("inline; filename=\"{}\"", filename)
+                                } else {
+                                    "inline".to_string()
+                                },
+                            ),
+                        ])
+                        .into();
+
+                        let bytes_stream = field
+                            .into_stream()
+                            .map_err(|err| std::io::Error::new(std::io::ErrorKind::Other, err));
+
+                        upload_file_internal(s3_client.clone(), &file_key, bytes_stream, options)
+                            .await?;
+
+                        files.entry(name).or_insert(vec![]).push(serde_json::json!({
+                            "s3": &file_key
+                        }));
+                    } else {
+                        body.insert(name, to_raw_value(&field.text().await.unwrap_or_default()));
+                    }
+                }
+            }
+
+            for (k, v) in files {
+                body.insert(k, to_raw_value(&v));
+            }
+
+            Ok(body)
+        } else {
+            Err(Error::BadRequest(format!(
+                "You need to connect your workspace to an S3 bucket to use multipart/form-data"
+            )))
+        }
+    }
+
+    pub async fn process_args(
+        self,
+        authed: &ApiAuthed,
+        db: &DB,
+        w_id: &str,
+        force_use_raw: Option<bool>,
+    ) -> Result<WebhookArgs, Error> {
+        let use_raw = force_use_raw.unwrap_or(self.metadata.query_use_raw);
+
+        match self.body {
+            RawBody::Multipart(multipart) => {
+                let body = Self::process_multipart(multipart, authed, db, w_id).await?;
+                Ok(WebhookArgs { body: Body::HashMap(body), metadata: self.metadata })
+            }
+            RawBody::Empty => {
+                let mut metadata = self.metadata;
+                if use_raw {
+                    metadata.raw_string = Some("".to_string());
+                }
+                Ok(WebhookArgs { body: Body::HashMap(HashMap::new()), metadata })
+            }
+            RawBody::Text(s) | RawBody::Xml(s) => Ok(WebhookArgs {
+                body: Body::HashMap(HashMap::new()),
+                metadata: WebhookArgsMetadata { raw_string: Some(s), ..self.metadata },
+            }),
+            RawBody::UrlEncoded(bytes) => {
+                let mut metadata = self.metadata;
+                if use_raw {
+                    let raw_string = String::from_utf8(bytes.to_vec())
+                        .map_err(|e| Error::BadRequest(format!("invalid utf8: {}", e)))?;
+                    metadata.raw_string = Some(raw_string);
+                }
+                let payload: HashMap<String, Option<String>> = serde_urlencoded::from_bytes(&bytes)
+                    .map_err(|e| Error::BadRequest(format!("invalid urlencoded data: {}", e)))?;
+                let payload = payload
+                    .into_iter()
+                    .map(|(k, v)| (k, to_raw_value(&v)))
+                    .collect::<HashMap<_, _>>();
+
+                Ok(WebhookArgs { body: Body::HashMap(payload), metadata })
+            }
+            RawBody::Json(s) => WebhookArgs::from_json(self.metadata, use_raw, s).await,
+            RawBody::CEJson(s) => WebhookArgs::from_ce_json(self.metadata, use_raw, s).await,
+        }
+    }
+
+    pub async fn to_main_args(
+        self,
         authed: &ApiAuthed,
         db: &DB,
         w_id: &str,
     ) -> Result<PushArgsOwned, Error> {
-        use futures::TryStreamExt;
+        let args = self.process_args(authed, db, w_id, None).await?;
+        args.to_main_args()
+    }
 
-        if let Some(mut multipart) = self.multipart {
-            {
-                let (_, s3_resource) =
-                    get_workspace_s3_resource(authed, db, None, "", w_id, None).await?;
+    pub async fn to_args_from_runnable(
+        self,
+        authed: &ApiAuthed,
+        db: &DB,
+        w_id: &str,
+        runnable_id: RunnableId,
+        skip_preprocessor: Option<bool>,
+    ) -> Result<PushArgsOwned, Error> {
+        let args = self.process_args(authed, db, w_id, None).await?;
+        args.to_args_from_runnable(db, w_id, runnable_id, skip_preprocessor)
+            .await
+    }
+}
 
-                if let Some(s3_resource) = s3_resource {
-                    let s3_client = build_object_store_client(&s3_resource).await?;
+#[derive(Serialize)]
+struct WebhookPreprocessorEvent {
+    kind: String,
+    body: Box<RawValue>,
+    raw_string: Option<String>,
+    headers: HashMap<String, Box<RawValue>>,
+    query: HashMap<String, Box<RawValue>>,
+}
 
-                    let mut body = HashMap::new();
-                    let mut files = HashMap::new();
+impl WebhookArgs {
+    pub fn to_main_args(self) -> Result<PushArgsOwned, Error> {
+        self.to_args_from_format(RunnableFormat {
+            has_preprocessor: false,
+            version: RunnableFormatVersion::V2,
+        })
+    }
 
-                    while let Some(field) = multipart.next_field().await.map_err(|e| {
-                        Error::BadRequest(format!(
-                            "Error reading multipart field: {}",
-                            e.body_text()
-                        ))
-                    })? {
-                        if let Some(name) = field.name().map(|x| x.to_string()) {
-                            if let Some(content_type) = field.content_type() {
-                                let ext = field
-                                    .file_name()
-                                    .map(|x| x.split('.').last())
-                                    .flatten()
-                                    .map(|x| x.to_string());
+    pub async fn to_args_from_runnable(
+        self,
+        db: &DB,
+        w_id: &str,
+        runnable_id: RunnableId,
+        skip_preprocessor: Option<bool>,
+    ) -> Result<PushArgsOwned, Error> {
+        if skip_preprocessor.unwrap_or(false) {
+            self.to_main_args()
+        } else {
+            let runnable_format =
+                get_runnable_format(runnable_id, w_id, db, &TriggerKind::Webhook).await?;
 
-                                let file_key = get_random_file_name(ext);
+            self.to_args_from_format(runnable_format)
+        }
+    }
 
-                                let options = Attributes::from_iter(vec![
-                                    (Attribute::ContentType, content_type.to_string()),
-                                    (
-                                        Attribute::ContentDisposition,
-                                        if let Some(filename) = field.file_name() {
-                                            format!("inline; filename=\"{}\"", filename)
-                                        } else {
-                                            "inline".to_string()
-                                        },
-                                    ),
-                                ])
-                                .into();
+    pub fn to_args_from_format(
+        self,
+        runnable_format: RunnableFormat,
+    ) -> Result<PushArgsOwned, Error> {
+        match runnable_format {
+            RunnableFormat { has_preprocessor: true, version: RunnableFormatVersion::V2 } => {
+                let mut args = HashMap::new();
 
-                                let bytes_stream = field.into_stream().map_err(|err| {
-                                    std::io::Error::new(std::io::ErrorKind::Other, err)
-                                });
+                args.insert(
+                    "event".to_string(),
+                    to_raw_value(&WebhookPreprocessorEvent {
+                        kind: "webhook".to_string(),
+                        body: to_raw_value(&self.body),
+                        raw_string: self.metadata.raw_string,
+                        headers: self.metadata.headers,
+                        query: self.metadata.query,
+                    }),
+                );
 
-                                upload_file_internal(
-                                    s3_client.clone(),
-                                    &file_key,
-                                    bytes_stream,
-                                    options,
-                                )
-                                .await?;
+                Ok(PushArgsOwned { args, extra: None })
+            }
+            RunnableFormat { has_preprocessor, .. } => {
+                let mut extra = HashMap::new();
 
-                                files.entry(name).or_insert(vec![]).push(serde_json::json!({
-                                    "s3": &file_key
-                                }));
-                            } else {
-                                body.insert(
-                                    name,
-                                    to_raw_value(&field.text().await.unwrap_or_default()),
-                                );
-                            }
+                let WebhookArgsMetadata { query, query_wrap_body, headers, raw_string, .. } =
+                    self.metadata;
+
+                for (k, v) in headers {
+                    extra.insert(k, v);
+                }
+
+                for (k, v) in query {
+                    extra.insert(k, v);
+                }
+
+                if let Some(raw_string) = raw_string {
+                    extra.insert("raw_string".to_string(), to_raw_value(&raw_string));
+                }
+
+                if has_preprocessor {
+                    // if has preprocessor, it has to be v1
+                    extra.insert(
+                        "wm_trigger".to_string(),
+                        to_raw_value(&serde_json::json!({
+                            "kind": "webhook",
+                        })),
+                    );
+                }
+
+                let extra = if extra.is_empty() { None } else { Some(extra) };
+
+                match self.body {
+                    Body::HashMap(mut body) => {
+                        if query_wrap_body {
+                            body = HashMap::from([("body".to_string(), to_raw_value(&body))]);
                         }
+                        Ok(PushArgsOwned { args: body, extra })
                     }
-
-                    for (k, v) in files {
-                        body.insert(k, to_raw_value(&v));
+                    Body::NoHashMap(args) => {
+                        let mut hm = HashMap::new();
+                        hm.insert("body".to_string(), args);
+                        Ok(PushArgsOwned { args: hm, extra })
                     }
-
-                    if self.wrap_body.unwrap_or(false) {
-                        self.args
-                            .args
-                            .insert("body".to_string(), to_raw_value(&body));
-                    } else {
-                        self.args.args.extend(body);
-                    }
-
-                    return Ok(self.args);
                 }
             }
-
-            return Err(Error::BadRequest(format!(
-                "You need to connect your workspace to an S3 bucket to use multipart/form-data"
-            )));
         }
-
-        Ok(self.args)
     }
 }
 
@@ -166,26 +348,36 @@ async fn req_to_string<S: Send + Sync>(
 pub async fn try_from_request_body<S>(
     request: Request,
     _state: &S,
-    use_raw: Option<bool>,
-    wrap_body: Option<bool>,
-) -> Result<WebhookArgs, Response>
+    is_http_trigger: bool,
+) -> Result<RawWebhookArgs, Response>
 where
     S: Send + Sync,
 {
-    let (content_type, mut extra, use_raw, wrap_body) = {
+    let (content_type, metadata) = {
         let headers_map = request.headers();
         let content_type_header = headers_map.get(CONTENT_TYPE);
         let content_type = content_type_header.and_then(|value| value.to_str().ok());
         let uri = request.uri();
-        let query = Query::<RequestQuery>::try_from_uri(uri).unwrap().0;
-        let mut extra = build_extra(&headers_map, query.include_header);
-        let query_decode = DecodeQueries::from_uri(uri);
+        let request_query = Query::<RequestQuery>::try_from_uri(uri).unwrap().0;
+        let headers = build_headers(&headers_map, request_query.include_header, is_http_trigger);
+        let query_decode = DecodeQueries::from_uri(uri, is_http_trigger);
+        let mut query = HashMap::new();
         if let Some(DecodeQueries(queries)) = query_decode {
-            extra.extend(queries);
+            query.extend(queries);
         }
-        let raw = query.raw.unwrap_or(use_raw.unwrap_or(false));
-        let wrap_body = query.wrap_body.unwrap_or(wrap_body.unwrap_or(false));
-        (content_type, extra, raw, wrap_body)
+        let raw = !is_http_trigger && request_query.raw.unwrap_or(false);
+        let wrap_body = !is_http_trigger && request_query.wrap_body.unwrap_or(false);
+        (
+            content_type,
+            WebhookArgsMetadata {
+                headers,
+                query,
+                method: request.method().clone(),
+                raw_string: None,
+                query_wrap_body: wrap_body,
+                query_use_raw: raw,
+            },
+        )
     };
 
     let no_content_type = content_type.is_none();
@@ -194,33 +386,19 @@ where
             .await
             .map_err(IntoResponse::into_response)?;
         if no_content_type && bytes.is_empty() {
-            if use_raw {
-                extra.insert("raw_string".to_string(), to_raw_value(&"".to_string()));
-            }
-            let mut args = HashMap::new();
-            if wrap_body {
-                args.insert("body".to_string(), to_raw_value(&serde_json::json!({})));
-            }
-            return Ok(WebhookArgs {
-                args: PushArgsOwned { extra: Some(extra), args: args },
-                ..Default::default()
-            });
+            Ok(RawWebhookArgs { body: RawBody::Empty, metadata })
+        } else {
+            let str = String::from_utf8(bytes.to_vec())
+                .map_err(|e| Error::BadRequest(format!("invalid utf8: {}", e)).into_response())?;
+            Ok(RawWebhookArgs { body: RawBody::Json(str), metadata })
         }
-        let str = String::from_utf8(bytes.to_vec())
-            .map_err(|e| Error::BadRequest(format!("invalid utf8: {}", e)).into_response())?;
-
-        PushArgsOwned::from_json(extra, use_raw, wrap_body, str)
-            .await
-            .map(|args| WebhookArgs { args, ..Default::default() })
     } else if content_type
         .unwrap()
         .starts_with("application/cloudevents+json")
     {
         let str = req_to_string(request, _state).await?;
 
-        PushArgsOwned::from_ce_json(extra, use_raw, str)
-            .await
-            .map(|args| WebhookArgs { args, ..Default::default() })
+        Ok(RawWebhookArgs { body: RawBody::CEJson(str), metadata })
     } else if content_type
         .unwrap()
         .starts_with("application/cloudevents-batch+json")
@@ -231,11 +409,7 @@ where
         )
     } else if content_type.unwrap().starts_with("text/plain") {
         let str = req_to_string(request, _state).await?;
-        extra.insert("raw_string".to_string(), to_raw_value(&str));
-        Ok(WebhookArgs {
-            args: PushArgsOwned { extra: Some(extra), args: HashMap::new() },
-            ..Default::default()
-        })
+        Ok(RawWebhookArgs { body: RawBody::Text(str), metadata })
     } else if content_type
         .unwrap()
         .starts_with("application/x-www-form-urlencoded")
@@ -244,58 +418,32 @@ where
             .await
             .map_err(IntoResponse::into_response)?;
 
-        if use_raw {
-            let raw_string = String::from_utf8(bytes.to_vec())
-                .map_err(|e| Error::BadRequest(format!("invalid utf8: {}", e)).into_response())?;
-            extra.insert("raw_string".to_string(), to_raw_value(&raw_string));
-        }
-
-        let payload: HashMap<String, Option<String>> = serde_urlencoded::from_bytes(&bytes)
-            .map_err(|e| {
-                Error::BadRequest(format!("invalid urlencoded data: {}", e)).into_response()
-            })?;
-        let payload = payload
-            .into_iter()
-            .map(|(k, v)| (k, to_raw_value(&v)))
-            .collect::<HashMap<_, _>>();
-
-        return Ok(WebhookArgs {
-            args: PushArgsOwned { extra: Some(extra), args: payload },
-            ..Default::default()
-        });
+        Ok(RawWebhookArgs { body: RawBody::UrlEncoded(bytes), metadata })
     } else if content_type.unwrap().starts_with("application/xml")
         || content_type.unwrap().starts_with("text/xml")
     {
         let str = req_to_string(request, _state).await?;
-        extra.insert("raw_string".to_string(), to_raw_value(&str));
-        Ok(WebhookArgs {
-            args: PushArgsOwned { extra: Some(extra), args: HashMap::new() },
-            ..Default::default()
-        })
+        Ok(RawWebhookArgs { body: RawBody::Xml(str), metadata })
     } else if content_type.unwrap().starts_with("multipart/form-data") {
         let multipart = Multipart::from_request(request, _state)
             .await
             .map_err(IntoResponse::into_response)?;
 
-        Ok(WebhookArgs {
-            args: PushArgsOwned { extra: Some(extra), args: HashMap::new() },
-            multipart: Some(multipart),
-            wrap_body: Some(wrap_body),
-        })
+        Ok(RawWebhookArgs { body: RawBody::Multipart(multipart), metadata })
     } else {
         Err(StatusCode::UNSUPPORTED_MEDIA_TYPE.into_response())
     }
 }
 
 #[axum::async_trait]
-impl<S> FromRequest<S, axum::body::Body> for WebhookArgs
+impl<S> FromRequest<S, axum::body::Body> for RawWebhookArgs
 where
     S: Send + Sync,
 {
     type Rejection = Response;
 
     async fn from_request(request: Request, _state: &S) -> Result<Self, Self::Rejection> {
-        let args = try_from_request_body(request, _state, None, None).await?;
+        let args = try_from_request_body(request, _state, false).await?;
 
         Ok(args)
     }
@@ -309,27 +457,37 @@ lazy_static::lazy_static! {
         .collect()).unwrap_or_default();
 }
 
-pub fn build_extra(
+pub fn build_headers(
     headers: &HeaderMap,
     include_header: Option<String>,
+    is_http_trigger: bool,
 ) -> HashMap<String, Box<RawValue>> {
-    let mut args = HashMap::new();
-    let whitelist = include_header
-        .map(|s| s.split(",").map(|s| s.to_string()).collect::<Vec<_>>())
-        .unwrap_or_default();
+    let mut selected_headers = HashMap::new();
 
-    whitelist
-        .iter()
-        .chain(INCLUDE_HEADERS.iter())
-        .for_each(|h| {
-            if let Some(v) = headers.get(h) {
-                args.insert(
-                    h.to_string().to_lowercase().replace('-', "_"),
-                    to_raw_value(&v.to_str().unwrap().to_string()),
-                );
-            }
-        });
-    args
+    if is_http_trigger {
+        for (k, v) in headers.iter() {
+            selected_headers.insert(
+                k.to_string(),
+                to_raw_value(&v.to_str().unwrap_or("").to_string()),
+            );
+        }
+    } else {
+        let whitelist = include_header
+            .map(|s| s.split(",").map(|s| s.to_string()).collect::<Vec<_>>())
+            .unwrap_or_default();
+        whitelist
+            .iter()
+            .chain(INCLUDE_HEADERS.iter())
+            .for_each(|h| {
+                if let Some(v) = headers.get(h) {
+                    selected_headers.insert(
+                        h.to_string().to_lowercase().replace('-', "_"),
+                        to_raw_value(&v.to_str().unwrap_or("").to_string()),
+                    );
+                }
+            });
+    }
+    selected_headers
 }
 
 #[derive(Deserialize)]
@@ -347,37 +505,49 @@ where
     type Rejection = Response;
 
     async fn from_request_parts(parts: &mut Parts, _state: &S) -> Result<Self, Self::Rejection> {
-        Ok(DecodeQueries::from_uri(&parts.uri).unwrap_or_else(|| DecodeQueries(HashMap::new())))
+        Ok(DecodeQueries::from_uri(&parts.uri, false)
+            .unwrap_or_else(|| DecodeQueries(HashMap::new())))
     }
 }
 
 impl DecodeQueries {
-    pub fn from_uri(uri: &Uri) -> Option<Self> {
+    pub fn from_uri(uri: &Uri, is_http_trigger: bool) -> Option<Self> {
         let query = uri.query();
         if query.is_none() {
             return None;
         }
         let query = query.unwrap();
-        let include_query = serde_urlencoded::from_str::<IncludeQuery>(query)
-            .map(|x| x.include_query)
-            .ok()
-            .flatten()
-            .unwrap_or_default();
-        let parse_query_args = include_query
-            .split(",")
-            .map(|s| s.to_string())
-            .collect::<Vec<_>>();
-        let mut args = HashMap::new();
-        if !parse_query_args.is_empty() {
+        if is_http_trigger {
             let queries =
                 serde_urlencoded::from_str::<HashMap<String, String>>(query).unwrap_or_default();
-            parse_query_args.iter().for_each(|h| {
-                if let Some(v) = queries.get(h) {
-                    args.insert(h.to_string(), to_raw_value(v));
-                }
-            });
+            Some(DecodeQueries(
+                queries
+                    .into_iter()
+                    .map(|(k, v)| (k, to_raw_value(&v)))
+                    .collect(),
+            ))
+        } else {
+            let include_query = serde_urlencoded::from_str::<IncludeQuery>(query)
+                .map(|x| x.include_query)
+                .ok()
+                .flatten()
+                .unwrap_or_default();
+            let parse_query_args = include_query
+                .split(",")
+                .map(|s| s.to_string())
+                .collect::<Vec<_>>();
+            let mut args = HashMap::new();
+            if !parse_query_args.is_empty() {
+                let queries = serde_urlencoded::from_str::<HashMap<String, String>>(query)
+                    .unwrap_or_default();
+                parse_query_args.iter().for_each(|h| {
+                    if let Some(v) = queries.get(h) {
+                        args.insert(h.to_string(), to_raw_value(v));
+                    }
+                });
+            }
+            Some(DecodeQueries(args))
         }
-        Some(DecodeQueries(args))
     }
 }
 
@@ -414,69 +584,50 @@ fn restructure_cloudevents_metadata(
     }
 }
 
-trait PushArgsOwnedExt: Sized {
+impl WebhookArgs {
     async fn from_json(
-        extra: HashMap<String, Box<RawValue>>,
-        use_raw: bool,
-        force_wrap_body: bool,
-        str: String,
-    ) -> Result<Self, Response>;
-
-    async fn from_ce_json(
-        extra: HashMap<String, Box<RawValue>>,
+        mut metadata: WebhookArgsMetadata,
         use_raw: bool,
         str: String,
-    ) -> Result<Self, Response>;
-}
-
-impl PushArgsOwnedExt for PushArgsOwned {
-    async fn from_json(
-        mut extra: HashMap<String, Box<RawValue>>,
-        use_raw: bool,
-        force_wrap_body: bool,
-        str: String,
-    ) -> Result<Self, Response> {
+    ) -> Result<Self, Error> {
         if use_raw {
-            extra.insert("raw_string".to_string(), to_raw_value(&str));
+            metadata.raw_string = Some(str.clone());
         }
 
-        let wrap_body = force_wrap_body || str.len() > 0 && str.chars().next().unwrap() != '{';
+        let no_hashmap = str.len() > 0 && str.chars().next().unwrap() != '{';
 
-        if wrap_body {
+        if no_hashmap {
             let args = serde_json::from_str::<Option<Box<RawValue>>>(&str)
-                .map_err(|e| Error::BadRequest(format!("invalid json: {}", e)).into_response())?
+                .map_err(|e| Error::BadRequest(format!("invalid json: {}", e)))?
                 .unwrap_or_else(|| to_raw_value(&serde_json::Value::Null));
-            let mut hm = HashMap::new();
-            hm.insert("body".to_string(), args);
-            Ok(PushArgsOwned { extra: Some(extra), args: hm })
+
+            Ok(Self { body: Body::NoHashMap(args), metadata })
         } else {
             let hm = serde_json::from_str::<Option<HashMap<String, Box<JsonRawValue>>>>(&str)
-                .map_err(|e| Error::BadRequest(format!("invalid json: {}", e)).into_response())?
+                .map_err(|e| Error::BadRequest(format!("invalid json: {}", e)))?
                 .unwrap_or_else(HashMap::new);
-            Ok(PushArgsOwned { extra: Some(extra), args: hm })
+            Ok(Self { body: Body::HashMap(hm), metadata })
         }
     }
 
     async fn from_ce_json(
-        mut extra: HashMap<String, Box<RawValue>>,
+        mut metadata: WebhookArgsMetadata,
         use_raw: bool,
         str: String,
-    ) -> Result<Self, Response> {
+    ) -> Result<Self, Error> {
         if use_raw {
-            extra.insert("raw_string".to_string(), to_raw_value(&str));
+            metadata.raw_string = Some(str.clone());
         }
 
-        let hm = serde_json::from_str::<HashMap<String, Box<RawValue>>>(&str).map_err(|e| {
-            Error::BadRequest(format!("invalid cloudevents+json: {}", e)).into_response()
-        })?;
-        let hm = restructure_cloudevents_metadata(hm).map_err(|e| e.into_response())?;
-        Ok(PushArgsOwned { extra: Some(extra), args: hm })
+        let hm = serde_json::from_str::<HashMap<String, Box<RawValue>>>(&str)
+            .map_err(|e| Error::BadRequest(format!("invalid cloudevents+json: {}", e)))?;
+        let hm = restructure_cloudevents_metadata(hm)?;
+        Ok(Self { body: Body::HashMap(hm), metadata })
     }
 }
 
 #[cfg(test)]
 mod tests {
-    use std::collections::HashMap;
 
     use super::*;
 
@@ -514,24 +665,35 @@ mod tests {
             "data" : 1.5
         }
         "#;
-        let extra = HashMap::new();
+        let metadata = WebhookArgsMetadata::default();
 
-        let a1 = PushArgsOwned::from_ce_json(extra.clone(), false, r1.to_string())
+        let a1 = WebhookArgs::from_ce_json(metadata.clone(), false, r1.to_string())
             .await
             .expect("Failed to parse the cloudevent");
-        let a2 = PushArgsOwned::from_ce_json(extra.clone(), false, r2.to_string())
+        let a2 = WebhookArgs::from_ce_json(metadata.clone(), false, r2.to_string())
             .await
             .expect("Failed to parse the cloudevent");
 
-        a1.args.get("WEBHOOK__METADATA__").expect(
-            "CloudEvents should generate a neighboring `webhook-metadata` field in PushArgs",
-        );
-        assert_eq!(
-            a2.args
-                .get("body")
-                .expect("Cloud events with a data field with no wrapping curly brackets should be inside of a `body` field in PushArgs")
-                .to_string(),
-            "1.5"
-        );
+        match a1.body {
+            Body::HashMap(body) => {
+                body.get("WEBHOOK__METADATA__").expect(
+                    "CloudEvents should generate a neighboring `webhook-metadata` field in PushArgs",
+                );
+            }
+            _ => panic!("Expected a HashMap"),
+        }
+
+        match a2.body {
+            Body::HashMap(body) => {
+                assert_eq!(
+                    body
+                        .get("body")
+                        .expect("Cloud events with a data field with no wrapping curly brackets should be inside of a `body` field in PushArgs")
+                        .to_string(),
+                    "1.5"
+                );
+            }
+            _ => panic!("Expected a HashMap"),
+        }
     }
 }

backend/windmill-api/src/capture.rs

@@ -8,20 +8,24 @@
 
 #[cfg(feature = "http_trigger")]
 use {
-    crate::{
-        args::try_from_request_body,
-        http_triggers::{build_http_trigger_extra, HttpMethod},
-    },
+    crate::http_trigger_args::{HttpMethod, RawHttpTriggerArgs},
     axum::response::{IntoResponse, Response},
     std::collections::HashMap,
 };
 
 #[cfg(all(feature = "enterprise", feature = "gcp_trigger"))]
-use crate::gcp_triggers_ee::{
-    manage_google_subscription, process_google_push_request, validate_jwt_token,
-    CreateUpdateConfig, SubscriptionMode,
+use {
+    crate::gcp_triggers_ee::{
+        manage_google_subscription, process_google_push_request, validate_jwt_token,
+        CreateUpdateConfig, SubscriptionMode,
+    },
+    axum::extract::Request,
+    http::HeaderMap,
 };
 
+#[cfg(all(feature = "enterprise", feature = "gcp_trigger"))]
+use windmill_common::utils::empty_as_none;
+
 #[cfg(all(feature = "enterprise", feature = "sqs_trigger"))]
 use windmill_common::auth::aws::AwsAuthResourceType;
 
@@ -29,12 +33,7 @@ use windmill_common::auth::aws::AwsAuthResourceType;
     feature = "http_trigger",
     all(feature = "enterprise", feature = "gcp_trigger")
 ))]
-use {
-    axum::extract::Request,
-    http::HeaderMap,
-    serde::de::DeserializeOwned,
-    windmill_common::{error::Error, utils::empty_string_as_none},
-};
+use {serde::de::DeserializeOwned, windmill_common::error::Error};
 
 #[cfg(all(feature = "enterprise", feature = "kafka"))]
 use crate::kafka_triggers_ee::KafkaTriggerConfigConnection;
@@ -56,7 +55,7 @@ use {
 };
 
 use crate::{
-    args::WebhookArgs,
+    args::RawWebhookArgs,
     db::{ApiAuthed, DB},
     users::fetch_api_authed,
     utils::RunnableKind,
@@ -76,11 +75,12 @@ use sqlx::types::Json as SqlxJson;
 use windmill_common::{
     db::UserDB,
     error::{JsonResult, Result},
+    triggers::{RunnableFormat, RunnableFormatVersion, TriggerKind},
     utils::{not_found_if_none, paginate, Pagination, StripPath},
     worker::{to_raw_value, CLOUD_HOSTED},
 };
 
-use windmill_queue::{PushArgs, PushArgsOwned, TriggerKind};
+use windmill_queue::{PushArgs, PushArgsOwned};
 
 const KEEP_LAST: i64 = 20;
 
@@ -164,9 +164,9 @@ pub struct SqsTriggerConfig {
 pub struct GcpTriggerConfig {
     pub gcp_resource_path: String,
     pub subscription_mode: SubscriptionMode,
-    #[serde(default, deserialize_with = "empty_string_as_none")]
+    #[serde(default, deserialize_with = "empty_as_none")]
     pub subscription_id: Option<String>,
-    #[serde(default, deserialize_with = "empty_string_as_none")]
+    #[serde(default, deserialize_with = "empty_as_none")]
     pub base_endpoint: Option<String>,
     #[serde(flatten)]
     pub create_update: Option<CreateUpdateConfig>,
@@ -383,6 +383,7 @@ async fn set_gcp_trigger_config(
         gcp_config.subscription_mode,
         gcp_config.create_update,
         false,
+        capture_config.is_flow,
     )
     .await?;
     gcp_config.create_update = Some(config);
@@ -496,8 +497,8 @@ struct Capture {
     id: i64,
     created_at: chrono::DateTime<chrono::Utc>,
     trigger_kind: TriggerKind,
-    payload: SqlxJson<Box<serde_json::value::RawValue>>,
-    trigger_extra: Option<SqlxJson<Box<serde_json::value::RawValue>>>,
+    main_args: SqlxJson<Box<serde_json::value::RawValue>>,
+    preprocessor_args: Option<SqlxJson<Box<serde_json::value::RawValue>>>,
 }
 
 #[derive(Deserialize)]
@@ -525,10 +526,13 @@ async fn list_captures(
             created_at, 
             trigger_kind AS "trigger_kind: _",
             CASE 
-                WHEN pg_column_size(payload) < 40000 THEN payload 
+                WHEN pg_column_size(main_args) < 40000 THEN main_args 
                 ELSE '"WINDMILL_TOO_BIG"'::jsonb 
-            END AS "payload!: _",
-            trigger_extra AS "trigger_extra: _"
+            END AS "main_args!: _",
+            CASE
+                WHEN pg_column_size(preprocessor_args) < 40000 THEN preprocessor_args
+                ELSE '"WINDMILL_TOO_BIG"'::jsonb
+            END AS "preprocessor_args: _"
         FROM 
             capture
         WHERE 
@@ -570,8 +574,8 @@ async fn get_capture(
             id, 
             created_at, 
             trigger_kind AS "trigger_kind: _", 
-            payload AS "payload!: _", 
-            trigger_extra AS "trigger_extra: _"
+            main_args AS "main_args!: _", 
+            preprocessor_args AS "preprocessor_args: _"
         FROM 
             capture
         WHERE 
@@ -820,15 +824,15 @@ pub async fn insert_capture_payload(
     path: &str,
     is_flow: bool,
     trigger_kind: &TriggerKind,
-    payload: PushArgsOwned,
-    trigger_extra: Option<Box<RawValue>>,
+    main_args: PushArgsOwned,
+    preprocessor_args: PushArgsOwned,
     owner: &str,
 ) -> Result<()> {
     sqlx::query!(
         r#"
     INSERT INTO 
         capture (
-            workspace_id, path, is_flow, trigger_kind, payload, trigger_extra, created_by
+            workspace_id, path, is_flow, trigger_kind, main_args, preprocessor_args, created_by
         )
     VALUES (
         $1, $2, $3, $4, $5, $6, $7
@@ -838,11 +842,9 @@ pub async fn insert_capture_payload(
         path,
         is_flow,
         trigger_kind as &TriggerKind,
-        SqlxJson(to_raw_value(&PushArgs {
-            args: &payload.args,
-            extra: payload.extra
-        })) as SqlxJson<Box<RawValue>>,
-        trigger_extra.map(SqlxJson) as Option<SqlxJson<Box<RawValue>>>,
+        SqlxJson(PushArgs { args: &main_args.args, extra: main_args.extra }) as SqlxJson<PushArgs>,
+        SqlxJson(PushArgs { args: &preprocessor_args.args, extra: preprocessor_args.extra })
+            as SqlxJson<PushArgs>,
         owner,
     )
     .execute(db)
@@ -856,7 +858,7 @@ pub async fn insert_capture_payload(
 async fn webhook_payload(
     Extension(db): Extension<DB>,
     Path((w_id, runnable_kind, path)): Path<(String, RunnableKind, StripPath)>,
-    args: WebhookArgs,
+    args: RawWebhookArgs,
 ) -> Result<StatusCode> {
     let (owner, email) = get_active_capture_owner_and_email(
         &db,
@@ -868,7 +870,15 @@ async fn webhook_payload(
     .await?;
 
     let authed = fetch_api_authed(owner.clone(), email, &w_id, &db, None).await?;
-    let args = args.to_push_args_owned(&authed, &db, &w_id).await?;
+
+    let args = args.process_args(&authed, &db, &w_id, None).await?;
+
+    let preprocessor_args = args.clone().to_args_from_format(RunnableFormat {
+        has_preprocessor: true,
+        version: RunnableFormatVersion::V2,
+    })?;
+
+    let main_args = args.to_main_args()?;
 
     insert_capture_payload(
         &db,
@@ -876,12 +886,8 @@ async fn webhook_payload(
         &path.to_path(),
         matches!(runnable_kind, RunnableKind::Flow),
         &TriggerKind::Webhook,
-        args,
-        Some(to_raw_value(&serde_json::json!({
-            "wm_trigger": {
-                "kind": "webhook",
-            }
-        }))),
+        main_args,
+        preprocessor_args,
         &owner,
     )
     .await?;
@@ -897,6 +903,8 @@ async fn gcp_payload(
     headers: HeaderMap,
     request: Request,
 ) -> Result<StatusCode> {
+    use crate::{gcp_triggers_ee::GcpTrigger, trigger_helpers::TriggerJobArgs};
+
     let is_flow = matches!(runnable_kind, RunnableKind::Flow);
     let (gcp_trigger_config, owner, email): (GcpTriggerConfig, _, _) =
         get_capture_trigger_config_and_owner(&db, &w_id, &path, is_flow, &TriggerKind::Gcp).await?;
@@ -918,9 +926,9 @@ async fn gcp_payload(
     )
     .await?;
 
-    let (args, extra) = process_google_push_request(headers, request).await?;
+    let (payload, gcp) = process_google_push_request(headers, request).await?;
 
-    let payload = PushArgsOwned { args, extra: None };
+    let (main_args, preprocessor_args) = GcpTrigger::build_capture_payloads(payload, gcp);
 
     let _ = insert_capture_payload(
         &db,
@@ -928,8 +936,8 @@ async fn gcp_payload(
         &path,
         is_flow,
         &TriggerKind::Gcp,
-        payload,
-        Some(to_raw_value(&extra)),
+        main_args,
+        preprocessor_args,
         &owner,
     )
     .await?;
@@ -941,10 +949,7 @@ async fn gcp_payload(
 async fn http_payload(
     Extension(db): Extension<DB>,
     Path((w_id, runnable_kind, path, route_path)): Path<(String, RunnableKind, String, StripPath)>,
-    Query(query): Query<HashMap<String, String>>,
-    method: http::Method,
-    headers: HeaderMap,
-    request: Request,
+    args: RawHttpTriggerArgs,
 ) -> std::result::Result<StatusCode, Response> {
     let path = path.replace(".", "/");
     let is_flow = matches!(runnable_kind, RunnableKind::Flow);
@@ -954,20 +959,17 @@ async fn http_payload(
             .await
             .map_err(|e| e.into_response())?;
 
-    let args = try_from_request_body(
-        request,
-        &(),
-        http_trigger_config.raw_string,
-        http_trigger_config.wrap_body,
-    )
-    .await
-    .map_err(|e| e.into_response())?;
-
     let authed = fetch_api_authed(owner.clone(), email, &w_id, &db, None)
         .await
         .map_err(|e| e.into_response())?;
-    let mut args = args
-        .to_push_args_owned(&authed, &db, &w_id)
+
+    let args = args
+        .process_args(
+            &authed,
+            &db,
+            &w_id,
+            http_trigger_config.raw_string.unwrap_or(false),
+        )
         .await
         .map_err(|e| e.into_response())?;
 
@@ -985,31 +987,23 @@ async fn http_payload(
         .map(|(k, v)| (k.to_string(), v.to_string()))
         .collect();
 
-    let extra = args.extra.get_or_insert_with(HashMap::new);
+    let preprocessor_args = args
+        .clone()
+        .to_v2_preprocessor_args(&http_trigger_config.route_path, &route_path, &params)
+        .map_err(|e| e.into_response())?;
 
-    extra.insert(
-        "wm_trigger".to_string(),
-        build_http_trigger_extra(
-            &http_trigger_config.route_path,
-            route_path,
-            &method,
-            &params,
-            &query,
-            &headers,
-        )
-        .await,
-    );
+    let main_args = args
+        .to_main_args(http_trigger_config.wrap_body.unwrap_or(false))
+        .map_err(|e| e.into_response())?;
 
-    let extra = Some(to_raw_value(&extra));
-    args.extra = None;
     insert_capture_payload(
         &db,
         &w_id,
         &path,
         is_flow,
         &TriggerKind::Http,
-        args,
-        extra,
+        main_args,
+        preprocessor_args,
         &owner,
     )
     .await

backend/windmill-api/src/db.rs

@@ -782,10 +782,32 @@ async fn fix_job_completed_index(db: &DB) -> Result<(), Error> {
         .execute(db)
         .await?;
     });
+
+    run_windmill_migration!("job_completed_completed_at", db, |tx| {
+        sqlx::query!(
+            "CREATE INDEX CONCURRENTLY IF NOT EXISTS ix_job_completed_completed_at ON v2_job_completed (completed_at DESC)"
+        )
+        .execute(db)
+        .await?;
+    });
+
+    run_windmill_migration!("alerts_by_workspace", db, |tx| {
+        sqlx::query!(
+            "CREATE INDEX CONCURRENTLY IF NOT EXISTS alerts_by_workspace ON alerts (workspace_id);"
+        )
+        .execute(db)
+        .await?;
+    });
+
+    run_windmill_migration!("remove_redundant_log_file_index", db, |tx| {
+        sqlx::query!("DROP INDEX CONCURRENTLY IF EXISTS log_file_hostname_log_ts_idx")
+            .execute(db)
+            .await?;
+    });
     Ok(())
 }
 
-#[derive(Clone, Debug)]
+#[derive(Clone, Debug, Hash, Eq, PartialEq)]
 pub struct ApiAuthed {
     pub email: String,
     pub username: String,

backend/windmill-api/src/gcp_triggers_ee.rs

@@ -1,4 +1,5 @@
 use crate::db::{ApiAuthed, DB};
+use crate::trigger_helpers::TriggerJobArgs;
 use axum::{extract::Request, Router};
 use http::HeaderMap;
 use serde::{Deserialize, Serialize};
@@ -7,9 +8,11 @@ use sqlx::prelude::FromRow;
 use sqlx::types::Json as SqlxJson;
 use std::collections::HashMap;
 use windmill_common::db::UserDB;
+use windmill_common::worker::to_raw_value;
 use windmill_common::{
     error::{Error as WindmillError, Result as WindmillResult},
-    utils::empty_string_as_none,
+    triggers::TriggerKind,
+    utils::empty_as_none,
 };
 
 #[derive(sqlx::Type, Debug, Deserialize, Serialize)]
@@ -30,9 +33,9 @@ impl Default for DeliveryType {
 #[derive(FromRow, Deserialize, Serialize, Debug)]
 #[allow(unused)]
 pub struct PushConfig {
-    #[serde(deserialize_with = "empty_string_as_none")]
+    #[serde(deserialize_with = "empty_as_none")]
     route_path: Option<String>,
-    #[serde(deserialize_with = "empty_string_as_none")]
+    #[serde(deserialize_with = "empty_as_none")]
     audience: Option<String>,
     authenticate: bool,
     base_endpoint: String,
@@ -41,7 +44,7 @@ pub struct PushConfig {
 #[allow(unused)]
 pub struct CreateUpdateConfig {
     pub delivery_type: DeliveryType,
-    #[serde(default, deserialize_with = "empty_string_as_none")]
+    #[serde(default, deserialize_with = "empty_as_none")]
     pub subscription_id: Option<String>,
     pub delivery_config: Option<SqlxJson<PushConfig>>,
 }
@@ -83,6 +86,7 @@ pub async fn manage_google_subscription(
     _subscription_mode: SubscriptionMode,
     _create_update_config: Option<CreateUpdateConfig>,
     _trigger_mode: bool,
+    _is_flow: bool,
 ) -> WindmillResult<CreateUpdateConfig> {
     Ok(CreateUpdateConfig::default())
 }
@@ -90,14 +94,8 @@ pub async fn manage_google_subscription(
 pub async fn process_google_push_request(
     _headers: HeaderMap,
     _request: Request,
-) -> Result<
-    (
-        HashMap<String, Box<RawValue>>,
-        Option<HashMap<String, Box<RawValue>>>,
-    ),
-    WindmillError,
-> {
-    Ok((HashMap::new(), None))
+) -> Result<(String, HashMap<String, Box<RawValue>>), WindmillError> {
+    Ok((String::new(), HashMap::new()))
 }
 
 pub async fn validate_jwt_token(
@@ -137,3 +135,13 @@ pub struct GcpTrigger {
     pub last_server_ping: Option<chrono::DateTime<chrono::Utc>>,
     pub enabled: bool,
 }
+
+impl TriggerJobArgs<String> for GcpTrigger {
+    fn v1_payload_fn(payload: String) -> HashMap<String, Box<RawValue>> {
+        HashMap::from([("payload".to_string(), to_raw_value(&payload))])
+    }
+
+    fn trigger_kind() -> TriggerKind {
+        TriggerKind::Gcp
+    }
+}

backend/windmill-api/src/http_trigger_args.rs

@@ -0,0 +1,210 @@
+use std::collections::HashMap;
+
+use axum::{
+    extract::{FromRequest, Request},
+    response::Response,
+};
+use serde::{Deserialize, Serialize};
+use serde_json::value::RawValue;
+use windmill_common::{
+    error::Error,
+    triggers::{RunnableFormat, RunnableFormatVersion},
+    worker::to_raw_value,
+    DB,
+};
+use windmill_queue::PushArgsOwned;
+
+use crate::{
+    args::{try_from_request_body, Body, RawWebhookArgs, WebhookArgs, WebhookArgsMetadata},
+    db::ApiAuthed,
+};
+
+pub struct RawHttpTriggerArgs(pub RawWebhookArgs);
+
+#[derive(Serialize, Deserialize, sqlx::Type, Debug, Clone, Hash, Eq, PartialEq)]
+#[sqlx(type_name = "HTTP_METHOD", rename_all = "lowercase")]
+#[serde(rename_all = "lowercase")]
+pub enum HttpMethod {
+    Get,
+    Post,
+    Put,
+    Delete,
+    Patch,
+}
+
+impl TryFrom<&http::Method> for HttpMethod {
+    type Error = Error;
+    fn try_from(method: &http::Method) -> Result<Self, Self::Error> {
+        match method {
+            &http::Method::GET => Ok(HttpMethod::Get),
+            &http::Method::POST => Ok(HttpMethod::Post),
+            &http::Method::PUT => Ok(HttpMethod::Put),
+            &http::Method::DELETE => Ok(HttpMethod::Delete),
+            &http::Method::PATCH => Ok(HttpMethod::Patch),
+            _ => Err(Error::BadRequest("Invalid HTTP method".to_string())),
+        }
+    }
+}
+
+#[axum::async_trait]
+impl<S> FromRequest<S, axum::body::Body> for RawHttpTriggerArgs
+where
+    S: Send + Sync,
+{
+    type Rejection = Response;
+
+    async fn from_request(request: Request, _state: &S) -> Result<Self, Self::Rejection> {
+        let args = try_from_request_body(request, _state, true).await?;
+
+        Ok(Self(args))
+    }
+}
+
+#[derive(Debug, Clone)]
+pub struct HttpTriggerArgs(pub WebhookArgs);
+
+impl RawHttpTriggerArgs {
+    pub async fn process_args(
+        self,
+        authed: &ApiAuthed,
+        db: &DB,
+        w_id: &str,
+        use_raw: bool,
+    ) -> Result<HttpTriggerArgs, Error> {
+        if self.0.metadata.query_use_raw || self.0.metadata.query_wrap_body {
+            return Err(Error::BadRequest(
+                "Specifying use raw or wrap body with query args is not supported anymore on http routes, please set it in the trigger config".to_string(),
+            )
+            .into());
+        }
+
+        let args = self.0.process_args(authed, db, w_id, Some(use_raw)).await?;
+
+        Ok(HttpTriggerArgs(args))
+    }
+}
+
+#[derive(Serialize)]
+struct HttpTriggerPreprocessorEvent<'a> {
+    kind: String,
+    route: &'a str,
+    path: &'a str,
+    body: Box<RawValue>,
+    raw_string: Option<String>,
+    params: &'a HashMap<String, String>,
+    headers: HashMap<String, Box<RawValue>>,
+    query: HashMap<String, Box<RawValue>>,
+    method: HttpMethod,
+}
+
+#[derive(Serialize)]
+struct HttpTriggerWmTrigger<'a> {
+    route: &'a str,
+    path: &'a str,
+    params: &'a HashMap<String, String>,
+    query: &'a HashMap<String, Box<RawValue>>,
+    headers: &'a HashMap<String, Box<RawValue>>,
+    method: HttpMethod,
+}
+
+impl HttpTriggerArgs {
+    pub fn to_main_args(self, wrap_body: bool) -> Result<PushArgsOwned, Error> {
+        let mut extra = HashMap::new();
+
+        let WebhookArgsMetadata { raw_string, .. } = self.0.metadata;
+
+        if let Some(raw_string) = raw_string {
+            extra.insert("raw_string".to_string(), to_raw_value(&raw_string));
+        }
+
+        let extra = if extra.is_empty() { None } else { Some(extra) };
+
+        match self.0.body {
+            Body::HashMap(mut body) => {
+                if wrap_body {
+                    body = HashMap::from([("body".to_string(), to_raw_value(&body))]);
+                }
+                Ok(PushArgsOwned { args: body, extra })
+            }
+            Body::NoHashMap(args) => {
+                let mut hm = HashMap::new();
+                hm.insert("body".to_string(), args);
+                Ok(PushArgsOwned { args: hm, extra })
+            }
+        }
+    }
+
+    pub fn to_args_from_format(
+        self,
+        route_path: &str,
+        called_path: &str,
+        params: &HashMap<String, String>,
+        format: RunnableFormat,
+        wrap_body: bool,
+    ) -> Result<PushArgsOwned, Error> {
+        match format {
+            RunnableFormat { has_preprocessor: true, version: RunnableFormatVersion::V2 } => {
+                // we don't care about wrap_body in v2
+                self.to_v2_preprocessor_args(route_path, called_path, params)
+            }
+            RunnableFormat { has_preprocessor: true, version: RunnableFormatVersion::V1 } => {
+                self.to_v1_preprocessor_args(route_path, called_path, params, wrap_body)
+            }
+            RunnableFormat { has_preprocessor: false, .. } => self.to_main_args(wrap_body),
+        }
+    }
+
+    fn to_v1_preprocessor_args(
+        self,
+        route_path: &str,
+        called_path: &str,
+        params: &HashMap<String, String>,
+        wrap_body: bool,
+    ) -> Result<PushArgsOwned, Error> {
+        let mut extra = HashMap::new();
+        let mut wm_trigger = HashMap::new();
+        wm_trigger.insert("kind".to_string(), to_raw_value(&"http".to_string()));
+        wm_trigger.insert(
+            "http".to_string(),
+            to_raw_value(&HttpTriggerWmTrigger {
+                route: route_path,
+                path: called_path,
+                method: (&self.0.metadata.method).try_into()?,
+                params,
+                query: &self.0.metadata.query,
+                headers: &self.0.metadata.headers,
+            }),
+        );
+        extra.insert("wm_trigger".to_string(), to_raw_value(&wm_trigger));
+
+        let mut args = self.to_main_args(wrap_body)?;
+
+        args.extra.get_or_insert_default().extend(extra);
+
+        Ok(args)
+    }
+
+    pub fn to_v2_preprocessor_args(
+        self,
+        route_path: &str,
+        called_path: &str,
+        params: &HashMap<String, String>,
+    ) -> Result<PushArgsOwned, Error> {
+        let mut args = HashMap::new();
+        args.insert(
+            "event".to_string(),
+            to_raw_value(&HttpTriggerPreprocessorEvent {
+                kind: "http".to_string(),
+                body: to_raw_value(&self.0.body),
+                raw_string: self.0.metadata.raw_string,
+                headers: self.0.metadata.headers,
+                query: self.0.metadata.query,
+                method: (&self.0.metadata.method).try_into()?,
+                route: route_path,
+                path: called_path,
+                params,
+            }),
+        );
+        Ok(PushArgsOwned { args, extra: None })
+    }
+}

backend/windmill-api/src/http_trigger_auth.rs

@@ -414,7 +414,7 @@ pub enum Encoding {
     Base64Uri,
     Hex,
 }
-#[derive(Debug, Serialize, Deserialize)]
+#[derive(Debug, Serialize, Deserialize, Clone)]
 pub struct SignatureAuthenticationMethod {
     algorithm: HmacAlgorithm,
     encoding: Encoding,
@@ -426,20 +426,20 @@ pub struct SignatureConfigData<'config> {
     secret_key: &'config str,
 }
 
-#[derive(Debug, Serialize, Deserialize)]
+#[derive(Debug, Serialize, Deserialize, Clone)]
 pub struct SignatureAuthentication {
     signature_provider: WebhookType,
     secret_key: String,
     authentication_config: Option<SignatureAuthenticationMethod>,
 }
 
-#[derive(Debug, Serialize, Deserialize)]
+#[derive(Debug, Serialize, Deserialize, Clone)]
 pub struct BasicAuthAuthentication {
     pub username: String,
     pub password: String,
 }
 
-#[derive(Debug, Serialize, Deserialize)]
+#[derive(Debug, Serialize, Deserialize, Clone)]
 pub struct ApiKeyAuthentication {
     api_key_header: String,
     api_key_secret: String,
@@ -558,7 +558,7 @@ pub fn verify_hmac_signature(
     Ok(())
 }
 
-#[derive(Debug, Serialize, Deserialize)]
+#[derive(Debug, Serialize, Deserialize, Clone)]
 #[serde(untagged)]
 pub enum AuthenticationMethod {
     Signature(SignatureAuthentication),

backend/windmill-api/src/http_triggers.rs

@@ -1,10 +1,11 @@
-use crate::http_trigger_auth::{self};
+#[cfg(feature = "http_trigger")]
+use crate::http_trigger_args::{HttpMethod, RawHttpTriggerArgs};
 #[cfg(feature = "parquet")]
 use crate::job_helpers_ee::get_workspace_s3_resource;
 use crate::resources::try_get_resource_from_db_as;
-use crate::utils::non_empty_str;
+use crate::trigger_helpers::{get_runnable_format, RunnableId};
+use crate::utils::{non_empty_str, ExpiringCacheEntry};
 use crate::{
-    args::try_from_request_body,
     auth::{AuthCache, OptTokened},
     db::{ApiAuthed, DB},
     jobs::{
@@ -15,7 +16,7 @@ use crate::{
 };
 use axum::response::Response;
 use axum::{
-    extract::{Path, Query, Request},
+    extract::{Path, Query},
     response::IntoResponse,
     routing::{delete, get, post},
     Extension, Json, Router,
@@ -23,11 +24,14 @@ use axum::{
 #[cfg(feature = "parquet")]
 use http::header::IF_NONE_MATCH;
 use http::{HeaderMap, StatusCode};
+use quick_cache::sync::Cache;
 use serde::{Deserialize, Serialize};
 use sql_builder::{bind::Bind, SqlBuilder};
 use sqlx::prelude::FromRow;
+use sqlx::PgTransaction;
 use std::borrow::Cow;
 use std::{collections::HashMap, sync::Arc};
+use tokio::sync::{RwLock, RwLockReadGuard};
 use tower_http::cors::CorsLayer;
 use windmill_audit::{audit_ee::audit_log, ActionKind};
 use windmill_common::error::Error;
@@ -37,9 +41,11 @@ use windmill_common::{
     db::UserDB,
     error::{self, JsonResult},
     s3_helpers::S3Object,
+    triggers::TriggerKind,
     utils::{not_found_if_none, paginate, require_admin, Pagination, StripPath},
-    worker::{to_raw_value, CLOUD_HOSTED},
+    worker::CLOUD_HOSTED,
 };
+use windmill_git_sync::handle_deployment_metadata;
 
 lazy_static::lazy_static! {
     static ref ROUTE_PATH_KEY_RE: regex::Regex = regex::Regex::new(r"/?:[-\w]+").unwrap();
@@ -81,31 +87,6 @@ pub fn workspaced_service() -> Router {
         .route("/route_exists", post(exists_route))
 }
 
-#[derive(Serialize, Deserialize, sqlx::Type, Debug)]
-#[sqlx(type_name = "HTTP_METHOD", rename_all = "lowercase")]
-#[serde(rename_all = "lowercase")]
-pub enum HttpMethod {
-    Get,
-    Post,
-    Put,
-    Delete,
-    Patch,
-}
-
-impl TryFrom<&http::Method> for HttpMethod {
-    type Error = error::Error;
-    fn try_from(method: &http::Method) -> Result<Self, Self::Error> {
-        match method {
-            &http::Method::GET => Ok(HttpMethod::Get),
-            &http::Method::POST => Ok(HttpMethod::Post),
-            &http::Method::PUT => Ok(HttpMethod::Put),
-            &http::Method::DELETE => Ok(HttpMethod::Delete),
-            &http::Method::PATCH => Ok(HttpMethod::Patch),
-            _ => Err(error::Error::BadRequest("Invalid HTTP method".to_string())),
-        }
-    }
-}
-
 #[derive(sqlx::Type, Serialize, Deserialize, Debug, PartialEq, Clone, Copy)]
 #[sqlx(type_name = "AUTHENTICATION_METHOD", rename_all = "snake_case")]
 #[serde(rename_all(serialize = "snake_case", deserialize = "snake_case"))]
@@ -306,6 +287,16 @@ fn validate_authentication_method(
     Ok(())
 }
 
+async fn increase_trigger_version_and_commit(mut tx: PgTransaction<'_>) -> error::Result<()> {
+    sqlx::query!("SELECT nextval('http_trigger_version_seq')",)
+        .fetch_one(&mut *tx)
+        .await?;
+
+    tx.commit().await?;
+
+    Ok(())
+}
+
 async fn create_trigger(
     authed: ApiAuthed,
     Extension(db): Extension<DB>,
@@ -405,7 +396,18 @@ async fn create_trigger(
     )
     .await?;
 
-    tx.commit().await?;
+    increase_trigger_version_and_commit(tx).await?;
+
+    handle_deployment_metadata(
+        &authed.email,
+        &authed.username,
+        &db,
+        &w_id,
+        windmill_git_sync::DeployedObject::HttpTrigger { path: ct.path.clone() },
+        Some(format!("HTTP trigger '{}' created", ct.path)),
+        true,
+    )
+    .await?;
 
     Ok((StatusCode::CREATED, format!("{}", ct.path)))
 }
@@ -554,20 +556,32 @@ async fn update_trigger(
         &mut *tx,
         &authed,
         "http_triggers.update",
-        ActionKind::Create,
+        ActionKind::Update,
         &w_id,
-        Some(path),
+        Some(&ct.path),
         None,
     )
     .await?;
 
-    tx.commit().await?;
+    increase_trigger_version_and_commit(tx).await?;
 
-    Ok(path.to_string())
+    handle_deployment_metadata(
+        &authed.email,
+        &authed.username,
+        &db,
+        &w_id,
+        windmill_git_sync::DeployedObject::HttpTrigger { path: ct.path.clone() },
+        Some(format!("HTTP trigger '{}' updated", ct.path)),
+        true,
+    )
+    .await?;
+
+    Ok(ct.path.to_string())
 }
 
 async fn delete_trigger(
     authed: ApiAuthed,
+    Extension(db): Extension<DB>,
     Extension(user_db): Extension<UserDB>,
     Path((w_id, path)): Path<(String, StripPath)>,
 ) -> error::Result<String> {
@@ -595,7 +609,18 @@ async fn delete_trigger(
     )
     .await?;
 
-    tx.commit().await?;
+    increase_trigger_version_and_commit(tx).await?;
+
+    handle_deployment_metadata(
+        &authed.email,
+        &authed.username,
+        &db,
+        &w_id,
+        windmill_git_sync::DeployedObject::HttpTrigger { path: path.to_string() },
+        Some(format!("HTTP trigger '{}' deleted", path)),
+        true,
+    )
+    .await?;
 
     Ok(format!("HTTP trigger {path} deleted"))
 }
@@ -708,8 +733,8 @@ async fn exists_route(
     Ok(Json(exists))
 }
 
-#[derive(Debug, Deserialize)]
-struct TriggerRoute {
+#[derive(Debug, Deserialize, Clone)]
+pub struct TriggerRoute {
     path: String,
     script_path: String,
     is_flow: bool,
@@ -727,6 +752,145 @@ struct TriggerRoute {
     raw_string: bool,
 }
 
+pub struct RoutersCache {
+    routers: HashMap<HttpMethod, matchit::Router<TriggerRoute>>,
+    version: i64,
+}
+
+lazy_static::lazy_static! {
+    static ref HTTP_ACCESS_CACHE: Cache<(String, String, ApiAuthed), ExpiringCacheEntry<()>> = Cache::new(100);
+    static ref HTTP_AUTH_CACHE: Cache<(String, String, ApiAuthed), ExpiringCacheEntry<crate::http_trigger_auth::AuthenticationMethod>> = Cache::new(100);
+
+    static ref HTTP_ROUTERS_CACHE: RwLock<RoutersCache> = RwLock::new(RoutersCache {
+        routers: HashMap::new(),
+        version: 0,
+    });
+}
+
+pub async fn refresh_routers_loop(
+    db: &DB,
+    mut killpill_rx: tokio::sync::broadcast::Receiver<()>,
+) -> () {
+    match refresh_routers(db).await {
+        Ok(_) => {
+            tracing::info!("Loaded HTTP routers");
+        }
+        Err(err) => {
+            tracing::error!("Error loading HTTP routers: {err:#}");
+        }
+    };
+    let db = db.clone();
+    tokio::spawn(async move {
+        loop {
+            tokio::select! {
+                _ = killpill_rx.recv() => {
+                    break;
+                }
+                _ = tokio::time::sleep(std::time::Duration::from_secs(60)) => {
+                    match refresh_routers(&db).await {
+                        Ok((true, _)) => {
+                            tracing::info!("Refreshed HTTP routers");
+                        }
+                        Err(err) => {
+                            tracing::error!("Error refreshing HTTP routers: {err:#}");
+                        }
+                        _ => {}
+                    }
+                }
+            }
+        }
+    });
+}
+
+pub async fn refresh_routers(db: &DB) -> Result<(bool, RwLockReadGuard<'_, RoutersCache>), Error> {
+    let version = sqlx::query_scalar!("SELECT last_value FROM http_trigger_version_seq",)
+        .fetch_one(db)
+        .await?;
+    let routers_cache = HTTP_ROUTERS_CACHE.read().await;
+    if routers_cache.version == 0 || version > routers_cache.version {
+        drop(routers_cache);
+        let mut routers = HashMap::new();
+
+        for http_method in [
+            HttpMethod::Get,
+            HttpMethod::Post,
+            HttpMethod::Put,
+            HttpMethod::Patch,
+            HttpMethod::Delete,
+        ] {
+            let triggers = sqlx::query_as!(
+                TriggerRoute,
+                r#"
+                    SELECT 
+                        path, 
+                        script_path, 
+                        is_flow, 
+                        route_path, 
+                        authentication_resource_path,
+                        workspace_id, 
+                        is_async, 
+                        authentication_method  AS "authentication_method: _", 
+                        edited_by, 
+                        email, 
+                        static_asset_config AS "static_asset_config: _",
+                        wrap_body,
+                        raw_string,
+                        workspaced_route,
+                        is_static_website
+                    FROM 
+                        http_trigger 
+                    WHERE 
+                        http_method = $1
+                    "#,
+                &http_method as &HttpMethod
+            )
+            .fetch_all(db)
+            .await?;
+
+            let mut router = matchit::Router::new();
+
+            for trigger in triggers {
+                let full_path = if trigger.workspaced_route || *CLOUD_HOSTED {
+                    format!("/{}/{}", trigger.workspace_id, trigger.route_path)
+                } else {
+                    format!("/{}", trigger.route_path)
+                };
+
+                if trigger.is_static_website {
+                    router
+                        .insert(format!("{}/*wm_subpath", full_path), trigger.clone())
+                        .unwrap_or_else(|e| {
+                            tracing::warn!(
+                                "Failed to consider http trigger route {}/*wm_subpath: {:?}",
+                                full_path,
+                                e,
+                            );
+                        });
+                }
+                router
+                    .insert(full_path.clone(), trigger.clone())
+                    .unwrap_or_else(|e| {
+                        tracing::warn!(
+                            "Failed to consider http trigger route {}: {:?}",
+                            full_path,
+                            e,
+                        );
+                    });
+            }
+
+            routers.insert(http_method, router);
+        }
+
+        let mut routers_cache = HTTP_ROUTERS_CACHE.write().await;
+        *routers_cache = RoutersCache { routers, version };
+
+        Ok((true, routers_cache.downgrade()))
+    } else {
+        tracing::debug!("No HTTP routers refresh needed");
+        Ok((false, routers_cache))
+    }
+}
+
 async fn get_http_route_trigger(
     route_path: &str,
     auth_cache: &Arc<AuthCache>,
@@ -736,111 +900,30 @@ async fn get_http_route_trigger(
     method: &http::Method,
 ) -> error::Result<(TriggerRoute, String, HashMap<String, String>, ApiAuthed)> {
     let http_method: HttpMethod = method.try_into()?;
-    let (mut triggers, route_path) = if *CLOUD_HOSTED {
-        let mut splitted = route_path.split("/");
-        let w_id = splitted.next().ok_or_else(|| {
-            error::Error::BadRequest("Missing workspace id in route path".to_string())
-        })?;
-        let route_path = StripPath(splitted.collect::<Vec<_>>().join("/"));
-        let triggers = sqlx::query_as!(
-            TriggerRoute,
-            r#"
-            SELECT 
-                path, 
-                script_path, 
-                is_flow, 
-                route_path, 
-                workspace_id, 
-                is_async, 
-                authentication_method  AS "authentication_method: _", 
-                edited_by, 
-                email,
-                static_asset_config AS "static_asset_config: _",
-                wrap_body,
-                raw_string,
-                workspaced_route,
-                is_static_website,
-                authentication_resource_path
-            FROM 
-                http_trigger 
-            WHERE 
-                workspace_id = $1 AND 
-                http_method = $2
-            "#,
-            w_id,
-            http_method as HttpMethod
-        )
-        .fetch_all(db)
-        .await?;
-        (triggers, route_path)
+
+    let requested_path = format!("/{}", route_path);
+
+    let routers_cache = HTTP_ROUTERS_CACHE.read().await;
+
+    let routers_cache = if routers_cache.routers.is_empty() {
+        tracing::warn!("HTTP routers are not loaded, loading from db");
+        let (_, routers_cache) = refresh_routers(db).await?;
+        routers_cache
     } else {
-        let triggers = sqlx::query_as!(
-            TriggerRoute,
-            r#"
-            SELECT 
-                path, 
-                script_path, 
-                is_flow, 
-                route_path, 
-                authentication_resource_path,
-                workspace_id, 
-                is_async, 
-                authentication_method  AS "authentication_method: _", 
-                edited_by, 
-                email, 
-                static_asset_config AS "static_asset_config: _",
-                wrap_body,
-                raw_string,
-                workspaced_route,
-                is_static_website
-            FROM 
-                http_trigger 
-            WHERE 
-                http_method = $1
-            "#,
-            http_method as HttpMethod
-        )
-        .fetch_all(db)
-        .await?;
-        (triggers, StripPath(route_path.to_string()))
+        routers_cache
     };
 
-    let mut router = matchit::Router::new();
+    let router = routers_cache
+        .routers
+        .get(&http_method)
+        .ok_or(error::Error::internal_err(
+            "HTTP routers could not be loaded".to_string(),
+        ))?;
 
-    for (idx, trigger) in triggers.iter().enumerate() {
-        let route_path = match trigger.workspaced_route {
-            true => format!("{}/{}", &trigger.workspace_id, &trigger.route_path),
-            _ => trigger.route_path.clone(),
-        };
-        if trigger.is_static_website {
-            router
-                .insert(format!("/{}/*wm_subpath", route_path), idx)
-                .unwrap_or_else(|e| {
-                    tracing::warn!(
-                        "Failed to consider http trigger route {}: {:?}",
-                        route_path,
-                        e,
-                    );
-                });
-        }
-        router
-            .insert(format!("/{}", route_path), idx)
-            .unwrap_or_else(|e| {
-                tracing::warn!(
-                    "Failed to consider http trigger route {}: {:?}",
-                    route_path,
-                    e,
-                );
-            });
-    }
+    let trigger_match = router.at(requested_path.as_str()).ok();
 
-    let requested_path = format!("/{}", route_path.0);
-    let trigger_idx = router.at(requested_path.as_str()).ok();
-
-    let matchit::Match { value: trigger_idx, params } =
-        not_found_if_none(trigger_idx, "Trigger", requested_path.as_str())?;
-
-    let trigger = triggers.remove(trigger_idx.to_owned());
+    let matchit::Match { value: trigger, params } =
+        not_found_if_none(trigger_match, "Trigger", requested_path.as_str())?;
 
     let params: HashMap<String, String> = params
         .iter()
@@ -857,25 +940,49 @@ async fn get_http_route_trigger(
         };
         if let Some(authed) = opt_authed {
             // check that the user has access to the trigger
-            let mut tx = user_db.begin(&authed).await?;
-            let exists = sqlx::query_scalar!(
-                r#"
-                SELECT EXISTS(
-                    SELECT 1 
-                    FROM 
-                        http_trigger 
-                    WHERE 
-                        workspace_id = $1 AND 
-                        path = $2
-                )
-                "#,
-                trigger.workspace_id,
-                trigger.path
-            )
-            .fetch_one(&mut *tx)
-            .await?
-            .unwrap_or(false);
-            tx.commit().await?;
+            let cache_key = (
+                trigger.workspace_id.clone(),
+                trigger.path.clone(),
+                authed.clone(),
+            );
+            let exists = match HTTP_ACCESS_CACHE.get(&cache_key) {
+                Some(cache_entry) if cache_entry.expiry > std::time::Instant::now() => {
+                    tracing::debug!("HTTP access cache hit for trigger {}", trigger.path);
+                    true
+                }
+                _ => {
+                    tracing::debug!("HTTP access cache miss for trigger {}", trigger.path);
+                    let mut tx = user_db.begin(&authed).await?;
+                    let exists = sqlx::query_scalar!(
+                        r#"
+                        SELECT EXISTS(
+                            SELECT 1
+                            FROM 
+                                http_trigger 
+                            WHERE 
+                                workspace_id = $1 AND 
+                                path = $2
+                        )
+                        "#,
+                        trigger.workspace_id,
+                        trigger.path
+                    )
+                    .fetch_one(&mut *tx)
+                    .await?
+                    .unwrap_or(false);
+                    if exists {
+                        HTTP_ACCESS_CACHE.insert(
+                            cache_key,
+                            ExpiringCacheEntry {
+                                value: (),
+                                expiry: std::time::Instant::now()
+                                    + std::time::Duration::from_secs(10),
+                            },
+                        );
+                    }
+                    exists
+                }
+            };
             if exists {
                 Some(authed.display_username().to_owned())
             } else {
@@ -899,33 +1006,7 @@ async fn get_http_route_trigger(
     )
     .await?;
 
-    Ok((trigger, route_path.0, params, authed))
-}
-
-pub async fn build_http_trigger_extra(
-    route_path: &str,
-    called_path: &str,
-    method: &http::Method,
-    params: &HashMap<String, String>,
-    query: &HashMap<String, String>,
-    headers: &HeaderMap,
-) -> Box<serde_json::value::RawValue> {
-    let headers = headers
-        .iter()
-        .map(|(k, v)| (k.to_string(), v.to_str().unwrap_or("").to_string()))
-        .collect::<HashMap<String, String>>();
-
-    to_raw_value(&serde_json::json!({
-        "kind": "http",
-        "http": {
-            "route": route_path,
-            "path": called_path,
-            "method": method.to_string().to_lowercase(),
-            "params": params,
-            "query": query,
-            "headers": headers
-        },
-    }))
+    Ok((trigger.clone(), route_path.to_string(), params, authed))
 }
 
 async fn route_job(
@@ -934,10 +1015,8 @@ async fn route_job(
     Extension(auth_cache): Extension<Arc<AuthCache>>,
     OptTokened { token }: OptTokened,
     Path(route_path): Path<StripPath>,
-    Query(query): Query<HashMap<String, String>>,
-    method: http::Method,
     headers: HeaderMap,
-    request: Request,
+    args: RawHttpTriggerArgs,
 ) -> Result<impl IntoResponse, Response> {
     let route_path = route_path.to_path().trim_end_matches("/");
     let (trigger, called_path, params, authed) = get_http_route_trigger(
@@ -946,25 +1025,21 @@ async fn route_job(
         token.as_ref(),
         &db,
         user_db.clone(),
-        &method,
+        &args.0.metadata.method,
     )
     .await
     .map_err(|e| e.into_response())?;
 
-    let args = try_from_request_body(
-        request,
-        &(),
-        Some(match trigger.authentication_method {
-            AuthenticationMethod::CustomScript | AuthenticationMethod::Signature => true,
-            _ => trigger.raw_string,
-        }),
-        Some(trigger.wrap_body),
-    )
-    .await
-    .map_err(|e| e.into_response())?;
-
-    let mut args = args
-        .to_push_args_owned(&authed, &db, &trigger.workspace_id)
+    let args = args
+        .process_args(
+            &authed,
+            &db,
+            &trigger.workspace_id,
+            match trigger.authentication_method {
+                AuthenticationMethod::CustomScript | AuthenticationMethod::Signature => true,
+                _ => trigger.raw_string,
+            },
+        )
         .await
         .map_err(|e| e.into_response())?;
 
@@ -983,34 +1058,45 @@ async fn route_job(
                 }
             };
 
-            let authentication_method =
-                try_get_resource_from_db_as::<http_trigger_auth::AuthenticationMethod>(
-                    authed.clone(),
-                    Some(user_db.clone()),
-                    &db,
-                    &resource_path,
-                    &trigger.workspace_id,
-                )
-                .await
-                .map_err(|e| e.into_response())?;
+            let cache_key = (
+                trigger.workspace_id.clone(),
+                resource_path.clone(),
+                authed.clone(),
+            );
 
-            let raw_payload = args
-                .extra
-                .as_ref()
-                .and_then(|extra| {
-                    extra
-                        .get("raw_string")
-                        .and_then(|value| Some(value.to_string()))
-                        .and_then(|raw_payload| Some(serde_json::from_str::<String>(&raw_payload)))
-                })
-                .transpose()
-                .map_err(|e| {
-                    windmill_common::error::Error::SerdeJson { location: e.to_string(), error: e }
-                        .into_response()
-                })?;
+            let authentication_method = match HTTP_AUTH_CACHE.get(&cache_key) {
+                Some(cache_entry) if cache_entry.expiry > std::time::Instant::now() => {
+                    tracing::debug!("HTTP auth method cache hit for trigger {}", trigger.path);
+                    cache_entry.value
+                }
+                _ => {
+                    tracing::debug!("HTTP auth method cache miss for trigger {}", trigger.path);
+                    let auth_method = try_get_resource_from_db_as::<
+                        crate::http_trigger_auth::AuthenticationMethod,
+                    >(
+                        authed.clone(),
+                        Some(user_db.clone()),
+                        &db,
+                        &resource_path,
+                        &trigger.workspace_id,
+                    )
+                    .await
+                    .map_err(|e| e.into_response())?;
+                    HTTP_AUTH_CACHE.insert(
+                        cache_key,
+                        ExpiringCacheEntry {
+                            value: auth_method.clone(),
+                            expiry: std::time::Instant::now() + std::time::Duration::from_secs(60),
+                        },
+                    );
+                    auth_method
+                }
+            };
+
+            let raw_payload = args.0.metadata.raw_string.as_ref();
 
             let response = authentication_method
-                .authenticate_http_request(&headers, raw_payload.as_ref())
+                .authenticate_http_request(&headers, raw_payload)
                 .map_err(|e| e.into_response())?;
 
             if let Some(response) = response {
@@ -1130,20 +1216,28 @@ async fn route_job(
         }
     }
 
-    let extra = args.extra.get_or_insert_with(HashMap::new);
+    let runnable_format = get_runnable_format(
+        if trigger.is_flow {
+            RunnableId::from_flow_path(&trigger.script_path)
+        } else {
+            RunnableId::from_script_path(&trigger.script_path)
+        },
+        &trigger.workspace_id,
+        &db,
+        &TriggerKind::Http,
+    )
+    .await
+    .map_err(|e| e.into_response())?;
 
-    extra.insert(
-        "wm_trigger".to_string(),
-        build_http_trigger_extra(
+    let args = args
+        .to_args_from_format(
             &trigger.route_path,
             &called_path,
-            &method,
             &params,
-            &query,
-            &headers,
+            runnable_format,
+            trigger.wrap_body,
         )
-        .await,
-    );
+        .map_err(|e| e.into_response())?;
 
     let run_query = RunJobQuery::default();
 
@@ -1157,7 +1251,6 @@ async fn route_job(
                 StripPath(trigger.script_path.to_owned()),
                 run_query,
                 args,
-                None,
             )
             .await
             .into_response()
@@ -1170,7 +1263,6 @@ async fn route_job(
                 user_db,
                 args,
                 trigger.workspace_id.clone(),
-                None,
             )
             .await
             .into_response()
@@ -1185,7 +1277,6 @@ async fn route_job(
                 StripPath(trigger.script_path.to_owned()),
                 run_query,
                 args,
-                None,
             )
             .await
             .into_response()
@@ -1198,7 +1289,6 @@ async fn route_job(
                 user_db,
                 trigger.workspace_id.clone(),
                 args,
-                None,
             )
             .await
             .into_response()

backend/windmill-api/src/jobs.rs

@@ -7,6 +7,7 @@
  */
 
 use axum::body::Body;
+use axum::extract::Request;
 use axum::http::HeaderValue;
 #[cfg(feature = "deno_core")]
 use deno_core::{op2, serde_v8, v8, JsRuntime, OpState};
@@ -37,10 +38,11 @@ use crate::add_webhook_allowed_origin;
 use crate::concurrency_groups::join_concurrency_key;
 use crate::db::ApiAuthed;
 
+use crate::trigger_helpers::RunnableId;
 use crate::users::get_scope_tags;
 use crate::utils::content_plain;
 use crate::{
-    args::{DecodeQueries, WebhookArgs},
+    args::{self, RawWebhookArgs},
     db::DB,
     users::{check_scopes, require_owner_of_path, OptAuthed},
     utils::require_super_admin,
@@ -55,7 +57,7 @@ use axum::{
 use base64::Engine;
 use chrono::Utc;
 use hmac::Mac;
-use hyper::{Request, StatusCode};
+use hyper::StatusCode;
 use serde::{de::DeserializeOwned, Deserialize, Serialize};
 use sql_builder::prelude::*;
 use sqlx::types::JsonRawValue;
@@ -86,7 +88,10 @@ use windmill_common::s3_helpers::OBJECT_STORE_CACHE_SETTINGS;
 #[cfg(feature = "prometheus")]
 use windmill_common::{METRICS_DEBUG_ENABLED, METRICS_ENABLED};
 
-use windmill_common::{get_latest_deployed_hash_for_path, BASE_URL};
+use windmill_common::{
+    get_latest_deployed_hash_for_path, get_latest_flow_version_info_for_path,
+    get_script_info_for_hash, FlowVersionInfo, ScriptHashInfo, BASE_URL,
+};
 use windmill_queue::{
     cancel_job, get_result_and_success_by_id_from_flow, job_is_complete, push, PushArgs,
     PushArgsOwned, PushIsolationLevel,
@@ -545,58 +550,6 @@ async fn force_cancel(
     }
 }
 
-pub async fn get_path_tag_limits_cache_for_hash(
-    mut tx: Transaction<'_, Postgres>,
-    w_id: &str,
-    hash: i64,
-) -> error::Result<(
-    String,
-    Option<String>,
-    Option<String>,
-    Option<i32>,
-    Option<i32>,
-    Option<i32>,
-    ScriptLang,
-    Option<bool>,
-    Option<i16>,
-    Option<bool>,
-    Option<i32>,
-    Option<bool>,
-    Option<String>,
-    String,
-)> {
-    let script = sqlx::query!(
-        "select path, tag, concurrency_key, concurrent_limit, concurrency_time_window_s, cache_ttl, language as \"language: ScriptLang\", dedicated_worker, priority, delete_after_use, timeout, has_preprocessor, on_behalf_of_email, created_by from script where hash = $1 AND workspace_id = $2",
-        hash,
-        w_id
-    )
-    .fetch_optional(&mut *tx)
-    .await
-    .map_err(|e| {
-        Error::internal_err(format!(
-            "querying getting path for hash {hash} in {w_id}: {e:#}"
-        ))
-    })?.ok_or_else(|| Error::NotFound(format!(
-        "deployed script not found at hash {hash} in workspace {w_id}"
-    )))?;
-    Ok((
-        script.path,
-        script.tag,
-        script.concurrency_key,
-        script.concurrent_limit,
-        script.concurrency_time_window_s,
-        script.cache_ttl,
-        script.language,
-        script.dedicated_worker,
-        script.priority,
-        script.delete_after_use,
-        script.timeout,
-        script.has_preprocessor,
-        script.on_behalf_of_email,
-        script.created_by,
-    ))
-}
-
 async fn get_flow_job_debug_info(
     OptAuthed(opt_authed): OptAuthed,
     Extension(db): Extension<DB>,
@@ -3466,7 +3419,6 @@ async fn batch_rerun_handle_job(
                 StripPath(job.script_path.clone()),
                 RunJobQuery { ..Default::default() },
                 PushArgsOwned { extra: None, args },
-                None,
             )
             .await;
             if let Ok((_, uuid)) = result {
@@ -3483,7 +3435,6 @@ async fn batch_rerun_handle_job(
                     StripPath(job.script_path.clone()),
                     RunJobQuery { ..Default::default() },
                     PushArgsOwned { extra: None, args },
-                    None,
                 )
                 .await
             } else {
@@ -3495,7 +3446,6 @@ async fn batch_rerun_handle_job(
                     job.script_hash,
                     RunJobQuery { ..Default::default() },
                     PushArgsOwned { extra: None, args },
-                    None,
                 )
                 .await
             };
@@ -3516,11 +3466,19 @@ pub async fn run_flow_by_path(
     Extension(user_db): Extension<UserDB>,
     Path((w_id, flow_path)): Path<(String, StripPath)>,
     Query(run_query): Query<RunJobQuery>,
-    args: WebhookArgs,
+    args: RawWebhookArgs,
 ) -> error::Result<(StatusCode, String)> {
-    let args = args.to_push_args_owned(&authed, &db, &w_id).await?;
+    let args = args
+        .to_args_from_runnable(
+            &authed,
+            &db,
+            &w_id,
+            RunnableId::from_flow_path(flow_path.to_path()),
+            run_query.skip_preprocessor,
+        )
+        .await?;
 
-    run_flow_by_path_inner(authed, db, user_db, w_id, flow_path, run_query, args, None).await
+    run_flow_by_path_inner(authed, db, user_db, w_id, flow_path, run_query, args).await
 }
 
 pub async fn run_flow_by_path_inner(
@@ -3531,7 +3489,6 @@ pub async fn run_flow_by_path_inner(
     flow_path: StripPath,
     run_query: RunJobQuery,
     args: PushArgsOwned,
-    label_prefix: Option<String>,
 ) -> error::Result<(StatusCode, String)> {
     #[cfg(feature = "enterprise")]
     check_license_key_valid().await?;
@@ -3539,23 +3496,17 @@ pub async fn run_flow_by_path_inner(
     check_scopes(&authed, || format!("run:flow/{flow_path}"))?;
 
     let mut tx = user_db.clone().begin(&authed).await?;
-    let (tag, dedicated_worker, has_preprocessor, on_behalf_of_email, edited_by) = sqlx::query!(
-        "SELECT tag, dedicated_worker, flow_version.value->>'preprocessor_module' IS NOT NULL as has_preprocessor, on_behalf_of_email, edited_by
-        FROM flow 
-        LEFT JOIN flow_version
-            ON flow_version.id = flow.versions[array_upper(flow.versions, 1)]
-        WHERE flow.path = $1 and flow.workspace_id = $2",
-        flow_path,
-        w_id
-    )
-    .fetch_optional(&mut *tx)
-    .await?
-    .map(|x| (x.tag, x.dedicated_worker, x.has_preprocessor, x.on_behalf_of_email, x.edited_by))
-    .ok_or_else(|| {
-        Error::NotFound(format!(
-            "flow not found at path {flow_path} in workspace {w_id}"
-        ))
-    })?;
+
+    let FlowVersionInfo {
+        version,
+        tag,
+        dedicated_worker,
+        has_preprocessor,
+        on_behalf_of_email,
+        edited_by,
+        ..
+    } = get_latest_flow_version_info_for_path(&mut *tx, &w_id, &flow_path, true).await?;
+
     drop(tx);
 
     let tag = run_query.tag.clone().or(tag);
@@ -3587,13 +3538,12 @@ pub async fn run_flow_by_path_inner(
         JobPayload::Flow {
             path: flow_path.to_string(),
             dedicated_worker,
+            version,
             apply_preprocessor: !run_query.skip_preprocessor.unwrap_or(false)
                 && has_preprocessor.unwrap_or(false),
         },
         PushArgs { args: &args.args, extra: args.extra },
-        &label_prefix
-            .map(|x| x + authed.display_username())
-            .unwrap_or_else(|| authed.display_username().to_string()),
+        authed.display_username(),
         email,
         permissioned_as,
         scheduled_for,
@@ -3715,20 +3665,19 @@ pub async fn run_script_by_path(
     Extension(user_db): Extension<UserDB>,
     Path((w_id, script_path)): Path<(String, StripPath)>,
     Query(run_query): Query<RunJobQuery>,
-    args: WebhookArgs,
+    args: RawWebhookArgs,
 ) -> error::Result<(StatusCode, String)> {
-    let args = args.to_push_args_owned(&authed, &db, &w_id).await?;
-    run_script_by_path_inner(
-        authed,
-        db,
-        user_db,
-        w_id,
-        script_path,
-        run_query,
-        args,
-        None,
-    )
-    .await
+    let args = args
+        .to_args_from_runnable(
+            &authed,
+            &db,
+            &w_id,
+            RunnableId::from_script_path(script_path.to_path()),
+            run_query.skip_preprocessor,
+        )
+        .await?;
+
+    run_script_by_path_inner(authed, db, user_db, w_id, script_path, run_query, args).await
 }
 
 pub async fn run_script_by_path_inner(
@@ -3739,7 +3688,6 @@ pub async fn run_script_by_path_inner(
     script_path: StripPath,
     run_query: RunJobQuery,
     args: PushArgsOwned,
-    label_prefix: Option<String>,
 ) -> error::Result<(StatusCode, String)> {
     #[cfg(feature = "enterprise")]
     check_license_key_valid().await?;
@@ -3780,9 +3728,7 @@ pub async fn run_script_by_path_inner(
         &w_id,
         job_payload,
         PushArgs { args: &args.args, extra: args.extra },
-        &label_prefix
-            .map(|x| x + authed.display_username())
-            .unwrap_or_else(|| authed.display_username().to_string()),
+        authed.display_username(),
         email,
         permissioned_as,
         scheduled_for,
@@ -4384,7 +4330,7 @@ pub async fn run_wait_result_job_by_path_get(
     Extension(db): Extension<DB>,
     Path((w_id, script_path)): Path<(String, StripPath)>,
     Query(run_query): Query<RunJobQuery>,
-    DecodeQueries(queries): DecodeQueries,
+    args: RawWebhookArgs,
 ) -> error::Result<Response> {
     #[cfg(feature = "enterprise")]
     check_license_key_valid().await?;
@@ -4396,20 +4342,27 @@ pub async fn run_wait_result_job_by_path_get(
         x.map_err(|e| Error::internal_err(format!("Impossible to decode query payload: {e:#?}")))
     });
 
-    let mut payload_args = if let Some(payload) = payload_r {
+    let payload_args = if let Some(payload) = payload_r {
         payload?
     } else {
         HashMap::new()
     };
-    queries.iter().for_each(|(k, v)| {
-        payload_args.insert(k.to_string(), v.clone());
-    });
 
-    let inner_args: HashMap<String, Box<RawValue>> = HashMap::new();
-    let args = PushArgs { extra: Some(payload_args), args: &inner_args };
+    let mut args = args.process_args(&authed, &db, &w_id, None).await?;
+    args.body = args::Body::HashMap(payload_args);
+
+    let script_path = script_path.to_path();
+
+    let args = args
+        .to_args_from_runnable(
+            &db,
+            &w_id,
+            RunnableId::from_script_path(script_path),
+            run_query.skip_preprocessor,
+        )
+        .await?;
 
     check_queue_too_long(&db, QUEUE_LIMIT_WAIT_RESULT.or(run_query.queue_limit)).await?;
-    let script_path = script_path.to_path();
     check_scopes(&authed, || format!("run:script/{script_path}"))?;
 
     let mut tx = user_db.clone().begin(&authed).await?;
@@ -4478,7 +4431,7 @@ pub async fn run_wait_result_flow_by_path_get(
     Extension(db): Extension<DB>,
     Path((w_id, flow_path)): Path<(String, StripPath)>,
     Query(run_query): Query<RunJobQuery>,
-    DecodeQueries(queries): DecodeQueries,
+    args: RawWebhookArgs,
 ) -> error::Result<Response> {
     #[cfg(feature = "enterprise")]
     check_license_key_valid().await?;
@@ -4492,22 +4445,26 @@ pub async fn run_wait_result_flow_by_path_get(
         })
     });
 
-    let mut payload_args = if let Some(payload) = payload_r {
+    let payload_args = if let Some(payload) = payload_r {
         payload?
     } else {
         HashMap::new()
     };
 
-    queries.iter().for_each(|(k, v)| {
-        payload_args.insert(k.to_string(), v.clone());
-    });
+    let mut args = args.process_args(&authed, &db, &w_id, None).await?;
+    args.body = args::Body::HashMap(payload_args);
 
-    let args = PushArgsOwned { extra: Some(payload_args), args: HashMap::new() };
+    let args = args
+        .to_args_from_runnable(
+            &db,
+            &w_id,
+            RunnableId::from_flow_path(flow_path.to_path()),
+            run_query.skip_preprocessor,
+        )
+        .await?;
 
-    run_wait_result_flow_by_path_internal(
-        db, run_query, flow_path, authed, user_db, args, w_id, None,
-    )
-    .await
+    run_wait_result_flow_by_path_internal(db, run_query, flow_path, authed, user_db, args, w_id)
+        .await
 }
 
 pub async fn run_wait_result_script_by_path(
@@ -4516,24 +4473,23 @@ pub async fn run_wait_result_script_by_path(
     Extension(db): Extension<DB>,
     Path((w_id, script_path)): Path<(String, StripPath)>,
     Query(run_query): Query<RunJobQuery>,
-    args: WebhookArgs,
+    args: RawWebhookArgs,
 ) -> error::Result<Response> {
     #[cfg(feature = "enterprise")]
     check_license_key_valid().await?;
 
-    let args = args.to_push_args_owned(&authed, &db, &w_id).await?;
+    let args = args
+        .to_args_from_runnable(
+            &authed,
+            &db,
+            &w_id,
+            RunnableId::from_script_path(script_path.to_path()),
+            run_query.skip_preprocessor,
+        )
+        .await?;
 
-    run_wait_result_script_by_path_internal(
-        db,
-        run_query,
-        script_path,
-        authed,
-        user_db,
-        w_id,
-        args,
-        None,
-    )
-    .await
+    run_wait_result_script_by_path_internal(db, run_query, script_path, authed, user_db, w_id, args)
+        .await
 }
 
 pub async fn run_wait_result_script_by_path_internal(
@@ -4544,7 +4500,6 @@ pub async fn run_wait_result_script_by_path_internal(
     user_db: UserDB,
     w_id: String,
     args: PushArgsOwned,
-    label_prefix: Option<String>,
 ) -> error::Result<Response> {
     check_queue_too_long(&db, QUEUE_LIMIT_WAIT_RESULT.or(run_query.queue_limit)).await?;
     let script_path = script_path.to_path();
@@ -4553,6 +4508,7 @@ pub async fn run_wait_result_script_by_path_internal(
     let mut tx = user_db.clone().begin(&authed).await?;
     let (job_payload, tag, delete_after_use, timeout, on_behalf_of) =
         script_path_to_payload(script_path, &mut *tx, &w_id, run_query.skip_preprocessor).await?;
+    drop(tx);
 
     let tag = run_query.tag.clone().or(tag);
     check_tag_available_for_workspace(&w_id, &tag, &authed).await?;
@@ -4580,9 +4536,7 @@ pub async fn run_wait_result_script_by_path_internal(
         &w_id,
         job_payload,
         PushArgs { args: &args.args, extra: args.extra },
-        &label_prefix
-            .map(|x| x + authed.display_username())
-            .unwrap_or_else(|| authed.display_username().to_string()),
+        authed.display_username(),
         email,
         permissioned_as,
         None,
@@ -4616,19 +4570,29 @@ pub async fn run_wait_result_script_by_hash(
     Extension(db): Extension<DB>,
     Path((w_id, script_hash)): Path<(String, ScriptHash)>,
     Query(run_query): Query<RunJobQuery>,
-    args: WebhookArgs,
+    args: RawWebhookArgs,
 ) -> error::Result<Response> {
     #[cfg(feature = "enterprise")]
     check_license_key_valid().await?;
 
-    let args = args.to_push_args_owned(&authed, &db, &w_id).await?;
+    let args = args
+        .to_args_from_runnable(
+            &authed,
+            &db,
+            &w_id,
+            RunnableId::from_script_hash(script_hash),
+            run_query.skip_preprocessor,
+        )
+        .await?;
+
     check_queue_too_long(&db, run_query.queue_limit).await?;
 
     let hash = script_hash.0;
-    let (
+    let mut tx = user_db.clone().begin(&authed).await?;
+    let ScriptHashInfo {
         path,
         tag,
-        custom_concurrency_key,
+        concurrency_key,
         concurrent_limit,
         concurrency_time_window_s,
         mut cache_ttl,
@@ -4640,8 +4604,8 @@ pub async fn run_wait_result_script_by_hash(
         has_preprocessor,
         on_behalf_of_email,
         created_by,
-    ) = get_path_tag_limits_cache_for_hash(user_db.clone().begin(&authed).await?, &w_id, hash)
-        .await?;
+        ..
+    } = get_script_info_for_hash(&mut *tx, &w_id, hash).await?;
     if let Some(run_query_cache_ttl) = run_query.cache_ttl {
         cache_ttl = Some(run_query_cache_ttl);
     }
@@ -4674,7 +4638,7 @@ pub async fn run_wait_result_script_by_hash(
         JobPayload::ScriptHash {
             hash: ScriptHash(hash),
             path: path,
-            custom_concurrency_key,
+            custom_concurrency_key: concurrency_key,
             concurrent_limit: concurrent_limit,
             concurrency_time_window_s: concurrency_time_window_s,
             cache_ttl,
@@ -4719,17 +4683,23 @@ pub async fn run_wait_result_flow_by_path(
     Extension(db): Extension<DB>,
     Path((w_id, flow_path)): Path<(String, StripPath)>,
     Query(run_query): Query<RunJobQuery>,
-    args: WebhookArgs,
+    args: RawWebhookArgs,
 ) -> error::Result<Response> {
     #[cfg(feature = "enterprise")]
     check_license_key_valid().await?;
 
-    let args = args.to_push_args_owned(&authed, &db, &w_id).await?;
+    let args = args
+        .to_args_from_runnable(
+            &authed,
+            &db,
+            &w_id,
+            RunnableId::from_flow_path(flow_path.to_path()),
+            run_query.skip_preprocessor,
+        )
+        .await?;
 
-    run_wait_result_flow_by_path_internal(
-        db, run_query, flow_path, authed, user_db, args, w_id, None,
-    )
-    .await
+    run_wait_result_flow_by_path_internal(db, run_query, flow_path, authed, user_db, args, w_id)
+        .await
 }
 
 pub async fn run_wait_result_flow_by_path_internal(
@@ -4740,7 +4710,6 @@ pub async fn run_wait_result_flow_by_path_internal(
     user_db: UserDB,
     args: PushArgsOwned,
     w_id: String,
-    label_prefix: Option<String>,
 ) -> error::Result<Response> {
     check_queue_too_long(&db, run_query.queue_limit).await?;
 
@@ -4750,23 +4719,17 @@ pub async fn run_wait_result_flow_by_path_internal(
     let scheduled_for = run_query.get_scheduled_for(&db).await?;
 
     let mut tx = user_db.clone().begin(&authed).await?;
-    let (tag, dedicated_worker, early_return, has_preprocessor, on_behalf_of_email, edited_by) = sqlx::query!(
-        "SELECT tag, dedicated_worker, flow_version.value->>'early_return' as early_return, flow_version.value->>'preprocessor_module' IS NOT NULL as has_preprocessor, on_behalf_of_email, edited_by
-        FROM flow 
-        LEFT JOIN flow_version
-            ON flow_version.id = flow.versions[array_upper(flow.versions, 1)]
-        WHERE flow.path = $1 and flow.workspace_id = $2",
-        flow_path,
-        w_id
-    )
-    .fetch_optional(&mut *tx)
-    .await?
-    .map(|x| (x.tag, x.dedicated_worker, x.early_return, x.has_preprocessor, x.on_behalf_of_email, x.edited_by))
-    .ok_or_else(|| {
-        Error::NotFound(format!(
-            "flow not found at path {flow_path} in workspace {w_id}"
-        ))
-    })?;
+
+    let FlowVersionInfo {
+        tag,
+        dedicated_worker,
+        early_return,
+        has_preprocessor,
+        on_behalf_of_email,
+        edited_by,
+        version,
+    } = get_latest_flow_version_info_for_path(&mut *tx, &w_id, &flow_path, true).await?;
+    drop(tx);
 
     let tag = run_query.tag.clone().or(tag);
     check_tag_available_for_workspace(&w_id, &tag, &authed).await?;
@@ -4795,13 +4758,12 @@ pub async fn run_wait_result_flow_by_path_internal(
         JobPayload::Flow {
             path: flow_path.to_string(),
             dedicated_worker,
+            version,
             apply_preprocessor: !run_query.skip_preprocessor.unwrap_or(false)
                 && has_preprocessor.unwrap_or(false),
         },
         PushArgs { args: &args.args, extra: args.extra },
-        &label_prefix
-            .map(|x| x + authed.display_username())
-            .unwrap_or_else(|| authed.display_username().to_string()),
+        authed.display_username(),
         email,
         permissioned_as,
         scheduled_for,
@@ -5255,29 +5217,23 @@ async fn add_batch_jobs(
         "script" => {
             if let Some(path) = batch_info.path {
                 let mut tx = user_db.clone().begin(&authed).await?;
-                let (
-                    script_hash,
-                    _tag,
-                    custom_concurrency_key,
+                let ScriptHashInfo {
+                    hash: script_hash,
+                    concurrency_key,
                     concurrent_limit,
                     concurrency_time_window_s,
-                    _cache_ttl,
                     language,
                     dedicated_worker,
-                    _priority,
-                    _delete_after_use,
                     timeout,
-                    _,
-                    _, // TODO: consider on_behalf_of_email and created_by for batch jobs
-                    _, // ------------------------------------------
-                ) = get_latest_deployed_hash_for_path(&mut *tx, &w_id, &path).await?;
+                    .. // TODO: consider on_behalf_of_email and created_by for batch jobs
+                 } = get_latest_deployed_hash_for_path(&mut *tx, &w_id, &path).await?;
                 (
                     Some(script_hash),
                     Some(path),
                     JobKind::Script,
                     Some(language),
                     dedicated_worker,
-                    custom_concurrency_key,
+                    concurrency_key,
                     concurrent_limit,
                     concurrency_time_window_s,
                     timeout,
@@ -5417,7 +5373,7 @@ async fn add_batch_jobs(
         raw_lock,
         raw_flow.map(sqlx::types::Json) as Option<sqlx::types::Json<FlowValue>>,
         tag,
-        hash.map(|h| h.0),
+        hash,
         path,
         job_kind.clone() as JobKind,
         language as ScriptLang,
@@ -5562,20 +5518,19 @@ pub async fn run_job_by_hash(
     Extension(user_db): Extension<UserDB>,
     Path((w_id, script_hash)): Path<(String, ScriptHash)>,
     Query(run_query): Query<RunJobQuery>,
-    args: WebhookArgs,
+    args: RawWebhookArgs,
 ) -> error::Result<(StatusCode, String)> {
-    let args = args.to_push_args_owned(&authed, &db, &w_id).await?;
-    run_job_by_hash_inner(
-        authed,
-        db,
-        user_db,
-        w_id,
-        script_hash,
-        run_query,
-        args,
-        None,
-    )
-    .await
+    let args = args
+        .to_args_from_runnable(
+            &authed,
+            &db,
+            &w_id,
+            RunnableId::from_script_hash(script_hash),
+            run_query.skip_preprocessor,
+        )
+        .await?;
+
+    run_job_by_hash_inner(authed, db, user_db, w_id, script_hash, run_query, args).await
 }
 
 pub async fn run_job_by_hash_inner(
@@ -5586,29 +5541,28 @@ pub async fn run_job_by_hash_inner(
     script_hash: ScriptHash,
     run_query: RunJobQuery,
     args: PushArgsOwned,
-    label_prefix: Option<String>,
 ) -> error::Result<(StatusCode, String)> {
     #[cfg(feature = "enterprise")]
     check_license_key_valid().await?;
 
     let hash = script_hash.0;
-    let (
+    let mut tx = user_db.clone().begin(&authed).await?;
+    let ScriptHashInfo {
         path,
         tag,
-        custom_concurrency_key,
+        concurrency_key,
         concurrent_limit,
         concurrency_time_window_s,
         mut cache_ttl,
         language,
         dedicated_worker,
         priority,
-        _delete_after_use, // not taken into account in async endpoints
         timeout,
         has_preprocessor,
         on_behalf_of_email,
         created_by,
-    ) = get_path_tag_limits_cache_for_hash(user_db.clone().begin(&authed).await?, &w_id, hash)
-        .await?;
+        .. // delete_after_use not taken into account in async endpoints
+    } = get_script_info_for_hash(&mut *tx, &w_id, hash).await?;
     check_scopes(&authed, || format!("run:script/{path}"))?;
     if let Some(run_query_cache_ttl) = run_query.cache_ttl {
         cache_ttl = Some(run_query_cache_ttl);
@@ -5642,7 +5596,7 @@ pub async fn run_job_by_hash_inner(
         JobPayload::ScriptHash {
             hash: ScriptHash(hash),
             path: path,
-            custom_concurrency_key,
+            custom_concurrency_key: concurrency_key,
             concurrent_limit: concurrent_limit,
             concurrency_time_window_s: concurrency_time_window_s,
             cache_ttl,
@@ -5653,9 +5607,7 @@ pub async fn run_job_by_hash_inner(
                 && has_preprocessor.unwrap_or(false),
         },
         PushArgs { args: &args.args, extra: args.extra },
-        &label_prefix
-            .map(|x| x + authed.display_username())
-            .unwrap_or_else(|| authed.display_username().to_string()),
+        authed.display_username(),
         email,
         permissioned_as,
         scheduled_for,

backend/windmill-api/src/lib.rs

@@ -81,15 +81,18 @@ mod folders;
 mod granular_acls;
 mod groups;
 #[cfg(feature = "http_trigger")]
+mod http_trigger_args;
+#[cfg(feature = "http_trigger")]
 mod http_trigger_auth;
 #[cfg(feature = "http_trigger")]
-mod http_triggers;
+pub mod http_triggers;
 mod indexer_ee;
 mod inputs;
 mod integration;
 #[cfg(feature = "postgres_trigger")]
 mod postgres_triggers;
 
+mod approvals;
 #[cfg(feature = "enterprise")]
 mod apps_ee;
 #[cfg(all(feature = "enterprise", feature = "gcp_trigger"))]
@@ -122,6 +125,8 @@ mod slack_approvals;
 mod smtp_server_ee;
 #[cfg(all(feature = "enterprise", feature = "sqs_trigger"))]
 mod sqs_triggers_ee;
+mod teams_approvals_ee;
+mod trigger_helpers;
 
 mod static_assets;
 #[cfg(all(feature = "stripe", feature = "enterprise"))]
@@ -401,6 +406,12 @@ pub async fn run_server(
         Router::new()
     };
 
+    #[cfg(feature = "http_trigger")]
+    {
+        let http_killpill_rx = killpill_rx.resubscribe();
+        http_triggers::refresh_routers_loop(&db, http_killpill_rx).await;
+    }
+
     let postgres_triggers_service = {
         #[cfg(feature = "postgres_trigger")]
         {
@@ -647,6 +658,10 @@ pub async fn run_server(
                     "/w/:workspace_id/jobs/slack_approval/:job_id",
                     get(slack_approvals::request_slack_approval),
                 )
+                .route(
+                    "/w/:workspace_id/jobs/teams_approval/:job_id",
+                    get(teams_approvals_ee::request_teams_approval),
+                )
                 .nest("/w/:workspace_id/github_app", {
                     #[cfg(feature = "enterprise")]
                     {

backend/windmill-api/src/mcp.rs

@@ -924,7 +924,6 @@ impl ServerHandler for Runner {
                 user_db.clone(),
                 w_id.clone(),
                 push_args,
-                None,
             )
             .await
         } else {
@@ -936,7 +935,6 @@ impl ServerHandler for Runner {
                 user_db.clone(),
                 push_args,
                 w_id.clone(),
-                None,
             )
             .await
         };

backend/windmill-api/src/mqtt_triggers.rs

@@ -3,9 +3,10 @@ use crate::{
     db::{ApiAuthed, DB},
     jobs::{run_flow_by_path_inner, run_script_by_path_inner, RunJobQuery},
     resources::try_get_resource_from_db_as,
+    trigger_helpers::TriggerJobArgs,
     users::fetch_api_authed,
 };
-use windmill_queue::TriggerKind;
+use windmill_git_sync::{handle_deployment_metadata, DeployedObject};
 
 use axum::{
     async_trait,
@@ -16,7 +17,7 @@ use axum::{
     routing::{delete, get, post},
     Router,
 };
-use base64::prelude::*;
+use base64::{engine, prelude::*};
 use bytes::Bytes;
 use http::StatusCode;
 use itertools::Itertools;
@@ -42,6 +43,7 @@ use windmill_audit::{audit_ee::audit_log, ActionKind};
 use windmill_common::{
     db::UserDB,
     error::{self, JsonResult},
+    triggers::TriggerKind,
     utils::{not_found_if_none, paginate, report_critical_error, Pagination, StripPath},
     worker::{to_raw_value, CLOUD_HOSTED},
     INSTANCE_NAME,
@@ -51,8 +53,6 @@ use rand::seq::SliceRandom;
 use serde_json::value::RawValue;
 use sqlx::types::Json as SqlxJson;
 
-use windmill_queue::PushArgsOwned;
-
 pub fn workspaced_service() -> Router {
     Router::new()
         .route("/create", post(create_mqtt_trigger))
@@ -82,12 +82,20 @@ enum Error {
 }
 
 async fn run_job(
-    args: Option<HashMap<String, Box<RawValue>>>,
-    extra: Option<HashMap<String, Box<RawValue>>>,
+    payload: &[u8],
+    trigger_info: HashMap<String, Box<RawValue>>,
     db: &DB,
     trigger: &MqttTrigger,
 ) -> anyhow::Result<()> {
-    let args = PushArgsOwned { args: args.unwrap_or_default(), extra };
+    let args = MqttTrigger::build_job_args(
+        &trigger.script_path,
+        trigger.is_flow,
+        &trigger.workspace_id,
+        db,
+        payload,
+        trigger_info,
+    )
+    .await?;
 
     let authed = fetch_api_authed(
         trigger.edited_by.clone(),
@@ -111,7 +119,6 @@ async fn run_job(
             StripPath(trigger.script_path.to_owned()),
             run_query,
             args,
-            None,
         )
         .await?;
     } else {
@@ -123,7 +130,6 @@ async fn run_job(
             StripPath(trigger.script_path.to_owned()),
             run_query,
             args,
-            None,
         )
         .await?;
     }
@@ -239,25 +245,25 @@ pub struct EditMqttTrigger {
 
 #[derive(Debug, Serialize, Deserialize, FromRow)]
 pub struct MqttTrigger {
-    mqtt_resource_path: String,
-    subscribe_topics: Vec<SqlxJson<SubscribeTopic>>,
-    v3_config: Option<SqlxJson<MqttV3Config>>,
-    v5_config: Option<SqlxJson<MqttV5Config>>,
-    client_id: Option<String>,
+    pub mqtt_resource_path: String,
+    pub subscribe_topics: Vec<SqlxJson<SubscribeTopic>>,
+    pub v3_config: Option<SqlxJson<MqttV3Config>>,
+    pub v5_config: Option<SqlxJson<MqttV5Config>>,
+    pub client_id: Option<String>,
     #[serde(skip_serializing_if = "Option::is_none")]
-    client_version: Option<MqttClientVersion>,
-    path: String,
-    script_path: String,
-    is_flow: bool,
-    workspace_id: String,
-    edited_by: String,
-    email: String,
-    edited_at: chrono::DateTime<chrono::Utc>,
-    extra_perms: Option<serde_json::Value>,
-    error: Option<String>,
-    server_id: Option<String>,
-    last_server_ping: Option<chrono::DateTime<chrono::Utc>>,
-    enabled: bool,
+    pub client_version: Option<MqttClientVersion>,
+    pub path: String,
+    pub script_path: String,
+    pub is_flow: bool,
+    pub workspace_id: String,
+    pub edited_by: String,
+    pub email: String,
+    pub edited_at: chrono::DateTime<chrono::Utc>,
+    pub extra_perms: Option<serde_json::Value>,
+    pub error: Option<String>,
+    pub server_id: Option<String>,
+    pub last_server_ping: Option<chrono::DateTime<chrono::Utc>>,
+    pub enabled: bool,
 }
 
 #[derive(Deserialize, Serialize)]
@@ -510,13 +516,14 @@ pub async fn test_mqtt_connection(
 
 pub async fn create_mqtt_trigger(
     authed: ApiAuthed,
+    Extension(db): Extension<DB>,
     Extension(user_db): Extension<UserDB>,
     Path(w_id): Path<String>,
     Json(new_mqtt_trigger): Json<NewMqttTrigger>,
 ) -> error::Result<(StatusCode, String)> {
     if *CLOUD_HOSTED {
         return Err(error::Error::BadRequest(
-            "Mqtt triggers are not supported on multi-tenant cloud, use dedicated cloud or self-host".to_string(),
+            "MQTT triggers are not supported on multi-tenant cloud, use dedicated cloud or self-host".to_string(),
         ));
     }
 
@@ -601,7 +608,18 @@ pub async fn create_mqtt_trigger(
 
     tx.commit().await?;
 
-    Ok((StatusCode::CREATED, path.to_string()))
+    handle_deployment_metadata(
+        &authed.email,
+        &authed.username,
+        &db,
+        &w_id,
+        DeployedObject::MqttTrigger { path: path.to_string() },
+        Some(format!("MQTT trigger '{}' created", path)),
+        true,
+    )
+    .await?;
+
+    Ok((StatusCode::CREATED, format!("{}", path.to_string())))
 }
 
 pub async fn list_mqtt_triggers(
@@ -714,6 +732,7 @@ pub async fn get_mqtt_trigger(
 
 pub async fn update_mqtt_trigger(
     authed: ApiAuthed,
+    Extension(db): Extension<DB>,
     Extension(user_db): Extension<UserDB>,
     Path((w_id, path)): Path<(String, StripPath)>,
     Json(mqtt_trigger): Json<EditMqttTrigger>,
@@ -782,7 +801,7 @@ pub async fn update_mqtt_trigger(
         &mut *tx,
         &authed,
         "mqtt_triggers.update",
-        ActionKind::Create,
+        ActionKind::Update,
         &w_id,
         Some(&path),
         None,
@@ -791,11 +810,23 @@ pub async fn update_mqtt_trigger(
 
     tx.commit().await?;
 
-    Ok(workspace_path.to_string())
+    handle_deployment_metadata(
+        &authed.email,
+        &authed.username,
+        &db,
+        &w_id,
+        DeployedObject::MqttTrigger { path: path.clone() },
+        Some(format!("MQTT trigger '{}' updated", path)),
+        true,
+    )
+    .await?;
+
+    Ok(path.to_string())
 }
 
 pub async fn delete_mqtt_trigger(
     authed: ApiAuthed,
+    Extension(db): Extension<DB>,
     Extension(user_db): Extension<UserDB>,
     Path((w_id, path)): Path<(String, StripPath)>,
 ) -> error::Result<String> {
@@ -829,7 +860,18 @@ pub async fn delete_mqtt_trigger(
 
     tx.commit().await?;
 
-    Ok(format!("Mqtt trigger {path} deleted"))
+    handle_deployment_metadata(
+        &authed.email,
+        &authed.username,
+        &db,
+        &w_id,
+        DeployedObject::MqttTrigger { path: path.to_string() },
+        Some(format!("MQTT trigger '{}' deleted", path)),
+        true,
+    )
+    .await?;
+
+    Ok(format!("MQTT trigger {path} deleted"))
 }
 
 pub async fn exists_mqtt_trigger(
@@ -859,6 +901,7 @@ pub async fn exists_mqtt_trigger(
 
 pub async fn set_enabled(
     authed: ApiAuthed,
+    Extension(db): Extension<DB>,
     Extension(user_db): Extension<UserDB>,
     Path((w_id, path)): Path<(String, StripPath)>,
     Json(payload): Json<SetEnabled>,
@@ -908,6 +951,17 @@ pub async fn set_enabled(
 
     tx.commit().await?;
 
+    handle_deployment_metadata(
+        &authed.email,
+        &authed.username,
+        &db,
+        &w_id,
+        DeployedObject::MqttTrigger { path: path.to_string() },
+        Some(format!("MQTT trigger '{}' updated", path)),
+        true,
+    )
+    .await?;
+
     Ok(format!(
         "successfully updated mqtt trigger at path {} to status {}",
         path, payload.enabled
@@ -1090,31 +1144,27 @@ impl EventLoop for V3EventLoop {
 }
 
 async fn handle_publish_packet(db: &DB, mqtt: &MqttConfig, payload: Bytes, publish: PublishData) {
-    let args = HashMap::from([("payload".to_string(), to_raw_value(&payload.as_ref()))]);
-    let extra = Some(HashMap::from([(
-        "wm_trigger".to_string(),
-        to_raw_value(&serde_json::json!({
-            "kind": "mqtt",
-            "mqtt": {
-                "topic": publish.topic,
-                "retain": publish.retain,
-                "pkid": publish.pkid,
-                "qos": publish.qos,
-                "v5": publish.v5.map(|properties| {
-                    serde_json::json!({
-                        "payload_format_indicator": properties.payload_format_indicator,
-                        "topic_alias": properties.topic_alias,
-                        "response_topic": properties.response_topic,
-                        "correlation_data": properties.correlation_data.as_deref(),
-                        "user_properties": properties.user_properties,
-                        "subscription_identifiers": properties.subscription_identifiers,
-                        "content_type": properties.content_type,
-                    })
+    let trigger_info = HashMap::from([
+        ("topic".to_string(), to_raw_value(&publish.topic)),
+        ("retain".to_string(), to_raw_value(&publish.retain)),
+        ("pkid".to_string(), to_raw_value(&publish.pkid)),
+        ("qos".to_string(), to_raw_value(&publish.qos)),
+        (
+            "v5".to_string(),
+            to_raw_value(&publish.v5.map(|properties| {
+                serde_json::json!({
+                    "payload_format_indicator": properties.payload_format_indicator,
+                    "topic_alias": properties.topic_alias,
+                    "response_topic": properties.response_topic,
+                    "correlation_data": properties.correlation_data.as_deref(),
+                    "user_properties": properties.user_properties,
+                    "subscription_identifiers": properties.subscription_identifiers,
+                    "content_type": properties.content_type,
                 })
-            }
-        })),
-    )]));
-    mqtt.handle(&db, Some(args), extra).await;
+            })),
+        ),
+    ]);
+    mqtt.handle(&db, payload.as_ref(), trigger_info).await;
 }
 
 async fn handle_event<E, H>(db: &DB, mqtt: &MqttConfig, handler: H, mut event_loop: E) -> ()
@@ -1225,12 +1275,12 @@ impl MqttConfig {
     async fn handle(
         &self,
         db: &DB,
-        args: Option<HashMap<String, Box<RawValue>>>,
-        extra: Option<HashMap<String, Box<RawValue>>>,
+        payload: &[u8],
+        trigger_info: HashMap<String, Box<RawValue>>,
     ) -> () {
         match self {
-            MqttConfig::Trigger(trigger) => trigger.handle(&db, args, extra).await,
-            MqttConfig::Capture(capture) => capture.handle(&db, args, extra).await,
+            MqttConfig::Trigger(trigger) => trigger.handle(&db, payload, trigger_info).await,
+            MqttConfig::Capture(capture) => capture.handle(&db, payload, trigger_info).await,
         }
     }
 }
@@ -1406,10 +1456,10 @@ impl MqttTrigger {
     async fn handle(
         &self,
         db: &DB,
-        args: Option<HashMap<String, Box<RawValue>>>,
-        extra: Option<HashMap<String, Box<RawValue>>>,
+        payload: &[u8],
+        trigger_info: HashMap<String, Box<RawValue>>,
     ) -> () {
-        if let Err(err) = run_job(args, extra, db, self).await {
+        if let Err(err) = run_job(payload, trigger_info, db, self).await {
             report_critical_error(
                 format!("Failed to trigger job from mqtt {}: {:?}", self.path, err),
                 db.clone(),
@@ -1421,6 +1471,21 @@ impl MqttTrigger {
     }
 }
 
+impl TriggerJobArgs<&[u8]> for MqttTrigger {
+    fn v1_payload_fn(payload: &[u8]) -> HashMap<String, Box<RawValue>> {
+        HashMap::from([("payload".to_string(), to_raw_value(&payload))])
+    }
+
+    fn v2_payload_fn(payload: &[u8]) -> HashMap<String, Box<RawValue>> {
+        let base64_payload = engine::general_purpose::STANDARD.encode(payload);
+        HashMap::from([("payload".to_string(), to_raw_value(&base64_payload))])
+    }
+
+    fn trigger_kind() -> TriggerKind {
+        TriggerKind::Mqtt
+    }
+}
+
 struct PublishData {
     topic: String,
     retain: bool,
@@ -1675,19 +1740,19 @@ impl CaptureConfigForMqttTrigger {
     async fn handle(
         &self,
         db: &DB,
-        args: Option<HashMap<String, Box<RawValue>>>,
-        extra: Option<HashMap<String, Box<RawValue>>>,
+        payload: &[u8],
+        trigger_info: HashMap<String, Box<RawValue>>,
     ) -> () {
-        let args = PushArgsOwned { args: args.unwrap_or_default(), extra: None };
-        let extra = extra.as_ref().map(to_raw_value);
+        let (main_args, preprocessor_args) =
+            MqttTrigger::build_capture_payloads(payload, trigger_info);
         if let Err(err) = insert_capture_payload(
             db,
             &self.workspace_id,
             &self.path,
             self.is_flow,
             &TriggerKind::Mqtt,
-            args,
-            extra,
+            main_args,
+            preprocessor_args,
             &self.owner,
         )
         .await

backend/windmill-api/src/postgres_triggers/handler.rs

@@ -25,9 +25,10 @@ use windmill_common::error::Error;
 use windmill_common::{
     db::UserDB,
     error::{self, JsonResult, Result},
-    utils::{not_found_if_none, paginate, Pagination, StripPath},
+    utils::{not_found_if_none, paginate, Pagination, StripPath, empty_as_none},
     worker::CLOUD_HOSTED,
 };
+use windmill_git_sync::{handle_deployment_metadata, DeployedObject};
 
 use super::{
     create_logical_replication_slot_query, create_publication_query, drop_publication_query,
@@ -45,7 +46,8 @@ pub struct Postgres {
     pub dbname: String,
     #[serde(default)]
     pub sslmode: String,
-    pub root_certificate_pem: String,
+    #[serde(default, deserialize_with = "empty_as_none")]
+    pub root_certificate_pem: Option<String>,
 }
 
 #[derive(Debug, Clone, FromRow, Serialize, Deserialize)]
@@ -453,6 +455,17 @@ pub async fn create_postgres_trigger(
 
     tx.commit().await?;
 
+    handle_deployment_metadata(
+        &authed.email,
+        &authed.username,
+        &db,
+        &w_id,
+        DeployedObject::PostgresTrigger { path: path.to_string() },
+        Some(format!("Postgres trigger '{}' created", path)),
+        true,
+    )
+    .await?;
+
     Ok((StatusCode::CREATED, path.to_string()))
 }
 
@@ -1158,7 +1171,7 @@ pub async fn update_postgres_trigger(
         &mut *tx,
         &authed,
         "postgres_triggers.update",
-        ActionKind::Create,
+        ActionKind::Update,
         &w_id,
         Some(&path),
         None,
@@ -1167,11 +1180,23 @@ pub async fn update_postgres_trigger(
 
     tx.commit().await?;
 
+    handle_deployment_metadata(
+        &authed.email,
+        &authed.username,
+        &db,
+        &w_id,
+        DeployedObject::PostgresTrigger { path: path.to_string() },
+        Some(format!("Postgres trigger '{}' updated", path)),
+        true,
+    )
+    .await?;
+
     Ok(workspace_path.to_string())
 }
 
 pub async fn delete_postgres_trigger(
     authed: ApiAuthed,
+    Extension(db): Extension<DB>,
     Extension(user_db): Extension<UserDB>,
     Path((w_id, path)): Path<(String, StripPath)>,
 ) -> Result<String> {
@@ -1203,6 +1228,17 @@ pub async fn delete_postgres_trigger(
 
     tx.commit().await?;
 
+    handle_deployment_metadata(
+        &authed.email,
+        &authed.username,
+        &db,
+        &w_id,
+        DeployedObject::PostgresTrigger { path: path.to_string() },
+        Some(format!("Postgres trigger '{}' deleted", path)),
+        true,
+    )
+    .await?;
+
     Ok(format!("Postgres trigger {path} deleted"))
 }
 
@@ -1231,6 +1267,7 @@ pub async fn exists_postgres_trigger(
 
 pub async fn set_enabled(
     authed: ApiAuthed,
+    Extension(db): Extension<DB>,
     Extension(user_db): Extension<UserDB>,
     Path((w_id, path)): Path<(String, StripPath)>,
     Json(payload): Json<SetEnabled>,
@@ -1279,6 +1316,17 @@ pub async fn set_enabled(
 
     tx.commit().await?;
 
+    handle_deployment_metadata(
+        &authed.email,
+        &authed.username,
+        &db,
+        &w_id,
+        DeployedObject::PostgresTrigger { path: path.to_string() },
+        Some(format!("Postgres trigger '{}' updated", path)),
+        true,
+    )
+    .await?;
+
     Ok(format!(
         "succesfully updated postgres trigger at path {} to status {}",
         path, payload.enabled

backend/windmill-api/src/postgres_triggers/mod.rs

@@ -2,6 +2,7 @@ use crate::{
     db::{ApiAuthed, DB},
     jobs::{run_flow_by_path_inner, run_script_by_path_inner, RunJobQuery},
     resources::try_get_resource_from_db_as,
+    trigger_helpers::TriggerJobArgs,
     users::fetch_api_authed,
 };
 use chrono::Utc;
@@ -30,7 +31,6 @@ use handler::{
     Relations,
 };
 use windmill_common::{db::UserDB, error::Error, utils::StripPath};
-use windmill_queue::PushArgsOwned;
 mod bool;
 mod converter;
 mod handler;
@@ -84,8 +84,8 @@ pub async fn get_raw_postgres_connection(
             }
         };
 
-        let options = if !db.root_certificate_pem.is_empty() {
-            options.ssl_root_cert_from_pem(db.root_certificate_pem.as_bytes().to_vec())
+        let options = if let Some(root_certificate_pem) = &db.root_certificate_pem {
+            options.ssl_root_cert_from_pem(root_certificate_pem.as_bytes().to_vec())
         } else {
             options
         };
@@ -96,7 +96,6 @@ pub async fn get_raw_postgres_connection(
             options
         }
     };
-
     Ok(PgConnection::connect_with(&options).await?)
 }
 
@@ -250,12 +249,19 @@ pub fn workspaced_service() -> Router {
 }
 
 async fn run_job(
-    args: Option<HashMap<String, Box<RawValue>>>,
-    extra: Option<HashMap<String, Box<RawValue>>>,
+    payload: HashMap<String, Box<RawValue>>,
     db: &DB,
     trigger: &PostgresTrigger,
 ) -> anyhow::Result<()> {
-    let args = PushArgsOwned { args: args.unwrap_or_default(), extra };
+    let args = PostgresTrigger::build_job_args(
+        &trigger.script_path,
+        trigger.is_flow,
+        &trigger.workspace_id,
+        db,
+        payload,
+        HashMap::new(),
+    )
+    .await?;
 
     let authed = fetch_api_authed(
         trigger.edited_by.clone(),
@@ -279,7 +285,6 @@ async fn run_job(
             StripPath(trigger.script_path.to_owned()),
             run_query,
             args,
-            None,
         )
         .await?;
     } else {
@@ -291,7 +296,6 @@ async fn run_job(
             StripPath(trigger.script_path.to_owned()),
             run_query,
             args,
-            None,
         )
         .await?;
     }

backend/windmill-api/src/postgres_triggers/trigger.rs

@@ -12,26 +12,26 @@ use crate::{
         run_job,
     },
     resources::try_get_resource_from_db_as,
+    trigger_helpers::TriggerJobArgs,
     users::fetch_api_authed,
 };
-use windmill_queue::TriggerKind;
 
 use bytes::{BufMut, Bytes, BytesMut};
 use chrono::TimeZone;
 use futures::{pin_mut, SinkExt, StreamExt};
-use native_tls::TlsConnector;
+use native_tls::{Certificate, TlsConnector};
 use pg_escape::{quote_identifier, quote_literal};
 use rand::seq::SliceRandom;
-use rust_postgres::{config::SslMode, Client, Config, CopyBothDuplex, SimpleQueryMessage};
+use rust_postgres::{config::SslMode, Client, Config, CopyBothDuplex, NoTls, SimpleQueryMessage};
 use rust_postgres_native_tls::MakeTlsConnector;
 use serde::Deserialize;
 use serde_json::value::RawValue;
 use sqlx::types::Json as SqlxJson;
 
 use windmill_common::{
-    db::UserDB, error, utils::report_critical_error, worker::to_raw_value, INSTANCE_NAME,
+    db::UserDB, error, triggers::TriggerKind, utils::report_critical_error, worker::to_raw_value,
+    INSTANCE_NAME,
 };
-use windmill_queue::PushArgsOwned;
 
 use super::{
     drop_logical_replication_slot_query, drop_publication_query, get_database_connection,
@@ -79,6 +79,45 @@ enum Error {
     Tls(#[from] native_tls::Error),
 }
 
+fn build_tls_connector(
+    ssl_mode: SslMode,
+    root_certificate_pem: Option<&String>,
+) -> Result<Option<MakeTlsConnector>, Error> {
+    let get_tls_builder_for_verify = |root_certificate: Option<&String>| {
+        let mut builder = TlsConnector::builder();
+        if let Some(root_certificate) = root_certificate {
+            let root_certificate_pem = Certificate::from_pem(root_certificate.as_bytes()).map_err(|e| {
+                Error::Common(error::Error::BadConfig(format!("Invalid Certs: {e:#}")))
+            })?;
+            builder.add_root_certificate(root_certificate_pem);
+        }
+        Ok::<_, Error>(builder)
+    };
+    let connector = match ssl_mode {
+        SslMode::Disable => return Ok(None),
+        SslMode::Require | SslMode::Prefer => {
+            let mut builder = TlsConnector::builder();
+            builder.danger_accept_invalid_certs(true);
+            builder.danger_accept_invalid_hostnames(true);
+            builder
+        }
+
+        SslMode::VerifyCa => {
+            let mut builder = get_tls_builder_for_verify(root_certificate_pem)?;
+            builder.danger_accept_invalid_hostnames(true);
+            builder
+        }
+
+        SslMode::VerifyFull => {
+            let builder = get_tls_builder_for_verify(root_certificate_pem)?;
+            builder
+        }
+        _ => unreachable!(),
+    };
+
+    Ok(Some(MakeTlsConnector::new(connector.build()?)))
+}
+
 pub struct PostgresSimpleClient(Client);
 
 impl PostgresSimpleClient {
@@ -112,20 +151,27 @@ impl PostgresSimpleClient {
             config.password(&database.password);
         }
 
-        if !database.root_certificate_pem.is_empty() {
-            config.ssl_root_cert(database.root_certificate_pem.as_bytes());
-        }
+        let connector = build_tls_connector(ssl_mode, database.root_certificate_pem.as_ref())?;
 
-        let connector = MakeTlsConnector::new(TlsConnector::new()?);
-
-        let (client, connection) = config.connect(connector).await?;
-
-        tokio::spawn(async move {
-            if let Err(e) = connection.await {
-                tracing::debug!("{:#?}", e);
-            };
-            tracing::info!("Successfully Connected into database");
-        });
+        let client = if let Some(connector) = connector {
+            let (client, connection) = config.connect(connector).await?;
+            tokio::spawn(async move {
+                if let Err(e) = connection.await {
+                    tracing::debug!("{:#?}", e);
+                };
+                tracing::info!("Successfully Connected into database");
+            });
+            client
+        } else {
+            let (client, connection) = config.connect(NoTls).await?;
+            tokio::spawn(async move {
+                if let Err(e) = connection.await {
+                    tracing::debug!("{:#?}", e);
+                };
+                tracing::info!("Successfully Connected into database");
+            });
+            client
+        };
 
         Ok(PostgresSimpleClient(client))
     }
@@ -369,13 +415,8 @@ impl PostgresTrigger {
         .await
     }
 
-    async fn handle(
-        &self,
-        db: &DB,
-        args: Option<HashMap<String, Box<RawValue>>>,
-        extra: Option<HashMap<String, Box<RawValue>>>,
-    ) -> () {
-        if let Err(err) = run_job(args, extra, db, self).await {
+    async fn handle(&self, db: &DB, payload: HashMap<String, Box<RawValue>>) -> () {
+        if let Err(err) = run_job(payload, db, self).await {
             report_critical_error(
                 format!(
                     "Failed to trigger job from postgres {}: {:?}",
@@ -390,6 +431,20 @@ impl PostgresTrigger {
     }
 }
 
+impl TriggerJobArgs<HashMap<String, Box<RawValue>>> for PostgresTrigger {
+    fn v1_payload_fn(payload: HashMap<String, Box<RawValue>>) -> HashMap<String, Box<RawValue>> {
+        payload
+    }
+
+    fn v2_payload_fn(payload: HashMap<String, Box<RawValue>>) -> HashMap<String, Box<RawValue>> {
+        payload
+    }
+
+    fn trigger_kind() -> TriggerKind {
+        TriggerKind::Postgres
+    }
+}
+
 struct PgInfo<'a> {
     postgres_resource_path: &'a str,
     publication_name: &'a str,
@@ -467,7 +522,6 @@ impl PostgresConfig {
 
         let client = PostgresSimpleClient::new(&database).await?;
 
-
         let publication = client
             .execute_query(&format!(
                 "SELECT pubname FROM pg_publication WHERE pubname = {}",
@@ -508,15 +562,10 @@ impl PostgresConfig {
         }
     }
 
-    async fn handle(
-        &self,
-        db: &DB,
-        args: Option<HashMap<String, Box<RawValue>>>,
-        extra: Option<HashMap<String, Box<RawValue>>>,
-    ) -> () {
+    async fn handle(&self, db: &DB, payload: HashMap<String, Box<RawValue>>) -> () {
         match self {
-            PostgresConfig::Trigger(trigger) => trigger.handle(&db, args, extra).await,
-            PostgresConfig::Capture(capture) => capture.handle(&db, args, extra).await,
+            PostgresConfig::Trigger(trigger) => trigger.handle(&db, payload).await,
+            PostgresConfig::Capture(capture) => capture.handle(&db, payload).await,
         }
     }
 
@@ -606,7 +655,7 @@ async fn listen_to_transactions(
                                         }
                                     };
 
-                                    
+
                                     let message = match message {
                                         Ok(message) => message,
                                         Err(err) => {
@@ -678,13 +727,9 @@ async fn listen_to_transactions(
                                                         ("old_row".to_string(), to_raw_value(&old_row)),
                                                         ("row".to_string(), to_raw_value(&row)),
                                                     ]);
-                                                    let extra = Some(HashMap::from([(
-                                                        "wm_trigger".to_string(),
-                                                        to_raw_value(&serde_json::json!({"kind": "postgres", })),
-                                                    )]));
-    
-    
-                                                    let _ = pg.handle(&db, Some(database_info), extra).await;
+
+
+                                                    let _ = pg.handle(&db, database_info).await;
                                                 }
                                                 Some((o_id, old_row, row, transaction_type)) => {
                                                     let relation = match relations.get_relation(o_id) {
@@ -694,7 +739,7 @@ async fn listen_to_transactions(
                                                             continue;
                                                         }
                                                     };
-                                                
+
                                                     if let Err(err) = old_row {
                                                         tracing::error!(
                                                             transaction_type = ?transaction_type,
@@ -707,7 +752,7 @@ async fn listen_to_transactions(
                                                             relation.name,
                                                         );
                                                     }
-                                                    
+
                                                     if let Err(err) = row {
                                                         tracing::error!(
                                                             transaction_type = ?transaction_type,
@@ -720,7 +765,7 @@ async fn listen_to_transactions(
                                                             relation.name,
                                                         );
                                                     }
-                                                    
+
                                                 }
                                                 _ => {}
                                             }
@@ -919,22 +964,17 @@ impl CaptureConfigForPostgresTrigger {
         }
     }
 
-    async fn handle(
-        &self,
-        db: &DB,
-        args: Option<HashMap<String, Box<RawValue>>>,
-        extra: Option<HashMap<String, Box<RawValue>>>,
-    ) -> () {
-        let args = PushArgsOwned { args: args.unwrap_or_default(), extra: None };
-        let extra = extra.as_ref().map(to_raw_value);
+    async fn handle(&self, db: &DB, payload: HashMap<String, Box<RawValue>>) -> () {
+        let main_args = PostgresTrigger::build_job_args_v2(false, payload.clone(), HashMap::new());
+        let preprocessor_args = PostgresTrigger::build_job_args_v2(true, payload, HashMap::new());
         if let Err(err) = insert_capture_payload(
             db,
             &self.workspace_id,
             &self.path,
             self.is_flow,
             &TriggerKind::Postgres,
-            args,
-            extra,
+            main_args,
+            preprocessor_args,
             &self.owner,
         )
         .await

backend/windmill-api/src/scripts.rs

@@ -223,7 +223,8 @@ async fn list_scripts(
             "draft_only",
             "ws_error_handler_muted",
             "no_main_func",
-            "codebase IS NOT NULL as use_codebase"
+            "codebase IS NOT NULL as use_codebase",
+            "kind"
         ])
         .left()
         .join("favorite")
@@ -298,7 +299,9 @@ async fn list_scripts(
     if let Some(it) = &lq.is_template {
         sqlb.and_where_eq("is_template", it);
     }
-    if let Some(lowercased_kinds) = lowercased_kinds {
+    if authed.is_operator {
+        sqlb.and_where_eq("kind", quote("script"));
+    } else if let Some(lowercased_kinds) = lowercased_kinds {
         let safe_kinds = lowercased_kinds
             .into_iter()
             .map(sql_builder::quote)
@@ -683,36 +686,40 @@ async fn create_script_internal<'c>(
 
     let validate_schema = should_validate_schema(&ns.content, &ns.language);
 
-    let (no_main_func, has_preprocessor) = match lang {
-        ScriptLang::Bun | ScriptLang::Bunnative | ScriptLang::Deno | ScriptLang::Nativets => {
-            let args = windmill_parser_ts::parse_deno_signature(&ns.content, true, true, None);
-            match args {
-                Ok(args) => (args.no_main_func, args.has_preprocessor),
-                Err(e) => {
-                    tracing::warn!(
-                        "Error parsing deno signature when deploying script {}: {:?}",
-                        ns.path,
-                        e
-                    );
-                    (None, None)
+    let (no_main_func, has_preprocessor) = if matches!(ns.kind, Some(ScriptKind::Preprocessor)) {
+        (ns.no_main_func, ns.has_preprocessor)
+    } else {
+        match lang {
+            ScriptLang::Bun | ScriptLang::Bunnative | ScriptLang::Deno | ScriptLang::Nativets => {
+                let args = windmill_parser_ts::parse_deno_signature(&ns.content, true, true, None);
+                match args {
+                    Ok(args) => (args.no_main_func, args.has_preprocessor),
+                    Err(e) => {
+                        tracing::warn!(
+                            "Error parsing deno signature when deploying script {}: {:?}",
+                            ns.path,
+                            e
+                        );
+                        (None, None)
+                    }
                 }
             }
-        }
-        ScriptLang::Python3 => {
-            let args = windmill_parser_py::parse_python_signature(&ns.content, None, true);
-            match args {
-                Ok(args) => (args.no_main_func, args.has_preprocessor),
-                Err(e) => {
-                    tracing::warn!(
-                        "Error parsing python signature when deploying script {}: {:?}",
-                        ns.path,
-                        e
-                    );
-                    (None, None)
+            ScriptLang::Python3 => {
+                let args = windmill_parser_py::parse_python_signature(&ns.content, None, true);
+                match args {
+                    Ok(args) => (args.no_main_func, args.has_preprocessor),
+                    Err(e) => {
+                        tracing::warn!(
+                            "Error parsing python signature when deploying script {}: {:?}",
+                            ns.path,
+                            e
+                        );
+                        (None, None)
+                    }
                 }
             }
+            _ => (ns.no_main_func, ns.has_preprocessor),
         }
-        _ => (ns.no_main_func, ns.has_preprocessor),
     };
 
     sqlx::query!(

backend/windmill-api/src/slack_approvals.rs

@@ -3,28 +3,21 @@ use axum::{
     Extension,
 };
 use hyper::StatusCode;
-use serde::{Deserialize, Serialize};
-use serde_json::value::{RawValue, Value};
-
-use sqlx::types::Uuid;
-use std::{collections::HashMap, str::FromStr};
-
-use regex::Regex;
 use reqwest::Client;
+use serde::{Deserialize, Serialize};
+use serde_json::Value;
+use sqlx::types::Uuid;
+use std::collections::HashMap;
+use windmill_common::error::Error;
+use windmill_common::variables::get_secret_value_as_admin;
 
+use crate::approvals::{
+    extract_w_id_from_resume_url, handle_resume_action, ApprovalFormDetails, FieldType,
+    MessageFormat, QueryDefaultArgsJson, QueryDynamicEnumJson, QueryFlowStepId, QueryMessage,
+    ResumeFormField, ResumeSchema,
+};
 use crate::db::{ApiAuthed, DB};
-use crate::jobs::{
-    cancel_suspended_job, get_resume_urls_internal, resume_suspended_job, QueryApprover,
-    QueryOrBody, ResumeUrls,
-};
-
-use windmill_common::{
-    cache,
-    error::{self, Error},
-    jobs::JobKind,
-    scripts::ScriptHash,
-    variables::get_secret_value_as_admin,
-};
+use crate::jobs::{QueryApprover, ResumeUrls};
 
 #[derive(Deserialize, Debug)]
 pub struct SlackFormData {
@@ -91,53 +84,6 @@ struct SelectedOption {
     value: String,
 }
 
-#[derive(Debug, Deserialize, Serialize)]
-struct ResumeSchema {
-    schema: Schema,
-}
-
-#[derive(Debug, Deserialize)]
-struct ResumeFormRow {
-    resume_form: Option<serde_json::Value>,
-    hide_cancel: Option<bool>,
-}
-
-#[derive(Debug, Deserialize, Serialize)]
-struct Schema {
-    order: Vec<String>,
-    required: Vec<String>,
-    properties: HashMap<String, ResumeFormField>,
-}
-
-#[derive(Debug, Deserialize, Serialize)]
-#[serde(rename_all = "lowercase")]
-enum FieldType {
-    Boolean,
-    String,
-    Number,
-    Integer,
-    #[serde(other)]
-    Unknown,
-}
-
-#[derive(Debug, Deserialize, Serialize)]
-struct ResumeFormField {
-    r#type: FieldType,
-    format: Option<String>,
-    default: Option<serde_json::Value>,
-    description: Option<String>,
-    title: Option<String>,
-    r#enum: Option<Vec<String>>,
-    #[serde(rename = "enumLabels")]
-    enum_labels: Option<HashMap<String, String>>,
-    nullable: Option<bool>,
-}
-
-#[derive(Deserialize)]
-pub struct QueryMessage {
-    message: Option<String>,
-}
-
 #[derive(Deserialize)]
 pub struct QueryResourcePath {
     slack_resource_path: String,
@@ -148,21 +94,6 @@ pub struct QueryChannelId {
     channel_id: String,
 }
 
-#[derive(Deserialize)]
-pub struct QueryFlowStepId {
-    flow_step_id: String,
-}
-
-#[derive(Deserialize, Debug)]
-pub struct QueryDefaultArgsJson {
-    default_args_json: Option<serde_json::Value>,
-}
-
-#[derive(Deserialize, Debug)]
-pub struct QueryDynamicEnumJson {
-    dynamic_enums_json: Option<serde_json::Value>,
-}
-
 #[derive(Deserialize, Debug)]
 struct ModalActionValue {
     w_id: String,
@@ -362,73 +293,21 @@ async fn handle_submission(
         return Ok(());
     }
 
-    // Use regex to extract information from private_metadata
-    let re = Regex::new(r"/api/w/(?P<w_id>[^/]+)/jobs_u/(?P<action>resume|cancel)/(?P<job_id>[^/]+)/(?P<resume_id>[^/]+)/(?P<secret>[a-fA-F0-9]+)(?:\?approver=(?P<approver>[^&]+))?").unwrap();
-    let captures = re.captures(resume_url.as_str()).ok_or_else(|| {
-        tracing::error!("Resume URL does not match the pattern.");
-        Error::BadRequest("Invalid URL format.".to_string())
-    })?;
+    // Use the common handler to process the resume/cancel action
+    handle_resume_action(authed, db.clone(), &resume_url, state_json, action).await?;
 
-    let (w_id, job_id, resume_id, secret, approver) = (
-        captures.name("w_id").map_or("", |m| m.as_str()),
-        captures.name("job_id").map_or("", |m| m.as_str()),
-        captures.name("resume_id").map_or("", |m| m.as_str()),
-        captures.name("secret").map_or("", |m| m.as_str()),
-        captures.name("approver").map(|m| m.as_str().to_string()),
-    );
-
-    let approver = QueryApprover { approver: approver };
-
-    // Convert job_id and resume_id to appropriate types
-    let job_uuid = Uuid::from_str(job_id)
-        .map_err(|_| Error::BadRequest("Invalid job ID format.".to_string()))?;
-
-    let resume_id_parsed = resume_id
-        .parse::<u32>()
-        .map_err(|_| Error::BadRequest("Invalid resume ID format.".to_string()))?;
-
-    // Call the appropriate function based on the action
-    let res = if action == "resume" {
-        resume_suspended_job(
-            authed,
-            Extension(db.clone()),
-            Path((
-                w_id.to_string(),
-                job_uuid,
-                resume_id_parsed,
-                secret.to_string(),
-            )),
-            Query(approver),
-            QueryOrBody(Some(state_json)),
-        )
-        .await
-    } else {
-        cancel_suspended_job(
-            authed,
-            Extension(db.clone()),
-            Path((
-                w_id.to_string(),
-                job_uuid,
-                resume_id_parsed,
-                secret.to_string(),
-            )),
-            Query(approver),
-            QueryOrBody(Some(state_json)),
-        )
-        .await
-    };
-    tracing::debug!("Resume job action result: {:#?}", res);
-    let slack_token = get_slack_token(&db, &resource_path, &w_id).await?;
+    let w_id = extract_w_id_from_resume_url(&resume_url)?;
+    let slack_token = get_slack_token(&db, &resource_path, w_id).await?;
     update_original_slack_message(action, slack_token, container).await?;
     Ok(())
 }
 
 async fn transform_schemas(
     text: &str,
-    properties: Option<&HashMap<String, ResumeFormField>>,
+    properties: Option<HashMap<String, ResumeFormField>>,
     urls: &ResumeUrls,
-    order: Option<&Vec<String>>,
-    required: Option<&Vec<String>>,
+    order: Option<Vec<String>>,
+    required: Option<Vec<String>>,
     default_args_json: Option<&serde_json::Value>,
     dynamic_enums_json: Option<&serde_json::Value>,
 ) -> Result<serde_json::Value, Error> {
@@ -443,16 +322,16 @@ async fn transform_schemas(
     })];
 
     if let Some(properties) = properties {
-        for key in order.unwrap() {
-            if let Some(schema) = properties.get(key) {
-                let is_required = required.unwrap().contains(key);
+        for key in order.unwrap_or_default() {
+            if let Some(schema) = properties.get(&key) {
+                let is_required = required.as_ref().map_or(false, |r| r.contains(&key));
 
-                let default_value = default_args_json.and_then(|json| json.get(key).cloned());
+                let default_value = default_args_json.and_then(|json| json.get(&key).cloned());
                 let dynamic_enums_value =
-                    dynamic_enums_json.and_then(|json| json.get(key).cloned());
+                    dynamic_enums_json.and_then(|json| json.get(&key).cloned());
 
                 let input_block = create_input_block(
-                    key,
+                    &key,
                     schema,
                     is_required,
                     default_value,
@@ -947,155 +826,59 @@ async fn get_modal_blocks(
     default_args_json: Option<&serde_json::Value>,
     dynamic_enums_json: Option<&serde_json::Value>,
 ) -> Result<axum::Json<serde_json::Value>, Error> {
-    let res = get_resume_urls_internal(
-        axum::Extension(db.clone()),
-        Path((w_id.to_string(), job_id, resume_id)),
-        Query(QueryApprover { approver: approver.map(|a| a.to_string()) }),
+    let approval_details = crate::approvals::get_approval_form_details(
+        db,
+        w_id,
+        job_id,
+        flow_step_id,
+        resume_id,
+        approver,
+        message,
+        MessageFormat::Slack,
     )
     .await?;
 
-    let urls = res.0;
+    let ApprovalFormDetails { message_str, urls, schema } = approval_details;
 
-    tracing::debug!("Job ID: {:?}", job_id);
-
-    let (job_kind, script_hash, raw_flow, parent_job_id, created_at, created_by, script_path, args) = sqlx::query!(
-        "SELECT
-            v2_as_queue.job_kind AS \"job_kind!: JobKind\",
-            v2_as_queue.script_hash AS \"script_hash: ScriptHash\",
-            v2_as_queue.raw_flow AS \"raw_flow: sqlx::types::Json<Box<RawValue>>\",
-            v2_as_completed_job.parent_job AS \"parent_job: Uuid\",
-            v2_as_completed_job.created_at AS \"created_at!: chrono::NaiveDateTime\",
-            v2_as_completed_job.created_by AS \"created_by!\",
-            v2_as_queue.script_path,
-            v2_as_queue.args AS \"args: sqlx::types::Json<Box<RawValue>>\"
-        FROM v2_as_queue
-        JOIN v2_as_completed_job ON v2_as_completed_job.parent_job = v2_as_queue.id
-        WHERE v2_as_completed_job.id = $1 AND v2_as_completed_job.workspace_id = $2
-        LIMIT 1",
-        job_id,
-        &w_id
+    // Get the card content
+    let card_content = transform_schemas(
+        &message_str,
+        schema
+            .as_ref()
+            .and_then(|s| s.resume_form.as_ref())
+            .map(|f| {
+                let inner_schema: ResumeSchema = serde_json::from_value(f.clone()).unwrap();
+                inner_schema.schema.properties
+            }),
+        &urls,
+        schema
+            .as_ref()
+            .and_then(|s| s.resume_form.as_ref())
+            .map(|f| {
+                let inner_schema: ResumeSchema = serde_json::from_value(f.clone()).unwrap();
+                inner_schema.schema.order
+            }),
+        schema
+            .as_ref()
+            .and_then(|s| s.resume_form.as_ref())
+            .map(|f| {
+                let inner_schema: ResumeSchema = serde_json::from_value(f.clone()).unwrap();
+                inner_schema.schema.required
+            }),
+        default_args_json,
+        dynamic_enums_json,
     )
-    .fetch_optional(&db)
-    .await
-    .map_err(|e| error::Error::BadRequest(e.to_string()))?
-    .ok_or_else(|| error::Error::BadRequest("This workflow is no longer running and has either already timed out or been cancelled or completed.".to_string()))
-    .map(|r| (r.job_kind, r.script_hash, r.raw_flow, r.parent_job, r.created_at, r.created_by, r.script_path, r.args))?;
+    .await?;
 
-    let flow_data = match cache::job::fetch_flow(&db, job_kind, script_hash).await {
-        Ok(data) => data,
-        Err(_) => {
-            if let Some(parent_job_id) = parent_job_id.as_ref() {
-                cache::job::fetch_preview_flow(&db, parent_job_id, raw_flow).await?
-            } else {
-                return Err(error::Error::BadRequest(
-                    "This workflow is no longer running and has either already timed out or been cancelled or completed.".to_string(),
-                ));
-            }
-        }
-    };
-
-    let flow_value = &flow_data.flow;
-    let flow_step_id = flow_step_id.unwrap_or("");
-    let module = flow_value.modules.iter().find(|m| m.id == flow_step_id);
-
-    tracing::debug!("Module: {:#?}", module);
-
-    let schema = module.and_then(|module| {
-        module.suspend.as_ref().map(|suspend| ResumeFormRow {
-            resume_form: suspend.resume_form.clone(),
-            hide_cancel: suspend.hide_cancel,
-        })
-    });
-
-    let args_str = args.map_or("None".to_string(), |a| a.get().to_string());
-    let parent_job_id_str = parent_job_id.map_or("None".to_string(), |id| id.to_string());
-    let script_path_str = script_path.as_deref().unwrap_or("None");
-
-    let created_at_formatted = created_at.format("%Y-%m-%d %H:%M:%S").to_string();
-
-    let mut message_str = format!(
-        "A workflow has been suspended and is waiting for approval:\n\n\
-        *Created by*: {created_by}\n\
-        *Created at*: {created_at_formatted}\n\
-        *Script path*: {script_path_str}\n\
-        *Args*: {args_str}\n\
-        *Flow ID*: {parent_job_id_str}\n\n"
-    );
-
-    // Append custom message if provided
-    if let Some(msg) = message {
-        message_str.push_str(msg);
-    }
-
-    tracing::debug!("Schema: {:#?}", schema);
-
-    if let Some(resume_schema) = schema {
-        let hide_cancel = resume_schema.hide_cancel.unwrap_or(false);
-
-        // if hide cancel is false add note to message
-        if !hide_cancel {
-            message_str.push_str("\n\n*NOTE*: closing this modal will cancel the workflow.\n\n");
-        }
-
-        // Convert message_str back to &str when needed
-        let message_str_ref: &str = &message_str;
-
-        if let Some(schema_obj) = resume_schema.resume_form {
-            let inner_schema: ResumeSchema =
-                serde_json::from_value(schema_obj.clone()).map_err(|e| {
-                    tracing::error!("Failed to deserialize form schema: {:?}", e);
-                    Error::BadRequest(
-                        "Failed to deserialize resume form schema! Unsupported form field used."
-                            .to_string(),
-                    )
-                })?;
-
-            let blocks = transform_schemas(
-                message_str_ref,
-                Some(&inner_schema.schema.properties),
-                &urls,
-                Some(&inner_schema.schema.order),
-                Some(&inner_schema.schema.required),
-                default_args_json,
-                dynamic_enums_json,
-            )
-            .await?;
-
-            tracing::debug!("Slack Blocks: {:#?}", blocks);
-            return Ok(axum::Json(construct_payload(
-                blocks,
-                hide_cancel,
-                trigger_id,
-                &urls.resume,
-                resource_path,
-                container,
-            )));
-        } else {
-            tracing::debug!("No suspend form found!");
-            let blocks = transform_schemas(
-                message_str_ref,
-                None,
-                &urls,
-                None,
-                None,
-                default_args_json,
-                dynamic_enums_json,
-            )
-            .await?;
-            return Ok(axum::Json(construct_payload(
-                blocks,
-                hide_cancel,
-                trigger_id,
-                &urls.resume,
-                resource_path,
-                container,
-            )));
-        }
-    } else {
-        Err(Error::BadRequest(
-            "No approval form schema found.".to_string(),
-        ))
-    }
+    tracing::debug!("Slack Blocks: {:#?}", card_content);
+    Ok(axum::Json(construct_payload(
+        card_content,
+        schema.as_ref().and_then(|s| s.hide_cancel).unwrap_or(false),
+        trigger_id,
+        &urls.resume,
+        resource_path,
+        container,
+    )))
 }
 
 fn construct_payload(

backend/windmill-api/src/teams_approvals_ee.rs

@@ -0,0 +1,7 @@
+use hyper::StatusCode;
+
+use windmill_common::error::Error;
+
+pub async fn request_teams_approval() -> Result<StatusCode, Error> {
+    Err(Error::InternalErr("enterprise feature only".to_string()))
+}

backend/windmill-api/src/trigger_helpers.rs

@@ -0,0 +1,444 @@
+use serde::Deserialize;
+use serde_json::value::RawValue;
+use std::collections::HashMap;
+use windmill_common::{
+    error::Result,
+    flows::FlowModuleValue,
+    get_latest_deployed_hash_for_path, get_latest_flow_version_info_for_path,
+    jobs::get_has_preprocessor_from_content_and_lang,
+    scripts::{get_full_hub_script_by_path, ScriptHash, ScriptLang},
+    triggers::{
+        HubOrWorkspaceId, RunnableFormat, RunnableFormatVersion, TriggerKind,
+        RUNNABLE_FORMAT_VERSION_CACHE,
+    },
+    utils::StripPath,
+    worker::to_raw_value,
+    FlowVersionInfo,
+};
+use windmill_queue::PushArgsOwned;
+
+use crate::{db::DB, HTTP_CLIENT};
+
+struct ScriptInfo {
+    has_preprocessor: Option<bool>,
+    language: ScriptLang,
+    content: String,
+    schema: Option<sqlx::types::Json<PartialSchema>>,
+}
+
+#[derive(Debug, Deserialize)]
+struct PropertyDefinition {
+    r#type: Option<String>,
+}
+
+#[derive(Debug, Deserialize)]
+struct PartialSchema {
+    properties: Option<HashMap<String, PropertyDefinition>>,
+}
+
+#[derive(Debug, Clone, Eq, PartialEq, Hash)]
+pub enum RunnableId {
+    FlowPath(String),
+    ScriptId(ScriptId),
+    HubScript(String),
+}
+
+impl RunnableId {
+    pub fn from_script_hash(hash: ScriptHash) -> Self {
+        Self::ScriptId(ScriptId::ScriptHash(hash))
+    }
+
+    pub fn from_script_path(path: &str) -> Self {
+        if path.starts_with("hub/") {
+            Self::HubScript(path.to_string())
+        } else {
+            Self::ScriptId(ScriptId::ScriptPath(path.to_string()))
+        }
+    }
+
+    pub fn from_flow_path(path: &str) -> Self {
+        Self::FlowPath(path.to_string())
+    }
+}
+
+#[derive(Debug, Clone, Eq, PartialEq, Hash)]
+pub enum ScriptId {
+    ScriptPath(String),
+    ScriptHash(ScriptHash),
+}
+
+impl ScriptId {
+    async fn get_script_hash(self, workspace_id: &str, db: &DB) -> Result<i64> {
+        let hash = match self {
+            ScriptId::ScriptPath(path) => {
+                let info = get_latest_deployed_hash_for_path(db, workspace_id, &path).await?;
+                info.hash
+            }
+            ScriptId::ScriptHash(hash) => hash.0,
+        };
+
+        Ok(hash)
+    }
+}
+
+async fn get_script_info(
+    db: &DB,
+    workspace_id: &str,
+    hash: i64,
+) -> std::result::Result<ScriptInfo, sqlx::Error> {
+    sqlx::query_as!(ScriptInfo, "SELECT has_preprocessor, language as \"language: _\", content, schema as \"schema: _\" FROM script WHERE workspace_id = $1 AND hash = $2", workspace_id, hash)
+        .fetch_one(db)
+        .await
+}
+
+fn runnable_format_from_schema_without_preprocessor(
+    trigger_kind: &TriggerKind,
+    schema: Option<sqlx::types::Json<PartialSchema>>,
+) -> RunnableFormat {
+    match trigger_kind {
+        TriggerKind::Mqtt
+            if schema.as_ref().is_some_and(|schema| {
+                schema.properties.as_ref().is_some_and(|properties| {
+                    properties.iter().any(|(key, def)| {
+                        key == "payload" && def.r#type.as_ref().is_some_and(|t| t == "array")
+                    })
+                })
+            }) =>
+        {
+            RunnableFormat { version: RunnableFormatVersion::V1, has_preprocessor: false }
+        }
+        TriggerKind::Kafka | TriggerKind::Nats
+            if schema.as_ref().is_some_and(|schema| {
+                schema
+                    .properties
+                    .as_ref()
+                    .is_some_and(|properties| properties.keys().any(|key| key == "msg"))
+            }) =>
+        {
+            RunnableFormat { version: RunnableFormatVersion::V1, has_preprocessor: false }
+        }
+        _ => RunnableFormat { version: RunnableFormatVersion::V2, has_preprocessor: false },
+    }
+}
+
+fn runnable_format_from_preprocessor_args(
+    args: Option<Vec<windmill_parser::Arg>>,
+) -> RunnableFormat {
+    if let Some(args) = args {
+        if args.iter().any(|arg| arg.name == "wm_trigger")
+            || (args.len() > 0 && args.iter().all(|arg| arg.name != "event"))
+        {
+            RunnableFormat { version: RunnableFormatVersion::V1, has_preprocessor: true }
+        } else {
+            RunnableFormat { version: RunnableFormatVersion::V2, has_preprocessor: true }
+        }
+    } else {
+        RunnableFormat { version: RunnableFormatVersion::V2, has_preprocessor: true }
+    }
+}
+
+enum PreprocessorInfo {
+    Preprocessor { content: String, language: ScriptLang },
+    NoPreprocessor { schema: Option<sqlx::types::Json<PartialSchema>> },
+}
+
+#[derive(Debug, Deserialize)]
+struct FlowInfo {
+    preprocessor_module: Option<sqlx::types::Json<FlowModuleValue>>,
+    schema: Option<sqlx::types::Json<PartialSchema>>,
+}
+
+fn get_preprocessor_args_from_content_and_language(
+    content: &str,
+    language: &ScriptLang,
+) -> Result<Option<Vec<windmill_parser::Arg>>> {
+    let args = match language {
+        ScriptLang::Bun | ScriptLang::Bunnative | ScriptLang::Deno | ScriptLang::Nativets => {
+            let args = windmill_parser_ts::parse_deno_signature(
+                &content,
+                true,
+                false,
+                Some("preprocessor".to_string()),
+            )?;
+            Some(args.args)
+        }
+        ScriptLang::Python3 => {
+            let args = windmill_parser_py::parse_python_signature(
+                &content,
+                Some("preprocessor".to_string()),
+                false,
+            )?;
+            Some(args.args)
+        }
+        _ => None,
+    };
+    Ok(args)
+}
+
+pub async fn get_runnable_format(
+    runnable_id: RunnableId,
+    workspace_id: &str,
+    db: &DB,
+    trigger_kind: &TriggerKind,
+) -> Result<RunnableFormat> {
+    let (key, preprocessor_info) = match runnable_id {
+        RunnableId::HubScript(path) => {
+            let Some(version) = path.split("/").nth(1) else {
+                return Err(windmill_common::error::Error::internal_err(
+                    "Invalid hub script path".to_string(),
+                ));
+            };
+
+            let version = match version.parse::<i64>() {
+                Ok(version) => version,
+                Err(_) => {
+                    return Err(windmill_common::error::Error::internal_err(
+                        "Invalid hub script version".to_string(),
+                    ));
+                }
+            };
+
+            let key = (HubOrWorkspaceId::Hub, version, trigger_kind.clone());
+
+            let runnable_format = RUNNABLE_FORMAT_VERSION_CACHE.get(&key);
+
+            if let Some(runnable_format) = runnable_format {
+                tracing::debug!("Using cached runnable format for hub script {path}");
+                return Ok(runnable_format);
+            }
+
+            let hub_script =
+                get_full_hub_script_by_path(StripPath(path.to_string()), &HTTP_CLIENT, Some(db))
+                    .await?;
+
+            let has_preprocessor = get_has_preprocessor_from_content_and_lang(
+                &hub_script.content,
+                &hub_script.language,
+            )?;
+
+            let partial_schema = serde_json::from_str(hub_script.schema.get())?;
+
+            (
+                key,
+                if has_preprocessor {
+                    PreprocessorInfo::Preprocessor {
+                        content: hub_script.content,
+                        language: hub_script.language,
+                    }
+                } else {
+                    PreprocessorInfo::NoPreprocessor {
+                        schema: Some(sqlx::types::Json(partial_schema)),
+                    }
+                },
+            )
+        }
+        RunnableId::FlowPath(path) => {
+            let FlowVersionInfo { version, .. } =
+                get_latest_flow_version_info_for_path(db, workspace_id, &path, true).await?;
+
+            let key = (
+                HubOrWorkspaceId::WorkspaceId(workspace_id.to_string()),
+                version,
+                trigger_kind.clone(),
+            );
+
+            let runnable_format = RUNNABLE_FORMAT_VERSION_CACHE.get(&key);
+
+            if let Some(runnable_format) = runnable_format {
+                tracing::debug!("Using cached runnable format for flow {path}");
+                return Ok(runnable_format);
+            }
+
+            let flow_info = sqlx::query_as!(
+                FlowInfo,
+                "SELECT
+                    value->'preprocessor_module'->'value' as \"preprocessor_module: _\",
+                    schema as \"schema: _\"
+                FROM flow_version
+                WHERE 
+                    path = $1
+                    AND workspace_id = $2
+                ORDER BY created_at DESC
+                LIMIT 1",
+                path,
+                workspace_id,
+            )
+            .fetch_one(db)
+            .await?;
+
+            if let Some(preprocessor_module) = flow_info.preprocessor_module {
+                match preprocessor_module.0 {
+                    FlowModuleValue::RawScript { content, language, .. } => {
+                        (key, PreprocessorInfo::Preprocessor { content, language })
+                    }
+                    FlowModuleValue::Script { path, hash, .. } => {
+                        let hash = if let Some(hash) = hash {
+                            hash.0
+                        } else {
+                            let script_hash =
+                                get_latest_deployed_hash_for_path(db, workspace_id, &path).await?;
+                            script_hash.hash
+                        };
+                        let script_info = get_script_info(db, workspace_id, hash).await?;
+                        (
+                            key,
+                            PreprocessorInfo::Preprocessor {
+                                content: script_info.content,
+                                language: script_info.language,
+                            },
+                        )
+                    }
+                    _ => {
+                        return Err(windmill_common::error::Error::internal_err(
+                            "Unsupported preprocessor module".to_string(),
+                        ));
+                    }
+                }
+            } else {
+                (
+                    key,
+                    PreprocessorInfo::NoPreprocessor { schema: flow_info.schema },
+                )
+            }
+        }
+        RunnableId::ScriptId(script_id) => {
+            let hash = script_id.get_script_hash(workspace_id, db).await?;
+            let key = (
+                HubOrWorkspaceId::WorkspaceId(workspace_id.to_string()),
+                hash,
+                trigger_kind.clone(),
+            );
+            let runnable_format = RUNNABLE_FORMAT_VERSION_CACHE.get(&key);
+
+            if let Some(runnable_format) = runnable_format {
+                tracing::debug!("Using cached runnable format for script {hash}");
+                return Ok(runnable_format);
+            }
+
+            let script_info = get_script_info(db, workspace_id, hash).await?;
+
+            if script_info.has_preprocessor.unwrap_or(false) {
+                (
+                    key,
+                    PreprocessorInfo::Preprocessor {
+                        content: script_info.content,
+                        language: script_info.language,
+                    },
+                )
+            } else {
+                (
+                    key,
+                    PreprocessorInfo::NoPreprocessor { schema: script_info.schema },
+                )
+            }
+        }
+    };
+
+    let runnable_format = match preprocessor_info {
+        PreprocessorInfo::Preprocessor { content, language } => {
+            let args = get_preprocessor_args_from_content_and_language(&content, &language)?;
+            runnable_format_from_preprocessor_args(args)
+        }
+        PreprocessorInfo::NoPreprocessor { schema } => {
+            runnable_format_from_schema_without_preprocessor(trigger_kind, schema)
+        }
+    };
+
+    RUNNABLE_FORMAT_VERSION_CACHE.insert(key, runnable_format);
+
+    Ok(runnable_format)
+}
+
+#[allow(dead_code)]
+pub trait TriggerJobArgs<T: Clone> {
+    fn v1_payload_fn(payload: T) -> HashMap<String, Box<RawValue>>;
+    fn v2_payload_fn(payload: T) -> HashMap<String, Box<RawValue>> {
+        Self::v1_payload_fn(payload)
+    }
+    fn trigger_kind() -> TriggerKind;
+
+    fn build_job_args_v2(
+        has_preprocessor: bool,
+        payload: T,
+        info: HashMap<String, Box<RawValue>>,
+    ) -> PushArgsOwned {
+        let trigger_kind = Self::trigger_kind();
+        let mut args = Self::v2_payload_fn(payload);
+        if has_preprocessor {
+            args.insert("kind".to_string(), to_raw_value(&trigger_kind.to_key()));
+            args.extend(info);
+            let args = HashMap::from([("event".to_string(), to_raw_value(&args))]);
+            PushArgsOwned { args, extra: None }
+        } else {
+            PushArgsOwned { args, extra: None }
+        }
+    }
+
+    fn build_job_args_v1(
+        has_preprocessor: bool,
+        payload: T,
+        info: HashMap<String, Box<RawValue>>,
+    ) -> PushArgsOwned {
+        let trigger_kind = Self::trigger_kind();
+        let trigger_key = trigger_kind.to_key();
+        let args = Self::v1_payload_fn(payload);
+        let extra = if has_preprocessor {
+            Some(HashMap::from([(
+                "wm_trigger".to_string(),
+                to_raw_value(&serde_json::json!({
+                    "kind": trigger_key,
+                    trigger_key: info
+                })),
+            )]))
+        } else {
+            None
+        };
+
+        PushArgsOwned { args, extra }
+    }
+
+    async fn build_job_args(
+        runnable_path: &str,
+        is_flow: bool,
+        w_id: &str,
+        db: &DB,
+        payload: T,
+        info: HashMap<String, Box<RawValue>>,
+    ) -> Result<PushArgsOwned> {
+        let runnable_id = if is_flow {
+            RunnableId::from_flow_path(runnable_path)
+        } else {
+            RunnableId::from_script_path(runnable_path)
+        };
+        Self::build_job_args_from_runnable_id(runnable_id, w_id, db, payload, info).await
+    }
+
+    async fn build_job_args_from_runnable_id(
+        runnable_id: RunnableId,
+        w_id: &str,
+        db: &DB,
+        payload: T,
+        info: HashMap<String, Box<RawValue>>,
+    ) -> Result<PushArgsOwned> {
+        let runnable_format =
+            get_runnable_format(runnable_id, w_id, db, &Self::trigger_kind()).await?;
+
+        match runnable_format {
+            RunnableFormat { version: RunnableFormatVersion::V1, has_preprocessor } => {
+                Ok(Self::build_job_args_v1(has_preprocessor, payload, info))
+            }
+            RunnableFormat { version: RunnableFormatVersion::V2, has_preprocessor } => {
+                Ok(Self::build_job_args_v2(has_preprocessor, payload, info))
+            }
+        }
+    }
+
+    fn build_capture_payloads(
+        payload: T,
+        info: HashMap<String, Box<RawValue>>,
+    ) -> (PushArgsOwned, PushArgsOwned) {
+        let main_args = Self::build_job_args_v2(false, payload.clone(), info.clone());
+        let preprocessor_args = Self::build_job_args_v2(true, payload, info);
+        (main_args, preprocessor_args)
+    }
+}