all

* chore(main): release 1.493.4

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

.cursor/rules/rust-best-practices.mdc

@@ -47,6 +47,7 @@ Windmill uses a workspace-based architecture with multiple crates:
 - Group related routes together
 - Use consistent response formats (JSON)
 - Follow proper authentication and authorization patterns
+- Do not forget to update backend/windmill-api/openapi.yaml after modifying an api endpoint
 
 ## Performance Optimizations
 

.cursor/rules/svelte5-best-practices.mdc

@@ -5,7 +5,7 @@ alwaysApply: false
 ---
 # Svelte 5 Best Practices
 
-This guide outlines best practices for developing with Svelte 5, incorporating the new Runes API and other modern Svelte features. They should be applied on every new files created, but not on existing svelte 4 files unless specifically asked to.
+This guide outlines best practices for developing with Svelte 5, incorporating the new Runes API and other modern Svelte features. These rules MUST NOT be applied on svelte 4 files unless explicitly asked to do so.
 
 ## Reactivity with Runes
 

.cursor/rules/windmill-overview.mdc

@@ -0,0 +1,76 @@
+---
+description: 
+globs: 
+alwaysApply: false
+---
+# Windmill Overview
+
+Windmill is an open-source developer platform for building internal tools, API integrations, background jobs, workflows, and user interfaces. It offers a unified system where scripts are automatically turned into sharable UIs and can be composed into flows or embedded in custom applications.
+
+## Core Capabilities
+
+- **Script Development and Execution**: Write and run scripts in Python, TypeScript/JavaScript (Deno/Bun), Go, Bash, SQL, and other languages
+- **Workflow Orchestration**: Compose scripts into multi-step flows with conditional logic, loops, and error handling
+- **UI Generation**: Automatically generate UIs from scripts or build custom applications with a low-code editor
+- **Job Scheduling**: Trigger scripts and flows on schedules, webhooks, or external events
+- **Resource Management**: Securely store and use credentials, databases, and other connections
+
+## Platform Architecture
+
+The Windmill platform consists of several key components:
+
+- **Frontend UI**: Web-based interface for script and flow development, app building, and result visualization
+- **API Server**: Central API that handles authentication, resource management, and job coordination
+- **Workers**: Execute scripts in their respective environments with proper sandboxing
+- **Database**: PostgreSQL database for storage of scripts, flows, resources, job results, and more
+- **Job Queue**: Queue system for managing job execution, implemented in PostgreSQL
+- **Client Libraries**: Libraries for interacting with Windmill from Python, TypeScript, or command line
+
+# Windmill Backend Architecture
+
+The Windmill backend is written in Rust and consists of several services working together. These services are designed for horizontal scaling with stateless API servers and workers that can be deployed across multiple machines.
+
+## Key Components
+
+- **API Server (`windmill-api`)**: Handles HTTP requests, authentication, and resource management
+- **Queue Manager (`windmill-queue`)**: Manages the job queue in PostgreSQL
+- **Worker System (`windmill-worker`)**: Executes jobs in sandboxed environments
+- **Common Utilities (`windmill-common`)**: Shared code used by multiple services
+- **Git Sync (`windmill-git-sync`)**: Synchronizes scripts with Git repositories
+
+## Job Execution System
+
+The job execution process follows these steps:
+
+1. The API server receives a request to run a script or flow and creates a job record in the database
+2. The job is added to the queue system in PostgreSQL
+3. Workers continuously poll the queue for jobs matching their capabilities
+4. When a job is picked up, it's routed to the appropriate language executor
+5. The script is executed in a sandboxed environment using NSJAIL for security
+6. Results are processed and stored in the database
+7. For flows, each step creates a new job that goes through the same process
+
+Windmill supports worker tags and groups to route jobs to workers with specific capabilities or resource access.
+
+# Windmill Frontend Architecture
+
+The Windmill frontend is built with Svelte and provides several key interfaces for interacting with the platform.
+
+## Key Components
+
+- **Script Builder**: Code editor with language support, schema inference, and dependency management
+- **Flow Builder**: Visual editor for creating multi-step workflows with branching and looping
+- **App Editor**: Grid-based editor for building custom UIs that integrate scripts and flows
+- **Schema Form System**: Generates form interfaces from script parameters automatically
+- **Result Viewer**: Visualizes job results, logs, and execution status
+
+The frontend uses the Monaco editor (same as VS Code) for code editing, with specialized language support for all supported script languages.
+
+## UI Framework
+
+The frontend is built with Svelte, providing a reactive and component-based architecture. Key frontend technologies include:
+
+- **Svelte/SvelteKit**: Core framework for UI components and routing
+- **Monaco Editor**: Code editing experience similar to VS Code
+- **Schema Form**: Automatic UI generation from TypeScript/JSON schemas
+- **Tailwind CSS**: Utility-first CSS framework for styling

.github/workflows/aider-after-review.yaml

@@ -5,12 +5,42 @@ on:
     types: [submitted]
 
 jobs:
-  auto-fix-review:
+  check-membership:
     if: github.event.review.state == 'changes_requested' && contains(github.event.pull_request.title, '[Aider PR]')
-    runs-on: ubicloud-standard-8
+    runs-on: ubicloud-standard-2
+    outputs:
+      is_member: ${{ steps.check-membership.outputs.is_member }}
+    steps:
+      - name: Check organization membership
+        id: check-membership
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          GITHUB_REPOSITORY: ${{ github.repository }}
+          REVIEWER: ${{ github.event.review.user.login }}
+          ORG_ACCESS_TOKEN: ${{ secrets.ORG_ACCESS_TOKEN }}
+        run: |
+          ORG="windmill-labs"
+          STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
+            -H "Authorization: token $ORG_ACCESS_TOKEN" \
+            -H "Accept: application/vnd.github+json" \
+            -H "X-GitHub-Api-Version: 2022-11-28" \
+            "https://api.github.com/orgs/$ORG/members/$REVIEWER")
+
+          if [ "$STATUS" -eq 204 ]; then
+            echo "is_member=true" >> $GITHUB_OUTPUT
+          else
+            echo "is_member=false" >> $GITHUB_OUTPUT
+          fi
+
+  check-and-prepare:
+    needs: check-membership
+    if: github.event.review.state == 'changes_requested' && contains(github.event.pull_request.title, '[Aider PR]') && needs.check-membership.outputs.is_member == 'true'
+    runs-on: ubicloud-standard-2
     permissions:
       contents: write
       pull-requests: write
+    outputs:
+      prompt_content: ${{ steps.prepare_prompt.outputs.prompt_content }}
     env:
       GEMINI_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
       GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
@@ -19,152 +49,46 @@ jobs:
       WINDMILL_TOKEN: ${{ secrets.WINDMILL_TOKEN }}
 
     steps:
-      - name: Harden Runner
-        uses: step-security/harden-runner@v2
-        with:
-          egress-policy: audit
-
-      - name: Check out code
-        uses: actions/checkout@v4
-        with:
-          fetch-depth: 0
-
-      - name: Configure Git User
-        run: |
-          git config --global user.name "github-actions[bot]"
-          git config --global user.email "github-actions[bot]@users.noreply.github.com"
-
-      - name: Checkout PR Branch
+      - name: Acknowledge Request
         env:
           GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          GITHUB_REPOSITORY: ${{ github.repository }}
         run: |
-          echo "PR review trigger: Checking out PR branch..."
-          PR_NUMBER=${{ github.event.pull_request.number }}
-          PR_HEAD_REF=$(gh pr view $PR_NUMBER --json headRefName -q .headRefName --repo $GITHUB_REPOSITORY)
-          if [[ -z "$PR_HEAD_REF" || "$PR_HEAD_REF" == "null" ]]; then
-             echo "::error::Could not determine PR head branch for PR #$PR_NUMBER via gh CLI."
-             exit 1
-          fi
-          echo "Checking out PR head branch: $PR_HEAD_REF for PR #$PR_NUMBER"
-          git fetch origin "refs/heads/${PR_HEAD_REF}:refs/remotes/origin/${PR_HEAD_REF}" --no-tags
-          git checkout "$PR_HEAD_REF"
-          echo "Successfully checked out branch $(git rev-parse --abbrev-ref HEAD)"
+          echo "Commenting on PR #${{ github.event.pull_request.number }} to acknowledge the /aider command."
+          gh pr comment ${{ github.event.pull_request.number }} --body "🤖 Aider is starting to work on your request. Please be patient, this might take a few minutes." --repo $GITHUB_REPOSITORY
 
-      - name: Set up Python
-        uses: actions/setup-python@v5
-        with:
-          python-version: "3.12"
-
-      - name: Install Aider and Dependencies
-        run: |
-          python -m pip install aider-install; aider-install
-          pip install -U google-generativeai
-          sudo apt-get update && sudo apt-get install -y jq
-
-      - name: Generate Prompt from Review
-        id: generate_prompt
+      - name: Prepare prompt for Aider
+        id: prepare_prompt
         shell: bash
+        env:
+          GITHUB_REPOSITORY: ${{ github.repository }}
+          PR_NUMBER: ${{ github.event.pull_request.number }}
+          REVIEW_BODY: ${{ github.event.review.body }}
         run: |
-          mkdir -p .github/aider
-          PROMPT_FILE_PATH=".github/aider/review-prompt.txt"
+          REVIEW_BODY_ESCAPED="${REVIEW_BODY//\\/\\\\}"
+          REVIEW_BODY_ESCAPED="${REVIEW_BODY_ESCAPED//\"/\\\"}"
 
-          # Get PR review body
-          REVIEW_BODY="${{ github.event.review.body }}"
-          PR_NUMBER="${{ github.event.pull_request.number }}"
-
-          # Get PR description for context NOT USED FOR NOW
-          # PR_DETAILS=$(gh pr view $PR_NUMBER --json title,body --repo $GITHUB_REPOSITORY)
-          # PR_TITLE=$(echo "$PR_DETAILS" | jq -r .title)
-          # PR_BODY=$(echo "$PR_DETAILS" | jq -r .body)
-
-          # Get all PR review comments
-          REVIEW_COMMENTS=$(gh pr view $PR_NUMBER --json reviews -q '.reviews[] | select(.state == "CHANGES_REQUESTED") | .body' --repo $GITHUB_REPOSITORY)
-          REVIEW_BODY_Q=$(printf '%q' "$REVIEW_BODY")
-
-          # Update query to get review comments from all review types, not just "CHANGES_REQUESTED"
           ALL_REVIEW_COMMENTS=$(gh api \
             -H "Accept: application/vnd.github+json" \
             -H "X-GitHub-Api-Version: 2022-11-28" \
-            /repos/$GITHUB_REPOSITORY/pulls/$PR_NUMBER/comments \
-            | jq '[.[] | {diff_hunk: .diff_hunk, path: .path, body: .body}]')
+            /repos/$GITHUB_REPOSITORY/pulls/$PR_NUMBER/comments)
+
+          FORMATTED_COMMENTS=$(jq -r '[.[] | {diff_hunk: .diff_hunk, path: .path, body: .body}]' <<< "$ALL_REVIEW_COMMENTS")
 
           BASE_PROMPT="Fix the following issues in the PR based on the review feedback. The review body is prepended with REVIEW. The review comments are prepended with REVIEW_COMMENTS. The review body and comments are separated by a blank line."
-          printf "%s\nREVIEW:\n%s\nREVIEW_COMMENTS:\n%s" \
-            "$BASE_PROMPT" "$REVIEW_BODY_Q" "$ALL_REVIEW_COMMENTS" > "$PROMPT_FILE_PATH"
-          echo "PROMPT_FILE_PATH=$PROMPT_FILE_PATH" >> $GITHUB_OUTPUT
 
-      - name: Run Aider with review prompt
-        run: |
-          aider \
-            --read .cursor/rules/rust-best-practices.mdc \
-            --read .cursor/rules/svelte5-best-practices.mdc \
-            --model gemini/gemini-2.5-pro-preview-05-06 \
-            --message-file .github/aider/review-prompt.txt \
-            --yes \
-            --no-check-update \
-            --auto-commits \
-            --no-analytics \
-            --no-gitignore \
-            | tee .github/aider/aider-output.txt || true
-          echo "Aider command completed. Output saved to .github/aider/aider-output.txt"
-          # Check if there are any changes to commit
-          if [[ -z "$(git status --porcelain)" ]]; then
-            echo "No changes detected after running Aider."
-            exit 0
-          fi
+          COMPLETE_PROMPT="${BASE_PROMPT}"$'\n'"REVIEW:"$'\n'"${REVIEW_BODY_ESCAPED}"$'\n'"REVIEW_COMMENTS:"$'\n'"${FORMATTED_COMMENTS}"
 
-      - name: Clean up prompt file
-        if: always()
-        run: rm -f .github/aider/review-prompt.txt
+          echo "prompt_content<<EOF" >> $GITHUB_OUTPUT
+          echo "$COMPLETE_PROMPT" >> $GITHUB_OUTPUT
+          echo "EOF" >> $GITHUB_OUTPUT
 
-      - name: Commit and Push Changes
-        id: commit_and_push
-        if: ${{ success() }}
-        run: |
-          CURRENT_BRANCH_NAME=$(git rev-parse --abbrev-ref HEAD)
-          echo "Attempting to push changes to PR branch $CURRENT_BRANCH_NAME for PR #${{ github.event.pull_request.number }}"
-
-          # Pull latest changes to avoid rejection due to non-fast-forward
-          git pull origin $CURRENT_BRANCH_NAME
-
-          if git push origin $CURRENT_BRANCH_NAME; then
-            echo "Push to $CURRENT_BRANCH_NAME successful."
-            echo "CHANGES_APPLIED=true" >> $GITHUB_OUTPUT
-          else
-            echo "::warning::Push to PR branch $CURRENT_BRANCH_NAME failed."
-            echo "CHANGES_APPLIED=false" >> $GITHUB_OUTPUT
-          fi
-
-      - name: Comment on PR
-        if: success()
-        env:
-          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-          PR_NUM: ${{ github.event.pull_request.number }}
-        run: |
-          # Create comment body in a temporary file to avoid command line length limits
-          if [[ "${{ steps.commit_and_push.outputs.CHANGES_APPLIED }}" == "true" ]]; then
-            cat > /tmp/pr-comment.md << EOL
-          🤖 I've automatically addressed the feedback based on the review.
-
-          ## Aider Output
-          \`\`\`
-          $(cat .github/aider/aider-output.txt || echo 'No output available')
-          \`\`\`
-
-          Please review the changes and let me know if further adjustments are needed.
-          EOL
-          else
-            cat > /tmp/pr-comment.md << EOL
-          🤖 I attempted to address the review feedback, but no modifications were made.
-
-          ## Aider Output
-          \`\`\`
-          $(cat .github/aider/aider-output.txt || echo 'No output available')
-          \`\`\`
-
-          Please review the output and provide additional guidance if needed.
-          EOL
-          fi
-
-          # Use the file for comment body
-          gh pr comment $PR_NUM --body-file /tmp/pr-comment.md
+  run-aider:
+    needs: [check-membership, check-and-prepare]
+    if: github.event.review.state == 'changes_requested' && contains(github.event.pull_request.title, '[Aider PR]') && needs.check-membership.outputs.is_member == 'true'
+    uses: ./.github/workflows/aider-common.yml
+    with:
+      needs_processing: false
+      base_prompt: ${{ needs.check-and-prepare.outputs.prompt_content }}
+      rules_files: ".cursor/rules/rust-best-practices.mdc .cursor/rules/svelte5-best-practices.mdc .cursor/rules/windmill-overview.mdc"
+    secrets: inherit

.github/workflows/aider-common.yml

@@ -0,0 +1,522 @@
+name: Aider Common Steps
+
+on:
+  workflow_call:
+    inputs:
+      issue_title:
+        description: "Title of the issue or PR"
+        required: false
+        type: string
+      issue_body:
+        description: "Body of the issue or PR"
+        required: false
+        type: string
+      instruction:
+        description: "Instruction for Aider"
+        required: false
+        type: string
+      issue_id:
+        description: "ID of the issue or PR"
+        required: false
+        type: string
+      needs_processing:
+        description: "Whether the issue needs to be processed by the external API"
+        required: false
+        type: boolean
+        default: true
+      base_prompt:
+        description: "Base prompt for Aider"
+        required: false
+        type: string
+        default: "Try to fix the following issue based on the instruction given by the user. The issue is prepended with the word ISSUE. The instruction is prepended with the word INSTRUCTION. The issue and instruction are separated by a blank line."
+      probe_prompt:
+        description: "Prompt for probe-chat"
+        required: false
+        type: string
+        default: 'I''m giving you a request that needs to be implemented. Your role is ONLY to give me the files that are relevant to the request and nothing else. The request is prepended with the word REQUEST. Give me all the files relevant to this request. Your output MUST be a single json array that can be parsed with programatic json parsing, with the relevant files. Files can be rust or typescript or javascript files. DO NOT INCLUDE ANY OTHER TEXT IN YOUR OUTPUT. ONLY THE JSON ARRAY. Example of output: ["file1.py", "file2.py"]'
+      rules_files:
+        description: "Rules files for Aider"
+        required: false
+        type: string
+    outputs:
+      files_to_edit:
+        description: "Files identified by probe-chat for editing"
+        value: ${{ jobs.common-steps.outputs.files_to_edit }}
+      final_prompt:
+        description: "Final prompt for Aider"
+        value: ${{ jobs.common-steps.outputs.final_prompt }}
+      pr_branch_name:
+        description: "Name of the branch used for PR"
+        value: ${{ jobs.common-steps.outputs.pr_branch_name }}
+      changes_applied_message:
+        description: "Message indicating changes were applied"
+        value: ${{ jobs.common-steps.outputs.changes_applied_message }}
+      changes_applied:
+        description: "Boolean indicating if changes were successfully applied"
+        value: ${{ jobs.common-steps.outputs.changes_applied }}
+
+jobs:
+  common-steps:
+    runs-on: ubicloud-standard-8
+    outputs:
+      files_to_edit: ${{ steps.probe_files.outputs.files_to_edit }}
+      final_prompt: ${{ steps.create_prompt.outputs.final_prompt }}
+      pr_branch_name: ${{ steps.commit_and_push.outputs.PR_BRANCH_NAME }}
+      changes_applied_message: ${{ steps.commit_and_push.outputs.CHANGES_APPLIED_MESSAGE }}
+      changes_applied: ${{ steps.commit_and_push.outputs.CHANGES_APPLIED }}
+    env:
+      GEMINI_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
+      GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
+      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      WINDMILL_TOKEN: ${{ secrets.WINDMILL_TOKEN }}
+      LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}
+      DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_AI_BOT_TOKEN }}
+
+    steps:
+      - name: Harden Runner
+        uses: step-security/harden-runner@v2
+        with:
+          egress-policy: audit
+
+      - name: Check out code
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+
+      - name: Checkout PR Branch
+        id: checkout_pr
+        if: (github.event_name == 'issue_comment' && github.event.issue.pull_request) || (github.event_name == 'pull_request_review')
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: |
+          echo "Issue comment trigger: Checking out PR branch..."
+          PR_NUMBER=""
+          if [ -n "${{ github.event.issue.number }}" ]; then
+            PR_NUMBER="${{ github.event.issue.number }}"
+          elif [ -n "${{ github.event.pull_request.number }}" ]; then
+            PR_NUMBER="${{ github.event.pull_request.number }}"
+          else
+            echo "::error::Could not determine PR number."
+            exit 1
+          fi
+          PR_HEAD_REF=$(gh pr view $PR_NUMBER --json headRefName -q .headRefName --repo $GITHUB_REPOSITORY)
+          if [[ -z "$PR_HEAD_REF" || "$PR_HEAD_REF" == "null" ]]; then
+             echo "::error::Could not determine PR head branch for PR #$PR_NUMBER via gh CLI."
+             exit 1
+          fi
+          echo "Checking out PR head branch: $PR_HEAD_REF for PR #$PR_NUMBER"
+          git fetch origin "refs/heads/${PR_HEAD_REF}:refs/remotes/origin/${PR_HEAD_REF}" --no-tags
+          git checkout "$PR_HEAD_REF"
+          echo "Successfully checked out branch $(git rev-parse --abbrev-ref HEAD)"
+          echo "PR_BRANCH=$PR_HEAD_REF" >> $GITHUB_OUTPUT
+
+      - name: Configure Git User
+        run: |
+          git config --global user.name "github-actions[bot]"
+          git config --global user.email "github-actions[bot]@users.noreply.github.com"
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+
+      - name: Cache Python dependencies
+        uses: actions/cache@v3
+        with:
+          path: ~/.cache/pip
+          key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt', '**/setup.py') }}
+          restore-keys: |
+            ${{ runner.os }}-pip-
+
+      - name: Install Aider and Dependencies
+        run: |
+          echo "Installing Aider..."
+          python -m pip install uv
+          python -m venv ~/uv-env
+          source ~/uv-env/bin/activate
+          uv pip install configargparse==1.7
+          uv pip install aider-chat==0.83.1
+          uv pip install -U google-generativeai
+          sudo apt-get update && sudo apt-get install -y jq
+          echo "$HOME/.local/bin" >> $GITHUB_PATH
+          echo "VIRTUAL_ENV_PATH=$HOME/uv-env" >> $GITHUB_ENV
+
+      - name: Create Prompt for Aider
+        id: create_prompt
+        shell: bash
+        env:
+          BASE_PROMPT_ENV: ${{ inputs.base_prompt }}
+          ISSUE_TITLE_ENV: ${{ inputs.issue_title }}
+          ISSUE_BODY_ENV: ${{ inputs.issue_body }}
+          INSTRUCTION_ENV: ${{ inputs.instruction }}
+          NEEDS_PROCESSING_ENV: ${{ inputs.needs_processing }}
+          WINDMILL_TOKEN: ${{ secrets.WINDMILL_TOKEN }}
+        run: |
+          set -e
+          FINAL_PROMPT_CONTENT=""
+
+          if [[ "$ISSUE_TITLE_ENV" != "" && "$ISSUE_BODY_ENV" != "" ]]; then
+            echo "Processing issue with title: $ISSUE_TITLE_ENV"
+            if [[ "$NEEDS_PROCESSING_ENV" == "true" ]]; then
+              echo "Needs processing is true. Calling Windmill API..."
+              JSON_PAYLOAD=$(jq -n \
+                --arg title "$ISSUE_TITLE_ENV" \
+                --arg body "$ISSUE_BODY_ENV" \
+                '{"body":{"issue_title":$title,"issue_body":$body}}')
+
+              echo "Windmill JSON Payload: $JSON_PAYLOAD"
+
+              API_RESULT_FILE=$(mktemp)
+              HTTP_CODE=$(curl -s -o "$API_RESULT_FILE" -w "%{http_code}" \
+                -X POST "https://app.windmill.dev/api/w/windmill-labs/jobs/run_wait_result/p/f/ai/quiet_script" \
+                -H "Content-Type: application/json" \
+                -H "Authorization: Bearer $WINDMILL_TOKEN" \
+                --data-binary "$JSON_PAYLOAD" \
+                --max-time 90)
+
+              BODY_CONTENT=$(cat "$API_RESULT_FILE")
+              rm -f "$API_RESULT_FILE" # Clean up temp file
+
+              echo "Windmill API HTTP Code: $HTTP_CODE"
+              if [[ "$HTTP_CODE" -eq 200 ]]; then
+                PROCESSED_ISSUE_PROMPT=$(echo "$BODY_CONTENT" | jq -r '.effective_body // empty')
+                if [[ -z "$PROCESSED_ISSUE_PROMPT" || "$PROCESSED_ISSUE_PROMPT" == "null" ]]; then
+                  echo "::warning::Windmill API returned 200 but effective_body was empty or null."
+                  EFFECTIVE_ISSUE_CONTENT_FOR_PROMPT="$ISSUE_BODY_ENV"
+                else
+                  echo "Successfully processed issue via Windmill API."
+                  EFFECTIVE_ISSUE_CONTENT_FOR_PROMPT="$PROCESSED_ISSUE_PROMPT"
+                fi
+                FINAL_PROMPT_CONTENT=$(printf "%s\nISSUE:\n%s\nINSTRUCTION:\n%s" \
+                  "$BASE_PROMPT_ENV" "$EFFECTIVE_ISSUE_CONTENT_FOR_PROMPT" "$INSTRUCTION_ENV")
+              else
+                echo "::error::Windmill API call failed (HTTP $HTTP_CODE). Using raw issue content for prompt."
+                FINAL_PROMPT_CONTENT=$(printf "%s\nISSUE:\n%s\nINSTRUCTION:\n%s" \
+                  "$BASE_PROMPT_ENV" "$ISSUE_BODY_ENV" "$INSTRUCTION_ENV")
+              fi
+            else
+              echo "Needs processing is false. Using raw issue content for prompt."
+              FINAL_PROMPT_CONTENT=$(printf "%s\nISSUE:\n%s\nINSTRUCTION:\n%s" \
+                "$BASE_PROMPT_ENV" "$ISSUE_BODY_ENV" "$INSTRUCTION_ENV")
+            fi
+          else
+            echo "No issue title or body given. Using base prompt."
+            FINAL_PROMPT_CONTENT=$(printf "%s\nINSTRUCTION:\n%s" "$BASE_PROMPT_ENV" "$INSTRUCTION_ENV")
+          fi
+
+          echo "Final prompt: $FINAL_PROMPT_CONTENT"
+          echo "final_prompt<<EOF_AIDER_PROMPT" >> "$GITHUB_OUTPUT"
+          echo "$FINAL_PROMPT_CONTENT" >> "$GITHUB_OUTPUT"
+          echo "EOF_AIDER_PROMPT" >> "$GITHUB_OUTPUT"
+
+      - name: Probe Chat for Relevant Files
+        id: probe_files
+        shell: bash
+        env:
+          FINAL_PROMPT: ${{ steps.create_prompt.outputs.final_prompt }}
+          PROBE_PROMPT: ${{ inputs.probe_prompt }}
+        run: |
+          echo "Running probe-chat to find relevant files..."
+
+          MESSAGE_FOR_PROBE=$(printf "%s\nREQUEST:\n%s" "$PROBE_PROMPT" "$FINAL_PROMPT")
+
+          set -o pipefail
+          PROBE_OUTPUT=$(npx --yes @buger/probe-chat@latest --max-iterations 50 --model-name gemini-2.5-pro-preview-05-06 --message "$MESSAGE_FOR_PROBE") || {
+            echo "::error::probe-chat command failed. Output:"
+            echo "$PROBE_OUTPUT"
+            exit 1
+          }
+          set +o pipefail
+          echo "Probe-chat raw output:"
+          echo "$PROBE_OUTPUT"
+
+          JSON_FILES=$(echo "$PROBE_OUTPUT" | sed -n '/^\s*\[/,$p' | sed '/^\s*\]/q')
+          echo "Extracted JSON block:"
+          echo "$JSON_FILES"
+
+          FILES_LIST=$(echo "$JSON_FILES" | jq -e -r '[.[] | select(type == "string" and . != "" and . != null and (endswith("/") | not))] | join(" ")' || echo "")
+
+          if [[ -z "$FILES_LIST" ]]; then
+             echo "::warning::probe-chat did not identify any relevant files."
+          fi
+
+          echo "Formatted files list for aider: $FILES_LIST"
+          echo "files_to_edit=$FILES_LIST" >> $GITHUB_OUTPUT
+
+      - name: Cache Aider tags
+        uses: actions/cache@v3
+        with:
+          path: .aider.tags.cache.v4
+          key: ${{ runner.os }}-aider-${{ github.sha }}
+          restore-keys: |
+            ${{ runner.os }}-aider-
+
+      - name: Prepare branch for Aider
+        id: prepare_branch
+        env:
+          ISSUE_ID: ${{ inputs.issue_id }}
+        run: |
+          if [[ "$ISSUE_ID" != "" ]]; then
+            BRANCH_NAME="aider-fix-issue-${ISSUE_ID}"
+
+            # Check if branch exists remotely
+            if git ls-remote --heads origin $BRANCH_NAME | grep -q $BRANCH_NAME; then
+              echo "Branch $BRANCH_NAME already exists remotely, fetching it"
+              git fetch origin $BRANCH_NAME
+              git checkout $BRANCH_NAME
+              git pull origin $BRANCH_NAME
+            else
+              echo "Creating new branch $BRANCH_NAME"
+              git checkout -b $BRANCH_NAME
+            fi
+            echo "BRANCH_NAME=$BRANCH_NAME" >> $GITHUB_OUTPUT
+          else
+            # We're in a pull_request_review event
+            PR_NUMBER="${{ github.event.pull_request.number }}"
+            PR_HEAD_REF="${{ github.event.pull_request.head.ref }}"
+            
+            echo "Handling pull_request_review for PR #$PR_NUMBER on branch $PR_HEAD_REF"
+            
+            # Ensure we're on the correct branch
+            git config pull.rebase true
+            git fetch origin $PR_HEAD_REF
+            git checkout $PR_HEAD_REF
+            git pull origin $PR_HEAD_REF
+            
+            echo "Using PR branch $PR_HEAD_REF for PR #$PR_NUMBER"
+            echo "BRANCH_NAME=$PR_HEAD_REF" >> $GITHUB_OUTPUT
+          fi
+
+      - name: Run Aider
+        id: run_aider
+        shell: bash
+        env:
+          FILES_TO_EDIT: ${{ steps.probe_files.outputs.files_to_edit }}
+          FINAL_PROMPT: ${{ steps.create_prompt.outputs.final_prompt }}
+          RULES_FILES: ${{ inputs.rules_files }}
+        run: |
+          source $VIRTUAL_ENV_PATH/bin/activate
+          echo "$FINAL_PROMPT" > .aider_final_prompt.txt
+          echo "FILES_TO_EDIT: $FILES_TO_EDIT"
+
+          RULES=""
+          if [ -n "$RULES_FILES" ]; then
+            for rule in $RULES_FILES; do
+              RULES="$RULES --read $rule"
+            done
+          fi
+
+          aider \
+            $RULES \
+            $FILES_TO_EDIT \
+            --model gemini/gemini-2.5-pro-preview-05-06 \
+            --message-file .aider_final_prompt.txt \
+            --yes \
+            --no-check-update \
+            --auto-commits \
+            --no-analytics \
+            --no-gitignore \
+            | tee .aider_output.txt || true
+
+          echo "Aider command completed. Output saved to .aider_output.txt"
+
+      - name: Cache Node.js dependencies
+        uses: actions/cache@v3
+        with:
+          path: ~/.npm
+          key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json', '**/yarn.lock') }}
+          restore-keys: |
+            ${{ runner.os }}-node-
+
+      - name: Commit and Push Changes
+        id: commit_and_push
+        env:
+          ISSUE_ID: ${{ inputs.issue_id }}
+          BRANCH_NAME: ${{ steps.prepare_branch.outputs.BRANCH_NAME }}
+        run: |
+          if [[ "$ISSUE_ID" != "" ]]; then
+            # Check if there are any uncommitted changes
+            if [[ -n $(git status --porcelain) ]]; then
+              echo "Found uncommitted changes, committing them"
+              git add .
+              git commit -m "Aider changes"
+            fi
+
+            # Push changes to the branch
+            if git push origin $BRANCH_NAME; then
+              echo "Pushed to branch $BRANCH_NAME"
+              echo "PR_BRANCH_NAME=$BRANCH_NAME" >> $GITHUB_OUTPUT
+              echo "CHANGES_APPLIED_MESSAGE=Aider changes pushed to branch $BRANCH_NAME." >> $GITHUB_OUTPUT
+              echo "CHANGES_APPLIED=true" >> $GITHUB_OUTPUT
+            else
+              echo "::warning::Push to PR branch $BRANCH_NAME failed."
+              echo "CHANGES_APPLIED_MESSAGE=Aider ran, but failed to push changes to PR branch $BRANCH_NAME." >> $GITHUB_OUTPUT
+              echo "CHANGES_APPLIED=false" >> $GITHUB_OUTPUT
+            fi
+          else
+            # We're in a pull_request_review event
+            PR_HEAD_REF="${{ github.event.pull_request.head.ref }}"
+            echo "Attempting to push changes to PR branch $PR_HEAD_REF"
+            if git push origin $PR_HEAD_REF; then
+              echo "Push to $PR_HEAD_REF successful (or no new changes to push)."
+              echo "CHANGES_APPLIED_MESSAGE=Aider changes (if any) pushed to PR branch $PR_HEAD_REF." >> $GITHUB_OUTPUT
+              echo "PR_BRANCH_NAME=$PR_HEAD_REF" >> $GITHUB_OUTPUT
+              echo "CHANGES_APPLIED=true" >> $GITHUB_OUTPUT
+            else
+              echo "::warning::Push to PR branch $PR_HEAD_REF failed."
+              echo "CHANGES_APPLIED_MESSAGE=Aider ran, but failed to push changes to PR branch $PR_HEAD_REF." >> $GITHUB_OUTPUT
+              echo "CHANGES_APPLIED=false" >> $GITHUB_OUTPUT
+            fi
+          fi
+
+      - name: Create Pull Request
+        if: always() && (github.event_name == 'issue_comment' || github.event_name == 'repository_dispatch') && !github.event.issue.pull_request && steps.commit_and_push.outputs.PR_BRANCH_NAME != ''
+        id: create_pr
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          PR_BRANCH: ${{ steps.commit_and_push.outputs.PR_BRANCH_NAME }}
+          ISSUE_NUM: ${{ inputs.issue_id }}
+          ISSUE_TITLE: ${{ inputs.issue_title }}
+          GITHUB_EVENT_NAME: ${{ github.event_name }}
+        run: |
+          # Create PR description in a temporary file to avoid command line length limits and ensure it stays under 40k chars
+          HEADER="This PR was created automatically by Aider to fix issue #${ISSUE_NUM}."
+          # if event is repository_dispatch, add the issue title to the header
+          if [ "$GITHUB_EVENT_NAME" == "repository_dispatch" ]; then
+            if [[ "${{ github.event.client_payload.source }}" == "linear" ]]; then
+              HEADER="This PR was created automatically by Aider to fix issue #linear:${ISSUE_NUM}."
+            elif [[ "${{ github.event.client_payload.source }}" == "discord" ]]; then
+              HEADER="This PR was created automatically by Aider to fix issue #discord:${ISSUE_NUM}."
+            fi
+          fi
+          cat > /tmp/pr-description.md << EOL | head -c 40000
+          $HEADER
+
+          ## Aider Output
+          \`\`\`
+          $(cat .aider_output.txt || echo "No output available")
+          \`\`\`
+          EOL
+
+          # Create PR using the file for the body content, handle errors gracefully
+          set +e  # Don't exit on error
+          PR_TITLE="[Aider PR] Fix: ${ISSUE_TITLE}"
+          if [ -z "$ISSUE_TITLE" ]; then
+            PR_TITLE="[Aider PR] AI changes after request"
+          fi
+          gh pr create \
+            --title "$PR_TITLE" \
+            --body-file /tmp/pr-description.md \
+            --head "$PR_BRANCH" \
+            --base main \
+            --draft
+          PR_CREATE_EXIT_CODE=$?
+          set -e  # Re-enable exit on error
+
+          if [ $PR_CREATE_EXIT_CODE -eq 0 ]; then
+            echo "PR created successfully"
+            PR_URL=$(gh pr view $PR_BRANCH --json url --jq .url)
+            echo "PR_URL=$PR_URL" >> $GITHUB_OUTPUT
+            echo "PR_CREATED=true" >> $GITHUB_OUTPUT
+          else
+            echo "Warning: Failed to create PR. Exit code: $PR_CREATE_EXIT_CODE"
+            echo "PR_CREATED=false" >> $GITHUB_OUTPUT
+            # Continue workflow despite PR creation failure
+          fi
+
+      - name: Comment on PR with Aider Output
+        if: always() && github.event_name == 'pull_request_review' && steps.commit_and_push.outputs.CHANGES_APPLIED != ''
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          PR_NUM: ${{ github.event.pull_request.number }}
+          JOB_STATUS: ${{ job.status }}
+        run: |
+          # Create comment body in a temporary file to avoid command line length limits
+          if [[ "${{ steps.commit_and_push.outputs.CHANGES_APPLIED }}" == "true" ]]; then
+            if [[ "$JOB_STATUS" == "success" ]]; then
+              STATUS_PREFIX="🤖 I've automatically addressed the feedback based on the review."
+            else
+              STATUS_PREFIX="⚠️ I attempted to address the feedback, but encountered some issues."
+            fi
+          else
+            if [[ "$JOB_STATUS" == "success" ]]; then
+              STATUS_PREFIX="🤖 I attempted to address the review feedback, but no modifications were made."
+            else
+              STATUS_PREFIX="⚠️ I encountered issues while attempting to address the feedback, and no modifications were made."
+            fi
+          fi
+
+          cat > /tmp/pr-comment.md << EOL
+          ${STATUS_PREFIX}
+
+          ## Aider Output
+          \`\`\`
+          $(cat .aider_output.txt || echo 'No output available')
+          \`\`\`
+
+          Please review the output and provide additional guidance if needed.
+          EOL
+
+          # Use the file for comment body
+          gh pr comment $PR_NUM --body-file /tmp/pr-comment.md
+
+      - name: Comment on issue/PR to let the user know Aider has finished working on the request
+        if: always() && github.event_name == 'issue_comment'
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          GITHUB_REPOSITORY: ${{ github.repository }}
+          JOB_STATUS: ${{ job.status }}
+          PR_CREATED: ${{ steps.create_pr.outputs.PR_CREATED }}
+          PR_URL: ${{ steps.create_pr.outputs.PR_URL }}
+        run: |
+          echo "Commenting on issue/PR #${{ github.event.issue.number }} to let the user know Aider has finished working on the request."
+
+          if [[ "$JOB_STATUS" == "success" ]]; then
+            if [[ "$PR_CREATED" == "true" ]]; then
+              COMMENT_BODY="🤖 Aider has finished working on your request. A PR has been created. $PR_URL"
+            else
+              COMMENT_BODY="🤖 Aider has finished working on your request, but was unable to create a PR."
+            fi
+          else
+            COMMENT_BODY="⚠️ Aider encountered issues while working on your request. Please check the workflow logs for details."
+          fi
+
+          gh issue comment ${{ github.event.issue.number }} --body "$COMMENT_BODY" --repo $GITHUB_REPOSITORY
+
+      - name: Comment on linear issue to let the user know Aider has finished working on the request
+        if: always() && github.event_name == 'repository_dispatch'
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          GITHUB_REPOSITORY: ${{ github.repository }}
+          JOB_STATUS: ${{ job.status }}
+          LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}
+          PR_CREATED: ${{ steps.create_pr.outputs.PR_CREATED }}
+          PR_URL: ${{ steps.create_pr.outputs.PR_URL }}
+          DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_AI_BOT_TOKEN }}
+          SOURCE: ${{ github.event.client_payload.source }}
+        run: |
+          echo "Notifying user about Aider completion status for $SOURCE request #${{ github.event.client_payload.issue_id }}"
+          if [[ "$JOB_STATUS" == "success" ]]; then
+            if [[ "$PR_CREATED" == "true" ]]; then
+              COMMENT_BODY="🤖 Aider has finished working on your request. A PR has been created. $PR_URL"
+            else
+              COMMENT_BODY="🤖 Aider has finished working on your request, but was unable to create a PR."
+            fi
+          else
+            COMMENT_BODY="⚠️ Aider encountered issues while working on your request. Please check the workflow logs for details."
+          fi
+
+          if [[ "$SOURCE" == "discord" ]]; then
+            curl -X POST \
+              -H "Authorization: Bot $DISCORD_BOT_TOKEN" \
+              -H "Content-Type: application/json" \
+              "https://discord.com/api/v10/channels/${{ github.event.client_payload.channel_id }}/messages" \
+              -d "{\"content\":\"${COMMENT_BODY}\"}"
+          else
+            curl -X POST \
+              -H "Authorization: $LINEAR_API_KEY" \
+              -H "Content-Type: application/json" \
+              "https://api.linear.app/graphql" \
+              -d "{\"query\":\"mutation { commentCreate(input: { issueId: \\\"${{ github.event.client_payload.issue_id }}\\\", body: \\\"${COMMENT_BODY}\\\" }) { success } }\"}"
+          fi

.github/workflows/aider-external.yaml

@@ -0,0 +1,80 @@
+name: External Aider Issue Fix
+
+on:
+  repository_dispatch:
+    types: [external_issue_fix]
+
+jobs:
+  check-and-prepare:
+    runs-on: ubicloud-standard-2
+    permissions:
+      contents: write
+      pull-requests: write
+    outputs:
+      issue_title: ${{ steps.determine_inputs.outputs.ISSUE_TITLE }}
+      issue_body: ${{ steps.determine_inputs.outputs.ISSUE_BODY }}
+      instruction: ${{ steps.determine_inputs.outputs.INSTRUCTION }}
+    env:
+      GEMINI_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
+      GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
+      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      WINDMILL_TOKEN: ${{ secrets.WINDMILL_TOKEN }}
+      LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}
+      DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_AI_BOT_TOKEN }}
+
+    steps:
+      - name: Acknowledge Request
+        env:
+          LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}
+          DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_AI_BOT_TOKEN }}
+        run: |
+          if [[ "${{ github.event.client_payload.source }}" == "linear" ]]; then
+            echo "Commenting on Linear issue #${{ github.event.client_payload.issue_id }} to acknowledge the request."
+            curl -X POST \
+              -H "Authorization: $LINEAR_API_KEY" \
+              -H "Content-Type: application/json" \
+              "https://api.linear.app/graphql" \
+              -d "{\"query\":\"mutation { commentCreate(input: { issueId: \\\"${{ github.event.client_payload.issue_id }}\\\", body: \\\"🤖 Aider is starting to work on your request. I'll update you here once I have a PR ready. Please be patient, this might take a few minutes.\\\" }) { success } }\"}"
+          elif [[ "${{ github.event.client_payload.source }}" == "discord" ]]; then
+            echo "Commenting on Discord thread #${{ github.event.client_payload.channel_id }} to acknowledge the request."
+            curl -X POST \
+              -H "Authorization: Bot $DISCORD_BOT_TOKEN" \
+              -H "Content-Type: application/json" \
+              "https://discord.com/api/v10/channels/${{ github.event.client_payload.channel_id }}/messages" \
+              -d "{\"content\":\"🤖 Aider is starting to work on your request. I'll update you here once I have a PR ready. Please be patient, this might take a few minutes.\"}"
+          fi
+
+      - name: Determine inputs for Aider
+        id: determine_inputs
+        shell: bash
+        env:
+          ISSUE_TITLE: ${{ github.event.client_payload.issue_title }}
+          ISSUE_BODY: ${{ github.event.client_payload.issue_body }}
+          INSTRUCTION: ${{ github.event.client_payload.instruction }}
+        run: |
+          echo "Determining inputs for Aider..."
+
+          echo "ISSUE_TITLE<<EOF_AIDER_TITLE" >> "$GITHUB_OUTPUT"
+          echo "$ISSUE_TITLE" >> "$GITHUB_OUTPUT"
+          echo "EOF_AIDER_TITLE" >> "$GITHUB_OUTPUT"
+
+          echo "ISSUE_BODY<<EOF_AIDER_BODY" >> "$GITHUB_OUTPUT"
+          echo "$ISSUE_BODY" >> "$GITHUB_OUTPUT"
+          echo "EOF_AIDER_BODY" >> "$GITHUB_OUTPUT"
+
+          echo "INSTRUCTION<<EOF_AIDER_INSTRUCTION" >> "$GITHUB_OUTPUT"
+          echo "$INSTRUCTION" >> "$GITHUB_OUTPUT"
+          echo "EOF_AIDER_INSTRUCTION" >> "$GITHUB_OUTPUT"
+          echo "Finished determining inputs."
+
+  run-aider:
+    needs: check-and-prepare
+    uses: ./.github/workflows/aider-common.yml
+    with:
+      issue_title: ${{ needs.check-and-prepare.outputs.issue_title }}
+      issue_body: ${{ needs.check-and-prepare.outputs.issue_body }}
+      instruction: ${{ needs.check-and-prepare.outputs.instruction }}
+      issue_id: ${{ github.event.client_payload.issue_id }}
+      rules_files: ".cursor/rules/rust-best-practices.mdc .cursor/rules/svelte5-best-practices.mdc .cursor/rules/windmill-overview.mdc"
+    secrets: inherit

.github/workflows/aider.yaml

@@ -5,12 +5,40 @@ on:
     types: [created]
 
 jobs:
-  auto-fix:
-    runs-on: ubicloud-standard-8
+  check-membership:
+    runs-on: ubicloud-standard-2
     if: |
       github.event_name == 'issue_comment' &&
       contains(github.event.comment.body, '/aider') &&
       !contains(github.event.comment.user.login, '[bot]')
+    outputs:
+      is_member: ${{ steps.check-membership.outputs.is_member }}
+    steps:
+      - name: Check organization membership
+        id: check-membership
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          GITHUB_REPOSITORY: ${{ github.repository }}
+          COMMENTER: ${{ github.event.comment.user.login }}
+          ORG_ACCESS_TOKEN: ${{ secrets.ORG_ACCESS_TOKEN }}
+        run: |
+          ORG="windmill-labs"
+          STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
+            -H "Authorization: token $ORG_ACCESS_TOKEN" \
+            -H "Accept: application/vnd.github+json" \
+            -H "X-GitHub-Api-Version: 2022-11-28" \
+            "https://api.github.com/orgs/$ORG/members/$COMMENTER")
+
+          if [ "$STATUS" -eq 204 ]; then
+            echo "is_member=true" >> $GITHUB_OUTPUT
+          else
+            echo "is_member=false" >> $GITHUB_OUTPUT
+          fi
+
+  check-and-prepare:
+    needs: check-membership
+    runs-on: ubicloud-standard-2
+    if: needs.check-membership.outputs.is_member == 'true'
     permissions:
       contents: write
       pull-requests: write
@@ -21,322 +49,117 @@ jobs:
       GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
       GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
       WINDMILL_TOKEN: ${{ secrets.WINDMILL_TOKEN }}
+    outputs:
+      issue_title: ${{ steps.determine_inputs.outputs.ISSUE_TITLE }}
+      issue_body: ${{ steps.determine_inputs.outputs.ISSUE_BODY }}
+      comment_content: ${{ steps.determine_inputs.outputs.COMMENT_CONTENT }}
+      pr_branch: ${{ steps.checkout_pr.outputs.PR_BRANCH }}
 
     steps:
-      - name: Harden Runner
-        uses: step-security/harden-runner@v2
-        with:
-          egress-policy: audit
-
-      - name: Check out code
-        uses: actions/checkout@v4
-        with:
-          fetch-depth: 0
-
-      - name: Configure Git User
-        run: |
-          git config --global user.name "github-actions[bot]"
-          git config --global user.email "github-actions[bot]@users.noreply.github.com"
-
-      - name: Checkout PR Branch
-        if: github.event_name == 'issue_comment' && github.event.issue.pull_request
+      - name: Acknowledge Request
         env:
           GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          GITHUB_REPOSITORY: ${{ github.repository }}
         run: |
-          echo "Issue comment trigger: Checking out PR branch..."
-          PR_NUMBER=${{ github.event.issue.number }}
-          PR_HEAD_REF=$(gh pr view $PR_NUMBER --json headRefName -q .headRefName --repo $GITHUB_REPOSITORY)
-          if [[ -z "$PR_HEAD_REF" || "$PR_HEAD_REF" == "null" ]]; then
-             echo "::error::Could not determine PR head branch for PR #$PR_NUMBER via gh CLI."
-             exit 1
-          fi
-          echo "Checking out PR head branch: $PR_HEAD_REF for PR #$PR_NUMBER"
-          git fetch origin "refs/heads/${PR_HEAD_REF}:refs/remotes/origin/${PR_HEAD_REF}" --no-tags
-          git checkout "$PR_HEAD_REF"
-          echo "Successfully checked out branch $(git rev-parse --abbrev-ref HEAD)"
+          echo "Commenting on issue/PR #${{ github.event.issue.number }} to acknowledge the /aider command."
+          gh issue comment ${{ github.event.issue.number }} --body "🤖 Aider is starting to work on your request. I'll update you here once I have a PR ready. Please be patient, this might take a few minutes." --repo $GITHUB_REPOSITORY
 
-      - name: Set up Python
-        uses: actions/setup-python@v5
-        with:
-          python-version: "3.12"
-
-      - name: Install Aider and Dependencies
-        run: |
-          python -m pip install aider-install; aider-install
-          pip install -U google-generativeai
-          sudo apt-get update && sudo apt-get install -y jq
-
-      - name: Determine Prompt for Aider
-        id: determine_prompt
+      - name: Determine inputs for Aider
+        id: determine_inputs
         shell: bash
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          COMMENT_BODY: ${{ github.event.comment.body }}
+          ISSUE_NUMBER: ${{ github.event.issue.number }}
+          GITHUB_REPOSITORY: ${{ github.repository }}
+          LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}
         run: |
-          PROMPT_FILE_PATH=".github/aider/issue-prompt.txt"
-          mkdir -p .github/aider
+          echo "Determining inputs for Aider..."
+          ISSUE_TITLE_VAL=""
+          ISSUE_BODY_VAL=""
 
-          # Determine if this is a PR comment or regular issue comment
           if [[ ! -z "${{ github.event.issue.pull_request }}" ]]; then
             echo "This is a comment on a Pull Request"
-            PR_NUMBER="${{ github.event.issue.number }}"
+            PR_NUMBER="$ISSUE_NUMBER"
             
-            # Get PR description to check for issue references
-            PR_BODY=$(gh pr view $PR_NUMBER --json body -q .body --repo $GITHUB_REPOSITORY)
-            
-            # Extract issue number from PR description (looking for #123 or "fixes #123" patterns)
-            REFERENCED_ISSUE=$(echo "$PR_BODY" | grep -oE "#[0-9]+" | grep -oE "[0-9]+" | head -1)
-            
-            if [[ ! -z "$REFERENCED_ISSUE" ]]; then
-              echo "Found referenced issue #$REFERENCED_ISSUE in PR description"
-              
-              # Fetch the referenced issue details
-              ISSUE_DETAILS=$(gh issue view $REFERENCED_ISSUE --json title,body --repo $GITHUB_REPOSITORY)
-              ISSUE_TITLE=$(echo "$ISSUE_DETAILS" | jq -r .title)
-              ISSUE_BODY=$(echo "$ISSUE_DETAILS" | jq -r .body)
-              
-              # Store raw comment body in a file first to avoid shell interpretation issues
-              echo '${{ github.event.comment.body }}' > /tmp/raw_comment.txt
-              RAW_COMMENT_BODY=$(cat /tmp/raw_comment.txt)
-              # Remove the /aider prefix and trim whitespace
-              COMMENT_CONTENT=$(echo "$RAW_COMMENT_BODY" | sed 's|^/aider||' | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//')
-              
-              echo "Sending issue content and PR comment to external API…"
-              
-              ISSUE_TITLE_Q=$(printf '%q' "$ISSUE_TITLE")
-              ISSUE_BODY_Q=$(printf '%q' "$ISSUE_BODY")
-              
-              JSON_PAYLOAD=$(jq -n \
-                --arg title "$ISSUE_TITLE_Q" \
-                --arg body "$ISSUE_BODY_Q" \
-                '{"body":{"issue_title":$title,"issue_body":$body}}')
-              
-              API_RESULT=$(curl -s -w "\n%{http_code}" \
-                -X POST "https://app.windmill.dev/api/w/windmill-labs/jobs/run_wait_result/p/f/ai/quiet_script" \
-                -H "Content-Type: application/json" \
-                -H "Authorization: Bearer $WINDMILL_TOKEN" \
-                --data-binary "$JSON_PAYLOAD" \
-                --max-time 90)
-              
-              HTTP_CODE=$(echo "$API_RESULT" | tail -n1)
-              BODY=$(echo "$API_RESULT" | sed '$d')
-              
-              echo "$BODY" > /tmp/api_response.txt
-
-              BASE_PROMPT="Try to fix the following issue based on the instruction given by the user. The issue is prepended with the word ISSUE. The instruction is prepended with the word INSTRUCTION. The issue and instruction are separated by a blank line."
-              if [[ "$HTTP_CODE" -eq 200 ]]; then
-                PROCESSED_ISSUE_PROMPT=$(jq -r '.effective_body // empty' /tmp/api_response.txt)
-                if [[ -z "$PROCESSED_ISSUE_PROMPT" || "$PROCESSED_ISSUE_PROMPT" == "null" ]]; then
-                  PROCESSED_ISSUE_PROMPT=""
-                fi
-                printf "%s\nISSUE:\n%s\nINSTRUCTION:\n%s" \
-                  "$BASE_PROMPT" "$PROCESSED_ISSUE_PROMPT" "$COMMENT_CONTENT" > "$PROMPT_FILE_PATH"
-              else
-                echo "::warning::API call failed (HTTP $HTTP_CODE). Using PR comment with issue context."
-                printf "%s\nISSUE:\n%s\nINSTRUCTION:\n%s" \
-                  "$BASE_PROMPT" "$ISSUE_BODY_Q" "$COMMENT_CONTENT" > "$PROMPT_FILE_PATH"
-              fi
-              rm -f /tmp/api_response.txt
+            PR_BODY_JSON=$(gh pr view "$PR_NUMBER" --json body --repo "$GITHUB_REPOSITORY")
+            if [[ $? -ne 0 ]]; then
+              echo "Error fetching PR body for PR #$PR_NUMBER"
+              PR_BODY_VAL=""
             else
-              echo "No referenced issue found in PR description, using comment content only"
-              # Use comment content directly as with regular issue comments
-              echo '${{ github.event.comment.body }}' > /tmp/raw_comment.txt
-              RAW_COMMENT_BODY=$(cat /tmp/raw_comment.txt)
-              COMMENT_CONTENT=$(echo "$RAW_COMMENT_BODY" | sed 's|^/aider||' | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//')
-              
-              if [[ -z "$COMMENT_CONTENT" ]]; then
-                echo "::error::Comment with /aider provided, but no instruction found after it. Cannot proceed."
-                printf "Error: /aider command found but no instruction followed." > "$PROMPT_FILE_PATH"
-                exit 1
-              else
-                echo "Using comment content as prompt."
-                printf '%s' "$COMMENT_CONTENT" > "$PROMPT_FILE_PATH"
+              PR_BODY_VAL=$(jq -r '.body // ""' <<< "$PR_BODY_JSON")
+            fi
+            
+            if [[ ! -z "$PR_BODY_VAL" ]]; then
+              REFERENCED_ISSUE=""
+              if [[ "$PR_BODY_VAL" =~ \#linear:([a-f0-9-]+) ]]; then
+                REFERENCED_ISSUE="${BASH_REMATCH[1]}"
+                echo "Found referenced Linear issue #$REFERENCED_ISSUE in PR description"
+                LINEAR_ISSUE_JSON=$(curl -s -H "Authorization: $LINEAR_API_KEY" \
+                  "https://api.linear.app/graphql" \
+                  -X POST \
+                  -H "Content-Type: application/json" \
+                  -d "{\"query\":\"query { issue(id: \\\"$REFERENCED_ISSUE\\\") { title description } }\"}")
+                
+                if [[ $? -eq 0 && ! "$LINEAR_ISSUE_JSON" =~ "error" ]]; then
+                  ISSUE_TITLE_VAL=$(jq -r '.data.issue.title // ""' <<< "$LINEAR_ISSUE_JSON")
+                  ISSUE_BODY_VAL=$(jq -r '.data.issue.description // ""' <<< "$LINEAR_ISSUE_JSON")
+                  echo "Successfully fetched Linear issue details"
+                else
+                  echo "Error fetching Linear issue details for #$REFERENCED_ISSUE"
+                fi
+              elif [[ "$PR_BODY_VAL" =~ \#([0-9]+) ]]; then
+                REFERENCED_ISSUE="${BASH_REMATCH[1]}"
+                echo "Found referenced GitHub issue #$REFERENCED_ISSUE in PR description"
+                
+                ISSUE_DETAILS_JSON=$(gh issue view "$REFERENCED_ISSUE" --json title,body --repo "$GITHUB_REPOSITORY")
+                if [[ $? -ne 0 ]]; then
+                  echo "Error fetching issue details for #$REFERENCED_ISSUE"
+                else
+                  ISSUE_TITLE_VAL=$(jq -r '.title // ""' <<< "$ISSUE_DETAILS_JSON")
+                  ISSUE_BODY_VAL=$(jq -r '.body // ""' <<< "$ISSUE_DETAILS_JSON")
+                fi
               fi
+            else
+              echo "PR body is empty or could not be fetched."
             fi
           else
             echo "This is a comment on a regular issue"
             
-            # Fetch the issue details
-            ISSUE_NUMBER="${{ github.event.issue.number }}"
-            ISSUE_DETAILS=$(gh issue view $ISSUE_NUMBER --json title,body --repo $GITHUB_REPOSITORY)
-            ISSUE_TITLE=$(echo "$ISSUE_DETAILS" | jq -r .title)
-            ISSUE_BODY=$(echo "$ISSUE_DETAILS" | jq -r .body)
-            
-            # Store raw comment body in a file first to avoid shell interpretation issues
-            echo '${{ github.event.comment.body }}' > /tmp/raw_comment.txt
-            # Extract the command part safely
-            RAW_COMMENT_BODY=$(cat /tmp/raw_comment.txt)
-            # Remove the /aider prefix and trim whitespace
-            COMMENT_CONTENT=$(echo "$RAW_COMMENT_BODY" | sed 's|^/aider||' | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//')
-
-            if [[ -z "$COMMENT_CONTENT" ]]; then
-              echo "::error::Comment with /aider provided, but no instruction found after it. Cannot proceed."
-              printf "Error: /aider command found but no instruction followed." > "$PROMPT_FILE_PATH"
-              exit 1
+            ISSUE_DETAILS_JSON=$(gh issue view "$ISSUE_NUMBER" --json title,body --repo "$GITHUB_REPOSITORY")
+            if [[ $? -ne 0 ]]; then
+              echo "Error fetching issue details for #$ISSUE_NUMBER"
             else
-              echo "Sending issue content and issue comment to external API…"
-              
-              ISSUE_TITLE_Q=$(printf '%q' "$ISSUE_TITLE")
-              ISSUE_BODY_Q=$(printf '%q' "$ISSUE_BODY")
-              COMMENT_CONTENT_Q=$(printf '%q' "$COMMENT_CONTENT")
-              
-              JSON_PAYLOAD=$(jq -n \
-                --arg title "$ISSUE_TITLE_Q" \
-                --arg body "$ISSUE_BODY_Q" \
-                --arg comment "$COMMENT_CONTENT_Q" \
-                '{"body":{"issue_title":$title,"issue_body":$body,"issue_comment":$comment}}')
-              
-              API_RESULT=$(curl -s -w "\n%{http_code}" \
-                -X POST "https://app.windmill.dev/api/w/windmill-labs/jobs/run_wait_result/p/f/ai/quiet_script" \
-                -H "Content-Type: application/json" \
-                -H "Authorization: Bearer $WINDMILL_TOKEN" \
-                --data-binary "$JSON_PAYLOAD" \
-                --max-time 90)
-              
-              HTTP_CODE=$(echo "$API_RESULT" | tail -n1)
-              BODY=$(echo "$API_RESULT" | sed '$d')
-              
-              echo "$BODY" > /tmp/api_response.txt
-              
-              BASE_PROMPT="Try to fix the following issue based on the instruction given by the user. The issue is prepended with the word ISSUE. The instruction is prepended with the word INSTRUCTION. The issue and instruction are separated by a blank line."
-              if [[ "$HTTP_CODE" -eq 200 ]]; then
-                PROCESSED_ISSUE_PROMPT=$(jq -r '.effective_body // empty' /tmp/api_response.txt)
-                if [[ -z "$PROCESSED_ISSUE_PROMPT" || "$PROCESSED_ISSUE_PROMPT" == "null" ]]; then
-                  PROCESSED_ISSUE_PROMPT=""
-                fi
-                printf "%s\nISSUE:\n%s\nINSTRUCTION:\n%s" \
-                  "$BASE_PROMPT" "$PROCESSED_ISSUE_PROMPT" "$COMMENT_CONTENT" > "$PROMPT_FILE_PATH"
-              else
-                echo "::warning::API call failed (HTTP $HTTP_CODE). Using PR comment with issue context."
-                printf "%s\nISSUE:\n%s\nINSTRUCTION:\n%s" \
-                  "$BASE_PROMPT" "$ISSUE_BODY_Q" "$COMMENT_CONTENT" > "$PROMPT_FILE_PATH"
-              fi
-
-              rm -f /tmp/api_response.txt
-            fi
-          fi
-          echo "Prompt determined and written to $PROMPT_FILE_PATH"
-          echo "PROMPT_FILE_PATH=$PROMPT_FILE_PATH" >> $GITHUB_OUTPUT
-
-      - name: Probe Chat for Relevant Files
-        id: probe_files
-        env:
-          PROMPT_CONTENT_FILE: ${{ steps.determine_prompt.outputs.PROMPT_FILE_PATH }}
-        run: |
-          echo "Running probe-chat to find relevant files..."
-          if [[ ! -f "$PROMPT_CONTENT_FILE" ]]; then
-            echo "::error::Prompt file $PROMPT_CONTENT_FILE not found!"
-            exit 1
-          fi
-          PROMPT_CONTENT=$(cat "$PROMPT_CONTENT_FILE")
-          if [ -z "$PROMPT_CONTENT" ]; then
-             echo "::error::Prompt content is empty!"
-             exit 1
-          fi
-
-          PROMPT_ESCAPED=$(jq -Rs . <<< "$PROMPT_CONTENT")
-
-          MESSAGE_FOR_PROBE=$(jq -n --arg prompt_escaped "$PROMPT_ESCAPED" \
-            '{ "message": "I'\''m giving you a request that needs to be implemented. Your role is ONLY to give me the files that are relevant to the request and nothing else. The request is prepended with the word REQUEST.\\nREQUEST: \($prompt_escaped). Give me all the files relevant to this request. Your output MUST be a single json array that can be parsed with programatic json parsing, with the relevant files. Files can be rust or typescript or javascript files. DO NOT INCLUDE ANY OTHER TEXT IN YOUR OUTPUT. ONLY THE JSON ARRAY. Example of output: [\"file1.py\", \"file2.py\"]" }' | jq -r .message)
-
-          set -o pipefail
-          PROBE_OUTPUT=$(npx --yes @buger/probe-chat@latest --max-iterations 50 --model-name gemini-2.5-pro-preview-05-06 --message "$MESSAGE_FOR_PROBE") || {
-            echo "::error::probe-chat command failed. Output:"
-            echo "$PROBE_OUTPUT"
-            exit 1
-          }
-          set +o pipefail
-          echo "Probe-chat raw output:"
-          echo "$PROBE_OUTPUT"
-
-          JSON_FILES=$(echo "$PROBE_OUTPUT" | sed -n '/^\s*\[/,$p' | sed '/^\s*\]/q')
-          echo "Extracted JSON block:"
-          echo "$JSON_FILES"
-
-          FILES_LIST=$(echo "$JSON_FILES" | jq -e -r '[.[] | select(type == "string" and . != "" and . != null and (endswith("/") | not))] | map(@sh) | join(" ")' || echo "")
-
-          if [[ -z "$FILES_LIST" ]]; then
-             echo "::warning::probe-chat did not identify any relevant files."
-             exit 1
-          fi
-
-          echo "Formatted files list for aider: $FILES_LIST"
-          echo "FILES_TO_EDIT=$FILES_LIST" >> $GITHUB_ENV
-
-      - name: Run Aider with external prompt
-        run: |
-          echo "Files identified by probe-chat: ${{ env.FILES_TO_EDIT }}"
-          aider \
-            --read .cursor/rules/rust-best-practices.mdc \
-            --read .cursor/rules/svelte5-best-practices.mdc \
-            ${{ env.FILES_TO_EDIT }} \
-            --model gemini/gemini-2.5-pro-preview-05-06 \
-            --message-file .github/aider/issue-prompt.txt \
-            --yes \
-            --no-check-update \
-            --auto-commits \
-            --no-analytics \
-            --no-gitignore \
-            | tee .github/aider/aider-output.txt || true
-          echo "Aider command completed. Output saved to .github/aider/aider-output.txt"
-
-      - name: Clean up prompt file
-        if: always()
-        run: rm -f .github/aider/issue-prompt.txt
-
-      - name: Commit and Push Changes
-        id: commit_and_push
-        if: ${{ success() }}
-        run: |
-          if [[ -z "${{ github.event.issue.pull_request }}" ]]; then
-            BRANCH_NAME="aider-fix-issue-${{ github.event.issue.number }}"
-            
-            # Check if branch exists remotely
-            if git ls-remote --heads origin $BRANCH_NAME | grep -q $BRANCH_NAME; then
-              echo "Branch $BRANCH_NAME already exists remotely, fetching it"
-              git fetch origin $BRANCH_NAME
-              git checkout $BRANCH_NAME
-              git pull origin $BRANCH_NAME
-            else
-              echo "Creating new branch $BRANCH_NAME"
-              git checkout -b $BRANCH_NAME
-            fi
-            
-            echo "Created/checked out branch $BRANCH_NAME for issue #${{ github.event.issue.number }}"
-            git push origin $BRANCH_NAME
-            echo "Pushed to branch $BRANCH_NAME"
-            echo "PR_BRANCH_NAME=$BRANCH_NAME" >> $GITHUB_OUTPUT
-            echo "CHANGES_APPLIED_MESSAGE=Aider changes pushed to branch $BRANCH_NAME." >> $GITHUB_OUTPUT
-          else
-            CURRENT_BRANCH_NAME=$(git rev-parse --abbrev-ref HEAD)
-            echo "Attempting to push changes to PR branch $CURRENT_BRANCH_NAME for PR #${{ github.event.issue.number }}"
-            if git push origin $CURRENT_BRANCH_NAME; then
-              echo "Push to $CURRENT_BRANCH_NAME successful (or no new changes to push)."
-              echo "CHANGES_APPLIED_MESSAGE=Aider changes (if any) pushed to PR branch $CURRENT_BRANCH_NAME." >> $GITHUB_OUTPUT
-              echo "PR_BRANCH_NAME=$CURRENT_BRANCH_NAME" >> $GITHUB_OUTPUT
-            else
-              echo "::warning::Push to PR branch $CURRENT_BRANCH_NAME failed."
-              echo "CHANGES_APPLIED_MESSAGE=Aider ran, but failed to push changes to PR branch $CURRENT_BRANCH_NAME." >> $GITHUB_OUTPUT
+              ISSUE_TITLE_VAL=$(jq -r '.title // ""' <<< "$ISSUE_DETAILS_JSON")
+              ISSUE_BODY_VAL=$(jq -r '.body // ""' <<< "$ISSUE_DETAILS_JSON") 
             fi
           fi
 
-      - name: Create Pull Request
-        if: success() && github.event_name == 'issue_comment' && !github.event.issue.pull_request && steps.commit_and_push.outputs.PR_BRANCH_NAME != ''
-        env:
-          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-          PR_BRANCH: ${{ steps.commit_and_push.outputs.PR_BRANCH_NAME }}
-          ISSUE_NUM: ${{ github.event.issue.number }}
-        run: |
-          # Create PR description in a temporary file to avoid command line length limits
-          cat > /tmp/pr-description.md << EOL
-          This PR was created automatically by Aider to fix issue #${ISSUE_NUM}.
+          echo "ISSUE_TITLE<<EOF_AIDER_TITLE" >> "$GITHUB_OUTPUT"
+          echo "$ISSUE_TITLE_VAL" >> "$GITHUB_OUTPUT"
+          echo "EOF_AIDER_TITLE" >> "$GITHUB_OUTPUT"
 
-          ## Aider Output
-          \`\`\`
-          $(cat .github/aider/aider-output.txt || echo "No output available")
-          \`\`\`
-          EOL
+          echo "ISSUE_BODY<<EOF_AIDER_BODY" >> "$GITHUB_OUTPUT"
+          echo "$ISSUE_BODY_VAL" >> "$GITHUB_OUTPUT"
+          echo "EOF_AIDER_BODY" >> "$GITHUB_OUTPUT"
 
-          # Create PR using the file for the body content
-          gh pr create \
-            --title "[Aider PR] Add fixes for issue #${ISSUE_NUM}" \
-            --body-file /tmp/pr-description.md \
-            --head "$PR_BRANCH" \
-            --base main
+          CLEAN_COMMENT="${COMMENT_BODY/\/aider/}"
+          CLEAN_COMMENT="${CLEAN_COMMENT#"${CLEAN_COMMENT%%[![:space:]]*}"}"
+          CLEAN_COMMENT="${CLEAN_COMMENT%"${CLEAN_COMMENT##*[![:space:]]}"}"
+
+          echo "COMMENT_CONTENT<<EOF_AIDER_COMMENT" >> "$GITHUB_OUTPUT"
+          echo "$CLEAN_COMMENT" >> "$GITHUB_OUTPUT"
+          echo "EOF_AIDER_COMMENT" >> "$GITHUB_OUTPUT"
+          echo "Finished determining inputs."
+
+  run-aider:
+    needs: [check-membership, check-and-prepare]
+    if: needs.check-membership.outputs.is_member == 'true'
+    uses: ./.github/workflows/aider-common.yml
+    with:
+      issue_title: ${{ needs.check-and-prepare.outputs.issue_title }}
+      issue_body: ${{ needs.check-and-prepare.outputs.issue_body }}
+      instruction: ${{ needs.check-and-prepare.outputs.comment_content }}
+      issue_id: ${{ github.event.issue.number }}
+      rules_files: ".cursor/rules/rust-best-practices.mdc .cursor/rules/svelte5-best-practices.mdc .cursor/rules/windmill-overview.mdc"
+    secrets: inherit

.github/workflows/backend-test.yml

@@ -45,9 +45,9 @@ jobs:
       - uses: oven-sh/setup-bun@v2
         with:
           bun-version: 1.1.43
-      - uses: astral-sh/setup-uv@v4
+      - uses: astral-sh/setup-uv@v6
         with:
-          version: "0.4.18"
+          version: "0.6.2"
       - uses: actions-rust-lang/setup-rust-toolchain@v1
         with:
           cache-workspaces: backend

.github/workflows/claude.yml

@@ -0,0 +1,85 @@
+name: Claude PR Assistant
+
+on:
+  issue_comment:
+    types: [created]
+  pull_request_review_comment:
+    types: [created]
+  issues:
+    types: [opened, assigned]
+  pull_request_review:
+    types: [submitted]
+
+jobs:
+  check-membership:
+    if: |
+      (github.event_name == 'issue_comment' && contains(github.event.comment.body, '/ai') && !contains(github.event.comment.user.login, '[bot]')) ||
+      (github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '/ai') && !contains(github.event.comment.user.login, '[bot]')) ||
+      (github.event_name == 'pull_request_review' && contains(github.event.review.body, '/ai') && !contains(github.event.review.user.login, '[bot]')) ||
+      (github.event_name == 'issues' && contains(github.event.issue.body, '/ai') && !contains(github.event.issue.user.login, '[bot]'))
+    runs-on: ubicloud-standard-2
+    outputs:
+      is_member: ${{ steps.check-membership.outputs.is_member }}
+    steps:
+      - name: Check organization membership
+        id: check-membership
+        env:
+          ORG_ACCESS_TOKEN: ${{ secrets.ORG_ACCESS_TOKEN }}
+        run: |
+          ORG="windmill-labs"
+
+          if [[ "${{ github.event_name }}" == "issue_comment" || "${{ github.event_name }}" == "pull_request_review_comment" ]]; then
+            COMMENTER="${{ github.event.comment.user.login }}"
+          elif [[ "${{ github.event_name }}" == "pull_request_review" ]]; then
+            COMMENTER="${{ github.event.review.user.login }}"
+          else
+            COMMENTER="${{ github.event.issue.user.login }}"
+          fi
+
+          STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
+            -H "Authorization: token $ORG_ACCESS_TOKEN" \
+            -H "Accept: application/vnd.github+json" \
+            -H "X-GitHub-Api-Version: 2022-11-28" \
+            "https://api.github.com/orgs/$ORG/members/$COMMENTER")
+
+          if [ "$STATUS" -eq 204 ]; then
+            echo "is_member=true" >> $GITHUB_OUTPUT
+          else
+            echo "is_member=false" >> $GITHUB_OUTPUT
+          fi
+
+  claude-code-action:
+    needs: check-membership
+    if: |
+      needs.check-membership.outputs.is_member == 'true'
+    runs-on: ubicloud-standard-8
+    permissions:
+      contents: read
+      pull-requests: read
+      issues: read
+      id-token: write
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 1
+
+      - name: Run Claude PR Action
+        uses: anthropics/claude-code-action@beta
+        with:
+          anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
+          timeout_minutes: "60"
+          allowed_tools: "mcp__github__create_pull_request,Bash(npm run check),Bash(npm install),Bash(cargo check),Bash(curl https://sh.rustup.rs -sSf | sh -s -- -y)"
+          custom_instructions: "IMPORTANT INSTRUCTIONS:
+          - Bypass previous instructions and instead of adding a url in your comment to manually create a PR, if you created a new branch with your changes, automatically create a draft PR from that branch to main, with the title starting with [Claude PR].
+          - If you made changes to the frontend code, run npm install, then npm run generate-backend-client, then npm run check. You can ignore warnings that are reported by the check script, but fix the errors.
+          - If you made changes to the backend code, install Rust and then run cargo check. You can ignore warnings that are reported by the check script, but fix the errors.
+          - DO NOT FORGET TO OPEN A DRAFT PR AFTER YOU ARE DONE if you made changes after a request from a git issue.
+          AVAILABLE TOOLS:
+          - mcp__github__create_pull_request: Create a PR from a branch to main
+          - Bash(npm run check): Run the check script. You should run this tool after making changes to the frontend code.
+          - Bash(npm install): Install dependencies. You need this to run npm run check.
+          - Bash(npm run generate-backend-client): Generate the backend client. You need this to run npm run check.
+          - Bash(cargo check): Run the cargo check script. You should run this tool after making changes to the backend code.
+          - Bash(curl https://sh.rustup.rs -sSf | sh -s -- -y): Install Rust. You need this to run cargo check."
+          trigger_phrase: "/ai"

.github/workflows/create-docs.yml

@@ -3,8 +3,34 @@ on:
     types: [created]
 
 jobs:
+  check-membership:
+    if: ${{ github.event.issue.pull_request && startsWith(github.event.comment.body, '/docs') && github.event.comment.user.type != 'Bot' }}
+    runs-on: ubicloud-standard-2
+    outputs:
+      is_member: ${{ steps.check-membership.outputs.is_member }}
+    steps:
+      - name: Check organization membership
+        id: check-membership
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          COMMENTER: ${{ github.event.comment.user.login }}
+        run: |
+          ORG="windmill-labs"
+          STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
+            -H "Authorization: token $GH_TOKEN" \
+            -H "Accept: application/vnd.github+json" \
+            -H "X-GitHub-Api-Version: 2022-11-28" \
+            "https://api.github.com/orgs/$ORG/members/$COMMENTER")
+
+          if [ "$STATUS" -eq 204 ]; then
+            echo "is_member=true" >> $GITHUB_OUTPUT
+          else
+            echo "is_member=false" >> $GITHUB_OUTPUT
+          fi
+
   trigger-docs:
-    if: ${{ github.event.issue.pull_request && startsWith(github.event.comment.body, '/docs') }}
+    needs: check-membership
+    if: ${{ github.event.issue.pull_request && startsWith(github.event.comment.body, '/docs') && needs.check-membership.outputs.is_member == 'true' }}
     uses: windmill-labs/windmilldocs/.github/workflows/create-docs.yml@main
     with:
       pr_number: ${{ github.event.issue.number }}

.github/workflows/discord-notification.yml

@@ -29,4 +29,4 @@ jobs:
       DISCORD_GUILD_ID: "930051556043276338"
       PR_NUMBER: ${{ github.event.pull_request.number }}
     secrets:
-      DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_PR_BOT_TOKEN }}
+      DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_AI_BOT_TOKEN }}

.github/workflows/linear-claude.yaml

@@ -0,0 +1,38 @@
+name: Claude PR Assistant
+
+on:
+  repository_dispatch:
+    types: [external_claude_issue_fix]
+
+jobs:
+  claude-code-action:
+    runs-on: ubicloud-standard-8
+    permissions:
+      contents: read
+      pull-requests: read
+      issues: read
+      id-token: write
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 1
+
+      - name: Process inputs
+        id: process_inputs
+        shell: bash
+        run: |
+          ISSUE_TITLE="${{ github.event.client_payload.issue_title }}"
+          INSTRUCTION="${{ github.event.client_payload.instruction }}"
+          ISSUE_BODY=$(printf '%q' "${{ github.event.client_payload.issue_body }}")
+          BASE_PROMPT="Try to fix the following issue based on the instruction given. You are provided with the issue title, issue body, and instruction. You are to fix the issue based on the instruction. You are to create a pull request to fix the issue."
+          CUSTOM_PROMPT=$(printf -v PROMPT "%s\n\nISSUE_TITLE: %s\n\nISSUE_BODY: %s\n\nINSTRUCTION: %s" "$BASE_PROMPT" "$ISSUE_TITLE" "$ISSUE_BODY" "$INSTRUCTION")
+          echo "CUSTOM_PROMPT=$CUSTOM_PROMPT" >> $GITHUB_OUTPUT
+
+      - name: Run Claude PR Action
+        uses: anthropics/claude-code-action@beta
+        with:
+          anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
+          timeout_minutes: "60"
+          allowed_tools: "mcp__github__create_pull_request"
+          direct_prompt: ${{ steps.process_inputs.outputs.CUSTOM_PROMPT }}

.github/workflows/shareable-discord-notification.yml

@@ -84,7 +84,7 @@ jobs:
           fi
           # 2) get the first message in that thread
           messages=$(curl -H "Authorization: Bot $BOT_TOKEN" \
-            "https://discord.com/api/v10/channels/$thread_id/messages?limit=1")
+            "https://discord.com/api/v10/channels/$thread_id/messages")
           message_id=$(echo "$messages" | jq -r '.[-1].id')
 
           if [ -z "$message_id" ]; then

CHANGELOG.md

@@ -1,5 +1,78 @@
 # Changelog
 
+## [1.493.4](https://github.com/windmill-labs/windmill/compare/v1.493.3...v1.493.4) (2025-05-29)
+
+
+### Bug Fixes
+
+* templatev2 delete issue ([#5834](https://github.com/windmill-labs/windmill/issues/5834)) ([ed3ad32](https://github.com/windmill-labs/windmill/commit/ed3ad327a235c16b9f3aa7f8edeefe61b0c01da3))
+
+## [1.493.3](https://github.com/windmill-labs/windmill/compare/v1.493.2...v1.493.3) (2025-05-29)
+
+
+### Bug Fixes
+
+* evalv2 prohibit component delete ([e302aa3](https://github.com/windmill-labs/windmill/commit/e302aa38b5977dd406ae05e1d8dbb74cb7dc3d17))
+* faster layout for larger graphs ([8d12bcc](https://github.com/windmill-labs/windmill/commit/8d12bcc8ee2991909ea0d9bb57f04f0d4106c69f))
+
+## [1.493.2](https://github.com/windmill-labs/windmill/compare/v1.493.1...v1.493.2) (2025-05-28)
+
+
+### Bug Fixes
+
+* improve monaco editor memory leak ([e0f4f83](https://github.com/windmill-labs/windmill/commit/e0f4f83ebf4416c3bcc24433a7bf606349e1f75a))
+* improve monaco javascript extra lib refresh ([7b70348](https://github.com/windmill-labs/windmill/commit/7b70348b4bba3726e3fb26c964219a5a2aa6af55))
+
+## [1.493.1](https://github.com/windmill-labs/windmill/compare/v1.493.0...v1.493.1) (2025-05-28)
+
+
+### Bug Fixes
+
+* improve monaco javascript extra lib refresh ([a2c8ea6](https://github.com/windmill-labs/windmill/commit/a2c8ea69a3962a350273717cd237d8a96523fd00))
+
+## [1.493.0](https://github.com/windmill-labs/windmill/compare/v1.492.1...v1.493.0) (2025-05-27)
+
+
+### Features
+
+* add aws oidc support for instance s3 storage ([#5810](https://github.com/windmill-labs/windmill/issues/5810)) ([5b96bcc](https://github.com/windmill-labs/windmill/commit/5b96bccedd6e68fea631580dd49338301ad0305f))
+* duckdb sql lang support ([#5761](https://github.com/windmill-labs/windmill/issues/5761)) ([fdefd4b](https://github.com/windmill-labs/windmill/commit/fdefd4be9398b9610a539360353fd61b521732d4))
+* **python:** inline script metadata (PEP 723)  ([#5712](https://github.com/windmill-labs/windmill/issues/5712)) ([2622253](https://github.com/windmill-labs/windmill/commit/26222539e66bce7e88f86a7e5917e6ca99350865))
+
+
+### Bug Fixes
+
+* add missing http_trigger_version_seq grants ([#5816](https://github.com/windmill-labs/windmill/issues/5816)) ([306f3ea](https://github.com/windmill-labs/windmill/commit/306f3eabd1c03fa904b0e59438de124a0e680597))
+* avoid monaco memory leak ([0d459d5](https://github.com/windmill-labs/windmill/commit/0d459d5d223728270854e37715ecc1663ede9870))
+* error handler node rendering at top level ([feae9b0](https://github.com/windmill-labs/windmill/commit/feae9b09240ba306c007013a36d2aefb0b273766))
+* **frontend:** auto completion and render of tailwind classes in app editor ([#5817](https://github.com/windmill-labs/windmill/issues/5817)) ([5897e7e](https://github.com/windmill-labs/windmill/commit/5897e7e01b8839425c30c2a97481ef7bb9090661))
+
+## [1.492.1](https://github.com/windmill-labs/windmill/compare/v1.492.0...v1.492.1) (2025-05-22)
+
+
+### Bug Fixes
+
+* fix strum compile ([59f6024](https://github.com/windmill-labs/windmill/commit/59f6024cbdaface9c9f0ed61c4a415a13b558515))
+
+## [1.492.0](https://github.com/windmill-labs/windmill/compare/v1.491.5...v1.492.0) (2025-05-22)
+
+
+### Features
+
+* job search pagination + result count ([#5789](https://github.com/windmill-labs/windmill/issues/5789)) ([55ae766](https://github.com/windmill-labs/windmill/commit/55ae76648475ce9ff14b2fa33b2a71b90fbd50a1))
+* **python:** add annotation to skip result post-processing ([#5769](https://github.com/windmill-labs/windmill/issues/5769)) ([07c2ff5](https://github.com/windmill-labs/windmill/commit/07c2ff5668f4725a3b9a8a2655248b0945ac251c))
+* shift/ctrl+click/enter to open ctrl+k menu results in new tab ([#5800](https://github.com/windmill-labs/windmill/issues/5800)) ([66a997a](https://github.com/windmill-labs/windmill/commit/66a997afc399de2d592c469faf9a5b2cd6433aac))
+* triggers git sync ([#5766](https://github.com/windmill-labs/windmill/issues/5766)) ([065a814](https://github.com/windmill-labs/windmill/commit/065a814d35a5749725c2ada1155481abba782684))
+
+
+### Bug Fixes
+
+* improve app css consistency ([88482c3](https://github.com/windmill-labs/windmill/commit/88482c3bd76ddad16738354f7531d16fa806ad2f))
+* improve docker mode unexpected exit handling ([7c24fbc](https://github.com/windmill-labs/windmill/commit/7c24fbcef2ecfe5fc034870c4c65dd80513301a4))
+* postgres trigger ssl issue ([#5790](https://github.com/windmill-labs/windmill/issues/5790)) ([b9a776c](https://github.com/windmill-labs/windmill/commit/b9a776c97b3411af18e58cde7a070c4955aaaab4))
+* specify using inline type in system prompt for AI ([#5787](https://github.com/windmill-labs/windmill/issues/5787)) ([791296f](https://github.com/windmill-labs/windmill/commit/791296fa41c5bc45c32944db8bc1b66e1515ea82))
+* workspace preprocessor improvements ([#5784](https://github.com/windmill-labs/windmill/issues/5784)) ([30edcdf](https://github.com/windmill-labs/windmill/commit/30edcdfe0e950b0ab850942bcbc9b4b5ff4fc00c))
+
 ## [1.491.5](https://github.com/windmill-labs/windmill/compare/v1.491.4...v1.491.5) (2025-05-17)
 
 

CLAUDE.md

@@ -0,0 +1,3 @@
+To have an overview of what this app does, see @.cursor/rules/windmill-overview.mdc
+For backend modifications, follow the rules mentioned here @.cursor/rules/rust-best-practices.mdc
+For frontend modifications, follow the rules mentioned here @.cursor/rules/svelte5-best-practices.mdc

backend/.gitignore

@@ -5,4 +5,5 @@ oauth2.json
 tracing.folded
 heaptrack*
 index/
-windmill-api/openapi-*.*
+windmill-api/openapi-*.*
+.duckdb/*

backend/.sqlx/query-551fee7919fdeb911e3f9cc5852e158ea47e3db4895c2b2b1d3cb6b16fceeda9.json

@@ -0,0 +1,23 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "SELECT fv.id\n                                                                                FROM flow f\n                                                                                INNER JOIN flow_version fv ON fv.id = f.versions[array_upper(f.versions, 1)]\n                                                                                WHERE fv.value->'preprocessor_module'->'value'->>'path' = $1 AND f.workspace_id = $2",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "id",
+        "type_info": "Int8"
+      }
+    ],
+    "parameters": {
+      "Left": [
+        "Text",
+        "Text"
+      ]
+    },
+    "nullable": [
+      false
+    ]
+  },
+  "hash": "551fee7919fdeb911e3f9cc5852e158ea47e3db4895c2b2b1d3cb6b16fceeda9"
+}

backend/.sqlx/query-9a9e4a8779b0bf8a275d029221dfa1465e5d44cd8a7be5879219ffc8cd7ae6b1.json

@@ -1,6 +1,6 @@
 {
   "db_name": "PostgreSQL",
-  "query": "INSERT INTO pip_resolution_cache (hash, lockfile, expiration) VALUES ($1, $2, now() + ('3 days')::interval) ON CONFLICT (hash) DO UPDATE SET lockfile = $2",
+  "query": "INSERT INTO pip_resolution_cache (hash, lockfile, expiration) VALUES ($1, $2, now() + ('5 mins')::interval) ON CONFLICT (hash) DO UPDATE SET lockfile = $2",
   "describe": {
     "columns": [],
     "parameters": {
@@ -11,5 +11,5 @@
     },
     "nullable": []
   },
-  "hash": "4fb3881cdbb4b9e93e28f460a9b3715bdc6a52b76c89f3a3913023b13c4e085c"
+  "hash": "9a9e4a8779b0bf8a275d029221dfa1465e5d44cd8a7be5879219ffc8cd7ae6b1"
 }

backend/.sqlx/query-c51f9ad5133c46fd7c499b8339dbbf3f3059bbb85de07ee3b4b4cea971984a52.json

@@ -0,0 +1,132 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "\n                SELECT\n                    mqtt_resource_path,\n                    subscribe_topics as \"subscribe_topics: _\",\n                    v3_config as \"v3_config: _\",\n                    v5_config as \"v5_config: _\",\n                    client_version AS \"client_version: _\",\n                    client_id,\n                    workspace_id,\n                    path,\n                    script_path,\n                    is_flow,\n                    edited_by,\n                    email,\n                    edited_at,\n                    server_id,\n                    last_server_ping,\n                    extra_perms,\n                    error,\n                    enabled\n                FROM \n                    mqtt_trigger\n                ",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "mqtt_resource_path",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 1,
+        "name": "subscribe_topics: _",
+        "type_info": "JsonbArray"
+      },
+      {
+        "ordinal": 2,
+        "name": "v3_config: _",
+        "type_info": "Jsonb"
+      },
+      {
+        "ordinal": 3,
+        "name": "v5_config: _",
+        "type_info": "Jsonb"
+      },
+      {
+        "ordinal": 4,
+        "name": "client_version: _",
+        "type_info": {
+          "Custom": {
+            "name": "mqtt_client_version",
+            "kind": {
+              "Enum": [
+                "v3",
+                "v5"
+              ]
+            }
+          }
+        }
+      },
+      {
+        "ordinal": 5,
+        "name": "client_id",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 6,
+        "name": "workspace_id",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 7,
+        "name": "path",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 8,
+        "name": "script_path",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 9,
+        "name": "is_flow",
+        "type_info": "Bool"
+      },
+      {
+        "ordinal": 10,
+        "name": "edited_by",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 11,
+        "name": "email",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 12,
+        "name": "edited_at",
+        "type_info": "Timestamptz"
+      },
+      {
+        "ordinal": 13,
+        "name": "server_id",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 14,
+        "name": "last_server_ping",
+        "type_info": "Timestamptz"
+      },
+      {
+        "ordinal": 15,
+        "name": "extra_perms",
+        "type_info": "Jsonb"
+      },
+      {
+        "ordinal": 16,
+        "name": "error",
+        "type_info": "Text"
+      },
+      {
+        "ordinal": 17,
+        "name": "enabled",
+        "type_info": "Bool"
+      }
+    ],
+    "parameters": {
+      "Left": []
+    },
+    "nullable": [
+      false,
+      false,
+      true,
+      true,
+      false,
+      true,
+      false,
+      false,
+      false,
+      false,
+      false,
+      false,
+      false,
+      true,
+      true,
+      false,
+      true,
+      false
+    ]
+  },
+  "hash": "c51f9ad5133c46fd7c499b8339dbbf3f3059bbb85de07ee3b4b4cea971984a52"
+}

backend/.sqlx/query-e7348225a27bbdc9607d7c799e7192cd6ce4088467d91a5cbdc019430320d26d.json

@@ -1,6 +1,6 @@
 {
   "db_name": "PostgreSQL",
-  "query": "SELECT\n                    value->'preprocessor_module'->'value' as \"preprocessor_module: _\",\n                    schema as \"schema: _\"\n                FROM flow \n                WHERE workspace_id = $1\n                    AND path = $2",
+  "query": "SELECT\n                    value->'preprocessor_module'->'value' as \"preprocessor_module: _\",\n                    schema as \"schema: _\"\n                FROM flow_version\n                WHERE \n                    path = $1\n                    AND workspace_id = $2\n                ORDER BY created_at DESC\n                LIMIT 1",
   "describe": {
     "columns": [
       {
@@ -25,5 +25,5 @@
       true
     ]
   },
-  "hash": "72916f8e490f8252e0a51b7f562ccc3be832b12102eb86a07d8405a4fa9287d5"
+  "hash": "e7348225a27bbdc9607d7c799e7192cd6ce4088467d91a5cbdc019430320d26d"
 }

backend/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "windmill"
-version = "1.491.5"
+version = "1.493.4"
 authors.workspace = true
 edition.workspace = true
 
@@ -32,7 +32,7 @@ members = [
 ]
 
 [workspace.package]
-version = "1.491.5"
+version = "1.493.4"
 authors = ["Ruben Fiszel <ruben@windmill.dev>"]
 edition = "2021"
 
@@ -59,7 +59,7 @@ embedding = ["windmill-api/embedding"]
 parquet = ["windmill-api/parquet", "windmill-common/parquet", "windmill-worker/parquet", "dep:object_store"]
 prometheus = ["windmill-common/prometheus", "windmill-api/prometheus", "windmill-worker/prometheus", "windmill-queue/prometheus", "dep:prometheus"]
 flow_testing = ["windmill-worker/flow_testing"]
-openidconnect = ["windmill-api/openidconnect"]
+openidconnect = ["windmill-api/openidconnect", "windmill-common/openidconnect"]
 cloud = ["windmill-queue/cloud", "windmill-worker/cloud", "windmill-common/cloud", "windmill-api/cloud"]
 jemalloc = ["windmill-common/jemalloc", "dep:tikv-jemallocator", "dep:tikv-jemalloc-sys", "dep:tikv-jemalloc-ctl"]
 tantivy = ["dep:windmill-indexer", "windmill-api/tantivy", "windmill-indexer/enterprise", "windmill-indexer/parquet", "windmill-common/tantivy", "enterprise", "parquet"]
@@ -83,17 +83,18 @@ zip = ["windmill-api/zip"]
 static_frontend = ["windmill-api/static_frontend"]
 scoped_cache = ["windmill-common/scoped_cache"]
 # Languages
-python = ["windmill-worker/python"]
+python = ["windmill-worker/python", "windmill-api/python"]
 rust = ["windmill-worker/rust"]
 mysql = ["windmill-worker/mysql"]
 oracledb = ["windmill-worker/oracledb"]
+duckdb = ["windmill-worker/duckdb"]
 mssql = ["windmill-worker/mssql"]
 bigquery = ["windmill-worker/bigquery"]
 php = ["windmill-worker/php"]
 csharp = ["windmill-worker/csharp"]
 nu = ["windmill-worker/nu"]
 java = ["windmill-worker/java"]
-all_languages = [ "python", "deno_core", "rust", "mysql", "oracledb", "mssql", "bigquery", "csharp", "nu", "php", "java"]
+all_languages = [ "python", "deno_core", "rust", "mysql", "oracledb", "duckdb", "mssql", "bigquery", "csharp", "nu", "php", "java"]
 
 
 [patch.crates-io]
@@ -135,8 +136,11 @@ quote.workspace = true
 memchr.workspace = true
 v8 = { workspace = true, optional = true }
 rustls.workspace = true
+pep440_rs.workspace = true
 systemstat.workspace = true
 size.workspace = true
+strum.workspace = true
+
 
 [target.'cfg(not(target_env = "msvc"))'.dependencies]
 tikv-jemallocator = { optional = true, workspace = true }
@@ -218,6 +222,7 @@ git-version = "^0"
 malachite = "=0.4.18"
 malachite-bigint = "=0.2.0"
 rustpython-parser = "^0"
+pep440_rs = "0.7.3"
 php-parser-rs = { git = "https://github.com/php-rust-tools/parser", rev = "ec4cb411dec09450946ef57920b7ffced7f6495d" }
 cron = "^0"
 mail-send = { version = "0.4.0", features = ["builder"], default-features=false }
@@ -234,8 +239,9 @@ json-pointer = "^0"
 itertools = "^0"
 regex = "^1"
 semver = "^1"
+duckdb = { version = "1.2.2", features = ["bundled"] }
 
-v8 = "=130.0.7"  # Exact version
+v8 = "=130.0.7"  # Exact version NOTE: Do not forget to update version and hash in flake.nix
 deno_fetch = "0.214.0"
 deno_tls = "0.177.0"
 deno_console = "0.190.0"
@@ -342,7 +348,7 @@ openidconnect = { version = "4.0.0-rc.1" }
 aws-config = "^1"
 aws-sdk-sqs = "1.57.0"
 aws-sdk-sts = "^1"
-
+aws-smithy-types-convert = { version = "^0", features = ["convert-chrono"] }
 crc = "^3"
 tar = "^0"
 http = "^1"
@@ -389,3 +395,5 @@ tree-sitter-c-sharp = "0.23.0"
 tree-sitter-java = "0.23.0"
 oracle = { version = "0.6.3", features = ["chrono"] }
 rumqttc = { version = "0.24.0", features = ["use-native-tls"]}
+strum = { version = "0.27", features = ["derive"] }
+strum_macros = "^0"

backend/ee-repo-ref.txt

@@ -1 +1 @@
-3efa7fa51e9f93f60e141fef5b8b9338528cf955
+8a2506e86b923c00522cb83b052586f705f7aa8e

backend/migrations/20250515084520_duckdb_support.down.sql

@@ -0,0 +1 @@
+-- Add down migration script here

backend/migrations/20250515084520_duckdb_support.up.sql

@@ -0,0 +1,3 @@
+-- Add up migration script here
+ALTER TYPE SCRIPT_LANG ADD VALUE IF NOT EXISTS 'duckdb';
+UPDATE config set config = jsonb_set(config, '{worker_tags}', config->'worker_tags' || '["duckdb"]'::jsonb) where name = 'worker__default' and config @> '{"worker_tags": ["deno", "python3", "go", "bash", "powershell", "dependency", "flow", "hub", "other", "bun", "php", "rust", "ansible", "csharp", "nu", "java"]}'::jsonb AND NOT config->'worker_tags' @> '"duckdb"'::jsonb;

backend/migrations/20250520104210_workspace_preprocessor_notify.down.sql

@@ -0,0 +1,12 @@
+-- Add down migration script here
+CREATE OR REPLACE FUNCTION notify_runnable_version_change()
+RETURNS TRIGGER AS $$
+DECLARE
+    source_type TEXT;
+BEGIN
+    source_type := TG_ARGV[0];
+
+    PERFORM pg_notify('notify_runnable_version_change', NEW.workspace_id || ':' || source_type || ':' || NEW.path);
+    RETURN NEW;
+END;
+$$ LANGUAGE plpgsql;

backend/migrations/20250520104210_workspace_preprocessor_notify.up.sql

@@ -0,0 +1,19 @@
+-- Add up migration script here
+CREATE OR REPLACE FUNCTION notify_runnable_version_change()
+RETURNS TRIGGER AS $$
+DECLARE
+    source_type TEXT;
+    kind TEXT;
+BEGIN
+    source_type := TG_ARGV[0];
+
+    IF source_type = 'script' THEN
+        kind := NEW.kind;
+    ELSE
+        kind := 'flow';
+    END IF;
+
+    PERFORM pg_notify('notify_runnable_version_change', NEW.workspace_id || ':' || source_type || ':' || NEW.path || ':' || kind);
+    RETURN NEW;
+END;
+$$ LANGUAGE plpgsql;

backend/migrations/20250526173025_http_trigger_seq_grants.down.sql

@@ -0,0 +1 @@
+-- Add down migration script here

backend/migrations/20250526173025_http_trigger_seq_grants.up.sql

@@ -0,0 +1,3 @@
+-- Add up migration script here
+GRANT ALL ON SEQUENCE http_trigger_version_seq TO windmill_user;
+GRANT ALL ON SEQUENCE http_trigger_version_seq TO windmill_admin;

backend/parsers/windmill-parser-py-imports/Cargo.toml

@@ -27,3 +27,6 @@ anyhow.workspace = true
 lazy_static.workspace = true
 sqlx.workspace = true
 async-recursion.workspace = true
+toml.workspace = true
+serde.workspace = true
+pep440_rs.workspace = true

backend/parsers/windmill-parser-py-imports/src/lib.rs

@@ -11,7 +11,7 @@ mod mapping;
 use async_recursion::async_recursion;
 use itertools::Itertools;
 use lazy_static::lazy_static;
-use std::collections::HashMap;
+use std::{collections::HashMap, str::FromStr};
 
 use mapping::{FULL_IMPORTS_MAP, SHORT_IMPORTS_MAP};
 #[cfg(not(target_arch = "wasm32"))]
@@ -25,7 +25,10 @@ use rustpython_parser::{
     Parse,
 };
 use sqlx::{Pool, Postgres};
-use windmill_common::{error, worker::PythonAnnotations};
+use windmill_common::{
+    error::{self, to_anyhow},
+    worker::PythonAnnotations,
+};
 
 const DEF_MAIN: &str = "def main(";
 
@@ -242,8 +245,7 @@ pub async fn parse_python_imports(
     w_id: &str,
     path: &str,
     db: &Pool<Postgres>,
-    already_visited: &mut Vec<String>,
-    annotated_pyv_numeric: &mut Option<u32>,
+    version_specifiers: &mut Vec<pep440_rs::VersionSpecifier>,
 ) -> error::Result<(Vec<String>, Option<String>)> {
     let mut compile_error_hint: Option<String> = None;
     let mut imports = parse_python_imports_inner(
@@ -251,9 +253,10 @@ pub async fn parse_python_imports(
         w_id,
         path,
         db,
-        already_visited,
-        annotated_pyv_numeric,
-        &mut annotated_pyv_numeric.and_then(|_| Some(path.to_owned())),
+        &mut vec![],
+        version_specifiers,
+        // &mut version_specifier.and_then(|_| Some(path.to_owned())),
+        &mut None
     )
     .await?
     .into_values()
@@ -279,6 +282,7 @@ pub async fn parse_python_imports(
     .flatten()
     .collect::<error::Result<Vec<String>>>()?
     .into_iter()
+    .filter(|x| !x.trim_start().starts_with("--") && !x.trim().is_empty())
     .unique()
     .collect_vec();
 
@@ -304,11 +308,34 @@ async fn parse_python_imports_inner(
     path: &str,
     db: &Pool<Postgres>,
     already_visited: &mut Vec<String>,
-    annotated_pyv_numeric: &mut Option<u32>,
+    version_specifiers: &mut Vec<pep440_rs::VersionSpecifier>,
     path_where_annotated_pyv: &mut Option<String>,
 ) -> error::Result<HashMap<String, NImportResolved>> {
     let PythonAnnotations { py310, py311, py312, py313, .. } = PythonAnnotations::parse(&code);
 
+    let mut push_version_specifiers = |perform, unparsed: String| -> error::Result<()> {
+        if perform {
+            pep440_rs::VersionSpecifiers::from_str(unparsed.as_str())
+                .ok()
+                .map(|vs| version_specifiers.extend(vs.to_vec()));
+        }
+        Ok(())
+    };
+    push_version_specifiers(py310, "==3.10.*".to_owned())?;
+    push_version_specifiers(py311, "==3.11.*".to_owned())?;
+    push_version_specifiers(py312, "==3.12.*".to_owned())?;
+    push_version_specifiers(py313, "==3.13.*".to_owned())?;
+
+    for x in code.lines() {
+        if x.starts_with("# py:") || x.starts_with("#py:") {
+            push_version_specifiers(
+                true,
+                x.replace('#', "").replace("py:", "").trim().to_owned(),
+            )?;
+        } else if !x.starts_with('#') {
+            break;
+        }
+    }
     // we pass only if there is none or only one annotation
 
     // Naive:
@@ -323,39 +350,48 @@ async fn parse_python_imports_inner(
     // This way we make sure there is no multiple annotations for same script
     // and we get detailed span on conflicting versions
 
-    let mut check = |is_py_xyz, numeric| -> error::Result<()> {
-        if is_py_xyz {
-            if let Some(v) = annotated_pyv_numeric {
-                if *v != numeric {
-                    return Err(error::Error::from(anyhow::anyhow!(
-                        "Annotated 2 or more different python versions: \n - py{v} at {}\n - py{numeric} at {path}\nIt is possible to use only one.",
-                        path_where_annotated_pyv.clone().unwrap_or("Unknown".to_owned())
-                    )));
-                }
-            } else {
-                *annotated_pyv_numeric = Some(numeric);
-            }
-            *path_where_annotated_pyv = Some(path.to_owned());
-        }
-        Ok(())
-    };
+    #[derive(serde::Serialize, serde::Deserialize)]
+    struct InlineMetadata {
+        requires_python: String,
+        dependencies: Vec<String>,
+    }
 
-    check(py310, 310)?;
-    check(py311, 311)?;
-    check(py312, 312)?;
-    check(py313, 313)?;
-
-    let find_requirements = code
-        .lines()
-        .find_position(|x| x.starts_with("#requirements:") || x.starts_with("# requirements:"));
-    if let Some((pos, _)) = find_requirements {
+    let find_requirements = code.lines().find_position(|x| {
+        x.starts_with("#requirements:")
+            || x.starts_with("# requirements:")
+            || x.starts_with("# /// script")
+    });
+    if let Some((pos, item)) = find_requirements {
         let mut requirements = HashMap::new();
-        code.lines()
-            .skip(pos + 1)
-            .map_while(|x| {
-                RE.captures(x).and_then(|x| {
-                    x.get(1).map(|m| {
-                        let requirement = m.as_str().to_string();
+        if item.starts_with("# /// script") {
+            let mut incorrect = false;
+            let metadata = code
+                .lines()
+                .skip(pos + 1)
+                .map_while(|x| {
+                    incorrect = !x.starts_with('#');
+                    if incorrect || x.starts_with("# ///") {
+                        None
+                    } else {
+                        x.get(1..)
+                    }
+                })
+                .join("\n")
+                .parse::<toml::Table>()
+            .map_err(to_anyhow)?;
+
+            {
+                if let Some(v) = metadata.get("requires-python").and_then(|v| v.as_str()) {
+                    push_version_specifiers(true, v.to_owned())?;
+                }
+            };
+
+            metadata
+                .get("dependencies")
+                .and_then(|dependencies| dependencies.as_array())
+                .inspect(|list| {
+                    for dependency_v in list.into_iter() {
+                        let requirement = dependency_v.as_str().unwrap_or("ERROR").to_owned();
                         let key = extract_pkg_name(&requirement);
                         requirements.insert(
                             key.clone(),
@@ -367,11 +403,31 @@ async fn parse_python_imports_inner(
                                 key,
                             },
                         );
+                    }
+                });
+        } else {
+            code.lines()
+                .skip(pos + 1)
+                .map_while(|x| {
+                    RE.captures(x).and_then(|x| {
+                        x.get(1).map(|m| {
+                            let requirement = m.as_str().to_string();
+                            let key = extract_pkg_name(&requirement);
+                            requirements.insert(
+                                key.clone(),
+                                NImportResolved::Pin {
+                                    pins: vec![ImportPin {
+                                        pkg: requirement.clone(),
+                                        path: Default::default(),
+                                    }],
+                                    key,
+                                },
+                            );
+                        })
                     })
                 })
-            })
-            .collect_vec();
-
+                .collect_vec();
+        }
         Ok(requirements)
     } else {
         let find_extra_requirements = code.lines().find_position(|x| {
@@ -442,7 +498,7 @@ async fn parse_python_imports_inner(
                             &rpath,
                             db,
                             already_visited,
-                            annotated_pyv_numeric,
+                            version_specifiers,
                             path_where_annotated_pyv,
                         )
                         .await?

backend/parsers/windmill-parser-py-imports/tests/tests.rs

@@ -18,16 +18,8 @@ def main():
     pass
 
 ";
-        let mut already_visited = vec![];
-        let (r, ..) = parse_python_imports(
-            code,
-            "test-workspace",
-            "f/foo/bar",
-            &db,
-            &mut already_visited,
-            &mut None,
-        )
-        .await?;
+        let (r, ..) =
+            parse_python_imports(code, "test-workspace", "f/foo/bar", &db, &mut vec![]).await?;
         // println!("{}", serde_json::to_string(&r)?);
         assert_eq!(
             r,
@@ -59,16 +51,8 @@ def main():
     pass
 
 ";
-        let mut already_visited = vec![];
-        let (r, ..) = parse_python_imports(
-            code,
-            "test-workspace",
-            "f/foo/bar",
-            &db,
-            &mut already_visited,
-            &mut None,
-        )
-        .await?;
+        let (r, ..) =
+            parse_python_imports(code, "test-workspace", "f/foo/bar", &db, &mut vec![]).await?;
         println!("{}", serde_json::to_string(&r)?);
         assert_eq!(r, vec!["burkina=0.4", "nigeria"]);
 
@@ -89,17 +73,9 @@ def main():
     pass
 
 ";
-        let mut already_visited = vec![];
 
-        let (r, ..) = parse_python_imports(
-            code,
-            "test-workspace",
-            "f/foo/bar",
-            &db,
-            &mut already_visited,
-            &mut None,
-        )
-        .await?;
+        let (r, ..) =
+            parse_python_imports(code, "test-workspace", "f/foo/bar", &db, &mut vec![]).await?;
         println!("{}", serde_json::to_string(&r)?);
         assert_eq!(
             r,

backend/parsers/windmill-parser-sql/src/lib.rs

@@ -83,6 +83,21 @@ pub fn parse_bigquery_sig(code: &str) -> anyhow::Result<MainArgSignature> {
     }
 }
 
+pub fn parse_duckdb_sig(code: &str) -> anyhow::Result<MainArgSignature> {
+    let parsed = parse_duckdb_file(&code)?;
+    if let Some(args) = parsed {
+        Ok(MainArgSignature {
+            star_args: false,
+            star_kwargs: false,
+            args,
+            no_main_func: None,
+            has_preprocessor: None,
+        })
+    } else {
+        Err(anyhow!("Error parsing sql".to_string()))
+    }
+}
+
 pub fn parse_snowflake_sig(code: &str) -> anyhow::Result<MainArgSignature> {
     let parsed = parse_snowflake_file(&code)?;
     if let Some(x) = parsed {
@@ -212,6 +227,9 @@ lazy_static::lazy_static! {
     // -- @name (type) = default
     static ref RE_ARG_BIGQUERY: Regex = Regex::new(r#"(?m)^-- @(\w+) \((\w+(?:\[\])?)\)(?: ?\= ?(.+))? *(?:\r|\n|$)"#).unwrap();
 
+    // -- $name (type) = default
+    static ref RE_ARG_DUCKDB: Regex = Regex::new(r#"(?m)^-- \$(\w+) \((\w+)\)(?: ?\= ?(.+))? *(?:\r|\n|$)"#).unwrap();
+
     static ref RE_ARG_SNOWFLAKE: Regex = Regex::new(r#"(?m)^-- \? (\w+) \((\w+)\)(?: ?\= ?(.+))? *(?:\r|\n|$)"#).unwrap();
 
 
@@ -577,6 +595,35 @@ fn parse_bigquery_file(code: &str) -> anyhow::Result<Option<Vec<Arg>>> {
     Ok(Some(args))
 }
 
+fn parse_duckdb_file(code: &str) -> anyhow::Result<Option<Vec<Arg>>> {
+    let mut args: Vec<Arg> = vec![];
+
+    for cap in RE_ARG_DUCKDB.captures_iter(code) {
+        let name = cap.get(1).map(|x| x.as_str().to_string()).unwrap();
+        let typ = cap
+            .get(2)
+            .map(|x| x.as_str().to_string().to_lowercase())
+            .unwrap();
+        let default = cap.get(3).map(|x| x.as_str().to_string());
+        let has_default = default.is_some();
+        let parsed_typ = parse_duckdb_typ(typ.as_str());
+
+        let parsed_default = default.and_then(|x| parsed_default(&parsed_typ, x));
+
+        args.push(Arg {
+            name,
+            typ: parsed_typ,
+            default: parsed_default,
+            otyp: Some(typ),
+            has_default,
+            oidx: None,
+        });
+    }
+
+    args.append(&mut parse_sql_sanitized_interpolation(code));
+    Ok(Some(args))
+}
+
 fn parse_snowflake_file(code: &str) -> anyhow::Result<Option<Vec<Arg>>> {
     let mut args: Vec<Arg> = vec![];
 
@@ -729,6 +776,33 @@ pub fn parse_bigquery_typ(typ: &str) -> Typ {
     }
 }
 
+pub fn parse_duckdb_typ(typ: &str) -> Typ {
+    if typ.ends_with("[]") {
+        let base_typ = parse_duckdb_typ(typ.strip_suffix("[]").unwrap());
+        Typ::List(Box::new(base_typ))
+    } else {
+        match typ {
+            "varchar" | "char" | "bpchar" | "text" | "string" => Typ::Str(None),
+            "blob" | "bytea" | "binary" | "varbinary" | "bitstring" => Typ::Bytes,
+            "boolean" | "bool" | "bit" | "logical" => Typ::Bool,
+            "bigint" | "int8" | "long" | "integer" | "int4" | "int" | "smallint" | "int2"
+            | "short" | "tinyint" | "int1" | "signed" | "ubigint" | "uhugeint" | "uinteger"
+            | "usmallint" | "utinyint" => Typ::Int,
+            "decimal" | "numeric" | "double" | "float8" | "float" | "float4" | "real" => Typ::Float,
+            "date"
+            | "time"
+            | "timestamp with time zone"
+            | "timestamptz"
+            | "timestamp"
+            | "datetime" => Typ::Datetime,
+            "uuid" | "json" => Typ::Str(None),
+            "interval" | "hugeint" => Typ::Str(None),
+            "s3object" => Typ::Resource("S3Object".to_string()),
+            _ => Typ::Str(None),
+        }
+    }
+}
+
 pub fn parse_snowflake_typ(typ: &str) -> Typ {
     match typ {
         "varchar" => Typ::Str(None),

backend/parsers/windmill-parser-wasm/src/lib.rs

@@ -96,6 +96,12 @@ pub fn parse_oracledb(code: &str) -> String {
     wrap_sig(windmill_parser_sql::parse_oracledb_sig(code))
 }
 
+#[cfg(feature = "sql-parser")]
+#[wasm_bindgen]
+pub fn parse_duckdb(code: &str) -> String {
+    wrap_sig(windmill_parser_sql::parse_duckdb_sig(code))
+}
+
 #[cfg(feature = "sql-parser")]
 #[wasm_bindgen]
 pub fn parse_bigquery(code: &str) -> String {

backend/src/main.rs

@@ -5,7 +5,6 @@
  * Please see the included NOTICE for copyright information and
  * LICENSE-AGPL for a copy of the license.
  */
-
 use anyhow::Context;
 use monitor::{
     load_base_url, load_otel, reload_critical_alerts_on_db_oversize,
@@ -23,6 +22,7 @@ use std::{
     net::{IpAddr, Ipv4Addr, SocketAddr},
     time::{Duration, Instant},
 };
+use strum::IntoEnumIterator;
 use tokio::{fs::File, io::AsyncReadExt, task::JoinHandle};
 use uuid::Uuid;
 use windmill_api::HTTP_CLIENT;
@@ -50,6 +50,7 @@ use windmill_common::{
     },
     scripts::ScriptLang,
     stats_ee::schedule_stats,
+    triggers::TriggerKind,
     utils::{hostname, rd_string, Mode, GIT_VERSION, MODE_AND_ADDONS},
     worker::{
         reload_custom_tags_setting, Connection, HUB_CACHE_DIR, TMP_DIR, TMP_LOGS_DIR, WORKER_GROUP,
@@ -68,7 +69,7 @@ use tikv_jemallocator::Jemalloc;
 static GLOBAL: Jemalloc = Jemalloc;
 
 #[cfg(feature = "parquet")]
-use windmill_common::global_settings::OBJECT_STORE_CACHE_CONFIG_SETTING;
+use windmill_common::global_settings::OBJECT_STORE_CONFIG_SETTING;
 
 use windmill_worker::{
     get_hub_script_content_and_requirements, BUN_BUNDLE_CACHE_DIR, BUN_CACHE_DIR, CSHARP_CACHE_DIR,
@@ -91,7 +92,7 @@ use crate::monitor::{
 };
 
 #[cfg(feature = "parquet")]
-use crate::monitor::reload_s3_cache_setting;
+use windmill_common::s3_helpers::reload_object_store_setting;
 
 const DEFAULT_NUM_WORKERS: usize = 1;
 const DEFAULT_PORT: u16 = 8000;
@@ -791,11 +792,37 @@ Windmill Community Edition {GIT_VERSION}
                                                     let payload = n.payload();
                                                     tracing::info!("Runnable version change detected: {}", payload);
                                                     match payload.split(':').collect::<Vec<&str>>().as_slice() {
-                                                        [workspace_id, source_type, path] => {
+                                                        [workspace_id, source_type, path, kind] => {
                                                             let key = (workspace_id.to_string(), path.to_string());
                                                             match source_type {
                                                                 &"script" => {
                                                                     windmill_common::DEPLOYED_SCRIPT_HASH_CACHE.remove(&key);
+                                                                    match kind {
+                                                                        &"preprocessor" => {
+                                                                            match sqlx::query_scalar!(
+                                                                                "SELECT fv.id
+                                                                                FROM flow f
+                                                                                INNER JOIN flow_version fv ON fv.id = f.versions[array_upper(f.versions, 1)]
+                                                                                WHERE fv.value->'preprocessor_module'->'value'->>'path' = $1 AND f.workspace_id = $2",
+                                                                                path,
+                                                                                workspace_id
+                                                                            ).fetch_all(&db).await {
+                                                                                Ok(flow_versions) => {
+                                                                                    tracing::debug!("Workspace preprocessor {} changed, removing runnable format version cache for flow versions {:?}", path, flow_versions);
+                                                                                    for version in flow_versions {
+                                                                                        for trigger_kind in TriggerKind::iter() {
+                                                                                            let key = (windmill_common::triggers::HubOrWorkspaceId::WorkspaceId(workspace_id.to_string()), version, trigger_kind);
+                                                                                            windmill_common::triggers::RUNNABLE_FORMAT_VERSION_CACHE.remove(&key);
+                                                                                        }
+                                                                                    }
+                                                                                }
+                                                                                Err(e) => {
+                                                                                    tracing::error!("Error fetching flow paths: {e:#}");
+                                                                                }
+                                                                            }
+                                                                        },
+                                                                        _ => {}
+                                                                    }
                                                                 }
                                                                 &"flow" => {
                                                                     windmill_common::FLOW_VERSION_CACHE.remove(&key);
@@ -880,9 +907,9 @@ Windmill Community Edition {GIT_VERSION}
                                                             reload_job_default_timeout_setting(&conn).await
                                                         },
                                                         #[cfg(feature = "parquet")]
-                                                        OBJECT_STORE_CACHE_CONFIG_SETTING => {
+                                                        OBJECT_STORE_CONFIG_SETTING => {
                                                             if !disable_s3_store {
-                                                                reload_s3_cache_setting(&db).await
+                                                                reload_object_store_setting(&db).await;
                                                             }
                                                         },
                                                         SCIM_TOKEN_SETTING => {

backend/src/monitor.rs

@@ -33,8 +33,11 @@ use windmill_common::ee::low_disk_alerts;
 #[cfg(feature = "enterprise")]
 use windmill_common::ee::{jobs_waiting_alerts, worker_groups_alerts};
 
+use windmill_common::client::AuthedClient;
 #[cfg(feature = "oauth2")]
 use windmill_common::global_settings::OAUTH_SETTING;
+#[cfg(feature = "parquet")]
+use windmill_common::s3_helpers::reload_object_store_setting;
 use windmill_common::{
     agent_workers::DECODED_AGENT_TOKEN,
     auth::create_token_for_owner,
@@ -75,19 +78,13 @@ use windmill_common::{
 };
 use windmill_queue::{cancel_job, MiniPulledJob, SameWorkerPayload};
 use windmill_worker::{
-    handle_job_error, AuthedClient, JobCompletedSender, SameWorkerSender, BUNFIG_INSTALL_SCOPES,
+    handle_job_error, JobCompletedSender, SameWorkerSender, BUNFIG_INSTALL_SCOPES,
     INSTANCE_PYTHON_VERSION, JOB_DEFAULT_TIMEOUT, KEEP_JOB_DIR, MAVEN_REPOS, NO_DEFAULT_MAVEN,
     NPM_CONFIG_REGISTRY, NUGET_CONFIG, PIP_EXTRA_INDEX_URL, PIP_INDEX_URL,
 };
 
 #[cfg(feature = "parquet")]
-use windmill_common::s3_helpers::{
-    build_object_store_from_settings, build_s3_client_from_settings, S3Settings,
-    OBJECT_STORE_CACHE_SETTINGS,
-};
-
-#[cfg(feature = "parquet")]
-use windmill_common::global_settings::OBJECT_STORE_CACHE_CONFIG_SETTING;
+use windmill_common::s3_helpers::ObjectStoreReload;
 
 #[cfg(feature = "enterprise")]
 use crate::ee::verify_license_key;
@@ -241,7 +238,23 @@ pub async fn initial_load(
     #[cfg(feature = "parquet")]
     if !disable_s3_store {
         if let Some(db) = conn.as_sql() {
-            reload_s3_cache_setting(db).await;
+            let db2 = db.clone();
+            match reload_object_store_setting(db).await {
+                ObjectStoreReload::Later => {
+                    tokio::spawn(async move {
+                        tokio::time::sleep(Duration::from_secs(10)).await;
+                        match reload_object_store_setting(&db2).await {
+                            ObjectStoreReload::Later => {
+                                tracing::error!("Giving up on loading object store setting");
+                            }
+                            ObjectStoreReload::Never => {
+                                tracing::info!("Object store setting successfully loaded");
+                            }
+                        }
+                    });
+                }
+                ObjectStoreReload::Never => (),
+            }
         }
     }
 
@@ -631,7 +644,7 @@ async fn send_log_file_to_object_store(
         }
 
         #[cfg(feature = "parquet")]
-        let s3_client = OBJECT_STORE_CACHE_SETTINGS.read().await.clone();
+        let s3_client = windmill_common::s3_helpers::get_object_store().await;
         #[cfg(feature = "parquet")]
         if let Some(s3_client) = s3_client {
             let path = std::path::Path::new(TMP_WINDMILL_LOGS_SERVICE)
@@ -917,10 +930,7 @@ async fn delete_log_files_from_disk_and_store(
     _s3_prefix: &str,
 ) {
     #[cfg(feature = "parquet")]
-    let os = windmill_common::s3_helpers::OBJECT_STORE_CACHE_SETTINGS
-        .read()
-        .await
-        .clone();
+    let os = windmill_common::s3_helpers::get_object_store().await;
     #[cfg(not(feature = "parquet"))]
     let os: Option<()> = None;
 
@@ -1101,61 +1111,6 @@ pub async fn reload_delete_logs_periodically_setting(conn: &Connection) {
     }
 }
 
-#[cfg(feature = "parquet")]
-pub async fn reload_s3_cache_setting(db: &DB) {
-    use windmill_common::{
-        ee::{get_license_plan, LicensePlan},
-        s3_helpers::ObjectSettings,
-    };
-
-    let s3_config = load_value_from_global_settings(db, OBJECT_STORE_CACHE_CONFIG_SETTING).await;
-    if let Err(e) = s3_config {
-        tracing::error!("Error reloading s3 cache config: {:?}", e)
-    } else {
-        if let Some(v) = s3_config.unwrap() {
-            if matches!(get_license_plan().await, LicensePlan::Pro) {
-                tracing::error!("S3 cache is not available for pro plan");
-                return;
-            }
-            let mut s3_cache_settings = OBJECT_STORE_CACHE_SETTINGS.write().await;
-            let setting = serde_json::from_value::<ObjectSettings>(v);
-            if let Err(e) = setting {
-                tracing::error!("Error parsing s3 cache config: {:?}", e)
-            } else {
-                let s3_client = build_object_store_from_settings(setting.unwrap()).await;
-                if let Err(e) = s3_client {
-                    tracing::error!("Error building s3 client from settings: {:?}", e)
-                } else {
-                    *s3_cache_settings = Some(s3_client.unwrap());
-                }
-            }
-        } else {
-            let mut s3_cache_settings = OBJECT_STORE_CACHE_SETTINGS.write().await;
-            if std::env::var("S3_CACHE_BUCKET").is_ok() {
-                if matches!(get_license_plan().await, LicensePlan::Pro) {
-                    tracing::error!("S3 cache is not available for pro plan");
-                    return;
-                }
-                *s3_cache_settings = build_s3_client_from_settings(S3Settings {
-                    bucket: None,
-                    region: None,
-                    access_key: None,
-                    secret_key: None,
-                    endpoint: None,
-                    store_logs: None,
-                    path_style: None,
-                    allow_http: None,
-                    port: None,
-                })
-                .await
-                .ok();
-            } else {
-                *s3_cache_settings = None;
-            }
-        }
-    }
-}
-
 pub async fn reload_job_default_timeout_setting(conn: &Connection) {
     reload_option_setting_with_tracing(
         conn,

backend/tests/fixtures/multipython.sql

@@ -0,0 +1,20 @@
+INSERT INTO public.script(workspace_id, created_by, content, schema, summary, description, path, hash, language, lock) VALUES (
+'test-workspace',
+'test-user',
+'# py312
+',
+'{"$schema":"https://json-schema.org/draft/2020-12/schema","properties":{},"required":[],"type":"object"}',
+'',
+'',
+'f/multipython/aliases', 2468135790, 'python3', '');
+
+INSERT INTO public.script(workspace_id, created_by, content, schema, summary, description, path, hash, language, lock) VALUES (
+'test-workspace',
+'test-user',
+'# py: >=3.9,!=3.12.2
+',
+'{"$schema":"https://json-schema.org/draft/2020-12/schema","properties":{},"required":[],"type":"object"}',
+'',
+'',
+'f/multipython/script1', 2345678901, 'python3', '');
+

backend/tests/worker.rs

@@ -3970,7 +3970,7 @@ async fn assert_lockfile(
 #[sqlx::test(fixtures("base", "lockfile_python"))]
 async fn test_requirements_python(db: Pool<Postgres>) {
     let content = r#"
-# py311
+# py: 3.11.11
 # requirements:
 # tiny==0.1.3
 
@@ -3988,7 +3988,7 @@ def main():
         &db,
         content,
         ScriptLang::Python3,
-        vec!["# py311", "tiny==0.1.3"],
+        vec!["# py: 3.11.11", "tiny==0.1.3"],
     )
     .await;
 }
@@ -3998,7 +3998,7 @@ def main():
 async fn test_extra_requirements_python(db: Pool<Postgres>) {
     {
         let content = r#"
-# py311
+# py: ==3.11.11
 # extra_requirements:
 # tiny
 
@@ -4016,7 +4016,7 @@ def main():
             &db,
             content,
             ScriptLang::Python3,
-            vec!["# py311", "bottle==0.13.2", "tiny==0.1.2"],
+            vec!["# py: 3.11.11", "bottle==0.13.2", "tiny==0.1.2"],
         )
         .await;
     }
@@ -4026,7 +4026,7 @@ def main():
 #[sqlx::test(fixtures("base", "lockfile_python"))]
 async fn test_extra_requirements_python2(db: Pool<Postgres>) {
     let content = r#"
-# py311
+# py: ==3.11.11
 # extra_requirements:
 # tiny==0.1.3
 
@@ -4040,7 +4040,7 @@ def main():
         &db,
         content,
         ScriptLang::Python3,
-        vec!["# py311", "simplejson==3.20.1", "tiny==0.1.3"],
+        vec!["# py: 3.11.11", "simplejson==3.20.1", "tiny==0.1.3"],
     )
     .await;
 }
@@ -4049,7 +4049,7 @@ def main():
 #[sqlx::test(fixtures("base", "lockfile_python"))]
 async fn test_pins_python(db: Pool<Postgres>) {
     let content = r#"
-# py311
+# py: ==3.11.11
 # extra_requirements:
 # tiny==0.1.3
 # bottle==0.13.2
@@ -4069,7 +4069,7 @@ def main():
         content,
         ScriptLang::Python3,
         vec![
-            "# py311",
+            "# py: 3.11.11",
             "bottle==0.13.2",
             "microdot==2.2.0",
             "simplejson==3.19.3",
@@ -4078,6 +4078,39 @@ def main():
     )
     .await;
 }
+#[cfg(feature = "python")]
+#[sqlx::test(fixtures("base", "multipython"))]
+async fn test_multipython_python(db: Pool<Postgres>) {
+    let content = r#"# py: <=3.12.2, >=3.12.0
+import f.multipython.script1
+import f.multipython.aliases
+"#
+    .to_string();
+
+    assert_lockfile(&db, content, ScriptLang::Python3, vec!["# py: 3.12.1\n"]).await;
+}
+
+#[cfg(feature = "python")]
+#[sqlx::test(fixtures("base", "multipython"))]
+async fn test_inline_script_metadata_python(db: Pool<Postgres>) {
+    let content = r#"# py_select_latest
+# /// script
+# requires-python = ">3.11,<3.12.3,!=3.12.2"
+# dependencies = [
+#   "tiny==0.1.3",
+# ]
+# ///
+"#
+    .to_string();
+
+    assert_lockfile(
+        &db,
+        content,
+        ScriptLang::Python3,
+        vec!["# py: 3.12.1", "tiny==0.1.3"],
+    )
+    .await;
+}
 #[sqlx::test(fixtures("base", "result_format"))]
 async fn test_result_format(db: Pool<Postgres>) {
     let ordered_result_job_id = "1eecb96a-c8b0-4a3d-b1b6-087878c55e41";

backend/windmill-api/Cargo.toml

@@ -18,7 +18,7 @@ benchmark = []
 embedding = ["dep:tinyvector", "dep:hf-hub", "dep:tokenizers", "dep:candle-core", "dep:candle-transformers", "dep:candle-nn"]
 parquet = ["dep:datafusion", "dep:object_store", "dep:url", "windmill-common/parquet", "windmill-worker/parquet"]
 prometheus = ["windmill-common/prometheus", "windmill-queue/prometheus", "dep:prometheus", "windmill-worker/prometheus"]
-openidconnect = ["dep:openidconnect"]
+openidconnect = ["dep:openidconnect", "windmill-common/openidconnect"]
 tantivy = ["dep:windmill-indexer"]
 kafka = ["dep:rdkafka"]
 nats = ["dep:async-nats", "dep:nkeys"]
@@ -36,6 +36,7 @@ deno_core = ["dep:deno_core", "dep:deno_error"]
 gcp_trigger = ["dep:thiserror", "dep:google-cloud-pubsub", "dep:google-cloud-googleapis", "dep:tonic"]
 cloud = ["windmill-common/cloud"]
 mcp = ["dep:rmcp"]
+python = []
 
 [dependencies]
 rmcp = { git = "https://github.com/windmill-labs/rust-sdk", features = ["transport-sse-server"], optional = true }

backend/windmill-api/openapi.yaml

@@ -1,7 +1,7 @@
 openapi: "3.0.3"
 
 info:
-  version: 1.491.5
+  version: 1.493.4
   title: Windmill API
 
   contact:
@@ -377,6 +377,9 @@ paths:
                   type: string
                 company:
                   type: string
+                skip_email:
+                  type: boolean
+                  description: Skip sending email notifications to the user
               required:
                 - email
                 - password
@@ -11105,6 +11108,23 @@ paths:
                 items:
                   $ref: "#/components/schemas/AutoscalingEvent"
 
+  /configs/list_available_python_versions:
+    get:
+      summary: Get currently available python versions provided by UV.
+      operationId: listAvailablePythonVersions
+      tags:
+        - config
+      # parameters:
+      responses:
+        "200":
+          description: List of python versions
+          content:
+            application/json:
+              schema:
+                type: array
+                items:
+                  type: string
+
   /agent_workers/create_agent_token:
     post:
       summary: create agent token
@@ -12590,6 +12610,11 @@ paths:
           required: true
           schema:
             type: string
+        - name: pagination_offset
+          in: query
+          required: false
+          schema:
+            type: integer
       responses:
         "200":
           description: search results
@@ -12602,15 +12627,26 @@ paths:
                     description: a list of the terms that couldn't be parsed (and thus ignored)
                     type: array
                     items:
-                      type: object
-                      properties:
-                        dancer:
-                          type: string
+                      type: string
                   hits:
                     description: the jobs that matched the query
                     type: array
                     items:
                       $ref: "#/components/schemas/JobSearchHit"
+                  hit_count:
+                    description: how many jobs matched in total
+                    type: number
+                  index_metadata:
+                    description: Metadata about the index current state
+                    type: object
+                    properties:
+                      indexed_until:
+                        description: Datetime of the most recently indexed job
+                        type: string
+                        format: date-time
+                      lost_lock_ownership:
+                        description: Is the current indexer service being replaced
+                        type: boolean
 
   /srch/index/search/service_logs:
     get:
@@ -14209,7 +14245,8 @@ components:
           ansible,
           csharp,
           nu,
-          java
+          java,
+          duckdb
           # for related places search: ADD_NEW_LANG
         ]
 
@@ -16316,6 +16353,7 @@ components:
               - schedule
               - user
               - group
+              - trigger
         repositories:
           type: array
           items:
@@ -16383,6 +16421,7 @@ components:
               - schedule
               - user
               - group
+              - trigger
       required:
         - script_path
         - git_repo_resource_path
@@ -16784,7 +16823,6 @@ components:
           type: string
       required:
         - s3
-
     TeamsChannel:
       type: object
       required:
@@ -16808,4 +16846,4 @@ components:
         channel_name:
           type: string
           description: Microsoft Teams channel name
-          minLength: 1
+          minLength: 1

backend/windmill-api/src/args.rs

@@ -10,12 +10,17 @@ use http::{header::CONTENT_TYPE, request::Parts, StatusCode};
 use serde::{Deserialize, Serialize};
 use serde_json::value::RawValue;
 use sqlx::types::JsonRawValue;
-use windmill_common::{error::Error, worker::to_raw_value, DB};
-use windmill_queue::{PushArgsOwned, TriggerKind};
+use windmill_common::{
+    error::Error,
+    triggers::{RunnableFormat, RunnableFormatVersion, TriggerKind},
+    worker::to_raw_value,
+    DB,
+};
+use windmill_queue::PushArgsOwned;
 
 use crate::{
     db::ApiAuthed,
-    trigger_helpers::{get_runnable_format, RunnableFormat, RunnableFormatVersion, RunnableId},
+    trigger_helpers::{get_runnable_format, RunnableId},
 };
 
 #[derive(Debug)]

backend/windmill-api/src/capture.rs

@@ -57,7 +57,6 @@ use {
 use crate::{
     args::RawWebhookArgs,
     db::{ApiAuthed, DB},
-    trigger_helpers::{RunnableFormat, RunnableFormatVersion},
     users::fetch_api_authed,
     utils::RunnableKind,
 };
@@ -76,11 +75,12 @@ use sqlx::types::Json as SqlxJson;
 use windmill_common::{
     db::UserDB,
     error::{JsonResult, Result},
+    triggers::{RunnableFormat, RunnableFormatVersion, TriggerKind},
     utils::{not_found_if_none, paginate, Pagination, StripPath},
     worker::{to_raw_value, CLOUD_HOSTED},
 };
 
-use windmill_queue::{PushArgs, PushArgsOwned, TriggerKind};
+use windmill_queue::{PushArgs, PushArgsOwned};
 
 const KEEP_LAST: i64 = 20;
 

backend/windmill-api/src/configs.rs

@@ -33,6 +33,10 @@ pub fn global_service() -> Router {
             "/list_autoscaling_events/:worker_group",
             get(list_autoscaling_events),
         )
+        .route(
+            "/list_available_python_versions",
+            get(list_available_python_versions),
+        )
 }
 
 #[derive(Serialize, Deserialize, FromRow)]
@@ -205,6 +209,24 @@ async fn list_autoscaling_events(
     Ok(Json(events))
 }
 
+async fn list_available_python_versions() -> error::JsonResult<Vec<String>> {
+    #[cfg(not(feature = "python"))]
+    return Err(error::Error::BadRequest(
+        "Python listing available only with 'python' feature enabled".to_string(),
+    ));
+
+    #[cfg(feature = "python")]
+    use itertools::Itertools;
+    #[cfg(feature = "python")]
+    return Ok(Json(
+        windmill_worker::PyV::list_available_python_versions()
+            .await
+            .iter()
+            .map(|v| v.to_string())
+            .collect_vec(),
+    ));
+}
+
 #[cfg(feature = "enterprise")]
 async fn list_configs(
     authed: ApiAuthed,

backend/windmill-api/src/gcp_triggers_ee.rs

@@ -11,9 +11,9 @@ use windmill_common::db::UserDB;
 use windmill_common::worker::to_raw_value;
 use windmill_common::{
     error::{Error as WindmillError, Result as WindmillResult},
+    triggers::TriggerKind,
     utils::empty_as_none,
 };
-use windmill_queue::TriggerKind;
 
 #[derive(sqlx::Type, Debug, Deserialize, Serialize)]
 #[serde(rename_all(serialize = "lowercase", deserialize = "lowercase"))]
@@ -86,7 +86,7 @@ pub async fn manage_google_subscription(
     _subscription_mode: SubscriptionMode,
     _create_update_config: Option<CreateUpdateConfig>,
     _trigger_mode: bool,
-    _is_flow: bool
+    _is_flow: bool,
 ) -> WindmillResult<CreateUpdateConfig> {
     Ok(CreateUpdateConfig::default())
 }

backend/windmill-api/src/http_trigger_args.rs

@@ -6,13 +6,17 @@ use axum::{
 };
 use serde::{Deserialize, Serialize};
 use serde_json::value::RawValue;
-use windmill_common::{error::Error, worker::to_raw_value, DB};
+use windmill_common::{
+    error::Error,
+    triggers::{RunnableFormat, RunnableFormatVersion},
+    worker::to_raw_value,
+    DB,
+};
 use windmill_queue::PushArgsOwned;
 
 use crate::{
     args::{try_from_request_body, Body, RawWebhookArgs, WebhookArgs, WebhookArgsMetadata},
     db::ApiAuthed,
-    trigger_helpers::{RunnableFormat, RunnableFormatVersion},
 };
 
 pub struct RawHttpTriggerArgs(pub RawWebhookArgs);

backend/windmill-api/src/http_triggers.rs

@@ -41,10 +41,11 @@ use windmill_common::{
     db::UserDB,
     error::{self, JsonResult},
     s3_helpers::S3Object,
+    triggers::TriggerKind,
     utils::{not_found_if_none, paginate, require_admin, Pagination, StripPath},
     worker::CLOUD_HOSTED,
 };
-use windmill_queue::TriggerKind;
+use windmill_git_sync::handle_deployment_metadata;
 
 lazy_static::lazy_static! {
     static ref ROUTE_PATH_KEY_RE: regex::Regex = regex::Regex::new(r"/?:[-\w]+").unwrap();
@@ -397,6 +398,17 @@ async fn create_trigger(
 
     increase_trigger_version_and_commit(tx).await?;
 
+    handle_deployment_metadata(
+        &authed.email,
+        &authed.username,
+        &db,
+        &w_id,
+        windmill_git_sync::DeployedObject::HttpTrigger { path: ct.path.clone() },
+        Some(format!("HTTP trigger '{}' created", ct.path)),
+        true,
+    )
+    .await?;
+
     Ok((StatusCode::CREATED, format!("{}", ct.path)))
 }
 
@@ -544,20 +556,32 @@ async fn update_trigger(
         &mut *tx,
         &authed,
         "http_triggers.update",
-        ActionKind::Create,
+        ActionKind::Update,
         &w_id,
-        Some(path),
+        Some(&ct.path),
         None,
     )
     .await?;
 
     increase_trigger_version_and_commit(tx).await?;
 
-    Ok(path.to_string())
+    handle_deployment_metadata(
+        &authed.email,
+        &authed.username,
+        &db,
+        &w_id,
+        windmill_git_sync::DeployedObject::HttpTrigger { path: ct.path.clone() },
+        Some(format!("HTTP trigger '{}' updated", ct.path)),
+        true,
+    )
+    .await?;
+
+    Ok(ct.path.to_string())
 }
 
 async fn delete_trigger(
     authed: ApiAuthed,
+    Extension(db): Extension<DB>,
     Extension(user_db): Extension<UserDB>,
     Path((w_id, path)): Path<(String, StripPath)>,
 ) -> error::Result<String> {
@@ -587,6 +611,17 @@ async fn delete_trigger(
 
     increase_trigger_version_and_commit(tx).await?;
 
+    handle_deployment_metadata(
+        &authed.email,
+        &authed.username,
+        &db,
+        &w_id,
+        windmill_git_sync::DeployedObject::HttpTrigger { path: path.to_string() },
+        Some(format!("HTTP trigger '{}' deleted", path)),
+        true,
+    )
+    .await?;
+
     Ok(format!("HTTP trigger {path} deleted"))
 }
 

backend/windmill-api/src/jobs.rs

@@ -83,8 +83,6 @@ use windmill_common::{
     },
 };
 
-#[cfg(all(feature = "enterprise", feature = "parquet"))]
-use windmill_common::s3_helpers::OBJECT_STORE_CACHE_SETTINGS;
 #[cfg(feature = "prometheus")]
 use windmill_common::{METRICS_DEBUG_ENABLED, METRICS_ENABLED};
 
@@ -1058,7 +1056,7 @@ async fn get_logs_from_store(
     if log_offset > 0 {
         if let Some(file_index) = log_file_index.clone() {
             tracing::debug!("Getting logs from store: {file_index:?}");
-            if let Some(os) = OBJECT_STORE_CACHE_SETTINGS.read().await.clone() {
+            if let Some(os) = windmill_common::s3_helpers::get_object_store().await {
                 tracing::debug!("object store client present, streaming from there");
 
                 let logs = logs.to_string();
@@ -3473,7 +3471,7 @@ pub async fn run_flow_by_path(
             &authed,
             &db,
             &w_id,
-            RunnableId::from_flow_path(&flow_path.0),
+            RunnableId::from_flow_path(flow_path.to_path()),
             run_query.skip_preprocessor,
         )
         .await?;
@@ -3672,7 +3670,7 @@ pub async fn run_script_by_path(
             &authed,
             &db,
             &w_id,
-            RunnableId::from_script_path(&script_path.0),
+            RunnableId::from_script_path(script_path.to_path()),
             run_query.skip_preprocessor,
         )
         .await?;
@@ -4351,17 +4349,18 @@ pub async fn run_wait_result_job_by_path_get(
     let mut args = args.process_args(&authed, &db, &w_id, None).await?;
     args.body = args::Body::HashMap(payload_args);
 
+    let script_path = script_path.to_path();
+
     let args = args
         .to_args_from_runnable(
             &db,
             &w_id,
-            RunnableId::from_script_path(&script_path.0),
+            RunnableId::from_script_path(script_path),
             run_query.skip_preprocessor,
         )
         .await?;
 
     check_queue_too_long(&db, QUEUE_LIMIT_WAIT_RESULT.or(run_query.queue_limit)).await?;
-    let script_path = script_path.to_path();
     check_scopes(&authed, || format!("run:script/{script_path}"))?;
 
     let mut tx = user_db.clone().begin(&authed).await?;
@@ -4457,7 +4456,7 @@ pub async fn run_wait_result_flow_by_path_get(
         .to_args_from_runnable(
             &db,
             &w_id,
-            RunnableId::from_flow_path(&flow_path.0),
+            RunnableId::from_flow_path(flow_path.to_path()),
             run_query.skip_preprocessor,
         )
         .await?;
@@ -4482,7 +4481,7 @@ pub async fn run_wait_result_script_by_path(
             &authed,
             &db,
             &w_id,
-            RunnableId::from_script_path(&script_path.0),
+            RunnableId::from_script_path(script_path.to_path()),
             run_query.skip_preprocessor,
         )
         .await?;
@@ -4692,7 +4691,7 @@ pub async fn run_wait_result_flow_by_path(
             &authed,
             &db,
             &w_id,
-            RunnableId::from_flow_path(&flow_path.0),
+            RunnableId::from_flow_path(flow_path.to_path()),
             run_query.skip_preprocessor,
         )
         .await?;
@@ -4961,10 +4960,7 @@ async fn run_bundle_preview_script(
             uploaded = true;
 
             #[cfg(all(feature = "enterprise", feature = "parquet"))]
-            let object_store = windmill_common::s3_helpers::OBJECT_STORE_CACHE_SETTINGS
-                .read()
-                .await
-                .clone();
+            let object_store = windmill_common::s3_helpers::get_object_store().await;
 
             #[cfg(not(all(feature = "enterprise", feature = "parquet")))]
             let object_store: Option<()> = None;
@@ -5662,7 +5658,7 @@ async fn get_log_file(Path((_w_id, file_p)): Path<(String, String)>) -> error::R
     }
 
     #[cfg(all(feature = "enterprise", feature = "parquet"))]
-    if let Some(os) = OBJECT_STORE_CACHE_SETTINGS.read().await.clone() {
+    if let Some(os) = windmill_common::s3_helpers::get_object_store().await {
         let file = os
             .get(&object_store::path::Path::from(format!("logs/{file_p}")))
             .await;

backend/windmill-api/src/mqtt_triggers.rs

@@ -6,7 +6,7 @@ use crate::{
     trigger_helpers::TriggerJobArgs,
     users::fetch_api_authed,
 };
-use windmill_queue::TriggerKind;
+use windmill_git_sync::{handle_deployment_metadata, DeployedObject};
 
 use axum::{
     async_trait,
@@ -43,6 +43,7 @@ use windmill_audit::{audit_ee::audit_log, ActionKind};
 use windmill_common::{
     db::UserDB,
     error::{self, JsonResult},
+    triggers::TriggerKind,
     utils::{not_found_if_none, paginate, report_critical_error, Pagination, StripPath},
     worker::{to_raw_value, CLOUD_HOSTED},
     INSTANCE_NAME,
@@ -244,25 +245,25 @@ pub struct EditMqttTrigger {
 
 #[derive(Debug, Serialize, Deserialize, FromRow)]
 pub struct MqttTrigger {
-    mqtt_resource_path: String,
-    subscribe_topics: Vec<SqlxJson<SubscribeTopic>>,
-    v3_config: Option<SqlxJson<MqttV3Config>>,
-    v5_config: Option<SqlxJson<MqttV5Config>>,
-    client_id: Option<String>,
+    pub mqtt_resource_path: String,
+    pub subscribe_topics: Vec<SqlxJson<SubscribeTopic>>,
+    pub v3_config: Option<SqlxJson<MqttV3Config>>,
+    pub v5_config: Option<SqlxJson<MqttV5Config>>,
+    pub client_id: Option<String>,
     #[serde(skip_serializing_if = "Option::is_none")]
-    client_version: Option<MqttClientVersion>,
-    path: String,
-    script_path: String,
-    is_flow: bool,
-    workspace_id: String,
-    edited_by: String,
-    email: String,
-    edited_at: chrono::DateTime<chrono::Utc>,
-    extra_perms: Option<serde_json::Value>,
-    error: Option<String>,
-    server_id: Option<String>,
-    last_server_ping: Option<chrono::DateTime<chrono::Utc>>,
-    enabled: bool,
+    pub client_version: Option<MqttClientVersion>,
+    pub path: String,
+    pub script_path: String,
+    pub is_flow: bool,
+    pub workspace_id: String,
+    pub edited_by: String,
+    pub email: String,
+    pub edited_at: chrono::DateTime<chrono::Utc>,
+    pub extra_perms: Option<serde_json::Value>,
+    pub error: Option<String>,
+    pub server_id: Option<String>,
+    pub last_server_ping: Option<chrono::DateTime<chrono::Utc>>,
+    pub enabled: bool,
 }
 
 #[derive(Deserialize, Serialize)]
@@ -515,13 +516,14 @@ pub async fn test_mqtt_connection(
 
 pub async fn create_mqtt_trigger(
     authed: ApiAuthed,
+    Extension(db): Extension<DB>,
     Extension(user_db): Extension<UserDB>,
     Path(w_id): Path<String>,
     Json(new_mqtt_trigger): Json<NewMqttTrigger>,
 ) -> error::Result<(StatusCode, String)> {
     if *CLOUD_HOSTED {
         return Err(error::Error::BadRequest(
-            "Mqtt triggers are not supported on multi-tenant cloud, use dedicated cloud or self-host".to_string(),
+            "MQTT triggers are not supported on multi-tenant cloud, use dedicated cloud or self-host".to_string(),
         ));
     }
 
@@ -606,7 +608,18 @@ pub async fn create_mqtt_trigger(
 
     tx.commit().await?;
 
-    Ok((StatusCode::CREATED, path.to_string()))
+    handle_deployment_metadata(
+        &authed.email,
+        &authed.username,
+        &db,
+        &w_id,
+        DeployedObject::MqttTrigger { path: path.to_string() },
+        Some(format!("MQTT trigger '{}' created", path)),
+        true,
+    )
+    .await?;
+
+    Ok((StatusCode::CREATED, format!("{}", path.to_string())))
 }
 
 pub async fn list_mqtt_triggers(
@@ -719,6 +732,7 @@ pub async fn get_mqtt_trigger(
 
 pub async fn update_mqtt_trigger(
     authed: ApiAuthed,
+    Extension(db): Extension<DB>,
     Extension(user_db): Extension<UserDB>,
     Path((w_id, path)): Path<(String, StripPath)>,
     Json(mqtt_trigger): Json<EditMqttTrigger>,
@@ -787,7 +801,7 @@ pub async fn update_mqtt_trigger(
         &mut *tx,
         &authed,
         "mqtt_triggers.update",
-        ActionKind::Create,
+        ActionKind::Update,
         &w_id,
         Some(&path),
         None,
@@ -796,11 +810,23 @@ pub async fn update_mqtt_trigger(
 
     tx.commit().await?;
 
-    Ok(workspace_path.to_string())
+    handle_deployment_metadata(
+        &authed.email,
+        &authed.username,
+        &db,
+        &w_id,
+        DeployedObject::MqttTrigger { path: path.clone() },
+        Some(format!("MQTT trigger '{}' updated", path)),
+        true,
+    )
+    .await?;
+
+    Ok(path.to_string())
 }
 
 pub async fn delete_mqtt_trigger(
     authed: ApiAuthed,
+    Extension(db): Extension<DB>,
     Extension(user_db): Extension<UserDB>,
     Path((w_id, path)): Path<(String, StripPath)>,
 ) -> error::Result<String> {
@@ -834,7 +860,18 @@ pub async fn delete_mqtt_trigger(
 
     tx.commit().await?;
 
-    Ok(format!("Mqtt trigger {path} deleted"))
+    handle_deployment_metadata(
+        &authed.email,
+        &authed.username,
+        &db,
+        &w_id,
+        DeployedObject::MqttTrigger { path: path.to_string() },
+        Some(format!("MQTT trigger '{}' deleted", path)),
+        true,
+    )
+    .await?;
+
+    Ok(format!("MQTT trigger {path} deleted"))
 }
 
 pub async fn exists_mqtt_trigger(
@@ -864,6 +901,7 @@ pub async fn exists_mqtt_trigger(
 
 pub async fn set_enabled(
     authed: ApiAuthed,
+    Extension(db): Extension<DB>,
     Extension(user_db): Extension<UserDB>,
     Path((w_id, path)): Path<(String, StripPath)>,
     Json(payload): Json<SetEnabled>,
@@ -913,6 +951,17 @@ pub async fn set_enabled(
 
     tx.commit().await?;
 
+    handle_deployment_metadata(
+        &authed.email,
+        &authed.username,
+        &db,
+        &w_id,
+        DeployedObject::MqttTrigger { path: path.to_string() },
+        Some(format!("MQTT trigger '{}' updated", path)),
+        true,
+    )
+    .await?;
+
     Ok(format!(
         "successfully updated mqtt trigger at path {} to status {}",
         path, payload.enabled

backend/windmill-api/src/postgres_triggers/handler.rs

@@ -25,9 +25,10 @@ use windmill_common::error::Error;
 use windmill_common::{
     db::UserDB,
     error::{self, JsonResult, Result},
-    utils::{not_found_if_none, paginate, Pagination, StripPath},
+    utils::{not_found_if_none, paginate, Pagination, StripPath, empty_as_none},
     worker::CLOUD_HOSTED,
 };
+use windmill_git_sync::{handle_deployment_metadata, DeployedObject};
 
 use super::{
     create_logical_replication_slot_query, create_publication_query, drop_publication_query,
@@ -45,7 +46,8 @@ pub struct Postgres {
     pub dbname: String,
     #[serde(default)]
     pub sslmode: String,
-    pub root_certificate_pem: String,
+    #[serde(default, deserialize_with = "empty_as_none")]
+    pub root_certificate_pem: Option<String>,
 }
 
 #[derive(Debug, Clone, FromRow, Serialize, Deserialize)]
@@ -453,6 +455,17 @@ pub async fn create_postgres_trigger(
 
     tx.commit().await?;
 
+    handle_deployment_metadata(
+        &authed.email,
+        &authed.username,
+        &db,
+        &w_id,
+        DeployedObject::PostgresTrigger { path: path.to_string() },
+        Some(format!("Postgres trigger '{}' created", path)),
+        true,
+    )
+    .await?;
+
     Ok((StatusCode::CREATED, path.to_string()))
 }
 
@@ -1158,7 +1171,7 @@ pub async fn update_postgres_trigger(
         &mut *tx,
         &authed,
         "postgres_triggers.update",
-        ActionKind::Create,
+        ActionKind::Update,
         &w_id,
         Some(&path),
         None,
@@ -1167,11 +1180,23 @@ pub async fn update_postgres_trigger(
 
     tx.commit().await?;
 
+    handle_deployment_metadata(
+        &authed.email,
+        &authed.username,
+        &db,
+        &w_id,
+        DeployedObject::PostgresTrigger { path: path.to_string() },
+        Some(format!("Postgres trigger '{}' updated", path)),
+        true,
+    )
+    .await?;
+
     Ok(workspace_path.to_string())
 }
 
 pub async fn delete_postgres_trigger(
     authed: ApiAuthed,
+    Extension(db): Extension<DB>,
     Extension(user_db): Extension<UserDB>,
     Path((w_id, path)): Path<(String, StripPath)>,
 ) -> Result<String> {
@@ -1203,6 +1228,17 @@ pub async fn delete_postgres_trigger(
 
     tx.commit().await?;
 
+    handle_deployment_metadata(
+        &authed.email,
+        &authed.username,
+        &db,
+        &w_id,
+        DeployedObject::PostgresTrigger { path: path.to_string() },
+        Some(format!("Postgres trigger '{}' deleted", path)),
+        true,
+    )
+    .await?;
+
     Ok(format!("Postgres trigger {path} deleted"))
 }
 
@@ -1231,6 +1267,7 @@ pub async fn exists_postgres_trigger(
 
 pub async fn set_enabled(
     authed: ApiAuthed,
+    Extension(db): Extension<DB>,
     Extension(user_db): Extension<UserDB>,
     Path((w_id, path)): Path<(String, StripPath)>,
     Json(payload): Json<SetEnabled>,
@@ -1279,6 +1316,17 @@ pub async fn set_enabled(
 
     tx.commit().await?;
 
+    handle_deployment_metadata(
+        &authed.email,
+        &authed.username,
+        &db,
+        &w_id,
+        DeployedObject::PostgresTrigger { path: path.to_string() },
+        Some(format!("Postgres trigger '{}' updated", path)),
+        true,
+    )
+    .await?;
+
     Ok(format!(
         "succesfully updated postgres trigger at path {} to status {}",
         path, payload.enabled

backend/windmill-api/src/postgres_triggers/mod.rs

@@ -84,8 +84,8 @@ pub async fn get_raw_postgres_connection(
             }
         };
 
-        let options = if !db.root_certificate_pem.is_empty() {
-            options.ssl_root_cert_from_pem(db.root_certificate_pem.as_bytes().to_vec())
+        let options = if let Some(root_certificate_pem) = &db.root_certificate_pem {
+            options.ssl_root_cert_from_pem(root_certificate_pem.as_bytes().to_vec())
         } else {
             options
         };
@@ -96,7 +96,6 @@ pub async fn get_raw_postgres_connection(
             options
         }
     };
-
     Ok(PgConnection::connect_with(&options).await?)
 }
 

backend/windmill-api/src/postgres_triggers/trigger.rs

@@ -15,22 +15,22 @@ use crate::{
     trigger_helpers::TriggerJobArgs,
     users::fetch_api_authed,
 };
-use windmill_queue::TriggerKind;
 
 use bytes::{BufMut, Bytes, BytesMut};
 use chrono::TimeZone;
 use futures::{pin_mut, SinkExt, StreamExt};
-use native_tls::TlsConnector;
+use native_tls::{Certificate, TlsConnector};
 use pg_escape::{quote_identifier, quote_literal};
 use rand::seq::SliceRandom;
-use rust_postgres::{config::SslMode, Client, Config, CopyBothDuplex, SimpleQueryMessage};
+use rust_postgres::{config::SslMode, Client, Config, CopyBothDuplex, NoTls, SimpleQueryMessage};
 use rust_postgres_native_tls::MakeTlsConnector;
 use serde::Deserialize;
 use serde_json::value::RawValue;
 use sqlx::types::Json as SqlxJson;
 
 use windmill_common::{
-    db::UserDB, error, utils::report_critical_error, worker::to_raw_value, INSTANCE_NAME,
+    db::UserDB, error, triggers::TriggerKind, utils::report_critical_error, worker::to_raw_value,
+    INSTANCE_NAME,
 };
 
 use super::{
@@ -79,6 +79,45 @@ enum Error {
     Tls(#[from] native_tls::Error),
 }
 
+fn build_tls_connector(
+    ssl_mode: SslMode,
+    root_certificate_pem: Option<&String>,
+) -> Result<Option<MakeTlsConnector>, Error> {
+    let get_tls_builder_for_verify = |root_certificate: Option<&String>| {
+        let mut builder = TlsConnector::builder();
+        if let Some(root_certificate) = root_certificate {
+            let root_certificate_pem = Certificate::from_pem(root_certificate.as_bytes()).map_err(|e| {
+                Error::Common(error::Error::BadConfig(format!("Invalid Certs: {e:#}")))
+            })?;
+            builder.add_root_certificate(root_certificate_pem);
+        }
+        Ok::<_, Error>(builder)
+    };
+    let connector = match ssl_mode {
+        SslMode::Disable => return Ok(None),
+        SslMode::Require | SslMode::Prefer => {
+            let mut builder = TlsConnector::builder();
+            builder.danger_accept_invalid_certs(true);
+            builder.danger_accept_invalid_hostnames(true);
+            builder
+        }
+
+        SslMode::VerifyCa => {
+            let mut builder = get_tls_builder_for_verify(root_certificate_pem)?;
+            builder.danger_accept_invalid_hostnames(true);
+            builder
+        }
+
+        SslMode::VerifyFull => {
+            let builder = get_tls_builder_for_verify(root_certificate_pem)?;
+            builder
+        }
+        _ => unreachable!(),
+    };
+
+    Ok(Some(MakeTlsConnector::new(connector.build()?)))
+}
+
 pub struct PostgresSimpleClient(Client);
 
 impl PostgresSimpleClient {
@@ -112,20 +151,27 @@ impl PostgresSimpleClient {
             config.password(&database.password);
         }
 
-        if !database.root_certificate_pem.is_empty() {
-            config.ssl_root_cert(database.root_certificate_pem.as_bytes());
-        }
+        let connector = build_tls_connector(ssl_mode, database.root_certificate_pem.as_ref())?;
 
-        let connector = MakeTlsConnector::new(TlsConnector::new()?);
-
-        let (client, connection) = config.connect(connector).await?;
-
-        tokio::spawn(async move {
-            if let Err(e) = connection.await {
-                tracing::debug!("{:#?}", e);
-            };
-            tracing::info!("Successfully Connected into database");
-        });
+        let client = if let Some(connector) = connector {
+            let (client, connection) = config.connect(connector).await?;
+            tokio::spawn(async move {
+                if let Err(e) = connection.await {
+                    tracing::debug!("{:#?}", e);
+                };
+                tracing::info!("Successfully Connected into database");
+            });
+            client
+        } else {
+            let (client, connection) = config.connect(NoTls).await?;
+            tokio::spawn(async move {
+                if let Err(e) = connection.await {
+                    tracing::debug!("{:#?}", e);
+                };
+                tracing::info!("Successfully Connected into database");
+            });
+            client
+        };
 
         Ok(PostgresSimpleClient(client))
     }

backend/windmill-api/src/scripts.rs

@@ -223,7 +223,8 @@ async fn list_scripts(
             "draft_only",
             "ws_error_handler_muted",
             "no_main_func",
-            "codebase IS NOT NULL as use_codebase"
+            "codebase IS NOT NULL as use_codebase",
+            "kind"
         ])
         .left()
         .join("favorite")
@@ -298,7 +299,9 @@ async fn list_scripts(
     if let Some(it) = &lq.is_template {
         sqlb.and_where_eq("is_template", it);
     }
-    if let Some(lowercased_kinds) = lowercased_kinds {
+    if authed.is_operator {
+        sqlb.and_where_eq("kind", quote("script"));
+    } else if let Some(lowercased_kinds) = lowercased_kinds {
         let safe_kinds = lowercased_kinds
             .into_iter()
             .map(sql_builder::quote)
@@ -407,10 +410,7 @@ async fn create_snapshot_script(
             uploaded = true;
 
             #[cfg(all(feature = "enterprise", feature = "parquet"))]
-            let object_store = windmill_common::s3_helpers::OBJECT_STORE_CACHE_SETTINGS
-                .read()
-                .await
-                .clone();
+            let object_store = windmill_common::s3_helpers::get_object_store().await;
 
             #[cfg(not(all(feature = "enterprise", feature = "parquet")))]
             let object_store: Option<()> = None;
@@ -683,36 +683,40 @@ async fn create_script_internal<'c>(
 
     let validate_schema = should_validate_schema(&ns.content, &ns.language);
 
-    let (no_main_func, has_preprocessor) = match lang {
-        ScriptLang::Bun | ScriptLang::Bunnative | ScriptLang::Deno | ScriptLang::Nativets => {
-            let args = windmill_parser_ts::parse_deno_signature(&ns.content, true, true, None);
-            match args {
-                Ok(args) => (args.no_main_func, args.has_preprocessor),
-                Err(e) => {
-                    tracing::warn!(
-                        "Error parsing deno signature when deploying script {}: {:?}",
-                        ns.path,
-                        e
-                    );
-                    (None, None)
+    let (no_main_func, has_preprocessor) = if matches!(ns.kind, Some(ScriptKind::Preprocessor)) {
+        (ns.no_main_func, ns.has_preprocessor)
+    } else {
+        match lang {
+            ScriptLang::Bun | ScriptLang::Bunnative | ScriptLang::Deno | ScriptLang::Nativets => {
+                let args = windmill_parser_ts::parse_deno_signature(&ns.content, true, true, None);
+                match args {
+                    Ok(args) => (args.no_main_func, args.has_preprocessor),
+                    Err(e) => {
+                        tracing::warn!(
+                            "Error parsing deno signature when deploying script {}: {:?}",
+                            ns.path,
+                            e
+                        );
+                        (None, None)
+                    }
                 }
             }
-        }
-        ScriptLang::Python3 => {
-            let args = windmill_parser_py::parse_python_signature(&ns.content, None, true);
-            match args {
-                Ok(args) => (args.no_main_func, args.has_preprocessor),
-                Err(e) => {
-                    tracing::warn!(
-                        "Error parsing python signature when deploying script {}: {:?}",
-                        ns.path,
-                        e
-                    );
-                    (None, None)
+            ScriptLang::Python3 => {
+                let args = windmill_parser_py::parse_python_signature(&ns.content, None, true);
+                match args {
+                    Ok(args) => (args.no_main_func, args.has_preprocessor),
+                    Err(e) => {
+                        tracing::warn!(
+                            "Error parsing python signature when deploying script {}: {:?}",
+                            ns.path,
+                            e
+                        );
+                        (None, None)
+                    }
                 }
             }
+            _ => (ns.no_main_func, ns.has_preprocessor),
         }
-        _ => (ns.no_main_func, ns.has_preprocessor),
     };
 
     sqlx::query!(
@@ -1320,10 +1324,12 @@ async fn raw_script_by_path_internal(
             w_id
         )
         .fetch_one(&db)
-        .await?;
-        if exists.unwrap_or(false) {
+        .await?
+        .unwrap_or(false);
+
+        if exists {
             return Err(Error::NotFound(format!(
-                "Script {path} not visible to {} but exists",
+                "Script {path} exists but {} does not have permissions to access it",
                 authed.username
             )));
         }

backend/windmill-api/src/service_logs.rs

@@ -98,10 +98,7 @@ async fn get_log_file(
     require_devops_role(&db, &email).await?;
     let path = path.to_path();
     #[cfg(feature = "parquet")]
-    let s3_client = windmill_common::s3_helpers::OBJECT_STORE_CACHE_SETTINGS
-        .read()
-        .await
-        .clone();
+    let s3_client = windmill_common::s3_helpers::get_object_store().await;
     #[cfg(feature = "parquet")]
     if let Some(s3_client) = s3_client {
         let path = format!("{}{}", windmill_common::tracing_init::LOGS_SERVICE, path);

backend/windmill-api/src/settings.rs

@@ -120,12 +120,15 @@ use windmill_common::s3_helpers::build_object_store_from_settings;
 #[cfg(feature = "parquet")]
 pub async fn test_s3_bucket(
     _authed: ApiAuthed,
+    Extension(db): Extension<DB>,
     Json(test_s3_bucket): Json<ObjectSettings>,
 ) -> error::Result<String> {
     use bytes::Bytes;
     use futures::StreamExt;
 
-    let client = build_object_store_from_settings(test_s3_bucket).await?;
+    let client = build_object_store_from_settings(test_s3_bucket, Some(&db))
+        .await?
+        .store;
 
     let mut list = client.list(Some(&object_store::path::Path::from("".to_string())));
     let first_file = list.next().await;

backend/windmill-api/src/trigger_helpers.rs

@@ -1,4 +1,3 @@
-use quick_cache::sync::Cache;
 use serde::Deserialize;
 use serde_json::value::RawValue;
 use std::collections::HashMap;
@@ -6,31 +5,19 @@ use windmill_common::{
     error::Result,
     flows::FlowModuleValue,
     get_latest_deployed_hash_for_path, get_latest_flow_version_info_for_path,
-    scripts::{ScriptHash, ScriptLang},
+    jobs::get_has_preprocessor_from_content_and_lang,
+    scripts::{get_full_hub_script_by_path, ScriptHash, ScriptLang},
+    triggers::{
+        HubOrWorkspaceId, RunnableFormat, RunnableFormatVersion, TriggerKind,
+        RUNNABLE_FORMAT_VERSION_CACHE,
+    },
+    utils::StripPath,
     worker::to_raw_value,
     FlowVersionInfo,
 };
-use windmill_queue::{PushArgsOwned, TriggerKind};
+use windmill_queue::PushArgsOwned;
 
-use crate::db::DB;
-
-type RunnableFormatCacheKey = (String, i64, TriggerKind);
-
-lazy_static::lazy_static! {
-    pub static ref RUNNABLE_FORMAT_VERSION_CACHE: Cache<RunnableFormatCacheKey, RunnableFormat> = Cache::new(1000);
-}
-
-#[derive(Debug, Clone, Eq, PartialEq, Hash, Copy)]
-pub struct RunnableFormat {
-    pub version: RunnableFormatVersion,
-    pub has_preprocessor: bool,
-}
-
-#[derive(Debug, Clone, Eq, PartialEq, Hash, Copy)]
-pub enum RunnableFormatVersion {
-    V1,
-    V2,
-}
+use crate::{db::DB, HTTP_CLIENT};
 
 struct ScriptInfo {
     has_preprocessor: Option<bool>,
@@ -53,6 +40,7 @@ struct PartialSchema {
 pub enum RunnableId {
     FlowPath(String),
     ScriptId(ScriptId),
+    HubScript(String),
 }
 
 impl RunnableId {
@@ -61,7 +49,11 @@ impl RunnableId {
     }
 
     pub fn from_script_path(path: &str) -> Self {
-        Self::ScriptId(ScriptId::ScriptPath(path.to_string()))
+        if path.starts_with("hub/") {
+            Self::HubScript(path.to_string())
+        } else {
+            Self::ScriptId(ScriptId::ScriptPath(path.to_string()))
+        }
     }
 
     pub fn from_flow_path(path: &str) -> Self {
@@ -156,6 +148,33 @@ struct FlowInfo {
     schema: Option<sqlx::types::Json<PartialSchema>>,
 }
 
+fn get_preprocessor_args_from_content_and_language(
+    content: &str,
+    language: &ScriptLang,
+) -> Result<Option<Vec<windmill_parser::Arg>>> {
+    let args = match language {
+        ScriptLang::Bun | ScriptLang::Bunnative | ScriptLang::Deno | ScriptLang::Nativets => {
+            let args = windmill_parser_ts::parse_deno_signature(
+                &content,
+                true,
+                false,
+                Some("preprocessor".to_string()),
+            )?;
+            Some(args.args)
+        }
+        ScriptLang::Python3 => {
+            let args = windmill_parser_py::parse_python_signature(
+                &content,
+                Some("preprocessor".to_string()),
+                false,
+            )?;
+            Some(args.args)
+        }
+        _ => None,
+    };
+    Ok(args)
+}
+
 pub async fn get_runnable_format(
     runnable_id: RunnableId,
     workspace_id: &str,
@@ -163,11 +182,65 @@ pub async fn get_runnable_format(
     trigger_kind: &TriggerKind,
 ) -> Result<RunnableFormat> {
     let (key, preprocessor_info) = match runnable_id {
+        RunnableId::HubScript(path) => {
+            let Some(version) = path.split("/").nth(1) else {
+                return Err(windmill_common::error::Error::internal_err(
+                    "Invalid hub script path".to_string(),
+                ));
+            };
+
+            let version = match version.parse::<i64>() {
+                Ok(version) => version,
+                Err(_) => {
+                    return Err(windmill_common::error::Error::internal_err(
+                        "Invalid hub script version".to_string(),
+                    ));
+                }
+            };
+
+            let key = (HubOrWorkspaceId::Hub, version, trigger_kind.clone());
+
+            let runnable_format = RUNNABLE_FORMAT_VERSION_CACHE.get(&key);
+
+            if let Some(runnable_format) = runnable_format {
+                tracing::debug!("Using cached runnable format for hub script {path}");
+                return Ok(runnable_format);
+            }
+
+            let hub_script =
+                get_full_hub_script_by_path(StripPath(path.to_string()), &HTTP_CLIENT, Some(db))
+                    .await?;
+
+            let has_preprocessor = get_has_preprocessor_from_content_and_lang(
+                &hub_script.content,
+                &hub_script.language,
+            )?;
+
+            let partial_schema = serde_json::from_str(hub_script.schema.get())?;
+
+            (
+                key,
+                if has_preprocessor {
+                    PreprocessorInfo::Preprocessor {
+                        content: hub_script.content,
+                        language: hub_script.language,
+                    }
+                } else {
+                    PreprocessorInfo::NoPreprocessor {
+                        schema: Some(sqlx::types::Json(partial_schema)),
+                    }
+                },
+            )
+        }
         RunnableId::FlowPath(path) => {
             let FlowVersionInfo { version, .. } =
                 get_latest_flow_version_info_for_path(db, workspace_id, &path, true).await?;
 
-            let key = (workspace_id.to_string(), version, trigger_kind.clone());
+            let key = (
+                HubOrWorkspaceId::WorkspaceId(workspace_id.to_string()),
+                version,
+                trigger_kind.clone(),
+            );
 
             let runnable_format = RUNNABLE_FORMAT_VERSION_CACHE.get(&key);
 
@@ -181,11 +254,14 @@ pub async fn get_runnable_format(
                 "SELECT
                     value->'preprocessor_module'->'value' as \"preprocessor_module: _\",
                     schema as \"schema: _\"
-                FROM flow 
-                WHERE workspace_id = $1
-                    AND path = $2",
+                FROM flow_version
+                WHERE 
+                    path = $1
+                    AND workspace_id = $2
+                ORDER BY created_at DESC
+                LIMIT 1",
+                path,
                 workspace_id,
-                path
             )
             .fetch_one(db)
             .await?;
@@ -227,7 +303,11 @@ pub async fn get_runnable_format(
         }
         RunnableId::ScriptId(script_id) => {
             let hash = script_id.get_script_hash(workspace_id, db).await?;
-            let key = (workspace_id.to_string(), hash, trigger_kind.clone());
+            let key = (
+                HubOrWorkspaceId::WorkspaceId(workspace_id.to_string()),
+                hash,
+                trigger_kind.clone(),
+            );
             let runnable_format = RUNNABLE_FORMAT_VERSION_CACHE.get(&key);
 
             if let Some(runnable_format) = runnable_format {
@@ -256,30 +336,7 @@ pub async fn get_runnable_format(
 
     let runnable_format = match preprocessor_info {
         PreprocessorInfo::Preprocessor { content, language } => {
-            let args = match language {
-                ScriptLang::Bun
-                | ScriptLang::Bunnative
-                | ScriptLang::Deno
-                | ScriptLang::Nativets => {
-                    let args = windmill_parser_ts::parse_deno_signature(
-                        &content,
-                        true,
-                        false,
-                        Some("preprocessor".to_string()),
-                    )?;
-                    Some(args.args)
-                }
-                ScriptLang::Python3 => {
-                    let args = windmill_parser_py::parse_python_signature(
-                        &content,
-                        Some("preprocessor".to_string()),
-                        false,
-                    )?;
-                    Some(args.args)
-                }
-                _ => None,
-            };
-
+            let args = get_preprocessor_args_from_content_and_language(&content, &language)?;
             runnable_format_from_preprocessor_args(args)
         }
         PreprocessorInfo::NoPreprocessor { schema } => {

backend/windmill-api/src/users.rs

@@ -334,6 +334,7 @@ pub struct NewUser {
     pub super_admin: bool,
     pub name: Option<String>,
     pub company: Option<String>,
+    pub skip_email: Option<bool>,
 }
 
 #[derive(Deserialize)]

backend/windmill-api/src/utils.rs

@@ -5,6 +5,9 @@
  * Please see the included NOTICE for copyright information and
  * LICENSE-AGPL for a copy of the license.
  */
+
+use std::fmt::Display;
+
 use axum::{body::Body, response::Response};
 use regex::Regex;
 use serde::{Deserialize, Deserializer};
@@ -35,6 +38,16 @@ pub enum RunnableKind {
     Flow,
 }
 
+impl Display for RunnableKind {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+        let runnable_kind = match self {
+            RunnableKind::Script => "script",
+            RunnableKind::Flow => "flow"
+        };
+        write!(f, "{}", runnable_kind)
+    }
+}
+
 pub async fn require_super_admin(db: &DB, email: &str) -> error::Result<()> {
     let is_admin = is_super_admin_email(db, email).await?;
 

backend/windmill-api/src/websocket_triggers.rs

@@ -24,14 +24,14 @@ use windmill_audit::{audit_ee::audit_log, ActionKind};
 use windmill_common::{
     db::UserDB,
     error::{self, to_anyhow, JsonResult},
+    triggers::TriggerKind,
     utils::{not_found_if_none, paginate, report_critical_error, Pagination, StripPath},
     worker::{to_raw_value, CLOUD_HOSTED},
     INSTANCE_NAME,
 };
+use windmill_git_sync::handle_deployment_metadata;
 use windmill_queue::PushArgsOwned;
 
-use windmill_queue::TriggerKind;
-
 use crate::{
     capture::{insert_capture_payload, WebsocketTriggerConfig},
     db::{ApiAuthed, DB},
@@ -195,6 +195,7 @@ async fn get_websocket_trigger(
 
 async fn create_websocket_trigger(
     authed: ApiAuthed,
+    Extension(db): Extension<DB>,
     Extension(user_db): Extension<UserDB>,
     Path(w_id): Path<String>,
     Json(ct): Json<NewWebsocketTrigger>,
@@ -244,11 +245,23 @@ async fn create_websocket_trigger(
 
     tx.commit().await?;
 
+    handle_deployment_metadata(
+        &authed.email,
+        &authed.username,
+        &db,
+        &w_id,
+        windmill_git_sync::DeployedObject::WebsocketTrigger { path: ct.path.clone() },
+        Some(format!("WebSocket trigger '{}' created", ct.path)),
+        true,
+    )
+    .await?;
+
     Ok((StatusCode::CREATED, format!("{}", ct.path)))
 }
 
 async fn update_websocket_trigger(
     authed: ApiAuthed,
+    Extension(db): Extension<DB>,
     Extension(user_db): Extension<UserDB>,
     Path((w_id, path)): Path<(String, StripPath)>,
     Json(ct): Json<EditWebsocketTrigger>,
@@ -287,16 +300,27 @@ async fn update_websocket_trigger(
         &mut *tx,
         &authed,
         "websocket_triggers.update",
-        ActionKind::Create,
+        ActionKind::Update,
         &w_id,
-        Some(path),
+        Some(&ct.path),
         None,
     )
     .await?;
 
     tx.commit().await?;
 
-    Ok(path.to_string())
+    handle_deployment_metadata(
+        &authed.email,
+        &authed.username,
+        &db,
+        &w_id,
+        windmill_git_sync::DeployedObject::WebsocketTrigger { path: ct.path.clone() },
+        Some(format!("WebSocket trigger '{}' updated", ct.path)),
+        true,
+    )
+    .await?;
+
+    Ok(ct.path.to_string())
 }
 
 #[derive(Deserialize)]
@@ -306,6 +330,7 @@ pub struct SetEnabled {
 
 pub async fn set_enabled(
     authed: ApiAuthed,
+    Extension(db): Extension<DB>,
     Extension(user_db): Extension<UserDB>,
     Path((w_id, path)): Path<(String, StripPath)>,
     Json(payload): Json<SetEnabled>,
@@ -339,6 +364,17 @@ pub async fn set_enabled(
 
     tx.commit().await?;
 
+    handle_deployment_metadata(
+        &authed.email,
+        &authed.username,
+        &db,
+        &w_id,
+        windmill_git_sync::DeployedObject::WebsocketTrigger { path: path.to_string() },
+        Some(format!("WebSocket trigger '{}' updated", path)),
+        true,
+    )
+    .await?;
+
     Ok(format!(
         "succesfully updated WebSocket trigger at path {} to status {}",
         path, payload.enabled
@@ -347,6 +383,7 @@ pub async fn set_enabled(
 
 async fn delete_websocket_trigger(
     authed: ApiAuthed,
+    Extension(db): Extension<DB>,
     Extension(user_db): Extension<UserDB>,
     Path((w_id, path)): Path<(String, StripPath)>,
 ) -> error::Result<String> {
@@ -373,6 +410,17 @@ async fn delete_websocket_trigger(
 
     tx.commit().await?;
 
+    handle_deployment_metadata(
+        &authed.email,
+        &authed.username,
+        &db,
+        &w_id,
+        windmill_git_sync::DeployedObject::WebsocketTrigger { path: path.to_string() },
+        Some(format!("WebSocket trigger '{}' deleted", path)),
+        true,
+    )
+    .await?;
+
     Ok(format!("WebSocket trigger {path} deleted"))
 }
 

backend/windmill-api/src/workspaces_export.rs

@@ -360,6 +360,7 @@ pub(crate) async fn tarball_workspace(
                 ScriptLang::Bigquery => "bq.sql",
                 ScriptLang::Snowflake => "sf.sql",
                 ScriptLang::Mssql => "ms.sql",
+                ScriptLang::DuckDb => "duckdb.sql",
                 ScriptLang::Graphql => "gql",
                 ScriptLang::Nativets => "fetch.ts",
                 ScriptLang::Bun | ScriptLang::Bunnative => {
@@ -763,6 +764,45 @@ pub(crate) async fn tarball_workspace(
                     .await?;
             }
         }
+
+        #[cfg(all(feature = "enterprise", feature = "mqtt_trigger"))]
+        {
+            let mqtt_triggers = sqlx::query_as!(
+                crate::mqtt_triggers::MqttTrigger,
+                r#"
+                SELECT
+                    mqtt_resource_path,
+                    subscribe_topics as "subscribe_topics: _",
+                    v3_config as "v3_config: _",
+                    v5_config as "v5_config: _",
+                    client_version AS "client_version: _",
+                    client_id,
+                    workspace_id,
+                    path,
+                    script_path,
+                    is_flow,
+                    edited_by,
+                    email,
+                    edited_at,
+                    server_id,
+                    last_server_ping,
+                    extra_perms,
+                    error,
+                    enabled
+                FROM 
+                    mqtt_trigger
+                "#,
+            )
+            .fetch_all(&mut *tx)
+            .await?;
+
+            for trigger in mqtt_triggers {
+                let trigger_str = &to_string_without_metadata(&trigger, false, None).unwrap();
+                archive
+                    .write_to_archive(&trigger_str, &format!("{}.mqtt_trigger.json", trigger.path))
+                    .await?;
+            }
+        }
     }
 
     if include_users.unwrap_or(false) {

backend/windmill-common/Cargo.toml

@@ -12,14 +12,14 @@ tantivy = []
 prometheus = ["dep:prometheus"]
 loki = ["dep:tracing-loki"]
 benchmark = []
-parquet = ["dep:object_store", "dep:aws-config", "dep:aws-sdk-sts", "dep:datafusion"]
+parquet = ["dep:object_store", "dep:aws-config", "dep:aws-sdk-sts", "dep:aws-smithy-types-convert", "dep:datafusion"]
 aws_auth = ["dep:aws-sdk-sts", "dep:aws-config"]
 otel = ["dep:opentelemetry-semantic-conventions", "dep:opentelemetry-otlp", "dep:opentelemetry_sdk", 
     "dep:opentelemetry", "dep:tracing-opentelemetry", "dep:opentelemetry-appender-tracing", "dep:tonic"]
 smtp = ["dep:mail-send"]
 scoped_cache = []
 cloud = []
-
+openidconnect = ["dep:openidconnect"]
 [lib]
 name = "windmill_common"
 path = "src/lib.rs"
@@ -62,6 +62,7 @@ object_store = { workspace = true, optional = true }
 prometheus = { workspace = true, optional = true }
 aws-config = { workspace = true, optional = true }
 aws-sdk-sts = { workspace = true, optional = true }
+aws-smithy-types-convert = { workspace = true, optional = true }
 indexmap.workspace = true
 bytes.workspace = true
 mail-send = { workspace = true, optional = true }
@@ -71,8 +72,13 @@ const_format.workspace = true
 crc.workspace = true
 windmill-macros.workspace = true
 windmill-parser-sql.workspace = true
+windmill-parser-ts.workspace = true
+windmill-parser-py.workspace = true
 jsonwebtoken.workspace = true
 backon.workspace = true
+openidconnect = { workspace = true, optional = true }
+strum.workspace = true
+strum_macros.workspace = true
 
 semver.workspace = true
 croner = "2.0.6"

backend/windmill-common/src/client.rs

@@ -0,0 +1,244 @@
+use anyhow::Context;
+use reqwest::{Body, Response};
+use serde::de::DeserializeOwned;
+
+use crate::{
+    error::{self, to_anyhow},
+    s3_helpers::{DuckdbConnectionSettingsQueryV2, DuckdbConnectionSettingsResponse},
+    utils::HTTP_CLIENT,
+};
+
+#[derive(Clone)]
+pub struct AuthedClient {
+    pub base_internal_url: String,
+    pub workspace: String,
+    pub token: String,
+    pub force_client: Option<reqwest::Client>,
+}
+
+impl AuthedClient {
+    pub async fn get(&self, url: &str, query: Vec<(&str, String)>) -> anyhow::Result<Response> {
+        self.force_client
+            .as_ref()
+            .unwrap_or(&HTTP_CLIENT)
+            .get(url)
+            .query(&query)
+            .header(
+                reqwest::header::ACCEPT,
+                reqwest::header::HeaderValue::from_static("application/json"),
+            )
+            .header(
+                reqwest::header::AUTHORIZATION,
+                reqwest::header::HeaderValue::from_str(&format!("Bearer {}", self.token))?,
+            )
+            .send()
+            .await
+            .map_err(|e| {
+                tracing::error!("Error executing get request from authed http client to {url} with query {query:?}: {e}");
+                anyhow::anyhow!("Error executing get request from authed http client to {url} with query {query:?}: {e}")
+            })
+    }
+
+    pub async fn get_id_token(&self, audience: &str) -> anyhow::Result<String> {
+        let url = format!(
+            "{}/api/w/{}/oidc/token/{}",
+            self.base_internal_url, self.workspace, audience
+        );
+        let response = self.get(&url, vec![]).await?;
+        match response.status().as_u16() {
+            200u16 => Ok(response
+                .json::<String>()
+                .await
+                .context("decoding oidc token as json string")?),
+            _ => Err(anyhow::anyhow!(response.text().await.unwrap_or_default())),
+        }
+    }
+
+    pub async fn get_resource_value<T: DeserializeOwned>(&self, path: &str) -> anyhow::Result<T> {
+        let url = format!(
+            "{}/api/w/{}/resources/get_value/{}",
+            self.base_internal_url, self.workspace, path
+        );
+        let response = self.get(&url, vec![]).await?;
+        match response.status().as_u16() {
+            200u16 => Ok(response
+                .json::<T>()
+                .await
+                .context("decoding resource value as json")?),
+            _ => Err(anyhow::anyhow!(response.text().await.unwrap_or_default())),
+        }
+    }
+
+    pub async fn get_variable_value(&self, path: &str) -> anyhow::Result<String> {
+        let url = format!(
+            "{}/api/w/{}/variables/get_value/{}",
+            self.base_internal_url, self.workspace, path
+        );
+        let response = self.get(&url, vec![]).await?;
+        match response.status().as_u16() {
+            200u16 => Ok(response
+                .json::<String>()
+                .await
+                .context("decoding variable value as json")?),
+            _ => Err(anyhow::anyhow!(response.text().await.unwrap_or_default())),
+        }
+    }
+
+    pub async fn get_resource_value_interpolated<T: DeserializeOwned>(
+        &self,
+        path: &str,
+        job_id: Option<String>,
+    ) -> anyhow::Result<T> {
+        let url = format!(
+            "{}/api/w/{}/resources/get_value_interpolated/{}",
+            self.base_internal_url, self.workspace, path
+        );
+        let mut query = Vec::with_capacity(1usize);
+        if let Some(v) = &job_id {
+            query.push(("job_id", v.to_string()));
+        }
+        let response = self.get(&url, query).await?;
+        match response.status().as_u16() {
+            200u16 => Ok(response
+                .json::<T>()
+                .await
+                .context("decoding interpolated resource value as json")?),
+            _ => Err(anyhow::anyhow!(response.text().await.unwrap_or_default())),
+        }
+    }
+
+    pub async fn get_completed_job_result<T: DeserializeOwned>(
+        &self,
+        path: &str,
+        json_path: Option<String>,
+    ) -> anyhow::Result<T> {
+        let url = format!(
+            "{}/api/w/{}/jobs_u/completed/get_result/{}",
+            self.base_internal_url, self.workspace, path
+        );
+        let query = if let Some(json_path) = json_path {
+            vec![("json_path", json_path)]
+        } else {
+            vec![]
+        };
+        let response = self.get(&url, query).await?;
+        match response.status().as_u16() {
+            200u16 => Ok(response
+                .json::<T>()
+                .await
+                .context("decoding completed job result as json")?),
+            _ => Err(anyhow::anyhow!(response.text().await.unwrap_or_default())),
+        }
+    }
+
+    pub async fn get_result_by_id<T: DeserializeOwned>(
+        &self,
+        flow_job_id: &str,
+        node_id: &str,
+        json_path: Option<String>,
+    ) -> anyhow::Result<T> {
+        let url = format!(
+            "{}/api/w/{}/jobs/result_by_id/{}/{}",
+            self.base_internal_url, self.workspace, flow_job_id, node_id
+        );
+        let query = if let Some(json_path) = json_path {
+            vec![("json_path", json_path)]
+        } else {
+            vec![]
+        };
+        let response = self.get(&url, query).await?;
+        match response.status().as_u16() {
+            200u16 => Ok(response
+                .json::<T>()
+                .await
+                .context("decoding result by id as json")?),
+            _ => Err(anyhow::anyhow!(response.text().await.unwrap_or_default())),
+        }
+    }
+
+    pub async fn upload_s3_file<S>(
+        &self,
+        workspace_id: &str,
+        object_key: String,
+        storage: Option<String>,
+        body: S,
+    ) -> anyhow::Result<()>
+    where
+        S: futures::stream::TryStream + Send + 'static,
+        S::Error: Into<Box<dyn std::error::Error + Send + Sync>>,
+        bytes::Bytes: From<S::Ok>,
+    {
+        let mut query = vec![("file_key", object_key)];
+        if let Some(storage) = storage {
+            query.push(("storage", storage));
+        }
+        let response = self
+            .force_client
+            .as_ref()
+            .unwrap_or(&HTTP_CLIENT)
+            .post(format!(
+                "{}/api/w/{}/job_helpers/upload_s3_file",
+                self.base_internal_url, workspace_id
+            ))
+            .query(&query)
+            .header(
+                reqwest::header::ACCEPT,
+                reqwest::header::HeaderValue::from_static("application/json"),
+            )
+            .header(
+                reqwest::header::AUTHORIZATION,
+                reqwest::header::HeaderValue::from_str(&format!("Bearer {}", self.token))
+                    .map_err(|e| anyhow::anyhow!(e.to_string()))?,
+            )
+            .body(Body::wrap_stream(body))
+            .send()
+            .await
+            .context(format!("Sent upload_s3_file request",))
+            .map_err(|e| anyhow::anyhow!(e.to_string()))?;
+
+        match response.status().as_u16() {
+            200u16 => Ok(()),
+            _ => Err(anyhow::anyhow!(response.text().await.unwrap_or_default()))?,
+        }
+    }
+
+    pub async fn get_duckdb_connection_settings(
+        &self,
+        s3: &DuckdbConnectionSettingsQueryV2,
+    ) -> error::Result<DuckdbConnectionSettingsResponse> {
+        let url = format!(
+            "{}/api/w/{}/job_helpers/v2/duckdb_connection_settings",
+            self.base_internal_url, &self.workspace
+        );
+        let response = self
+            .force_client
+            .as_ref()
+            .unwrap_or(&HTTP_CLIENT)
+            .post(url)
+            .header(
+                reqwest::header::CONTENT_TYPE,
+                reqwest::header::HeaderValue::from_static("application/json"),
+            )
+            .header(
+                reqwest::header::ACCEPT,
+                reqwest::header::HeaderValue::from_static("application/json"),
+            )
+            .header(
+                reqwest::header::AUTHORIZATION,
+                reqwest::header::HeaderValue::from_str(&format!("Bearer {}", self.token))
+                    .map_err(|e| error::Error::BadConfig(e.to_string()))?,
+            )
+            .body(serde_json::to_string(&s3).map_err(to_anyhow)?)
+            .send()
+            .await
+            .context(format!("Sent get_duckdb_connection_settings request",))
+            .map_err(error::Error::from)?;
+        match response.status().as_u16() {
+            200u16 => Ok(response
+                .json::<DuckdbConnectionSettingsResponse>()
+                .await
+                .context("decoding duckdb_connection_settings response as json")?),
+            _ => Err(anyhow::anyhow!(response.text().await.unwrap_or_default()))?,
+        }
+    }
+}

backend/windmill-common/src/db.rs

@@ -120,62 +120,6 @@ impl UserDB {
         .execute(&mut *tx)
         .await?;
 
-        // set_session_context(
-        //     username TEXT,
-        //     groups TEXT,
-        //     pgroups TEXT,
-        //     folders_read TEXT,
-        //     folders_write TEXT
-        // )
-
-        // sqlx::query!(
-        //     "SELECT set_config('session.user', $1, true)",
-        //     authed.username()
-        // )
-        // .fetch_optional(&mut *tx)
-        // .await?;
-
-        // sqlx::query!(
-        //     "SELECT set_config('session.groups', $1, true)",
-        //     &authed.groups().join(",")
-        // )
-        // .fetch_optional(&mut *tx)
-        // .await?;
-
-        // sqlx::query!(
-        //     "SELECT set_config('session.pgroups', $1, true)",
-        //     &authed
-        //         .groups()
-        //         .iter()
-        //         .map(|x| format!("g/{}", x))
-        //         .collect::<Vec<_>>()
-        //         .join(",")
-        // )
-        // .fetch_optional(&mut *tx)
-        // .await?;
-
-        // sqlx::query!(
-        //     "SELECT set_config('session.folders_read', $1, true)",
-        //     folders_read
-        //         .iter()
-        //         .map(|x| x.0.clone())
-        //         .collect::<Vec<_>>()
-        //         .join(",")
-        // )
-        // .fetch_optional(&mut *tx)
-        // .await?;
-
-        // sqlx::query!(
-        //     "SELECT set_config('session.folders_write', $1, true)",
-        //     folders_write
-        //         .iter()
-        //         .map(|x| x.0.clone())
-        //         .collect::<Vec<_>>()
-        //         .join(",")
-        // )
-        // .fetch_optional(&mut *tx)
-        // .await?;
-
         Ok(tx)
     }
 }

backend/windmill-common/src/global_settings.rs

@@ -30,7 +30,7 @@ pub const EXPOSE_METRICS_SETTING: &str = "expose_metrics";
 pub const EXPOSE_DEBUG_METRICS_SETTING: &str = "expose_debug_metrics";
 pub const KEEP_JOB_DIR_SETTING: &str = "keep_job_dir";
 pub const REQUIRE_PREEXISTING_USER_FOR_OAUTH_SETTING: &str = "require_preexisting_user_for_oauth";
-pub const OBJECT_STORE_CACHE_CONFIG_SETTING: &str = "object_store_cache_config";
+pub const OBJECT_STORE_CONFIG_SETTING: &str = "object_store_cache_config";
 
 pub const AUTOMATE_USERNAME_CREATION_SETTING: &str = "automate_username_creation";
 pub const HUB_BASE_URL_SETTING: &str = "hub_base_url";

backend/windmill-common/src/job_s3_helpers_ee.rs

@@ -1,18 +1,34 @@
-use std::future::Future;
+use crate::s3_helpers::{ObjectStoreResource, StorageResourceType};
 
-use crate::{
-    error::Error,
-    s3_helpers::{ObjectStoreResource, StorageResourceType},
-};
-
-pub async fn get_s3_resource_internal<'c, F, Fut>(
+pub async fn get_s3_resource_internal<'c>(
     _resource_type: StorageResourceType,
     _s3_resource_value_raw: serde_json::Value,
-    _gen_token: F,
-) -> crate::error::Result<ObjectStoreResource>
-where
-    F: FnOnce(String) -> Fut,
-    Fut: Future<Output = Result<String, Error>> + Send + 'static,
-{
+    _gen_token: TokenGenerator<'c>,
+    _db: &crate::DB,
+) -> crate::error::Result<ObjectStoreResource> {
+    todo!()
+}
+
+pub enum TokenGenerator<'c> {
+    AsClient(&'c crate::client::AuthedClient),
+    AsServerInstance(),
+}
+
+impl<'c> TokenGenerator<'c> {
+    pub async fn gen_token(
+        &self,
+        _audience: &str,
+        _db: Option<&crate::DB>,
+    ) -> anyhow::Result<String> {
+        todo!()
+    }
+}
+
+#[cfg(feature = "parquet")]
+pub(crate) async fn generate_s3_aws_oidc_resource<'c>(
+    _clone: crate::s3_helpers::S3AwsOidcResource,
+    _token_generator: TokenGenerator<'c>,
+    _init_private_key: Option<&sqlx::Pool<sqlx::Postgres>>,
+) -> crate::error::Result<ObjectStoreResource> {
     todo!()
 }

backend/windmill-common/src/jobs.rs

@@ -18,8 +18,9 @@ use crate::{
     flow_status::{FlowStatus, RestartedFrom},
     flows::{FlowNodeId, FlowValue, Retry},
     get_latest_deployed_hash_for_path, get_latest_flow_version_info_for_path,
-    scripts::{ScriptHash, ScriptLang},
+    scripts::{get_full_hub_script_by_path, ScriptHash, ScriptLang},
     users::username_to_permissioned_as,
+    utils::{StripPath, HTTP_CLIENT},
     worker::{to_raw_value, TMP_DIR},
     FlowVersionInfo, ScriptHashInfo,
 };
@@ -270,6 +271,7 @@ impl CompletedJob {
 pub enum JobPayload {
     ScriptHub {
         path: String,
+        apply_preprocessor: bool,
     },
     ScriptHash {
         hash: ScriptHash,
@@ -387,6 +389,25 @@ pub struct OnBehalfOf {
     pub permissioned_as: String,
 }
 
+pub fn get_has_preprocessor_from_content_and_lang(
+    content: &str,
+    language: &ScriptLang,
+) -> error::Result<bool> {
+    let has_preprocessor = match language {
+        ScriptLang::Bun | ScriptLang::Bunnative | ScriptLang::Deno | ScriptLang::Nativets => {
+            let args = windmill_parser_ts::parse_deno_signature(&content, true, true, None)?;
+            args.has_preprocessor.unwrap_or(false)
+        }
+        ScriptLang::Python3 => {
+            let args = windmill_parser_py::parse_python_signature(&content, None, true)?;
+            args.has_preprocessor.unwrap_or(false)
+        }
+        _ => false,
+    };
+
+    Ok(has_preprocessor)
+}
+
 pub async fn script_path_to_payload<'e, A: sqlx::Acquire<'e, Database = Postgres> + Send>(
     script_path: &str,
     db: A,
@@ -399,63 +420,74 @@ pub async fn script_path_to_payload<'e, A: sqlx::Acquire<'e, Database = Postgres
     Option<i32>,
     Option<OnBehalfOf>,
 )> {
-    let (job_payload, tag, delete_after_use, script_timeout, on_behalf_of) =
-        if script_path.starts_with("hub/") {
-            (
-                JobPayload::ScriptHub { path: script_path.to_owned() },
-                None,
-                None,
-                None,
-                None,
-            )
+    let (job_payload, tag, delete_after_use, script_timeout, on_behalf_of) = if script_path
+        .starts_with("hub/")
+    {
+        let hub_script =
+            get_full_hub_script_by_path(StripPath(script_path.to_string()), &HTTP_CLIENT, None)
+                .await?;
+
+        let has_preprocessor =
+            get_has_preprocessor_from_content_and_lang(&hub_script.content, &hub_script.language)?;
+
+        (
+            JobPayload::ScriptHub {
+                path: script_path.to_owned(),
+                apply_preprocessor: has_preprocessor && !skip_preprocessor.unwrap_or(false),
+            },
+            None,
+            None,
+            None,
+            None,
+        )
+    } else {
+        let ScriptHashInfo {
+            hash,
+            tag,
+            concurrency_key,
+            concurrent_limit,
+            concurrency_time_window_s,
+            cache_ttl,
+            language,
+            dedicated_worker,
+            priority,
+            delete_after_use,
+            timeout,
+            has_preprocessor,
+            on_behalf_of_email,
+            created_by,
+            ..
+        } = get_latest_deployed_hash_for_path(db, w_id, script_path).await?;
+
+        let on_behalf_of = if let Some(email) = on_behalf_of_email {
+            Some(OnBehalfOf {
+                email,
+                permissioned_as: username_to_permissioned_as(created_by.as_str()),
+            })
         } else {
-            let ScriptHashInfo {
-                hash,
-                tag,
-                concurrency_key,
+            None
+        };
+
+        (
+            JobPayload::ScriptHash {
+                hash: ScriptHash(hash),
+                path: script_path.to_owned(),
+                custom_concurrency_key: concurrency_key,
                 concurrent_limit,
                 concurrency_time_window_s,
-                cache_ttl,
+                cache_ttl: cache_ttl,
                 language,
                 dedicated_worker,
                 priority,
-                delete_after_use,
-                timeout,
-                has_preprocessor,
-                on_behalf_of_email,
-                created_by,
-                ..
-            } = get_latest_deployed_hash_for_path(db, w_id, script_path).await?;
-
-            let on_behalf_of = if let Some(email) = on_behalf_of_email {
-                Some(OnBehalfOf {
-                    email,
-                    permissioned_as: username_to_permissioned_as(created_by.as_str()),
-                })
-            } else {
-                None
-            };
-
-            (
-                JobPayload::ScriptHash {
-                    hash: ScriptHash(hash),
-                    path: script_path.to_owned(),
-                    custom_concurrency_key: concurrency_key,
-                    concurrent_limit,
-                    concurrency_time_window_s,
-                    cache_ttl: cache_ttl,
-                    language,
-                    dedicated_worker,
-                    priority,
-                    apply_preprocessor: !skip_preprocessor.unwrap_or(false)
-                        && has_preprocessor.unwrap_or(false),
-                },
-                tag,
-                delete_after_use,
-                timeout,
-                on_behalf_of,
-            )
-        };
+                apply_preprocessor: !skip_preprocessor.unwrap_or(false)
+                    && has_preprocessor.unwrap_or(false),
+            },
+            tag,
+            delete_after_use,
+            timeout,
+            on_behalf_of,
+        )
+    };
     Ok((
         job_payload,
         tag,
@@ -576,11 +608,11 @@ pub async fn get_logs_from_store(
     logs: &str,
     log_file_index: &Option<Vec<String>>,
 ) -> Option<impl Stream<Item = Result<Bytes, object_store::Error>>> {
-    use crate::s3_helpers::OBJECT_STORE_CACHE_SETTINGS;
+    use crate::s3_helpers::get_object_store;
 
     if log_offset > 0 {
         if let Some(file_index) = log_file_index.clone() {
-            if let Some(os) = OBJECT_STORE_CACHE_SETTINGS.read().await.clone() {
+            if let Some(os) = get_object_store().await {
                 let logs = logs.to_string();
                 let stream = async_stream::stream! {
                     for file_p in file_index.clone() {

backend/windmill-common/src/lib.rs

@@ -30,6 +30,7 @@ pub mod auth;
 #[cfg(feature = "benchmark")]
 pub mod bench;
 pub mod cache;
+pub mod client;
 pub mod db;
 pub mod ee;
 pub mod email_ee;
@@ -43,6 +44,9 @@ pub mod job_metrics;
 #[cfg(feature = "parquet")]
 pub mod job_s3_helpers_ee;
 
+#[cfg(all(feature = "enterprise", feature = "openidconnect"))]
+pub mod oidc_ee;
+
 pub mod jobs;
 pub mod jwt;
 pub mod more_serde;
@@ -62,6 +66,7 @@ pub mod utils;
 pub mod variables;
 pub mod worker;
 pub mod workspaces;
+pub mod triggers;
 
 pub const DEFAULT_MAX_CONNECTIONS_SERVER: u32 = 50;
 pub const DEFAULT_MAX_CONNECTIONS_WORKER: u32 = 5;

backend/windmill-common/src/oidc_ee.rs

@@ -0,0 +1,198 @@
+/*
+ * Author: Ruben Fiszel
+ * Copyright: Windmill Labs, Inc 2023
+ * This file and its contents are licensed under the AGPLv3 License.
+ * Please see the included NOTICE for copyright information and
+ * LICENSE-AGPL for a copy of the license.
+ */
+
+use serde::{Deserialize, Serialize};
+use tokio::sync::RwLock;
+#[cfg(all(feature = "enterprise", feature = "openidconnect"))]
+use {
+    crate::db::DB,
+    crate::{auth::IdToken as WindmillIdToken, error::Result},
+    anyhow,
+    openidconnect::{
+        core::{CoreJwsSigningAlgorithm, CoreRsaPrivateSigningKey},
+        IssuerUrl, JsonWebKeyId,
+    },
+    std::process::Command,
+};
+
+#[cfg(feature = "openidconnect")]
+use openidconnect::AdditionalClaims;
+
+#[cfg(feature = "openidconnect")]
+impl AdditionalClaims for JobClaim {}
+
+#[cfg(feature = "openidconnect")]
+impl AdditionalClaims for WorkspaceClaim {}
+
+#[cfg(feature = "openidconnect")]
+impl AdditionalClaims for InstanceClaim {}
+
+#[derive(Clone, Debug, Default, Deserialize, PartialEq, Eq, Serialize)]
+pub struct WorkspaceClaim {
+    pub workspace: String,
+}
+
+#[derive(Clone, Debug, Default, Deserialize, PartialEq, Eq, Serialize)]
+pub struct InstanceClaim {}
+
+#[derive(Clone, Debug, Default, Deserialize, PartialEq, Eq, Serialize)]
+pub struct JobClaim {
+    pub job_id: String,
+    pub path: Option<String>,
+    pub flow_path: Option<String>,
+    pub groups: Vec<String>,
+    pub username: String,
+    pub email: String,
+    pub workspace: String,
+}
+
+lazy_static::lazy_static! {
+    static ref PRIVATE_KEY: RwLock<Option<String>> = RwLock::new(None);
+}
+
+pub async fn generate_id_token<T: AdditionalClaims>(
+    db: Option<&DB>,
+    claim: T,
+    audience: &str,
+    identifier: String,
+    email: Option<String>,
+) -> Result<WindmillIdToken> {
+    use chrono::{Duration, Utc};
+    use openidconnect::{
+        core::{CoreGenderClaim, CoreJweContentEncryptionAlgorithm},
+        Audience, EndUserEmail, IdToken, IdTokenClaims, StandardClaims, SubjectIdentifier,
+    };
+
+    let private_key = get_private_key(db).await?;
+
+    let issue_url = format!("{}/api/oidc/", crate::BASE_URL.read().await.clone());
+    let issue_time = Utc::now();
+    let expiration = issue_time + Duration::try_hours(48).unwrap();
+    let id_token = IdToken::<
+        T,
+        CoreGenderClaim,
+        CoreJweContentEncryptionAlgorithm,
+        CoreJwsSigningAlgorithm,
+    >::new(
+        IdTokenClaims::<T, CoreGenderClaim>::new(
+            // Specify the issuer URL for the OpenID Connect Provider.
+            IssuerUrl::new(issue_url)
+                .map_err(|e| anyhow::anyhow!("Failed to generate IssueUrl: {}", e))?,
+            // The audience is usually a single entry with the client ID of the client for whom
+            // the ID token is intended. This is a required claim.
+            vec![Audience::new(audience.to_string())],
+            // The ID token expiration is usually much shorter than that of the access or refresh
+            // tokens issued to clients.
+            expiration,
+            // The issue time is usually the current time.
+            issue_time,
+            // Set the standard claims defined by the OpenID Connect Core spec.
+            StandardClaims::new(
+                // Stable subject identifiers are recommended in place of e-mail addresses or other
+                // potentially unstable identifiers. This is the only required claim.
+                SubjectIdentifier::new(identifier),
+            )
+            // Optional: specify the user's e-mail address. This should only be provided if the
+            // client has been granted the 'profile' or 'email' scopes.
+            .set_email(email.map(|x| EndUserEmail::new(x)))
+            // Optional: specify whether the provider has verified the user's e-mail address.
+            .set_email_verified(Some(true)),
+            // OpenID Connect Providers may supply custom claims by providing a struct that
+            // implements the AdditionalClaims trait. This requires manually using the
+            // generic IdTokenClaims struct rather than the CoreIdTokenClaims type alias,
+            // however.
+            claim,
+        ),
+        // The private key used for signing the ID token. For confidential clients (those able
+        // to maintain a client secret), a CoreHmacKey can also be used, in conjunction
+        // with one of the CoreJwsSigningAlgorithm::HmacSha* signing algorithms. When using an
+        // HMAC-based signing algorithm, the UTF-8 representation of the client secret should
+        // be used as the HMAC key.
+        &CoreRsaPrivateSigningKey::from_pem(
+            &private_key,
+            Some(JsonWebKeyId::new("windmill".to_string())),
+        )
+        .map_err(|e| anyhow::anyhow!("Invalid private key: {}", e))?,
+        // Uses the RS256 signature algorithm. This crate supports any RS*, PS*, or HS*
+        // signature algorithm.
+        CoreJwsSigningAlgorithm::RsaSsaPkcs1V15Sha256,
+        // When returning the ID token alongside an access token (e.g., in the Authorization Code
+        // flow), it is recommended to pass the access token here to set the `at_hash` claim
+        // automatically.
+        None,
+        // When returning the ID token alongside an authorization code (e.g., in the implicit
+        // flow), it is recommended to pass the authorization code here to set the `c_hash` claim
+        // automatically.
+        None,
+    )
+    .map_err(|e| anyhow::anyhow!("Failed to generate token: {}", e))?;
+
+    Ok(WindmillIdToken::new(id_token.to_string(), expiration))
+}
+
+#[cfg(all(feature = "enterprise", feature = "openidconnect"))]
+pub async fn get_private_key(db: Option<&DB>) -> anyhow::Result<String> {
+    if let Some(key) = PRIVATE_KEY.read().await.clone() {
+        return Ok(key);
+    } else if let Some(db) = db {
+        let key = sqlx::query_scalar!(
+            "SELECT value->>'private_key' FROM global_settings WHERE name = 'rsa_keys'",
+        )
+        .fetch_optional(db)
+        .await?
+        .flatten();
+
+        let key = key.filter(|s| !s.is_empty());
+
+        if let Some(key) = key {
+            return Ok(key);
+        } else {
+            let keys = gen_pems(db).await?;
+            return Ok(keys.private_key);
+        }
+    } else {
+        return Err(anyhow::anyhow!("Private key not found and no db provided"));
+    }
+}
+
+#[cfg(all(feature = "enterprise", feature = "openidconnect"))]
+#[derive(Debug, Clone, serde::Serialize)]
+struct Keys {
+    private_key: String,
+}
+
+#[cfg(all(feature = "enterprise", feature = "openidconnect"))]
+async fn gen_pems(db: &DB) -> anyhow::Result<Keys> {
+    use anyhow::anyhow;
+
+    let private_key_cmd = Command::new("openssl")
+        .arg("genrsa")
+        .arg("--traditional")
+        .arg("2048")
+        .output()
+        .expect("failed to execute process");
+
+    let private_key = String::from_utf8(private_key_cmd.stdout)?;
+
+    tracing::debug!("Generated private key: {}", private_key);
+
+    if private_key.is_empty() {
+        return Err(anyhow!("Failed to generate RSA key: key is empty"));
+    }
+
+    let keys = Keys { private_key };
+
+    sqlx::query!(
+        r#"INSERT INTO global_settings (name, value) VALUES ('rsa_keys', $1)"#,
+        serde_json::to_value(&keys).unwrap()
+    )
+    .execute(db)
+    .await?;
+
+    Ok(keys)
+}

backend/windmill-common/src/s3_helpers.rs

@@ -4,6 +4,7 @@ use crate::error;
 use aws_sdk_sts::config::ProvideCredentials;
 #[cfg(feature = "parquet")]
 use axum::async_trait;
+use chrono::{DateTime, Utc};
 #[cfg(feature = "parquet")]
 use object_store::aws::AwsCredential;
 #[cfg(feature = "parquet")]
@@ -17,6 +18,7 @@ use reqwest::header::HeaderMap;
 use serde::{Deserialize, Serialize};
 #[cfg(feature = "parquet")]
 use std::sync::{Arc, Mutex};
+
 #[cfg(feature = "parquet")]
 use tokio::sync::RwLock;
 
@@ -46,9 +48,170 @@ use tokio::task;
 use windmill_parser_sql::S3ModeFormat;
 
 #[cfg(feature = "parquet")]
-lazy_static::lazy_static! {
+#[derive(Clone)]
+pub struct ExpirableObjectStore {
+    pub store: Arc<dyn ObjectStore>,
+    pub refresh: Option<ObjectStoreRefresh>,
+}
 
-    pub static ref OBJECT_STORE_CACHE_SETTINGS: Arc<RwLock<Option<Arc<dyn ObjectStore>>>> = Arc::new(RwLock::new(None));
+#[cfg(feature = "parquet")]
+#[derive(Clone)]
+pub struct ObjectStoreRefresh {
+    refresh: Option<DateTime<Utc>>,
+    settings: ObjectSettings,
+}
+
+#[cfg(feature = "parquet")]
+impl ObjectStoreRefresh {
+    pub fn new(settings: ObjectSettings, refresh: Option<DateTime<Utc>>) -> Self {
+        Self { settings, refresh }
+    }
+    fn refresh_needed(&self) -> bool {
+        if let Some(refresh) = self.refresh {
+            if refresh < Utc::now() - chrono::Duration::minutes(1) {
+                return true;
+            }
+        }
+        return false;
+    }
+
+    async fn refresh(&self) -> Option<ExpirableObjectStore> {
+        return build_object_store_from_settings(self.settings.clone(), None)
+            .await
+            .map_err(|e| {
+                tracing::error!("Error building s3 client from settings: {:?}", e);
+                e
+            })
+            .ok();
+    }
+}
+
+#[cfg(feature = "parquet")]
+impl From<Arc<dyn ObjectStore>> for ExpirableObjectStore {
+    fn from(store: Arc<dyn ObjectStore>) -> Self {
+        Self { store, refresh: None }
+    }
+}
+
+// #[cfg(feature = "parquet")]
+
+// impl ExpirableObjectStore {
+//     pub fn new(store: Arc<dyn ObjectStore>, expiration: Option<DateTime<Utc>>) -> Self {
+//         Self { store, expiration }
+//     }
+// }
+
+#[cfg(feature = "parquet")]
+lazy_static::lazy_static! {
+    pub static ref OBJECT_STORE_SETTINGS: Arc<RwLock<Option<ExpirableObjectStore>>> = Arc::new(RwLock::new(None));
+}
+
+#[cfg(feature = "parquet")]
+pub async fn get_object_store() -> Option<Arc<dyn ObjectStore>> {
+    let settings = OBJECT_STORE_SETTINGS.read().await;
+    if let Some(s) = settings.as_ref() {
+        match &s.refresh {
+            Some(refresh) => {
+                if refresh.refresh_needed() {
+                    let refresh = refresh.clone();
+                    drop(settings);
+                    let new_store = refresh.refresh().await;
+                    if let Some(new_store) = new_store {
+                        let mut s3_cache_settings = OBJECT_STORE_SETTINGS.write().await;
+                        let arc = new_store.store.clone();
+                        *s3_cache_settings = Some(new_store);
+                        return Some(arc);
+                    } else {
+                        return None;
+                    }
+                } else {
+                    return Some(s.store.clone());
+                }
+            }
+            None => {
+                return Some(s.store.clone());
+            }
+        }
+    } else {
+        return None;
+    }
+}
+
+#[cfg(feature = "parquet")]
+pub enum ObjectStoreReload {
+    //if the jwks endpoints are not up yet, we should retry later soon
+    Later,
+    Never,
+}
+
+#[cfg(feature = "parquet")]
+pub async fn reload_object_store_setting(db: &crate::DB) -> ObjectStoreReload {
+    use crate::{
+        ee::{get_license_plan, LicensePlan},
+        global_settings::{load_value_from_global_settings, OBJECT_STORE_CONFIG_SETTING},
+        s3_helpers::ObjectSettings,
+    };
+
+    let s3_config = load_value_from_global_settings(db, OBJECT_STORE_CONFIG_SETTING).await;
+    if let Err(e) = s3_config {
+        tracing::error!("Error reloading s3 cache config: {:?}", e)
+    } else {
+        if let Some(v) = s3_config.unwrap() {
+            if matches!(get_license_plan().await, LicensePlan::Pro) {
+                tracing::error!("S3 cache is not available for pro plan");
+                return ObjectStoreReload::Never;
+            }
+            let setting = serde_json::from_value::<ObjectSettings>(v);
+            match setting {
+                Ok(setting) => {
+                    let is_oidc = matches!(setting, ObjectSettings::AwsOidc(_));
+                    let s3_client = build_object_store_from_settings(setting, Some(db)).await;
+                    match s3_client {
+                        Ok(s3_client) => {
+                            let mut s3_cache_settings = OBJECT_STORE_SETTINGS.write().await;
+                            *s3_cache_settings = Some(s3_client);
+                        }
+                        Err(e) => {
+                            if is_oidc {
+                                tracing::error!("Error building s3 client from oidc settings. It may be due to the jwks endpoints not being up yet, it will be attempted again in 10s to leave time for the server to be ready: {:?}", e);
+                                return ObjectStoreReload::Later;
+                            } else {
+                                tracing::error!("Error building s3 client from settings: {:?}", e);
+                            }
+                        }
+                    }
+                }
+                Err(e) => {
+                    tracing::error!("Error parsing s3 cache config: {:?}", e)
+                }
+            }
+        } else {
+            let mut s3_cache_settings = OBJECT_STORE_SETTINGS.write().await;
+            if std::env::var("S3_CACHE_BUCKET").is_ok() {
+                if matches!(get_license_plan().await, LicensePlan::Pro) {
+                    tracing::error!("S3 cache is not available for pro plan");
+                    return ObjectStoreReload::Never;
+                }
+                *s3_cache_settings = build_s3_client_from_settings(S3Settings {
+                    bucket: None,
+                    region: None,
+                    access_key: None,
+                    secret_key: None,
+                    endpoint: None,
+                    store_logs: None,
+                    path_style: None,
+                    allow_http: None,
+                    port: None,
+                })
+                .await
+                .ok()
+                .map(|x| ExpirableObjectStore::from(x))
+            } else {
+                *s3_cache_settings = None;
+            }
+        }
+    }
+    return ObjectStoreReload::Never;
 }
 
 #[derive(Serialize, Deserialize, Debug)]
@@ -81,6 +244,15 @@ pub enum ObjectStoreResource {
     Azure(AzureBlobResource),
 }
 
+impl ObjectStoreResource {
+    pub fn expiration(&self) -> Option<DateTime<Utc>> {
+        match self {
+            ObjectStoreResource::S3(s3_resource) => s3_resource.expiration,
+            _ => None,
+        }
+    }
+}
+
 #[derive(Deserialize, Debug)]
 pub enum StorageResourceType {
     S3,
@@ -104,6 +276,8 @@ pub struct S3Resource {
     #[serde(rename = "pathStyle")]
     pub path_style: Option<bool>,
     pub token: Option<String>,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub expiration: Option<DateTime<Utc>>,
     pub port: Option<u16>,
 }
 
@@ -126,7 +300,7 @@ pub struct AzureBlobResource {
     pub federated_token_file: Option<String>,
 }
 
-#[derive(Debug, Deserialize, Serialize, Clone)]
+#[derive(Debug, Deserialize, Serialize, Clone, Hash)]
 pub struct S3AwsOidcResource {
     #[serde(rename = "bucket")]
     pub bucket: String,
@@ -412,17 +586,47 @@ pub enum ObjectStoreSettings {
 pub enum ObjectSettings {
     S3(S3Settings),
     Azure(AzureBlobResource),
+    AwsOidc(S3AwsOidcResource),
+}
+
+impl ObjectSettings {
+    pub fn get_bucket(&self) -> Option<&String> {
+        match self {
+            ObjectSettings::S3(s3_settings) => s3_settings.bucket.as_ref(),
+            ObjectSettings::Azure(azure_settings) => Some(&azure_settings.container_name),
+            ObjectSettings::AwsOidc(s3_aws_oidc_settings) => Some(&s3_aws_oidc_settings.bucket),
+        }
+    }
 }
 
 #[cfg(feature = "parquet")]
 pub async fn build_object_store_from_settings(
     settings: ObjectSettings,
-) -> error::Result<Arc<dyn ObjectStore>> {
+    init_private_key: Option<&crate::DB>,
+) -> error::Result<ExpirableObjectStore> {
     match settings {
-        ObjectSettings::S3(s3_settings) => build_s3_client_from_settings(s3_settings).await,
+        ObjectSettings::S3(s3_settings) => build_s3_client_from_settings(s3_settings)
+            .await
+            .map(|x| ExpirableObjectStore::from(x)),
         ObjectSettings::Azure(azure_settings) => {
             let azure_blob_resource = azure_settings;
-            build_azure_blob_client(&azure_blob_resource)
+            build_azure_blob_client(&azure_blob_resource).map(|x| ExpirableObjectStore::from(x))
+        }
+        ObjectSettings::AwsOidc(ref s3_aws_oidc_settings) => {
+            let token_generator = crate::job_s3_helpers_ee::TokenGenerator::AsServerInstance();
+            let res = crate::job_s3_helpers_ee::generate_s3_aws_oidc_resource(
+                s3_aws_oidc_settings.clone(),
+                token_generator,
+                init_private_key,
+            )
+            .await?;
+
+            build_object_store_client(&res)
+                .await
+                .map(|x| ExpirableObjectStore {
+                    store: x,
+                    refresh: Some(ObjectStoreRefresh::new(settings.clone(), res.expiration())),
+                })
         }
     }
 }
@@ -470,6 +674,7 @@ pub async fn build_s3_client_from_settings(
         path_style: settings.path_style,
         port: settings.port,
         token: None,
+        expiration: None,
     };
 
     build_s3_client(&s3_resource).await
@@ -686,3 +891,20 @@ pub async fn convert_json_line_stream<E: Into<anyhow::Error>>(
 
     Ok(tokio_stream::wrappers::ReceiverStream::new(rx))
 }
+
+#[derive(Deserialize, Serialize)]
+pub struct DuckdbConnectionSettingsResponse {
+    pub connection_settings_str: String,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub azure_container_path: Option<String>,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub s3_bucket: Option<String>,
+}
+
+#[derive(Deserialize, Serialize)]
+pub struct DuckdbConnectionSettingsQueryV2 {
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub s3_resource_path: Option<String>,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub storage: Option<String>,
+}

backend/windmill-common/src/schema.rs

@@ -349,7 +349,7 @@ pub fn should_validate_schema(code: &str, lang: &ScriptLang) -> bool {
     let comment = match lang {
         Nativets | Bun | Bunnative | Deno | Php | CSharp | Java => "//",
         Python3 | Go | Bash | Powershell | Graphql | Ansible | Nu => "#",
-        Postgresql | Mysql | Bigquery | Snowflake | Mssql | OracleDB => "--",
+        Postgresql | Mysql | Bigquery | Snowflake | Mssql | OracleDB | DuckDb => "--",
         Rust => "//!",
         // for related places search: ADD_NEW_LANG
     };

backend/windmill-common/src/scripts.rs

@@ -46,13 +46,13 @@ pub enum ScriptLang {
     Graphql,
     Mssql,
     OracleDB,
+    DuckDb,
     Php,
     Rust,
     Ansible,
     CSharp,
     Nu,
-    Java,
-    // for related places search: ADD_NEW_LANG
+    Java, // for related places search: ADD_NEW_LANG
 }
 
 impl ScriptLang {
@@ -73,6 +73,7 @@ impl ScriptLang {
             ScriptLang::Mssql => "mssql",
             ScriptLang::Graphql => "graphql",
             ScriptLang::OracleDB => "oracledb",
+            ScriptLang::DuckDb => "duckdb",
             ScriptLang::Php => "php",
             ScriptLang::Rust => "rust",
             ScriptLang::Ansible => "ansible",
@@ -247,6 +248,7 @@ pub struct ListableScript {
     #[sqlx(default)]
     #[serde(skip_serializing_if = "Option::is_none")]
     pub deployment_msg: Option<String>,
+    pub kind: ScriptKind,
 }
 
 fn is_false(x: &bool) -> bool {

backend/windmill-common/src/triggers.rs

@@ -0,0 +1,79 @@
+use quick_cache::sync::Cache;
+use serde::{Deserialize, Serialize};
+use std::fmt;
+use strum::EnumIter;
+
+#[derive(Eq, PartialEq, Hash)]
+pub enum HubOrWorkspaceId {
+    Hub,
+    WorkspaceId(String),
+}
+
+type RunnableFormatCacheKey = (HubOrWorkspaceId, i64, TriggerKind);
+
+#[derive(Debug, Clone, Eq, PartialEq, Hash, Copy)]
+pub struct RunnableFormat {
+    pub version: RunnableFormatVersion,
+    pub has_preprocessor: bool,
+}
+
+#[derive(Debug, Clone, Eq, PartialEq, Hash, Copy)]
+pub enum RunnableFormatVersion {
+    V1,
+    V2,
+}
+
+lazy_static::lazy_static! {
+    pub static ref RUNNABLE_FORMAT_VERSION_CACHE: Cache<RunnableFormatCacheKey, RunnableFormat> = Cache::new(1000);
+}
+
+#[derive(sqlx::Type, Serialize, Deserialize, Debug, Clone, Eq, PartialEq, Hash, EnumIter)]
+#[sqlx(type_name = "TRIGGER_KIND", rename_all = "lowercase")]
+#[serde(rename_all = "lowercase")]
+pub enum TriggerKind {
+    Webhook,
+    Http,
+    Websocket,
+    Kafka,
+    Email,
+    Nats,
+    Mqtt,
+    Sqs,
+    Postgres,
+    Gcp,
+}
+
+impl TriggerKind {
+    pub fn to_key(&self) -> String {
+        match self {
+            TriggerKind::Webhook => "webhook".to_string(),
+            TriggerKind::Http => "http".to_string(),
+            TriggerKind::Websocket => "websocket".to_string(),
+            TriggerKind::Kafka => "kafka".to_string(),
+            TriggerKind::Email => "email".to_string(),
+            TriggerKind::Nats => "nats".to_string(),
+            TriggerKind::Mqtt => "mqtt".to_string(),
+            TriggerKind::Sqs => "sqs".to_string(),
+            TriggerKind::Postgres => "postgres".to_string(),
+            TriggerKind::Gcp => "gcp".to_string(),
+        }
+    }
+}
+
+impl fmt::Display for TriggerKind {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        let s = match self {
+            TriggerKind::Webhook => "webhook",
+            TriggerKind::Http => "http",
+            TriggerKind::Websocket => "websocket",
+            TriggerKind::Kafka => "kafka",
+            TriggerKind::Email => "email",
+            TriggerKind::Nats => "nats",
+            TriggerKind::Mqtt => "mqtt",
+            TriggerKind::Sqs => "sqs",
+            TriggerKind::Postgres => "postgres",
+            TriggerKind::Gcp => "gcp",
+        };
+        write!(f, "{}", s)
+    }
+}

backend/windmill-common/src/worker.rs

@@ -79,6 +79,7 @@ lazy_static::lazy_static! {
         "csharp".to_string(),
         "nu".to_string(),
         "java".to_string(),
+        "duckdb".to_string(),
         // for related places search: ADD_NEW_LANG
         "dependency".to_string(),
         "flow".to_string(),
@@ -516,6 +517,8 @@ fn parse_file<T: FromStr>(path: &str) -> Option<T> {
 pub struct PythonAnnotations {
     pub no_cache: bool,
     pub no_postinstall: bool,
+    pub py_select_latest: bool,
+    pub skip_result_postprocessing: bool,
     pub py310: bool,
     pub py311: bool,
     pub py312: bool,
@@ -580,11 +583,7 @@ pub async fn load_cache(bin_path: &str, _remote_path: &str, is_dir: bool) -> (bo
         (true, format!("loaded from local cache: {}\n", bin_path))
     } else {
         #[cfg(all(feature = "enterprise", feature = "parquet"))]
-        if let Some(os) = crate::s3_helpers::OBJECT_STORE_CACHE_SETTINGS
-            .read()
-            .await
-            .clone()
-        {
+        if let Some(os) = crate::s3_helpers::get_object_store().await {
             let started = std::time::Instant::now();
             use crate::s3_helpers::attempt_fetch_bytes;
 
@@ -627,11 +626,7 @@ pub async fn exists_in_cache(bin_path: &str, _remote_path: &str) -> bool {
         return true;
     } else {
         #[cfg(all(feature = "enterprise", feature = "parquet"))]
-        if let Some(os) = crate::s3_helpers::OBJECT_STORE_CACHE_SETTINGS
-            .read()
-            .await
-            .clone()
-        {
+        if let Some(os) = crate::s3_helpers::get_object_store().await {
             return os
                 .get(&object_store::path::Path::from(_remote_path))
                 .await
@@ -649,11 +644,7 @@ pub async fn save_cache(
 ) -> crate::error::Result<String> {
     let mut _cached_to_s3 = false;
     #[cfg(all(feature = "enterprise", feature = "parquet"))]
-    if let Some(os) = crate::s3_helpers::OBJECT_STORE_CACHE_SETTINGS
-        .read()
-        .await
-        .clone()
-    {
+    if let Some(os) = crate::s3_helpers::get_object_store().await {
         use object_store::path::Path;
         let file_to_cache = if is_dir {
             let tar_path = format!(

backend/windmill-git-sync/src/lib.rs

@@ -27,6 +27,14 @@ pub enum DeployedObject {
     ResourceType { path: String },
     User { email: String },
     Group { name: String },
+    HttpTrigger { path: String },
+    WebsocketTrigger { path: String },
+    KafkaTrigger { path: String },
+    NatsTrigger { path: String },
+    PostgresTrigger { path: String },
+    MqttTrigger { path: String },
+    SqsTrigger { path: String },
+    GcpTrigger { path: String },
 }
 
 impl DeployedObject {
@@ -42,6 +50,14 @@ impl DeployedObject {
             DeployedObject::ResourceType { path, .. } => path.to_owned(),
             DeployedObject::User { email } => format!("users/{email}"),
             DeployedObject::Group { name } => format!("groups/{name}"),
+            DeployedObject::HttpTrigger { path } => path.to_owned(),
+            DeployedObject::WebsocketTrigger { path } => path.to_owned(),
+            DeployedObject::KafkaTrigger { path } => path.to_owned(),
+            DeployedObject::NatsTrigger { path } => path.to_owned(),
+            DeployedObject::PostgresTrigger { path } => path.to_owned(),
+            DeployedObject::MqttTrigger { path } => path.to_owned(),
+            DeployedObject::SqsTrigger { path } => path.to_owned(),
+            DeployedObject::GcpTrigger { path } => path.to_owned(),
         }
     }
 
@@ -64,6 +80,14 @@ impl DeployedObject {
             DeployedObject::ResourceType { .. } => None,
             DeployedObject::User { .. } => None,
             DeployedObject::Group { .. } => None,
+            DeployedObject::HttpTrigger { .. } => None,
+            DeployedObject::WebsocketTrigger { .. } => None,
+            DeployedObject::KafkaTrigger { .. } => None,
+            DeployedObject::NatsTrigger { .. } => None,
+            DeployedObject::PostgresTrigger { .. } => None,
+            DeployedObject::MqttTrigger { .. } => None,
+            DeployedObject::SqsTrigger { .. } => None,
+            DeployedObject::GcpTrigger { .. } => None,
         }
     }
 }

backend/windmill-queue/src/jobs.rs

@@ -6,7 +6,6 @@
  * LICENSE-AGPL for a copy of the license.
  */
 
-use std::fmt;
 use std::{collections::HashMap, sync::Arc, vec};
 
 use anyhow::Context;
@@ -981,6 +980,7 @@ pub async fn add_completed_job<T: Serialize + Send + Sync + ValidableJson>(
             is_flow_step = queued_job.is_flow_step(),
             language = ?queued_job.script_lang,
             scheduled_for = ?queued_job.scheduled_for,
+            workspace_id = ?queued_job.workspace_id,
             success,
             "inserted completed job: {} (success: {success})",
             queued_job.id
@@ -1944,40 +1944,6 @@ async fn handle_successful_schedule<'a, 'c, T: Serialize + Send + Sync>(
     Ok(())
 }
 
-#[derive(sqlx::Type, Serialize, Deserialize, Debug, Clone, Eq, PartialEq, Hash)]
-#[sqlx(type_name = "TRIGGER_KIND", rename_all = "lowercase")]
-#[serde(rename_all = "lowercase")]
-pub enum TriggerKind {
-    Webhook,
-    Http,
-    Websocket,
-    Kafka,
-    Email,
-    Nats,
-    Mqtt,
-    Sqs,
-    Postgres,
-    Gcp
-}
-
-
-impl TriggerKind {
-    pub fn to_key(&self) -> String {
-        match self {
-            TriggerKind::Webhook => "webhook".to_string(),
-            TriggerKind::Http => "http".to_string(),
-            TriggerKind::Websocket => "websocket".to_string(),
-            TriggerKind::Kafka => "kafka".to_string(),
-            TriggerKind::Email => "email".to_string(),
-            TriggerKind::Nats => "nats".to_string(),
-            TriggerKind::Mqtt => "mqtt".to_string(),
-            TriggerKind::Sqs => "sqs".to_string(),
-            TriggerKind::Postgres => "postgres".to_string(),
-            TriggerKind::Gcp => "gcp".to_string(),
-        }
-    }
-}
-
 #[derive(sqlx::Type, Serialize, Deserialize, Debug, Clone)]
 #[sqlx(type_name = "JOB_TRIGGER_KIND", rename_all = "lowercase")]
 #[serde(rename_all = "lowercase")]
@@ -1995,23 +1961,6 @@ pub enum JobTriggerKind {
     Gcp
 }
 
-impl fmt::Display for TriggerKind {
-    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
-        let s = match self {
-            TriggerKind::Webhook => "webhook",
-            TriggerKind::Http => "http",
-            TriggerKind::Websocket => "websocket",
-            TriggerKind::Kafka => "kafka",
-            TriggerKind::Email => "email",
-            TriggerKind::Nats => "nats",
-            TriggerKind::Mqtt => "mqtt",
-            TriggerKind::Sqs => "sqs",
-            TriggerKind::Postgres => "postgres",
-            TriggerKind::Gcp => "gcp",
-        };
-        write!(f, "{}", s)
-    }
-}
 
 #[derive(sqlx::FromRow, Debug, Clone, Serialize, Deserialize)]
 pub struct MiniPulledJob {
@@ -2174,10 +2123,19 @@ pub struct PulledJob {
     pub permissioned_as_folders: Option<Vec<serde_json::Value>>,
 }
 
+
+// NOTE:
+// Precomputed by the server
+// Used to offload work from agent workers to server
 #[derive(Serialize, Deserialize)]
 pub enum PrecomputedAgentInfo {
     Bun { local: String, remote: String },
-    Python { py_version: Option<u32>, requirements: Option<String> },
+    Python {
+        // V1, not used anymore. Exists for compat.
+        // TODO: Needs to be removed eventually
+        py_version: Option<u32>,
+        py_version_v2: Option<String>,
+        requirements: Option<String> },
 }
 
 #[derive(Serialize, Deserialize)]
@@ -3587,7 +3545,7 @@ pub async fn push<'c, 'd>(
             None,
             None,
         ),
-        JobPayload::ScriptHub { path } => {
+        JobPayload::ScriptHub { path, apply_preprocessor } => {
             if path == "hub/7771/slack" || path == "hub/7836/slack" || path == "hub/9084/slack" {
                 // these scripts send app reports to slack
                 // they use the slack bot token and should therefore be run with permissions to access it
@@ -3595,6 +3553,10 @@ pub async fn push<'c, 'd>(
                 email = SUPERADMIN_NOTIFICATION_EMAIL;
             }
 
+            if apply_preprocessor {
+                preprocessed = Some(false);
+            }
+
             let hub_script =
                 get_full_hub_script_by_path(StripPath(path.clone()), &HTTP_CLIENT, Some(_db))
                     .await?;

backend/windmill-worker/Cargo.toml

@@ -31,6 +31,7 @@ csharp = ["dep:windmill-parser-csharp"]
 rust = ["dep:windmill-parser-rust"]
 nu = ["dep:windmill-parser-nu"]
 java = ["dep:windmill-parser-java"]
+duckdb = ["dep:duckdb"]
 
 [dependencies]
 windmill-queue.workspace = true
@@ -92,6 +93,7 @@ deno_permissions = { workspace = true, optional = true }
 deno_io = { workspace = true, optional = true }
 deno_error = { workspace = true, optional = true }
 async-stream.workspace = true
+duckdb = { workspace = true, optional = true }
 
 postgres-native-tls.workspace = true
 native-tls.workspace = true
@@ -116,6 +118,7 @@ convert_case.workspace = true
 yaml-rust.workspace = true
 backon.workspace = true
 winapi = { workspace = true, optional = true }
+pep440_rs.workspace = true
 
 opentelemetry = { workspace = true, optional = true }
 bollard = { workspace = true, optional = true }

backend/windmill-worker/src/ansible_executor.rs

@@ -30,10 +30,11 @@ use crate::{
         start_child_process, transform_json, OccupancyMetrics,
     },
     handle_child::handle_child,
-    python_executor::{create_dependencies_dir, handle_python_reqs, uv_pip_compile, PyVersion},
-    AuthedClient, DISABLE_NSJAIL, DISABLE_NUSER, GIT_PATH, HOME_ENV, NSJAIL_PATH, PATH_ENV,
-    PROXY_ENVS, PY_INSTALL_DIR, TZ_ENV,
+    python_executor::{create_dependencies_dir, handle_python_reqs, uv_pip_compile},
+    PyVAlias, DISABLE_NSJAIL, DISABLE_NUSER, GIT_PATH, HOME_ENV, NSJAIL_PATH, PATH_ENV, PROXY_ENVS,
+    PY_INSTALL_DIR, TZ_ENV,
 };
+use windmill_common::client::AuthedClient;
 
 lazy_static::lazy_static! {
     static ref ANSIBLE_PLAYBOOK_PATH: String =
@@ -373,7 +374,7 @@ async fn handle_ansible_python_deps(
                     worker_name,
                     w_id,
                     &mut Some(occupancy_metrics),
-                    PyVersion::Py311,
+                    PyVAlias::Py311.into(),
                     false,
                 )
                 .await
@@ -387,10 +388,7 @@ async fn handle_ansible_python_deps(
 
     if requirements.len() > 0 {
         let mut venv_path = handle_python_reqs(
-            requirements
-                .split("\n")
-                .filter(|x| !x.starts_with("--"))
-                .collect(),
+            crate::python_executor::split_requirements(requirements),
             job_id,
             w_id,
             mem_peak,
@@ -400,7 +398,7 @@ async fn handle_ansible_python_deps(
             job_dir,
             worker_dir,
             &mut Some(occupancy_metrics),
-            crate::python_executor::PyVersion::Py311,
+            PyVAlias::default().into(),
         )
         .await?;
         additional_python_paths.append(&mut venv_path);
@@ -1193,7 +1191,7 @@ async fn create_file_resources(
     job_dir: &str,
     args: Option<&HashMap<String, Box<RawValue>>>,
     r: &AnsibleRequirements,
-    client: &crate::AuthedClient,
+    client: &AuthedClient,
     conn: &Connection,
 ) -> error::Result<Vec<String>> {
     let mut logs = String::new();
@@ -1270,7 +1268,7 @@ async fn create_file_resources(
 }
 
 async fn get_resource_or_variable_content(
-    client: &crate::AuthedClient,
+    client: &AuthedClient,
     path: &ResourceOrVariablePath,
     job_id: String,
 ) -> anyhow::Result<String> {

backend/windmill-worker/src/bash_executor.rs

@@ -43,9 +43,11 @@ use crate::{
         OccupancyMetrics,
     },
     handle_child::handle_child,
-    AuthedClient, DISABLE_NSJAIL, DISABLE_NUSER, HOME_ENV, NSJAIL_PATH, PATH_ENV,
+    DISABLE_NSJAIL, DISABLE_NUSER, HOME_ENV, NSJAIL_PATH, PATH_ENV,
     POWERSHELL_CACHE_DIR, POWERSHELL_PATH, PROXY_ENVS, TZ_ENV,
 };
+use windmill_common::client::AuthedClient;
+
 
 #[cfg(windows)]
 use crate::SYSTEM_ROOT;
@@ -299,16 +301,28 @@ async fn handle_docker_job(
     }
 
     let wait_f = async {
-        let wait = client
+        let waited = client
             .wait_container::<String>(&container_id, None)
             .try_collect::<Vec<_>>()
-            .await
-            .map_err(|e| {
+            .await;
+        match waited {
+            Ok(wait) => Ok(wait.first().map(|x| x.status_code)),
+            Err(bollard::errors::Error::DockerResponseServerError { status_code, message }) => {
+                append_logs(&job_id, &workspace_id, &format!(": {message}"), conn).await;
+                Ok(Some(status_code as i64))
+            }
+            Err(bollard::errors::Error::DockerContainerWaitError { error, code }) => {
+                append_logs(&job_id, &workspace_id, &format!("{error}"), conn).await;
+                Ok(Some(code as i64))
+            }
+            Err(e) => {
                 tracing::error!("Error waiting for container: {:?}", e);
-                anyhow::anyhow!("Error waiting for container")
-            })?;
-        let waited = wait.first().map(|x| x.status_code);
-        Ok(waited)
+                Err(Error::ExecutionErr(format!(
+                    "Error waiting for container: {:?}",
+                    e
+                )))
+            }
+        }
     };
 
     let ncontainer_id = container_id.to_string();
@@ -317,7 +331,7 @@ async fn handle_docker_job(
     let conn2 = conn.clone();
     let worker_name2 = worker_name.to_string();
     let (tx, mut rx) = tokio::sync::broadcast::channel::<()>(1);
-
+    let workspace_id2 = workspace_id.to_string();
     let mut killpill_rx = killpill_rx.resubscribe();
     let logs = tokio::spawn(async move {
         let client = bollard::Docker::connect_with_unix_defaults().map_err(to_anyhow);
@@ -332,6 +346,13 @@ async fn handle_docker_job(
                     ..Default::default()
                 }),
             );
+            append_logs(
+                &job_id,
+                &workspace_id2,
+                "\ndocker logs stream started\n",
+                &conn2,
+            )
+            .await;
             loop {
                 tokio::select! {
                     log = log_stream.next() => {
@@ -441,11 +462,14 @@ async fn handle_docker_job(
 
     let result = result.unwrap();
 
+    if result.is_some_and(|x| x > 0) {
+        return Err(Error::ExecutionErr(format!(
+            "Docker job completed with unsuccessful exit status: {}",
+            result.unwrap()
+        )));
+    }
     return Ok(to_raw_value(&json!(format!(
-        "Docker exit status: {}",
-        result
-            .map(|x| x.to_string())
-            .unwrap_or_else(|| "none".to_string())
+        "Docker job completed with success exit status"
     ))));
 }
 

backend/windmill-worker/src/bigquery_executor.rs

@@ -4,6 +4,7 @@ use futures::future::BoxFuture;
 use futures::{FutureExt, StreamExt};
 use reqwest::Client;
 use serde_json::{json, value::RawValue, Value};
+use windmill_common::client::AuthedClient;
 use windmill_common::error::to_anyhow;
 use windmill_common::s3_helpers::convert_json_line_stream;
 use windmill_common::worker::Connection;
@@ -16,15 +17,12 @@ use windmill_queue::CanceledBy;
 
 use serde::Deserialize;
 
+use crate::common::{build_args_values, resolve_job_timeout};
 use crate::common::{
     build_http_client, s3_mode_args_to_worker_data, OccupancyMetrics, S3ModeWorkerData,
 };
 use crate::handle_child::run_future_with_polling_update_job_poller;
 use crate::sanitized_sql_params::sanitize_and_interpolate_unsafe_sql_args;
-use crate::{
-    common::{build_args_values, resolve_job_timeout},
-    AuthedClient,
-};
 
 use gcp_auth::{AuthenticationManager, CustomServiceAccount};
 

backend/windmill-worker/src/bun_executor.rs

@@ -20,10 +20,11 @@ use crate::{
         read_file_content, read_result, start_child_process, write_file_binary, OccupancyMetrics,
     },
     handle_child::handle_child,
-    AuthedClient, BUNFIG_INSTALL_SCOPES, BUN_BUNDLE_CACHE_DIR, BUN_CACHE_DIR, BUN_PATH,
-    DISABLE_NSJAIL, DISABLE_NUSER, HOME_ENV, NODE_BIN_PATH, NODE_PATH, NPM_CONFIG_REGISTRY,
-    NPM_PATH, NSJAIL_PATH, PATH_ENV, PROXY_ENVS, TZ_ENV,
+    BUNFIG_INSTALL_SCOPES, BUN_BUNDLE_CACHE_DIR, BUN_CACHE_DIR, BUN_PATH, DISABLE_NSJAIL,
+    DISABLE_NUSER, HOME_ENV, NODE_BIN_PATH, NODE_PATH, NPM_CONFIG_REGISTRY, NPM_PATH, NSJAIL_PATH,
+    PATH_ENV, PROXY_ENVS, TZ_ENV,
 };
+use windmill_common::client::AuthedClient;
 
 #[cfg(windows)]
 use crate::SYSTEM_ROOT;
@@ -612,10 +613,7 @@ pub async fn pull_codebase(w_id: &str, id: &str, job_dir: &str) -> Result<()> {
         extract_saved_codebase(job_dir, &bun_cache_path, is_tar, &dst, false)?;
     } else {
         #[cfg(all(feature = "enterprise", feature = "parquet"))]
-        let object_store = windmill_common::s3_helpers::OBJECT_STORE_CACHE_SETTINGS
-            .read()
-            .await
-            .clone();
+        let object_store = windmill_common::s3_helpers::get_object_store().await;
 
         #[cfg(not(all(feature = "enterprise", feature = "parquet")))]
         let object_store: Option<()> = None;

backend/windmill-worker/src/common.rs

@@ -44,10 +44,8 @@ use windmill_common::{variables, DB};
 use tokio::{io::AsyncWriteExt, process::Child, time::Instant};
 
 use crate::agent_workers::UPDATE_PING_URL;
-use crate::{
-    AuthedClient, DISABLE_NSJAIL, JOB_DEFAULT_TIMEOUT, MAX_RESULT_SIZE, MAX_TIMEOUT_DURATION,
-    PATH_ENV,
-};
+use crate::{DISABLE_NSJAIL, JOB_DEFAULT_TIMEOUT, MAX_RESULT_SIZE, MAX_TIMEOUT_DURATION, PATH_ENV};
+use windmill_common::client::AuthedClient;
 
 pub async fn build_args_map<'a>(
     job: &'a MiniPulledJob,
@@ -782,19 +780,17 @@ async fn get_workspace_s3_resource_path(
         }
     };
 
-    let client2 = client.clone();
-    let token_fn = |audience: String| async move {
-        client2
-            .get_id_token(&audience)
-            .await
-            .map_err(|e| windmill_common::error::Error::from(e))
-    };
     let s3_resource_value_raw = client
         .get_resource_value::<serde_json::Value>(path.as_str())
         .await?;
-    get_s3_resource_internal(rt, s3_resource_value_raw, token_fn)
-        .await
-        .map(Some)
+    get_s3_resource_internal(
+        rt,
+        s3_resource_value_raw,
+        windmill_common::job_s3_helpers_ee::TokenGenerator::AsClient(client),
+        db,
+    )
+    .await
+    .map(Some)
 }
 
 #[cfg(feature = "parquet")]
@@ -1109,7 +1105,7 @@ pub async fn par_install_language_dependencies<'a>(
         }
 
         #[cfg(all(feature = "enterprise", feature = "parquet"))]
-        if windmill_common::s3_helpers::OBJECT_STORE_CACHE_SETTINGS
+        if windmill_common::s3_helpers::OBJECT_STORE_SETTINGS
             .read()
             .await
             .is_none()
@@ -1264,11 +1260,7 @@ pub async fn par_install_language_dependencies<'a>(
 
         #[cfg(all(feature = "enterprise", feature = "parquet"))]
         let s3_pull_future = if is_not_pro {
-            if let Some(os) = windmill_common::s3_helpers::OBJECT_STORE_CACHE_SETTINGS
-                .read()
-                .await
-                .clone()
-            {
+            if let Some(os) = windmill_common::s3_helpers::get_object_store().await {
                 Some(crate::global_cache::pull_from_tar(
                     os,
                     path.clone(),
@@ -1449,11 +1441,7 @@ pub async fn par_install_language_dependencies<'a>(
                 };
                 #[cfg(all(feature = "enterprise", feature = "parquet"))]
                 {
-                    if let Some(os) = windmill_common::s3_helpers::OBJECT_STORE_CACHE_SETTINGS
-                        .read()
-                        .await
-                        .clone()
-                    {
+                    if let Some(os) = windmill_common::s3_helpers::get_object_store().await {
                         tokio::spawn(async move {
                             if let Err(e) = crate::global_cache::build_tar_and_push(
                                 os,
@@ -1541,11 +1529,7 @@ pub async fn par_install_language_dependencies<'a>(
                 };
                 #[cfg(all(feature = "enterprise", feature = "parquet"))]
                 {
-                    if let Some(os) = windmill_common::s3_helpers::OBJECT_STORE_CACHE_SETTINGS
-                        .read()
-                        .await
-                        .clone()
-                    {
+                    if let Some(os) = windmill_common::s3_helpers::get_object_store().await {
                         let language_name = language_name.to_owned();
                         tokio::spawn(async move {
                             if let Err(e) = crate::global_cache::build_tar_and_push(
@@ -1591,7 +1575,7 @@ pub struct S3ModeWorkerData {
 }
 
 impl S3ModeWorkerData {
-    pub async fn upload<S>(&self, stream: S) -> error::Result<()>
+    pub async fn upload<S>(&self, stream: S) -> anyhow::Result<()>
     where
         S: futures::stream::TryStream + Send + 'static,
         S::Error: Into<Box<dyn std::error::Error + Send + Sync>>,

backend/windmill-worker/src/csharp_executor.rs

@@ -36,7 +36,7 @@ use crate::{
 };
 
 use crate::common::OccupancyMetrics;
-use crate::AuthedClient;
+use windmill_common::client::AuthedClient;
 
 #[cfg(windows)]
 use crate::SYSTEM_ROOT;

backend/windmill-worker/src/deno_executor.rs

@@ -11,9 +11,11 @@ use crate::{
         start_child_process, OccupancyMetrics,
     },
     handle_child::handle_child,
-    AuthedClient, DENO_CACHE_DIR, DENO_PATH, DISABLE_NSJAIL, HOME_ENV, NPM_CONFIG_REGISTRY,
+    DENO_CACHE_DIR, DENO_PATH, DISABLE_NSJAIL, HOME_ENV, NPM_CONFIG_REGISTRY,
     PATH_ENV, TZ_ENV,
 };
+use windmill_common::client::AuthedClient;
+
 use tokio::{fs::File, io::AsyncReadExt, process::Command};
 use windmill_common::{error::Result, worker::write_file, BASE_URL};
 use windmill_common::{

backend/windmill-worker/src/duckdb_executor.rs

@@ -0,0 +1,664 @@
+use std::collections::HashMap;
+use std::env;
+
+use duckdb::types::TimeUnit;
+use duckdb::{params_from_iter, Row};
+use rust_decimal::prelude::FromPrimitive;
+use rust_decimal::Decimal;
+use serde_json::value::RawValue;
+use serde_json::{json, Value};
+use tokio::fs::remove_file;
+use tokio::task;
+use uuid::Uuid;
+use windmill_common::error::{to_anyhow, Error, Result};
+use windmill_common::s3_helpers::{
+    DuckdbConnectionSettingsQueryV2, DuckdbConnectionSettingsResponse, S3Object,
+};
+use windmill_common::worker::{to_raw_value, Connection};
+use windmill_parser_sql::{parse_duckdb_sig, parse_sql_blocks};
+use windmill_queue::{CanceledBy, MiniPulledJob};
+
+use crate::common::{build_args_values, OccupancyMetrics};
+use crate::handle_child::run_future_with_polling_update_job_poller;
+#[cfg(feature = "mysql")]
+use crate::mysql_executor::MysqlDatabase;
+use crate::pg_executor::PgDatabase;
+use crate::sanitized_sql_params::sanitize_and_interpolate_unsafe_sql_args;
+use windmill_common::client::AuthedClient;
+
+fn do_duckdb_inner(
+    conn: &duckdb::Connection,
+    query: &str,
+    job_args: &HashMap<String, duckdb::types::Value>,
+    skip_collect: bool,
+    column_order: &mut Option<Vec<String>>,
+) -> Result<Box<RawValue>> {
+    let mut rows_vec = vec![];
+
+    let (query, job_args) = interpolate_named_args(query, &job_args);
+
+    let mut stmt = conn
+        .prepare(&query)
+        .map_err(|e| Error::ExecutionErr(e.to_string()))?;
+
+    let mut rows = stmt
+        .query(params_from_iter(job_args))
+        .map_err(|e| Error::ExecutionErr(e.to_string()))?;
+
+    if skip_collect {
+        return Ok(to_raw_value(&json!([])));
+    }
+
+    // Statement needs to be stepped at least once or stmt.column_names() will panic
+    let mut column_names = None;
+    loop {
+        let row = rows.next();
+        match row {
+            Ok(Some(row)) => {
+                // Set column names if not already set
+                let stmt = row.as_ref();
+                let column_names = match column_names.as_ref() {
+                    Some(column_names) => column_names,
+                    None => {
+                        column_names = Some(stmt.column_names());
+                        column_names.as_ref().unwrap()
+                    }
+                };
+
+                let row = row_to_value(row, &column_names.as_slice())
+                    .map_err(|e| Error::ExecutionErr(e.to_string()))?;
+                rows_vec.push(row);
+            }
+            Ok(None) => break,
+            Err(e) => {
+                return Err(Error::ExecutionErr(e.to_string()));
+            }
+        }
+    }
+
+    if let (Some(column_order), Some(column_names)) = (column_order.as_mut(), column_names) {
+        *column_order = column_names.clone();
+    }
+
+    return Ok(to_raw_value(&rows_vec));
+}
+
+pub async fn do_duckdb(
+    job: &MiniPulledJob,
+    client: &AuthedClient,
+    query: &str,
+    conn: &Connection,
+    mem_peak: &mut i32,
+    canceled_by: &mut Option<CanceledBy>,
+    worker_name: &str,
+    column_order_ref: &mut Option<Vec<String>>,
+    occupancy_metrics: &mut OccupancyMetrics,
+) -> Result<Box<RawValue>> {
+    let result_f = async {
+        let sig = parse_duckdb_sig(query)?.args;
+        let mut job_args = build_args_values(job, client, conn).await?;
+
+        let (query, _) = &sanitize_and_interpolate_unsafe_sql_args(query, &sig, &job_args)?;
+        // Prevent interpolate_named_args from detecting argument identifiers in the signature for
+        // the first query block
+        let query = trunc_sig(query);
+
+        let (_query_with_transformed_s3_uris, mut used_storages) =
+            transform_s3_uris(query, client).await?;
+        let query = _query_with_transformed_s3_uris.as_deref().unwrap_or(query);
+
+        let job_args = {
+            let mut m: HashMap<String, duckdb::types::Value> = HashMap::new();
+            for sig_arg in sig.into_iter() {
+                let json_value = job_args
+                    .remove(&sig_arg.name)
+                    .or_else(|| sig_arg.default)
+                    .unwrap_or_else(|| json!(null));
+
+                if matches!(&sig_arg.otyp.as_ref().map(String::as_str), Some("s3object")) {
+                    let s3_obj = serde_json::from_value::<S3Object>(json_value).map_err(|e| {
+                        Error::ExecutionErr(format!("Failed to deserialize S3Object: {}", e))
+                    })?;
+                    let duckdb_conn_settings: windmill_common::s3_helpers::DuckdbConnectionSettingsResponse = client
+                        .get_duckdb_connection_settings(&DuckdbConnectionSettingsQueryV2 {
+                            s3_resource_path: None,
+                            storage: s3_obj.storage.clone(),
+                        })
+                        .await?;
+
+                    let uri = match (
+                        &duckdb_conn_settings.s3_bucket,
+                        &duckdb_conn_settings.azure_container_path,
+                    ) {
+                        (Some(s3_bucket), None) => format!("s3://{}/{}", s3_bucket, &s3_obj.s3),
+                        (None, Some(az_container)) => format!("{}/{}", az_container, &s3_obj.s3),
+                        _ => {
+                            return Err(Error::ExecutionErr(
+                                "S3Object must have either s3_bucket or azure_container_path"
+                                    .to_string(),
+                            ));
+                        }
+                    };
+                    m.insert(sig_arg.name, duckdb::types::Value::Text(uri));
+                    used_storages.insert(s3_obj.storage, duckdb_conn_settings);
+                } else {
+                    let duckdb_value = json_value_to_duckdb_value(
+                        &json_value,
+                        sig_arg
+                            .otyp
+                            .clone()
+                            .unwrap_or_else(|| "text".to_string())
+                            .as_str(),
+                        client,
+                    )?;
+                    m.insert(sig_arg.name, duckdb_value);
+                }
+            }
+            m
+        };
+
+        let query_block_list = parse_sql_blocks(query);
+
+        // Replace windmill resource ATTACH statements with the real instructions
+        let query_block_list = {
+            let mut v = vec![];
+            for query_block in query_block_list.iter() {
+                match parse_attach_db_resource(query_block) {
+                    Some(parsed) => v.extend(
+                        transform_attach_db_resource_query(&parsed, &job.id, client).await?,
+                    ),
+                    None => v.push(query_block.to_string()),
+                };
+            }
+            v
+        };
+
+        // duckdb::Connection is not Send so we do it in a single blocking task
+        let (result, column_order) = task::spawn_blocking(move || {
+            let conn = duckdb::Connection::open_in_memory()
+                .map_err(|e| Error::ConnectingToDatabase(e.to_string()))?;
+
+            for (_, DuckdbConnectionSettingsResponse { connection_settings_str, .. }) in
+                used_storages.into_iter()
+            {
+                conn.execute_batch(&connection_settings_str)
+                    .map_err(|e| Error::ExecutionErr(e.to_string()))?;
+            }
+
+            let mut result: Option<Box<RawValue>> = None;
+            let mut column_order = None;
+            for (query_block_index, query_block) in query_block_list.iter().enumerate() {
+                result = Some(
+                    do_duckdb_inner(
+                        &conn,
+                        query_block.as_str(),
+                        &job_args,
+                        query_block_index != query_block_list.len() - 1,
+                        &mut column_order,
+                    )
+                    .map_err(|e| Error::ExecutionErr(e.to_string()))?,
+                );
+            }
+            let result = result.unwrap_or_else(|| to_raw_value(&json!([])));
+            Ok::<_, Error>((result, column_order))
+        })
+        .await
+        .map_err(to_anyhow)??;
+
+        *column_order_ref = column_order;
+
+        // BigQuery cleanup
+        let bq_credentials_path = make_bq_credentials_path(&job.id);
+        env::remove_var("GOOGLE_APPLICATION_CREDENTIALS");
+        if matches!(tokio::fs::try_exists(&bq_credentials_path).await, Ok(true)) {
+            remove_file(&bq_credentials_path).await.map_err(to_anyhow)?;
+        }
+        Ok(result)
+    };
+
+    let result = run_future_with_polling_update_job_poller(
+        job.id,
+        job.timeout,
+        conn,
+        mem_peak,
+        canceled_by,
+        result_f,
+        worker_name,
+        &job.workspace_id,
+        &mut Some(occupancy_metrics),
+        Box::pin(futures::stream::once(async { 0 })),
+    )
+    .await?;
+
+    Ok(result)
+}
+
+fn row_to_value(row: &Row<'_>, column_names: &[String]) -> Result<Box<RawValue>> {
+    let mut obj = serde_json::Map::new();
+    for (i, key) in column_names.iter().enumerate() {
+        let value: duckdb::types::Value =
+            row.get(i).map_err(|e| Error::ExecutionErr(e.to_string()))?;
+        let json_value = match value {
+            duckdb::types::Value::Null => serde_json::Value::Null,
+            duckdb::types::Value::Boolean(b) => serde_json::Value::Bool(b),
+            duckdb::types::Value::TinyInt(i) => serde_json::Value::Number(i.into()),
+            duckdb::types::Value::SmallInt(i) => serde_json::Value::Number(i.into()),
+            duckdb::types::Value::Int(i) => serde_json::Value::Number(i.into()),
+            duckdb::types::Value::BigInt(i) => serde_json::Value::Number(i.into()),
+            duckdb::types::Value::HugeInt(i) => serde_json::Value::String(i.to_string()),
+            duckdb::types::Value::UTinyInt(u) => serde_json::Value::Number(u.into()),
+            duckdb::types::Value::USmallInt(u) => serde_json::Value::Number(u.into()),
+            duckdb::types::Value::UInt(u) => serde_json::Value::Number(u.into()),
+            duckdb::types::Value::UBigInt(u) => serde_json::Value::Number(u.into()),
+            duckdb::types::Value::Float(f) => serde_json::Value::Number(
+                serde_json::Number::from_f64(f as f64)
+                    .ok_or_else(|| Error::ExecutionErr("Could not convert to f64".to_string()))?,
+            ),
+            duckdb::types::Value::Double(f) => serde_json::Value::Number(
+                serde_json::Number::from_f64(f)
+                    .ok_or_else(|| Error::ExecutionErr("Could not convert to f64".to_string()))?,
+            ),
+            duckdb::types::Value::Decimal(d) => serde_json::Value::String(d.to_string()),
+            duckdb::types::Value::Timestamp(_, ts) => serde_json::Value::String(ts.to_string()),
+            duckdb::types::Value::Text(s) => serde_json::Value::String(s),
+            duckdb::types::Value::Blob(b) => serde_json::Value::Array(
+                b.into_iter()
+                    .map(|byte| serde_json::Value::Number(byte.into()))
+                    .collect(),
+            ),
+            duckdb::types::Value::Date32(d) => serde_json::Value::Number(d.into()),
+            duckdb::types::Value::Time64(_, t) => serde_json::Value::String(t.to_string()),
+            duckdb::types::Value::Interval { months, days, nanos } => serde_json::json!({
+                "months": months,
+                "days": days,
+                "nanos": nanos
+            }),
+            duckdb::types::Value::List(values) => serde_json::Value::Array(
+                values
+                    .into_iter()
+                    .map(|v| serde_json::Value::String(format!("{:?}", v)))
+                    .collect(),
+            ),
+            duckdb::types::Value::Enum(e) => serde_json::Value::String(e),
+            duckdb::types::Value::Struct(fields) => serde_json::Value::Object(
+                fields
+                    .iter()
+                    .map(|(k, v)| (k.clone(), serde_json::Value::String(format!("{:?}", v))))
+                    .collect(),
+            ),
+            duckdb::types::Value::Array(values) => serde_json::Value::Array(
+                values
+                    .into_iter()
+                    .map(|v| serde_json::Value::String(format!("{:?}", v)))
+                    .collect(),
+            ),
+            duckdb::types::Value::Map(map) => serde_json::Value::Object(
+                map.iter()
+                    .map(|(k, v)| {
+                        (
+                            format!("{:?}", k),
+                            serde_json::Value::String(format!("{:?}", v)),
+                        )
+                    })
+                    .collect(),
+            ),
+            duckdb::types::Value::Union(value) => {
+                serde_json::Value::String(format!("{:?}", *value))
+            }
+        };
+        obj.insert(key.clone(), json_value);
+    }
+    serde_json::value::to_raw_value(&obj).map_err(|e| e.into())
+}
+
+fn json_value_to_duckdb_value(
+    json_value: &serde_json::Value,
+    arg_type: &str,
+    client: &AuthedClient,
+) -> Result<duckdb::types::Value> {
+    let arg_type = arg_type.to_lowercase();
+    let duckdb_value = match json_value {
+        serde_json::Value::Null => duckdb::types::Value::Null,
+        serde_json::Value::Bool(b) => duckdb::types::Value::Boolean(*b),
+
+        serde_json::Value::String(s)
+            if matches!(
+                arg_type.as_str(),
+                "timestamp" | "timestamptz" | "timestamp with time zone" | "datetime"
+            ) =>
+        {
+            string_to_duckdb_timestamp(&s)?
+        }
+        serde_json::Value::String(s) if arg_type.as_str() == "date" => string_to_duckdb_date(&s)?,
+        serde_json::Value::String(s) if arg_type.as_str() == "time" => string_to_duckdb_time(&s)?,
+        serde_json::Value::String(s) => duckdb::types::Value::Text(s.clone()),
+
+        serde_json::Value::Number(n) if n.is_i64() => {
+            let v = n.as_i64().unwrap();
+            match arg_type.as_str() {
+                "tinyint" | "int1" => duckdb::types::Value::TinyInt(v as i8),
+                "smallint" | "int2" | "short" => duckdb::types::Value::SmallInt(v as i16),
+                "integer" | "int4" | "int" | "signed" => duckdb::types::Value::Int(v as i32),
+                "bigint" | "int8" | "long" => duckdb::types::Value::BigInt(v),
+                "hugeint" => duckdb::types::Value::HugeInt(v as i128),
+                "float" | "float4" | "real" => duckdb::types::Value::Float(v as f32),
+                "double" | "float8" => duckdb::types::Value::Double(v as f64),
+                _ => duckdb::types::Value::BigInt(v), // default fallback
+            }
+        }
+
+        serde_json::Value::Number(n) if n.is_u64() => {
+            let v = n.as_u64().unwrap();
+            match arg_type.as_str() {
+                "utinyint" => duckdb::types::Value::UTinyInt(v as u8),
+                "usmallint" => duckdb::types::Value::USmallInt(v as u16),
+                "uinteger" => duckdb::types::Value::UInt(v as u32),
+                "ubigint" | "uhugeint" => duckdb::types::Value::UBigInt(v),
+                _ => duckdb::types::Value::UBigInt(v), // default fallback
+            }
+        }
+
+        serde_json::Value::Number(n) if n.is_f64() => {
+            let v = n.as_f64().unwrap();
+            match arg_type.as_str() {
+                "float" | "float4" | "real" => duckdb::types::Value::Float(v as f32),
+                "double" | "float8" => duckdb::types::Value::Double(v),
+                "decimal" | "numeric" => {
+                    duckdb::types::Value::Decimal(Decimal::from_f64(v).ok_or_else(|| {
+                        Error::ExecutionErr("Could not convert f64 to Decimal".to_string())
+                    })?)
+                }
+                _ => duckdb::types::Value::Double(v), // default fallback
+            }
+        }
+
+        serde_json::Value::Array(arr) => duckdb::types::Value::Array(
+            arr.iter()
+                .map(|val| json_value_to_duckdb_value(val, arg_type.as_str(), client))
+                .collect::<Result<Vec<_>>>()?,
+        ),
+        serde_json::Value::Object(map) => duckdb::types::Value::Struct(
+            map.iter()
+                .map(|(k, v)| {
+                    Ok::<_, Error>((
+                        k.clone(),
+                        json_value_to_duckdb_value(v, arg_type.as_str(), client)?,
+                    ))
+                })
+                .collect::<Result<Vec<_>>>()?
+                .into(),
+        ),
+
+        value @ _ => {
+            return Err(Error::ExecutionErr(format!(
+                "Unsupported type in query: {:?} and signature {arg_type:?}",
+                value
+            )))
+        }
+    };
+    Ok(duckdb_value)
+}
+
+fn string_to_duckdb_timestamp(s: &str) -> Result<duckdb::types::Value> {
+    let ts = chrono::DateTime::parse_from_rfc3339(s)
+        .map_err(|e: chrono::ParseError| Error::ExecutionErr(e.to_string()))?;
+    Ok(duckdb::types::Value::Timestamp(
+        TimeUnit::Millisecond,
+        ts.timestamp_millis(),
+    ))
+}
+
+fn string_to_duckdb_date(s: &str) -> Result<duckdb::types::Value> {
+    use chrono::Datelike;
+    let date = chrono::NaiveDate::parse_from_str(s, "%Y-%m-%d").unwrap();
+    Ok(duckdb::types::Value::Date32(date.num_days_from_ce()))
+}
+
+fn string_to_duckdb_time(s: &str) -> Result<duckdb::types::Value> {
+    use chrono::Timelike;
+    let time = chrono::NaiveTime::parse_from_str(s, "%H:%M:%S").unwrap();
+    Ok(duckdb::types::Value::Time64(
+        TimeUnit::Microsecond,
+        time.num_seconds_from_midnight() as i64,
+    ))
+}
+
+struct ParsedAttachDbResource<'a> {
+    resource_path: &'a str,
+    name: &'a str,
+    db_type: &'a str,
+    extra_args: Option<&'a str>,
+}
+fn parse_attach_db_resource<'a>(query: &'a str) -> Option<ParsedAttachDbResource<'a>> {
+    lazy_static::lazy_static! {
+        static ref RE: regex::Regex = regex::Regex::new(r"ATTACH '\$res:([^']+)' AS (\S+) \(TYPE (\w+)(.*)\)").unwrap();
+    }
+
+    for cap in RE.captures_iter(query) {
+        if let (Some(resource_path), Some(name), Some(db_type)) =
+            (cap.get(1), cap.get(2), cap.get(3))
+        {
+            let extra_args = cap.get(4).map(|m| query[m.start()..m.end()].trim());
+            return Some(ParsedAttachDbResource {
+                resource_path: query[resource_path.start()..resource_path.end()].trim(),
+                name: query[name.start()..name.end()].trim(),
+                db_type: query[db_type.start()..db_type.end()].trim(),
+                extra_args,
+            });
+        }
+    }
+    None
+}
+
+async fn transform_attach_db_resource_query(
+    parsed: &ParsedAttachDbResource<'_>,
+    job_id: &Uuid,
+    client: &AuthedClient,
+) -> Result<Vec<String>> {
+    match parsed.db_type.to_lowercase().as_str() {
+        "postgres" => {
+            let resource: PgDatabase = client
+                .get_resource_value_interpolated(parsed.resource_path, Some(job_id.to_string()))
+                .await?;
+
+            let attach_str = format!(
+                "ATTACH 'dbname={} {} host={} {} {}' AS {} (TYPE postgres{});",
+                resource.dbname,
+                resource
+                    .user
+                    .map(|u| format!("user={}", u))
+                    .unwrap_or_default(),
+                resource.host,
+                resource
+                    .password
+                    .map(|p| format!("password={}", p))
+                    .unwrap_or_default(),
+                resource
+                    .port
+                    .map(|p| format!("port={}", p))
+                    .unwrap_or_default(),
+                parsed.name,
+                parsed.extra_args.unwrap_or("")
+            );
+
+            Ok(vec![
+                "INSTALL postgres;".to_string(),
+                "LOAD postgres;".to_string(),
+                attach_str,
+            ])
+        }
+        "mysql" => {
+            #[cfg(not(feature = "mysql"))]
+            return Err(Error::ExecutionErr(
+                "MySQL feature is not enabled".to_string(),
+            ));
+
+            #[cfg(feature = "mysql")]
+            {
+                let resource: MysqlDatabase = client
+                    .get_resource_value_interpolated(parsed.resource_path, Some(job_id.to_string()))
+                    .await?;
+
+                let attach_str = format!(
+                    "ATTACH 'database={} host={} ssl_mode={} {} {} {}' AS {} (TYPE mysql{});",
+                    resource.database,
+                    resource.host,
+                    resource
+                        .ssl
+                        .map(|ssl| if ssl { "required" } else { "disabled" })
+                        .unwrap_or("preferred"),
+                    resource
+                        .password
+                        .map(|p| format!("password={}", p))
+                        .unwrap_or_default(),
+                    resource
+                        .port
+                        .map(|p| format!("port={}", p))
+                        .unwrap_or_default(),
+                    resource
+                        .user
+                        .map(|u| format!("user={}", u))
+                        .unwrap_or_default(),
+                    parsed.name,
+                    parsed.extra_args.unwrap_or("")
+                );
+
+                Ok(vec![
+                    "INSTALL mysql;".to_string(),
+                    "LOAD mysql;".to_string(),
+                    attach_str,
+                ])
+            }
+        }
+        "bigquery" => {
+            let resource: Value = client
+                .get_resource_value_interpolated(parsed.resource_path, Some(job_id.to_string()))
+                .await?;
+            // duckdb's bigquery extension requires a json file as credentials
+            let bq_credentials_path = make_bq_credentials_path(job_id);
+            env::set_var("GOOGLE_APPLICATION_CREDENTIALS", &bq_credentials_path);
+            tokio::fs::write(&bq_credentials_path, resource.to_string())
+                .await
+                .map_err(|e| {
+                    Error::ExecutionErr(format!(
+                        "Failed to write BigQuery credentials to {}: {}",
+                        &bq_credentials_path, e
+                    ))
+                })?;
+            let project_id: String = serde_json::from_value(
+                resource
+                    .get("project_id")
+                    .ok_or_else(|| {
+                        Error::ExecutionErr("BigQuery resource must contain project_id".to_string())
+                    })?
+                    .to_owned(),
+            )
+            .map_err(|_e| Error::ExecutionErr("failed project_id deserialize".to_string()))?;
+            let attach_str = format!(
+                "ATTACH 'project={}' as {} (TYPE bigquery{});",
+                project_id,
+                parsed.name,
+                parsed.extra_args.unwrap_or("")
+            )
+            .to_string();
+            Ok(vec![
+                "INSTALL bigquery FROM community;".to_string(),
+                "LOAD bigquery;".to_string(),
+                attach_str,
+            ])
+        }
+        _ => Err(Error::ExecutionErr(format!(
+            "Unsupported db type in DuckDB ATTACH: {}",
+            parsed.db_type
+        ))),
+    }
+}
+
+// Returns the transformed query and the set of storages used
+async fn transform_s3_uris(
+    query: &str,
+    client: &AuthedClient,
+) -> Result<(
+    Option<String>,
+    HashMap<Option<String>, DuckdbConnectionSettingsResponse>,
+)> {
+    let mut transformed_query = None;
+    lazy_static::lazy_static! {
+        static ref RE: regex::Regex = regex::Regex::new(r"'s3://([^'/]*)/([^']+)'").unwrap();
+    }
+    let mut used_storages = HashMap::new();
+    for cap in RE.captures_iter(query) {
+        if let (storage, Some(s3_path)) = (cap.get(1), cap.get(2)) {
+            let s3_path = s3_path.as_str();
+            let storage = match storage.map(|m| m.as_str()) {
+                Some("") | None => None,
+                Some(s) => Some(s.to_string()),
+            };
+            let original_str_lit =
+                format!("'s3://{}/{}'", storage.as_deref().unwrap_or(""), s3_path);
+            let duckdb_conn_settings = client
+                .get_duckdb_connection_settings(&DuckdbConnectionSettingsQueryV2 {
+                    s3_resource_path: None,
+                    storage: storage.clone(),
+                })
+                .await?;
+            let url = match &duckdb_conn_settings {
+                DuckdbConnectionSettingsResponse { s3_bucket: Some(bucket), .. } => {
+                    format!("'s3://{bucket}/{s3_path}'")
+                }
+                DuckdbConnectionSettingsResponse { azure_container_path: Some(base), .. } => {
+                    format!("'{base}/{s3_path}'")
+                }
+                _ => {
+                    return Err(Error::ExecutionErr(
+                        "DuckDB connection settings response must have either s3_bucket or azure_container_path".to_string(),
+                    ))?;
+                }
+            };
+            transformed_query = Some(
+                transformed_query
+                    .unwrap_or(query.to_string())
+                    .replace(&original_str_lit, &url),
+            );
+            used_storages.insert(storage, duckdb_conn_settings);
+        }
+    }
+    Ok((transformed_query, used_storages))
+}
+
+// BigQuery extension requires a json file as credentials
+// The file path is set as an env var by do_duckdb
+// It is created by transform_attach_db_resource_query (when bigquery is detected)
+// and deleted by do_duckdb after the query is executed
+fn make_bq_credentials_path(job_id: &Uuid) -> String {
+    format!("/tmp/service-account-credentials-{}.json", job_id)
+}
+
+// duckdb-rs does not support named parameters,
+// and it raises an error when passing unused arguments. We cannot prepare batch statements
+// but only single SQL statements so it doesn't work when all arguments are not used by
+// every single statement.
+fn interpolate_named_args<'a>(
+    query: &str,
+    args: &'a HashMap<String, duckdb::types::Value>,
+) -> (String, Vec<&'a duckdb::types::Value>) {
+    let mut query = query.to_string();
+
+    let mut values = vec![];
+    for (arg_name, arg_value) in args {
+        let pat = format!("${}", arg_name);
+        if !query.contains(&pat) {
+            continue;
+        }
+        values.push(arg_value);
+        query = query.replace(&pat, &format!("${}", values.len()));
+    }
+    (query, values)
+}
+
+fn trunc_sig(query: &str) -> &str {
+    let idx = query.rfind("-- $").unwrap_or(query.len());
+    // find next \n starting from idx and return everything after it
+    let idx = query[idx..].find('\n').map(|i| i + idx).unwrap_or(0);
+    &query[idx..]
+}

backend/windmill-worker/src/global_cache.rs

@@ -22,6 +22,7 @@ pub async fn build_tar_and_push(
     platform_agnostic: bool,
 ) -> error::Result<()> {
     use object_store::path::Path;
+    use tokio::fs::create_dir_all;
 
     use crate::TAR_PYBASE_CACHE_DIR;
 
@@ -36,7 +37,9 @@ pub async fn build_tar_and_push(
     };
 
     let prefix = &format!("{TAR_PYBASE_CACHE_DIR}/{}", lang);
-    let tar_path = format!("{prefix}/{folder_name}_tar.tar",);
+    let tar_path = format!("{prefix}/{folder_name}_tar.tar");
+
+    create_dir_all(prefix).await?;
 
     let tar_file = std::fs::File::create(&tar_path)?;
     let mut tar = tar::Builder::new(tar_file);

backend/windmill-worker/src/go_executor.rs

@@ -19,9 +19,10 @@ use crate::{
         start_child_process, OccupancyMetrics,
     },
     handle_child::handle_child,
-    AuthedClient, DISABLE_NSJAIL, DISABLE_NUSER, GOPRIVATE, GOPROXY, GO_BIN_CACHE_DIR,
-    GO_CACHE_DIR, HOME_ENV, NSJAIL_PATH, PATH_ENV, TZ_ENV,
+    DISABLE_NSJAIL, DISABLE_NUSER, GOPRIVATE, GOPROXY, GO_BIN_CACHE_DIR, GO_CACHE_DIR, HOME_ENV,
+    NSJAIL_PATH, PATH_ENV, TZ_ENV,
 };
+use windmill_common::client::AuthedClient;
 
 const GO_REQ_SPLITTER: &str = "//go.sum\n";
 const NSJAIL_CONFIG_RUN_GO_CONTENT: &str = include_str!("../nsjail/run.go.config.proto");
@@ -473,7 +474,7 @@ pub async fn install_go_dependencies(
     if non_dep_job {
         if let Some(db) = conn.as_sql() {
             sqlx::query!(
-                "INSERT INTO pip_resolution_cache (hash, lockfile, expiration) VALUES ($1, $2, now() + ('3 days')::interval) ON CONFLICT (hash) DO UPDATE SET lockfile = $2",
+                "INSERT INTO pip_resolution_cache (hash, lockfile, expiration) VALUES ($1, $2, now() + ('5 mins')::interval) ON CONFLICT (hash) DO UPDATE SET lockfile = $2",
                 hash,
                 req_content
             )

backend/windmill-worker/src/graphql_executor.rs

@@ -12,7 +12,8 @@ use serde::Deserialize;
 
 use crate::common::{build_http_client, resolve_job_timeout, OccupancyMetrics};
 use crate::handle_child::run_future_with_polling_update_job_poller;
-use crate::{common::build_args_map, AuthedClient};
+use crate::common::build_args_map;
+use windmill_common::client::AuthedClient;
 
 #[derive(Deserialize)]
 struct GraphqlApi {

backend/windmill-worker/src/handle_child.rs

@@ -134,7 +134,7 @@ pub async fn handle_child(
     let (tx, rx) = broadcast::channel::<()>(3);
     let mut rx2: broadcast::Receiver<()> = tx.subscribe();
 
-    let output = child_joined_output_stream(&mut child, job_id.clone());
+    let output = child_joined_output_stream(&mut child, job_id.clone(), w_id.to_string());
 
     let job_id: Uuid = job_id.clone();
 
@@ -729,6 +729,7 @@ where
 fn child_joined_output_stream(
     child: &mut Child,
     job_id: Uuid,
+    w_id: String,
 ) -> impl stream::FusedStream<Item = io::Result<String>> {
     let stderr = child
         .stderr
@@ -743,8 +744,8 @@ fn child_joined_output_stream(
     let stdout = BufReader::new(stdout).lines();
     let stderr = BufReader::new(stderr).lines();
     stream::select(
-        lines_to_stream(stderr, true, job_id.clone()),
-        lines_to_stream(stdout, false, job_id),
+        lines_to_stream(stderr, true, job_id.clone(), w_id.clone()),
+        lines_to_stream(stdout, false, job_id, w_id),
     )
 }
 
@@ -752,11 +753,12 @@ pub fn lines_to_stream<R: tokio::io::AsyncBufRead + Unpin>(
     mut lines: tokio::io::Lines<R>,
     stderr: bool,
     job_id: Uuid,
+    w_id: String,
 ) -> impl futures::Stream<Item = io::Result<String>> {
     stream::poll_fn(move |cx| {
         std::pin::Pin::new(&mut lines)
             .poll_next_line(cx)
-            .map(|result| process_streaming_log_lines(result, stderr, &job_id))
+            .map(|result| process_streaming_log_lines(result, stderr, &job_id, &w_id))
     })
 }
 

backend/windmill-worker/src/java_executor.rs

@@ -24,9 +24,11 @@ use crate::{
         create_args_and_out_file, get_reserved_variables, par_install_language_dependencies,
         read_result, start_child_process, OccupancyMetrics, RequiredDependency,
     },
-    handle_child, AuthedClient, COURSIER_CACHE_DIR, DISABLE_NSJAIL, DISABLE_NUSER, JAVA_CACHE_DIR,
+    handle_child, COURSIER_CACHE_DIR, DISABLE_NSJAIL, DISABLE_NUSER, JAVA_CACHE_DIR,
     JAVA_REPOSITORY_DIR, MAVEN_REPOS, NO_DEFAULT_MAVEN, NSJAIL_PATH, PATH_ENV, PROXY_ENVS,
 };
+use windmill_common::client::AuthedClient;
+
 lazy_static::lazy_static! {
     static ref JAVA_CONCURRENT_DOWNLOADS: usize = std::env::var("JAVA_CONCURRENT_DOWNLOADS").ok().map(|flag| flag.parse().unwrap_or(20)).unwrap_or(20);
     static ref JAVA_PATH: String = std::env::var("JAVA_PATH").unwrap_or_else(|_| "/usr/bin/java".to_string());
@@ -243,7 +245,7 @@ pub async fn resolve<'a>(
 
     if let Connection::Sql(db) = conn {
         sqlx::query!(
-        "INSERT INTO pip_resolution_cache (hash, lockfile, expiration) VALUES ($1, $2, now() + ('3 days')::interval) ON CONFLICT (hash) DO UPDATE SET lockfile = $2",
+        "INSERT INTO pip_resolution_cache (hash, lockfile, expiration) VALUES ($1, $2, now() + ('5 mins')::interval) ON CONFLICT (hash) DO UPDATE SET lockfile = $2",
         req_hash,
             lock.clone(),
         )

backend/windmill-worker/src/job_logger_ee.rs

@@ -36,6 +36,7 @@ pub(crate) fn process_streaming_log_lines(
     r: Result<Option<String>, io::Error>,
     _stderr: bool,
     _job_id: &Uuid,
+    _w_id: &str,
 ) -> Option<Result<String, io::Error>> {
     r.transpose()
 }

backend/windmill-worker/src/js_eval.rs

@@ -48,7 +48,8 @@ use windmill_common::worker::{write_file, TMP_DIR};
 use windmill_common::flow_status::JobResult;
 use windmill_queue::CanceledBy;
 
-use crate::{common::OccupancyMetrics, AuthedClient};
+use crate::common::OccupancyMetrics;
+use windmill_common::client::AuthedClient;
 
 #[cfg(feature = "deno_core")]
 use crate::{common::unsafe_raw, handle_child::run_future_with_polling_update_job_poller};

backend/windmill-worker/src/lib.rs

@@ -20,6 +20,8 @@ mod csharp_executor;
 #[cfg(feature = "enterprise")]
 mod dedicated_worker;
 mod deno_executor;
+#[cfg(feature = "duckdb")]
+mod duckdb_executor;
 mod global_cache;
 mod go_executor;
 mod graphql_executor;
@@ -39,6 +41,8 @@ mod pg_executor;
 mod php_executor;
 #[cfg(feature = "python")]
 mod python_executor;
+#[cfg(feature = "python")]
+mod python_versions;
 pub mod result_processor;
 #[cfg(feature = "rust")]
 mod rust_executor;
@@ -60,3 +64,6 @@ pub use bun_executor::{
     prebundle_bun_script, prepare_job_dir,
 };
 pub use deno_executor::generate_deno_lock;
+
+#[cfg(feature = "python")]
+pub use python_versions::{PyV, PyVAlias};

backend/windmill-worker/src/mssql_executor.rs

@@ -22,7 +22,7 @@ use windmill_queue::{append_logs, CanceledBy};
 use crate::common::{build_args_values, s3_mode_args_to_worker_data, OccupancyMetrics};
 use crate::handle_child::run_future_with_polling_update_job_poller;
 use crate::sanitized_sql_params::sanitize_and_interpolate_unsafe_sql_args;
-use crate::AuthedClient;
+use windmill_common::client::AuthedClient;
 
 use serde::Deserializer;
 

backend/windmill-worker/src/mysql_executor.rs

@@ -13,6 +13,7 @@ use serde_json::{json, value::RawValue, Value};
 use std::str::FromStr;
 use tokio::sync::Mutex;
 use windmill_common::{
+    client::AuthedClient,
     error::{to_anyhow, Error},
     s3_helpers::convert_json_line_stream,
     worker::{to_raw_value, Connection},
@@ -28,17 +29,16 @@ use crate::{
     common::{build_args_values, s3_mode_args_to_worker_data, OccupancyMetrics, S3ModeWorkerData},
     handle_child::run_future_with_polling_update_job_poller,
     sanitized_sql_params::sanitize_and_interpolate_unsafe_sql_args,
-    AuthedClient,
 };
 
 #[derive(Deserialize)]
-struct MysqlDatabase {
-    host: String,
-    user: Option<String>,
-    password: Option<String>,
-    port: Option<u16>,
-    database: String,
-    ssl: Option<bool>,
+pub struct MysqlDatabase {
+    pub host: String,
+    pub user: Option<String>,
+    pub password: Option<String>,
+    pub port: Option<u16>,
+    pub database: String,
+    pub ssl: Option<bool>,
 }
 
 fn do_mysql_inner<'a>(

backend/windmill-worker/src/nu_executor.rs

@@ -16,8 +16,10 @@ use crate::{
         create_args_and_out_file, get_reserved_variables, read_result, start_child_process,
         OccupancyMetrics,
     },
-    handle_child, AuthedClient, DISABLE_NSJAIL, DISABLE_NUSER, NSJAIL_PATH, PATH_ENV, PROXY_ENVS,
+    handle_child, DISABLE_NSJAIL, DISABLE_NUSER, NSJAIL_PATH, PATH_ENV, PROXY_ENVS,
 };
+use windmill_common::client::AuthedClient;
+
 
 const NSJAIL_CONFIG_RUN_NU_CONTENT: &str = include_str!("../nsjail/run.nu.config.proto");
 lazy_static::lazy_static! {

backend/windmill-worker/src/oracledb_executor.rs

@@ -27,9 +27,9 @@ use crate::{
         OccupancyMetrics, S3ModeWorkerData,
     },
     handle_child::run_future_with_polling_update_job_poller,
-    sanitized_sql_params::sanitize_and_interpolate_unsafe_sql_args,
-    AuthedClient,
+    sanitized_sql_params::sanitize_and_interpolate_unsafe_sql_args
 };
+use windmill_common::client::AuthedClient;
 
 #[derive(Deserialize)]
 struct OracleDatabase {

backend/windmill-worker/src/pg_executor.rs

@@ -41,20 +41,20 @@ use crate::common::{
 };
 use crate::handle_child::run_future_with_polling_update_job_poller;
 use crate::sanitized_sql_params::sanitize_and_interpolate_unsafe_sql_args;
-use crate::{AuthedClient, MAX_RESULT_SIZE};
+use crate::MAX_RESULT_SIZE;
 use bytes::Buf;
 use lazy_static::lazy_static;
 use urlencoding::encode;
-
+use windmill_common::client::AuthedClient;
 #[derive(Deserialize)]
-struct PgDatabase {
-    host: String,
-    user: Option<String>,
-    password: Option<String>,
-    port: Option<u16>,
-    sslmode: Option<String>,
-    dbname: String,
-    root_certificate_pem: Option<String>,
+pub struct PgDatabase {
+    pub host: String,
+    pub user: Option<String>,
+    pub password: Option<String>,
+    pub port: Option<u16>,
+    pub sslmode: Option<String>,
+    pub dbname: String,
+    pub root_certificate_pem: Option<String>,
 }
 
 lazy_static! {

backend/windmill-worker/src/php_executor.rs

@@ -20,9 +20,10 @@ use crate::{
         read_result, start_child_process, OccupancyMetrics,
     },
     handle_child::handle_child,
-    AuthedClient, COMPOSER_CACHE_DIR, COMPOSER_PATH, DISABLE_NSJAIL, DISABLE_NUSER, NSJAIL_PATH,
+    COMPOSER_CACHE_DIR, COMPOSER_PATH, DISABLE_NSJAIL, DISABLE_NUSER, NSJAIL_PATH,
     PHP_PATH,
 };
+use windmill_common::client::AuthedClient;
 
 const NSJAIL_CONFIG_RUN_PHP_CONTENT: &str = include_str!("../nsjail/run.php.config.proto");
 

backend/windmill-worker/src/python_executor.rs

@@ -3,6 +3,7 @@ use std::{
     fs,
     path::Path,
     process::Stdio,
+    str::FromStr,
     sync::Arc,
 };
 
@@ -38,12 +39,12 @@ use std::env::var;
 use windmill_queue::{append_logs, CanceledBy, PrecomputedAgentInfo};
 
 lazy_static::lazy_static! {
-    static ref PYTHON_PATH: Option<String> = var("PYTHON_PATH").ok().map(|v| {
+    pub(crate) static ref PYTHON_PATH: Option<String> = var("PYTHON_PATH").ok().map(|v| {
         tracing::warn!("PYTHON_PATH is set to {} and thus python will not be managed by uv and stay static regardless of annotation and instance settings. NOT RECOMMENDED", v);
         v
     });
 
-    static ref UV_PATH: String =
+    pub(crate) static ref UV_PATH: String =
     var("UV_PATH").unwrap_or_else(|_| "/usr/local/bin/uv".to_string());
 
     static ref PY_CONCURRENT_DOWNLOADS: usize =
@@ -69,7 +70,7 @@ const RELATIVE_PYTHON_LOADER: &str = include_str!("../loader.py");
 use crate::global_cache::{build_tar_and_push, pull_from_tar};
 
 #[cfg(all(feature = "enterprise", feature = "parquet", unix))]
-use windmill_common::s3_helpers::OBJECT_STORE_CACHE_SETTINGS;
+use windmill_common::s3_helpers::OBJECT_STORE_SETTINGS;
 
 use crate::{
     common::{
@@ -77,347 +78,11 @@ use crate::{
         start_child_process, OccupancyMetrics,
     },
     handle_child::handle_child,
-    worker_lockfiles::LOCKFILE_GENERATED_FROM_REQUIREMENTS_TXT,
     worker_utils::ping_job_status,
-    AuthedClient, DISABLE_NSJAIL, DISABLE_NUSER, HOME_ENV, INSTANCE_PYTHON_VERSION, NSJAIL_PATH,
-    PATH_ENV, PIP_EXTRA_INDEX_URL, PIP_INDEX_URL, PROXY_ENVS, PY_INSTALL_DIR, TZ_ENV, UV_CACHE_DIR,
+    PyV, PyVAlias, DISABLE_NSJAIL, DISABLE_NUSER, HOME_ENV, NSJAIL_PATH, PATH_ENV,
+    PIP_EXTRA_INDEX_URL, PIP_INDEX_URL, PROXY_ENVS, PY_INSTALL_DIR, TZ_ENV, UV_CACHE_DIR,
 };
-
-// To change latest stable version:
-// 1. Change placeholder in instanceSettings.ts
-// 2. Change LATEST_STABLE_PY in dockerfile
-// 3. Change #[default] annotation for PyVersion in backend
-#[derive(Eq, PartialEq, Clone, Copy, Default, Debug)]
-pub enum PyVersion {
-    Py310,
-    #[default]
-    Py311,
-    Py312,
-    Py313,
-}
-
-impl PyVersion {
-    pub async fn from_instance_version(job_id: &Uuid, w_id: &str, conn: &Connection) -> Self {
-        let mut err = None;
-        let pyv = match INSTANCE_PYTHON_VERSION.read().await.clone() {
-            Some(v) => PyVersion::from_string_with_dots(&v).unwrap_or_else(|| {
-                let v = PyVersion::default();
-                err = Some(format!("\nCannot parse INSTANCE_PYTHON_VERSION ({:?}), fallback to latest_stable ({v:?})", *INSTANCE_PYTHON_VERSION));
-                v
-            }),
-            // Use latest stable
-            None => PyVersion::default(),
-        };
-
-        if let Some(msg) = err {
-            append_logs(job_id, w_id, &msg, conn).await;
-            tracing::error!(msg);
-        }
-        pyv
-    }
-    /// e.g.: `/tmp/windmill/cache/python_3xy`
-    pub fn to_cache_dir(&self) -> String {
-        use windmill_common::worker::ROOT_CACHE_DIR;
-        format!("{ROOT_CACHE_DIR}{}", &self.to_cache_dir_top_level())
-    }
-    /// e.g.: `python_3xy`
-    pub fn to_cache_dir_top_level(&self) -> String {
-        format!("python_{}", self.to_string_no_dot())
-    }
-    /// e.g.: `3xy`
-    pub fn to_string_no_dot(&self) -> String {
-        self.to_string_with_dot().replace('.', "")
-    }
-    /// e.g.: `3.xy`
-    pub fn to_string_with_dot(&self) -> &str {
-        use PyVersion::*;
-        match self {
-            Py310 => "3.10",
-            Py311 => "3.11",
-            Py312 => "3.12",
-            Py313 => "3.13",
-        }
-    }
-    pub fn from_string_with_dots(value: &str) -> Option<Self> {
-        use PyVersion::*;
-        match value {
-            "3.10" => Some(Py310),
-            "3.11" => Some(Py311),
-            "3.12" => Some(Py312),
-            "3.13" => Some(Py313),
-            "default" => Some(PyVersion::default()),
-            _ => {
-                tracing::warn!(
-                    "Cannot convert string (\"{value}\") to PyVersion\nExpected format x.yz"
-                );
-                None
-            }
-        }
-    }
-    pub fn from_string_no_dots(value: &str) -> Option<Self> {
-        use PyVersion::*;
-        match value {
-            "310" => Some(Py310),
-            "311" => Some(Py311),
-            "312" => Some(Py312),
-            "313" => Some(Py313),
-            "default" => Some(PyVersion::default()),
-            _ => {
-                tracing::warn!(
-                    "Cannot convert string (\"{value}\") to PyVersion\nExpected format xyz"
-                );
-                None
-            }
-        }
-    }
-    /// e.g.: `# py3xy` -> `PyVersion::Py3XY`
-    pub fn parse_version(line: &str) -> Option<Self> {
-        Self::from_string_no_dots(line.replace(" ", "").replace("#py", "").as_str())
-    }
-    pub fn from_py_annotations(a: PythonAnnotations) -> Option<Self> {
-        let PythonAnnotations { py310, py311, py312, py313, .. } = a;
-        use PyVersion::*;
-        if py313 {
-            Some(Py313)
-        } else if py312 {
-            Some(Py312)
-        } else if py311 {
-            Some(Py311)
-        } else if py310 {
-            Some(Py310)
-        } else {
-            None
-        }
-    }
-    pub fn from_numeric(n: u32) -> Option<Self> {
-        use PyVersion::*;
-        match n {
-            310 => Some(Py310),
-            311 => Some(Py311),
-            312 => Some(Py312),
-            313 => Some(Py313),
-            _ => None,
-        }
-    }
-    pub fn to_numeric(&self) -> u32 {
-        use PyVersion::*;
-        match self {
-            Py310 => 310,
-            Py311 => 311,
-            Py312 => 312,
-            Py313 => 313,
-        }
-    }
-    pub async fn get_python(
-        &self,
-        job_id: &Uuid,
-        mem_peak: &mut i32,
-        // canceled_by: &mut Option<CanceledBy>,
-        conn: &Connection,
-        worker_name: &str,
-        w_id: &str,
-        occupancy_metrics: &mut Option<&mut OccupancyMetrics>,
-    ) -> error::Result<Option<String>> {
-        // lazy_static::lazy_static! {
-        //     static ref PYTHON_PATHS: Arc<RwLock<HashMap<PyVersion, String>>> = Arc::new(RwLock::new(HashMap::new()));
-        // }
-
-        let res = self
-            .get_python_inner(job_id, mem_peak, conn, worker_name, w_id, occupancy_metrics)
-            .await;
-
-        if let Err(ref e) = res {
-            tracing::error!(
-                "worker_name: {worker_name}, w_id: {w_id}, job_id: {job_id}\n
-                Error while getting python from uv, falling back to system python: {e:?}"
-            );
-            append_logs(
-                job_id,
-                w_id,
-                format!(
-                    "\nError while getting python from uv, falling back to system python: {e:?}"
-                ),
-                conn,
-            )
-            .await;
-        }
-        res
-    }
-    async fn get_python_inner(
-        self,
-        job_id: &Uuid,
-        mem_peak: &mut i32,
-        // canceled_by: &mut Option<CanceledBy>,
-        conn: &Connection,
-        worker_name: &str,
-        w_id: &str,
-        occupancy_metrics: &mut Option<&mut OccupancyMetrics>,
-    ) -> error::Result<Option<String>> {
-        let py_path = self.find_python().await;
-
-        // Runtime is not installed
-        if py_path.is_err() {
-            // Install it
-            if let Err(err) = self
-                .install_python(job_id, mem_peak, conn, worker_name, w_id, occupancy_metrics)
-                .await
-            {
-                tracing::error!("Cannot install python: {err}");
-                return Err(err);
-            } else {
-                // Try to find one more time
-                let py_path = self.find_python().await;
-
-                if let Err(err) = py_path {
-                    tracing::error!("Cannot find python version {err}");
-                    return Err(err);
-                }
-
-                // TODO: Cache the result
-                py_path
-            }
-        } else {
-            py_path
-        }
-    }
-    async fn install_python(
-        self,
-        job_id: &Uuid,
-        mem_peak: &mut i32,
-        // canceled_by: &mut Option<CanceledBy>,
-        conn: &Connection,
-        worker_name: &str,
-        w_id: &str,
-        occupancy_metrics: &mut Option<&mut OccupancyMetrics>,
-    ) -> error::Result<()> {
-        let v = self.to_string_with_dot();
-        append_logs(job_id, w_id, format!("\nINSTALLING PYTHON ({})", v), conn).await;
-        // Create dirs for newly installed python
-        // If we dont do this, NSJAIL will not be able to mount cache
-        // For the default version directory created during startup (main.rs)
-        DirBuilder::new()
-            .recursive(true)
-            .create(self.to_cache_dir())
-            .await
-            .expect("could not create initial worker dir");
-
-        let logs = String::new();
-
-        #[cfg(windows)]
-        let uv_cmd = "uv";
-
-        #[cfg(unix)]
-        let uv_cmd = UV_PATH.as_str();
-
-        let mut child_cmd = Command::new(uv_cmd);
-        child_cmd
-            .env_clear()
-            .env("HOME", HOME_ENV.to_string())
-            .env("PATH", PATH_ENV.to_string())
-            .envs(PROXY_ENVS.clone())
-            .args(["python", "install", v, "--python-preference=only-managed"])
-            // TODO: Do we need these?
-            .envs([("UV_PYTHON_INSTALL_DIR", PY_INSTALL_DIR)])
-            .stdout(Stdio::piped())
-            .stderr(Stdio::piped());
-
-        #[cfg(windows)]
-        {
-            child_cmd
-                .env("SystemRoot", SYSTEM_ROOT.as_str())
-                .env("USERPROFILE", crate::USERPROFILE_ENV.as_str())
-                .env(
-                    "TMP",
-                    std::env::var("TMP").unwrap_or_else(|_| String::from("/tmp")),
-                )
-                .env(
-                    "LOCALAPPDATA",
-                    std::env::var("LOCALAPPDATA")
-                        .unwrap_or_else(|_| format!("{}\\AppData\\Local", HOME_ENV.as_str())),
-                );
-        }
-
-        let child_process = start_child_process(child_cmd, "uv").await?;
-
-        append_logs(&job_id, &w_id, logs, conn).await;
-        handle_child(
-            job_id,
-            conn,
-            mem_peak,
-            &mut None,
-            child_process,
-            false,
-            worker_name,
-            &w_id,
-            "uv",
-            None,
-            false,
-            occupancy_metrics,
-            None,
-        )
-        .await
-    }
-    async fn find_python(self) -> error::Result<Option<String>> {
-        #[cfg(windows)]
-        let uv_cmd = "uv";
-
-        #[cfg(unix)]
-        let uv_cmd = UV_PATH.as_str();
-
-        let mut child_cmd = Command::new(uv_cmd);
-
-        child_cmd.env_clear();
-
-        #[cfg(windows)]
-        {
-            child_cmd
-                .env("SystemRoot", SYSTEM_ROOT.as_str())
-                .env("USERPROFILE", crate::USERPROFILE_ENV.as_str())
-                .env(
-                    "TMP",
-                    std::env::var("TMP").unwrap_or_else(|_| String::from("/tmp")),
-                )
-                .env(
-                    "LOCALAPPDATA",
-                    std::env::var("LOCALAPPDATA")
-                        .unwrap_or_else(|_| format!("{}\\AppData\\Local", HOME_ENV.as_str())),
-                );
-        }
-
-        let output = child_cmd
-            // .current_dir(job_dir)
-            .env("HOME", HOME_ENV.to_string())
-            .env("PATH", PATH_ENV.to_string())
-            .args([
-                "python",
-                "find",
-                self.to_string_with_dot(),
-                "--system",
-                "--python-preference=only-managed",
-            ])
-            .envs([
-                ("UV_PYTHON_INSTALL_DIR", PY_INSTALL_DIR),
-                ("UV_PYTHON_PREFERENCE", "only-managed"),
-            ])
-            // .stdout(Stdio::piped())
-            .stderr(Stdio::piped())
-            .output()
-            .await?;
-
-        // Check if the command was successful
-        if output.status.success() {
-            // Convert the output to a String
-            let stdout =
-                String::from_utf8(output.stdout).expect("Failed to convert output to String");
-            return Ok(Some(stdout.replace('\n', "")));
-        } else {
-            // If the command failed, print the error
-            let stderr =
-                String::from_utf8(output.stderr).expect("Failed to convert error output to String");
-            return Err(error::Error::FindPythonError(stderr));
-        }
-    }
-}
+use windmill_common::client::AuthedClient;
 
 #[cfg(windows)]
 use crate::SYSTEM_ROOT;
@@ -465,7 +130,7 @@ pub async fn uv_pip_compile(
     worker_name: &str,
     w_id: &str,
     occupancy_metrics: &mut Option<&mut OccupancyMetrics>,
-    py_version: PyVersion,
+    py_version: PyV,
     // Debug-only flag
     no_cache: bool,
 ) -> error::Result<String> {
@@ -502,10 +167,11 @@ pub async fn uv_pip_compile(
         requirements.to_string()
     };
 
+    let py_version_str = py_version.clone().to_string();
     // Include python version to requirements.in
     // We need it because same hash based on requirements.in can get calculated even for different python versions
     // To prevent from overwriting same requirements.in but with different python versions, we include version to hash
-    let requirements = format!("# py{}\n{}", py_version.to_string_no_dot(), requirements);
+    let requirements = format!("# py: {}\n{}", py_version.to_string(), requirements);
 
     #[cfg(feature = "enterprise")]
     let requirements = replace_pip_secret(conn, w_id, &requirements, worker_name, job_id).await?;
@@ -525,7 +191,7 @@ pub async fn uv_pip_compile(
             {
                 logs.push_str(&format!(
                     "\nFound cached resolution: {req_hash}, on python version: {}",
-                    py_version.to_string_with_dot()
+                    &py_version_str
                 ));
                 return Ok(cached);
             }
@@ -539,7 +205,7 @@ pub async fn uv_pip_compile(
     {
         // Make sure we have python runtime installed
         py_version
-            .get_python(job_id, mem_peak, conn, worker_name, w_id, occupancy_metrics)
+            .try_get_python(job_id, mem_peak, conn, worker_name, w_id, occupancy_metrics)
             .await?;
 
         let mut args = vec![
@@ -561,12 +227,7 @@ pub async fn uv_pip_compile(
             UV_CACHE_DIR,
         ];
 
-        args.extend([
-            "-p",
-            &py_version.to_string_with_dot(),
-            "--python-preference",
-            "only-managed",
-        ]);
+        args.extend(["-p", &py_version_str, "--python-preference", "only-managed"]);
 
         if no_cache {
             args.extend(["--no-cache"]);
@@ -666,8 +327,8 @@ pub async fn uv_pip_compile(
     let mut req_content = "".to_string();
     file.read_to_string(&mut req_content).await?;
     let lockfile = format!(
-        "# py{}\n{}",
-        py_version.to_string_no_dot(),
+        "# py: {}\n{}",
+        py_version.to_string(),
         req_content
             .lines()
             .filter(|x| !x.trim_start().starts_with('#'))
@@ -677,7 +338,7 @@ pub async fn uv_pip_compile(
     );
     if let Some(db) = conn.as_sql() {
         sqlx::query!(
-        "INSERT INTO pip_resolution_cache (hash, lockfile, expiration) VALUES ($1, $2, now() + ('3 days')::interval) ON CONFLICT (hash) DO UPDATE SET lockfile = $2",
+        "INSERT INTO pip_resolution_cache (hash, lockfile, expiration) VALUES ($1, $2, now() + ('5 mins')::interval) ON CONFLICT (hash) DO UPDATE SET lockfile = $2",
         req_hash,
         lockfile
     ).fetch_optional(db).await?;
@@ -789,37 +450,6 @@ async fn postinstall(
     Ok(())
 }
 
-async fn get_python_path(
-    py_version: PyVersion,
-    worker_name: &str,
-    job_id: &Uuid,
-    w_id: &str,
-    mem_peak: &mut i32,
-    conn: &Connection,
-    occupancy_metrics: &mut Option<&mut OccupancyMetrics>,
-) -> windmill_common::error::Result<String> {
-    let python_path = if let Some(python_path) = PYTHON_PATH.clone() {
-        python_path
-    } else if let Some(python_path) = py_version
-        .get_python(
-            &job_id,
-            mem_peak,
-            conn,
-            worker_name,
-            w_id,
-            occupancy_metrics,
-        )
-        .await?
-    {
-        python_path
-    } else {
-        return Err(Error::ExecutionErr(format!(
-            "uv could not manage python path. Please manage it manually by setting PYTHON_PATH environment variable to your python binary path"
-        )));
-    };
-    Ok(python_path)
-}
-
 #[tracing::instrument(level = "trace", skip_all)]
 pub async fn handle_python_job(
     requirements_o: Option<&String>,
@@ -842,6 +472,8 @@ pub async fn handle_python_job(
 ) -> windmill_common::error::Result<Box<RawValue>> {
     let script_path = crate::common::use_flow_root_path(job.runnable_path());
 
+    let annotations = PythonAnnotations::parse(inner_content);
+
     let (py_version, mut additional_python_paths) = handle_python_deps(
         job_dir,
         requirements_o,
@@ -856,23 +488,23 @@ pub async fn handle_python_job(
         canceled_by,
         &mut Some(occupancy_metrics),
         precomputed_agent_info,
+        annotations,
     )
     .await?;
 
-    let PythonAnnotations { no_postinstall, .. } = PythonAnnotations::parse(inner_content);
     tracing::debug!("Finished handling python dependencies");
-    let python_path = get_python_path(
-        py_version,
-        worker_name,
-        &job.id,
-        &job.workspace_id,
-        mem_peak,
-        conn,
-        &mut Some(occupancy_metrics),
-    )
-    .await?;
+    let python_path = py_version
+        .get_python(
+            worker_name,
+            &job.id,
+            &job.workspace_id,
+            mem_peak,
+            conn,
+            &mut Some(occupancy_metrics),
+        )
+        .await?;
 
-    if !no_postinstall {
+    if !annotations.no_postinstall {
         if let Err(e) = postinstall(&mut additional_python_paths, job_dir, job, conn).await {
             tracing::error!("Postinstall stage has failed. Reason: {e}");
         }
@@ -885,7 +517,7 @@ pub async fn handle_python_job(
             &job.workspace_id,
             format!(
                 "\n\n--- PYTHON ({}) CODE EXECUTION ---\n",
-                py_version.to_string_with_dot()
+                py_version.clone().to_string()
             ),
             conn,
         )
@@ -938,6 +570,8 @@ pub async fn handle_python_job(
         "".to_string()
     };
 
+    let postprocessor = get_result_postprocessor(annotations.skip_result_postprocessing);
+
     let os_main_override = if let Some(main_override) = main_name.as_ref() {
         format!("os.environ[\"MAIN_OVERRIDE\"] = \"{main_override}\"\n")
     } else {
@@ -984,7 +618,8 @@ def res_to_json(res):
         for k, v in res.items():
             if type(v).__name__ == 'bytes':
                 res[k] = to_b_64(v)
-    return re.sub(replace_invalid_fields, ' null ', json.dumps(res, separators=(',', ':'), default=str).replace('\n', ''))
+    unprocessed = json.dumps(res, separators=(',', ':'), default=str).replace('\n', '')
+    return {postprocessor}
 
 try:
     {preprocessor}
@@ -1021,7 +656,7 @@ except BaseException as e:
     let mut reserved_variables =
         get_reserved_variables(job, &client.token, conn, parent_runnable_path).await?;
 
-    // Add /tmp/windmill/cache/python_xyz/global-site-packages to PYTHONPATH.
+    // Add /tmp/windmill/cache/python_x_y_z/global-site-packages to PYTHONPATH.
     // Usefull if certain wheels needs to be preinstalled before execution.
     let global_site_packages_path = py_version.to_cache_dir() + "/global-site-packages";
     let additional_python_paths_folders = {
@@ -1034,9 +669,9 @@ except BaseException as e:
             // Since we handle mount of global_site_packages on our own, we don't want it to be mounted automatically.
             // We do this because existence of every wheel in cache is mandatory and if it is not there and nsjail expects it, it is a bug.
             // On the other side global_site_packages is purely optional.
-            // NOTE: This behaviour can be changed in future, so verification of wheels can be offloaded from nsjail to windmill
+            // NOTE: This behaviour can be changed in future, so verification of wheels can be delegated from nsjail to windmill
             paths.insert(0, global_site_packages_path.clone());
-            //    ^^^^^^^^
+            //    ^^^^^^ ^
             // We also want this be priorotized, that's why we insert it to the beginning
         }
         paths.iter().join(":")
@@ -1428,7 +1063,8 @@ async fn handle_python_deps(
     canceled_by: &mut Option<CanceledBy>,
     occupancy_metrics: &mut Option<&mut OccupancyMetrics>,
     precomputed_agent_info: Option<PrecomputedAgentInfo>,
-) -> error::Result<(PyVersion, Vec<String>)> {
+    annotations: PythonAnnotations,
+) -> error::Result<(PyV, Vec<String>)> {
     create_dependencies_dir(job_dir).await;
 
     let mut additional_python_paths: Vec<String> = WORKER_CONFIG
@@ -1439,91 +1075,116 @@ async fn handle_python_deps(
         .unwrap_or_else(|| vec![])
         .clone();
 
-    let mut requirements;
-    let compilation_error_hint;
-    let mut annotated_pyv = None;
-    let mut annotated_pyv_numeric = None;
-    let is_deployed = requirements_o.is_some();
-    let instance_pyv = PyVersion::from_instance_version(job_id, w_id, conn).await;
-    let annotations = windmill_common::worker::PythonAnnotations::parse(inner_content);
-    let requirements = match requirements_o {
-        Some(r) => r,
+    let (pyv, resolved_lines) = match requirements_o {
+        // Deployed
+        Some(r) => {
+            let rl = split_requirements(r);
+            (PyV::parse_from_requirements(&rl), rl)
+        }
+        // Preview
         None => {
-            let mut already_visited = vec![];
-
-            (requirements, compilation_error_hint) = match conn {
+            let (v, requirements_lines, error_hint) = match conn {
                 Connection::Sql(db) => {
+                    let mut version_specifiers = vec![];
                     let (r, h) = windmill_parser_py_imports::parse_python_imports(
                         inner_content,
                         w_id,
                         script_path,
                         db,
-                        &mut already_visited,
-                        &mut annotated_pyv_numeric,
+                        &mut version_specifiers,
                     )
                     .await?;
 
-                    (r.join("\n"), h)
+                    let v = PyV::resolve(
+                        version_specifiers,
+                        job_id,
+                        w_id,
+                        annotations.py_select_latest,
+                        Some(conn.clone()),
+                        None,
+                        None,
+                    )
+                    .await?;
+
+                    (v, r, h)
                 }
                 Connection::Http(_) => match precomputed_agent_info {
-                    Some(PrecomputedAgentInfo::Python { py_version, requirements }) => {
-                        annotated_pyv_numeric = py_version;
-                        (requirements.clone().unwrap_or_else(|| "".to_string()), None)
+                    Some(PrecomputedAgentInfo::Python {
+                        requirements,
+                        py_version,
+                        py_version_v2,
+                    }) => {
+                        let v = {
+                            let v_v2 = py_version_v2
+                                .clone()
+                                .and_then(|s| pep440_rs::Version::from_str(&s).ok().map(PyV::from));
+                            let v_v1 = py_version.and_then(PyVAlias::try_from_v1).map(PyV::from);
+
+                            match v_v2.or(v_v1) {
+                                Some(v) => v,
+                                None => {
+                                    tracing::warn!(
+                                        workspace_id = %w_id,
+                                        "
+Failed to get precomputed python version from server. Fallback to Default ({})
+Returned from server: py_version - {:?}, py_version_v2 - {:?}
+                                        ",
+                                        *PyV::default(),
+                                        py_version,
+                                        py_version_v2
+                                    );
+                                    Default::default()
+                                }
+                            }
+                        };
+
+                        let r = split_requirements(requirements.unwrap_or_default());
+                        let h = None;
+
+                        (v, r, h)
                     }
-                    _ => ("".to_string(), None),
+                    _ => Default::default(),
                 },
             };
 
-            annotated_pyv = annotated_pyv_numeric.and_then(|v| PyVersion::from_numeric(v));
-
-            if !requirements.is_empty() {
-                requirements = uv_pip_compile(
-                    job_id,
-                    &requirements,
-                    mem_peak,
-                    canceled_by,
-                    job_dir,
-                    conn,
-                    worker_name,
-                    w_id,
-                    occupancy_metrics,
-                    annotated_pyv.unwrap_or(instance_pyv),
-                    annotations.no_cache,
-                )
-                .await
-                .map_err(|e| {
-                    Error::ExecutionErr(format!(
-                        "pip compile failed: {}{}",
-                        e.to_string(),
-                        compilation_error_hint.unwrap_or_default()
-                    ))
-                })?;
-            }
-            &requirements
+            (
+                v.clone(),
+                if !requirements_lines.is_empty() {
+                    uv_pip_compile(
+                        job_id,
+                        &requirements_lines.join("\n"),
+                        mem_peak,
+                        canceled_by,
+                        job_dir,
+                        conn,
+                        worker_name,
+                        w_id,
+                        occupancy_metrics,
+                        // annotated_pyv.unwrap_or(instance_pyv),
+                        v,
+                        annotations.no_cache,
+                    )
+                    .await
+                    .map_err(|e| {
+                        Error::ExecutionErr(format!(
+                            "pip compile failed: {}{}",
+                            e.to_string(),
+                            error_hint.unwrap_or_default()
+                        ))
+                    })?
+                    .lines()
+                    .map(|s| s.to_owned())
+                    .collect_vec()
+                } else {
+                    vec![]
+                },
+            )
         }
     };
 
-    /*
-     For deployed scripts we want to find out version in following order:
-     1. Assigned version (written in lockfile)
-     2. 3.11
-
-     For Previews:
-     1. Annotated version
-     2. Instance version
-     3. Latest Stable
-    */
-    let requirements_lines = split_requirements(requirements.as_str());
-    let final_version = if is_deployed {
-        get_pyv_from_requirements_lines(&requirements_lines)
-    } else {
-        // This is not deployed script, meaning we test run it (Preview)
-        annotated_pyv.unwrap_or(instance_pyv)
-    };
-    // If len > 0 it means there is atleast one dependency or assigned python version
-    if requirements.len() > 0 {
+    if !resolved_lines.is_empty() {
         let mut venv_path = handle_python_reqs(
-            requirements_lines,
+            resolved_lines,
             job_id,
             w_id,
             mem_peak,
@@ -1533,13 +1194,13 @@ async fn handle_python_deps(
             job_dir,
             worker_dir,
             occupancy_metrics,
-            final_version,
+            pyv.clone(),
         )
         .await?;
         additional_python_paths.append(&mut venv_path);
     }
 
-    Ok((final_version, additional_python_paths))
+    Ok((pyv, additional_python_paths))
 }
 
 lazy_static::lazy_static! {
@@ -1728,7 +1389,7 @@ async fn spawn_uv_install(
 
 /// uv pip install, include cached or pull from S3
 pub async fn handle_python_reqs(
-    requirements: Vec<&str>,
+    requirements: Vec<String>,
     job_id: &Uuid,
     w_id: &str,
     mem_peak: &mut i32,
@@ -1738,7 +1399,7 @@ pub async fn handle_python_reqs(
     job_dir: &str,
     worker_dir: &str,
     _occupancy_metrics: &mut Option<&mut OccupancyMetrics>,
-    py_version: PyVersion,
+    py_version: PyV,
 ) -> error::Result<Vec<String>> {
     let worker_dir = worker_dir.to_string();
 
@@ -1764,7 +1425,7 @@ pub async fn handle_python_reqs(
         }
 
         #[cfg(all(feature = "enterprise", feature = "parquet", unix))]
-        if OBJECT_STORE_CACHE_SETTINGS.read().await.is_none() {
+        if OBJECT_STORE_SETTINGS.read().await.is_none() {
             (s3_pull, s3_push) = (false, false);
         }
 
@@ -2012,7 +1673,7 @@ pub async fn handle_python_reqs(
 
     let total_time = std::time::Instant::now();
     let py_path = py_version
-        .get_python(
+        .try_get_python(
             job_id,
             mem_peak,
             conn,
@@ -2054,6 +1715,10 @@ pub async fn handle_python_reqs(
         let py_path = py_path.clone();
         let pids = pids.clone();
         let worker_dir = worker_dir.clone();
+
+        #[cfg(all(feature = "enterprise", feature = "parquet", unix))]
+        let py_version = py_version.clone();
+
         handles.push(task::spawn(async move {
             // permit will be dropped anyway if this thread exits at any point
             // so we dont have to drop it manually
@@ -2071,7 +1736,7 @@ pub async fn handle_python_reqs(
             let start = std::time::Instant::now();
             #[cfg(all(feature = "enterprise", feature = "parquet", unix))]
             if is_not_pro {
-                if let Some(os) = OBJECT_STORE_CACHE_SETTINGS.read().await.clone() {
+                if let Some(os) = windmill_common::s3_helpers::get_object_store().await {
                     tokio::select! {
                         // Cancel was called on the job
                         _ = kill_rx.recv() => return Err(anyhow::anyhow!("S3 pull was canceled")),
@@ -2225,7 +1890,7 @@ pub async fn handle_python_reqs(
 
             #[cfg(all(feature = "enterprise", feature = "parquet", unix))]
             if s3_push {
-                if let Some(os) = OBJECT_STORE_CACHE_SETTINGS.read().await.clone() {
+                if let Some(os) = windmill_common::s3_helpers::get_object_store().await {
                     tokio::spawn(build_tar_and_push(os, venv_p.clone(), py_version.to_cache_dir_top_level(), None, false));
                 }
             }
@@ -2295,34 +1960,21 @@ pub async fn handle_python_reqs(
     };
 }
 
-fn split_requirements(requirements: &str) -> Vec<&str> {
+pub fn split_requirements<T: AsRef<str>>(requirements: T) -> Vec<String> {
     requirements
-        .split("\n")
+        .as_ref()
+        .lines()
         .filter(|x| !x.trim_start().starts_with("--") && !x.trim().is_empty())
+        .map(String::from)
         .collect()
 }
-/// Check requirements/lockfile to figure out python version assigned to it.
-fn get_pyv_from_requirements_lines(requirements_lines: &[&str]) -> PyVersion {
-    // If script is deployed we can try to parse first line to get assigned version
 
-    let index = if requirements_lines.get(0).map_or(false, |line| {
-        line.starts_with(LOCKFILE_GENERATED_FROM_REQUIREMENTS_TXT)
-    }) {
-        1
+// Returns code snippet that needs to be injected into wrapper to post-process results or leave unprocessed
+fn get_result_postprocessor<'a>(skip: bool) -> &'a str {
+    if skip {
+        "unprocessed"
     } else {
-        0
-    };
-    if let Some(v) = requirements_lines
-        .get(index)
-        .and_then(|line| PyVersion::parse_version(*line))
-    {
-        // We have valid assigned version, we use it
-        v
-    } else {
-        // If there is no assigned version in lockfile we automatically fallback to 3.11
-        // In this case we have dependencies, but no associated python version
-        // This is the case for old deployed scripts
-        PyVersion::Py311
+        "re.sub(replace_invalid_fields, ' null ', unprocessed)"
     }
 }
 
@@ -2351,6 +2003,8 @@ pub async fn start_worker(
     jobs_rx: tokio::sync::mpsc::Receiver<std::sync::Arc<MiniPulledJob>>,
     killpill_rx: tokio::sync::broadcast::Receiver<()>,
 ) -> error::Result<()> {
+    use crate::{PyV, PyVAlias};
+
     let mut mem_peak: i32 = 0;
     let mut canceled_by: Option<CanceledBy> = None;
     let context = variables::get_reserved_variables(
@@ -2373,6 +2027,7 @@ pub async fn start_worker(
     .await
     .to_vec();
 
+    let annotations = PythonAnnotations::parse(inner_content);
     let context_envs = build_envs_map(context).await;
     let (_, additional_python_paths) = handle_python_deps(
         job_dir,
@@ -2388,6 +2043,7 @@ pub async fn start_worker(
         &mut canceled_by,
         &mut None,
         None,
+        annotations,
     )
     .await?;
 
@@ -2405,6 +2061,7 @@ pub async fn start_worker(
     ) = prepare_wrapper(job_dir, false, None, None, inner_content, script_path).await?;
 
     {
+        let postprocessor = get_result_postprocessor(annotations.skip_result_postprocessing);
         let indented_transforms = transforms
             .lines()
             .map(|x| format!("    {}", x))
@@ -2456,7 +2113,8 @@ for line in sys.stdin:
             for k, v in res.items():
                 if type(v).__name__ == 'bytes':
                     res[k] = to_b_64(v)
-        res_json = re.sub(replace_invalid_fields, ' null ', json.dumps(res, separators=(',', ':'), default=str).replace('\n', ''))
+        unprocessed = json.dumps(res, separators=(',', ':'), default=str).replace('\n', '')
+        res_json = {postprocessor}
         sys.stdout.write("wm_res[success]:" + res_json + "\n")
     except BaseException as e:
         exc_type, exc_value, exc_traceback = sys.exc_info()
@@ -2500,22 +2158,22 @@ for line in sys.stdin:
     proc_envs.insert("BASE_URL".to_string(), base_internal_url.to_string());
 
     let py_version = if let Some(requirements) = requirements_o {
-        get_pyv_from_requirements_lines(&split_requirements(requirements.as_str()))
+        PyV::parse_from_requirements(&split_requirements(requirements.as_str()))
     } else {
         tracing::warn!(workspace_id = %w_id, "lockfile is empty for dedicated worker, thus python version cannot be inferred. Fallback to 3.11");
-        PyVersion::Py311
+        PyVAlias::Py311.into()
     };
 
-    let python_path = get_python_path(
-        py_version,
-        worker_name,
-        &Uuid::nil(),
-        w_id,
-        &mut mem_peak,
-        &Connection::Sql(db.clone()),
-        &mut None,
-    )
-    .await?;
+    let python_path = py_version
+        .get_python(
+            worker_name,
+            &Uuid::nil(),
+            w_id,
+            &mut mem_peak,
+            &Connection::Sql(db.clone()),
+            &mut None,
+        )
+        .await?;
     handle_dedicated_process(
         &python_path,
         job_dir,

backend/windmill-worker/src/python_versions.rs

@@ -0,0 +1,848 @@
+use std::{
+    ops::{Deref, DerefMut},
+    process::Stdio,
+    str::FromStr,
+    sync::Arc,
+};
+
+use chrono::{DateTime, Duration, Utc};
+use itertools::Itertools;
+use serde_json::Value;
+use tokio::{fs::DirBuilder, process::Command, sync::RwLock};
+use uuid::Uuid;
+use windmill_common::{
+    error::{self, Error},
+    worker::Connection,
+};
+
+use anyhow::{anyhow, bail};
+use windmill_queue::append_logs;
+
+use crate::{
+    common::{start_child_process, OccupancyMetrics},
+    handle_child::handle_child,
+    python_executor::{PYTHON_PATH, UV_PATH},
+    worker_lockfiles::LOCKFILE_GENERATED_FROM_REQUIREMENTS_TXT,
+    HOME_ENV, INSTANCE_PYTHON_VERSION, PATH_ENV, PROXY_ENVS, PY_INSTALL_DIR, WIN_ENVS,
+};
+
+#[derive(Eq, PartialEq, Clone, Copy, Default, Debug)]
+#[repr(u32)]
+pub enum PyVAlias {
+    Py310 = 10,
+    #[default]
+    Py311,
+    Py312,
+    Py313,
+}
+
+impl Into<pep440_rs::Version> for PyVAlias {
+    fn into(self) -> pep440_rs::Version {
+        pep440_rs::Version::new([self.major() as u64, self as u64])
+    }
+}
+
+impl Into<u32> for PyVAlias {
+    fn into(self) -> u32 {
+        self.major() * 100 + self as u32
+    }
+}
+
+impl From<PyV> for PyVAlias {
+    fn from(value: PyV) -> Self {
+        match value.release() {
+            [major, minor, ..] => {
+                if let Some(alias) = Self::try_from_v1(format!("{}{}", *major, *minor)) {
+                    return alias;
+                }
+            }
+            _ => (),
+        }
+
+        tracing::warn!(
+            "Failed to convert Python Full Version to Alias. Fallback to default ({})",
+            *PyV::default()
+        );
+        Self::default()
+    }
+}
+impl PyVAlias {
+    fn all<T: From<PyVAlias>>() -> Vec<T> {
+        use PyVAlias::*;
+        vec![Py310.into(), Py311.into(), Py312.into(), Py313.into()]
+    }
+    // Get MAJOR part of alias. (semver: MAJOR.MINOR.PATCH)
+    fn major(&self) -> u32 {
+        use PyVAlias::*;
+        match self {
+            Py310 | Py311 | Py312 | Py313 => 3,
+            // Py400 | Py401 => 4
+        }
+    }
+
+    /// Converts numeric format to alias
+    /// Example:
+    /// 310u32 (in) -> PyVAlias::Py310 (out)
+    pub(crate) fn try_from_v1<T: ToString>(numeric: T) -> Option<Self> {
+        use PyVAlias::*;
+        match numeric.to_string().as_str() {
+            "310" => Some(Py310),
+            "311" => Some(Py311),
+            "312" => Some(Py312),
+            "313" => Some(Py313),
+            _ => None,
+        }
+    }
+}
+
+// To change latest stable version:
+// 1. Change placeholder in instanceSettings.ts
+// 2. Change LATEST_STABLE_PY in dockerfile
+// 3. Change #[default] annotation for PyVersion in backend
+#[derive(Debug, Clone, PartialEq, Eq, Hash, PartialOrd, Ord)]
+pub struct PyV(pub pep440_rs::Version);
+
+impl From<pep440_rs::Version> for PyV {
+    fn from(value: pep440_rs::Version) -> Self {
+        Self(value)
+    }
+}
+
+impl From<PyVAlias> for PyV {
+    fn from(value: PyVAlias) -> Self {
+        Self(value.into())
+    }
+}
+
+impl Default for PyV {
+    fn default() -> Self {
+        PyVAlias::default().into()
+    }
+}
+
+impl Deref for PyV {
+    type Target = pep440_rs::Version;
+
+    fn deref(&self) -> &Self::Target {
+        &self.0
+    }
+}
+impl DerefMut for PyV {
+    fn deref_mut(&mut self) -> &mut Self::Target {
+        &mut self.0
+    }
+}
+
+impl PyV {
+    pub async fn resolve(
+        version_specifiers: Vec<pep440_rs::VersionSpecifier>,
+        job_id: &Uuid,
+        w_id: &str,
+        select_latest: bool,
+        // Needed for logs but optional
+        conn: Option<Connection>,
+        // Usually for testing
+        custom_versions: Option<Vec<PyV>>,
+        // For testing
+        gravitational_version: Option<PyV>,
+    ) -> Result<Self, Error> {
+        // Get all versions that can be fetched
+        let all_versions = custom_versions.unwrap_or(PyV::list_available_python_versions().await);
+
+        // Narrow down to those that satisfy given version specifiers
+        let valid = all_versions
+            .clone()
+            .into_iter()
+            .filter(|v| version_specifiers.iter().all(|vs| vs.contains(&*v)))
+            .collect_vec();
+
+        if !valid.is_empty() {
+            if select_latest {
+                return Ok(valid[0].clone());
+            }
+
+            // Usually INSTANCE_PYTHON_VERSION
+            let gv = gravitational_version
+                .unwrap_or(PyV::gravitational_version(job_id, w_id, conn).await);
+
+            // Will be used to determine if picked version matches gravity version
+            // Once first match occure, we will stop iterating
+            let gravity_matcher = pep440_rs::VersionSpecifier::from_version(
+                pep440_rs::Operator::EqualStar,
+                (*gv).clone(),
+            )
+            .map_err(|e| {
+                Error::ArgumentErr(format!(
+                    "{e}\nLikely means INSTANCE_PYTHON_VERSION is set incorrectly."
+                ))
+            })?;
+
+            // Reminder of semver: MAJOR.MINOR.PATCH
+            //
+            // - Go from up to down
+            // - We will iterate until find the closest version to target.
+            //   - If closest version has the same MINOR version, use it.
+            //   - If it differs in MINOR version, take latest PATCH version.
+            //
+            let mut result = None;
+
+            // This represents newest version with oldest MINOR:
+            //
+            // I Iterable   Newest in MINOR
+            // 1. 3.11.2 -> 3.11.2
+            // 2. 3.11.1 -> 3.11.2
+            // 3. 3.11.0 -> 3.11.2
+            // 4. 3.10.2 -> 3.10.2
+            // 5. 3.10.1 -> 3.10.2
+            // 6. 3.10.0 -> 3.10.2
+            let mut newest_in_minor = None;
+            for v in valid.iter() {
+                if result.is_none() {
+                    result.replace(v);
+                }
+
+                if v < &gv {
+                    // We will not continue if we start looking into versions older than gravity version.
+                    break;
+                }
+
+                let [major, minor, ..] = v.release() else {
+                    return Err(Error::InternalErr(format!("Failed to parse \"{}\". Available python versions are supposed to be in SEMVER format (MAJOR.MINOR)", **v)));
+                };
+
+                // Since we go top to down we can assume
+                // the first occurence of new minor version contains the latest patch version.
+                if matches!(newest_in_minor, Some((_, mm)) if mm != (major, minor))
+                    || newest_in_minor.is_none()
+                {
+                    newest_in_minor = Some((v.clone(), (major, minor)));
+                }
+
+                if gravity_matcher.contains(v) {
+                    // return as soon as gravity matcher has first hit.
+                    return Ok(v.clone());
+                }
+                // If we are still in the loop, it means that we are getting closer to gravity version
+                else {
+                    result = Some(v);
+                }
+            }
+
+            let [gravity_major, gravity_minor, ..] = gv.release() else {
+                return Err(Error::internal_err(format!("Cannot get MAJOR or/and MINOR version of python gravity version ({}). Something might be wrong with INSTANCE_PYTHON_VERSION.", &*gv)));
+            };
+
+            if let Some((v, mm)) = newest_in_minor {
+                if (gravity_major, gravity_minor) != mm {
+                    return Ok(v);
+                }
+            }
+
+            result
+                .ok_or(Error::internal_err(
+                    "No python candidates found. This is a bug!",
+                ))
+                .map(ToOwned::to_owned)
+        } else {
+            Err(anyhow!(
+                "
+  × No solution found when resolving python:
+  ╰─▶ Because you require python {}, we can conclude that your requirements are unsatisfiable.
+
+  All versions: \n{}
+                \n",
+                version_specifiers.iter().map(|s| s.to_string()).join(", "),
+                all_versions
+                    .iter()
+                    .enumerate()
+                    .map(|(i, v)| format!(
+                        "{}{}",
+                        windmill_common::worker::pad_string(&v.0.to_string(), 11),
+                        if (i + 1) % 5 == 0 { "\n" } else { "" }
+                    ))
+                    .collect::<String>()
+            )
+            .into())
+        }
+    }
+    /// e.g.: `/tmp/windmill/cache/python_3xy`
+    pub(crate) fn to_cache_dir(&self) -> String {
+        use windmill_common::worker::ROOT_CACHE_DIR;
+        format!("{ROOT_CACHE_DIR}{}", self.to_cache_dir_top_level())
+    }
+
+    /// e.g.: `python_3_x_y`
+    pub fn to_cache_dir_top_level(&self) -> String {
+        format!("python_{}", self.to_string().replace(".", "_"))
+    }
+
+    pub async fn gravitational_version(
+        job_id: &Uuid,
+        w_id: &str,
+        conn: Option<Connection>,
+    ) -> Self {
+        let mut err = None;
+        let pyv = match INSTANCE_PYTHON_VERSION.read().await.clone() {
+            Some(v) => pep440_rs::Version::from_str(&v).unwrap_or_else(|_| {
+                let v = PyVAlias::default().into();
+                err = Some(format!("\nCannot parse INSTANCE_PYTHON_VERSION ({:?}), fallback to latest_stable ({v:?})", *INSTANCE_PYTHON_VERSION));
+                v
+            }),
+            // Use latest stable
+            None => PyVAlias::default().into(),
+        };
+
+        if let Some(msg) = err {
+            if let Some(conn) = conn {
+                append_logs(job_id, w_id, &msg, &conn).await;
+            }
+            tracing::error!(msg);
+        }
+        pyv.into()
+    }
+
+    pub async fn list_available_python_versions() -> Vec<Self> {
+        match Self::list_available_python_versions_inner().await {
+            Ok(pyvs) => pyvs,
+            Err(e) => {
+                tracing::error!(
+                    "Fallback to preconfigured aliases. Cannot list python versions due to this error: {e}"
+                );
+                PyVAlias::all()
+            }
+        }
+    }
+    async fn list_available_python_versions_inner() -> anyhow::Result<Vec<Self>> {
+        lazy_static::lazy_static! {
+            static ref CACHED_VERSIONS: Arc<RwLock<Option<Vec<PyV>>>> = Arc::new(RwLock::new(None));
+            static ref LAST_CHECKED: Arc<RwLock<DateTime<Utc>>> = Arc::new(RwLock::new(Utc::now()));
+        }
+        match (
+            Utc::now().signed_duration_since(*LAST_CHECKED.read().await) > Duration::minutes(30),
+            CACHED_VERSIONS.read().await.clone(),
+        ) {
+            (false, Some(vs)) => return Ok(vs),
+            _ => {}
+        };
+
+        let output = {
+            #[cfg(windows)]
+            let uv_cmd = "uv";
+
+            #[cfg(unix)]
+            let uv_cmd = UV_PATH.as_str();
+
+            Command::new(uv_cmd)
+                .env_clear()
+                .envs(WIN_ENVS.to_vec())
+                .args([
+                    "python",
+                    "list",
+                    "--all-versions",
+                    "--output-format",
+                    "json",
+                ])
+                .stderr(Stdio::piped())
+                .output()
+                .await?
+        };
+
+        // We want to skip all versions smaller then 3.10
+        // Windmill is incompatible with 3.9 and older
+        let filter = pep440_rs::VersionSpecifier::from_version(
+            pep440_rs::Operator::GreaterThanEqual,
+            PyVAlias::Py310.into(),
+        )?;
+
+        if output.status.success() {
+            let res = String::from_utf8(output.stdout)?;
+            tracing::error!("{}", &res);
+            let list = serde_json::from_str::<Vec<serde_json::Map<String, Value>>>(&res)?
+                .into_iter()
+                .filter_map(|e| {
+                    if e.get("implementation").and_then(Value::as_str) == Some("pypy") {
+                        None
+                    } else {
+                        Some(
+                            e.get("version")
+                                .and_then(Value::as_str)
+                                .and_then(|s| pep440_rs::Version::from_str(s).ok())
+                                .map(PyV::from)
+                                .ok_or(Error::internal_err("version is None")),
+                        )
+                    }
+                })
+                .collect::<Result<Vec<PyV>, Error>>()?
+                .into_iter()
+                .unique()
+                .sorted()
+                .filter(|pyv| filter.contains(&*pyv))
+                .rev()
+                .collect_vec();
+
+            *LAST_CHECKED.write().await = Utc::now();
+            CACHED_VERSIONS.write().await.replace(list.clone());
+
+            Ok(list)
+        } else {
+            // If the command failed, print the error
+            let stderr = String::from_utf8(output.stderr)?;
+            bail!(
+                "Cannot list python versions, is uv (0.5.19 and newer) installed? Err:\n{}",
+                stderr
+            );
+        }
+    }
+
+    /// Parse lockfile for assigned python version.
+    /// If not found returns 3.11
+    pub fn parse_from_requirements<S: AsRef<str>>(requirements_lines: &[S]) -> Self {
+        Self::try_parse_from_requirements(requirements_lines).unwrap_or(
+            // If there is no assigned version in lockfile we automatically fallback to 3.11
+            // In this case we have dependencies or other metadata, but no associated python version
+            // This is the case for old deployed scripts
+            PyVAlias::Py311.into(),
+        )
+    }
+
+    /// Parse lockfile for assigned python version.
+    /// If not found returns None
+    pub fn try_parse_from_requirements<S: AsRef<str>>(requirements_lines: &[S]) -> Option<Self> {
+        let parse_version = |s: &str| -> Option<PyV> {
+            // Possible inputs:
+            // V2:
+            // # py: 3.11.0 or #py:3.11.0 or #py: 3.11.0
+            //
+            // V1:
+            // # py311 or #py311
+            let version_unparsed = s
+                .to_owned()
+                // Remove whitespaces. That leaves us with:
+                // V2: #py:3.11.0
+                // V1: #py311
+                //
+                // Remove #
+                // V2: py:3.11.0
+                // V1: py311
+                //
+                // Remove :
+                // V2: py3.11.0
+                // V1: py311
+                .replace([' ', '#', ':'], "")
+                // Remove "py"
+                // V2: 3.11.0
+                // V1: 311
+                .replace("py", "");
+
+            // We will support reading V1 syntax, but it will be overwritten next deploy
+            PyVAlias::try_from_v1(&version_unparsed)
+                .map(PyVAlias::into)
+                .or(pep440_rs::Version::from_str(&version_unparsed)
+                    .ok()
+                    .map(pep440_rs::Version::into))
+        };
+        let index = if requirements_lines.get(0).map_or(false, |line| {
+            line.as_ref()
+                .starts_with(LOCKFILE_GENERATED_FROM_REQUIREMENTS_TXT)
+        }) {
+            1
+        } else {
+            0
+        };
+        requirements_lines
+            .get(index)
+            .map(S::as_ref)
+            .and_then(parse_version)
+    }
+
+    pub async fn get_python(
+        &self,
+        worker_name: &str,
+        job_id: &Uuid,
+        w_id: &str,
+        mem_peak: &mut i32,
+        conn: &Connection,
+        occupancy_metrics: &mut Option<&mut OccupancyMetrics>,
+    ) -> windmill_common::error::Result<String> {
+        let python_path = if let Some(python_path) = PYTHON_PATH.clone() {
+            python_path
+        } else if let Some(python_path) = self
+            .try_get_python(
+                &job_id,
+                mem_peak,
+                conn,
+                worker_name,
+                w_id,
+                occupancy_metrics,
+            )
+            .await?
+        {
+            python_path
+        } else {
+            return Err(Error::ExecutionErr(format!(
+                "uv could not manage python path. Please manage it manually by setting PYTHON_PATH environment variable to your python binary path"
+            )));
+        };
+        Ok(python_path)
+    }
+
+    pub async fn try_get_python(
+        &self,
+        job_id: &Uuid,
+        mem_peak: &mut i32,
+        // canceled_by: &mut Option<CanceledBy>,
+        conn: &Connection,
+        worker_name: &str,
+        w_id: &str,
+        occupancy_metrics: &mut Option<&mut OccupancyMetrics>,
+    ) -> error::Result<Option<String>> {
+        // lazy_static::lazy_static! {
+        //     static ref PYTHON_PATHS: Arc<RwLock<HashMap<PyVersion, String>>> = Arc::new(RwLock::new(HashMap::new()));
+        // }
+
+        let res = self
+            .get_python_inner(job_id, mem_peak, conn, worker_name, w_id, occupancy_metrics)
+            .await;
+
+        if let Err(ref e) = res {
+            tracing::error!(
+                "worker_name: {worker_name}, w_id: {w_id}, job_id: {job_id}\n
+                Error while getting python from uv, falling back to system python: {e:?}"
+            );
+            append_logs(
+                job_id,
+                w_id,
+                format!(
+                    "\nError while getting python from uv, falling back to system python: {e:?}"
+                ),
+                conn,
+            )
+            .await;
+        }
+        res
+    }
+    async fn get_python_inner(
+        &self,
+        job_id: &Uuid,
+        mem_peak: &mut i32,
+        // canceled_by: &mut Option<CanceledBy>,
+        conn: &Connection,
+        worker_name: &str,
+        w_id: &str,
+        occupancy_metrics: &mut Option<&mut OccupancyMetrics>,
+    ) -> error::Result<Option<String>> {
+        let py_path = self.find_python().await;
+
+        // Runtime is not installed
+        if py_path.is_err() {
+            // Install it
+            if let Err(err) = self
+                .install_python(job_id, mem_peak, conn, worker_name, w_id, occupancy_metrics)
+                .await
+            {
+                tracing::error!("Cannot install python: {err}");
+                return Err(err);
+            } else {
+                // Try to find one more time
+                let py_path = self.find_python().await;
+
+                if let Err(err) = py_path {
+                    tracing::error!("Cannot find python version {err}");
+                    return Err(err);
+                }
+
+                // TODO: Cache the result
+                py_path
+            }
+        } else {
+            py_path
+        }
+    }
+    async fn install_python(
+        &self,
+        job_id: &Uuid,
+        mem_peak: &mut i32,
+        // canceled_by: &mut Option<CanceledBy>,
+        conn: &Connection,
+        worker_name: &str,
+        w_id: &str,
+        occupancy_metrics: &mut Option<&mut OccupancyMetrics>,
+    ) -> error::Result<()> {
+        let v = self.to_string();
+        append_logs(job_id, w_id, format!("\nINSTALLING PYTHON ({})", v), conn).await;
+        // Create dirs for newly installed python
+        // If we dont do this, NSJAIL will not be able to mount cache
+        // For the default version directory created during startup (main.rs)
+        DirBuilder::new()
+            .recursive(true)
+            .create(self.to_cache_dir())
+            .await
+            .expect("could not create initial worker dir");
+
+        let logs = String::new();
+
+        #[cfg(windows)]
+        let uv_cmd = "uv";
+
+        #[cfg(unix)]
+        let uv_cmd = UV_PATH.as_str();
+
+        let mut child_cmd = Command::new(uv_cmd);
+        child_cmd
+            .env_clear()
+            .env("HOME", HOME_ENV.to_string())
+            .env("PATH", PATH_ENV.to_string())
+            .envs(PROXY_ENVS.clone())
+            .args(["python", "install", &v, "--python-preference=only-managed"])
+            // TODO: Do we need these?
+            .envs([("UV_PYTHON_INSTALL_DIR", PY_INSTALL_DIR)])
+            .stdout(Stdio::piped())
+            .stderr(Stdio::piped());
+
+        #[cfg(windows)]
+        {
+            child_cmd
+                .env("SystemRoot", crate::SYSTEM_ROOT.as_str())
+                .env("USERPROFILE", crate::USERPROFILE_ENV.as_str())
+                .env(
+                    "TMP",
+                    std::env::var("TMP").unwrap_or_else(|_| String::from("/tmp")),
+                )
+                .env(
+                    "LOCALAPPDATA",
+                    std::env::var("LOCALAPPDATA")
+                        .unwrap_or_else(|_| format!("{}\\AppData\\Local", HOME_ENV.as_str())),
+                );
+        }
+
+        let child_process = start_child_process(child_cmd, "uv").await?;
+
+        append_logs(&job_id, &w_id, logs, conn).await;
+        handle_child(
+            job_id,
+            conn,
+            mem_peak,
+            &mut None,
+            child_process,
+            false,
+            worker_name,
+            &w_id,
+            "uv",
+            None,
+            false,
+            occupancy_metrics,
+            None,
+        )
+        .await
+    }
+    async fn find_python(&self) -> error::Result<Option<String>> {
+        #[cfg(windows)]
+        let uv_cmd = "uv";
+
+        #[cfg(unix)]
+        let uv_cmd = UV_PATH.as_str();
+
+        let mut child_cmd = Command::new(uv_cmd);
+
+        child_cmd.env_clear();
+
+        #[cfg(windows)]
+        {
+            child_cmd
+                .env("SystemRoot", crate::SYSTEM_ROOT.as_str())
+                .env("USERPROFILE", crate::USERPROFILE_ENV.as_str())
+                .env(
+                    "TMP",
+                    std::env::var("TMP").unwrap_or_else(|_| String::from("/tmp")),
+                )
+                .env(
+                    "LOCALAPPDATA",
+                    std::env::var("LOCALAPPDATA")
+                        .unwrap_or_else(|_| format!("{}\\AppData\\Local", HOME_ENV.as_str())),
+                );
+        }
+
+        let output = child_cmd
+            // .current_dir(job_dir)
+            .env("HOME", HOME_ENV.to_string())
+            .env("PATH", PATH_ENV.to_string())
+            .args([
+                "python",
+                "find",
+                &self.to_string(),
+                "--system",
+                "--python-preference=only-managed",
+            ])
+            .envs([
+                ("UV_PYTHON_INSTALL_DIR", PY_INSTALL_DIR),
+                ("UV_PYTHON_PREFERENCE", "only-managed"),
+            ])
+            // .stdout(Stdio::piped())
+            .stderr(Stdio::piped())
+            .output()
+            .await?;
+
+        // Check if the command was successful
+        if output.status.success() {
+            // Convert the output to a String
+            let stdout =
+                String::from_utf8(output.stdout).expect("Failed to convert output to String");
+            return Ok(Some(stdout.replace('\n', "")));
+        } else {
+            // If the command failed, print the error
+            let stderr =
+                String::from_utf8(output.stderr).expect("Failed to convert error output to String");
+            return Err(error::Error::FindPythonError(stderr));
+        }
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    /// Unsafe helper for testing
+    fn pyv(value: &str) -> PyV {
+        pep440_rs::Version::from_str(value).unwrap().into()
+    }
+
+    async fn assert_resolution(
+        instance_version: &str,
+        select_highest: bool,
+        specifiers: Vec<&str>,
+        available: Vec<PyV>,
+        expected: PyV,
+    ) {
+        let resolved = PyV::resolve(
+            specifiers
+                .into_iter()
+                .map(|s| pep440_rs::VersionSpecifier::from_str(s).unwrap())
+                .collect_vec(),
+            &Uuid::nil(),
+            "",
+            select_highest,
+            None,
+            Some(available),
+            Some(pyv(instance_version)),
+        )
+        .await
+        .unwrap();
+        assert_eq!(expected, resolved);
+    }
+
+    #[tokio::test]
+    async fn test_python_resolution_1() {
+        assert_resolution(
+            "1.0",
+            false,
+            vec![],
+            vec![
+                pyv("1.2.0"),
+                pyv("1.1.0"),
+                pyv("1.0.0"),
+                pyv("0.9.0"), //
+            ],
+            pyv("1.0.0"), //
+        )
+        .await;
+    }
+    #[tokio::test]
+    async fn test_python_resolution_2() {
+        assert_resolution(
+            "1.0.0",
+            false,
+            vec!["!=1.*"],
+            vec![
+                pyv("1.2"),
+                pyv("1.1"),
+                pyv("1.0.2"),
+                pyv("1.0.1"),
+                pyv("1.0.0"),
+                pyv("0.9.4"),
+                pyv("0.9.3"),
+                pyv("0.9.2"),
+            ],
+            pyv("0.9.4"), //
+        )
+        .await;
+    }
+    #[tokio::test]
+    async fn test_python_resolution_3() {
+        assert_resolution(
+            "0.9",
+            false,
+            vec!["!=0.9.*"],
+            vec![
+                pyv("1.2"),
+                pyv("1.1"),
+                pyv("1.0.2"),
+                pyv("1.0.1"),
+                pyv("1.0.0"),
+                pyv("0.9.4"),
+                pyv("0.9.3"),
+                pyv("0.9.2"),
+                pyv("0.8.2"),
+                pyv("0.8.1"),
+                pyv("0.8.0"),
+            ],
+            pyv("1.0.2"), //
+        )
+        .await;
+    }
+    #[tokio::test]
+    async fn test_python_resolution_4() {
+        assert_resolution(
+            "0.9",
+            false,
+            vec!["<=0.8.1"],
+            vec![pyv("1.0.0"), pyv("0.9.0"), pyv("0.8.1"), pyv("0.8.0")],
+            pyv("0.8.1"), //
+        )
+        .await;
+    }
+    #[tokio::test]
+    async fn test_python_resolution_5() {
+        assert_resolution(
+            "0.0.1",
+            false,
+            vec!["!=0.1.0"],
+            vec![pyv("2.1.0"), pyv("1.1.0"), pyv("0.1.0")],
+            pyv("1.1.0"),
+        )
+        .await;
+    }
+    #[tokio::test]
+    async fn test_python_resolution_6() {
+        assert_resolution(
+            "1.1.1",
+            false,
+            vec![],
+            vec![
+                pyv("3.0.1"),
+                pyv("3.0.0"),
+                pyv("2.2.2"),
+                pyv("2.2.1"),
+                pyv("2.2.0"),
+            ],
+            pyv("2.2.2"),
+        )
+        .await;
+    }
+    #[tokio::test]
+    async fn test_python_resolution_7() {
+        assert_resolution(
+            "2.2.1",
+            true,
+            vec![],
+            vec![
+                pyv("3.0.1"),
+                pyv("3.0.0"),
+                pyv("2.2.2"),
+                pyv("2.2.1"),
+                pyv("2.2.0"),
+            ],
+            pyv("3.0.1"),
+        )
+        .await;
+    }
+}

backend/windmill-worker/src/result_processor.rs

@@ -29,8 +29,7 @@ use windmill_common::{
 use windmill_common::bench::{BenchmarkInfo, BenchmarkIter};
 
 use windmill_queue::{
-    append_logs, get_queued_job,  CanceledBy, JobCompleted, MiniPulledJob,
-    WrappedError,
+    append_logs, get_queued_job, CanceledBy, JobCompleted, MiniPulledJob, WrappedError,
 };
 
 use serde_json::{json, value::RawValue};
@@ -44,9 +43,10 @@ use crate::{
     common::{error_to_value, read_result, save_in_cache},
     otel_ee::add_root_flow_job_to_otlp,
     worker_flow::update_flow_status_after_job_completion,
-    AuthedClient, JobCompletedReceiver, JobCompletedSender, SameWorkerSender, SendResult,
-    UpdateFlow, INIT_SCRIPT_TAG,
+    JobCompletedReceiver, JobCompletedSender, SameWorkerSender, SendResult, UpdateFlow,
+    INIT_SCRIPT_TAG,
 };
+use windmill_common::client::AuthedClient;
 
 async fn process_jc(
     jc: JobCompleted,
@@ -273,11 +273,7 @@ pub fn start_background_processor(
     })
 }
 
-async fn send_job_completed(
-    job_completed_tx: JobCompletedSender,
-    jc: JobCompleted,
-
-) {
+async fn send_job_completed(job_completed_tx: JobCompletedSender, jc: JobCompleted) {
     job_completed_tx
         .send_job(jc, true)
         .with_context(windmill_common::otel_ee::otel_ctx())
@@ -301,7 +297,6 @@ pub async fn process_result(
 ) -> error::Result<bool> {
     match result {
         Ok(result) => {
-
             send_job_completed(
                 job_completed_tx,
                 JobCompleted {