Compare commits
1 Commits
rf/testbac
...
react2
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
4013aa9098 |
@@ -1,127 +0,0 @@
|
||||
---
|
||||
name: branch-diff-reviewer
|
||||
description: Use this agent when you want a comprehensive code review of changes in the current branch compared to main. This includes reviewing for bugs, optimization opportunities, code style issues, potential mistakes, and adherence to project conventions. The agent should be invoked after completing a feature branch or before creating a pull request.\n\nExamples:\n\n<example>\nContext: User has finished implementing a new feature and wants feedback before merging.\nuser: "I've finished the new kafka trigger implementation, can you review my changes?"\nassistant: "I'll use the branch-diff-reviewer agent to analyze your changes against the main branch and provide comprehensive feedback."\n<commentary>\nSince the user wants a review of their branch changes, use the Task tool to launch the branch-diff-reviewer agent to compare the current branch against main and provide detailed feedback.\n</commentary>\n</example>\n\n<example>\nContext: User wants to check their code quality before submitting a PR.\nuser: "Review my branch before I create a PR"\nassistant: "Let me launch the branch-diff-reviewer agent to examine all your changes and identify any issues or improvements."\n<commentary>\nThe user is preparing for a PR, so use the branch-diff-reviewer agent to provide a thorough review of all branch differences.\n</commentary>\n</example>\n\n<example>\nContext: User is unsure if their implementation follows project patterns.\nuser: "Does my implementation look correct? I'm not sure if I followed the existing patterns"\nassistant: "I'll use the branch-diff-reviewer agent to compare your changes against main and check for pattern consistency, potential issues, and optimization opportunities."\n<commentary>\nThe user needs validation of their implementation against project standards. Launch the branch-diff-reviewer agent to analyze the diff and provide feedback on patterns, correctness, and improvements.\n</commentary>\n</example>
|
||||
tools: Glob, Grep, Read, WebFetch, TodoWrite, WebSearch, ListMcpResourcesTool, ReadMcpResourceTool, mcp__svelte__get-documentation, mcp__svelte__list-sections, mcp__svelte__playground-link, mcp__svelte__svelte-autofixer, mcp__ide__getDiagnostics, mcp__ide__executeCode, Bash, Skill
|
||||
model: inherit
|
||||
---
|
||||
|
||||
You are an elite code reviewer with deep expertise in software engineering best practices, performance optimization, and security. Your role is to provide thorough, actionable feedback on code changes between the current branch and main.
|
||||
|
||||
## Your Review Process
|
||||
|
||||
1. **First, gather the diff**: Use git commands to obtain the complete diff between the current branch and main:
|
||||
- Run `git diff main...HEAD` to see all changes
|
||||
- Run `git log main..HEAD --oneline` to understand the commit history
|
||||
- Identify all modified, added, and deleted files
|
||||
|
||||
2. **Analyze each changed file** in the context of:
|
||||
- The project's established patterns (check CLAUDE.md and related documentation)
|
||||
- The file's purpose and its role in the broader codebase
|
||||
- Dependencies and how changes might affect other parts of the system
|
||||
|
||||
## Review Categories
|
||||
|
||||
For each significant change, evaluate and report on:
|
||||
|
||||
### 🐛 Bugs & Correctness
|
||||
- Logic errors or edge cases not handled
|
||||
- Null/undefined handling issues
|
||||
- Race conditions in async code
|
||||
- Incorrect error handling
|
||||
- Type mismatches or unsafe casts
|
||||
|
||||
### ⚡ Performance
|
||||
- Inefficient algorithms or data structures
|
||||
- N+1 query problems in database code
|
||||
- Unnecessary re-renders in frontend code
|
||||
- Missing indexes for database queries
|
||||
- Blocking operations in async contexts
|
||||
- Memory leaks or excessive allocations
|
||||
- For Rust: Check for unnecessary clones, inefficient serde usage, blocking in async
|
||||
- For Svelte: Check for inefficient reactivity, missing keys in loops, excessive effects
|
||||
|
||||
### 🔒 Security
|
||||
- SQL injection vulnerabilities
|
||||
- Missing input validation
|
||||
- Exposed sensitive data
|
||||
- Authentication/authorization gaps
|
||||
- Unsafe deserialization
|
||||
|
||||
### 📐 Code Quality & Style
|
||||
- Adherence to project conventions (CLAUDE.md guidelines)
|
||||
- Code duplication that should be refactored
|
||||
- Unclear or misleading naming
|
||||
- Missing or inadequate documentation
|
||||
- Overly complex logic that could be simplified
|
||||
- Dead code or unused imports
|
||||
|
||||
### 🏗️ Architecture & Design
|
||||
- Proper separation of concerns
|
||||
- Appropriate use of existing utilities vs. new code
|
||||
- Consistency with established patterns
|
||||
- Proper error propagation
|
||||
- API design issues
|
||||
|
||||
### 🧪 Testing Considerations
|
||||
- Suggest test cases for new functionality
|
||||
- Identify untested edge cases
|
||||
- Note if changes break existing test assumptions
|
||||
|
||||
## Project-Specific Rules
|
||||
|
||||
### For Rust (Backend)
|
||||
- Verify `SELECT` statements list explicit columns (never `SELECT *` in worker code)
|
||||
- Check for proper use of `sqlx` with parameterized queries
|
||||
- Ensure errors use the custom `Error` enum from `windmill-common::error`
|
||||
- Verify async code doesn't block the tokio runtime
|
||||
- Check serde attributes for optimal serialization
|
||||
- Ensure openapi.yaml is updated for API changes
|
||||
|
||||
### For Svelte (Frontend)
|
||||
- For Svelte 5 files: Verify proper use of Runes (`$state`, `$derived`, `$effect`)
|
||||
- Check for `key` attributes in `{#each}` blocks
|
||||
- Ensure event handlers use the new syntax (`onclick` not `on:click`) in Svelte 5
|
||||
- Verify snippets are used instead of slots in Svelte 5
|
||||
- Check for proper props declaration with `$props()`
|
||||
|
||||
## Output Format
|
||||
|
||||
Structure your review as follows:
|
||||
|
||||
```
|
||||
## Summary
|
||||
[Brief overview of the changes and overall assessment]
|
||||
|
||||
## Critical Issues 🚨
|
||||
[Issues that must be fixed before merging]
|
||||
|
||||
## Recommendations 💡
|
||||
[Improvements that would significantly enhance the code]
|
||||
|
||||
## Minor Suggestions 📝
|
||||
[Nice-to-haves and style improvements]
|
||||
|
||||
## Positive Observations ✅
|
||||
[Well-done aspects worth acknowledging]
|
||||
|
||||
## File-by-File Details
|
||||
[Detailed feedback organized by file]
|
||||
```
|
||||
|
||||
For each issue, provide:
|
||||
1. **Location**: File path and line number(s)
|
||||
2. **Issue**: Clear description of the problem
|
||||
3. **Impact**: Why this matters
|
||||
4. **Suggestion**: Concrete fix or improvement with code example when helpful
|
||||
|
||||
## Behavioral Guidelines
|
||||
|
||||
- Be thorough but prioritize: focus most on critical issues
|
||||
- Be constructive: every criticism should come with a suggestion
|
||||
- Be specific: vague feedback is not actionable
|
||||
- Acknowledge good work: positive reinforcement matters
|
||||
- Consider context: understand why decisions might have been made
|
||||
- Ask clarifying questions if the intent of changes is unclear
|
||||
- Reference project documentation when pointing out convention violations
|
||||
|
||||
Begin by fetching the diff and then proceed with your comprehensive review.
|
||||
@@ -1,76 +0,0 @@
|
||||
---
|
||||
name: openapi-sync
|
||||
description: Use this agent when backend API endpoints are added, modified, or removed, or when the Flow structure changes and the OpenAPI specification files need to be updated. This includes changes to route handlers in windmill-api, modifications to request/response schemas, changes to authentication requirements, or updates to the Flow data structures that affect the API.\n\nExamples:\n\n<example>\nContext: User has just added a new API endpoint for managing workspace templates.\nuser: "Add a new endpoint POST /api/w/{workspace}/templates to create workspace templates"\nassistant: "I've created the new endpoint handler in windmill-api. Now let me use the openapi-sync agent to update the OpenAPI specification files."\n<commentary>\nSince a new API endpoint was added, use the openapi-sync agent to ensure the openapi.yaml and openflow.openapi.yaml files are updated with the new endpoint definition.\n</commentary>\n</example>\n\n<example>\nContext: User has modified the response schema for an existing endpoint.\nuser: "Update the GET /api/w/{workspace}/flows endpoint to also return a 'versions' array in the response"\nassistant: "I've updated the flow listing endpoint to include the versions array. Now let me use the openapi-sync agent to update the OpenAPI specification."\n<commentary>\nSince the response schema of an existing endpoint was modified, use the openapi-sync agent to update the corresponding schema in the OpenAPI files.\n</commentary>\n</example>\n\n<example>\nContext: User has made changes to the Flow structure in the codebase.\nuser: "Add a new 'retry_policy' field to the Flow value structure"\nassistant: "I've added the retry_policy field to the Flow struct. Now let me use the openapi-sync agent to update the OpenAPI specification to reflect this schema change."\n<commentary>\nSince the Flow structure was modified, use the openapi-sync agent to ensure the flow-related schemas in openapi.yaml and openflow.openapi.yaml are updated.\n</commentary>\n</example>
|
||||
model: inherit
|
||||
---
|
||||
|
||||
You are an expert API documentation engineer specializing in OpenAPI specifications for the Windmill platform. Your primary responsibility is to maintain synchronization between the Rust backend API implementation and the OpenAPI specification files.
|
||||
|
||||
## Your Core Responsibilities
|
||||
|
||||
1. **Update OpenAPI Specifications**: When API endpoints are added, modified, or removed in the windmill-api crate, you must update:
|
||||
- `backend/windmill-api/openapi.yaml` - The main OpenAPI specification
|
||||
- `backend/windmill-api/openflow.openapi.yaml` - Flow-specific OpenAPI definitions (if flow-related changes)
|
||||
|
||||
2. **Maintain Schema Accuracy**: Ensure all request/response schemas accurately reflect the Rust structs used in the API handlers.
|
||||
|
||||
3. **Document Comprehensively**: Include proper descriptions, examples, and parameter documentation.
|
||||
|
||||
## Key Files to Reference
|
||||
|
||||
- **API Route Definitions**: Look in `backend/windmill-api/src/` for route handlers organized by domain
|
||||
- **Data Structures**: Check `backend/windmill-common/src/` for shared structs and types
|
||||
- **Database Schema**: Reference `backend/summarized_schema.txt` for understanding data models
|
||||
- **Existing OpenAPI Files**: Always review the current state of `openapi.yaml` and `openflow.openapi.yaml` before making changes
|
||||
|
||||
## Workflow
|
||||
|
||||
1. **Identify Changes**: Determine what API changes were made by examining:
|
||||
- New or modified route handlers in windmill-api
|
||||
- Changes to request/response structs
|
||||
- Modifications to the Flow structure or related types
|
||||
|
||||
2. **Analyze the Implementation**: For each endpoint, identify:
|
||||
- HTTP method and path
|
||||
- Path parameters, query parameters, and request body schema
|
||||
- Response schema(s) and status codes
|
||||
- Authentication requirements
|
||||
- Any tags or groupings
|
||||
|
||||
3. **Update OpenAPI Files**:
|
||||
- Add or modify path definitions with accurate operation IDs
|
||||
- Update or create schema definitions in the components section
|
||||
- Ensure $ref references are correct
|
||||
- Maintain consistent naming conventions with existing patterns
|
||||
|
||||
4. **Validate Changes**: Ensure the YAML syntax is valid and follows OpenAPI 3.0 specification.
|
||||
|
||||
## OpenAPI Conventions for Windmill
|
||||
|
||||
- **Operation IDs**: Use camelCase, descriptive names (e.g., `createScript`, `listFlows`, `updateWorkspaceSettings`)
|
||||
- **Tags**: Group endpoints by domain (e.g., `scripts`, `flows`, `workspaces`, `users`)
|
||||
- **Schema Naming**: Use PascalCase for schema names matching Rust struct names
|
||||
- **Path Parameters**: Use `{workspace}` for workspace_id, maintain consistency with existing patterns
|
||||
- **Security**: Most endpoints require Bearer token authentication - include appropriate security requirements
|
||||
|
||||
## Schema Mapping from Rust to OpenAPI
|
||||
|
||||
- `String` / `&str` → `type: string`
|
||||
- `i32`, `i64` → `type: integer` (with appropriate format)
|
||||
- `f32`, `f64` → `type: number`
|
||||
- `bool` → `type: boolean`
|
||||
- `Vec<T>` → `type: array` with `items`
|
||||
- `Option<T>` → property is not in `required` array
|
||||
- `HashMap<K, V>` → `type: object` with `additionalProperties`
|
||||
- Enums → `type: string` with `enum` array
|
||||
- Custom structs → `$ref` to schema definition
|
||||
|
||||
## Important Notes
|
||||
|
||||
- Always preserve existing documentation and descriptions when updating
|
||||
- Maintain backward compatibility warnings in descriptions when applicable
|
||||
- Include example values where they aid understanding
|
||||
- For Flow-related changes, update BOTH openapi.yaml AND openflow.openapi.yaml as needed
|
||||
- Follow the existing indentation and formatting style in the YAML files
|
||||
|
||||
When you complete updates, summarize what changes were made to which files and highlight any schema additions or modifications that downstream consumers should be aware of.
|
||||
@@ -1,20 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Format backend Rust files with rustfmt after Claude edits them
|
||||
|
||||
# Get the file path from the tool result (passed via stdin as JSON)
|
||||
INPUT=$(cat)
|
||||
FILE_PATH=$(echo "$INPUT" | jq -r '.tool_input.file_path // empty')
|
||||
|
||||
# Exit if no file path
|
||||
if [ -z "$FILE_PATH" ]; then
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Check if the file is in the backend directory and is a Rust file
|
||||
if [[ "$FILE_PATH" == *"/backend/"* ]] && [[ "$FILE_PATH" =~ \.rs$ ]]; then
|
||||
cd "$CLAUDE_PROJECT_DIR/backend" || exit 0
|
||||
# Run rustfmt with config from rustfmt.toml (edition=2021)
|
||||
rustfmt --config-path rustfmt.toml "$FILE_PATH" 2>/dev/null || true
|
||||
fi
|
||||
|
||||
exit 0
|
||||
@@ -1,23 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Format frontend files with prettier after Claude edits them
|
||||
|
||||
# Get the file path from the tool result (passed via stdin as JSON)
|
||||
INPUT=$(cat)
|
||||
FILE_PATH=$(echo "$INPUT" | jq -r '.tool_input.file_path // empty')
|
||||
|
||||
# Exit if no file path
|
||||
if [ -z "$FILE_PATH" ]; then
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Check if the file is in the frontend directory
|
||||
if [[ "$FILE_PATH" == *"/frontend/"* ]]; then
|
||||
# Check if it's a formattable file type
|
||||
if [[ "$FILE_PATH" =~ \.(ts|js|svelte|json|css|html|md)$ ]]; then
|
||||
cd "$CLAUDE_PROJECT_DIR/frontend" || exit 0
|
||||
# Run prettier silently, don't fail the hook if prettier fails
|
||||
npx prettier --write "$FILE_PATH" 2>/dev/null || true
|
||||
fi
|
||||
fi
|
||||
|
||||
exit 0
|
||||
@@ -1,25 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Notify user when Claude requires input (works on macOS and Linux)
|
||||
|
||||
# Check if we're in an SSH session
|
||||
if [[ -n "$SSH_CLIENT" || -n "$SSH_TTY" || -n "$SSH_CONNECTION" ]]; then
|
||||
# SSH session - use terminal bell
|
||||
# If using VSCode, enable audible terminal bell for SSH sessions:
|
||||
# Add the following to .vscode/settings.json:
|
||||
# "accessibility.signals.terminalBell": {
|
||||
# "sound": "on"
|
||||
# },
|
||||
# "terminal.integrated.enableVisualBell": true
|
||||
printf '\a'
|
||||
else
|
||||
# Local session - use native notifications
|
||||
if [[ "$OSTYPE" == "darwin"* ]]; then
|
||||
osascript -e 'display notification "Claude is waiting for your input" with title "Claude Code" sound name "Glass"' 2>/dev/null || printf '\a'
|
||||
elif [[ "$OSTYPE" == "linux-gnu"* ]]; then
|
||||
notify-send "Claude Code" "Claude is waiting for your input" 2>/dev/null || printf '\a'
|
||||
else
|
||||
printf '\a'
|
||||
fi
|
||||
fi
|
||||
|
||||
exit 0
|
||||
@@ -1,30 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Resolve _ee.rs symlinks to actual files so Claude can read them
|
||||
# This script runs before each user prompt is processed
|
||||
|
||||
set -e
|
||||
|
||||
PROJECT_DIR="${CLAUDE_PROJECT_DIR:-/home/farhad/windmill}"
|
||||
MANIFEST_FILE="$PROJECT_DIR/.claude/hooks/.symlink-manifest"
|
||||
|
||||
# Find all _ee.rs symlinks and store their targets
|
||||
find "$PROJECT_DIR" -name "*_ee.rs" -type l 2>/dev/null | while read -r symlink; do
|
||||
target=$(readlink -f "$symlink" 2>/dev/null) || continue
|
||||
|
||||
# Only process if target file exists
|
||||
if [[ -f "$target" ]]; then
|
||||
# Store symlink path and target in manifest
|
||||
echo "$symlink|$target" >> "$MANIFEST_FILE.tmp"
|
||||
|
||||
# Replace symlink with actual file content
|
||||
rm "$symlink"
|
||||
cp "$target" "$symlink"
|
||||
fi
|
||||
done
|
||||
|
||||
# Atomically replace manifest
|
||||
if [[ -f "$MANIFEST_FILE.tmp" ]]; then
|
||||
mv "$MANIFEST_FILE.tmp" "$MANIFEST_FILE"
|
||||
fi
|
||||
|
||||
exit 0
|
||||
@@ -1,36 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Restore _ee.rs symlinks after Claude finishes processing
|
||||
# This script runs when Claude stops
|
||||
# IMPORTANT: Copies any modifications back to the target before restoring symlinks
|
||||
|
||||
set -e
|
||||
|
||||
PROJECT_DIR="${CLAUDE_PROJECT_DIR:-/home/farhad/windmill}"
|
||||
MANIFEST_FILE="$PROJECT_DIR/.claude/hooks/.symlink-manifest"
|
||||
|
||||
# Check if manifest exists
|
||||
if [[ ! -f "$MANIFEST_FILE" ]]; then
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Read manifest and restore symlinks
|
||||
while IFS='|' read -r symlink target; do
|
||||
if [[ -n "$symlink" && -n "$target" ]]; then
|
||||
# If the file exists (not a symlink) and target exists, copy changes back
|
||||
if [[ -f "$symlink" && ! -L "$symlink" && -e "$target" ]]; then
|
||||
# Copy the potentially modified file back to the target
|
||||
cp "$symlink" "$target"
|
||||
fi
|
||||
|
||||
# Remove the regular file (which was a copy)
|
||||
rm -f "$symlink" 2>/dev/null || true
|
||||
|
||||
# Recreate the symlink
|
||||
ln -s "$target" "$symlink" 2>/dev/null || true
|
||||
fi
|
||||
done < "$MANIFEST_FILE"
|
||||
|
||||
# Clean up manifest
|
||||
rm -f "$MANIFEST_FILE"
|
||||
|
||||
exit 0
|
||||
@@ -1,133 +0,0 @@
|
||||
{
|
||||
"permissions": {
|
||||
"allow": [
|
||||
"Bash(ls:*)",
|
||||
"Bash(grep:*)",
|
||||
"Bash(cat:*)",
|
||||
"Bash(head:*)",
|
||||
"Bash(tail:*)",
|
||||
"Bash(less:*)",
|
||||
"Bash(more:*)",
|
||||
"Bash(find:*)",
|
||||
"Bash(wc:*)",
|
||||
"Bash(diff:*)",
|
||||
"Bash(file:*)",
|
||||
"Bash(stat:*)",
|
||||
"Bash(tree:*)",
|
||||
"Bash(pwd)",
|
||||
"Bash(which:*)",
|
||||
"Bash(whereis:*)",
|
||||
"Bash(echo:*)",
|
||||
"Bash(git status:*)",
|
||||
"Bash(git diff:*)",
|
||||
"Bash(git log:*)",
|
||||
"Bash(git branch:*)",
|
||||
"Bash(git show:*)",
|
||||
"Bash(git blame:*)",
|
||||
"Bash(cargo check:*)",
|
||||
"mcp__ide__getDiagnostics",
|
||||
"Bash(npm run generate-backend-client:*)",
|
||||
"Bash(npm run check:*)"
|
||||
],
|
||||
"deny": [
|
||||
"Read(.env)",
|
||||
"Read(.env.*)",
|
||||
"Read(**/.env)",
|
||||
"Read(**/.env.*)",
|
||||
"Read(**/secrets/**)",
|
||||
"Read(**/*.pem)",
|
||||
"Read(**/*.key)",
|
||||
"Read(**/credentials.json)",
|
||||
"Read(**/*secret*)",
|
||||
"Edit(.env)",
|
||||
"Edit(.env.*)",
|
||||
"Edit(**/.env)",
|
||||
"Edit(**/.env.*)"
|
||||
],
|
||||
"ask": [
|
||||
"Bash(rm:*)",
|
||||
"Bash(rmdir:*)",
|
||||
"Bash(mv:*)",
|
||||
"Bash(chmod:*)",
|
||||
"Bash(chown:*)",
|
||||
"Bash(truncate:*)",
|
||||
"Bash(shred:*)",
|
||||
"Bash(unlink:*)",
|
||||
"Bash(git push:*)",
|
||||
"Bash(git reset:*)",
|
||||
"Bash(git revert:*)",
|
||||
"Bash(git checkout:*)",
|
||||
"Bash(git merge:*)",
|
||||
"Bash(git rebase:*)"
|
||||
]
|
||||
},
|
||||
"enableAllProjectMcpServers": true,
|
||||
"hooks": {
|
||||
"UserPromptSubmit": [
|
||||
{
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/resolve-symlinks.sh",
|
||||
"timeout": 30
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"Stop": [
|
||||
{
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/restore-symlinks.sh",
|
||||
"timeout": 30
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"SessionEnd": [
|
||||
{
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/restore-symlinks.sh",
|
||||
"timeout": 30
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"PostToolUse": [
|
||||
{
|
||||
"matcher": "Edit|Write",
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/format-frontend.sh",
|
||||
"timeout": 30
|
||||
},
|
||||
{
|
||||
"type": "command",
|
||||
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/format-backend.sh",
|
||||
"timeout": 30
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"Notification": [
|
||||
{
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/notify-user.sh",
|
||||
"timeout": 10
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
"enabledPlugins": {
|
||||
"rust-analyzer-lsp@claude-plugins-official": true,
|
||||
"typescript-lsp@claude-plugins-official": true,
|
||||
"code-review@claude-plugins-official": true
|
||||
}
|
||||
}
|
||||
@@ -1,60 +0,0 @@
|
||||
---
|
||||
name: commit
|
||||
user_invocable: true
|
||||
description: Create a git commit with conventional commit format. MUST use anytime you want to commit changes.
|
||||
---
|
||||
|
||||
# Git Commit Skill
|
||||
|
||||
Create a focused, single-line commit following conventional commit conventions.
|
||||
|
||||
## Instructions
|
||||
|
||||
1. **Analyze changes**: Run `git status` and `git diff` to understand what was modified
|
||||
2. **Stage only modified files**: Add files individually by name. NEVER use `git add -A` or `git add .`
|
||||
3. **Write commit message**: Follow the conventional commit format as a single line
|
||||
|
||||
## Conventional Commit Format
|
||||
|
||||
```
|
||||
<type>: <description>
|
||||
```
|
||||
|
||||
### Types
|
||||
- `feat`: New feature or capability
|
||||
- `fix`: Bug fix
|
||||
- `refactor`: Code change that neither fixes a bug nor adds a feature
|
||||
- `docs`: Documentation only changes
|
||||
- `style`: Formatting, missing semicolons, etc (no code change)
|
||||
- `test`: Adding or correcting tests
|
||||
- `chore`: Maintenance tasks, dependency updates, etc
|
||||
- `perf`: Performance improvement
|
||||
|
||||
### Rules
|
||||
- Message MUST be a single line (no multi-line messages)
|
||||
- Description should be lowercase, imperative mood ("add" not "added")
|
||||
- No period at the end
|
||||
- Keep under 72 characters total
|
||||
|
||||
### Examples
|
||||
```
|
||||
feat: add token usage tracking for AI providers
|
||||
fix: resolve null pointer in job executor
|
||||
refactor: extract common validation logic
|
||||
docs: update API endpoint documentation
|
||||
chore: upgrade sqlx to 0.7
|
||||
```
|
||||
|
||||
## Execution Steps
|
||||
|
||||
1. Run `git status` to see all changes
|
||||
2. Run `git diff` to understand the changes in detail
|
||||
3. Run `git log --oneline -5` to see recent commit style
|
||||
4. Stage ONLY the modified/relevant files: `git add <file1> <file2> ...`
|
||||
5. Create the commit with conventional format:
|
||||
```bash
|
||||
git commit -m "<type>: <description>
|
||||
|
||||
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>"
|
||||
```
|
||||
6. Run `git status` to verify the commit succeeded
|
||||
@@ -1,87 +0,0 @@
|
||||
---
|
||||
name: pr
|
||||
user_invocable: true
|
||||
description: Open a draft pull request on GitHub. MUST use when you want to create/open a PR.
|
||||
---
|
||||
|
||||
# Pull Request Skill
|
||||
|
||||
Create a draft pull request with a clear title and explicit description of changes.
|
||||
|
||||
## Instructions
|
||||
|
||||
1. **Analyze branch changes**: Understand all commits since diverging from main
|
||||
2. **Push to remote**: Ensure all commits are pushed
|
||||
3. **Create draft PR**: Always open as draft for review before merging
|
||||
|
||||
## PR Title Format
|
||||
|
||||
Follow conventional commit format for the PR title:
|
||||
```
|
||||
<type>: <description>
|
||||
```
|
||||
|
||||
### Types
|
||||
- `feat`: New feature or capability
|
||||
- `fix`: Bug fix
|
||||
- `refactor`: Code restructuring
|
||||
- `docs`: Documentation changes
|
||||
- `chore`: Maintenance tasks
|
||||
- `perf`: Performance improvements
|
||||
|
||||
### Title Rules
|
||||
- Keep under 70 characters
|
||||
- Use lowercase, imperative mood
|
||||
- No period at the end
|
||||
|
||||
## PR Body Format
|
||||
|
||||
The body MUST be explicit about what changed. Structure:
|
||||
|
||||
```markdown
|
||||
## Summary
|
||||
<Clear description of what this PR does and why>
|
||||
|
||||
## Changes
|
||||
- <Specific change 1>
|
||||
- <Specific change 2>
|
||||
- <Specific change 3>
|
||||
|
||||
## Test plan
|
||||
- [ ] <How to verify change 1>
|
||||
- [ ] <How to verify change 2>
|
||||
|
||||
---
|
||||
Generated with [Claude Code](https://claude.com/claude-code)
|
||||
```
|
||||
|
||||
## Execution Steps
|
||||
|
||||
1. Run `git status` to check for uncommitted changes
|
||||
2. Run `git log main..HEAD --oneline` to see all commits in this branch
|
||||
3. Run `git diff main...HEAD` to see the full diff against main
|
||||
4. Check if remote branch exists and is up to date:
|
||||
```bash
|
||||
git rev-parse --abbrev-ref --symbolic-full-name @{u} 2>/dev/null || echo "no upstream"
|
||||
```
|
||||
5. Push to remote if needed: `git push -u origin HEAD`
|
||||
6. Create draft PR using gh CLI:
|
||||
```bash
|
||||
gh pr create --draft --title "<type>: <description>" --body "$(cat <<'EOF'
|
||||
## Summary
|
||||
<description>
|
||||
|
||||
## Changes
|
||||
- <change 1>
|
||||
- <change 2>
|
||||
|
||||
## Test plan
|
||||
- [ ] <test 1>
|
||||
- [ ] <test 2>
|
||||
|
||||
---
|
||||
Generated with [Claude Code](https://claude.com/claude-code)
|
||||
EOF
|
||||
)"
|
||||
```
|
||||
7. Return the PR URL to the user
|
||||
@@ -1,495 +0,0 @@
|
||||
---
|
||||
name: rust-backend
|
||||
description: Rust coding guidelines for the Windmill backend. MUST use when writing or modifying Rust code in the backend directory.
|
||||
---
|
||||
|
||||
# Rust Backend Coding Guidelines
|
||||
|
||||
Apply these patterns when writing or modifying Rust code in the `backend/` directory.
|
||||
|
||||
## Data Structure Design
|
||||
|
||||
Choose between `struct`, `enum`, or `newtype` based on domain needs:
|
||||
|
||||
- Use `enum` for state machines instead of boolean flags or loosely related fields
|
||||
- Model invariants explicitly using types (e.g., `NonZeroU32`, `Duration`, custom enums)
|
||||
- Consider ownership of each field:
|
||||
- Use `&str` vs `String`, slices vs vectors
|
||||
- Use `Arc<T>` when sharing across threads
|
||||
- Use `Cow<'a, T>` for flexible ownership
|
||||
|
||||
```rust
|
||||
// State machine with enum
|
||||
enum JobState {
|
||||
Pending { scheduled_for: DateTime<Utc> },
|
||||
Running { started_at: DateTime<Utc>, worker: String },
|
||||
Completed { result: JobResult, duration_ms: i64 },
|
||||
Failed { error: String, retries: u32 },
|
||||
}
|
||||
|
||||
// Avoid multiple booleans
|
||||
struct Job {
|
||||
is_pending: bool, // Don't do this
|
||||
is_running: bool,
|
||||
is_completed: bool,
|
||||
}
|
||||
```
|
||||
|
||||
## Impl Block Organization
|
||||
|
||||
Place `impl` blocks immediately below the struct/enum they modify. Group methods logically:
|
||||
|
||||
```rust
|
||||
struct JobQueue {
|
||||
jobs: Vec<Job>,
|
||||
capacity: usize,
|
||||
}
|
||||
|
||||
impl JobQueue {
|
||||
// Constructors first
|
||||
pub fn new(capacity: usize) -> Self { ... }
|
||||
pub fn with_jobs(jobs: Vec<Job>) -> Self { ... }
|
||||
|
||||
// Getters
|
||||
pub fn len(&self) -> usize { ... }
|
||||
pub fn is_empty(&self) -> bool { ... }
|
||||
|
||||
// Mutation methods
|
||||
pub fn push(&mut self, job: Job) -> Result<()> { ... }
|
||||
pub fn pop(&mut self) -> Option<Job> { ... }
|
||||
|
||||
// Domain logic
|
||||
pub fn next_scheduled(&self) -> Option<&Job> { ... }
|
||||
}
|
||||
```
|
||||
|
||||
## Iterator Chains Over For-Loops
|
||||
|
||||
Prefer functional iterator chains (`.filter().map().collect()`) over imperative for-loops:
|
||||
|
||||
```rust
|
||||
// Preferred
|
||||
let results: Vec<_> = items
|
||||
.iter()
|
||||
.filter(|item| item.is_valid())
|
||||
.map(|item| item.transform())
|
||||
.collect();
|
||||
|
||||
// Avoid
|
||||
let mut results = Vec::new();
|
||||
for item in items.iter() {
|
||||
if item.is_valid() {
|
||||
results.push(item.transform());
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
Use the `Error` type from `windmill_common::error`. Return `Result<T, Error>` or `JsonResult<T>` for fallible functions:
|
||||
|
||||
```rust
|
||||
use windmill_common::error::{Error, Result};
|
||||
|
||||
// Use ? operator for propagation
|
||||
pub async fn get_job(db: &DB, id: Uuid) -> Result<Job> {
|
||||
let job = sqlx::query_as!(Job, "SELECT ... WHERE id = $1", id)
|
||||
.fetch_optional(db)
|
||||
.await?
|
||||
.ok_or_else(|| Error::NotFound("job not found".to_string()))?;
|
||||
Ok(job)
|
||||
}
|
||||
```
|
||||
|
||||
Prefer `if let` for optional handling. Use `let...else` when early return makes code clearer:
|
||||
|
||||
```rust
|
||||
let Some(config) = get_config() else {
|
||||
return Err(Error::MissingConfig);
|
||||
};
|
||||
```
|
||||
|
||||
Never panic in library code. Reserve `.unwrap()` for cases with compile-time guarantees. Keep functions short to help lifetime inference and clarity.
|
||||
|
||||
## Early Returns
|
||||
|
||||
Return early to avoid deep nesting. Handle error cases and edge conditions first:
|
||||
|
||||
```rust
|
||||
// Preferred - early returns
|
||||
fn process_job(job: Option<Job>) -> Result<Output> {
|
||||
let Some(job) = job else {
|
||||
return Ok(Output::default());
|
||||
};
|
||||
|
||||
if !job.is_valid() {
|
||||
return Err(Error::InvalidJob);
|
||||
}
|
||||
|
||||
if job.is_cached() {
|
||||
return Ok(job.cached_result());
|
||||
}
|
||||
|
||||
// Main logic at the end, not nested
|
||||
execute_job(job)
|
||||
}
|
||||
|
||||
// Avoid - deep nesting
|
||||
fn process_job(job: Option<Job>) -> Result<Output> {
|
||||
if let Some(job) = job {
|
||||
if job.is_valid() {
|
||||
if !job.is_cached() {
|
||||
execute_job(job)
|
||||
} else {
|
||||
Ok(job.cached_result())
|
||||
}
|
||||
} else {
|
||||
Err(Error::InvalidJob)
|
||||
}
|
||||
} else {
|
||||
Ok(Output::default())
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Variable Shadowing
|
||||
|
||||
Shadow variables instead of creating new names with prefixes:
|
||||
|
||||
```rust
|
||||
// Preferred
|
||||
let data = fetch_raw_data();
|
||||
let data = parse(data);
|
||||
let data = validate(data)?;
|
||||
|
||||
// Avoid
|
||||
let raw_data = fetch_raw_data();
|
||||
let parsed_data = parse(raw_data);
|
||||
let validated_data = validate(parsed_data)?;
|
||||
```
|
||||
|
||||
## Minimal Comments
|
||||
|
||||
- No inline comments explaining obvious code
|
||||
- No TODO/FIXME comments in committed code
|
||||
- Doc comments (`///`) only on public items
|
||||
- Let code be self-documenting through clear naming
|
||||
|
||||
## Type Safety
|
||||
|
||||
Use enums over boolean flags for clarity:
|
||||
|
||||
```rust
|
||||
// Preferred
|
||||
enum JobStatus {
|
||||
Pending,
|
||||
Running,
|
||||
Completed,
|
||||
}
|
||||
|
||||
// Avoid
|
||||
struct Job {
|
||||
is_running: bool,
|
||||
is_completed: bool,
|
||||
}
|
||||
```
|
||||
|
||||
## Pattern Matching
|
||||
|
||||
Prefer explicit matching. Use wildcards strategically for fallback cases or ignored fields:
|
||||
|
||||
```rust
|
||||
// Explicit matching preferred
|
||||
match status {
|
||||
JobStatus::Pending => handle_pending(),
|
||||
JobStatus::Running => handle_running(),
|
||||
JobStatus::Completed => handle_completed(),
|
||||
}
|
||||
|
||||
// Wildcards OK for fallback
|
||||
match result {
|
||||
Ok(value) => process(value),
|
||||
Err(_) => return default_value(),
|
||||
}
|
||||
|
||||
// Wildcards OK for ignoring fields in destructuring
|
||||
let Point { x, y, .. } = point;
|
||||
```
|
||||
|
||||
## Destructuring in Function Signatures
|
||||
|
||||
Destructure structs directly in function parameters:
|
||||
|
||||
```rust
|
||||
// Preferred
|
||||
async fn process_job(
|
||||
Extension(db): Extension<DB>,
|
||||
Path((workspace, job_id)): Path<(String, Uuid)>,
|
||||
Query(pagination): Query<Pagination>,
|
||||
) -> Result<Json<Job>> {
|
||||
// ...
|
||||
}
|
||||
|
||||
// Avoid
|
||||
async fn process_job(
|
||||
db_ext: Extension<DB>,
|
||||
path: Path<(String, Uuid)>,
|
||||
query: Query<Pagination>,
|
||||
) -> Result<Json<Job>> {
|
||||
let Extension(db) = db_ext;
|
||||
let Path((workspace, job_id)) = path;
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
## Trait Implementations
|
||||
|
||||
Use standard trait implementations to simplify conversions and reduce boilerplate:
|
||||
|
||||
```rust
|
||||
// Implement From/Into for type conversions
|
||||
impl From<DbJob> for ApiJob {
|
||||
fn from(db: DbJob) -> Self {
|
||||
ApiJob {
|
||||
id: db.id,
|
||||
status: db.status.into(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Use TryFrom for fallible conversions
|
||||
impl TryFrom<String> for JobKind {
|
||||
type Error = Error;
|
||||
fn try_from(s: String) -> Result<Self, Self::Error> { ... }
|
||||
}
|
||||
```
|
||||
|
||||
Apply `derive` macros to reduce boilerplate:
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct Job { ... }
|
||||
```
|
||||
|
||||
## Module Structure
|
||||
|
||||
- Use `pub(crate)` instead of `pub` when possible; expose only what needs exposing
|
||||
- Keep APIs small and expressive; avoid leaking internal types
|
||||
- Organize code into modules reflecting ownership and domain boundaries
|
||||
|
||||
```rust
|
||||
// Prefer restricted visibility
|
||||
pub(crate) fn internal_helper() { ... }
|
||||
|
||||
// Only pub for external API
|
||||
pub fn create_job(...) -> Result<Job> { ... }
|
||||
```
|
||||
|
||||
## Code Navigation
|
||||
|
||||
Always use rust-analyzer LSP for:
|
||||
- Go to definition
|
||||
- Find references
|
||||
- Type information
|
||||
- Import resolution
|
||||
|
||||
Do not guess at module paths or type definitions.
|
||||
|
||||
## JSON Handling
|
||||
|
||||
Prefer `Box<serde_json::value::RawValue>` over `serde_json::Value` when:
|
||||
- Storing JSON in the database (JSONB columns)
|
||||
- Passing JSON through without modification
|
||||
- The JSON structure doesn't need inspection
|
||||
|
||||
```rust
|
||||
// Preferred - avoids parsing/serialization overhead
|
||||
pub struct Job {
|
||||
pub id: Uuid,
|
||||
pub args: Option<Box<serde_json::value::RawValue>>,
|
||||
}
|
||||
|
||||
// Only use Value when you need to inspect/modify JSON
|
||||
let value: serde_json::Value = serde_json::from_str(&json)?;
|
||||
if let Some(field) = value.get("field") {
|
||||
// modify or inspect
|
||||
}
|
||||
```
|
||||
|
||||
## Serde Optimizations
|
||||
|
||||
Use serde attributes to optimize serialization:
|
||||
|
||||
```rust
|
||||
#[derive(Serialize, Deserialize)]
|
||||
pub struct Job {
|
||||
#[serde(rename = "jobId")]
|
||||
pub id: Uuid,
|
||||
|
||||
#[serde(default)]
|
||||
pub priority: i32,
|
||||
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub parent_job: Option<Uuid>,
|
||||
|
||||
#[serde(skip_serializing_if = "Vec::is_empty")]
|
||||
pub tags: Vec<String>,
|
||||
}
|
||||
```
|
||||
|
||||
Prefer borrowing for zero-copy deserialization when lifetimes allow:
|
||||
|
||||
```rust
|
||||
#[derive(Deserialize)]
|
||||
pub struct JobInput<'a> {
|
||||
#[serde(borrow)]
|
||||
pub workspace_id: Cow<'a, str>,
|
||||
|
||||
#[serde(borrow)]
|
||||
pub script_path: &'a str,
|
||||
}
|
||||
```
|
||||
|
||||
## SQLx Patterns
|
||||
|
||||
**Never use `SELECT *`** - always list columns explicitly. This is critical for backwards compatibility when workers run behind the API server version:
|
||||
|
||||
```rust
|
||||
// Preferred - explicit columns
|
||||
sqlx::query_as!(
|
||||
Job,
|
||||
"SELECT id, workspace_id, path, created_at FROM v2_job WHERE id = $1",
|
||||
job_id
|
||||
)
|
||||
|
||||
// Avoid - breaks when columns are added
|
||||
sqlx::query_as!(Job, "SELECT * FROM v2_job WHERE id = $1", job_id)
|
||||
```
|
||||
|
||||
Use batch operations to minimize round trips:
|
||||
|
||||
```rust
|
||||
// Preferred - single query with multiple values
|
||||
sqlx::query!(
|
||||
"INSERT INTO job_logs (job_id, logs) VALUES ($1, $2), ($3, $4)",
|
||||
id1, log1, id2, log2
|
||||
)
|
||||
|
||||
// Avoid N+1 queries
|
||||
for id in ids {
|
||||
sqlx::query!("SELECT ... WHERE id = $1", id).fetch_one(db).await?;
|
||||
}
|
||||
|
||||
// Preferred - single query with IN clause
|
||||
sqlx::query!("SELECT ... WHERE id = ANY($1)", &ids[..]).fetch_all(db).await?
|
||||
```
|
||||
|
||||
Use transactions for multi-step operations and parameterize all queries.
|
||||
|
||||
## Async & Tokio Patterns
|
||||
|
||||
Never block the async runtime. Use `spawn_blocking` for CPU-intensive or blocking I/O:
|
||||
|
||||
```rust
|
||||
// Preferred - offload blocking work
|
||||
let result = tokio::task::spawn_blocking(move || {
|
||||
expensive_computation(&data)
|
||||
}).await?;
|
||||
|
||||
// Avoid - blocks the runtime
|
||||
let result = expensive_computation(&data); // Don't do this in async
|
||||
```
|
||||
|
||||
Use tokio primitives for sleep and channels:
|
||||
|
||||
```rust
|
||||
use tokio::sync::mpsc;
|
||||
use tokio::time::sleep;
|
||||
|
||||
// Avoid in async contexts
|
||||
use std::thread::sleep; // Blocks the runtime
|
||||
```
|
||||
|
||||
Use bounded channels for backpressure:
|
||||
|
||||
```rust
|
||||
// Preferred - bounded channel prevents overwhelming
|
||||
let (tx, rx) = tokio::sync::mpsc::channel(100);
|
||||
|
||||
// Be careful with unbounded
|
||||
let (tx, rx) = tokio::sync::mpsc::unbounded_channel();
|
||||
```
|
||||
|
||||
## Mutex Selection in Async Code
|
||||
|
||||
**Prefer `std::sync::Mutex` (or `parking_lot::Mutex`) over `tokio::sync::Mutex`** for protecting data in async code. The async mutex is more expensive and only needed when holding locks across `.await` points.
|
||||
|
||||
```rust
|
||||
// Preferred for data protection - std mutex is faster
|
||||
use std::sync::Mutex;
|
||||
|
||||
struct Cache {
|
||||
data: Mutex<HashMap<String, Value>>,
|
||||
}
|
||||
|
||||
impl Cache {
|
||||
fn get(&self, key: &str) -> Option<Value> {
|
||||
self.data.lock().unwrap().get(key).cloned()
|
||||
}
|
||||
|
||||
fn insert(&self, key: String, value: Value) {
|
||||
self.data.lock().unwrap().insert(key, value);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Use `tokio::sync::Mutex` only when you must hold the lock across `.await` points**, typically for IO resources like database connections:
|
||||
|
||||
```rust
|
||||
use tokio::sync::Mutex;
|
||||
use std::sync::Arc;
|
||||
|
||||
// Async mutex for IO resources held across await points
|
||||
let conn = Arc::new(Mutex::new(db_connection));
|
||||
|
||||
async fn execute_query(conn: Arc<Mutex<DbConn>>, query: &str) {
|
||||
let mut lock = conn.lock().await;
|
||||
lock.execute(query).await; // Lock held across .await
|
||||
}
|
||||
```
|
||||
|
||||
**Common pattern**: Wrap `Arc<Mutex<...>>` in a struct with non-async methods that lock internally, keeping lock scope minimal:
|
||||
|
||||
```rust
|
||||
struct SharedState {
|
||||
inner: std::sync::Mutex<StateInner>,
|
||||
}
|
||||
|
||||
impl SharedState {
|
||||
fn update(&self, value: i32) {
|
||||
self.inner.lock().unwrap().value = value;
|
||||
}
|
||||
|
||||
fn get(&self) -> i32 {
|
||||
self.inner.lock().unwrap().value
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Alternative for IO resources**: Spawn a dedicated task to manage the resource and communicate via message passing:
|
||||
|
||||
```rust
|
||||
let (tx, mut rx) = tokio::sync::mpsc::channel(32);
|
||||
|
||||
tokio::spawn(async move {
|
||||
while let Some(cmd) = rx.recv().await {
|
||||
handle_io_command(&mut resource, cmd).await;
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
## Build & Tooling
|
||||
|
||||
Build speed tips:
|
||||
- Use `cargo check` during rapid iteration over `cargo build`
|
||||
- Minimize unnecessary dependencies and feature flags
|
||||
109
.cursor/rules/rust-best-practices.mdc
Normal file
109
.cursor/rules/rust-best-practices.mdc
Normal file
@@ -0,0 +1,109 @@
|
||||
---
|
||||
description:
|
||||
globs: backend/**/*.rs
|
||||
alwaysApply: false
|
||||
---
|
||||
# Windmill Backend - Rust Best Practices
|
||||
|
||||
## Project Structure
|
||||
|
||||
Windmill uses a workspace-based architecture with multiple crates:
|
||||
|
||||
- **windmill-api**: API server functionality
|
||||
- **windmill-worker**: Job execution
|
||||
- **windmill-common**: Shared code used by all crates
|
||||
- **windmill-queue**: Job & flow queuing
|
||||
- **windmill-audit**: Audit logging
|
||||
- Other specialized crates (git-sync, autoscaling, etc.)
|
||||
|
||||
## Adding New Code
|
||||
|
||||
### Module Organization
|
||||
|
||||
- Place new code in the appropriate crate based on functionality
|
||||
- For API endpoints, create or modify files in `windmill-api/src/` organized by domain
|
||||
- For shared functionality, use `windmill-common/src/`
|
||||
- Use the `_ee.rs` suffix for enterprise-only modules
|
||||
- Follow existing patterns for file structure and organization
|
||||
|
||||
### Error Handling
|
||||
|
||||
- Use the custom `Error` enum from `windmill-common::error`
|
||||
- Return `Result<T, Error>` or `JsonResult<T>` for functions that can fail
|
||||
- Use the `?` operator for error propagation
|
||||
- Add location tracking to errors using `#[track_caller]`
|
||||
|
||||
### Database Operations
|
||||
|
||||
- Use `sqlx` for database operations with prepared statements
|
||||
- Leverage existing database helper functions in `db.rs` modules
|
||||
- Use transactions for multi-step operations
|
||||
- Handle database errors properly
|
||||
|
||||
### API Endpoints
|
||||
|
||||
- Follow existing patterns in the `windmill-api` crate
|
||||
- Use axum's routing system and extractors
|
||||
- Group related routes together
|
||||
- Use consistent response formats (JSON)
|
||||
- Follow proper authentication and authorization patterns
|
||||
|
||||
## Performance Optimizations
|
||||
|
||||
When generating code, especially involving `serde`, `sqlx`, and `tokio`, prioritize performance by applying the following principles:
|
||||
|
||||
### Serde Optimizations (Serialization & Deserialization)
|
||||
|
||||
- **Specify Structure Explicitly:** When defining structs for Serde (`#[derive(Serialize, Deserialize)]`), use `#[serde(...` attributes extensively. This includes:
|
||||
* `#[serde(rename = "...")]` or `#[serde(alias = "...")]` to map external names precisely, avoiding dynamic lookups.
|
||||
* `#[serde(default)]` for optional fields with default values, reducing parsing complexity.
|
||||
* `#[serde(skip_serializing_if = "...")]` to avoid writing fields that meet a certain condition (e.g., `Option::is_none()`, `Vec::is_empty()`, or a custom function), reducing output size and serialization work.
|
||||
* `#[serde(skip_serializing)]` or `#[serde(skip_deserializing)]` for fields that should *not* be included.
|
||||
- **Prefer Borrowing:** Where possible and safe (data lifetime allows), use `Cow<'a, str>` or `&'a str` (with `#[serde(borrow)]`) instead of `String` for string fields during deserialization. This avoids allocating new strings, enabling zero-copy reading from the input buffer. Apply this principle to byte slices (`&'a [u8]` / `Cow<'a, [u8]>`) and potentially borrowed vectors as well.
|
||||
- **Avoid Intermediate `Value`:** Unless the data structure is truly dynamic or unknown at compile time, deserialize directly into a well-defined struct or enum rather than into `serde_json::Value` (or equivalent for other formats). This avoids unnecessary heap allocations and type switching.
|
||||
|
||||
### SQLx Optimizations (Database Interaction)
|
||||
|
||||
- **Select Only Necessary Columns:** In `SELECT` queries, list specific column names rather than using `SELECT *`. This reduces data transferred from the database and the work needed for hydration/deserialization.
|
||||
- **Batch Operations:** For multiple `INSERT`, `UPDATE`, or `DELETE` statements, prefer executing them in a single query if the database and driver support it efficiently (e.g., `INSERT INTO ... VALUES (...), (...), ...`). This minimizes round trips to the database.
|
||||
- **Avoid N+1 Queries:** Do not loop through results of one query and execute a separate query for each item (e.g., fetching users, then querying for each user's profile in a loop). Instead, use JOINs or a single query with an `IN` clause to fetch related data efficiently.
|
||||
- **Deserialize Directly:** Use `#[derive(FromRow)]` on structs and ensure the struct fields match the selected columns in the query. This allows SQLx to hydrate objects directly, avoiding intermediate data structures.
|
||||
- **Parameterize Queries:** Always use SQLx's query methods (`.bind(...)`) to pass values as parameters rather than string formatting. This prevents SQL injection and allows the database to cache query plans, improving performance on repeated executions.
|
||||
|
||||
### Tokio Optimizations (Asynchronous Runtime)
|
||||
|
||||
- **Avoid Blocking Operations:** **Crucially**, never perform blocking operations (synchronous file I/O, `std::thread::sleep`, CPU-bound loops, `std::sync::Mutex::lock`, blocking network calls without `tokio::net`) directly within an `async fn` or a standard `tokio::spawn` task. Blocking pauses the entire worker thread, potentially starving other tasks. Use `tokio::task::spawn_blocking` for CPU-intensive work or blocking I/O.
|
||||
- **Use Tokio's Async Primitives:** Prefer `tokio::sync` (channels, mutexes, semaphores), `tokio::io`, `tokio::net`, and `tokio::time` over their `std` counterparts in asynchronous contexts. These are designed to yield control back to the scheduler.
|
||||
- **Manage Concurrency:** Be mindful of how many tasks are spawned. Creating a new task for every tiny piece of work can introduce overhead. Group related asynchronous operations where appropriate.
|
||||
- **Handle Shared State Efficiently:** Use `Arc` for shared ownership in concurrent tasks. When shared state needs mutation, prefer `tokio::sync::Mutex` over `std::sync::Mutex` in `async` code. Consider `tokio::sync::RwLock` if reads significantly outnumber writes. Minimize the duration for which locks are held.
|
||||
- **Understand `.await`:** Place `.await` strategically to allow the runtime to switch to other ready tasks. Ensure that `.await` points to genuinely asynchronous operations.
|
||||
- **Backpressure:** If dealing with data streams or queues between tasks, implement backpressure mechanisms (e.g., bounded channels like `tokio::sync::mpsc::channel`) to prevent one component from overwhelming another or critical resources like the database.
|
||||
|
||||
## Enterprise Features
|
||||
|
||||
- Use feature flags for enterprise functionality
|
||||
- Conditionally compile with `#[cfg(feature = "enterprise")]`
|
||||
- Isolate enterprise code in separate modules
|
||||
|
||||
## Code Style
|
||||
|
||||
- Group imports by external and internal crates
|
||||
- Place struct/enum definitions before implementations
|
||||
- Group similar functionality together
|
||||
- Use descriptive naming consistent with the codebase
|
||||
- Follow existing patterns for async code using tokio
|
||||
|
||||
## Testing
|
||||
|
||||
- Write unit tests for core functionality
|
||||
- Use the `#[cfg(test)]` module for test code
|
||||
- For database tests, use the existing test utilities
|
||||
|
||||
## Common Crates Used
|
||||
|
||||
- **tokio**: For async runtime
|
||||
- **axum**: For web server and routing
|
||||
- **sqlx**: For database operations
|
||||
- **serde**: For serialization/deserialization
|
||||
- **tracing**: For logging and diagnostics
|
||||
- **reqwest**: For HTTP client functionality
|
||||
@@ -5,7 +5,7 @@ alwaysApply: false
|
||||
---
|
||||
# Svelte 5 Best Practices
|
||||
|
||||
This guide outlines best practices for developing with Svelte 5, incorporating the new Runes API and other modern Svelte features. These rules MUST NOT be applied on svelte 4 files unless explicitly asked to do so.
|
||||
This guide outlines best practices for developing with Svelte 5, incorporating the new Runes API and other modern Svelte features. They should be applied on every new files created, but not on existing svelte 4 files unless specifically asked to.
|
||||
|
||||
## Reactivity with Runes
|
||||
|
||||
@@ -3,4 +3,3 @@ frontend/build/
|
||||
frontend/.svelte-kit/
|
||||
|
||||
backend/target/
|
||||
backend/windmill-duckdb-ffi-internal/target/
|
||||
6
.github/CODEOWNERS
vendored
6
.github/CODEOWNERS
vendored
@@ -1,4 +1,4 @@
|
||||
* @rubenfiszel @hugocasa @alpetric
|
||||
* @rubenfiszel @HugoCasa @alpetric
|
||||
|
||||
/community/ @rubenfiszel @hugocasa @alpetric
|
||||
/frontend/ @rubenfiszel @hugocasa @alpetric
|
||||
/community/ @rubenfiszel @HugoCasa @alpetric
|
||||
/frontend/ @rubenfiszel @HugoCasa @alpetric
|
||||
|
||||
12
.github/DockerfileBackendTests
vendored
12
.github/DockerfileBackendTests
vendored
@@ -28,7 +28,7 @@ ENV PATH="${PATH}:/usr/local/go/bin"
|
||||
ENV GO_PATH=/usr/local/go/bin/go
|
||||
|
||||
# UV
|
||||
RUN curl --proto '=https' --tlsv1.2 -LsSf https://github.com/astral-sh/uv/releases/download/0.9.24/uv-installer.sh | sh && mv /usr/local/cargo/bin/uv /usr/local/bin/uv
|
||||
RUN curl --proto '=https' --tlsv1.2 -LsSf https://github.com/astral-sh/uv/releases/download/0.4.18/uv-installer.sh | sh && mv /usr/local/cargo/bin/uv /usr/local/bin/uv
|
||||
|
||||
ENV TZ=Etc/UTC
|
||||
|
||||
@@ -42,7 +42,7 @@ RUN wget https://www.python.org/ftp/python/${PYTHON_VERSION}/Python-${PYTHON_VER
|
||||
RUN /usr/local/bin/python3 -m pip install pip-tools
|
||||
|
||||
# Bun
|
||||
COPY --from=oven/bun:1.3.8 /usr/local/bin/bun /usr/bin/bun
|
||||
COPY --from=oven/bun:1.2.4 /usr/local/bin/bun /usr/bin/bun
|
||||
|
||||
ARG TARGETPLATFORM
|
||||
|
||||
@@ -57,12 +57,8 @@ RUN apt-get update \
|
||||
RUN rustup component add rustfmt
|
||||
|
||||
# C#
|
||||
RUN wget https://dot.net/v1/dotnet-install.sh -O dotnet-install.sh \
|
||||
&& chmod +x dotnet-install.sh \
|
||||
&& ./dotnet-install.sh --channel 9.0 --install-dir /usr/share/dotnet \
|
||||
&& ln -s /usr/share/dotnet/dotnet /usr/bin/dotnet \
|
||||
&& rm dotnet-install.sh
|
||||
|
||||
COPY --from=bitnami/dotnet-sdk:9.0.101-debian-12-r0 /opt/bitnami/dotnet-sdk /opt/dotnet-sdk
|
||||
RUN ln -s /opt/dotnet-sdk/bin/dotnet /usr/bin/dotnet
|
||||
|
||||
# Nushell
|
||||
COPY --from=ghcr.io/nushell/nushell:0.101.0-bookworm /usr/bin/nu /usr/bin/nu
|
||||
|
||||
2
.github/change-versions-mac.sh
vendored
2
.github/change-versions-mac.sh
vendored
@@ -7,7 +7,7 @@ VERSION=$1
|
||||
echo "Updating versions to: $VERSION"
|
||||
|
||||
sed -i '' -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/backend/Cargo.toml
|
||||
sed -i '' -e "/^export const VERSION =/s/= .*/= \"v$VERSION\";/" ${root_dirpath}/cli/src/main.ts
|
||||
sed -i '' -e "/^export const VERSION =/s/= .*/= \"v$VERSION\";/" ${root_dirpath}/cli/main.ts
|
||||
sed -i '' -e "/^export const VERSION =/s/= .*/= \"v$VERSION\";/" ${root_dirpath}/benchmarks/lib.ts
|
||||
sed -i '' -e "/version: /s/: .*/: $VERSION/" ${root_dirpath}/backend/windmill-api/openapi.yaml
|
||||
sed -i '' -e "/version: /s/: .*/: $VERSION/" ${root_dirpath}/openflow.openapi.yaml
|
||||
|
||||
2
.github/change-versions.sh
vendored
2
.github/change-versions.sh
vendored
@@ -7,7 +7,7 @@ VERSION=$1
|
||||
echo "Updating versions to: $VERSION"
|
||||
|
||||
sed -i -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/backend/Cargo.toml
|
||||
sed -i -e "/^export const VERSION =/s/= .*/= \"$VERSION\";/" ${root_dirpath}/cli/src/main.ts
|
||||
sed -i -e "/^export const VERSION =/s/= .*/= \"$VERSION\";/" ${root_dirpath}/cli/main.ts
|
||||
sed -i -e "/^export const VERSION =/s/= .*/= \"v$VERSION\";/" ${root_dirpath}/benchmarks/lib.ts
|
||||
sed -i -e "/version: /s/: .*/: $VERSION/" ${root_dirpath}/backend/windmill-api/openapi.yaml
|
||||
sed -i -e "/version: /s/: .*/: $VERSION/" ${root_dirpath}/openflow.openapi.yaml
|
||||
|
||||
170
.github/workflows/aider-after-review.yaml
vendored
Normal file
170
.github/workflows/aider-after-review.yaml
vendored
Normal file
@@ -0,0 +1,170 @@
|
||||
name: Aider Auto-fix PR Review Change Requests
|
||||
|
||||
on:
|
||||
pull_request_review:
|
||||
types: [submitted]
|
||||
|
||||
jobs:
|
||||
auto-fix-review:
|
||||
if: github.event.review.state == 'changes_requested' && contains(github.event.pull_request.title, '[Aider PR]')
|
||||
runs-on: ubicloud-standard-8
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
env:
|
||||
GEMINI_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
|
||||
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
WINDMILL_TOKEN: ${{ secrets.WINDMILL_TOKEN }}
|
||||
|
||||
steps:
|
||||
- name: Harden Runner
|
||||
uses: step-security/harden-runner@v2
|
||||
with:
|
||||
egress-policy: audit
|
||||
|
||||
- name: Check out code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Configure Git User
|
||||
run: |
|
||||
git config --global user.name "github-actions[bot]"
|
||||
git config --global user.email "github-actions[bot]@users.noreply.github.com"
|
||||
|
||||
- name: Checkout PR Branch
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
run: |
|
||||
echo "PR review trigger: Checking out PR branch..."
|
||||
PR_NUMBER=${{ github.event.pull_request.number }}
|
||||
PR_HEAD_REF=$(gh pr view $PR_NUMBER --json headRefName -q .headRefName --repo $GITHUB_REPOSITORY)
|
||||
if [[ -z "$PR_HEAD_REF" || "$PR_HEAD_REF" == "null" ]]; then
|
||||
echo "::error::Could not determine PR head branch for PR #$PR_NUMBER via gh CLI."
|
||||
exit 1
|
||||
fi
|
||||
echo "Checking out PR head branch: $PR_HEAD_REF for PR #$PR_NUMBER"
|
||||
git fetch origin "refs/heads/${PR_HEAD_REF}:refs/remotes/origin/${PR_HEAD_REF}" --no-tags
|
||||
git checkout "$PR_HEAD_REF"
|
||||
echo "Successfully checked out branch $(git rev-parse --abbrev-ref HEAD)"
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Install Aider and Dependencies
|
||||
run: |
|
||||
python -m pip install aider-install; aider-install
|
||||
pip install -U google-generativeai
|
||||
sudo apt-get update && sudo apt-get install -y jq
|
||||
|
||||
- name: Generate Prompt from Review
|
||||
id: generate_prompt
|
||||
shell: bash
|
||||
run: |
|
||||
mkdir -p .github/aider
|
||||
PROMPT_FILE_PATH=".github/aider/review-prompt.txt"
|
||||
|
||||
# Get PR review body
|
||||
REVIEW_BODY="${{ github.event.review.body }}"
|
||||
PR_NUMBER="${{ github.event.pull_request.number }}"
|
||||
|
||||
# Get PR description for context NOT USED FOR NOW
|
||||
# PR_DETAILS=$(gh pr view $PR_NUMBER --json title,body --repo $GITHUB_REPOSITORY)
|
||||
# PR_TITLE=$(echo "$PR_DETAILS" | jq -r .title)
|
||||
# PR_BODY=$(echo "$PR_DETAILS" | jq -r .body)
|
||||
|
||||
# Get all PR review comments
|
||||
REVIEW_COMMENTS=$(gh pr view $PR_NUMBER --json reviews -q '.reviews[] | select(.state == "CHANGES_REQUESTED") | .body' --repo $GITHUB_REPOSITORY)
|
||||
REVIEW_BODY_Q=$(printf '%q' "$REVIEW_BODY")
|
||||
|
||||
# Update query to get review comments from all review types, not just "CHANGES_REQUESTED"
|
||||
ALL_REVIEW_COMMENTS=$(gh api \
|
||||
-H "Accept: application/vnd.github+json" \
|
||||
-H "X-GitHub-Api-Version: 2022-11-28" \
|
||||
/repos/$GITHUB_REPOSITORY/pulls/$PR_NUMBER/comments \
|
||||
| jq '[.[] | {diff_hunk: .diff_hunk, path: .path, body: .body}]')
|
||||
|
||||
BASE_PROMPT="Fix the following issues in the PR based on the review feedback. The review body is prepended with REVIEW. The review comments are prepended with REVIEW_COMMENTS. The review body and comments are separated by a blank line."
|
||||
printf "%s\nREVIEW:\n%s\nREVIEW_COMMENTS:\n%s" \
|
||||
"$BASE_PROMPT" "$REVIEW_BODY_Q" "$ALL_REVIEW_COMMENTS" > "$PROMPT_FILE_PATH"
|
||||
echo "PROMPT_FILE_PATH=$PROMPT_FILE_PATH" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Run Aider with review prompt
|
||||
run: |
|
||||
aider \
|
||||
--read .cursor/rules/rust-best-practices.mdc \
|
||||
--read .cursor/rules/svelte5-best-practices.mdc \
|
||||
--model gemini/gemini-2.5-pro-preview-05-06 \
|
||||
--message-file .github/aider/review-prompt.txt \
|
||||
--yes \
|
||||
--no-check-update \
|
||||
--auto-commits \
|
||||
--no-analytics \
|
||||
--no-gitignore \
|
||||
| tee .github/aider/aider-output.txt || true
|
||||
echo "Aider command completed. Output saved to .github/aider/aider-output.txt"
|
||||
# Check if there are any changes to commit
|
||||
if [[ -z "$(git status --porcelain)" ]]; then
|
||||
echo "No changes detected after running Aider."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
- name: Clean up prompt file
|
||||
if: always()
|
||||
run: rm -f .github/aider/review-prompt.txt
|
||||
|
||||
- name: Commit and Push Changes
|
||||
id: commit_and_push
|
||||
if: ${{ success() }}
|
||||
run: |
|
||||
CURRENT_BRANCH_NAME=$(git rev-parse --abbrev-ref HEAD)
|
||||
echo "Attempting to push changes to PR branch $CURRENT_BRANCH_NAME for PR #${{ github.event.pull_request.number }}"
|
||||
|
||||
# Pull latest changes to avoid rejection due to non-fast-forward
|
||||
git pull origin $CURRENT_BRANCH_NAME
|
||||
|
||||
if git push origin $CURRENT_BRANCH_NAME; then
|
||||
echo "Push to $CURRENT_BRANCH_NAME successful."
|
||||
echo "CHANGES_APPLIED=true" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "::warning::Push to PR branch $CURRENT_BRANCH_NAME failed."
|
||||
echo "CHANGES_APPLIED=false" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
- name: Comment on PR
|
||||
if: success()
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
PR_NUM: ${{ github.event.pull_request.number }}
|
||||
run: |
|
||||
# Create comment body in a temporary file to avoid command line length limits
|
||||
if [[ "${{ steps.commit_and_push.outputs.CHANGES_APPLIED }}" == "true" ]]; then
|
||||
cat > /tmp/pr-comment.md << EOL
|
||||
🤖 I've automatically addressed the feedback based on the review.
|
||||
|
||||
## Aider Output
|
||||
\`\`\`
|
||||
$(cat .github/aider/aider-output.txt || echo 'No output available')
|
||||
\`\`\`
|
||||
|
||||
Please review the changes and let me know if further adjustments are needed.
|
||||
EOL
|
||||
else
|
||||
cat > /tmp/pr-comment.md << EOL
|
||||
🤖 I attempted to address the review feedback, but no modifications were made.
|
||||
|
||||
## Aider Output
|
||||
\`\`\`
|
||||
$(cat .github/aider/aider-output.txt || echo 'No output available')
|
||||
\`\`\`
|
||||
|
||||
Please review the output and provide additional guidance if needed.
|
||||
EOL
|
||||
fi
|
||||
|
||||
# Use the file for comment body
|
||||
gh pr comment $PR_NUM --body-file /tmp/pr-comment.md
|
||||
@@ -1,94 +0,0 @@
|
||||
name: Aider Auto-fix PR Review Change Requests
|
||||
|
||||
on:
|
||||
pull_request_review:
|
||||
types: [submitted]
|
||||
|
||||
jobs:
|
||||
check-membership:
|
||||
if: github.event.review.state == 'changes_requested' && contains(github.event.pull_request.title, '[Aider PR]')
|
||||
runs-on: ubicloud-standard-2
|
||||
outputs:
|
||||
is_member: ${{ steps.check-membership.outputs.is_member }}
|
||||
steps:
|
||||
- name: Check organization membership
|
||||
id: check-membership
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
REVIEWER: ${{ github.event.review.user.login }}
|
||||
ORG_ACCESS_TOKEN: ${{ secrets.ORG_ACCESS_TOKEN }}
|
||||
run: |
|
||||
ORG="windmill-labs"
|
||||
STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
|
||||
-H "Authorization: token $ORG_ACCESS_TOKEN" \
|
||||
-H "Accept: application/vnd.github+json" \
|
||||
-H "X-GitHub-Api-Version: 2022-11-28" \
|
||||
"https://api.github.com/orgs/$ORG/members/$REVIEWER")
|
||||
|
||||
if [ "$STATUS" -eq 204 ]; then
|
||||
echo "is_member=true" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "is_member=false" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
check-and-prepare:
|
||||
needs: check-membership
|
||||
if: github.event.review.state == 'changes_requested' && contains(github.event.pull_request.title, '[Aider PR]') && needs.check-membership.outputs.is_member == 'true'
|
||||
runs-on: ubicloud-standard-2
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
outputs:
|
||||
prompt_content: ${{ steps.prepare_prompt.outputs.prompt_content }}
|
||||
env:
|
||||
GEMINI_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
|
||||
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
WINDMILL_TOKEN: ${{ secrets.WINDMILL_TOKEN }}
|
||||
|
||||
steps:
|
||||
- name: Acknowledge Request
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
run: |
|
||||
echo "Commenting on PR #${{ github.event.pull_request.number }} to acknowledge the /aider command."
|
||||
gh pr comment ${{ github.event.pull_request.number }} --body "🤖 Aider is starting to work on your request. Please be patient, this might take a few minutes." --repo $GITHUB_REPOSITORY
|
||||
|
||||
- name: Prepare prompt for Aider
|
||||
id: prepare_prompt
|
||||
shell: bash
|
||||
env:
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
REVIEW_BODY: ${{ github.event.review.body }}
|
||||
run: |
|
||||
REVIEW_BODY_ESCAPED="${REVIEW_BODY//\\/\\\\}"
|
||||
REVIEW_BODY_ESCAPED="${REVIEW_BODY_ESCAPED//\"/\\\"}"
|
||||
|
||||
ALL_REVIEW_COMMENTS=$(gh api \
|
||||
-H "Accept: application/vnd.github+json" \
|
||||
-H "X-GitHub-Api-Version: 2022-11-28" \
|
||||
/repos/$GITHUB_REPOSITORY/pulls/$PR_NUMBER/comments)
|
||||
|
||||
FORMATTED_COMMENTS=$(jq -r '[.[] | {diff_hunk: .diff_hunk, path: .path, body: .body}]' <<< "$ALL_REVIEW_COMMENTS")
|
||||
|
||||
BASE_PROMPT="Fix the following issues in the PR based on the review feedback. The review body is prepended with REVIEW. The review comments are prepended with REVIEW_COMMENTS. The review body and comments are separated by a blank line."
|
||||
|
||||
COMPLETE_PROMPT="${BASE_PROMPT}"$'\n'"REVIEW:"$'\n'"${REVIEW_BODY_ESCAPED}"$'\n'"REVIEW_COMMENTS:"$'\n'"${FORMATTED_COMMENTS}"
|
||||
|
||||
echo "prompt_content<<EOF" >> $GITHUB_OUTPUT
|
||||
echo "$COMPLETE_PROMPT" >> $GITHUB_OUTPUT
|
||||
echo "EOF" >> $GITHUB_OUTPUT
|
||||
|
||||
run-aider:
|
||||
needs: [check-membership, check-and-prepare]
|
||||
if: github.event.review.state == 'changes_requested' && contains(github.event.pull_request.title, '[Aider PR]') && needs.check-membership.outputs.is_member == 'true'
|
||||
uses: ./.github/workflows/aider-common.yml
|
||||
with:
|
||||
needs_processing: false
|
||||
base_prompt: ${{ needs.check-and-prepare.outputs.prompt_content }}
|
||||
rules_files: ".cursor/rules/rust-best-practices.mdc .cursor/rules/svelte5-best-practices.mdc .cursor/rules/windmill-overview.mdc"
|
||||
secrets: inherit
|
||||
522
.github/workflows/aider-common.yml.archived
vendored
522
.github/workflows/aider-common.yml.archived
vendored
@@ -1,522 +0,0 @@
|
||||
name: Aider Common Steps
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
issue_title:
|
||||
description: "Title of the issue or PR"
|
||||
required: false
|
||||
type: string
|
||||
issue_body:
|
||||
description: "Body of the issue or PR"
|
||||
required: false
|
||||
type: string
|
||||
instruction:
|
||||
description: "Instruction for Aider"
|
||||
required: false
|
||||
type: string
|
||||
issue_id:
|
||||
description: "ID of the issue or PR"
|
||||
required: false
|
||||
type: string
|
||||
needs_processing:
|
||||
description: "Whether the issue needs to be processed by the external API"
|
||||
required: false
|
||||
type: boolean
|
||||
default: true
|
||||
base_prompt:
|
||||
description: "Base prompt for Aider"
|
||||
required: false
|
||||
type: string
|
||||
default: "Try to fix the following issue based on the instruction given by the user. The issue is prepended with the word ISSUE. The instruction is prepended with the word INSTRUCTION. The issue and instruction are separated by a blank line."
|
||||
probe_prompt:
|
||||
description: "Prompt for probe-chat"
|
||||
required: false
|
||||
type: string
|
||||
default: 'I''m giving you a request that needs to be implemented. Your role is ONLY to give me the files that are relevant to the request and nothing else. The request is prepended with the word REQUEST. Give me all the files relevant to this request. Your output MUST be a single json array that can be parsed with programatic json parsing, with the relevant files. Files can be rust or typescript or javascript files. DO NOT INCLUDE ANY OTHER TEXT IN YOUR OUTPUT. ONLY THE JSON ARRAY. Example of output: ["file1.py", "file2.py"]'
|
||||
rules_files:
|
||||
description: "Rules files for Aider"
|
||||
required: false
|
||||
type: string
|
||||
outputs:
|
||||
files_to_edit:
|
||||
description: "Files identified by probe-chat for editing"
|
||||
value: ${{ jobs.common-steps.outputs.files_to_edit }}
|
||||
final_prompt:
|
||||
description: "Final prompt for Aider"
|
||||
value: ${{ jobs.common-steps.outputs.final_prompt }}
|
||||
pr_branch_name:
|
||||
description: "Name of the branch used for PR"
|
||||
value: ${{ jobs.common-steps.outputs.pr_branch_name }}
|
||||
changes_applied_message:
|
||||
description: "Message indicating changes were applied"
|
||||
value: ${{ jobs.common-steps.outputs.changes_applied_message }}
|
||||
changes_applied:
|
||||
description: "Boolean indicating if changes were successfully applied"
|
||||
value: ${{ jobs.common-steps.outputs.changes_applied }}
|
||||
|
||||
jobs:
|
||||
common-steps:
|
||||
runs-on: ubicloud-standard-8
|
||||
outputs:
|
||||
files_to_edit: ${{ steps.probe_files.outputs.files_to_edit }}
|
||||
final_prompt: ${{ steps.create_prompt.outputs.final_prompt }}
|
||||
pr_branch_name: ${{ steps.commit_and_push.outputs.PR_BRANCH_NAME }}
|
||||
changes_applied_message: ${{ steps.commit_and_push.outputs.CHANGES_APPLIED_MESSAGE }}
|
||||
changes_applied: ${{ steps.commit_and_push.outputs.CHANGES_APPLIED }}
|
||||
env:
|
||||
GEMINI_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
|
||||
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
WINDMILL_TOKEN: ${{ secrets.WINDMILL_TOKEN }}
|
||||
LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}
|
||||
DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_AI_BOT_TOKEN }}
|
||||
|
||||
steps:
|
||||
- name: Harden Runner
|
||||
uses: step-security/harden-runner@v2
|
||||
with:
|
||||
egress-policy: audit
|
||||
|
||||
- name: Check out code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Checkout PR Branch
|
||||
id: checkout_pr
|
||||
if: (github.event_name == 'issue_comment' && github.event.issue.pull_request) || (github.event_name == 'pull_request_review')
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
run: |
|
||||
echo "Issue comment trigger: Checking out PR branch..."
|
||||
PR_NUMBER=""
|
||||
if [ -n "${{ github.event.issue.number }}" ]; then
|
||||
PR_NUMBER="${{ github.event.issue.number }}"
|
||||
elif [ -n "${{ github.event.pull_request.number }}" ]; then
|
||||
PR_NUMBER="${{ github.event.pull_request.number }}"
|
||||
else
|
||||
echo "::error::Could not determine PR number."
|
||||
exit 1
|
||||
fi
|
||||
PR_HEAD_REF=$(gh pr view $PR_NUMBER --json headRefName -q .headRefName --repo $GITHUB_REPOSITORY)
|
||||
if [[ -z "$PR_HEAD_REF" || "$PR_HEAD_REF" == "null" ]]; then
|
||||
echo "::error::Could not determine PR head branch for PR #$PR_NUMBER via gh CLI."
|
||||
exit 1
|
||||
fi
|
||||
echo "Checking out PR head branch: $PR_HEAD_REF for PR #$PR_NUMBER"
|
||||
git fetch origin "refs/heads/${PR_HEAD_REF}:refs/remotes/origin/${PR_HEAD_REF}" --no-tags
|
||||
git checkout "$PR_HEAD_REF"
|
||||
echo "Successfully checked out branch $(git rev-parse --abbrev-ref HEAD)"
|
||||
echo "PR_BRANCH=$PR_HEAD_REF" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Configure Git User
|
||||
run: |
|
||||
git config --global user.name "github-actions[bot]"
|
||||
git config --global user.email "github-actions[bot]@users.noreply.github.com"
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.11"
|
||||
|
||||
- name: Cache Python dependencies
|
||||
uses: actions/cache@v3
|
||||
with:
|
||||
path: ~/.cache/pip
|
||||
key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt', '**/setup.py') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-pip-
|
||||
|
||||
- name: Install Aider and Dependencies
|
||||
run: |
|
||||
echo "Installing Aider..."
|
||||
python -m pip install uv
|
||||
python -m venv ~/uv-env
|
||||
source ~/uv-env/bin/activate
|
||||
uv pip install configargparse==1.7
|
||||
uv pip install aider-chat==0.83.1
|
||||
uv pip install -U google-generativeai
|
||||
sudo apt-get update && sudo apt-get install -y jq
|
||||
echo "$HOME/.local/bin" >> $GITHUB_PATH
|
||||
echo "VIRTUAL_ENV_PATH=$HOME/uv-env" >> $GITHUB_ENV
|
||||
|
||||
- name: Create Prompt for Aider
|
||||
id: create_prompt
|
||||
shell: bash
|
||||
env:
|
||||
BASE_PROMPT_ENV: ${{ inputs.base_prompt }}
|
||||
ISSUE_TITLE_ENV: ${{ inputs.issue_title }}
|
||||
ISSUE_BODY_ENV: ${{ inputs.issue_body }}
|
||||
INSTRUCTION_ENV: ${{ inputs.instruction }}
|
||||
NEEDS_PROCESSING_ENV: ${{ inputs.needs_processing }}
|
||||
WINDMILL_TOKEN: ${{ secrets.WINDMILL_TOKEN }}
|
||||
run: |
|
||||
set -e
|
||||
FINAL_PROMPT_CONTENT=""
|
||||
|
||||
if [[ "$ISSUE_TITLE_ENV" != "" && "$ISSUE_BODY_ENV" != "" ]]; then
|
||||
echo "Processing issue with title: $ISSUE_TITLE_ENV"
|
||||
if [[ "$NEEDS_PROCESSING_ENV" == "true" ]]; then
|
||||
echo "Needs processing is true. Calling Windmill API..."
|
||||
JSON_PAYLOAD=$(jq -n \
|
||||
--arg title "$ISSUE_TITLE_ENV" \
|
||||
--arg body "$ISSUE_BODY_ENV" \
|
||||
'{"body":{"issue_title":$title,"issue_body":$body}}')
|
||||
|
||||
echo "Windmill JSON Payload: $JSON_PAYLOAD"
|
||||
|
||||
API_RESULT_FILE=$(mktemp)
|
||||
HTTP_CODE=$(curl -s -o "$API_RESULT_FILE" -w "%{http_code}" \
|
||||
-X POST "https://app.windmill.dev/api/w/windmill-labs/jobs/run_wait_result/p/f/ai/quiet_script" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer $WINDMILL_TOKEN" \
|
||||
--data-binary "$JSON_PAYLOAD" \
|
||||
--max-time 90)
|
||||
|
||||
BODY_CONTENT=$(cat "$API_RESULT_FILE")
|
||||
rm -f "$API_RESULT_FILE" # Clean up temp file
|
||||
|
||||
echo "Windmill API HTTP Code: $HTTP_CODE"
|
||||
if [[ "$HTTP_CODE" -eq 200 ]]; then
|
||||
PROCESSED_ISSUE_PROMPT=$(echo "$BODY_CONTENT" | jq -r '.effective_body // empty')
|
||||
if [[ -z "$PROCESSED_ISSUE_PROMPT" || "$PROCESSED_ISSUE_PROMPT" == "null" ]]; then
|
||||
echo "::warning::Windmill API returned 200 but effective_body was empty or null."
|
||||
EFFECTIVE_ISSUE_CONTENT_FOR_PROMPT="$ISSUE_BODY_ENV"
|
||||
else
|
||||
echo "Successfully processed issue via Windmill API."
|
||||
EFFECTIVE_ISSUE_CONTENT_FOR_PROMPT="$PROCESSED_ISSUE_PROMPT"
|
||||
fi
|
||||
FINAL_PROMPT_CONTENT=$(printf "%s\nISSUE:\n%s\nINSTRUCTION:\n%s" \
|
||||
"$BASE_PROMPT_ENV" "$EFFECTIVE_ISSUE_CONTENT_FOR_PROMPT" "$INSTRUCTION_ENV")
|
||||
else
|
||||
echo "::error::Windmill API call failed (HTTP $HTTP_CODE). Using raw issue content for prompt."
|
||||
FINAL_PROMPT_CONTENT=$(printf "%s\nISSUE:\n%s\nINSTRUCTION:\n%s" \
|
||||
"$BASE_PROMPT_ENV" "$ISSUE_BODY_ENV" "$INSTRUCTION_ENV")
|
||||
fi
|
||||
else
|
||||
echo "Needs processing is false. Using raw issue content for prompt."
|
||||
FINAL_PROMPT_CONTENT=$(printf "%s\nISSUE:\n%s\nINSTRUCTION:\n%s" \
|
||||
"$BASE_PROMPT_ENV" "$ISSUE_BODY_ENV" "$INSTRUCTION_ENV")
|
||||
fi
|
||||
else
|
||||
echo "No issue title or body given. Using base prompt."
|
||||
FINAL_PROMPT_CONTENT=$(printf "%s\nINSTRUCTION:\n%s" "$BASE_PROMPT_ENV" "$INSTRUCTION_ENV")
|
||||
fi
|
||||
|
||||
echo "Final prompt: $FINAL_PROMPT_CONTENT"
|
||||
echo "final_prompt<<EOF_AIDER_PROMPT" >> "$GITHUB_OUTPUT"
|
||||
echo "$FINAL_PROMPT_CONTENT" >> "$GITHUB_OUTPUT"
|
||||
echo "EOF_AIDER_PROMPT" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Probe Chat for Relevant Files
|
||||
id: probe_files
|
||||
shell: bash
|
||||
env:
|
||||
FINAL_PROMPT: ${{ steps.create_prompt.outputs.final_prompt }}
|
||||
PROBE_PROMPT: ${{ inputs.probe_prompt }}
|
||||
run: |
|
||||
echo "Running probe-chat to find relevant files..."
|
||||
|
||||
MESSAGE_FOR_PROBE=$(printf "%s\nREQUEST:\n%s" "$PROBE_PROMPT" "$FINAL_PROMPT")
|
||||
|
||||
set -o pipefail
|
||||
PROBE_OUTPUT=$(npx --yes @buger/probe-chat@latest --max-iterations 50 --model-name gemini-2.5-pro-preview-05-06 --message "$MESSAGE_FOR_PROBE") || {
|
||||
echo "::error::probe-chat command failed. Output:"
|
||||
echo "$PROBE_OUTPUT"
|
||||
exit 1
|
||||
}
|
||||
set +o pipefail
|
||||
echo "Probe-chat raw output:"
|
||||
echo "$PROBE_OUTPUT"
|
||||
|
||||
JSON_FILES=$(echo "$PROBE_OUTPUT" | sed -n '/^\s*\[/,$p' | sed '/^\s*\]/q')
|
||||
echo "Extracted JSON block:"
|
||||
echo "$JSON_FILES"
|
||||
|
||||
FILES_LIST=$(echo "$JSON_FILES" | jq -e -r '[.[] | select(type == "string" and . != "" and . != null and (endswith("/") | not))] | join(" ")' || echo "")
|
||||
|
||||
if [[ -z "$FILES_LIST" ]]; then
|
||||
echo "::warning::probe-chat did not identify any relevant files."
|
||||
fi
|
||||
|
||||
echo "Formatted files list for aider: $FILES_LIST"
|
||||
echo "files_to_edit=$FILES_LIST" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Cache Aider tags
|
||||
uses: actions/cache@v3
|
||||
with:
|
||||
path: .aider.tags.cache.v4
|
||||
key: ${{ runner.os }}-aider-${{ github.sha }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-aider-
|
||||
|
||||
- name: Prepare branch for Aider
|
||||
id: prepare_branch
|
||||
env:
|
||||
ISSUE_ID: ${{ inputs.issue_id }}
|
||||
run: |
|
||||
if [[ "$ISSUE_ID" != "" ]]; then
|
||||
BRANCH_NAME="aider-fix-issue-${ISSUE_ID}"
|
||||
|
||||
# Check if branch exists remotely
|
||||
if git ls-remote --heads origin $BRANCH_NAME | grep -q $BRANCH_NAME; then
|
||||
echo "Branch $BRANCH_NAME already exists remotely, fetching it"
|
||||
git fetch origin $BRANCH_NAME
|
||||
git checkout $BRANCH_NAME
|
||||
git pull origin $BRANCH_NAME
|
||||
else
|
||||
echo "Creating new branch $BRANCH_NAME"
|
||||
git checkout -b $BRANCH_NAME
|
||||
fi
|
||||
echo "BRANCH_NAME=$BRANCH_NAME" >> $GITHUB_OUTPUT
|
||||
else
|
||||
# We're in a pull_request_review event
|
||||
PR_NUMBER="${{ github.event.pull_request.number }}"
|
||||
PR_HEAD_REF="${{ github.event.pull_request.head.ref }}"
|
||||
|
||||
echo "Handling pull_request_review for PR #$PR_NUMBER on branch $PR_HEAD_REF"
|
||||
|
||||
# Ensure we're on the correct branch
|
||||
git config pull.rebase true
|
||||
git fetch origin $PR_HEAD_REF
|
||||
git checkout $PR_HEAD_REF
|
||||
git pull origin $PR_HEAD_REF
|
||||
|
||||
echo "Using PR branch $PR_HEAD_REF for PR #$PR_NUMBER"
|
||||
echo "BRANCH_NAME=$PR_HEAD_REF" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
- name: Run Aider
|
||||
id: run_aider
|
||||
shell: bash
|
||||
env:
|
||||
FILES_TO_EDIT: ${{ steps.probe_files.outputs.files_to_edit }}
|
||||
FINAL_PROMPT: ${{ steps.create_prompt.outputs.final_prompt }}
|
||||
RULES_FILES: ${{ inputs.rules_files }}
|
||||
run: |
|
||||
source $VIRTUAL_ENV_PATH/bin/activate
|
||||
echo "$FINAL_PROMPT" > .aider_final_prompt.txt
|
||||
echo "FILES_TO_EDIT: $FILES_TO_EDIT"
|
||||
|
||||
RULES=""
|
||||
if [ -n "$RULES_FILES" ]; then
|
||||
for rule in $RULES_FILES; do
|
||||
RULES="$RULES --read $rule"
|
||||
done
|
||||
fi
|
||||
|
||||
aider \
|
||||
$RULES \
|
||||
$FILES_TO_EDIT \
|
||||
--model gemini/gemini-2.5-pro-preview-05-06 \
|
||||
--message-file .aider_final_prompt.txt \
|
||||
--yes \
|
||||
--no-check-update \
|
||||
--auto-commits \
|
||||
--no-analytics \
|
||||
--no-gitignore \
|
||||
| tee .aider_output.txt || true
|
||||
|
||||
echo "Aider command completed. Output saved to .aider_output.txt"
|
||||
|
||||
- name: Cache Node.js dependencies
|
||||
uses: actions/cache@v3
|
||||
with:
|
||||
path: ~/.npm
|
||||
key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json', '**/yarn.lock') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-node-
|
||||
|
||||
- name: Commit and Push Changes
|
||||
id: commit_and_push
|
||||
env:
|
||||
ISSUE_ID: ${{ inputs.issue_id }}
|
||||
BRANCH_NAME: ${{ steps.prepare_branch.outputs.BRANCH_NAME }}
|
||||
run: |
|
||||
if [[ "$ISSUE_ID" != "" ]]; then
|
||||
# Check if there are any uncommitted changes
|
||||
if [[ -n $(git status --porcelain) ]]; then
|
||||
echo "Found uncommitted changes, committing them"
|
||||
git add .
|
||||
git commit -m "Aider changes"
|
||||
fi
|
||||
|
||||
# Push changes to the branch
|
||||
if git push origin $BRANCH_NAME; then
|
||||
echo "Pushed to branch $BRANCH_NAME"
|
||||
echo "PR_BRANCH_NAME=$BRANCH_NAME" >> $GITHUB_OUTPUT
|
||||
echo "CHANGES_APPLIED_MESSAGE=Aider changes pushed to branch $BRANCH_NAME." >> $GITHUB_OUTPUT
|
||||
echo "CHANGES_APPLIED=true" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "::warning::Push to PR branch $BRANCH_NAME failed."
|
||||
echo "CHANGES_APPLIED_MESSAGE=Aider ran, but failed to push changes to PR branch $BRANCH_NAME." >> $GITHUB_OUTPUT
|
||||
echo "CHANGES_APPLIED=false" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
else
|
||||
# We're in a pull_request_review event
|
||||
PR_HEAD_REF="${{ github.event.pull_request.head.ref }}"
|
||||
echo "Attempting to push changes to PR branch $PR_HEAD_REF"
|
||||
if git push origin $PR_HEAD_REF; then
|
||||
echo "Push to $PR_HEAD_REF successful (or no new changes to push)."
|
||||
echo "CHANGES_APPLIED_MESSAGE=Aider changes (if any) pushed to PR branch $PR_HEAD_REF." >> $GITHUB_OUTPUT
|
||||
echo "PR_BRANCH_NAME=$PR_HEAD_REF" >> $GITHUB_OUTPUT
|
||||
echo "CHANGES_APPLIED=true" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "::warning::Push to PR branch $PR_HEAD_REF failed."
|
||||
echo "CHANGES_APPLIED_MESSAGE=Aider ran, but failed to push changes to PR branch $PR_HEAD_REF." >> $GITHUB_OUTPUT
|
||||
echo "CHANGES_APPLIED=false" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
fi
|
||||
|
||||
- name: Create Pull Request
|
||||
if: always() && (github.event_name == 'issue_comment' || github.event_name == 'repository_dispatch') && !github.event.issue.pull_request && steps.commit_and_push.outputs.PR_BRANCH_NAME != ''
|
||||
id: create_pr
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
PR_BRANCH: ${{ steps.commit_and_push.outputs.PR_BRANCH_NAME }}
|
||||
ISSUE_NUM: ${{ inputs.issue_id }}
|
||||
ISSUE_TITLE: ${{ inputs.issue_title }}
|
||||
GITHUB_EVENT_NAME: ${{ github.event_name }}
|
||||
run: |
|
||||
# Create PR description in a temporary file to avoid command line length limits and ensure it stays under 40k chars
|
||||
HEADER="This PR was created automatically by Aider to fix issue #${ISSUE_NUM}."
|
||||
# if event is repository_dispatch, add the issue title to the header
|
||||
if [ "$GITHUB_EVENT_NAME" == "repository_dispatch" ]; then
|
||||
if [[ "${{ github.event.client_payload.source }}" == "linear" ]]; then
|
||||
HEADER="This PR was created automatically by Aider to fix issue #linear:${ISSUE_NUM}."
|
||||
elif [[ "${{ github.event.client_payload.source }}" == "discord" ]]; then
|
||||
HEADER="This PR was created automatically by Aider to fix issue #discord:${ISSUE_NUM}."
|
||||
fi
|
||||
fi
|
||||
cat > /tmp/pr-description.md << EOL | head -c 40000
|
||||
$HEADER
|
||||
|
||||
## Aider Output
|
||||
\`\`\`
|
||||
$(cat .aider_output.txt || echo "No output available")
|
||||
\`\`\`
|
||||
EOL
|
||||
|
||||
# Create PR using the file for the body content, handle errors gracefully
|
||||
set +e # Don't exit on error
|
||||
PR_TITLE="[Aider PR] Fix: ${ISSUE_TITLE}"
|
||||
if [ -z "$ISSUE_TITLE" ]; then
|
||||
PR_TITLE="[Aider PR] AI changes after request"
|
||||
fi
|
||||
gh pr create \
|
||||
--title "$PR_TITLE" \
|
||||
--body-file /tmp/pr-description.md \
|
||||
--head "$PR_BRANCH" \
|
||||
--base main \
|
||||
--draft
|
||||
PR_CREATE_EXIT_CODE=$?
|
||||
set -e # Re-enable exit on error
|
||||
|
||||
if [ $PR_CREATE_EXIT_CODE -eq 0 ]; then
|
||||
echo "PR created successfully"
|
||||
PR_URL=$(gh pr view $PR_BRANCH --json url --jq .url)
|
||||
echo "PR_URL=$PR_URL" >> $GITHUB_OUTPUT
|
||||
echo "PR_CREATED=true" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "Warning: Failed to create PR. Exit code: $PR_CREATE_EXIT_CODE"
|
||||
echo "PR_CREATED=false" >> $GITHUB_OUTPUT
|
||||
# Continue workflow despite PR creation failure
|
||||
fi
|
||||
|
||||
- name: Comment on PR with Aider Output
|
||||
if: always() && github.event_name == 'pull_request_review' && steps.commit_and_push.outputs.CHANGES_APPLIED != ''
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
PR_NUM: ${{ github.event.pull_request.number }}
|
||||
JOB_STATUS: ${{ job.status }}
|
||||
run: |
|
||||
# Create comment body in a temporary file to avoid command line length limits
|
||||
if [[ "${{ steps.commit_and_push.outputs.CHANGES_APPLIED }}" == "true" ]]; then
|
||||
if [[ "$JOB_STATUS" == "success" ]]; then
|
||||
STATUS_PREFIX="🤖 I've automatically addressed the feedback based on the review."
|
||||
else
|
||||
STATUS_PREFIX="⚠️ I attempted to address the feedback, but encountered some issues."
|
||||
fi
|
||||
else
|
||||
if [[ "$JOB_STATUS" == "success" ]]; then
|
||||
STATUS_PREFIX="🤖 I attempted to address the review feedback, but no modifications were made."
|
||||
else
|
||||
STATUS_PREFIX="⚠️ I encountered issues while attempting to address the feedback, and no modifications were made."
|
||||
fi
|
||||
fi
|
||||
|
||||
cat > /tmp/pr-comment.md << EOL
|
||||
${STATUS_PREFIX}
|
||||
|
||||
## Aider Output
|
||||
\`\`\`
|
||||
$(cat .aider_output.txt || echo 'No output available')
|
||||
\`\`\`
|
||||
|
||||
Please review the output and provide additional guidance if needed.
|
||||
EOL
|
||||
|
||||
# Use the file for comment body
|
||||
gh pr comment $PR_NUM --body-file /tmp/pr-comment.md
|
||||
|
||||
- name: Comment on issue/PR to let the user know Aider has finished working on the request
|
||||
if: always() && github.event_name == 'issue_comment'
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
JOB_STATUS: ${{ job.status }}
|
||||
PR_CREATED: ${{ steps.create_pr.outputs.PR_CREATED }}
|
||||
PR_URL: ${{ steps.create_pr.outputs.PR_URL }}
|
||||
run: |
|
||||
echo "Commenting on issue/PR #${{ github.event.issue.number }} to let the user know Aider has finished working on the request."
|
||||
|
||||
if [[ "$JOB_STATUS" == "success" ]]; then
|
||||
if [[ "$PR_CREATED" == "true" ]]; then
|
||||
COMMENT_BODY="🤖 Aider has finished working on your request. A PR has been created. $PR_URL"
|
||||
else
|
||||
COMMENT_BODY="🤖 Aider has finished working on your request, but was unable to create a PR."
|
||||
fi
|
||||
else
|
||||
COMMENT_BODY="⚠️ Aider encountered issues while working on your request. Please check the workflow logs for details."
|
||||
fi
|
||||
|
||||
gh issue comment ${{ github.event.issue.number }} --body "$COMMENT_BODY" --repo $GITHUB_REPOSITORY
|
||||
|
||||
- name: Comment on linear issue to let the user know Aider has finished working on the request
|
||||
if: always() && github.event_name == 'repository_dispatch'
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
JOB_STATUS: ${{ job.status }}
|
||||
LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}
|
||||
PR_CREATED: ${{ steps.create_pr.outputs.PR_CREATED }}
|
||||
PR_URL: ${{ steps.create_pr.outputs.PR_URL }}
|
||||
DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_AI_BOT_TOKEN }}
|
||||
SOURCE: ${{ github.event.client_payload.source }}
|
||||
run: |
|
||||
echo "Notifying user about Aider completion status for $SOURCE request #${{ github.event.client_payload.issue_id }}"
|
||||
if [[ "$JOB_STATUS" == "success" ]]; then
|
||||
if [[ "$PR_CREATED" == "true" ]]; then
|
||||
COMMENT_BODY="🤖 Aider has finished working on your request. A PR has been created. $PR_URL"
|
||||
else
|
||||
COMMENT_BODY="🤖 Aider has finished working on your request, but was unable to create a PR."
|
||||
fi
|
||||
else
|
||||
COMMENT_BODY="⚠️ Aider encountered issues while working on your request. Please check the workflow logs for details."
|
||||
fi
|
||||
|
||||
if [[ "$SOURCE" == "discord" ]]; then
|
||||
curl -X POST \
|
||||
-H "Authorization: Bot $DISCORD_BOT_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
"https://discord.com/api/v10/channels/${{ github.event.client_payload.channel_id }}/messages" \
|
||||
-d "{\"content\":\"${COMMENT_BODY}\"}"
|
||||
else
|
||||
curl -X POST \
|
||||
-H "Authorization: $LINEAR_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
"https://api.linear.app/graphql" \
|
||||
-d "{\"query\":\"mutation { commentCreate(input: { issueId: \\\"${{ github.event.client_payload.issue_id }}\\\", body: \\\"${COMMENT_BODY}\\\" }) { success } }\"}"
|
||||
fi
|
||||
80
.github/workflows/aider-external.yaml.archived
vendored
80
.github/workflows/aider-external.yaml.archived
vendored
@@ -1,80 +0,0 @@
|
||||
name: External Aider Issue Fix
|
||||
|
||||
on:
|
||||
repository_dispatch:
|
||||
types: [external_issue_fix]
|
||||
|
||||
jobs:
|
||||
check-and-prepare:
|
||||
runs-on: ubicloud-standard-2
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
outputs:
|
||||
issue_title: ${{ steps.determine_inputs.outputs.ISSUE_TITLE }}
|
||||
issue_body: ${{ steps.determine_inputs.outputs.ISSUE_BODY }}
|
||||
instruction: ${{ steps.determine_inputs.outputs.INSTRUCTION }}
|
||||
env:
|
||||
GEMINI_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
|
||||
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
WINDMILL_TOKEN: ${{ secrets.WINDMILL_TOKEN }}
|
||||
LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}
|
||||
DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_AI_BOT_TOKEN }}
|
||||
|
||||
steps:
|
||||
- name: Acknowledge Request
|
||||
env:
|
||||
LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}
|
||||
DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_AI_BOT_TOKEN }}
|
||||
run: |
|
||||
if [[ "${{ github.event.client_payload.source }}" == "linear" ]]; then
|
||||
echo "Commenting on Linear issue #${{ github.event.client_payload.issue_id }} to acknowledge the request."
|
||||
curl -X POST \
|
||||
-H "Authorization: $LINEAR_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
"https://api.linear.app/graphql" \
|
||||
-d "{\"query\":\"mutation { commentCreate(input: { issueId: \\\"${{ github.event.client_payload.issue_id }}\\\", body: \\\"🤖 Aider is starting to work on your request. I'll update you here once I have a PR ready. Please be patient, this might take a few minutes.\\\" }) { success } }\"}"
|
||||
elif [[ "${{ github.event.client_payload.source }}" == "discord" ]]; then
|
||||
echo "Commenting on Discord thread #${{ github.event.client_payload.channel_id }} to acknowledge the request."
|
||||
curl -X POST \
|
||||
-H "Authorization: Bot $DISCORD_BOT_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
"https://discord.com/api/v10/channels/${{ github.event.client_payload.channel_id }}/messages" \
|
||||
-d "{\"content\":\"🤖 Aider is starting to work on your request. I'll update you here once I have a PR ready. Please be patient, this might take a few minutes.\"}"
|
||||
fi
|
||||
|
||||
- name: Determine inputs for Aider
|
||||
id: determine_inputs
|
||||
shell: bash
|
||||
env:
|
||||
ISSUE_TITLE: ${{ github.event.client_payload.issue_title }}
|
||||
ISSUE_BODY: ${{ github.event.client_payload.issue_body }}
|
||||
INSTRUCTION: ${{ github.event.client_payload.instruction }}
|
||||
run: |
|
||||
echo "Determining inputs for Aider..."
|
||||
|
||||
echo "ISSUE_TITLE<<EOF_AIDER_TITLE" >> "$GITHUB_OUTPUT"
|
||||
echo "$ISSUE_TITLE" >> "$GITHUB_OUTPUT"
|
||||
echo "EOF_AIDER_TITLE" >> "$GITHUB_OUTPUT"
|
||||
|
||||
echo "ISSUE_BODY<<EOF_AIDER_BODY" >> "$GITHUB_OUTPUT"
|
||||
echo "$ISSUE_BODY" >> "$GITHUB_OUTPUT"
|
||||
echo "EOF_AIDER_BODY" >> "$GITHUB_OUTPUT"
|
||||
|
||||
echo "INSTRUCTION<<EOF_AIDER_INSTRUCTION" >> "$GITHUB_OUTPUT"
|
||||
echo "$INSTRUCTION" >> "$GITHUB_OUTPUT"
|
||||
echo "EOF_AIDER_INSTRUCTION" >> "$GITHUB_OUTPUT"
|
||||
echo "Finished determining inputs."
|
||||
|
||||
run-aider:
|
||||
needs: check-and-prepare
|
||||
uses: ./.github/workflows/aider-common.yml
|
||||
with:
|
||||
issue_title: ${{ needs.check-and-prepare.outputs.issue_title }}
|
||||
issue_body: ${{ needs.check-and-prepare.outputs.issue_body }}
|
||||
instruction: ${{ needs.check-and-prepare.outputs.instruction }}
|
||||
issue_id: ${{ github.event.client_payload.issue_id }}
|
||||
rules_files: ".cursor/rules/rust-best-practices.mdc .cursor/rules/svelte5-best-practices.mdc .cursor/rules/windmill-overview.mdc"
|
||||
secrets: inherit
|
||||
342
.github/workflows/aider.yaml
vendored
Normal file
342
.github/workflows/aider.yaml
vendored
Normal file
@@ -0,0 +1,342 @@
|
||||
name: Aider Auto-fix issues and PR comments via external prompt
|
||||
|
||||
on:
|
||||
issue_comment:
|
||||
types: [created]
|
||||
|
||||
jobs:
|
||||
auto-fix:
|
||||
runs-on: ubicloud-standard-8
|
||||
if: |
|
||||
github.event_name == 'issue_comment' &&
|
||||
contains(github.event.comment.body, '/aider') &&
|
||||
!contains(github.event.comment.user.login, '[bot]')
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
issues: write
|
||||
env:
|
||||
GEMINI_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
|
||||
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
WINDMILL_TOKEN: ${{ secrets.WINDMILL_TOKEN }}
|
||||
|
||||
steps:
|
||||
- name: Harden Runner
|
||||
uses: step-security/harden-runner@v2
|
||||
with:
|
||||
egress-policy: audit
|
||||
|
||||
- name: Check out code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Configure Git User
|
||||
run: |
|
||||
git config --global user.name "github-actions[bot]"
|
||||
git config --global user.email "github-actions[bot]@users.noreply.github.com"
|
||||
|
||||
- name: Checkout PR Branch
|
||||
if: github.event_name == 'issue_comment' && github.event.issue.pull_request
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
run: |
|
||||
echo "Issue comment trigger: Checking out PR branch..."
|
||||
PR_NUMBER=${{ github.event.issue.number }}
|
||||
PR_HEAD_REF=$(gh pr view $PR_NUMBER --json headRefName -q .headRefName --repo $GITHUB_REPOSITORY)
|
||||
if [[ -z "$PR_HEAD_REF" || "$PR_HEAD_REF" == "null" ]]; then
|
||||
echo "::error::Could not determine PR head branch for PR #$PR_NUMBER via gh CLI."
|
||||
exit 1
|
||||
fi
|
||||
echo "Checking out PR head branch: $PR_HEAD_REF for PR #$PR_NUMBER"
|
||||
git fetch origin "refs/heads/${PR_HEAD_REF}:refs/remotes/origin/${PR_HEAD_REF}" --no-tags
|
||||
git checkout "$PR_HEAD_REF"
|
||||
echo "Successfully checked out branch $(git rev-parse --abbrev-ref HEAD)"
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Install Aider and Dependencies
|
||||
run: |
|
||||
python -m pip install aider-install; aider-install
|
||||
pip install -U google-generativeai
|
||||
sudo apt-get update && sudo apt-get install -y jq
|
||||
|
||||
- name: Determine Prompt for Aider
|
||||
id: determine_prompt
|
||||
shell: bash
|
||||
run: |
|
||||
PROMPT_FILE_PATH=".github/aider/issue-prompt.txt"
|
||||
mkdir -p .github/aider
|
||||
|
||||
# Determine if this is a PR comment or regular issue comment
|
||||
if [[ ! -z "${{ github.event.issue.pull_request }}" ]]; then
|
||||
echo "This is a comment on a Pull Request"
|
||||
PR_NUMBER="${{ github.event.issue.number }}"
|
||||
|
||||
# Get PR description to check for issue references
|
||||
PR_BODY=$(gh pr view $PR_NUMBER --json body -q .body --repo $GITHUB_REPOSITORY)
|
||||
|
||||
# Extract issue number from PR description (looking for #123 or "fixes #123" patterns)
|
||||
REFERENCED_ISSUE=$(echo "$PR_BODY" | grep -oE "#[0-9]+" | grep -oE "[0-9]+" | head -1)
|
||||
|
||||
if [[ ! -z "$REFERENCED_ISSUE" ]]; then
|
||||
echo "Found referenced issue #$REFERENCED_ISSUE in PR description"
|
||||
|
||||
# Fetch the referenced issue details
|
||||
ISSUE_DETAILS=$(gh issue view $REFERENCED_ISSUE --json title,body --repo $GITHUB_REPOSITORY)
|
||||
ISSUE_TITLE=$(echo "$ISSUE_DETAILS" | jq -r .title)
|
||||
ISSUE_BODY=$(echo "$ISSUE_DETAILS" | jq -r .body)
|
||||
|
||||
# Store raw comment body in a file first to avoid shell interpretation issues
|
||||
echo '${{ github.event.comment.body }}' > /tmp/raw_comment.txt
|
||||
RAW_COMMENT_BODY=$(cat /tmp/raw_comment.txt)
|
||||
# Remove the /aider prefix and trim whitespace
|
||||
COMMENT_CONTENT=$(echo "$RAW_COMMENT_BODY" | sed 's|^/aider||' | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//')
|
||||
|
||||
echo "Sending issue content and PR comment to external API…"
|
||||
|
||||
ISSUE_TITLE_Q=$(printf '%q' "$ISSUE_TITLE")
|
||||
ISSUE_BODY_Q=$(printf '%q' "$ISSUE_BODY")
|
||||
|
||||
JSON_PAYLOAD=$(jq -n \
|
||||
--arg title "$ISSUE_TITLE_Q" \
|
||||
--arg body "$ISSUE_BODY_Q" \
|
||||
'{"body":{"issue_title":$title,"issue_body":$body}}')
|
||||
|
||||
API_RESULT=$(curl -s -w "\n%{http_code}" \
|
||||
-X POST "https://app.windmill.dev/api/w/windmill-labs/jobs/run_wait_result/p/f/ai/quiet_script" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer $WINDMILL_TOKEN" \
|
||||
--data-binary "$JSON_PAYLOAD" \
|
||||
--max-time 90)
|
||||
|
||||
HTTP_CODE=$(echo "$API_RESULT" | tail -n1)
|
||||
BODY=$(echo "$API_RESULT" | sed '$d')
|
||||
|
||||
echo "$BODY" > /tmp/api_response.txt
|
||||
|
||||
BASE_PROMPT="Try to fix the following issue based on the instruction given by the user. The issue is prepended with the word ISSUE. The instruction is prepended with the word INSTRUCTION. The issue and instruction are separated by a blank line."
|
||||
if [[ "$HTTP_CODE" -eq 200 ]]; then
|
||||
PROCESSED_ISSUE_PROMPT=$(jq -r '.effective_body // empty' /tmp/api_response.txt)
|
||||
if [[ -z "$PROCESSED_ISSUE_PROMPT" || "$PROCESSED_ISSUE_PROMPT" == "null" ]]; then
|
||||
PROCESSED_ISSUE_PROMPT=""
|
||||
fi
|
||||
printf "%s\nISSUE:\n%s\nINSTRUCTION:\n%s" \
|
||||
"$BASE_PROMPT" "$PROCESSED_ISSUE_PROMPT" "$COMMENT_CONTENT" > "$PROMPT_FILE_PATH"
|
||||
else
|
||||
echo "::warning::API call failed (HTTP $HTTP_CODE). Using PR comment with issue context."
|
||||
printf "%s\nISSUE:\n%s\nINSTRUCTION:\n%s" \
|
||||
"$BASE_PROMPT" "$ISSUE_BODY_Q" "$COMMENT_CONTENT" > "$PROMPT_FILE_PATH"
|
||||
fi
|
||||
rm -f /tmp/api_response.txt
|
||||
else
|
||||
echo "No referenced issue found in PR description, using comment content only"
|
||||
# Use comment content directly as with regular issue comments
|
||||
echo '${{ github.event.comment.body }}' > /tmp/raw_comment.txt
|
||||
RAW_COMMENT_BODY=$(cat /tmp/raw_comment.txt)
|
||||
COMMENT_CONTENT=$(echo "$RAW_COMMENT_BODY" | sed 's|^/aider||' | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//')
|
||||
|
||||
if [[ -z "$COMMENT_CONTENT" ]]; then
|
||||
echo "::error::Comment with /aider provided, but no instruction found after it. Cannot proceed."
|
||||
printf "Error: /aider command found but no instruction followed." > "$PROMPT_FILE_PATH"
|
||||
exit 1
|
||||
else
|
||||
echo "Using comment content as prompt."
|
||||
printf '%s' "$COMMENT_CONTENT" > "$PROMPT_FILE_PATH"
|
||||
fi
|
||||
fi
|
||||
else
|
||||
echo "This is a comment on a regular issue"
|
||||
|
||||
# Fetch the issue details
|
||||
ISSUE_NUMBER="${{ github.event.issue.number }}"
|
||||
ISSUE_DETAILS=$(gh issue view $ISSUE_NUMBER --json title,body --repo $GITHUB_REPOSITORY)
|
||||
ISSUE_TITLE=$(echo "$ISSUE_DETAILS" | jq -r .title)
|
||||
ISSUE_BODY=$(echo "$ISSUE_DETAILS" | jq -r .body)
|
||||
|
||||
# Store raw comment body in a file first to avoid shell interpretation issues
|
||||
echo '${{ github.event.comment.body }}' > /tmp/raw_comment.txt
|
||||
# Extract the command part safely
|
||||
RAW_COMMENT_BODY=$(cat /tmp/raw_comment.txt)
|
||||
# Remove the /aider prefix and trim whitespace
|
||||
COMMENT_CONTENT=$(echo "$RAW_COMMENT_BODY" | sed 's|^/aider||' | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//')
|
||||
|
||||
if [[ -z "$COMMENT_CONTENT" ]]; then
|
||||
echo "::error::Comment with /aider provided, but no instruction found after it. Cannot proceed."
|
||||
printf "Error: /aider command found but no instruction followed." > "$PROMPT_FILE_PATH"
|
||||
exit 1
|
||||
else
|
||||
echo "Sending issue content and issue comment to external API…"
|
||||
|
||||
ISSUE_TITLE_Q=$(printf '%q' "$ISSUE_TITLE")
|
||||
ISSUE_BODY_Q=$(printf '%q' "$ISSUE_BODY")
|
||||
COMMENT_CONTENT_Q=$(printf '%q' "$COMMENT_CONTENT")
|
||||
|
||||
JSON_PAYLOAD=$(jq -n \
|
||||
--arg title "$ISSUE_TITLE_Q" \
|
||||
--arg body "$ISSUE_BODY_Q" \
|
||||
--arg comment "$COMMENT_CONTENT_Q" \
|
||||
'{"body":{"issue_title":$title,"issue_body":$body,"issue_comment":$comment}}')
|
||||
|
||||
API_RESULT=$(curl -s -w "\n%{http_code}" \
|
||||
-X POST "https://app.windmill.dev/api/w/windmill-labs/jobs/run_wait_result/p/f/ai/quiet_script" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer $WINDMILL_TOKEN" \
|
||||
--data-binary "$JSON_PAYLOAD" \
|
||||
--max-time 90)
|
||||
|
||||
HTTP_CODE=$(echo "$API_RESULT" | tail -n1)
|
||||
BODY=$(echo "$API_RESULT" | sed '$d')
|
||||
|
||||
echo "$BODY" > /tmp/api_response.txt
|
||||
|
||||
BASE_PROMPT="Try to fix the following issue based on the instruction given by the user. The issue is prepended with the word ISSUE. The instruction is prepended with the word INSTRUCTION. The issue and instruction are separated by a blank line."
|
||||
if [[ "$HTTP_CODE" -eq 200 ]]; then
|
||||
PROCESSED_ISSUE_PROMPT=$(jq -r '.effective_body // empty' /tmp/api_response.txt)
|
||||
if [[ -z "$PROCESSED_ISSUE_PROMPT" || "$PROCESSED_ISSUE_PROMPT" == "null" ]]; then
|
||||
PROCESSED_ISSUE_PROMPT=""
|
||||
fi
|
||||
printf "%s\nISSUE:\n%s\nINSTRUCTION:\n%s" \
|
||||
"$BASE_PROMPT" "$PROCESSED_ISSUE_PROMPT" "$COMMENT_CONTENT" > "$PROMPT_FILE_PATH"
|
||||
else
|
||||
echo "::warning::API call failed (HTTP $HTTP_CODE). Using PR comment with issue context."
|
||||
printf "%s\nISSUE:\n%s\nINSTRUCTION:\n%s" \
|
||||
"$BASE_PROMPT" "$ISSUE_BODY_Q" "$COMMENT_CONTENT" > "$PROMPT_FILE_PATH"
|
||||
fi
|
||||
|
||||
rm -f /tmp/api_response.txt
|
||||
fi
|
||||
fi
|
||||
echo "Prompt determined and written to $PROMPT_FILE_PATH"
|
||||
echo "PROMPT_FILE_PATH=$PROMPT_FILE_PATH" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Probe Chat for Relevant Files
|
||||
id: probe_files
|
||||
env:
|
||||
PROMPT_CONTENT_FILE: ${{ steps.determine_prompt.outputs.PROMPT_FILE_PATH }}
|
||||
run: |
|
||||
echo "Running probe-chat to find relevant files..."
|
||||
if [[ ! -f "$PROMPT_CONTENT_FILE" ]]; then
|
||||
echo "::error::Prompt file $PROMPT_CONTENT_FILE not found!"
|
||||
exit 1
|
||||
fi
|
||||
PROMPT_CONTENT=$(cat "$PROMPT_CONTENT_FILE")
|
||||
if [ -z "$PROMPT_CONTENT" ]; then
|
||||
echo "::error::Prompt content is empty!"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
PROMPT_ESCAPED=$(jq -Rs . <<< "$PROMPT_CONTENT")
|
||||
|
||||
MESSAGE_FOR_PROBE=$(jq -n --arg prompt_escaped "$PROMPT_ESCAPED" \
|
||||
'{ "message": "I'\''m giving you a request that needs to be implemented. Your role is ONLY to give me the files that are relevant to the request and nothing else. The request is prepended with the word REQUEST.\\nREQUEST: \($prompt_escaped). Give me all the files relevant to this request. Your output MUST be a single json array that can be parsed with programatic json parsing, with the relevant files. Files can be rust or typescript or javascript files. DO NOT INCLUDE ANY OTHER TEXT IN YOUR OUTPUT. ONLY THE JSON ARRAY. Example of output: [\"file1.py\", \"file2.py\"]" }' | jq -r .message)
|
||||
|
||||
set -o pipefail
|
||||
PROBE_OUTPUT=$(npx --yes @buger/probe-chat@latest --max-iterations 50 --model-name gemini-2.5-pro-preview-05-06 --message "$MESSAGE_FOR_PROBE") || {
|
||||
echo "::error::probe-chat command failed. Output:"
|
||||
echo "$PROBE_OUTPUT"
|
||||
exit 1
|
||||
}
|
||||
set +o pipefail
|
||||
echo "Probe-chat raw output:"
|
||||
echo "$PROBE_OUTPUT"
|
||||
|
||||
JSON_FILES=$(echo "$PROBE_OUTPUT" | sed -n '/^\s*\[/,$p' | sed '/^\s*\]/q')
|
||||
echo "Extracted JSON block:"
|
||||
echo "$JSON_FILES"
|
||||
|
||||
FILES_LIST=$(echo "$JSON_FILES" | jq -e -r '[.[] | select(type == "string" and . != "" and . != null and (endswith("/") | not))] | map(@sh) | join(" ")' || echo "")
|
||||
|
||||
if [[ -z "$FILES_LIST" ]]; then
|
||||
echo "::warning::probe-chat did not identify any relevant files."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Formatted files list for aider: $FILES_LIST"
|
||||
echo "FILES_TO_EDIT=$FILES_LIST" >> $GITHUB_ENV
|
||||
|
||||
- name: Run Aider with external prompt
|
||||
run: |
|
||||
echo "Files identified by probe-chat: ${{ env.FILES_TO_EDIT }}"
|
||||
aider \
|
||||
--read .cursor/rules/rust-best-practices.mdc \
|
||||
--read .cursor/rules/svelte5-best-practices.mdc \
|
||||
${{ env.FILES_TO_EDIT }} \
|
||||
--model gemini/gemini-2.5-pro-preview-05-06 \
|
||||
--message-file .github/aider/issue-prompt.txt \
|
||||
--yes \
|
||||
--no-check-update \
|
||||
--auto-commits \
|
||||
--no-analytics \
|
||||
--no-gitignore \
|
||||
| tee .github/aider/aider-output.txt || true
|
||||
echo "Aider command completed. Output saved to .github/aider/aider-output.txt"
|
||||
|
||||
- name: Clean up prompt file
|
||||
if: always()
|
||||
run: rm -f .github/aider/issue-prompt.txt
|
||||
|
||||
- name: Commit and Push Changes
|
||||
id: commit_and_push
|
||||
if: ${{ success() }}
|
||||
run: |
|
||||
if [[ -z "${{ github.event.issue.pull_request }}" ]]; then
|
||||
BRANCH_NAME="aider-fix-issue-${{ github.event.issue.number }}"
|
||||
|
||||
# Check if branch exists remotely
|
||||
if git ls-remote --heads origin $BRANCH_NAME | grep -q $BRANCH_NAME; then
|
||||
echo "Branch $BRANCH_NAME already exists remotely, fetching it"
|
||||
git fetch origin $BRANCH_NAME
|
||||
git checkout $BRANCH_NAME
|
||||
git pull origin $BRANCH_NAME
|
||||
else
|
||||
echo "Creating new branch $BRANCH_NAME"
|
||||
git checkout -b $BRANCH_NAME
|
||||
fi
|
||||
|
||||
echo "Created/checked out branch $BRANCH_NAME for issue #${{ github.event.issue.number }}"
|
||||
git push origin $BRANCH_NAME
|
||||
echo "Pushed to branch $BRANCH_NAME"
|
||||
echo "PR_BRANCH_NAME=$BRANCH_NAME" >> $GITHUB_OUTPUT
|
||||
echo "CHANGES_APPLIED_MESSAGE=Aider changes pushed to branch $BRANCH_NAME." >> $GITHUB_OUTPUT
|
||||
else
|
||||
CURRENT_BRANCH_NAME=$(git rev-parse --abbrev-ref HEAD)
|
||||
echo "Attempting to push changes to PR branch $CURRENT_BRANCH_NAME for PR #${{ github.event.issue.number }}"
|
||||
if git push origin $CURRENT_BRANCH_NAME; then
|
||||
echo "Push to $CURRENT_BRANCH_NAME successful (or no new changes to push)."
|
||||
echo "CHANGES_APPLIED_MESSAGE=Aider changes (if any) pushed to PR branch $CURRENT_BRANCH_NAME." >> $GITHUB_OUTPUT
|
||||
echo "PR_BRANCH_NAME=$CURRENT_BRANCH_NAME" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "::warning::Push to PR branch $CURRENT_BRANCH_NAME failed."
|
||||
echo "CHANGES_APPLIED_MESSAGE=Aider ran, but failed to push changes to PR branch $CURRENT_BRANCH_NAME." >> $GITHUB_OUTPUT
|
||||
fi
|
||||
fi
|
||||
|
||||
- name: Create Pull Request
|
||||
if: success() && github.event_name == 'issue_comment' && !github.event.issue.pull_request && steps.commit_and_push.outputs.PR_BRANCH_NAME != ''
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
PR_BRANCH: ${{ steps.commit_and_push.outputs.PR_BRANCH_NAME }}
|
||||
ISSUE_NUM: ${{ github.event.issue.number }}
|
||||
run: |
|
||||
# Create PR description in a temporary file to avoid command line length limits
|
||||
cat > /tmp/pr-description.md << EOL
|
||||
This PR was created automatically by Aider to fix issue #${ISSUE_NUM}.
|
||||
|
||||
## Aider Output
|
||||
\`\`\`
|
||||
$(cat .github/aider/aider-output.txt || echo "No output available")
|
||||
\`\`\`
|
||||
EOL
|
||||
|
||||
# Create PR using the file for the body content
|
||||
gh pr create \
|
||||
--title "[Aider PR] Add fixes for issue #${ISSUE_NUM}" \
|
||||
--body-file /tmp/pr-description.md \
|
||||
--head "$PR_BRANCH" \
|
||||
--base main
|
||||
165
.github/workflows/aider.yaml.archived
vendored
165
.github/workflows/aider.yaml.archived
vendored
@@ -1,165 +0,0 @@
|
||||
name: Aider Auto-fix issues and PR comments via external prompt
|
||||
|
||||
on:
|
||||
issue_comment:
|
||||
types: [created]
|
||||
|
||||
jobs:
|
||||
check-membership:
|
||||
runs-on: ubicloud-standard-2
|
||||
if: |
|
||||
github.event_name == 'issue_comment' &&
|
||||
contains(github.event.comment.body, '/aider') &&
|
||||
!contains(github.event.comment.user.login, '[bot]')
|
||||
outputs:
|
||||
is_member: ${{ steps.check-membership.outputs.is_member }}
|
||||
steps:
|
||||
- name: Check organization membership
|
||||
id: check-membership
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
COMMENTER: ${{ github.event.comment.user.login }}
|
||||
ORG_ACCESS_TOKEN: ${{ secrets.ORG_ACCESS_TOKEN }}
|
||||
run: |
|
||||
ORG="windmill-labs"
|
||||
STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
|
||||
-H "Authorization: token $ORG_ACCESS_TOKEN" \
|
||||
-H "Accept: application/vnd.github+json" \
|
||||
-H "X-GitHub-Api-Version: 2022-11-28" \
|
||||
"https://api.github.com/orgs/$ORG/members/$COMMENTER")
|
||||
|
||||
if [ "$STATUS" -eq 204 ]; then
|
||||
echo "is_member=true" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "is_member=false" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
check-and-prepare:
|
||||
needs: check-membership
|
||||
runs-on: ubicloud-standard-2
|
||||
if: needs.check-membership.outputs.is_member == 'true'
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
issues: write
|
||||
env:
|
||||
GEMINI_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
|
||||
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
WINDMILL_TOKEN: ${{ secrets.WINDMILL_TOKEN }}
|
||||
outputs:
|
||||
issue_title: ${{ steps.determine_inputs.outputs.ISSUE_TITLE }}
|
||||
issue_body: ${{ steps.determine_inputs.outputs.ISSUE_BODY }}
|
||||
comment_content: ${{ steps.determine_inputs.outputs.COMMENT_CONTENT }}
|
||||
pr_branch: ${{ steps.checkout_pr.outputs.PR_BRANCH }}
|
||||
|
||||
steps:
|
||||
- name: Acknowledge Request
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
run: |
|
||||
echo "Commenting on issue/PR #${{ github.event.issue.number }} to acknowledge the /aider command."
|
||||
gh issue comment ${{ github.event.issue.number }} --body "🤖 Aider is starting to work on your request. I'll update you here once I have a PR ready. Please be patient, this might take a few minutes." --repo $GITHUB_REPOSITORY
|
||||
|
||||
- name: Determine inputs for Aider
|
||||
id: determine_inputs
|
||||
shell: bash
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
COMMENT_BODY: ${{ github.event.comment.body }}
|
||||
ISSUE_NUMBER: ${{ github.event.issue.number }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}
|
||||
run: |
|
||||
echo "Determining inputs for Aider..."
|
||||
ISSUE_TITLE_VAL=""
|
||||
ISSUE_BODY_VAL=""
|
||||
|
||||
if [[ ! -z "${{ github.event.issue.pull_request }}" ]]; then
|
||||
echo "This is a comment on a Pull Request"
|
||||
PR_NUMBER="$ISSUE_NUMBER"
|
||||
|
||||
PR_BODY_JSON=$(gh pr view "$PR_NUMBER" --json body --repo "$GITHUB_REPOSITORY")
|
||||
if [[ $? -ne 0 ]]; then
|
||||
echo "Error fetching PR body for PR #$PR_NUMBER"
|
||||
PR_BODY_VAL=""
|
||||
else
|
||||
PR_BODY_VAL=$(jq -r '.body // ""' <<< "$PR_BODY_JSON")
|
||||
fi
|
||||
|
||||
if [[ ! -z "$PR_BODY_VAL" ]]; then
|
||||
REFERENCED_ISSUE=""
|
||||
if [[ "$PR_BODY_VAL" =~ \#linear:([a-f0-9-]+) ]]; then
|
||||
REFERENCED_ISSUE="${BASH_REMATCH[1]}"
|
||||
echo "Found referenced Linear issue #$REFERENCED_ISSUE in PR description"
|
||||
LINEAR_ISSUE_JSON=$(curl -s -H "Authorization: $LINEAR_API_KEY" \
|
||||
"https://api.linear.app/graphql" \
|
||||
-X POST \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "{\"query\":\"query { issue(id: \\\"$REFERENCED_ISSUE\\\") { title description } }\"}")
|
||||
|
||||
if [[ $? -eq 0 && ! "$LINEAR_ISSUE_JSON" =~ "error" ]]; then
|
||||
ISSUE_TITLE_VAL=$(jq -r '.data.issue.title // ""' <<< "$LINEAR_ISSUE_JSON")
|
||||
ISSUE_BODY_VAL=$(jq -r '.data.issue.description // ""' <<< "$LINEAR_ISSUE_JSON")
|
||||
echo "Successfully fetched Linear issue details"
|
||||
else
|
||||
echo "Error fetching Linear issue details for #$REFERENCED_ISSUE"
|
||||
fi
|
||||
elif [[ "$PR_BODY_VAL" =~ \#([0-9]+) ]]; then
|
||||
REFERENCED_ISSUE="${BASH_REMATCH[1]}"
|
||||
echo "Found referenced GitHub issue #$REFERENCED_ISSUE in PR description"
|
||||
|
||||
ISSUE_DETAILS_JSON=$(gh issue view "$REFERENCED_ISSUE" --json title,body --repo "$GITHUB_REPOSITORY")
|
||||
if [[ $? -ne 0 ]]; then
|
||||
echo "Error fetching issue details for #$REFERENCED_ISSUE"
|
||||
else
|
||||
ISSUE_TITLE_VAL=$(jq -r '.title // ""' <<< "$ISSUE_DETAILS_JSON")
|
||||
ISSUE_BODY_VAL=$(jq -r '.body // ""' <<< "$ISSUE_DETAILS_JSON")
|
||||
fi
|
||||
fi
|
||||
else
|
||||
echo "PR body is empty or could not be fetched."
|
||||
fi
|
||||
else
|
||||
echo "This is a comment on a regular issue"
|
||||
|
||||
ISSUE_DETAILS_JSON=$(gh issue view "$ISSUE_NUMBER" --json title,body --repo "$GITHUB_REPOSITORY")
|
||||
if [[ $? -ne 0 ]]; then
|
||||
echo "Error fetching issue details for #$ISSUE_NUMBER"
|
||||
else
|
||||
ISSUE_TITLE_VAL=$(jq -r '.title // ""' <<< "$ISSUE_DETAILS_JSON")
|
||||
ISSUE_BODY_VAL=$(jq -r '.body // ""' <<< "$ISSUE_DETAILS_JSON")
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "ISSUE_TITLE<<EOF_AIDER_TITLE" >> "$GITHUB_OUTPUT"
|
||||
echo "$ISSUE_TITLE_VAL" >> "$GITHUB_OUTPUT"
|
||||
echo "EOF_AIDER_TITLE" >> "$GITHUB_OUTPUT"
|
||||
|
||||
echo "ISSUE_BODY<<EOF_AIDER_BODY" >> "$GITHUB_OUTPUT"
|
||||
echo "$ISSUE_BODY_VAL" >> "$GITHUB_OUTPUT"
|
||||
echo "EOF_AIDER_BODY" >> "$GITHUB_OUTPUT"
|
||||
|
||||
CLEAN_COMMENT="${COMMENT_BODY/\/aider/}"
|
||||
CLEAN_COMMENT="${CLEAN_COMMENT#"${CLEAN_COMMENT%%[![:space:]]*}"}"
|
||||
CLEAN_COMMENT="${CLEAN_COMMENT%"${CLEAN_COMMENT##*[![:space:]]}"}"
|
||||
|
||||
echo "COMMENT_CONTENT<<EOF_AIDER_COMMENT" >> "$GITHUB_OUTPUT"
|
||||
echo "$CLEAN_COMMENT" >> "$GITHUB_OUTPUT"
|
||||
echo "EOF_AIDER_COMMENT" >> "$GITHUB_OUTPUT"
|
||||
echo "Finished determining inputs."
|
||||
|
||||
run-aider:
|
||||
needs: [check-membership, check-and-prepare]
|
||||
if: needs.check-membership.outputs.is_member == 'true'
|
||||
uses: ./.github/workflows/aider-common.yml
|
||||
with:
|
||||
issue_title: ${{ needs.check-and-prepare.outputs.issue_title }}
|
||||
issue_body: ${{ needs.check-and-prepare.outputs.issue_body }}
|
||||
instruction: ${{ needs.check-and-prepare.outputs.comment_content }}
|
||||
issue_id: ${{ github.event.issue.number }}
|
||||
rules_files: ".cursor/rules/rust-best-practices.mdc .cursor/rules/svelte5-best-practices.mdc .cursor/rules/windmill-overview.mdc"
|
||||
secrets: inherit
|
||||
44
.github/workflows/backend-check.yml
vendored
44
.github/workflows/backend-check.yml
vendored
@@ -17,13 +17,13 @@ jobs:
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Install mold and clang
|
||||
run: sudo apt-get update && sudo apt-get install -y mold clang
|
||||
|
||||
- uses: actions-rust-lang/setup-rust-toolchain@v1
|
||||
with:
|
||||
cache: false
|
||||
toolchain: 1.93.0
|
||||
cache-workspaces: backend
|
||||
toolchain: 1.85.0
|
||||
- uses: Swatinem/rust-cache@v2
|
||||
with:
|
||||
workspaces: backend
|
||||
- name: cargo check
|
||||
working-directory: ./backend
|
||||
timeout-minutes: 16
|
||||
@@ -36,21 +36,24 @@ jobs:
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: install xmlsec1 and gssapi
|
||||
- name: install xmlsec1
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y libxml2-dev libxmlsec1-dev libkrb5-dev libsasl2-dev libcurl4-openssl-dev mold clang
|
||||
sudo apt-get install -y libxml2-dev libxmlsec1-dev
|
||||
|
||||
- uses: actions-rust-lang/setup-rust-toolchain@v1
|
||||
with:
|
||||
cache: false
|
||||
toolchain: 1.93.0
|
||||
cache-workspaces: backend
|
||||
toolchain: 1.85.0
|
||||
- uses: Swatinem/rust-cache@v2
|
||||
with:
|
||||
workspaces: backend
|
||||
- name: cargo check
|
||||
working-directory: ./backend
|
||||
timeout-minutes: 16
|
||||
run: |
|
||||
mkdir -p fake_frontend_build
|
||||
FRONTEND_BUILD_DIR=$(pwd)/fake_frontend_build SQLX_OFFLINE=true cargo check --features all_sqlx_features
|
||||
FRONTEND_BUILD_DIR=$(pwd)/fake_frontend_build SQLX_OFFLINE=true cargo check --all-features
|
||||
|
||||
check_ee:
|
||||
runs-on: ubicloud-standard-8
|
||||
@@ -75,13 +78,13 @@ jobs:
|
||||
run: |
|
||||
./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
|
||||
|
||||
- name: Install mold and clang
|
||||
run: sudo apt-get update && sudo apt-get install -y mold clang
|
||||
|
||||
- uses: actions-rust-lang/setup-rust-toolchain@v1
|
||||
with:
|
||||
cache: false
|
||||
toolchain: 1.93.0
|
||||
cache-workspaces: backend
|
||||
toolchain: 1.85.0
|
||||
- uses: Swatinem/rust-cache@v2
|
||||
with:
|
||||
workspaces: backend
|
||||
- name: cargo check
|
||||
working-directory: ./backend
|
||||
timeout-minutes: 16
|
||||
@@ -106,10 +109,10 @@ jobs:
|
||||
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
fetch-depth: 0
|
||||
|
||||
- name: install xmlsec1 and gssapi
|
||||
- name: install xmlsec1
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y libxml2-dev libxmlsec1-dev libkrb5-dev libsasl2-dev libcurl4-openssl-dev mold clang
|
||||
sudo apt-get install -y libxml2-dev libxmlsec1-dev
|
||||
|
||||
- name: Substitute EE code (EE logic is behind feature flag)
|
||||
run: |
|
||||
@@ -118,10 +121,13 @@ jobs:
|
||||
- uses: actions-rust-lang/setup-rust-toolchain@v1
|
||||
with:
|
||||
cache-workspaces: backend
|
||||
toolchain: 1.93.0
|
||||
toolchain: 1.85.0
|
||||
- uses: Swatinem/rust-cache@v2
|
||||
with:
|
||||
workspaces: backend
|
||||
- name: cargo check
|
||||
timeout-minutes: 16
|
||||
working-directory: ./backend
|
||||
run: |
|
||||
mkdir -p fake_frontend_build
|
||||
FRONTEND_BUILD_DIR=$(pwd)/fake_frontend_build SQLX_OFFLINE=true cargo check --features all_sqlx_features,private
|
||||
FRONTEND_BUILD_DIR=$(pwd)/fake_frontend_build SQLX_OFFLINE=true cargo check --all-features
|
||||
|
||||
201
.github/workflows/backend-test.yml
vendored
201
.github/workflows/backend-test.yml
vendored
@@ -19,7 +19,7 @@ defaults:
|
||||
|
||||
jobs:
|
||||
cargo_test:
|
||||
runs-on: blacksmith-32vcpu-ubuntu-2404
|
||||
runs-on: ubicloud-standard-8
|
||||
services:
|
||||
postgres:
|
||||
image: postgres
|
||||
@@ -28,20 +28,9 @@ jobs:
|
||||
env:
|
||||
POSTGRES_DB: windmill
|
||||
POSTGRES_PASSWORD: changeme
|
||||
POSTGRES_INITDB_ARGS: "-c max_connections=500"
|
||||
options: >-
|
||||
--health-cmd pg_isready --health-interval 10s --health-timeout 5s
|
||||
--health-retries 5 --shm-size=256mb
|
||||
mysql:
|
||||
image: mysql:8.0
|
||||
ports:
|
||||
- 3306:3306
|
||||
env:
|
||||
MYSQL_ROOT_PASSWORD: changeme
|
||||
MYSQL_DATABASE: windmill_test
|
||||
options: >-
|
||||
--health-cmd "mysqladmin ping -h localhost" --health-interval 10s
|
||||
--health-timeout 5s --health-retries 5
|
||||
--health-retries 5
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-dotnet@v4
|
||||
@@ -50,178 +39,30 @@ jobs:
|
||||
- uses: denoland/setup-deno@v2
|
||||
with:
|
||||
deno-version: v2.x
|
||||
- uses: actions/setup-go@v6
|
||||
- uses: actions/setup-go@v2
|
||||
with:
|
||||
go-version: 1.25.0
|
||||
go-version: 1.21.5
|
||||
- uses: oven-sh/setup-bun@v2
|
||||
with:
|
||||
bun-version: 1.3.8
|
||||
- uses: actions/setup-node@v4
|
||||
bun-version: 1.1.43
|
||||
- uses: astral-sh/setup-uv@v4
|
||||
with:
|
||||
node-version: "20"
|
||||
- uses: astral-sh/setup-uv@v6.2.1
|
||||
with:
|
||||
version: "0.9.24"
|
||||
- uses: shivammathur/setup-php@v2
|
||||
with:
|
||||
php-version: "8.3"
|
||||
tools: composer
|
||||
- uses: ruby/setup-ruby@v1
|
||||
with:
|
||||
ruby-version: "3.3"
|
||||
bundler-cache: false
|
||||
- name: Install PowerShell, mold and clang
|
||||
run: |
|
||||
dotnet tool install --global PowerShell
|
||||
echo "$HOME/.dotnet/tools" >> $GITHUB_PATH
|
||||
sudo apt-get update && sudo apt-get install -y mold clang libcurl4-openssl-dev
|
||||
working-directory: /
|
||||
version: "0.4.18"
|
||||
- uses: actions-rust-lang/setup-rust-toolchain@v1
|
||||
with:
|
||||
cache: true
|
||||
toolchain: 1.93.0
|
||||
- name: Read EE repo commit hash
|
||||
run: |
|
||||
echo "ee_repo_ref=$(cat ./ee-repo-ref.txt)" >> "$GITHUB_ENV"
|
||||
|
||||
- uses: actions/checkout@v4
|
||||
cache-workspaces: backend
|
||||
toolchain: 1.85.0
|
||||
- uses: Swatinem/rust-cache@v2
|
||||
with:
|
||||
repository: windmill-labs/windmill-ee-private
|
||||
path: ./windmill-ee-private
|
||||
ref: ${{ env.ee_repo_ref }}
|
||||
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Substitute EE code (EE logic is behind feature flag)
|
||||
run: |
|
||||
./substitute_ee_code.sh --copy --dir ./windmill-ee-private
|
||||
- name: Setup private npm registry with test package
|
||||
working-directory: /tmp
|
||||
run: |
|
||||
set -e
|
||||
|
||||
# Install Verdaccio globally
|
||||
npm install -g verdaccio
|
||||
|
||||
# Create Verdaccio config that requires authentication for @windmill-test packages
|
||||
mkdir -p /tmp/verdaccio/storage
|
||||
cat > /tmp/verdaccio/config.yaml << 'VERDACCIO_CONFIG'
|
||||
storage: /tmp/verdaccio/storage
|
||||
auth:
|
||||
htpasswd:
|
||||
file: /tmp/verdaccio/htpasswd
|
||||
max_users: 100
|
||||
uplinks:
|
||||
npmjs:
|
||||
url: https://registry.npmjs.org/
|
||||
packages:
|
||||
'@windmill-test/*':
|
||||
access: $authenticated
|
||||
publish: $authenticated
|
||||
'@*/*':
|
||||
access: $all
|
||||
publish: $authenticated
|
||||
proxy: npmjs
|
||||
'**':
|
||||
access: $all
|
||||
publish: $authenticated
|
||||
proxy: npmjs
|
||||
server:
|
||||
keepAliveTimeout: 60
|
||||
middlewares:
|
||||
audit:
|
||||
enabled: true
|
||||
log: { type: stdout, format: pretty, level: warn }
|
||||
VERDACCIO_CONFIG
|
||||
|
||||
# Create empty htpasswd file (users will be created via API)
|
||||
touch /tmp/verdaccio/htpasswd
|
||||
|
||||
# Start Verdaccio in background
|
||||
verdaccio --config /tmp/verdaccio/config.yaml &
|
||||
VERDACCIO_PID=$!
|
||||
|
||||
# Wait for Verdaccio to be ready
|
||||
echo "Waiting for Verdaccio to start..."
|
||||
for i in {1..30}; do
|
||||
if curl -s http://localhost:4873/-/ping > /dev/null 2>&1; then
|
||||
echo "Verdaccio is ready"
|
||||
break
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
|
||||
# Login to get a token
|
||||
echo "Getting auth token..."
|
||||
RESPONSE=$(curl -s -X PUT \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"name":"testuser","password":"testpass123"}' \
|
||||
http://localhost:4873/-/user/org.couchdb.user:testuser)
|
||||
|
||||
echo "Auth response: $RESPONSE"
|
||||
NPM_TOKEN=$(echo "$RESPONSE" | jq -r '.token')
|
||||
|
||||
if [ -z "$NPM_TOKEN" ] || [ "$NPM_TOKEN" = "null" ]; then
|
||||
echo "Failed to get NPM token from response"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "NPM_TOKEN=${NPM_TOKEN}" >> $GITHUB_ENV
|
||||
echo "Got NPM token successfully: ${NPM_TOKEN:0:10}..."
|
||||
|
||||
# Configure npm globally with the auth token
|
||||
echo "//localhost:4873/:_authToken=${NPM_TOKEN}" > ~/.npmrc
|
||||
echo "Configured ~/.npmrc with auth token"
|
||||
|
||||
# Create a simple test package
|
||||
mkdir -p /tmp/windmill-test-private-pkg
|
||||
cat > /tmp/windmill-test-private-pkg/package.json << 'PKG_JSON'
|
||||
{
|
||||
"name": "@windmill-test/private-pkg",
|
||||
"version": "1.0.0",
|
||||
"main": "index.js"
|
||||
}
|
||||
PKG_JSON
|
||||
cat > /tmp/windmill-test-private-pkg/index.js << 'PKG_JS'
|
||||
module.exports.greet = (name) => `Hello from private package, ${name}!`;
|
||||
PKG_JS
|
||||
|
||||
# Publish to Verdaccio with auth
|
||||
cd /tmp/windmill-test-private-pkg
|
||||
echo "Publishing package..."
|
||||
npm publish --registry http://localhost:4873
|
||||
echo "Package published successfully"
|
||||
|
||||
# Verify the package requires auth by trying anonymous access (should fail)
|
||||
rm -f ~/.npmrc
|
||||
echo "Testing anonymous access (should fail)..."
|
||||
if npm view @windmill-test/private-pkg --registry http://localhost:4873 2>/dev/null; then
|
||||
echo "ERROR: Package should require authentication but anonymous access worked"
|
||||
exit 1
|
||||
fi
|
||||
echo "Verified: Package requires authentication for @windmill-test/private-pkg"
|
||||
- name: Cache DuckDB FFI module build
|
||||
uses: actions/cache@v3
|
||||
with:
|
||||
path: ./backend/windmill-duckdb-ffi-internal/target
|
||||
key: ${{ runner.os }}-duckdb-ffi-${{ hashFiles('./backend/windmill-duckdb-ffi-internal/src/**/*.rs', './backend/windmill-duckdb-ffi-internal/Cargo.toml', './backend/windmill-duckdb-ffi-internal/Cargo.lock') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-duckdb-ffi-
|
||||
workspaces: backend
|
||||
- name: cargo test
|
||||
timeout-minutes: 30
|
||||
env:
|
||||
SQLX_OFFLINE: true
|
||||
DATABASE_URL: postgres://postgres:changeme@localhost:5432/windmill
|
||||
DISABLE_EMBEDDING: true
|
||||
RUST_LOG: "off"
|
||||
RUST_LOG_STYLE: never
|
||||
CARGO_NET_GIT_FETCH_WITH_CLI: true
|
||||
CARGO_BUILD_JOBS: 16
|
||||
WMDEBUG_FORCE_V0_WORKSPACE_DEPENDENCIES: 1
|
||||
WMDEBUG_FORCE_RUNNABLE_SETTINGS_V0: 1
|
||||
WMDEBUG_FORCE_NO_LEGACY_DEBOUNCING_COMPAT: 1
|
||||
TEST_NPM_REGISTRY: "http://localhost:4873/:_authToken=${{ env.NPM_TOKEN }}"
|
||||
run: |
|
||||
deno --version && bun -v && node --version && go version && python3 --version && php --version && ruby --version && pwsh --version && dotnet --version
|
||||
cd windmill-duckdb-ffi-internal && ./build_dev.sh && cd ..
|
||||
DENO_PATH=$(which deno) BUN_PATH=$(which bun) NODE_BIN_PATH=$(which node) GO_PATH=$(which go) UV_PATH=$(which uv) PHP_PATH=$(which php) COMPOSER_PATH=$(which composer) RUBY_PATH=$(which ruby) RUBY_BUNDLE_PATH=$(which bundle) RUBY_GEM_PATH=$(which gem) POWERSHELL_PATH=$(which pwsh) DOTNET_PATH=$(which dotnet) cargo test --features enterprise,deno_core,duckdb,license,python,rust,scoped_cache,parquet,private,private_registry_test,csharp,php,ruby,mysql,quickjs,mcp --all -- --nocapture --test-threads=12
|
||||
timeout-minutes: 16
|
||||
run:
|
||||
deno --version && bun -v && go version && python3 --version &&
|
||||
SQLX_OFFLINE=true
|
||||
DATABASE_URL=postgres://postgres:changeme@localhost:5432/windmill
|
||||
DISABLE_EMBEDDING=true RUST_LOG=info
|
||||
DENO_PATH=$(which deno) BUN_PATH=$(which bun) GO_PATH=$(which go)
|
||||
UV_PATH=$(which uv) cargo test --features
|
||||
enterprise,deno_core,license,python,rust,scoped_cache --all --
|
||||
--nocapture
|
||||
|
||||
65
.github/workflows/build-extra-image.yml
vendored
65
.github/workflows/build-extra-image.yml
vendored
@@ -1,65 +0,0 @@
|
||||
env:
|
||||
REGISTRY: ghcr.io
|
||||
IMAGE_NAME: ${{ github.repository }}
|
||||
|
||||
name: Build windmill-extra
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
tag:
|
||||
description: "Tag for the image"
|
||||
required: false
|
||||
default: "dev"
|
||||
type: string
|
||||
|
||||
permissions: write-all
|
||||
|
||||
jobs:
|
||||
sleep:
|
||||
runs-on: ubicloud
|
||||
steps:
|
||||
- name: Sleep for 900 seconds waiting for pypi to update index
|
||||
if: startsWith(github.ref, 'refs/tags/v')
|
||||
run: sleep 900
|
||||
shell: bash
|
||||
build_extra:
|
||||
runs-on: ubicloud
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
ref: ${{ github.ref }}
|
||||
fetch-depth: 0
|
||||
|
||||
- uses: depot/setup-action@v1
|
||||
|
||||
- name: Docker meta
|
||||
id: meta
|
||||
uses: docker/metadata-action@v5
|
||||
with:
|
||||
images: |
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-extra
|
||||
flavor: |
|
||||
latest=false
|
||||
tags: |
|
||||
type=raw,value=${{ github.event.inputs.tag }}
|
||||
type=sha,enable=true,priority=100,prefix=,suffix=,format=short
|
||||
|
||||
- name: Login to registry
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
registry: ${{ env.REGISTRY }}
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Build and push
|
||||
uses: depot/build-push-action@v1
|
||||
with:
|
||||
context: .
|
||||
platforms: linux/amd64,linux/arm64
|
||||
push: true
|
||||
file: "./docker/DockerfileExtra"
|
||||
tags: |
|
||||
${{ steps.meta.outputs.tags }}
|
||||
labels: |
|
||||
${{ steps.meta.outputs.labels }}
|
||||
94
.github/workflows/build-publish-rh-image.yml
vendored
94
.github/workflows/build-publish-rh-image.yml
vendored
@@ -27,6 +27,8 @@ jobs:
|
||||
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
fetch-depth: 0
|
||||
|
||||
# - name: Set up Docker Buildx
|
||||
# uses: docker/setup-buildx-action@v2
|
||||
- uses: depot/setup-action@v1
|
||||
|
||||
- name: Docker meta
|
||||
@@ -55,62 +57,74 @@ jobs:
|
||||
run: |
|
||||
cp ./docker/RHEL9/Dockerfile ./Dockerfile
|
||||
|
||||
- name: Build and push EE (multi-arch)
|
||||
- name: Build and push publicly ee amd64
|
||||
uses: depot/build-push-action@v1
|
||||
with:
|
||||
context: .
|
||||
platforms: linux/amd64,linux/arm64
|
||||
platforms: linux/amd64
|
||||
push: true
|
||||
build-args: |
|
||||
features=ee_rhel
|
||||
features=enterprise,enterprise_saml,stripe,embedding,parquet,prometheus,openidconnect,cloud,jemalloc,license,otel,http_trigger,zip,oauth2,kafka,sqs_trigger,nats,postgres_trigger,gcp_trigger,mqtt_trigger,websocket,smtp,static_frontend,all_languages,deno_core,mcp
|
||||
secrets: |
|
||||
rh_username=${{ secrets.RH_USERNAME }}
|
||||
rh_password=${{ secrets.RH_PASSWORD }}
|
||||
tags: |
|
||||
${{ steps.meta-ee-public.outputs.tags }}
|
||||
${{ steps.meta-ee-public.outputs.tags }}-amd64
|
||||
labels: |
|
||||
${{ steps.meta-ee-public.outputs.labels }}
|
||||
${{ steps.meta-ee-public.outputs.labels }}-amd64
|
||||
org.opencontainers.image.licenses=Windmill-Enterprise-License
|
||||
|
||||
- name: Install crane
|
||||
uses: imjasonh/setup-crane@v0.4
|
||||
|
||||
- name: Extract binaries with crane
|
||||
run: |
|
||||
mkdir -p extracted
|
||||
|
||||
# Extract arm64 binary (include deps/ for hard link resolution)
|
||||
mkdir -p /tmp/arm64
|
||||
crane export --platform linux/arm64 ${{ steps.meta-ee-public.outputs.tags }} - \
|
||||
| tar -xf - -C /tmp/arm64 windmill/target/release/ usr/src/app/libwindmill_duckdb_ffi_internal.so
|
||||
cp /tmp/arm64/windmill/target/release/windmill extracted/windmill-ee-arm64-rhel9
|
||||
cp /tmp/arm64/usr/src/app/libwindmill_duckdb_ffi_internal.so extracted/libwindmill_duckdb_ffi_internal-arm64.so
|
||||
rm -rf /tmp/arm64
|
||||
|
||||
# Extract amd64 binary
|
||||
mkdir -p /tmp/amd64
|
||||
crane export --platform linux/amd64 ${{ steps.meta-ee-public.outputs.tags }} - \
|
||||
| tar -xf - -C /tmp/amd64 windmill/target/release/ usr/src/app/libwindmill_duckdb_ffi_internal.so
|
||||
cp /tmp/amd64/windmill/target/release/windmill extracted/windmill-ee-amd64-rhel9
|
||||
cp /tmp/amd64/usr/src/app/libwindmill_duckdb_ffi_internal.so extracted/libwindmill_duckdb_ffi_internal-amd64.so
|
||||
rm -rf /tmp/amd64
|
||||
|
||||
- uses: actions/upload-artifact@v4
|
||||
- name: Build and push publicly ee arm64
|
||||
uses: depot/build-push-action@v1
|
||||
with:
|
||||
name: RHEL9-arm64 build
|
||||
path: extracted/windmill-ee-arm64-rhel9
|
||||
context: .
|
||||
platforms: linux/arm64
|
||||
push: true
|
||||
build-args: |
|
||||
features=enterprise,enterprise_saml,stripe,embedding,parquet,prometheus,openidconnect,cloud,jemalloc,license,otel,http_trigger,zip,oauth2,kafka,sqs_trigger,nats,postgres_trigger,gcp_trigger,mqtt_trigger,websocket,smtp,static_frontend,all_languages,deno_core,mcp
|
||||
secrets: |
|
||||
rh_username=${{ secrets.RH_USERNAME }}
|
||||
rh_password=${{ secrets.RH_PASSWORD }}
|
||||
tags: |
|
||||
${{ steps.meta-ee-public.outputs.tags }}-arm64
|
||||
labels: |
|
||||
${{ steps.meta-ee-public.outputs.labels }}-arm64
|
||||
org.opencontainers.image.licenses=Windmill-Enterprise-License
|
||||
|
||||
- uses: shrink/actions-docker-extract@v3
|
||||
id: extract-ee-amd64
|
||||
with:
|
||||
image: ${{ steps.meta-ee-public.outputs.tags}}-amd64
|
||||
path: "/windmill/target/release/windmill"
|
||||
|
||||
# - uses: shrink/actions-docker-extract@v3
|
||||
# id: extract-ee-arm64
|
||||
# with:
|
||||
# image: ${{ steps.meta-ee-public.outputs.tags}}-arm64
|
||||
# path: "/windmill/target/release/windmill"
|
||||
|
||||
- name: Rename binary with corresponding architecture
|
||||
run: |
|
||||
mv "${{ steps.extract-ee-amd64.outputs.destination }}/windmill" "${{ steps.extract-ee-amd64.outputs.destination }}/windmill-ee-amd64-rhel9"
|
||||
# mv "${{ steps.extract-ee-arm64.outputs.destination }}/windmill" "${{ steps.extract-ee-arm64.outputs.destination }}/windmill-ee-arm64-rhel9"
|
||||
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: RHEL9-amd64 build
|
||||
path: extracted/windmill-ee-amd64-rhel9
|
||||
path: ${{ steps.extract-ee-amd64.outputs.destination
|
||||
}}/windmill-ee-amd64-rhel9
|
||||
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: RHEL9-arm64 dynamic libraries build
|
||||
path: extracted/libwindmill_duckdb_ffi_internal-arm64.so
|
||||
# - uses: actions/upload-artifact@v4
|
||||
# with:
|
||||
# name: RHEL9-arm64 build
|
||||
# path:
|
||||
# ${{ steps.extract-ee-arm64.outputs.destination
|
||||
# }}/windmill-ee-arm64-rhel9
|
||||
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: RHEL9-amd64 dynamic libraries build
|
||||
path: extracted/libwindmill_duckdb_ffi_internal-amd64.so
|
||||
# - name: Attach binary to release
|
||||
# uses: softprops/action-gh-release@v2
|
||||
# if: startsWith(github.ref, 'refs/tags/')
|
||||
# with:
|
||||
# files: |
|
||||
# ${{ steps.extract-ee-arm64.outputs.destination }}/windmill-ee-arm64-rhel9
|
||||
# ${{ steps.extract-ee-amd64.outputs.destination }}/windmill-ee-amd64-rhel9
|
||||
|
||||
140
.github/workflows/build-publish-rh8-image.yml
vendored
140
.github/workflows/build-publish-rh8-image.yml
vendored
@@ -1,140 +0,0 @@
|
||||
env:
|
||||
REGISTRY: ghcr.io
|
||||
IMAGE_NAME: ${{ github.repository }}
|
||||
|
||||
name: Build and publish windmill for RHEL8
|
||||
on: workflow_dispatch
|
||||
|
||||
permissions: write-all
|
||||
|
||||
jobs:
|
||||
build_ee:
|
||||
runs-on: ubicloud
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Read EE repo commit hash
|
||||
run: |
|
||||
echo "ee_repo_ref=$(cat ./backend/ee-repo-ref.txt)" >> "$GITHUB_ENV"
|
||||
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
repository: windmill-labs/windmill-ee-private
|
||||
path: ./windmill-ee-private
|
||||
ref: ${{ env.ee_repo_ref }}
|
||||
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
fetch-depth: 0
|
||||
|
||||
# - name: Set up Docker Buildx
|
||||
# uses: docker/setup-buildx-action@v2
|
||||
- uses: depot/setup-action@v1
|
||||
|
||||
- name: Docker meta
|
||||
id: meta-ee-public
|
||||
uses: docker/metadata-action@v5
|
||||
with:
|
||||
images: |
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee-rhel8
|
||||
flavor: |
|
||||
latest=false
|
||||
tags: |
|
||||
type=sha
|
||||
|
||||
- name: Login to registry
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
registry: ${{ env.REGISTRY }}
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Substitute EE code
|
||||
run: |
|
||||
./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
|
||||
|
||||
- name: Copy RHEL8 Dockerfile
|
||||
run: |
|
||||
cp ./docker/RHEL8/Dockerfile ./Dockerfile
|
||||
|
||||
- name: Build and push publicly ee amd64
|
||||
uses: depot/build-push-action@v1
|
||||
with:
|
||||
context: .
|
||||
platforms: linux/amd64
|
||||
push: true
|
||||
build-args: |
|
||||
features=ee_rhel
|
||||
secrets: |
|
||||
rh_username=${{ secrets.RH_USERNAME }}
|
||||
rh_password=${{ secrets.RH_PASSWORD }}
|
||||
tags: |
|
||||
${{ steps.meta-ee-public.outputs.tags }}-amd64
|
||||
labels: |
|
||||
${{ steps.meta-ee-public.outputs.labels }}-amd64
|
||||
org.opencontainers.image.licenses=Windmill-Enterprise-License
|
||||
|
||||
- name: Build and push publicly ee arm64
|
||||
uses: depot/build-push-action@v1
|
||||
with:
|
||||
context: .
|
||||
platforms: linux/arm64
|
||||
push: true
|
||||
build-args: |
|
||||
features=ee_rhel
|
||||
secrets: |
|
||||
rh_username=${{ secrets.RH_USERNAME }}
|
||||
rh_password=${{ secrets.RH_PASSWORD }}
|
||||
tags: |
|
||||
${{ steps.meta-ee-public.outputs.tags }}-arm64
|
||||
labels: |
|
||||
${{ steps.meta-ee-public.outputs.labels }}-arm64
|
||||
org.opencontainers.image.licenses=Windmill-Enterprise-License
|
||||
|
||||
- uses: shrink/actions-docker-extract@v3
|
||||
id: extract-ee-amd64
|
||||
with:
|
||||
image: ${{ steps.meta-ee-public.outputs.tags}}-amd64
|
||||
path: "/windmill/target/release/windmill"
|
||||
|
||||
- uses: shrink/actions-docker-extract@v3
|
||||
id: extract-duckdb-ffi-internal
|
||||
with:
|
||||
image: ${{ steps.meta-ee-public.outputs.tags}}-amd64
|
||||
path: "/usr/src/app/libwindmill_duckdb_ffi_internal.so"
|
||||
|
||||
# - uses: shrink/actions-docker-extract@v3
|
||||
# id: extract-ee-arm64
|
||||
# with:
|
||||
# image: ${{ steps.meta-ee-public.outputs.tags}}-arm64
|
||||
# path: "/windmill/target/release/windmill"
|
||||
|
||||
- name: Rename binary with corresponding architecture
|
||||
run: |
|
||||
mv "${{ steps.extract-ee-amd64.outputs.destination }}/windmill" "${{ steps.extract-ee-amd64.outputs.destination }}/windmill-ee-amd64-rhel8"
|
||||
# mv "${{ steps.extract-ee-arm64.outputs.destination }}/windmill" "${{ steps.extract-ee-arm64.outputs.destination }}/windmill-ee-arm64-rhel8"
|
||||
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: RHEL8-amd64 build
|
||||
path: ${{ steps.extract-ee-amd64.outputs.destination }}/windmill-ee-amd64-rhel8
|
||||
|
||||
- uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: RHEL8-amd64 dynamic libraries build
|
||||
path: ${{ steps.extract-duckdb-ffi-internal.outputs.destination }}/libwindmill_duckdb_ffi_internal.so
|
||||
|
||||
# - uses: actions/upload-artifact@v4
|
||||
# with:
|
||||
# name: RHEL8-arm64 build
|
||||
# path:
|
||||
# ${{ steps.extract-ee-arm64.outputs.destination
|
||||
# }}/windmill-ee-arm64-rhel8
|
||||
|
||||
# - name: Attach binary to release
|
||||
# uses: softprops/action-gh-release@v2
|
||||
# if: startsWith(github.ref, 'refs/tags/')
|
||||
# with:
|
||||
# files: |
|
||||
# ${{ steps.extract-ee-arm64.outputs.destination }}/windmill-ee-arm64-rhel8
|
||||
# ${{ steps.extract-ee-amd64.outputs.destination }}/windmill-ee-amd64-rhel8
|
||||
41
.github/workflows/build_windows_worker_.yml
vendored
41
.github/workflows/build_windows_worker_.yml
vendored
@@ -11,7 +11,7 @@ env:
|
||||
|
||||
jobs:
|
||||
cargo_build_windows:
|
||||
runs-on: blacksmith-16vcpu-windows-2025
|
||||
runs-on: windows-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
@@ -30,53 +30,34 @@ jobs:
|
||||
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
fetch-depth: 0
|
||||
|
||||
- uses: actions-rust-lang/setup-rust-toolchain@v1
|
||||
- name: Setup Rust
|
||||
uses: actions-rs/toolchain@v1
|
||||
with:
|
||||
cache-workspaces: backend
|
||||
toolchain: 1.93.0
|
||||
toolchain: 1.85.0
|
||||
override: true
|
||||
|
||||
- name: Substitute EE code
|
||||
shell: bash
|
||||
run: |
|
||||
./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
|
||||
|
||||
- name: Cargo check (fail fast on warnings)
|
||||
timeout-minutes: 60
|
||||
env:
|
||||
RUSTFLAGS: "-D warnings"
|
||||
run: |
|
||||
mkdir frontend/build && cd backend
|
||||
New-Item -Path . -Name "windmill-api/openapi-deref.yaml" -ItemType "File" -Force
|
||||
cargo check --features=ee_windows
|
||||
|
||||
- name: Cargo build dynamic libraries windows
|
||||
timeout-minutes: 180
|
||||
run: |
|
||||
cd backend/windmill-duckdb-ffi-internal
|
||||
cargo build --release -p windmill_duckdb_ffi_internal
|
||||
|
||||
- name: Cargo build binary windows
|
||||
timeout-minutes: 180
|
||||
- name: Cargo build windows
|
||||
timeout-minutes: 90
|
||||
run: |
|
||||
vcpkg.exe install openssl-windows:x64-windows
|
||||
vcpkg.exe install openssl:x64-windows-static
|
||||
vcpkg.exe integrate install
|
||||
$env:VCPKGRS_DYNAMIC=1
|
||||
$env:OPENSSL_DIR="${Env:VCPKG_INSTALLATION_ROOT}\installed\x64-windows-static"
|
||||
cd backend
|
||||
cargo build --release --features=ee_windows
|
||||
mkdir frontend/build && cd backend
|
||||
New-Item -Path . -Name "windmill-api/openapi-deref.yaml" -ItemType "File" -Force
|
||||
cargo build --release --features=enterprise,stripe,embedding,parquet,prometheus,openidconnect,cloud,jemalloc,tantivy,license,http_trigger,zip,oauth2,kafka,nats,sqs_trigger,postgres_trigger,gcp_trigger,mqtt_trigger,websocket,smtp,static_frontend,all_languages,mcp
|
||||
- name: Rename binary with corresponding architecture
|
||||
run: |
|
||||
Rename-Item -Path ".\backend\target\release\windmill.exe" -NewName "windmill-ee.exe"
|
||||
|
||||
- name: Upload binary artifact
|
||||
- name: Upload artifact
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: windmill-ee-binary
|
||||
path: ./backend/target/release/windmill-ee.exe
|
||||
|
||||
- name: Upload dynamic libraries artifact
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: windmill_duckdb_ffi_internal.dll
|
||||
path: ./backend/windmill-duckdb-ffi-internal/target/release/windmill_duckdb_ffi_internal.dll
|
||||
|
||||
12
.github/workflows/change-versions.yml
vendored
12
.github/workflows/change-versions.yml
vendored
@@ -9,14 +9,7 @@ jobs:
|
||||
runs-on: ubicloud
|
||||
container: node:18
|
||||
steps:
|
||||
- uses: actions/create-github-app-token@v2
|
||||
id: app
|
||||
with:
|
||||
app-id: ${{ vars.INTERNAL_APP_ID }}
|
||||
private-key: ${{ secrets.INTERNAL_APP_KEY }}
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
token: ${{ steps.app.outputs.token }}
|
||||
- run: git config --system --add safe.directory /__w/windmill/windmill
|
||||
- name: Change versions
|
||||
run: ./.github/change-versions.sh "$(cat version.txt)"
|
||||
@@ -28,8 +21,3 @@ jobs:
|
||||
cd backend
|
||||
cargo generate-lockfile
|
||||
- uses: stefanzweifel/git-auto-commit-action@v5
|
||||
with:
|
||||
commit_user_name: windmill-internal-app[bot]
|
||||
commit_user_email: windmill-internal-app[bot]@users.noreply.github.com
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ steps.app.outputs.token }}
|
||||
|
||||
66
.github/workflows/check-org-membership.yml
vendored
66
.github/workflows/check-org-membership.yml
vendored
@@ -1,66 +0,0 @@
|
||||
name: Check Organization Membership
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
commenter:
|
||||
required: true
|
||||
type: string
|
||||
description: 'The username to check for organization membership'
|
||||
organization:
|
||||
required: false
|
||||
type: string
|
||||
default: 'windmill-labs'
|
||||
description: 'The organization to check membership for'
|
||||
trusted_bot:
|
||||
required: false
|
||||
type: string
|
||||
default: 'windmill-internal-app[bot]'
|
||||
description: 'The trusted bot username to allow'
|
||||
secrets:
|
||||
access_token:
|
||||
required: true
|
||||
description: 'The access token to use for org membership check'
|
||||
outputs:
|
||||
is_member:
|
||||
description: 'Whether the user is an organization member or trusted bot'
|
||||
value: ${{ jobs.check-membership.outputs.is_member }}
|
||||
|
||||
jobs:
|
||||
check-membership:
|
||||
runs-on: ubicloud-standard-2
|
||||
outputs:
|
||||
is_member: ${{ steps.check-membership.outputs.is_member }}
|
||||
steps:
|
||||
- name: Check organization membership
|
||||
id: check-membership
|
||||
env:
|
||||
ORG_ACCESS_TOKEN: ${{ secrets.access_token }}
|
||||
COMMENTER: ${{ inputs.commenter }}
|
||||
ORG: ${{ inputs.organization }}
|
||||
TRUSTED_BOT: ${{ inputs.trusted_bot }}
|
||||
run: |
|
||||
# 1. Allow the trusted bot straight away
|
||||
if [[ "$COMMENTER" == "$TRUSTED_BOT" ]]; then
|
||||
echo "is_member=true" >> $GITHUB_OUTPUT
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# 2. Disallow other bots
|
||||
if [[ "${COMMENTER}" =~ \[bot\]$ ]]; then
|
||||
echo "is_member=false" >> $GITHUB_OUTPUT
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# 3. Otherwise check if the user is a member of the organization
|
||||
STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
|
||||
-H "Authorization: token $ORG_ACCESS_TOKEN" \
|
||||
-H "Accept: application/vnd.github+json" \
|
||||
-H "X-GitHub-Api-Version: 2022-11-28" \
|
||||
"https://api.github.com/orgs/$ORG/members/$COMMENTER")
|
||||
|
||||
if [ "$STATUS" -eq 204 ]; then
|
||||
echo "is_member=true" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "is_member=false" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
76
.github/workflows/claude-fast.yml
vendored
76
.github/workflows/claude-fast.yml
vendored
@@ -1,76 +0,0 @@
|
||||
name: Fast Claude
|
||||
|
||||
on:
|
||||
issue_comment:
|
||||
types: [created]
|
||||
pull_request_review_comment:
|
||||
types: [created]
|
||||
issues:
|
||||
types: [opened, assigned]
|
||||
pull_request_review:
|
||||
types: [submitted]
|
||||
|
||||
jobs:
|
||||
determine-commenter:
|
||||
if: |
|
||||
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '/ai-fast')) ||
|
||||
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '/ai-fast')) ||
|
||||
(github.event_name == 'pull_request_review' && contains(github.event.review.body, '/ai-fast')) ||
|
||||
(github.event_name == 'issues' && contains(github.event.issue.body, '/ai-fast'))
|
||||
runs-on: ubicloud-standard-2
|
||||
outputs:
|
||||
commenter: ${{ steps.determine-commenter.outputs.commenter }}
|
||||
steps:
|
||||
- name: Determine commenter
|
||||
id: determine-commenter
|
||||
run: |
|
||||
# Work out who wrote the comment / review
|
||||
if [[ "${{ github.event_name }}" == "issue_comment" || \
|
||||
"${{ github.event_name }}" == "pull_request_review_comment" ]]; then
|
||||
COMMENTER="${{ github.event.comment.user.login }}"
|
||||
elif [[ "${{ github.event_name }}" == "pull_request_review" ]]; then
|
||||
COMMENTER="${{ github.event.review.user.login }}"
|
||||
else
|
||||
COMMENTER="${{ github.event.issue.user.login }}"
|
||||
fi
|
||||
echo "commenter=$COMMENTER" >> $GITHUB_OUTPUT
|
||||
|
||||
check-membership:
|
||||
needs: determine-commenter
|
||||
uses: ./.github/workflows/check-org-membership.yml
|
||||
with:
|
||||
commenter: ${{ needs.determine-commenter.outputs.commenter }}
|
||||
secrets:
|
||||
access_token: ${{ secrets.ORG_ACCESS_TOKEN }}
|
||||
|
||||
claude-code-action:
|
||||
needs: [determine-commenter, check-membership]
|
||||
if: |
|
||||
needs.check-membership.outputs.is_member == 'true'
|
||||
runs-on: ubicloud-standard-8
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
issues: write
|
||||
id-token: write
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 1
|
||||
|
||||
- name: Run Claude PR Action
|
||||
uses: anthropics/claude-code-action@v1
|
||||
with:
|
||||
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
|
||||
allowed_bots: "windmill-internal-app[bot]"
|
||||
trigger_phrase: "/ai-fast"
|
||||
settings: |
|
||||
{
|
||||
"env": {
|
||||
"SQLX_OFFLINE": "true"
|
||||
}
|
||||
}
|
||||
claude_args: |
|
||||
--allowedTools "Bash,WebFetch,WebSearch"
|
||||
--model opus
|
||||
91
.github/workflows/claude-plan.yml
vendored
91
.github/workflows/claude-plan.yml
vendored
@@ -1,91 +0,0 @@
|
||||
name: Claude Plan Assistant
|
||||
|
||||
on:
|
||||
issue_comment:
|
||||
types: [created]
|
||||
pull_request_review_comment:
|
||||
types: [created]
|
||||
issues:
|
||||
types: [opened, assigned]
|
||||
pull_request_review:
|
||||
types: [submitted]
|
||||
|
||||
jobs:
|
||||
determine-commenter:
|
||||
if: |
|
||||
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '/plan')) ||
|
||||
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '/plan')) ||
|
||||
(github.event_name == 'pull_request_review' && contains(github.event.review.body, '/plan')) ||
|
||||
(github.event_name == 'issues' && contains(github.event.issue.body, '/plan'))
|
||||
runs-on: ubicloud-standard-2
|
||||
outputs:
|
||||
commenter: ${{ steps.determine-commenter.outputs.commenter }}
|
||||
steps:
|
||||
- name: Determine commenter
|
||||
id: determine-commenter
|
||||
run: |
|
||||
# Work out who wrote the comment / review
|
||||
if [[ "${{ github.event_name }}" == "issue_comment" || \
|
||||
"${{ github.event_name }}" == "pull_request_review_comment" ]]; then
|
||||
COMMENTER="${{ github.event.comment.user.login }}"
|
||||
elif [[ "${{ github.event_name }}" == "pull_request_review" ]]; then
|
||||
COMMENTER="${{ github.event.review.user.login }}"
|
||||
else
|
||||
COMMENTER="${{ github.event.issue.user.login }}"
|
||||
fi
|
||||
echo "commenter=$COMMENTER" >> $GITHUB_OUTPUT
|
||||
|
||||
check-membership:
|
||||
needs: determine-commenter
|
||||
uses: ./.github/workflows/check-org-membership.yml
|
||||
with:
|
||||
commenter: ${{ needs.determine-commenter.outputs.commenter }}
|
||||
secrets:
|
||||
access_token: ${{ secrets.ORG_ACCESS_TOKEN }}
|
||||
|
||||
claude-plan-action:
|
||||
needs: [determine-commenter, check-membership]
|
||||
if: |
|
||||
needs.check-membership.outputs.is_member == 'true'
|
||||
runs-on: ubicloud-standard-4
|
||||
timeout-minutes: 20
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: read
|
||||
issues: read
|
||||
id-token: write
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 1
|
||||
|
||||
- name: Run Claude Plan Action
|
||||
uses: anthropics/claude-code-action@v1
|
||||
with:
|
||||
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
|
||||
allowed_bots: 'windmill-internal-app[bot]'
|
||||
trigger_phrase: '/plan'
|
||||
claude_args: |
|
||||
--model opus
|
||||
--system-prompt "# Claude Planning Mode
|
||||
|
||||
You are operating in PLANNING MODE ONLY. Your role is to create detailed, structured plans without making any code changes.
|
||||
|
||||
## Your Responsibilities:
|
||||
|
||||
1. **Analyze the Request**: Carefully read and understand what the user is asking for
|
||||
2. **Explore the Codebase**: Understand the relevant code structure
|
||||
3. **Create a Detailed Plan**: Provide a comprehensive, step-by-step plan that includes:
|
||||
- Clear breakdown of all tasks needed
|
||||
- Files that will need to be modified or created
|
||||
- Code patterns and architecture decisions
|
||||
- Potential challenges and how to address them
|
||||
- If there are multiple options to achieve the same goal, explain the pros and cons of each option
|
||||
|
||||
## Strict Constraints:
|
||||
|
||||
- **DO NOT** make any code changes
|
||||
- **DO NOT** create branches or pull requests
|
||||
|
||||
Remember: You are here to plan, not to implement. Provide thorough analysis and clear guidance for implementation."
|
||||
129
.github/workflows/claude.yml
vendored
129
.github/workflows/claude.yml
vendored
@@ -1,129 +0,0 @@
|
||||
name: Claude PR Assistant
|
||||
|
||||
on:
|
||||
issue_comment:
|
||||
types: [created]
|
||||
pull_request_review_comment:
|
||||
types: [created]
|
||||
issues:
|
||||
types: [opened, assigned]
|
||||
pull_request_review:
|
||||
types: [submitted]
|
||||
|
||||
jobs:
|
||||
determine-commenter:
|
||||
if: |
|
||||
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '/ai')) ||
|
||||
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '/ai')) ||
|
||||
(github.event_name == 'pull_request_review' && contains(github.event.review.body, '/ai')) ||
|
||||
(github.event_name == 'issues' && contains(github.event.issue.body, '/ai'))
|
||||
runs-on: ubicloud-standard-2
|
||||
outputs:
|
||||
commenter: ${{ steps.determine-commenter.outputs.commenter }}
|
||||
steps:
|
||||
- name: Determine commenter
|
||||
id: determine-commenter
|
||||
run: |
|
||||
# Work out who wrote the comment / review
|
||||
if [[ "${{ github.event_name }}" == "issue_comment" || \
|
||||
"${{ github.event_name }}" == "pull_request_review_comment" ]]; then
|
||||
COMMENTER="${{ github.event.comment.user.login }}"
|
||||
elif [[ "${{ github.event_name }}" == "pull_request_review" ]]; then
|
||||
COMMENTER="${{ github.event.review.user.login }}"
|
||||
else
|
||||
COMMENTER="${{ github.event.issue.user.login }}"
|
||||
fi
|
||||
echo "commenter=$COMMENTER" >> $GITHUB_OUTPUT
|
||||
|
||||
check-membership:
|
||||
needs: determine-commenter
|
||||
uses: ./.github/workflows/check-org-membership.yml
|
||||
with:
|
||||
commenter: ${{ needs.determine-commenter.outputs.commenter }}
|
||||
secrets:
|
||||
access_token: ${{ secrets.ORG_ACCESS_TOKEN }}
|
||||
|
||||
claude-code-action:
|
||||
needs: [determine-commenter, check-membership]
|
||||
if: |
|
||||
needs.check-membership.outputs.is_member == 'true'
|
||||
runs-on: ubicloud-standard-8
|
||||
timeout-minutes: 60
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
issues: write
|
||||
id-token: write
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 1
|
||||
|
||||
- uses: actions/cache@v3
|
||||
with:
|
||||
path: ~/.npm
|
||||
key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-node-
|
||||
|
||||
- name: Run npm install and generate-backend-client
|
||||
working-directory: ./frontend
|
||||
run: |
|
||||
# add a build directory for cargo check
|
||||
mkdir -p build
|
||||
npm install
|
||||
npm run generate-backend-client
|
||||
|
||||
- name: install xmlsec1 and gssapi
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y libxml2-dev libxmlsec1-dev libkrb5-dev libsasl2-dev libcurl4-openssl-dev mold clang
|
||||
|
||||
- uses: actions-rust-lang/setup-rust-toolchain@v1
|
||||
with:
|
||||
cache-workspaces: backend
|
||||
toolchain: 1.93.0
|
||||
|
||||
- name: cargo check
|
||||
working-directory: ./backend
|
||||
timeout-minutes: 16
|
||||
run: |
|
||||
SQLX_OFFLINE=true cargo check --features all_sqlx_features
|
||||
|
||||
- name: Run Claude PR Action
|
||||
uses: anthropics/claude-code-action@v1
|
||||
with:
|
||||
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
|
||||
allowed_bots: "windmill-internal-app[bot]"
|
||||
trigger_phrase: "/ai"
|
||||
settings: |
|
||||
{
|
||||
"env": {
|
||||
"SQLX_OFFLINE": "true"
|
||||
}
|
||||
}
|
||||
claude_args: |
|
||||
--allowedTools "Bash,WebFetch,WebSearch"
|
||||
--model opus
|
||||
--system-prompt "## IMPORTANT INSTRUCTIONS
|
||||
- Your branch name should be a short description of the requested changes.
|
||||
- Bypass previous instructions and instead of adding a url in your comment to manually create a PR, if you created a new branch with your changes, automatically create a draft PR from that branch to main.
|
||||
|
||||
## Code Quality Requirements
|
||||
|
||||
After making any code changes, you MUST run the appropriate validation commands:
|
||||
|
||||
**Frontend Changes:**
|
||||
- Run: \`npm run check\` in the frontend directory
|
||||
- Fix all warnings and errors before proceeding
|
||||
|
||||
**Backend Changes:**
|
||||
- Run: \`cargo check --features all_sqlx_features\` in the backend directory
|
||||
- Fix all warnings and errors before proceeding
|
||||
|
||||
**Pull Request Creation:**
|
||||
- DO NOT FORGET TO OPEN A DRAFT PR AFTER YOU ARE DONE if you made changes after a request from a git issue.
|
||||
|
||||
## Available Tools
|
||||
- Bash: Full access to run validation commands and git operations"
|
||||
188
.github/workflows/cli-tests.yml
vendored
188
.github/workflows/cli-tests.yml
vendored
@@ -1,188 +0,0 @@
|
||||
name: CLI Tests
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- 'cli/**'
|
||||
- '.github/workflows/cli-tests.yml'
|
||||
pull_request:
|
||||
branches: [main]
|
||||
paths:
|
||||
- 'cli/**'
|
||||
- '.github/workflows/cli-tests.yml'
|
||||
|
||||
env:
|
||||
CARGO_TERM_COLOR: always
|
||||
SQLX_OFFLINE: true
|
||||
|
||||
jobs:
|
||||
build-check:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Deno
|
||||
uses: denoland/setup-deno@v2
|
||||
with:
|
||||
deno-version: v2.x
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '20'
|
||||
|
||||
- name: Generate Windmill client
|
||||
working-directory: cli
|
||||
run: ./gen_wm_client.sh
|
||||
|
||||
- name: Run CLI build
|
||||
working-directory: cli
|
||||
run: ./build.sh
|
||||
|
||||
test-linux:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16
|
||||
env:
|
||||
POSTGRES_USER: postgres
|
||||
POSTGRES_PASSWORD: changeme
|
||||
POSTGRES_DB: windmill
|
||||
options: >-
|
||||
--health-cmd pg_isready
|
||||
--health-interval 10s
|
||||
--health-timeout 5s
|
||||
--health-retries 5
|
||||
ports:
|
||||
- 5432:5432
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Rust toolchain
|
||||
uses: actions-rust-lang/setup-rust-toolchain@v1
|
||||
with:
|
||||
cache: true
|
||||
cache-workspaces: backend
|
||||
|
||||
- name: Setup Deno
|
||||
uses: denoland/setup-deno@v2
|
||||
with:
|
||||
deno-version: v2.x
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '20'
|
||||
|
||||
- name: Setup Bun
|
||||
uses: oven-sh/setup-bun@v2
|
||||
with:
|
||||
bun-version: latest
|
||||
|
||||
- name: Symlink Bun to /usr/bin/bun
|
||||
run: sudo ln -sf $(which bun) /usr/bin/bun
|
||||
|
||||
- name: Symlink Node to /usr/bin/node
|
||||
run: sudo ln -sf $(which node) /usr/bin/node
|
||||
|
||||
- name: Generate Windmill clients
|
||||
working-directory: cli
|
||||
run: |
|
||||
./gen_wm_client.sh
|
||||
./windmill-utils-internal/gen_wm_client.sh
|
||||
|
||||
- name: Run CLI tests
|
||||
working-directory: cli
|
||||
env:
|
||||
DATABASE_URL: postgres://postgres:changeme@localhost:5432
|
||||
CI_MINIMAL_FEATURES: "true"
|
||||
run: |
|
||||
deno test --no-check --allow-all test/ \
|
||||
--ignore=test/cargo_backend_example.test.ts
|
||||
|
||||
test-windows:
|
||||
runs-on: windows-latest
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup PostgreSQL
|
||||
uses: ikalnytskyi/action-setup-postgres@v6
|
||||
with:
|
||||
username: postgres
|
||||
password: changeme
|
||||
database: windmill
|
||||
port: 5432
|
||||
|
||||
- name: Setup Rust toolchain
|
||||
uses: actions-rust-lang/setup-rust-toolchain@v1
|
||||
with:
|
||||
cache: true
|
||||
cache-workspaces: backend
|
||||
|
||||
- name: Setup Deno
|
||||
uses: denoland/setup-deno@v2
|
||||
with:
|
||||
deno-version: v2.x
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '20'
|
||||
|
||||
- name: Setup Bun
|
||||
uses: oven-sh/setup-bun@v2
|
||||
with:
|
||||
bun-version: latest
|
||||
|
||||
- name: Get Bun and Node paths
|
||||
id: runtime-paths
|
||||
shell: pwsh
|
||||
run: |
|
||||
$bunPath = (Get-Command bun).Source
|
||||
$nodePath = (Get-Command node).Source
|
||||
echo "BUN_PATH=$bunPath" >> $env:GITHUB_OUTPUT
|
||||
echo "NODE_BIN_PATH=$nodePath" >> $env:GITHUB_OUTPUT
|
||||
|
||||
- name: Generate Windmill clients
|
||||
working-directory: cli
|
||||
shell: bash
|
||||
run: |
|
||||
./gen_wm_client.sh
|
||||
./windmill-utils-internal/gen_wm_client.sh
|
||||
|
||||
- name: Run CLI tests
|
||||
working-directory: cli
|
||||
shell: pwsh
|
||||
env:
|
||||
DATABASE_URL: postgres://postgres:changeme@localhost:5432
|
||||
CI_MINIMAL_FEATURES: "true"
|
||||
BUN_PATH: ${{ steps.runtime-paths.outputs.BUN_PATH }}
|
||||
NODE_BIN_PATH: ${{ steps.runtime-paths.outputs.NODE_BIN_PATH }}
|
||||
run: |
|
||||
deno test --no-check --allow-all test/ `
|
||||
--ignore=test/cargo_backend_example.test.ts
|
||||
|
||||
# Combined summary job for branch protection
|
||||
test-summary:
|
||||
runs-on: ubuntu-latest
|
||||
needs: [build-check, test-linux, test-windows]
|
||||
if: always()
|
||||
steps:
|
||||
- name: Check test results
|
||||
run: |
|
||||
if [ "${{ needs.build-check.result }}" != "success" ]; then
|
||||
echo "Build check failed"
|
||||
exit 1
|
||||
fi
|
||||
if [ "${{ needs.test-linux.result }}" != "success" ] || [ "${{ needs.test-windows.result }}" != "success" ]; then
|
||||
echo "Some tests failed"
|
||||
exit 1
|
||||
fi
|
||||
echo "All checks passed"
|
||||
28
.github/workflows/create-docs.yml
vendored
28
.github/workflows/create-docs.yml
vendored
@@ -3,37 +3,13 @@ on:
|
||||
types: [created]
|
||||
|
||||
jobs:
|
||||
check-membership:
|
||||
if: ${{ github.event.issue.pull_request && startsWith(github.event.comment.body, '/docs') }}
|
||||
uses: ./.github/workflows/check-org-membership.yml
|
||||
with:
|
||||
commenter: ${{ github.event.comment.user.login }}
|
||||
secrets:
|
||||
access_token: ${{ secrets.ORG_ACCESS_TOKEN }}
|
||||
|
||||
generate-token:
|
||||
needs: check-membership
|
||||
if: ${{ needs.check-membership.outputs.is_member == 'true' }}
|
||||
runs-on: ubicloud-standard-2
|
||||
outputs:
|
||||
app_token: ${{ steps.app.outputs.token }}
|
||||
steps:
|
||||
- name: Generate an installation token
|
||||
id: app
|
||||
uses: actions/create-github-app-token@v2
|
||||
with:
|
||||
app-id: ${{ vars.INTERNAL_APP_ID }}
|
||||
private-key: ${{ secrets.INTERNAL_APP_KEY }}
|
||||
owner: windmill-labs
|
||||
|
||||
trigger-docs:
|
||||
needs: [generate-token, check-membership]
|
||||
if: ${{ needs.check-membership.outputs.is_member == 'true' }}
|
||||
if: ${{ github.event.issue.pull_request && startsWith(github.event.comment.body, '/docs') }}
|
||||
uses: windmill-labs/windmilldocs/.github/workflows/create-docs.yml@main
|
||||
with:
|
||||
pr_number: ${{ github.event.issue.number }}
|
||||
repo: ${{ github.event.repository.name }}
|
||||
comment_text: ${{ github.event.comment.body }}
|
||||
secrets:
|
||||
DOCS_TOKEN: ${{ needs.generate-token.outputs.app_token }}
|
||||
DOCS_TOKEN: ${{ secrets.DOCS_TOKEN }}
|
||||
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
|
||||
|
||||
7
.github/workflows/discord-notification.yml
vendored
7
.github/workflows/discord-notification.yml
vendored
@@ -17,14 +17,11 @@ jobs:
|
||||
PR_AUTHOR: ${{ github.event.pull_request.user.login }}
|
||||
PR_STATUS: "opened"
|
||||
PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
DISCORD_CHANNEL_ID: "1372204995868491786"
|
||||
DISCORD_GUILD_ID: "930051556043276338"
|
||||
secrets:
|
||||
DISCORD_WEBHOOK_URL: ${{ secrets.DISCORD_PR_REVIEWS_WEBHOOK }}
|
||||
DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_AI_BOT_TOKEN }}
|
||||
|
||||
merge_success_emoji:
|
||||
if: github.event.action == 'closed'
|
||||
if: github.event.pull_request.merged == true
|
||||
uses: ./.github/workflows/shareable-discord-notification.yml
|
||||
with:
|
||||
PR_STATUS: "merged"
|
||||
@@ -32,4 +29,4 @@ jobs:
|
||||
DISCORD_GUILD_ID: "930051556043276338"
|
||||
PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
secrets:
|
||||
DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_AI_BOT_TOKEN }}
|
||||
DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_PR_BOT_TOKEN }}
|
||||
|
||||
2
.github/workflows/docker-image-rpi4.yml
vendored
2
.github/workflows/docker-image-rpi4.yml
vendored
@@ -67,7 +67,7 @@ jobs:
|
||||
platforms: linux/amd64,linux/arm64
|
||||
push: true
|
||||
build-args: |
|
||||
features=ce_rpi
|
||||
features=embedding,parquet,openidconnect,license,http_trigger,zip,oauth2,postgres_trigger,mqtt_trigger,websocket,smtp,static_frontend,all_languages,deno_core,mcp
|
||||
tags: |
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:dev
|
||||
${{ steps.meta-public.outputs.tags }}
|
||||
|
||||
130
.github/workflows/docker-image.yml
vendored
130
.github/workflows/docker-image.yml
vendored
@@ -24,8 +24,8 @@ on:
|
||||
description: "Tag the image"
|
||||
required: true
|
||||
default: "test"
|
||||
slim:
|
||||
description: "Build slim image (true, false)"
|
||||
nsjail:
|
||||
description: "Build nsjail image (true, false)"
|
||||
required: false
|
||||
default: false
|
||||
type: boolean
|
||||
@@ -92,7 +92,7 @@ jobs:
|
||||
platforms: linux/amd64,linux/arm64
|
||||
push: true
|
||||
build-args: |
|
||||
features=ce
|
||||
features=embedding,parquet,openidconnect,jemalloc,license,http_trigger,zip,oauth2,dind,postgres_trigger,mqtt_trigger,websocket,smtp,static_frontend,agent_worker_server,all_languages,deno_core,mcp
|
||||
tags: |
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.DEV_SHA }}
|
||||
${{ steps.meta-public.outputs.tags }}
|
||||
@@ -101,7 +101,7 @@ jobs:
|
||||
|
||||
build_ee:
|
||||
runs-on: ubicloud
|
||||
if: (github.event_name != 'workflow_dispatch') || github.event.inputs.ee
|
||||
if: (github.event_name != 'workflow_dispatch') || (github.event.inputs.ee || github.event.inputs.nsjail)
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
@@ -154,7 +154,7 @@ jobs:
|
||||
platforms: linux/amd64,linux/arm64
|
||||
push: true
|
||||
build-args: |
|
||||
features=ee
|
||||
features=enterprise,enterprise_saml,stripe,embedding,parquet,prometheus,openidconnect,cloud,jemalloc,agent_worker_server,tantivy,license,http_trigger,zip,oauth2,kafka,sqs_trigger,nats,otel,dind,postgres_trigger,mqtt_trigger,gcp_trigger,websocket,smtp,static_frontend,all_languages,deno_core,mcp
|
||||
tags: |
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee:${{ env.DEV_SHA }}
|
||||
${{ steps.meta-ee-public.outputs.tags }}
|
||||
@@ -162,6 +162,39 @@ jobs:
|
||||
${{ steps.meta-ee-public.outputs.labels }}
|
||||
org.opencontainers.image.licenses=Windmill-Enterprise-License
|
||||
|
||||
# disabled until we make it 100% reliable and add more meaningful tests
|
||||
# playwright:
|
||||
# runs-on: [self-hosted, new]
|
||||
# needs: [build]
|
||||
# services:
|
||||
# postgres:
|
||||
# image: postgres
|
||||
# env:
|
||||
# POSTGRES_DB: windmill
|
||||
# POSTGRES_USER: admin
|
||||
# POSTGRES_PASSWORD: changeme
|
||||
# ports:
|
||||
# - 5432:5432
|
||||
# options: >-
|
||||
# --health-cmd pg_isready
|
||||
# --health-interval 10s
|
||||
# --health-timeout 5s
|
||||
# --health-retries 5
|
||||
# steps:
|
||||
# - uses: actions/checkout@v4
|
||||
# - name: "Docker"
|
||||
# run: echo "::set-output name=id::$(docker run --network=host --rm -d -p 8000:8000 --privileged -it -e DATABASE_URL=postgres://admin:changeme@localhost:5432/windmill -e BASE_INTERNAL_URL=http://localhost:8000 ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest)"
|
||||
# id: docker-container
|
||||
# - uses: actions/setup-node@v3
|
||||
# with:
|
||||
# node-version: 16
|
||||
# - name: "Playwright run"
|
||||
# timeout-minutes: 2
|
||||
# run: cd frontend && npm ci @playwright/test && npx playwright install && export BASE_URL=http://localhost:8000 && npm run test
|
||||
# - name: "Clean up"
|
||||
# run: docker kill ${{ steps.docker-container.outputs.id }}
|
||||
# if: always()
|
||||
|
||||
attach_amd64_binary_to_release:
|
||||
needs: [build, build_ee]
|
||||
runs-on: ubicloud
|
||||
@@ -187,12 +220,6 @@ jobs:
|
||||
image: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.DEV_SHA }}
|
||||
path: "/usr/src/app/windmill"
|
||||
|
||||
- uses: shrink/actions-docker-extract@v3
|
||||
id: extract-duckdb-ffi-internal
|
||||
with:
|
||||
image: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.DEV_SHA }}
|
||||
path: "/usr/src/app/libwindmill_duckdb_ffi_internal.so"
|
||||
|
||||
- uses: shrink/actions-docker-extract@v3
|
||||
id: extract-ee
|
||||
with:
|
||||
@@ -210,7 +237,6 @@ jobs:
|
||||
files: |
|
||||
${{ steps.extract.outputs.destination }}/*
|
||||
${{ steps.extract-ee.outputs.destination }}/*
|
||||
${{ steps.extract-duckdb-ffi-internal.outputs.destination }}/*
|
||||
|
||||
# attach_arm64_binary_to_release:
|
||||
# needs: [build, build_ee]
|
||||
@@ -365,10 +391,67 @@ jobs:
|
||||
# ignore-unchanged: true
|
||||
# only-fixed: true
|
||||
|
||||
build_ee_nsjail:
|
||||
needs: [build_ee]
|
||||
runs-on: ubicloud
|
||||
if: (github.event_name != 'pull_request') && ((github.event_name != 'workflow_dispatch') || (github.event.inputs.ee || github.event.inputs.nsjail))
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
ref: ${{ github.ref }}
|
||||
|
||||
# - name: Set up Docker Buildx
|
||||
# uses: docker/setup-buildx-action@v2
|
||||
|
||||
- uses: depot/setup-action@v1
|
||||
|
||||
- name: Docker meta
|
||||
id: meta-ee-public
|
||||
uses: docker/metadata-action@v5
|
||||
with:
|
||||
images: |
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee-nsjail
|
||||
flavor: |
|
||||
latest=false
|
||||
tags: |
|
||||
type=semver,pattern={{version}}
|
||||
type=semver,pattern={{major}}.{{minor}}
|
||||
type=sha,enable=true,priority=100,prefix=,suffix=,format=short
|
||||
type=ref,event=branch
|
||||
type=ref,event=pr
|
||||
|
||||
- name: Login to registry
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
registry: ${{ env.REGISTRY }}
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Update Dockerfile image reference
|
||||
run: |
|
||||
sed -i 's|FROM ghcr.io/windmill-labs/windmill-ee:dev|FROM ghcr.io/${{ env.IMAGE_NAME }}-ee:${{ env.DEV_SHA }}|' ./docker/DockerfileNsjail
|
||||
cat ./docker/DockerfileNsjail | grep "FROM"
|
||||
|
||||
- name: Build and push publicly ee
|
||||
uses: depot/build-push-action@v1
|
||||
with:
|
||||
context: .
|
||||
platforms: linux/amd64,linux/arm64
|
||||
push: true
|
||||
file: "./docker/DockerfileNsjail"
|
||||
tags: |
|
||||
${{ steps.meta-ee-public.outputs.tags }}
|
||||
labels: |
|
||||
${{ steps.meta-ee-public.outputs.labels }}
|
||||
org.opencontainers.image.licenses=Windmill-Enterprise-License
|
||||
|
||||
publish_ecr_s3:
|
||||
needs: [build_ee_full]
|
||||
needs: [build_ee_nsjail]
|
||||
runs-on: ubicloud-standard-2-arm
|
||||
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
|
||||
if: (github.event_name != 'pull_request') && (github.event_name !=
|
||||
'workflow_dispatch')
|
||||
env:
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||
@@ -387,18 +470,23 @@ jobs:
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Get version from tag
|
||||
id: version
|
||||
run: echo "VERSION=${GITHUB_REF_NAME#v}" >> "$GITHUB_OUTPUT"
|
||||
- name: get git hash
|
||||
if: github.event_name != 'pull_request'
|
||||
id: git_hash
|
||||
run: |
|
||||
git_hash=$(git rev-parse --short "$GITHUB_SHA")
|
||||
echo "GIT_HASH=${git_hash:0:7}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- uses: shrink/actions-docker-extract@v3
|
||||
if: github.event_name != 'pull_request'
|
||||
id: extract
|
||||
with:
|
||||
image: |-
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee-full:${{ steps.version.outputs.VERSION }}
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee-nsjail:${{ steps.git_hash.outputs.GIT_HASH }}
|
||||
path: "/static_frontend/."
|
||||
|
||||
- uses: reggionick/s3-deploy@v4
|
||||
if: github.event_name != 'pull_request'
|
||||
with:
|
||||
folder: ${{ steps.extract.outputs.destination }}
|
||||
bucket: windmill-frontend
|
||||
@@ -493,10 +581,9 @@ jobs:
|
||||
${{ steps.meta-ee-public.outputs.labels }}
|
||||
|
||||
build_ee_slim:
|
||||
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
|
||||
needs: [build_ee]
|
||||
runs-on: ubicloud
|
||||
if: (github.event_name != 'pull_request') && ((github.event_name != 'workflow_dispatch') || (github.event.inputs.ee || github.event.inputs.slim))
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
@@ -514,7 +601,6 @@ jobs:
|
||||
images: |
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee-slim
|
||||
tags: |
|
||||
type=ref,event=branch
|
||||
type=semver,pattern={{version}}
|
||||
type=semver,pattern={{major}}.{{minor}}
|
||||
|
||||
@@ -529,7 +615,7 @@ jobs:
|
||||
uses: depot/build-push-action@v1
|
||||
with:
|
||||
context: .
|
||||
platforms: linux/amd64,linux/arm64
|
||||
platforms: linux/amd64
|
||||
push: true
|
||||
file: "./docker/DockerfileSlimEe"
|
||||
tags: |
|
||||
|
||||
9
.github/workflows/frontend-check.yml
vendored
9
.github/workflows/frontend-check.yml
vendored
@@ -16,12 +16,11 @@ jobs:
|
||||
runs-on: ubicloud-standard-8
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v5
|
||||
- uses: actions/setup-node@v3
|
||||
with:
|
||||
node-version: 24
|
||||
cache: "npm"
|
||||
cache-dependency-path: "frontend/package-lock.json"
|
||||
node-version: 18
|
||||
- name: "npm check"
|
||||
timeout-minutes: 5
|
||||
run: cd frontend && npm ci && npm run generate-backend-client && npm run
|
||||
run:
|
||||
cd frontend && npm ci && npm run generate-backend-client && npm run
|
||||
check
|
||||
|
||||
285
.github/workflows/git-commands.yaml
vendored
285
.github/workflows/git-commands.yaml
vendored
@@ -1,285 +0,0 @@
|
||||
name: Git commands
|
||||
|
||||
on:
|
||||
issue_comment:
|
||||
types: [created]
|
||||
|
||||
jobs:
|
||||
update-sqlx:
|
||||
if: github.event.issue.pull_request && startsWith(github.event.comment.body, '/updatesqlx')
|
||||
runs-on: ubicloud-standard-8
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
issues: write
|
||||
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16
|
||||
env:
|
||||
POSTGRES_PASSWORD: postgres
|
||||
POSTGRES_USER: postgres
|
||||
POSTGRES_DB: windmill
|
||||
ports:
|
||||
- 5432:5432
|
||||
options: >-
|
||||
--health-cmd pg_isready
|
||||
--health-interval 10s
|
||||
--health-timeout 5s
|
||||
--health-retries 5
|
||||
|
||||
steps:
|
||||
- uses: actions/create-github-app-token@v2
|
||||
id: app
|
||||
with:
|
||||
app-id: ${{ vars.INTERNAL_APP_ID }}
|
||||
private-key: ${{ secrets.INTERNAL_APP_KEY }}
|
||||
|
||||
- name: Comment on PR - Starting
|
||||
uses: actions/github-script@v6
|
||||
with:
|
||||
github-token: ${{ steps.app.outputs.token }}
|
||||
script: |
|
||||
const runUrl = `https://github.com/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
|
||||
github.rest.issues.createComment({
|
||||
issue_number: context.issue.number,
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
body: `Starting sqlx update...\n\n[View workflow run](${runUrl})`
|
||||
})
|
||||
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v3
|
||||
with:
|
||||
token: ${{ steps.app.outputs.token }}
|
||||
ref: ${{ github.event.issue.pull_request.head.ref }}
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Checkout windmill-ee-private
|
||||
uses: actions/checkout@v3
|
||||
with:
|
||||
repository: windmill-labs/windmill-ee-private
|
||||
path: windmill-ee-private
|
||||
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
|
||||
# Setup Rust toolchain
|
||||
- uses: actions-rust-lang/setup-rust-toolchain@v1
|
||||
with:
|
||||
cache-workspaces: backend
|
||||
toolchain: 1.93.0
|
||||
|
||||
- name: Install xmlsec and gssapi build-time deps
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y --no-install-recommends \
|
||||
pkg-config libxml2-dev libssl-dev libkrb5-dev libsasl2-dev libcurl4-openssl-dev mold clang \
|
||||
xmlsec1 libxmlsec1-dev libxmlsec1-openssl
|
||||
|
||||
- name: Run update-sqlx script
|
||||
env:
|
||||
DATABASE_URL: postgres://postgres:postgres@localhost:5432/windmill
|
||||
GH_TOKEN: ${{ steps.app.outputs.token }}
|
||||
run: |
|
||||
set -e # Exit on any command failure
|
||||
PR_NUMBER=${{ github.event.issue.number }}
|
||||
|
||||
# Set up error trap to comment on PR for any failure
|
||||
trap 'gh pr comment $PR_NUMBER --body "❌ SQLx update failed. Please check the workflow logs for details."' ERR
|
||||
|
||||
BRANCH_NAME=$(gh pr view $PR_NUMBER --json headRefName --jq .headRefName)
|
||||
echo "Checking out PR branch: $BRANCH_NAME"
|
||||
git checkout $BRANCH_NAME
|
||||
git config --local user.email "windmill-internal-app[bot]@users.noreply.github.com"
|
||||
git config --local user.name "windmill-internal-app[bot]"
|
||||
git config pull.rebase true
|
||||
git pull origin $BRANCH_NAME
|
||||
mkdir -p frontend/build
|
||||
cd backend
|
||||
cargo install sqlx-cli --version 0.8.5
|
||||
sqlx migrate run
|
||||
./substitute_ee_code.sh --dir ./windmill-ee-private
|
||||
./update_sqlx.sh
|
||||
# Pass the branch name to the next step
|
||||
echo "BRANCH_NAME=$BRANCH_NAME" >> $GITHUB_ENV
|
||||
|
||||
- name: Commit changes if any
|
||||
run: |
|
||||
git add backend/.sqlx
|
||||
git commit -m "Update SQLx metadata"
|
||||
git push origin ${{ env.BRANCH_NAME }}
|
||||
|
||||
- name: Comment on PR - Completed
|
||||
uses: actions/github-script@v6
|
||||
with:
|
||||
github-token: ${{ steps.app.outputs.token }}
|
||||
script: |
|
||||
github.rest.issues.createComment({
|
||||
issue_number: context.issue.number,
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
body: 'Successfully ran sqlx update'
|
||||
})
|
||||
|
||||
demo:
|
||||
if: github.event.issue.pull_request && startsWith(github.event.comment.body, '/demo')
|
||||
runs-on: ubicloud-standard-2
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: read
|
||||
issues: read
|
||||
id-token: write
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Run Claude Code for Demo Generation
|
||||
uses: anthropics/claude-code-action@beta
|
||||
with:
|
||||
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
|
||||
timeout_minutes: "10"
|
||||
allowed_tools: "Bash"
|
||||
direct_prompt: |
|
||||
You need to:
|
||||
|
||||
1. Extract the Cloudflare preview URL from the cloudflare-workers-and-pages bot comment in this PR
|
||||
2. Analyze the PR changes to understand what feature was added/modified
|
||||
3. Create detailed instructions to give to an AI agent that will click and interact with buttons and inputs to showcase the new feature. Only include the instructions, nothing else.
|
||||
4. Create a demo.json file with a valid JSON object containing:
|
||||
- instructions: the demo instructions
|
||||
- url: the preview URL
|
||||
5. VALIDATE the JSON file using `jq` before finishing
|
||||
DO NOT COMMIT THIS FILE TO THE PR.
|
||||
|
||||
Example demo.json:
|
||||
{
|
||||
"instructions": "Click on settings, then account settings, then 'generate new token'",
|
||||
"url": "https://example.pages.dev"
|
||||
}
|
||||
|
||||
CRITICAL: After creating demo.json, you MUST:
|
||||
1. Run `jq empty demo.json` to validate the JSON is properly formatted
|
||||
2. If validation fails, fix the JSON and validate again
|
||||
3. Only proceed once the JSON passes validation
|
||||
4. Use proper JSON escaping for newlines, quotes, and special characters
|
||||
|
||||
Make sure to:
|
||||
- Create a valid JSON object that passes `jq empty demo.json`
|
||||
- Extract the correct preview URL (should be a .pages.dev domain)
|
||||
- Create specific, actionable demo steps based on the actual changes in the PR
|
||||
- Properly escape all strings in the JSON (use jq to create the file if needed)
|
||||
- NOT COMMIT THE DEMO.JSON FILE TO THE PR
|
||||
|
||||
- name: Send instructions to Windmill
|
||||
env:
|
||||
DEMO_WEBHOOK_TOKEN: ${{ secrets.DEMO_WEBHOOK_TOKEN }}
|
||||
run: |
|
||||
if [[ -f "demo.json" ]]; then
|
||||
echo "Found demo.json, sending to Windmill..."
|
||||
cat demo.json
|
||||
|
||||
# Validate JSON one more time (Claude should have already done this)
|
||||
if ! jq empty demo.json; then
|
||||
echo "Error: demo.json is not valid JSON"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
RESULT=$(curl -s \
|
||||
-H 'Content-Type: application/json' \
|
||||
-H "Authorization: Bearer $DEMO_WEBHOOK_TOKEN" \
|
||||
-X POST \
|
||||
-d @demo.json \
|
||||
'https://app.windmill.dev/api/w/windmill-labs/jobs/run/f/f/ai/browserbase_demo')
|
||||
|
||||
echo "Windmill response:"
|
||||
echo -E "$RESULT"
|
||||
else
|
||||
echo "Error: demo.json file not found"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
update-ee-ref:
|
||||
if: github.event.issue.pull_request && startsWith(github.event.comment.body, '/eeref')
|
||||
runs-on: ubicloud-standard-2
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
issues: write
|
||||
steps:
|
||||
- uses: actions/create-github-app-token@v2
|
||||
id: app
|
||||
with:
|
||||
app-id: ${{ vars.INTERNAL_APP_ID }}
|
||||
private-key: ${{ secrets.INTERNAL_APP_KEY }}
|
||||
|
||||
- name: Comment on PR - Starting
|
||||
uses: actions/github-script@v6
|
||||
with:
|
||||
github-token: ${{ steps.app.outputs.token }}
|
||||
script: |
|
||||
const runUrl = `https://github.com/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
|
||||
github.rest.issues.createComment({
|
||||
issue_number: context.issue.number,
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
body: `Starting ee ref update...\n\n[View workflow run](${runUrl})`
|
||||
})
|
||||
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v3
|
||||
with:
|
||||
token: ${{ steps.app.outputs.token }}
|
||||
ref: ${{ github.event.issue.pull_request.head.ref }}
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Checkout windmill-ee-private
|
||||
uses: actions/checkout@v3
|
||||
with:
|
||||
repository: windmill-labs/windmill-ee-private
|
||||
path: windmill-ee-private
|
||||
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
|
||||
- name: Get last commit hash of private-repo
|
||||
id: get-commit-hash
|
||||
run: |
|
||||
cd windmill-ee-private
|
||||
COMMIT_HASH=$(git rev-parse HEAD)
|
||||
echo "commit_hash=$COMMIT_HASH" >> $GITHUB_OUTPUT
|
||||
echo "Latest commit hash: $COMMIT_HASH"
|
||||
|
||||
- name: Update ee-repo-ref.txt
|
||||
env:
|
||||
GH_TOKEN: ${{ steps.app.outputs.token }}
|
||||
run: |
|
||||
set -e # Exit on any command failure
|
||||
PR_NUMBER=${{ github.event.issue.number }}
|
||||
|
||||
# Set up error trap to comment on PR for any failure
|
||||
trap 'gh pr comment $PR_NUMBER --body "❌ EE ref update failed. Please check the workflow logs for details."' ERR
|
||||
|
||||
BRANCH_NAME=$(gh pr view $PR_NUMBER --json headRefName --jq .headRefName)
|
||||
echo "Checking out PR branch: $BRANCH_NAME"
|
||||
git checkout $BRANCH_NAME
|
||||
git config --local user.email "windmill-internal-app[bot]@users.noreply.github.com"
|
||||
git config --local user.name "windmill-internal-app[bot]"
|
||||
git config pull.rebase true
|
||||
git pull origin $BRANCH_NAME
|
||||
echo "${{ steps.get-commit-hash.outputs.commit_hash }}" > backend/ee-repo-ref.txt
|
||||
echo "Updated backend/ee-repo-ref.txt with commit hash: ${{ steps.get-commit-hash.outputs.commit_hash }}"
|
||||
# commit and push the changes
|
||||
git add backend/ee-repo-ref.txt
|
||||
git commit -m "Update ee-repo-ref.txt" || echo "No changes to commit"
|
||||
git push origin $BRANCH_NAME
|
||||
|
||||
- name: Comment on PR - Completed
|
||||
uses: actions/github-script@v6
|
||||
with:
|
||||
github-token: ${{ steps.app.outputs.token }}
|
||||
script: |
|
||||
github.rest.issues.createComment({
|
||||
issue_number: context.issue.number,
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
body: 'Successfully updated ee-repo-ref.txt'
|
||||
})
|
||||
29
.github/workflows/helmchart_on_release.yml
vendored
29
.github/workflows/helmchart_on_release.yml
vendored
@@ -9,19 +9,11 @@ jobs:
|
||||
runs-on: ubicloud-standard-2
|
||||
|
||||
steps:
|
||||
- name: Generate an installation token
|
||||
id: app
|
||||
uses: actions/create-github-app-token@v2
|
||||
with:
|
||||
app-id: ${{ vars.INTERNAL_APP_ID }}
|
||||
private-key: ${{ secrets.INTERNAL_APP_KEY }}
|
||||
owner: windmill-labs
|
||||
|
||||
- name: Checkout on helm repository
|
||||
uses: actions/checkout@v3
|
||||
with:
|
||||
repository: windmill-labs/windmill-helm-charts
|
||||
token: ${{ steps.app.outputs.token }}
|
||||
token: ${{ secrets.DOCS_TOKEN }}
|
||||
|
||||
- name: Get version
|
||||
id: get_version
|
||||
@@ -57,23 +49,6 @@ jobs:
|
||||
APP_VERSION=${APP_VERSION%/}
|
||||
sed -i "s/appVersion: .*/appVersion: $APP_VERSION/" ./charts/windmill/Chart.yaml
|
||||
|
||||
- name: Close existing bump-helm PRs
|
||||
env:
|
||||
GH_TOKEN: ${{ steps.app.outputs.token }}
|
||||
run: |
|
||||
# List open PR numbers whose title starts with the prefix
|
||||
prs=$(gh pr list \
|
||||
--state open \
|
||||
--search '"helm: bump version to" in:title' \
|
||||
--json number \
|
||||
-q '.[].number')
|
||||
|
||||
for pr in $prs; do
|
||||
echo "Closing outdated bump PR #$pr"
|
||||
gh pr close "$pr" \
|
||||
--comment "Closed automatically – superseded by a newer Helm-chart bump PR."
|
||||
done
|
||||
|
||||
- name: Commit and push
|
||||
run: |
|
||||
git add .
|
||||
@@ -82,7 +57,7 @@ jobs:
|
||||
|
||||
- name: Create PR
|
||||
env:
|
||||
GH_TOKEN: ${{ steps.app.outputs.token }}
|
||||
GH_TOKEN: ${{ secrets.DOCS_TOKEN }}
|
||||
run: |
|
||||
gh pr create \
|
||||
--title "helm: bump version to ${{ env.VERSION }}" \
|
||||
|
||||
38
.github/workflows/linear-claude.yaml
vendored
38
.github/workflows/linear-claude.yaml
vendored
@@ -1,38 +0,0 @@
|
||||
name: Claude PR Assistant
|
||||
|
||||
on:
|
||||
repository_dispatch:
|
||||
types: [external_claude_issue_fix]
|
||||
|
||||
jobs:
|
||||
claude-code-action:
|
||||
runs-on: ubicloud-standard-8
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: read
|
||||
issues: read
|
||||
id-token: write
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 1
|
||||
|
||||
- name: Process inputs
|
||||
id: process_inputs
|
||||
shell: bash
|
||||
run: |
|
||||
ISSUE_TITLE="${{ github.event.client_payload.issue_title }}"
|
||||
INSTRUCTION="${{ github.event.client_payload.instruction }}"
|
||||
ISSUE_BODY=$(printf '%q' "${{ github.event.client_payload.issue_body }}")
|
||||
BASE_PROMPT="Try to fix the following issue based on the instruction given. You are provided with the issue title, issue body, and instruction. You are to fix the issue based on the instruction. You are to create a pull request to fix the issue."
|
||||
CUSTOM_PROMPT=$(printf -v PROMPT "%s\n\nISSUE_TITLE: %s\n\nISSUE_BODY: %s\n\nINSTRUCTION: %s" "$BASE_PROMPT" "$ISSUE_TITLE" "$ISSUE_BODY" "$INSTRUCTION")
|
||||
echo "CUSTOM_PROMPT=$CUSTOM_PROMPT" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Run Claude PR Action
|
||||
uses: anthropics/claude-code-action@beta
|
||||
with:
|
||||
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
timeout_minutes: "60"
|
||||
allowed_tools: "mcp__github__create_pull_request"
|
||||
direct_prompt: ${{ steps.process_inputs.outputs.CUSTOM_PROMPT }}
|
||||
48
.github/workflows/pr-ready-review.yml
vendored
48
.github/workflows/pr-ready-review.yml
vendored
@@ -1,48 +0,0 @@
|
||||
name: Claude Auto Review
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
types: [ready_for_review, opened]
|
||||
|
||||
concurrency:
|
||||
group: claude-review-${{ github.event.pull_request.number }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
auto-review:
|
||||
runs-on: ubuntu-latest
|
||||
if: github.event.pull_request.draft == false || github.event.pull_request.ready_for_review == true
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: read
|
||||
id-token: write
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 1
|
||||
|
||||
- name: Automatic PR Review
|
||||
uses: anthropics/claude-code-action@v1
|
||||
with:
|
||||
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
|
||||
track_progress: true
|
||||
prompt: |
|
||||
REPO: ${{ github.repository }}
|
||||
PR NUMBER: ${{ github.event.pull_request.number }}
|
||||
|
||||
Please review this pull request and provide comprehensive feedback.
|
||||
|
||||
Focus on:
|
||||
- Code quality and best practices
|
||||
- Potential bugs or issues
|
||||
- Performance considerations
|
||||
- Security implications
|
||||
|
||||
Provide detailed feedback using inline comments for specific issues.
|
||||
Use top-level comments for general observations or praise.
|
||||
|
||||
At the end of your review, add complete instructions to reproduce the added changes through the app interface. These instructions will be given to a tester so he can verify the changes. It should be a short descriptive text (not a step by step or a list) on how to navigate the app (what page, what action, what input, etc) to see the changes.
|
||||
claude_args: |
|
||||
--allowedTools "mcp__github_inline_comment__create_inline_comment,Bash(gh pr comment:*),Bash(gh pr diff:*),Bash(gh pr view:*)"
|
||||
--model opus
|
||||
126
.github/workflows/publish_extra.yml
vendored
126
.github/workflows/publish_extra.yml
vendored
@@ -1,126 +0,0 @@
|
||||
env:
|
||||
REGISTRY: ghcr.io
|
||||
ECR_REGISTRY: 976079455550.dkr.ecr.us-east-1.amazonaws.com
|
||||
IMAGE_NAME: ${{ github.repository }}-extra
|
||||
|
||||
name: Publish windmill-extra
|
||||
on:
|
||||
push:
|
||||
tags:
|
||||
- "v*"
|
||||
workflow_dispatch:
|
||||
|
||||
permissions: write-all
|
||||
|
||||
jobs:
|
||||
sleep:
|
||||
runs-on: ubicloud
|
||||
steps:
|
||||
- name: Sleep for 900 seconds waiting for pypi to update index
|
||||
if: startsWith(github.ref, 'refs/tags/v')
|
||||
run: sleep 900
|
||||
shell: bash
|
||||
|
||||
# Build and test the image before publishing
|
||||
test_extra:
|
||||
needs: [sleep]
|
||||
runs-on: ubicloud-standard-8
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Setup Bun
|
||||
uses: oven-sh/setup-bun@v2
|
||||
with:
|
||||
bun-version: latest
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v3
|
||||
|
||||
- name: Build test image
|
||||
uses: docker/build-push-action@v5
|
||||
with:
|
||||
context: .
|
||||
file: ./docker/DockerfileExtra
|
||||
load: true
|
||||
tags: windmill-extra:test
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
- name: Start container
|
||||
run: |
|
||||
docker run -d --name windmill-extra-test \
|
||||
-p 3001:3001 -p 3002:3002 -p 3003:3003 \
|
||||
-e ENABLE_LSP=true \
|
||||
-e ENABLE_MULTIPLAYER=true \
|
||||
-e ENABLE_DEBUGGER=true \
|
||||
-e DEBUGGER_PORT=3003 \
|
||||
-e REQUIRE_SIGNED_DEBUG_REQUESTS=false \
|
||||
windmill-extra:test
|
||||
|
||||
# Wait for container to start
|
||||
echo "Waiting for container to initialize..."
|
||||
sleep 10
|
||||
|
||||
# Show container logs for debugging
|
||||
docker logs windmill-extra-test
|
||||
|
||||
- name: Run integration tests
|
||||
run: |
|
||||
bun run docker/test_windmill_extra.ts
|
||||
|
||||
- name: Show container logs on failure
|
||||
if: failure()
|
||||
run: |
|
||||
echo "=== Container logs ==="
|
||||
docker logs windmill-extra-test
|
||||
|
||||
- name: Cleanup
|
||||
if: always()
|
||||
run: |
|
||||
docker stop windmill-extra-test || true
|
||||
docker rm windmill-extra-test || true
|
||||
|
||||
publish_extra:
|
||||
needs: [sleep, test_extra]
|
||||
runs-on: ubicloud-standard-8
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- uses: depot/setup-action@v1
|
||||
|
||||
- name: Docker meta
|
||||
id: meta
|
||||
uses: docker/metadata-action@v5
|
||||
with:
|
||||
images: |
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
|
||||
tags: |
|
||||
type=ref,event=branch
|
||||
type=ref,event=pr
|
||||
type=semver,pattern={{version}}
|
||||
type=semver,pattern={{major}}.{{minor}}
|
||||
|
||||
- name: Login to registry
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
registry: ${{ env.REGISTRY }}
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Build and push publicly
|
||||
uses: depot/build-push-action@v1
|
||||
with:
|
||||
context: .
|
||||
file: ./docker/DockerfileExtra
|
||||
platforms: linux/amd64,linux/arm64
|
||||
push: true
|
||||
tags: |
|
||||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest
|
||||
${{ steps.meta.outputs.tags }}
|
||||
labels: |
|
||||
${{ steps.meta.outputs.labels }}
|
||||
org.opencontainers.image.licenses=AGPLv3
|
||||
25
.github/workflows/publish_windows_worker.yml
vendored
25
.github/workflows/publish_windows_worker.yml
vendored
@@ -13,7 +13,7 @@ env:
|
||||
|
||||
jobs:
|
||||
cargo_build_windows:
|
||||
runs-on: blacksmith-16vcpu-windows-2025
|
||||
runs-on: windows-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
@@ -32,24 +32,19 @@ jobs:
|
||||
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
|
||||
fetch-depth: 0
|
||||
|
||||
- uses: actions-rust-lang/setup-rust-toolchain@v1
|
||||
- name: Setup Rust
|
||||
uses: actions-rs/toolchain@v1
|
||||
with:
|
||||
cache-workspaces: backend
|
||||
toolchain: 1.93.0
|
||||
toolchain: 1.85.0
|
||||
override: true
|
||||
|
||||
- name: Substitute EE code
|
||||
shell: bash
|
||||
run: |
|
||||
./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
|
||||
|
||||
- name: Cargo build dynamic libraries windows
|
||||
timeout-minutes: 180
|
||||
run: |
|
||||
cd backend/windmill-duckdb-ffi-internal
|
||||
cargo build --release -p windmill_duckdb_ffi_internal
|
||||
|
||||
- name: Cargo build windows
|
||||
timeout-minutes: 180
|
||||
timeout-minutes: 90
|
||||
run: |
|
||||
vcpkg.exe install openssl-windows:x64-windows
|
||||
vcpkg.exe install openssl:x64-windows-static
|
||||
@@ -58,7 +53,7 @@ jobs:
|
||||
$env:OPENSSL_DIR="${Env:VCPKG_INSTALLATION_ROOT}\installed\x64-windows-static"
|
||||
mkdir frontend/build && cd backend
|
||||
New-Item -Path . -Name "windmill-api/openapi-deref.yaml" -ItemType "File" -Force
|
||||
cargo build --release --features=ee_windows
|
||||
cargo build --release --features=enterprise,stripe,embedding,parquet,prometheus,openidconnect,cloud,jemalloc,tantivy,license,http_trigger,zip,oauth2,kafka,sqs_trigger,nats,postgres_trigger,mqtt_trigger,gcp_trigger,websocket,smtp,static_frontend,all_languages,mcp
|
||||
- name: Rename binary with corresponding architecture
|
||||
run: |
|
||||
Rename-Item -Path ".\backend\target\release\windmill.exe" -NewName "windmill-ee.exe"
|
||||
@@ -68,9 +63,3 @@ jobs:
|
||||
with:
|
||||
files: |
|
||||
./backend/target/release/windmill-ee.exe
|
||||
|
||||
- name: Attach dynamic libraries to release
|
||||
uses: softprops/action-gh-release@v2
|
||||
with:
|
||||
files: |
|
||||
./backend/windmill-duckdb-ffi-internal/target/release/windmill_duckdb_ffi_internal.dll
|
||||
|
||||
27
.github/workflows/rust-client-check.yml
vendored
27
.github/workflows/rust-client-check.yml
vendored
@@ -1,27 +0,0 @@
|
||||
name: Rust Client Check
|
||||
on:
|
||||
workflow_run:
|
||||
workflows: ["Change versions"]
|
||||
types:
|
||||
- completed
|
||||
push:
|
||||
paths:
|
||||
- "rust-client/**"
|
||||
- "backend/**/*.rs"
|
||||
- "backend/windmill-api/openapi.yaml"
|
||||
- "version.txt"
|
||||
- "flake.nix"
|
||||
- ".github/workflows/rust-client-check.yml"
|
||||
|
||||
jobs:
|
||||
check_rust_client:
|
||||
runs-on: ubicloud-standard-8
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: cachix/install-nix-action@v20
|
||||
with:
|
||||
extra_nix_config: |
|
||||
experimental-features = nix-command flakes
|
||||
- name: Check rust client builds
|
||||
run: cd rust-client && nix develop ../ --command ./dev.nu --check
|
||||
timeout-minutes: 16
|
||||
19
.github/workflows/rust_on_release.yml
vendored
19
.github/workflows/rust_on_release.yml
vendored
@@ -1,19 +0,0 @@
|
||||
name: Publish rust-client to crates.io on release
|
||||
on:
|
||||
push:
|
||||
tags:
|
||||
- "v*"
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
build_rust_and_publish_to_crates_io:
|
||||
runs-on: ubicloud-standard-8
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: cachix/install-nix-action@v20
|
||||
with:
|
||||
extra_nix_config: |
|
||||
experimental-features = nix-command flakes
|
||||
- run: cd rust-client && nix develop ../ --command ./dev.nu --check --publish
|
||||
env:
|
||||
CRATES_IO_TOKEN: ${{ secrets.CRATES_IO_TOKEN }}
|
||||
@@ -38,45 +38,24 @@ jobs:
|
||||
- name: Send Discord notification and start a thread
|
||||
env:
|
||||
WEBHOOK_URL: ${{ secrets.DISCORD_WEBHOOK_URL }}
|
||||
BOT_TOKEN: ${{ secrets.DISCORD_BOT_TOKEN }}
|
||||
CHANNEL_ID: ${{ inputs.DISCORD_CHANNEL_ID }}
|
||||
GUILD_ID: ${{ inputs.DISCORD_GUILD_ID }}
|
||||
PR_TITLE: ${{ inputs.PR_TITLE }}
|
||||
PR_NUMBER: ${{ inputs.PR_NUMBER }}
|
||||
PR_URL: ${{ inputs.PR_URL }}
|
||||
PR_AUTHOR: ${{ inputs.PR_AUTHOR }}
|
||||
run: |
|
||||
# Check if thread already exists
|
||||
thread_exists=false
|
||||
if threads=$(curl -s -H "Authorization: Bot $BOT_TOKEN" "https://discord.com/api/v10/guilds/${GUILD_ID}/threads/active"); then
|
||||
if thread_id=$(echo "$threads" | jq -r --arg cid "$CHANNEL_ID" --arg pref "#${PR_NUMBER}:" '.threads[] | select(.parent_id == $cid and (.name | startswith($pref))) | .id' 2>/dev/null); then
|
||||
if [ -n "$thread_id" ]; then
|
||||
thread_exists=true
|
||||
echo "Thread already exists, skipping creation"
|
||||
fi
|
||||
fi
|
||||
else
|
||||
echo "Failed to check for existing threads, will create new thread"
|
||||
fi
|
||||
|
||||
# Create thread if it doesn't exist or if check failed
|
||||
if [ "$thread_exists" = false ]; then
|
||||
echo "Creating new thread"
|
||||
THREAD_TITLE="#${PR_NUMBER}: ${PR_TITLE} by \`${PR_AUTHOR}\`"
|
||||
payload=$(jq -n \
|
||||
--arg content "${PR_URL}" \
|
||||
--arg thread "${THREAD_TITLE:0:99}" \
|
||||
'{
|
||||
content: $content,
|
||||
thread_name: $thread,
|
||||
auto_archive_duration: 10080
|
||||
}'
|
||||
)
|
||||
curl -H "Content-Type: application/json" \
|
||||
-X POST \
|
||||
-d "$payload" \
|
||||
"$WEBHOOK_URL"
|
||||
fi
|
||||
payload=$(jq -n \
|
||||
--arg content "${PR_URL}" \
|
||||
--arg thread "#${PR_NUMBER}: $PR_TITLE by \`${PR_AUTHOR}\`" \
|
||||
'{
|
||||
content: $content,
|
||||
thread_name: $thread,
|
||||
auto_archive_duration: 10080
|
||||
}'
|
||||
)
|
||||
curl -H "Content-Type: application/json" \
|
||||
-X POST \
|
||||
-d "$payload" \
|
||||
"$WEBHOOK_URL"
|
||||
|
||||
merge_success_emoji:
|
||||
runs-on: ubuntu-latest
|
||||
@@ -105,7 +84,7 @@ jobs:
|
||||
fi
|
||||
# 2) get the first message in that thread
|
||||
messages=$(curl -H "Authorization: Bot $BOT_TOKEN" \
|
||||
"https://discord.com/api/v10/channels/$thread_id/messages")
|
||||
"https://discord.com/api/v10/channels/$thread_id/messages?limit=1")
|
||||
message_id=$(echo "$messages" | jq -r '.[-1].id')
|
||||
|
||||
if [ -z "$message_id" ]; then
|
||||
|
||||
148
.github/workflows/spawn-ephemeral-backend.yml
vendored
148
.github/workflows/spawn-ephemeral-backend.yml
vendored
@@ -1,148 +0,0 @@
|
||||
name: Spawn Ephemeral Backend
|
||||
|
||||
on:
|
||||
issue_comment:
|
||||
types: [created]
|
||||
pull_request_review_comment:
|
||||
types: [created]
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
pr_number:
|
||||
description: "PR number"
|
||||
required: true
|
||||
type: number
|
||||
|
||||
jobs:
|
||||
determine-commenter:
|
||||
if: |
|
||||
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '/spawnbackend')) ||
|
||||
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '/spawnbackend'))
|
||||
runs-on: ubicloud-standard-2
|
||||
outputs:
|
||||
commenter: ${{ steps.determine-commenter.outputs.commenter }}
|
||||
steps:
|
||||
- name: Determine commenter
|
||||
id: determine-commenter
|
||||
run: |
|
||||
# Work out who wrote the comment / review
|
||||
if [[ "${{ github.event_name }}" == "issue_comment" || \
|
||||
"${{ github.event_name }}" == "pull_request_review_comment" ]]; then
|
||||
COMMENTER="${{ github.event.comment.user.login }}"
|
||||
elif [[ "${{ github.event_name }}" == "pull_request_review" ]]; then
|
||||
COMMENTER="${{ github.event.review.user.login }}"
|
||||
else
|
||||
COMMENTER="${{ github.event.issue.user.login }}"
|
||||
fi
|
||||
echo "commenter=$COMMENTER" >> $GITHUB_OUTPUT
|
||||
|
||||
check-membership:
|
||||
needs: determine-commenter
|
||||
uses: ./.github/workflows/check-org-membership.yml
|
||||
with:
|
||||
commenter: ${{ needs.determine-commenter.outputs.commenter }}
|
||||
secrets:
|
||||
access_token: ${{ secrets.ORG_ACCESS_TOKEN }}
|
||||
|
||||
spawn-backend:
|
||||
needs: [determine-commenter, check-membership]
|
||||
# Only run on PR comments that contain /spawn-backend, or manual dispatch
|
||||
if: |
|
||||
github.event_name == 'workflow_dispatch' ||
|
||||
(github.event.issue.pull_request && needs.check-membership.outputs.is_member == 'true')
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
pull-requests: write
|
||||
contents: read
|
||||
|
||||
steps:
|
||||
- name: Get PR details
|
||||
id: pr-details
|
||||
uses: actions/github-script@v7
|
||||
with:
|
||||
script: |
|
||||
const prNumber = context.eventName === 'workflow_dispatch'
|
||||
? context.payload.inputs.pr_number
|
||||
: context.issue.number;
|
||||
|
||||
const pr = await github.rest.pulls.get({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
pull_number: prNumber
|
||||
});
|
||||
|
||||
// Get branch name and format it for Cloudflare Pages
|
||||
// Replace '/' with '-' for the URL
|
||||
const branchName = pr.data.head.ref;
|
||||
const formattedBranch = branchName.replace(/\//g, '-');
|
||||
const cfFrontendUrl = `https://${formattedBranch}.windmill.pages.dev`;
|
||||
|
||||
core.setOutput('commit_hash', pr.data.head.sha);
|
||||
core.setOutput('pr_number', prNumber);
|
||||
core.setOutput('branch_name', branchName);
|
||||
core.setOutput('cf_frontend_url', cfFrontendUrl);
|
||||
|
||||
- name: Check manager URL
|
||||
id: check-manager-url
|
||||
run: |
|
||||
if [ -z "${{ secrets.EPHEMERAL_BACKEND_QUEUE_URL }}" ]; then
|
||||
echo "manager_url_set=false" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "manager_url_set=true" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
- name: Post error comment if manager not running
|
||||
if: steps.check-manager-url.outputs.manager_url_set == 'false'
|
||||
uses: actions/github-script@v7
|
||||
with:
|
||||
script: |
|
||||
const prNumber = context.eventName === 'workflow_dispatch'
|
||||
? Number(context.payload.inputs.pr_number)
|
||||
: context.issue.number;
|
||||
|
||||
await github.rest.issues.createComment({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: prNumber,
|
||||
body: `❌ Manager URL not set (did you start the ephemeral backend manager?)\n\nThe ephemeral backend manager needs to be running to spawn backends. Please start the manager first.`
|
||||
});
|
||||
|
||||
- name: Fail if manager not running
|
||||
if: steps.check-manager-url.outputs.manager_url_set == 'false'
|
||||
run: |
|
||||
echo "Error: EPHEMERAL_BACKEND_QUEUE_URL secret is not set"
|
||||
exit 1
|
||||
|
||||
- name: Trigger Windmill flow
|
||||
if: steps.check-manager-url.outputs.manager_url_set == 'true'
|
||||
id: trigger-flow
|
||||
run: |
|
||||
JOB_UUID=$(curl -s -X POST "https://app.windmill.dev/api/w/windmill-labs/jobs/run/f/f/all/run_ephemeral_backend" \
|
||||
-H "Authorization: Bearer ${{ secrets.WINDMILL_RUN_FLOW_TOKEN }}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"manager_url": "${{ secrets.EPHEMERAL_BACKEND_QUEUE_URL }}",
|
||||
"commit_hash": "${{ steps.pr-details.outputs.commit_hash }}",
|
||||
"pr_number": ${{ steps.pr-details.outputs.pr_number }},
|
||||
"cf_frontend_url": "${{ steps.pr-details.outputs.cf_frontend_url }}"
|
||||
}' | tr -d '"')
|
||||
|
||||
echo "Job UUID: $JOB_UUID"
|
||||
echo "job_uuid=$JOB_UUID" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Post comment with job link
|
||||
if: steps.check-manager-url.outputs.manager_url_set == 'true'
|
||||
uses: actions/github-script@v7
|
||||
with:
|
||||
script: |
|
||||
const jobUuid = '${{ steps.trigger-flow.outputs.job_uuid }}';
|
||||
const appUrl = `https://app.windmill.dev/public/windmill-labs/a106bad0256c1dfa7a4f9279c42b1a4b#${jobUuid}`;
|
||||
const prNumber = context.eventName === 'workflow_dispatch'
|
||||
? Number(context.payload.inputs.pr_number)
|
||||
: context.issue.number;
|
||||
|
||||
await github.rest.issues.createComment({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: prNumber,
|
||||
body: `🚀 Spawning new ephemeral backend!\n\n${appUrl}`
|
||||
});
|
||||
34
.github/workflows/validate-openapi.yml
vendored
34
.github/workflows/validate-openapi.yml
vendored
@@ -1,34 +0,0 @@
|
||||
name: Validate OpenAPI Spec
|
||||
|
||||
on:
|
||||
push:
|
||||
paths:
|
||||
- 'backend/windmill-api/openapi*'
|
||||
pull_request:
|
||||
paths:
|
||||
- 'backend/windmill-api/openapi*'
|
||||
jobs:
|
||||
validate:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '20'
|
||||
|
||||
- name: Install openapi-generator-cli
|
||||
run: npm install @openapitools/openapi-generator-cli -g
|
||||
|
||||
- name: Validate openapi.yaml
|
||||
run: npx @openapitools/openapi-generator-cli validate -i backend/windmill-api/openapi.yaml
|
||||
|
||||
- name: Validate openapi-deref.json
|
||||
run: npx @openapitools/openapi-generator-cli validate -i backend/windmill-api/openapi-deref.json
|
||||
|
||||
# Does not work well with dereferenced yaml
|
||||
# - name: Validate openapi-deref.yaml
|
||||
# run: npx @openapitools/openapi-generator-cli validate -i backend/windmill-api/openapi-deref.yaml
|
||||
|
||||
145
.github/workflows/weekly-pr-summary.yml
vendored
145
.github/workflows/weekly-pr-summary.yml
vendored
@@ -1,145 +0,0 @@
|
||||
name: Weekly PR Summary
|
||||
|
||||
on:
|
||||
schedule:
|
||||
# Every Friday at 8:00 AM UTC
|
||||
- cron: '0 8 * * 5'
|
||||
workflow_dispatch:
|
||||
# Allow manual triggering for testing
|
||||
|
||||
jobs:
|
||||
weekly-pr-summary:
|
||||
runs-on: ubicloud-standard-4
|
||||
timeout-minutes: 30
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: read
|
||||
issues: write
|
||||
id-token: write
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 1
|
||||
|
||||
- name: Generate Weekly PR Summary
|
||||
uses: anthropics/claude-code-action@v1
|
||||
with:
|
||||
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
|
||||
prompt: |
|
||||
REPO: ${{ github.repository }}
|
||||
|
||||
Generate a categorized weekly summary of ONLY MERGED Pull Requests from the past 7 days.
|
||||
|
||||
## Your Task:
|
||||
|
||||
1. **Calculate Date Range**:
|
||||
- Run: `CUTOFF_DATE=$(date --date='7 days ago' --iso-8601)`
|
||||
- Run: `TODAY=$(date --iso-8601)`
|
||||
- This gives you the exact 7-day window (store these in variables for use in commands)
|
||||
|
||||
2. **Fetch ONLY Merged PRs from Past Week**:
|
||||
- Command: `gh pr list --repo ${{ github.repository }} --state merged --search "merged:>=$CUTOFF_DATE" --limit 100 --json number,title,author,mergedAt,url`
|
||||
- This returns ONLY PRs that were merged in the last 7 days
|
||||
- The --search flag filters by merge date using GitHub's search syntax
|
||||
- **FILTER OUT** any PRs with titles starting with "chore: release" or "chore(release)"
|
||||
|
||||
3. **Categorize PRs**: Group PRs into three categories by analyzing titles and labels:
|
||||
- **Features**: PRs with titles starting with "feat:", "feature:", or containing "add", "implement", "new"
|
||||
- **Bug Fixes**: PRs with titles starting with "fix:", "bug:", or containing "fix", "resolve", "patch"
|
||||
- **Other**: All remaining PRs (improvements, refactors, docs, chores, etc.)
|
||||
|
||||
4. **Gather Details**: For each feature and bug fix merged PR, include:
|
||||
- Full PR title (NO truncation, NO links)
|
||||
- Author (extract login from author.login in JSON)
|
||||
- Brief summary: Use `gh pr view <number> --json body` to get PR description, then extract first paragraph or key points (1-2 sentences max)
|
||||
|
||||
5. **Character Limit Enforcement**:
|
||||
- The final summary MUST be under 5000 characters
|
||||
- If the summary exceeds 5000 characters, truncate PR descriptions (NOT titles) and add at the end: "_and X more PRs_" where X is the count of omitted PRs
|
||||
|
||||
6. **Save Summary to Markdown File**: Write the summary to a file for webhook delivery:
|
||||
- Save the complete formatted markdown to: `summary.md`
|
||||
- Do not commit the file to the repository
|
||||
|
||||
## Output Format:
|
||||
|
||||
```markdown
|
||||
### 📈 Weekly overview
|
||||
- **Total merged**: X
|
||||
- **Features**: Y
|
||||
- **Bug Fixes**: Z
|
||||
- **Other**: W
|
||||
|
||||
### ✨ Features (Y)
|
||||
- **[Full PR Title]** by @username - [brief impact description]
|
||||
- **[Full PR Title]** by @username - [brief impact description]
|
||||
|
||||
### 🐛 Bug Fixes (Z)
|
||||
- **[Full PR Title]** by @username - [brief impact description]
|
||||
- **[Full PR Title]** by @username - [brief impact description]
|
||||
|
||||
_and X more PRs_
|
||||
```
|
||||
|
||||
## Important Notes:
|
||||
- **CRITICAL**: ONLY include PRs with state "merged" from the last 7 days
|
||||
- **CRITICAL**: EXCLUDE all PRs with titles starting with "chore: release" or "chore(release)"
|
||||
- **CRITICAL**: Total character count MUST be under 5000 characters
|
||||
- Count the number of "Other" PRs but do not include a section for them in the output
|
||||
- Only use ### markdown headers for major sections and emoji indicators
|
||||
- NO links to PRs
|
||||
- NO merged date in output
|
||||
- NEVER truncate PR titles - show full titles
|
||||
- Use GitHub CLI (`gh`) for all operations
|
||||
- Sort PRs within each category by merge date (most recent first)
|
||||
- If a PR has no description, write "(No description provided)"
|
||||
- Extract meaningful summary from PR body - look for the first paragraph or key bullet points
|
||||
- Parse JSON responses carefully using `jq` or similar tools
|
||||
- If summary exceeds 5000 chars, shorten PR descriptions and add "_and X more PRs_" at the end
|
||||
- Count PRs in each category and display in both overview and section headers
|
||||
|
||||
## Saving the Markdown Output:
|
||||
After generating the markdown summary, save it to a file, BUT DO NOT COMMIT IT TO THE REPOSITORY.
|
||||
|
||||
## Write Tool Fallback:
|
||||
- First, attempt to use the Write tool to create `summary.md` with the markdown content
|
||||
- If the Write tool returns ANY error or fails:
|
||||
1. Use the Bash tool with the `echo` command instead
|
||||
2. Use a heredoc to write the content: `cat > summary.md << 'EOF'` followed by your markdown content and `EOF` on a new line
|
||||
3. Example: `cat > summary.md << 'EOF'\n[your markdown content here]\nEOF`
|
||||
4. This ensures the file is always created regardless of Write tool issues
|
||||
- Verify the file was created by running: `ls -lh summary.md`
|
||||
claude_args: |
|
||||
--allowedTools "Edit,MultiEdit,Write,Read,Glob,Grep,LS,Bash"
|
||||
--model haiku
|
||||
|
||||
- name: Send Summary to Windmill
|
||||
if: hashFiles('summary.md') != ''
|
||||
env:
|
||||
WEEKLY_SUMMARY_TOKEN: ${{ secrets.WEEKLY_SUMMARY_TOKEN }}
|
||||
run: |
|
||||
if [[ -f "summary.md" ]]; then
|
||||
echo "Found summary.md, sending to Windmill..."
|
||||
|
||||
# Read the markdown content
|
||||
MARKDOWN_CONTENT=$(cat summary.md)
|
||||
|
||||
# Create JSON payload
|
||||
PAYLOAD=$(jq -n --arg markdown "$MARKDOWN_CONTENT" '{markdown: $markdown}')
|
||||
|
||||
# Send to Windmill webhook
|
||||
RESULT=$(curl -s \
|
||||
-H 'Content-Type: application/json' \
|
||||
-H "Authorization: Bearer $WEEKLY_SUMMARY_TOKEN" \
|
||||
-X POST \
|
||||
-d "$PAYLOAD" \
|
||||
'https://app.windmill.dev/api/w/windmill-labs/jobs/run/f/f/ai/send_past_week_pr_summaries_to_discord')
|
||||
|
||||
echo "Windmill response:"
|
||||
echo -E "$RESULT"
|
||||
echo "✅ Summary sent successfully to Windmill!"
|
||||
else
|
||||
echo "⚠️ Warning: summary.md not found, skipping delivery"
|
||||
exit 1
|
||||
fi
|
||||
12
.gitignore
vendored
12
.gitignore
vendored
@@ -7,16 +7,8 @@ CaddyfileRemoteMalo
|
||||
*.swp
|
||||
**/.idea/
|
||||
.direnv
|
||||
/.vscode
|
||||
.vscode
|
||||
.dev-docker-wrapper*
|
||||
backend/.minio-data
|
||||
.aider*
|
||||
!.aiderignore
|
||||
rust-client/Cargo.toml
|
||||
|
||||
# Symlinked cache directories (for git worktrees)
|
||||
backend/target
|
||||
frontend/node_modules
|
||||
typescript-client/node_modules
|
||||
frontend/.svelte-kit
|
||||
backend/chrome_profiler.json
|
||||
!.aiderignore
|
||||
@@ -1,8 +0,0 @@
|
||||
{
|
||||
"mcpServers": {
|
||||
"svelte": {
|
||||
"type": "http",
|
||||
"url": "https://mcp.svelte.dev/mcp"
|
||||
}
|
||||
}
|
||||
}
|
||||
3330
CHANGELOG.md
3330
CHANGELOG.md
File diff suppressed because it is too large
Load Diff
35
CLAUDE.md
35
CLAUDE.md
@@ -1,35 +0,0 @@
|
||||
# Windmill Development Guide
|
||||
|
||||
## Overview
|
||||
|
||||
Windmill is an open-source developer platform for building internal tools, workflows, API integrations, background jobs, workflows, and user interfaces. See @windmill-overview.mdc for full platform details.
|
||||
|
||||
## New Feature Implementation Guidelines
|
||||
|
||||
When implementing new features in Windmill, follow these best practices:
|
||||
|
||||
- **Clean Code First**: Write clean, readable, and maintainable code. Prioritize clarity over cleverness.
|
||||
- **Avoid Duplication at All Costs**: Before writing new code, thoroughly search for existing implementations that can be reused or extended.
|
||||
- **Adapt Existing Code**: Refactor and generalize existing code when necessary to avoid logic duplication. Extract common patterns into reusable utilities.
|
||||
- **Follow Established Patterns**: Study existing code patterns in the codebase and maintain consistency with established conventions.
|
||||
- **Single Responsibility**: Each function, component, and module should have a single, well-defined responsibility.
|
||||
- **Incremental Implementation**: Break large features into smaller, reviewable chunks that can be implemented and tested incrementally.
|
||||
|
||||
## Language-Specific Guides
|
||||
|
||||
- Backend (Rust): @backend/rust-best-practices.mdc + @backend/summarized_schema.txt
|
||||
- Frontend (Svelte 5): @frontend/svelte5-best-practices.mdc
|
||||
|
||||
## Querying the Database
|
||||
|
||||
To query the database directly, use psql with the following connection string:
|
||||
|
||||
```bash
|
||||
psql postgres://postgres:changeme@localhost:5432/windmill
|
||||
```
|
||||
|
||||
This can be helpful for:
|
||||
|
||||
- Inspecting database state during development
|
||||
- Testing queries before implementing them in Rust
|
||||
- Debugging data-related issues
|
||||
21
Caddyfile
21
Caddyfile
@@ -10,26 +10,9 @@
|
||||
|
||||
{$BASE_URL} {
|
||||
bind {$ADDRESS}
|
||||
|
||||
# LSP - Language Server Protocol for code intelligence (windmill_extra:3001)
|
||||
reverse_proxy /ws/* http://windmill_extra:3001
|
||||
|
||||
# Multiplayer - Real-time collaboration, Enterprise Edition (windmill_extra:3002)
|
||||
# Uncomment and set ENABLE_MULTIPLAYER=true in docker-compose.yml
|
||||
# reverse_proxy /ws_mp/* http://windmill_extra:3002
|
||||
|
||||
# Debugger - Interactive debugging via DAP WebSocket (windmill_extra:3003)
|
||||
# Set ENABLE_DEBUGGER=true in docker-compose.yml to enable
|
||||
handle_path /ws_debug/* {
|
||||
reverse_proxy http://windmill_extra:3003
|
||||
}
|
||||
|
||||
# Search indexer, Enterprise Edition (windmill_indexer:8002)
|
||||
reverse_proxy /ws/* http://lsp:3001
|
||||
# reverse_proxy /ws_mp/* http://multiplayer:3002
|
||||
# reverse_proxy /api/srch/* http://windmill_indexer:8002
|
||||
|
||||
# Default: Windmill server
|
||||
reverse_proxy /* http://windmill_server:8000
|
||||
|
||||
# TLS with custom certificates
|
||||
# tls /certs/cert.pem /certs/key.pem
|
||||
}
|
||||
|
||||
123
Dockerfile
123
Dockerfile
@@ -1,29 +1,9 @@
|
||||
ARG DEBIAN_IMAGE=debian:bookworm-slim
|
||||
ARG RUST_IMAGE=rust:1.93-slim-bookworm
|
||||
|
||||
FROM debian:bookworm-slim AS nsjail
|
||||
|
||||
WORKDIR /nsjail
|
||||
|
||||
RUN apt-get -y update \
|
||||
&& apt-get install -y \
|
||||
bison=2:3.8.* \
|
||||
flex=2.6.* \
|
||||
g++=4:12.2.* \
|
||||
gcc=4:12.2.* \
|
||||
git=1:2.39.* \
|
||||
libprotobuf-dev=3.21.* \
|
||||
libnl-route-3-dev=3.7.* \
|
||||
make=4.3-4.1 \
|
||||
pkg-config=1.8.* \
|
||||
protobuf-compiler=3.21.*
|
||||
|
||||
RUN git clone -b master --single-branch https://github.com/google/nsjail.git . && git checkout dccf911fd2659e7b08ce9507c25b2b38ec2c5800
|
||||
RUN make
|
||||
ARG RUST_IMAGE=rust:1.86-slim-bookworm
|
||||
|
||||
FROM ${RUST_IMAGE} AS rust_base
|
||||
|
||||
RUN apt-get update && apt-get install -y git libssl-dev pkg-config npm mold clang
|
||||
RUN apt-get update && apt-get install -y git libssl-dev pkg-config npm
|
||||
|
||||
RUN apt-get -y update \
|
||||
&& apt-get install -y \
|
||||
@@ -40,21 +20,7 @@ WORKDIR /windmill
|
||||
ENV SQLX_OFFLINE=true
|
||||
# ENV CARGO_INCREMENTAL=1
|
||||
|
||||
FROM rust_base AS windmill_duckdb_ffi_internal_builder
|
||||
|
||||
WORKDIR /windmill-duckdb-ffi-internal
|
||||
|
||||
RUN apt-get update && apt-get install -y clang=1:14.0-55.* libclang-dev=1:14.0-55.* cmake=3.25.* && \
|
||||
apt-get clean && \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
|
||||
COPY ./backend/windmill-duckdb-ffi-internal .
|
||||
|
||||
RUN --mount=type=cache,target=/usr/local/cargo/registry \
|
||||
--mount=type=cache,target=$SCCACHE_DIR,sharing=locked \
|
||||
cargo build --release -p windmill_duckdb_ffi_internal
|
||||
|
||||
FROM node:24-alpine as frontend
|
||||
FROM node:20-alpine as frontend
|
||||
|
||||
# install dependencies
|
||||
WORKDIR /frontend
|
||||
@@ -68,18 +34,16 @@ RUN mkdir /backend
|
||||
COPY /backend/windmill-api/openapi.yaml /backend/windmill-api/openapi.yaml
|
||||
COPY /openflow.openapi.yaml /openflow.openapi.yaml
|
||||
COPY /backend/windmill-api/build_openapi.sh /backend/windmill-api/build_openapi.sh
|
||||
COPY /system_prompts/auto-generated /system_prompts/auto-generated
|
||||
|
||||
RUN cd /backend/windmill-api && . ./build_openapi.sh
|
||||
COPY /backend/parsers/windmill-parser-wasm/pkg/ /backend/parsers/windmill-parser-wasm/pkg/
|
||||
COPY /typescript-client/docs/ /frontend/static/tsdocs/
|
||||
COPY /python-client/docs/ /frontend/static/pydocs/
|
||||
|
||||
RUN npm run generate-backend-client
|
||||
ENV NODE_OPTIONS "--max-old-space-size=8192"
|
||||
ARG VITE_BASE_URL ""
|
||||
# Read more about macro in docker/dev.nu
|
||||
# -- MACRO-SPREAD-WASM-PARSER-DEV-ONLY -- #
|
||||
# -- MACRO-SPREAD-WASM-PARSER-DEV-ONLY -- #
|
||||
RUN npm run build
|
||||
|
||||
|
||||
@@ -97,7 +61,7 @@ ARG features=""
|
||||
|
||||
COPY --from=planner /windmill/recipe.json recipe.json
|
||||
|
||||
RUN apt-get update && apt-get install -y libxml2-dev=2.9.* libxmlsec1-dev=1.2.* libkrb5-dev libsasl2-dev libcurl4-openssl-dev clang=1:14.0-55.* libclang-dev=1:14.0-55.* cmake=3.25.* && \
|
||||
RUN apt-get update && apt-get install -y libxml2-dev=2.9.* libxmlsec1-dev=1.2.* clang=1:14.0-55.* libclang-dev=1:14.0-55.* cmake=3.25.* && \
|
||||
apt-get clean && \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
|
||||
@@ -118,6 +82,7 @@ RUN --mount=type=cache,target=/usr/local/cargo/registry \
|
||||
--mount=type=cache,target=$SCCACHE_DIR,sharing=locked \
|
||||
CARGO_NET_GIT_FETCH_WITH_CLI=true cargo build --release --features "$features"
|
||||
|
||||
|
||||
FROM ${DEBIAN_IMAGE}
|
||||
|
||||
ARG TARGETPLATFORM
|
||||
@@ -125,8 +90,7 @@ ARG POWERSHELL_VERSION=7.5.0
|
||||
ARG POWERSHELL_DEB_VERSION=7.5.0-1
|
||||
ARG KUBECTL_VERSION=1.28.7
|
||||
ARG HELM_VERSION=3.14.3
|
||||
# NOTE: If changing, also change go version in workspace dependencies template at WorkspaceDependenciesEditor.svelte
|
||||
ARG GO_VERSION=1.25.0
|
||||
ARG GO_VERSION=1.22.5
|
||||
ARG APP=/usr/src/app
|
||||
ARG WITH_POWERSHELL=true
|
||||
ARG WITH_KUBECTL=true
|
||||
@@ -137,19 +101,16 @@ ARG WITH_GIT=true
|
||||
# 1. Change placeholder in instanceSettings.ts
|
||||
# 2. Change LATEST_STABLE_PY in dockerfile
|
||||
# 3. Change #[default] annotation for PyVersion in backend
|
||||
ARG LATEST_STABLE_PY=3.12
|
||||
ARG LATEST_STABLE_PY=3.11.10
|
||||
ENV UV_PYTHON_INSTALL_DIR=/tmp/windmill/cache/py_runtime
|
||||
ENV UV_PYTHON_PREFERENCE=only-managed
|
||||
|
||||
RUN mkdir -p /usr/local/uv
|
||||
ENV UV_TOOL_BIN_DIR=/usr/local/bin
|
||||
ENV UV_TOOL_DIR=/usr/local/uv
|
||||
|
||||
ENV PATH /usr/local/bin:/root/.local/bin:/tmp/.local/bin:$PATH
|
||||
ENV PATH /usr/local/bin:/root/.local/bin:$PATH
|
||||
|
||||
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends netbase tzdata ca-certificates wget curl jq unzip build-essential unixodbc xmlsec1 software-properties-common tini libsasl2-modules-gssapi-mit \
|
||||
&& apt-get install -y --no-install-recommends netbase tzdata ca-certificates wget curl jq unzip build-essential unixodbc xmlsec1 software-properties-common \
|
||||
&& apt-get clean \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
@@ -210,51 +171,30 @@ ENV PATH="${PATH}:/usr/local/go/bin"
|
||||
ENV GO_PATH=/usr/local/go/bin/go
|
||||
|
||||
# Install UV
|
||||
RUN curl --proto '=https' --tlsv1.2 -LsSf https://github.com/astral-sh/uv/releases/download/0.9.24/uv-installer.sh | sh && mv /root/.local/bin/uv /usr/local/bin/uv
|
||||
RUN curl --proto '=https' --tlsv1.2 -LsSf https://github.com/astral-sh/uv/releases/download/0.6.2/uv-installer.sh | sh && mv /root/.local/bin/uv /usr/local/bin/uv
|
||||
|
||||
# Preinstall python runtimes to temp build location (will copy with world-writable perms later)
|
||||
RUN UV_CACHE_DIR=/tmp/build_cache/uv UV_PYTHON_INSTALL_DIR=/tmp/build_cache/py_runtime uv python install 3.11
|
||||
RUN UV_CACHE_DIR=/tmp/build_cache/uv UV_PYTHON_INSTALL_DIR=/tmp/build_cache/py_runtime uv python install $LATEST_STABLE_PY
|
||||
# Preinstall python runtimes
|
||||
RUN uv python install 3.11
|
||||
RUN uv python install $LATEST_STABLE_PY
|
||||
|
||||
RUN uv venv
|
||||
|
||||
|
||||
RUN curl -sL https://deb.nodesource.com/setup_20.x | bash -
|
||||
RUN curl -sL https://deb.nodesource.com/setup_20.x | bash -
|
||||
RUN apt-get -y update && apt-get install -y curl procps nodejs awscli && apt-get clean \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# go build is slower the first time it is ran, so we prewarm it in the build
|
||||
# This mirrors Windmill's Go wrapper structure: main.go imports inner package, uses encoding/json, os, fmt
|
||||
RUN export GOCACHE=/tmp/build_cache/go && \
|
||||
mkdir -p /tmp/gobuildwarm/inner && \
|
||||
cd /tmp/gobuildwarm && \
|
||||
go mod init mymod && \
|
||||
printf 'package main\nimport (\n\t"encoding/json"\n\t"os"\n\t"fmt"\n\t"mymod/inner"\n)\nfunc main() {\n\tdat, _ := os.ReadFile("args.json")\n\tvar req inner.Req\n\tjson.Unmarshal(dat, &req)\n\tres, _ := inner.Run(req)\n\tres_json, _ := json.Marshal(res)\n\tfmt.Println(string(res_json))\n}' > main.go && \
|
||||
printf 'package inner\ntype Req struct {\n\tX int `json:"x"`\n}\nfunc Run(req Req) (interface{}, error) {\n\treturn main(req.X)\n}\nfunc main(x int) (interface{}, error) {\n\treturn x, nil\n}' > inner/inner.go && \
|
||||
go build -x . && \
|
||||
rm -rf /tmp/gobuildwarm
|
||||
|
||||
# Copy build caches to final location, then add write permissions for any UID
|
||||
# chmod a+rw adds read+write WITHOUT removing execute bits (755->777, 644->666)
|
||||
# Note: uv python install only creates py_runtime, not uv cache - we create uv/go dirs for runtime
|
||||
RUN mkdir -p /tmp/windmill/cache && \
|
||||
cp -r /tmp/build_cache/* /tmp/windmill/cache/ && \
|
||||
chmod -R a+rw /tmp/windmill/cache && \
|
||||
rm -rf /tmp/build_cache && \
|
||||
mkdir -p -m 777 /tmp/windmill/cache/uv /tmp/windmill/cache/go
|
||||
|
||||
# Runtime cache locations
|
||||
ENV UV_CACHE_DIR=/tmp/windmill/cache/uv
|
||||
ENV UV_PYTHON_INSTALL_DIR=/tmp/windmill/cache/py_runtime
|
||||
ENV GOCACHE=/tmp/windmill/cache/go
|
||||
RUN mkdir -p /tmp/gobuildwarm && cd /tmp/gobuildwarm && go mod init gobuildwarm && printf "package foo\nimport (\"fmt\")\nfunc main() { fmt.Println(42) }" > warm.go && go mod tidy && go build -x && rm -rf /tmp/gobuildwarm
|
||||
|
||||
ENV TZ=Etc/UTC
|
||||
|
||||
COPY --from=builder /frontend/build /static_frontend
|
||||
COPY --from=builder /windmill/target/release/windmill ${APP}/windmill
|
||||
COPY --from=windmill_duckdb_ffi_internal_builder /windmill-duckdb-ffi-internal/target/release/libwindmill_duckdb_ffi_internal.so ${APP}/libwindmill_duckdb_ffi_internal.so
|
||||
|
||||
COPY --from=denoland/deno:2.2.1 --chmod=755 /usr/bin/deno /usr/bin/deno
|
||||
|
||||
COPY --from=oven/bun:1.3.8 /usr/local/bin/bun /usr/bin/bun
|
||||
COPY --from=oven/bun:1.2.4 /usr/local/bin/bun /usr/bin/bun
|
||||
|
||||
COPY --from=php:8.3.7-cli /usr/local/bin/php /usr/bin/php
|
||||
COPY --from=composer:2.7.6 /usr/bin/composer /usr/bin/composer
|
||||
@@ -264,12 +204,6 @@ COPY --from=docker:dind /usr/local/bin/docker /usr/local/bin/
|
||||
|
||||
ENV RUSTUP_HOME="/usr/local/rustup"
|
||||
ENV CARGO_HOME="/usr/local/cargo"
|
||||
ENV LD_LIBRARY_PATH="."
|
||||
|
||||
# nsjail runtime deps and binary
|
||||
RUN apt-get update && apt-get install -y libprotobuf-dev libnl-route-3-dev \
|
||||
&& apt-get clean && rm -rf /var/lib/apt/lists/*
|
||||
COPY --from=nsjail /nsjail/nsjail /bin/nsjail
|
||||
|
||||
WORKDIR ${APP}
|
||||
|
||||
@@ -277,20 +211,23 @@ RUN ln -s ${APP}/windmill /usr/local/bin/windmill
|
||||
|
||||
COPY ./frontend/src/lib/hubPaths.json ${APP}/hubPaths.json
|
||||
|
||||
RUN windmill cache ${APP}/hubPaths.json && rm ${APP}/hubPaths.json
|
||||
|
||||
RUN windmill cache-rt
|
||||
RUN windmill cache ${APP}/hubPaths.json && rm ${APP}/hubPaths.json && chmod -R 777 /tmp/windmill
|
||||
|
||||
# Create a non-root user 'windmill' with UID and GID 1000
|
||||
RUN addgroup --gid 1000 windmill && \
|
||||
adduser --disabled-password --gecos "" --uid 1000 --gid 1000 windmill
|
||||
|
||||
# /tmp/.cache may be created by earlier build steps with 755; chmod ensures any UID can write
|
||||
RUN mkdir -p -m 777 /tmp/windmill/logs /tmp/windmill/search /tmp/.cache && chmod 777 /tmp/.cache
|
||||
RUN cp -r /root/.cache /home/windmill/.cache
|
||||
|
||||
# Make directories world-accessible for any UID
|
||||
# (cache files already have 666 from umask copy above, cache_nomount is read-only)
|
||||
RUN find ${APP} /tmp/windmill -type d -exec chmod 777 {} +
|
||||
RUN mkdir -p /tmp/windmill/logs && \
|
||||
mkdir -p /tmp/windmill/search
|
||||
|
||||
# Make directories world-readable and writable
|
||||
RUN chmod -R 777 ${APP} && \
|
||||
chmod -R 777 /tmp/windmill && \
|
||||
chmod -R 777 /home/windmill/.cache
|
||||
|
||||
USER root
|
||||
|
||||
EXPOSE 8000
|
||||
|
||||
|
||||
335
README.md
335
README.md
@@ -3,10 +3,10 @@
|
||||
</p>
|
||||
|
||||
<p align=center>
|
||||
Open-source developer platform for internal code: APIs, background jobs, workflows and UIs. Self-hostable alternative to Retool, Pipedream, Superblocks and a simplified Temporal with autogenerated UIs and custom UIs to trigger workflows and scripts as internal apps.
|
||||
Open-source developer infrastructure for internal tools (APIs, background jobs, workflows and UIs). Self-hostable alternative to Retool, Pipedream, Superblocks and a simplified Temporal with autogenerated UIs and custom UIs to trigger workflows and scripts as internal apps.
|
||||
|
||||
<p align=center>
|
||||
Scripts are turned into sharable UIs automatically, and can be composed together into flows or used into richer apps built with low-code. Supported languages: Python, TypeScript, Go, Bash, SQL, GraphQL, PowerShell, Rust, and more.
|
||||
Scripts are turned into sharable UIs automatically, and can be composed together into flows or used into richer apps built with low-code. Supported script languages supported are: Python, TypeScript, Go, Bash, SQL, and GraphQL.
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
@@ -36,58 +36,75 @@ Scripts are turned into sharable UIs automatically, and can be composed together
|
||||
|
||||
# Windmill - Developer platform for APIs, background jobs, workflows and UIs
|
||||
|
||||
Windmill is fully open-sourced (AGPLv3) and Windmill Labs offers dedicated instances and commercial support and licenses.
|
||||
Windmill is <b>fully open-sourced (AGPLv3)</b> and Windmill Labs offers
|
||||
dedicated instance and commercial support and licenses.
|
||||
|
||||

|
||||
|
||||
https://github.com/user-attachments/assets/d80de1d9-64de-4d89-aacd-6df23fa81fc4
|
||||
https://github.com/windmill-labs/windmill/assets/122811744/0b132cd1-ee67-4505-822f-0c7ee7104252
|
||||
|
||||
- [Windmill - Developer platform for APIs, background jobs, workflows and UIs](#windmill---developer-platform-for-apis-background-jobs-workflows-and-uis)
|
||||
- [Main Concepts](#main-concepts)
|
||||
- [Show me some actual script code](#show-me-some-actual-script-code)
|
||||
- [Local Development](#local-development)
|
||||
- [CLI](#cli)
|
||||
- [Running scripts locally](#running-scripts-locally)
|
||||
- [Stack](#stack)
|
||||
- [Fastest Self-Hostable Workflow Engine](#fastest-self-hostable-workflow-engine)
|
||||
- [Security](#security)
|
||||
- [Sandboxing](#sandboxing)
|
||||
- [Secrets, credentials and sensitive values](#secrets-credentials-and-sensitive-values)
|
||||
- [Performance](#performance)
|
||||
- [Architecture](#architecture)
|
||||
- [How to self-host](#how-to-self-host)
|
||||
- [Docker compose](#docker-compose)
|
||||
- [Kubernetes (Helm charts)](#kubernetes-helm-charts)
|
||||
- [Cloud providers](#cloud-providers)
|
||||
- [Kubernetes (k8s) and Helm charts](#kubernetes-k8s-and-helm-charts)
|
||||
- [Run from binaries](#run-from-binaries)
|
||||
- [OAuth, SSO \& SMTP](#oauth-sso--smtp)
|
||||
- [License](#license)
|
||||
- [Commercial license](#commercial-license)
|
||||
- [Integrations](#integrations)
|
||||
- [Environment Variables](#environment-variables)
|
||||
- [Run a local dev setup](#run-a-local-dev-setup)
|
||||
- [Frontend only](#frontend-only)
|
||||
- [only Frontend](#only-frontend)
|
||||
- [Backend + Frontend](#backend--frontend)
|
||||
- [Contributors](#contributors)
|
||||
- [Copyright](#copyright)
|
||||
|
||||
## Main Concepts
|
||||
|
||||
1. Define a minimal and generic script in Python, TypeScript, Go or Bash that solves a specific task. The code can be defined in the provided Web IDE or synchronized with your own GitHub repo (e.g. through VS Code extension): [provided Web IDE](https://www.windmill.dev/docs/code_editor) or [synchronized with your own GitHub repo](https://www.windmill.dev/docs/advanced/cli/sync) (e.g. through [VS Code](https://www.windmill.dev/docs/cli_local_dev/vscode-extension) extension):
|
||||
1. Define a minimal and generic script in Python, TypeScript, Go or Bash that
|
||||
solves a specific task. The code can be defined in the
|
||||
[provided Web IDE](https://www.windmill.dev/docs/code_editor) or
|
||||
[synchronized with your own GitHub repo](https://www.windmill.dev/docs/advanced/cli/sync)
|
||||
(e.g. through
|
||||
[VS Code](https://www.windmill.dev/docs/cli_local_dev/vscode-extension)
|
||||
extension):
|
||||
|
||||

|
||||

|
||||
|
||||
2. Your scripts parameters are automatically parsed and [generate a frontend](https://www.windmill.dev/docs/core_concepts/auto_generated_uis).
|
||||
2. Your scripts parameters are automatically parsed and
|
||||
[generate a frontend](https://www.windmill.dev/docs/core_concepts/auto_generated_uis).
|
||||
|
||||

|
||||
|
||||

|
||||
|
||||
3. Make it [flow](https://www.windmill.dev/docs/flows/flow_editor)! You can chain your scripts or scripts made by the community shared on [WindmillHub](https://hub.windmill.dev).
|
||||
3. Make it [flow](https://www.windmill.dev/docs/flows/flow_editor)! You can
|
||||
chain your scripts or scripts made by the community shared on
|
||||
[WindmillHub](https://hub.windmill.dev).
|
||||
|
||||

|
||||

|
||||
|
||||
4. Build [complex UIs](https://www.windmill.dev/docs/apps/app_editor) on top of your scripts and flows.
|
||||
4. Build [complex UIs](https://www.windmill.dev/docs/apps/app_editor) on top of
|
||||
your scripts and flows.
|
||||
|
||||

|
||||

|
||||
|
||||
Scripts and flows can be triggered by [schedules](https://www.windmill.dev/docs/core_concepts/scheduling), [webhooks](https://www.windmill.dev/docs/core_concepts/webhooks), [HTTP routes](https://www.windmill.dev/docs/core_concepts/http_routing), [Kafka](https://www.windmill.dev/docs/core_concepts/kafka_triggers), [WebSockets](https://www.windmill.dev/docs/core_concepts/websocket_triggers), [emails](https://www.windmill.dev/docs/core_concepts/email_triggers), and more.
|
||||
Scripts and flows can also be triggered by a
|
||||
[cron schedule](https://www.windmill.dev/docs/core_concepts/scheduling) (e.g.
|
||||
'_/5 _ \* \* \*') or through
|
||||
[webhooks](https://www.windmill.dev/docs/core_concepts/webhooks).
|
||||
|
||||
Build your entire infra on top of Windmill!
|
||||
You can build your entire infra on top of Windmill!
|
||||
|
||||
## Show me some actual script code
|
||||
|
||||
@@ -127,31 +144,43 @@ export async function main(
|
||||
}
|
||||
```
|
||||
|
||||
## Local Development
|
||||
## CLI
|
||||
|
||||
Windmill supports multiple ways to develop locally and sync with your instance:
|
||||
We have a powerful CLI to interact with the windmill platform and sync your
|
||||
scripts from local files, GitHub repos and to run scripts and flows on the
|
||||
instance from local commands. See
|
||||
[more details](https://www.windmill.dev/docs/advanced/cli).
|
||||
|
||||
| Tool | Description |
|
||||
|------|-------------|
|
||||
| **[CLI](https://www.windmill.dev/docs/advanced/cli)** | Sync scripts from local files or GitHub, run scripts/flows from the command line |
|
||||
| **[VS Code Extension](https://www.windmill.dev/docs/cli_local_dev/vscode-extension)** | Edit and test scripts & flows directly from VS Code / Cursor with full IDE support |
|
||||
| **[Git Sync](https://www.windmill.dev/docs/advanced/git_sync)** | Two-way sync between Windmill and your Git repository |
|
||||
| **[Claude Code](https://www.windmill.dev/docs/core_concepts/ai_generation)** | AI-assisted development with Claude for scripts, flows, and apps |
|
||||

|
||||
|
||||
https://github.com/user-attachments/assets/c541c326-e9ae-4602-a09a-1989aaded1e9
|
||||
### Running scripts locally
|
||||
|
||||
You can run scripts locally by passing the right environment variables for the `wmill` client library to fetch resources and variables from your instance. See [local development docs](https://www.windmill.dev/docs/advanced/local_development).
|
||||
You can run your script locally easily, you simply need to pass the right
|
||||
environment variables for the `wmill` client library to fetch resources and
|
||||
variables from your instance if necessary. See more:
|
||||
<https://www.windmill.dev/docs/advanced/local_development>.
|
||||
|
||||
To develop & test locally scripts & flows, we recommend using the Windmill VS
|
||||
Code extension: <https://www.windmill.dev/docs/cli_local_dev/vscode-extension>.
|
||||
|
||||
## Stack
|
||||
|
||||
- **Database**: Postgres (compatible with Aurora, Cloud SQL, Neon, Azure PostgreSQL)
|
||||
- **Backend**: Rust - stateless API servers and workers pulling jobs from a Postgres queue
|
||||
- **Frontend**: Svelte 5
|
||||
- **Sandboxing**: [nsjail](https://github.com/google/nsjail) and PID namespace isolation
|
||||
- **Runtimes**:
|
||||
- TypeScript/JavaScript: Bun (default) and Deno
|
||||
- Python: python3 with uv for dependency management
|
||||
- Go, Bash, PowerShell, PHP, Rust, C#, Java, Ansible
|
||||
- Postgres as the database.
|
||||
- Backend in Rust with the following highly-available and horizontally scalable.
|
||||
Architecture:
|
||||
- Stateless API backend.
|
||||
- Workers that pull jobs from a queue in Postgres (and later, Kafka or Redis.
|
||||
Upvote [#173](#https://github.com/windmill-labs/windmill/issues/173) if
|
||||
interested).
|
||||
- Frontend in Svelte.
|
||||
- Scripts executions are sandboxed using Google's
|
||||
[nsjail](https://github.com/google/nsjail).
|
||||
- Javascript runtime is the
|
||||
[deno_core rust library](https://denolib.gitbook.io/guide/) (which itself uses
|
||||
the [rusty_v8](https://github.com/denoland/rusty_v8) and hence V8 underneath).
|
||||
- TypeScript runtime is Bun and deno.
|
||||
- Python runtime is python3.
|
||||
- Golang runtime is 1.19.1.
|
||||
|
||||
## Fastest Self-Hostable Workflow Engine
|
||||
|
||||
@@ -168,10 +197,19 @@ page.
|
||||
|
||||
## Security
|
||||
|
||||
- **Sandboxing**: [nsjail](https://github.com/google/nsjail) for filesystem/resource isolation, and PID namespace isolation (enabled by default) to prevent jobs from accessing worker process memory
|
||||
- **Secrets**: One encryption key per workspace for credentials stored in Windmill's K/V store. We recommend encrypting the Postgres database as well.
|
||||
### Sandboxing
|
||||
|
||||
See [Security documentation](https://www.windmill.dev/docs/advanced/security_isolation) for details.
|
||||
Windmill can use [nsjail](https://github.com/google/nsjail). It is production
|
||||
multi-tenant grade secure. Do not take our word for it, take
|
||||
[fly.io's one](https://fly.io/blog/sandboxing-and-workload-isolation/).
|
||||
|
||||
### Secrets, credentials and sensitive values
|
||||
|
||||
There is one encryption key per workspace to encrypt the credentials and secrets
|
||||
stored in Windmill's K/V store.
|
||||
|
||||
In addition, we strongly recommend that you encrypt the whole Postgres database.
|
||||
That is what we do at <https://app.windmill.dev>.
|
||||
|
||||
## Performance
|
||||
|
||||
@@ -191,13 +229,19 @@ back to the database is ~50ms. A typical lightweight deno job will take around
|
||||
|
||||
## How to self-host
|
||||
|
||||
For detailed setup options, see [Self-Host documentation](https://www.windmill.dev/docs/advanced/self_host).
|
||||
We only provide docker-compose setup here. For more advanced setups, like
|
||||
compiling from source or using without a postgres super user, see
|
||||
[Self-Host documentation](https://www.windmill.dev/docs/advanced/self_host).
|
||||
|
||||
### Docker compose
|
||||
|
||||
Deploy Windmill with 3 files ([docker-compose.yml](./docker-compose.yml), [Caddyfile](./Caddyfile), [.env](./.env)):
|
||||
Windmill can be deployed using 3 files:
|
||||
([docker-compose.yml](./docker-compose.yml), [Caddyfile](./Caddyfile) and a
|
||||
[.env](./.env)) in a single command.
|
||||
|
||||
```bash
|
||||
Make sure Docker is started, and run:
|
||||
|
||||
```
|
||||
curl https://raw.githubusercontent.com/windmill-labs/windmill/main/docker-compose.yml -o docker-compose.yml
|
||||
curl https://raw.githubusercontent.com/windmill-labs/windmill/main/Caddyfile -o Caddyfile
|
||||
curl https://raw.githubusercontent.com/windmill-labs/windmill/main/.env -o .env
|
||||
@@ -205,126 +249,177 @@ curl https://raw.githubusercontent.com/windmill-labs/windmill/main/.env -o .env
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
Go to http://localhost - default credentials: `admin@windmill.dev` / `changeme`
|
||||
Go to http://localhost et voilà :)
|
||||
|
||||
**Using an external database**: Set `DATABASE_URL` in `.env` to point to your managed Postgres (AWS RDS, GCP Cloud SQL, Azure, Neon, etc.) and set db replicas to 0.
|
||||
The default super-admin user is: admin@windmill.dev / changeme.
|
||||
|
||||
### Kubernetes (Helm charts)
|
||||
From there, you can follow the setup app and create other users.
|
||||
|
||||
More details in
|
||||
[Self-Host Documention](https://www.windmill.dev/docs/advanced/self_host#docker).
|
||||
|
||||
### Kubernetes (k8s) and Helm charts
|
||||
|
||||
We publish helm charts at:
|
||||
<https://github.com/windmill-labs/windmill-helm-charts>.
|
||||
|
||||
### Run from binaries
|
||||
|
||||
Each release includes the corresponding binaries for x86_64. You can simply
|
||||
download the latest `windmill` binary using the following set of bash commands.
|
||||
|
||||
```bash
|
||||
helm repo add windmill https://windmill-labs.github.io/windmill-helm-charts/
|
||||
helm install windmill-chart windmill/windmill --namespace=windmill --create-namespace
|
||||
BINARY_NAME='windmill-amd64' # or windmill-ee-amd64 for the enterprise edition
|
||||
LATEST_RELEASE=$(curl -L -s -H 'Accept: application/json' https://github.com/windmill-labs/windmill/releases/latest)
|
||||
LATEST_VERSION=$(echo $LATEST_RELEASE | sed -e 's/.*"tag_name":"\([^"]*\)".*/\1/')
|
||||
ARTIFACT_URL="https://github.com/windmill-labs/windmill/releases/download/$LATEST_VERSION/$BINARY_NAME"
|
||||
wget "$ARTIFACT_URL" -O windmill
|
||||
```
|
||||
|
||||
See [windmill-helm-charts](https://github.com/windmill-labs/windmill-helm-charts) for configuration options.
|
||||
|
||||
### Cloud providers
|
||||
|
||||
Windmill works on AWS (EKS/ECS), GCP, Azure, Ubicloud, Fly.io, Render.com, Hetzner, Digital Ocean, and others. Rule of thumb: 1 worker per 1vCPU and 1-2 GB RAM.
|
||||
|
||||
### OAuth, SSO & SMTP
|
||||
|
||||
Configure OAuth and SSO (Google Workspace, Microsoft/Azure, Okta) directly from the superadmin UI. [See documentation](https://www.windmill.dev/docs/misc/setup_oauth).
|
||||
Windmill Community Edition allows to configure the OAuth, SSO (including Google
|
||||
Workspace SSO, Microsoft/Azure and Okta) directly from the UI in the superadmin
|
||||
settings. Do note that there is a limit of 10 SSO users on the community
|
||||
edition.
|
||||
|
||||
### License
|
||||
[See documentation](https://www.windmill.dev/docs/misc/setup_oauth).
|
||||
|
||||
The Community Edition is free to use internally. For commercial redistribution or managed services, contact <sales@windmill.dev>. See [LICENSE](./LICENSE) and [Pricing](https://www.windmill.dev/pricing) for details.
|
||||
### Commercial license
|
||||
|
||||
The "Community Edition" of Windmill available in the docker images hosted under ghcr.io/windmill-labs/windmill and the github binary releases contains the files under the AGPLv3 and Apache 2 sources but also includes proprietary and non-public code and features which are not open source and under the following terms: Windmill Labs, Inc. grants a right to use all the features of the "Community Edition" for free without restrictions other than the limits and quotas set in the software and a right to distribute the community edition as is but not to sell, resell, serve Windmill as a managed service, modify or wrap under any form without an explicit agreement.
|
||||
See the [LICENSE](https://github.com/windmill-labs/windmill/blob/main/LICENSE)
|
||||
file for the full license text.
|
||||
|
||||
The binary compilable from source code in this repository without the "enterprise" feature flag is open-source under the [LICENSE-AGPLv3](https://github.com/windmill-labs/windmill/blob/main/LICENSE-AGPL) License terms and conditions.
|
||||
The "Community Edition" of Windmill available in the docker images hosted under
|
||||
ghcr.io/windmill-labs/windmill and the github binary releases contains the files
|
||||
under the AGPLv3 and Apache 2 sources but also includes proprietary and
|
||||
non-public code and features which are not open source and under the following
|
||||
terms: Windmill Labs, Inc. grants a right to use all the features of the
|
||||
"Community Edition" for free without restrictions other than the limits and
|
||||
quotas set in the software and a right to distribute the community edition as is
|
||||
but not to sell, resell, serve Windmill as a managed service, modify or wrap
|
||||
under any form without an explicit agreement.
|
||||
|
||||
To [re-expose directly any Windmill parts to your users](https://www.windmill.dev/docs/misc/white_labelling) as a feature of your product, with the exception of iframed public Windmill "apps", or to build a feature on top of "Windmill Community Edition" that you sell commercially or embed in a distributable product or binary, you must get a commercial license. Contact us at <sales@windmill.dev> if you have any questions. To do the same from the binary compiled from the source code in this repository without the "enterprise" feature flag, you must comply with the AGPLv3 license terms and conditions or get a commercial license from Windmill Labs, Inc.
|
||||
The binary compilable from source code in this repository without the
|
||||
"enterprise" feature flag is open-source under the
|
||||
[LICENSE-AGPLv3](https://github.com/windmill-labs/windmill/blob/main/LICENSE-AGPL)
|
||||
License terms and conditions.
|
||||
|
||||
To use Windmill "Community Edition" as is internally in your organization, or to use its APIs as is, you do NOT need a commercial license.
|
||||
To
|
||||
[re-expose directly any Windmill parts to your users](https://www.windmill.dev/docs/misc/white_labelling)
|
||||
as a feature of your product, with the exception of iframed public Windmill
|
||||
"apps", or to build a feature on top of "Windmill Community Edition" that you
|
||||
sell commercially or embed in a distributable product or binary, you must get a
|
||||
commercial license. Contact us at <sales@windmill.dev> if you have any
|
||||
questions. To do the same from the binary compiled from the source code in this
|
||||
repository without the "enterprise" feature flag, you must comply with the
|
||||
AGPLv3 license terms and conditions or get a commercial license from Windmill
|
||||
Labs, Inc.
|
||||
|
||||
To use Windmill "Community Edition" as is internally in your organization, or to
|
||||
use its APIs as is, you do NOT need a commercial license.
|
||||
|
||||
### Integrations
|
||||
|
||||
In Windmill, integrations are referred to as [resources and resource types](https://www.windmill.dev/docs/core_concepts/resources_and_types). Each Resource has a Resource Type that defines the schema that the resource
|
||||
In Windmill, integrations are referred to as
|
||||
[resources and resource types](https://www.windmill.dev/docs/core_concepts/resources_and_types).
|
||||
Each Resource has a Resource Type that defines the schema that the resource
|
||||
needs to implement.
|
||||
|
||||
On self-hosted instances, you might want to import all the approved resource types from [WindmillHub](https://hub.windmill.dev). A setup script will prompt you to have it being synced automatically everyday.
|
||||
On self-hosted instances, you might want to import all the approved resource
|
||||
types from [WindmillHub](https://hub.windmill.dev). A setup script will prompt
|
||||
you to have it being synced automatically everyday.
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Environment Variable name | Default | Description | Api Server/Worker/All |
|
||||
| ----------------------------------- | -------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------- |
|
||||
| DATABASE_URL | | The Postgres database url. | All |
|
||||
| WORKER_GROUP | default | The worker group the worker belongs to and get its configuration pulled from | Worker |
|
||||
| MODE | standalone | The mode if the binary. Possible values: standalone, worker, server, agent | All |
|
||||
| METRICS_ADDR | None | (ee only) The socket addr at which to expose Prometheus metrics at the /metrics path. Set to "true" to expose it on port 8001 | All |
|
||||
| JSON_FMT | false | Output the logs in json format instead of logfmt | All |
|
||||
| BASE_URL | http://localhost:8000 | The base url that is exposed publicly to access your instance. Is overriden by the instance settings if any. | Server |
|
||||
| ZOMBIE_JOB_TIMEOUT | 30 | The timeout after which a job is considered to be zombie if the worker did not send pings about processing the job (every server check for zombie jobs every 30s) | Server |
|
||||
| RESTART_ZOMBIE_JOBS | true | If true then a zombie job is restarted (in-place with the same uuid and some logs), if false the zombie job is failed | Server |
|
||||
| SLEEP_QUEUE | 50 | The number of ms to sleep in between the last check for new jobs in the DB. It is multiplied by NUM_WORKERS such that in average, for one worker instance, there is one pull every SLEEP_QUEUE ms. | Worker |
|
||||
| KEEP_JOB_DIR | false | Keep the job directory after the job is done. Useful for debugging. | Worker |
|
||||
| LICENSE_KEY (EE only) | None | License key checked at startup for the Enterprise Edition of Windmill | Worker |
|
||||
| SLACK_SIGNING_SECRET | None | The signing secret of your Slack app. See [Slack documentation](https://api.slack.com/authentication/verifying-requests-from-slack) | Server |
|
||||
| COOKIE_DOMAIN | None | The domain of the cookie. If not set, the cookie will be set by the browser based on the full origin | Server |
|
||||
| DENO_PATH | /usr/bin/deno | The path to the deno binary. | Worker |
|
||||
| PYTHON_PATH | | The path to the python binary if wanting to not have it managed by uv. | Worker |
|
||||
| GO_PATH | /usr/bin/go | The path to the go binary. | Worker |
|
||||
| GOPRIVATE | | The GOPRIVATE env variable to use private go modules | Worker |
|
||||
| GOPROXY | | The GOPROXY env variable to use | Worker |
|
||||
| NETRC | | The netrc content to use a private go registry | Worker |
|
||||
| PY_CONCURRENT_DOWNLOADS | 20 | Sets the maximum number of in-flight concurrent python downloads that windmill will perform at any given time. | Worker |
|
||||
| PATH | None | The path environment variable, usually inherited | Worker |
|
||||
| HOME | None | The home directory to use for Go and Bash , usually inherited | Worker |
|
||||
| DATABASE_CONNECTIONS | 50 (Server)/3 (Worker) | The max number of connections in the database connection pool | All |
|
||||
| SUPERADMIN_SECRET | None | A token that would let the caller act as a virtual superadmin superadmin@windmill.dev | Server |
|
||||
| TIMEOUT_WAIT_RESULT | 20 | The number of seconds to wait before timeout on the 'run_wait_result' endpoint | Worker |
|
||||
| QUEUE_LIMIT_WAIT_RESULT | None | The number of max jobs in the queue before rejecting immediately the request in 'run_wait_result' endpoint. Takes precedence on the query arg. If none is specified, there are no limit. | Worker |
|
||||
| DENO_AUTH_TOKENS | None | Custom DENO_AUTH_TOKENS to pass to worker to allow the use of private modules | Worker |
|
||||
| DISABLE_RESPONSE_LOGS | false | Disable response logs | Server |
|
||||
| CREATE_WORKSPACE_REQUIRE_SUPERADMIN | true | If true, only superadmins can create new workspaces | Server |
|
||||
| MIN_FREE_DISK_SPACE_MB | 15000 | Minimum amount of free space on worker. Sends critical alert if worker has less free space. | Worker |
|
||||
| RUN_UPDATE_CA_CERTIFICATE_AT_START | false | If true, runs CA certificate update command at startup before other initialization | All |
|
||||
| RUN_UPDATE_CA_CERTIFICATE_PATH | /usr/sbin/update-ca-certificates | Path to the CA certificate update command/script to run when RUN_UPDATE_CA_CERTIFICATE_AT_START is true | All |
|
||||
| Environment Variable name | Default | Description | Api Server/Worker/All |
|
||||
| ----------------------------------- | ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------- |
|
||||
| DATABASE_URL | | The Postgres database url. | All |
|
||||
| WORKER_GROUP | default | The worker group the worker belongs to and get its configuration pulled from | Worker |
|
||||
| MODE | standalone | The mode if the binary. Possible values: standalone, worker, server, agent | All |
|
||||
| METRICS_ADDR | None | (ee only) The socket addr at which to expose Prometheus metrics at the /metrics path. Set to "true" to expose it on port 8001 | All |
|
||||
| JSON_FMT | false | Output the logs in json format instead of logfmt | All |
|
||||
| BASE_URL | http://localhost:8000 | The base url that is exposed publicly to access your instance. Is overriden by the instance settings if any. | Server |
|
||||
| ZOMBIE_JOB_TIMEOUT | 30 | The timeout after which a job is considered to be zombie if the worker did not send pings about processing the job (every server check for zombie jobs every 30s) | Server |
|
||||
| RESTART_ZOMBIE_JOBS | true | If true then a zombie job is restarted (in-place with the same uuid and some logs), if false the zombie job is failed | Server |
|
||||
| SLEEP_QUEUE | 50 | The number of ms to sleep in between the last check for new jobs in the DB. It is multiplied by NUM_WORKERS such that in average, for one worker instance, there is one pull every SLEEP_QUEUE ms. | Worker |
|
||||
| KEEP_JOB_DIR | false | Keep the job directory after the job is done. Useful for debugging. | Worker |
|
||||
| LICENSE_KEY (EE only) | None | License key checked at startup for the Enterprise Edition of Windmill | Worker |
|
||||
| SLACK_SIGNING_SECRET | None | The signing secret of your Slack app. See [Slack documentation](https://api.slack.com/authentication/verifying-requests-from-slack) | Server |
|
||||
| COOKIE_DOMAIN | None | The domain of the cookie. If not set, the cookie will be set by the browser based on the full origin | Server |
|
||||
| DENO_PATH | /usr/bin/deno | The path to the deno binary. | Worker |
|
||||
| PYTHON_PATH | | The path to the python binary if wanting to not have it managed by uv. | Worker |
|
||||
| GO_PATH | /usr/bin/go | The path to the go binary. | Worker |
|
||||
| GOPRIVATE | | The GOPRIVATE env variable to use private go modules | Worker |
|
||||
| GOPROXY | | The GOPROXY env variable to use | Worker |
|
||||
| NETRC | | The netrc content to use a private go registry | Worker |
|
||||
| PY_CONCURRENT_DOWNLOADS | 20 | Sets the maximum number of in-flight concurrent python downloads that windmill will perform at any given time. | Worker |
|
||||
| PATH | None | The path environment variable, usually inherited | Worker |
|
||||
| HOME | None | The home directory to use for Go and Bash , usually inherited | Worker |
|
||||
| DATABASE_CONNECTIONS | 50 (Server)/3 (Worker) | The max number of connections in the database connection pool | All |
|
||||
| SUPERADMIN_SECRET | None | A token that would let the caller act as a virtual superadmin superadmin@windmill.dev | Server |
|
||||
| TIMEOUT_WAIT_RESULT | 20 | The number of seconds to wait before timeout on the 'run_wait_result' endpoint | Worker |
|
||||
| QUEUE_LIMIT_WAIT_RESULT | None | The number of max jobs in the queue before rejecting immediately the request in 'run_wait_result' endpoint. Takes precedence on the query arg. If none is specified, there are no limit. | Worker |
|
||||
| DENO_AUTH_TOKENS | None | Custom DENO_AUTH_TOKENS to pass to worker to allow the use of private modules | Worker |
|
||||
| DISABLE_RESPONSE_LOGS | false | Disable response logs | Server |
|
||||
| CREATE_WORKSPACE_REQUIRE_SUPERADMIN | true | If true, only superadmins can create new workspaces | Server |
|
||||
| MIN_FREE_DISK_SPACE_MB | 15000 | Minimum amount of free space on worker. Sends critical alert if worker has less free space. | Worker |
|
||||
|
||||
## Run a local dev setup
|
||||
|
||||
We recommend using [Nix](./frontend/README_DEV.md#nix). See [./frontend/README_DEV.md](./frontend/README_DEV.md) for all options.
|
||||
See the [./frontend/README_DEV.md](./frontend/README_DEV.md) file for all
|
||||
running options.
|
||||
|
||||
### Frontend only
|
||||
Using [Nix](./frontend/README_DEV.md#nix).
|
||||
|
||||
Uses the backend of <https://app.windmill.dev> with local frontend (hot-reload):
|
||||
### only Frontend
|
||||
|
||||
```bash
|
||||
cd frontend
|
||||
npm install
|
||||
npm run generate-backend-client # or generate-backend-client-mac on Mac
|
||||
npm run dev
|
||||
This will use the backend of <https://app.windmill.dev> but your own frontend
|
||||
with hot-code reloading. Note that you will need to use a username / password
|
||||
login due to CSRF checks using a different auth provider.
|
||||
|
||||
In the `frontend/` directory:
|
||||
|
||||
1. install the dependencies with `npm install` (or `pnpm install` or `yarn`)
|
||||
2. generate the windmill client:
|
||||
|
||||
```
|
||||
npm run generate-backend-client
|
||||
## on mac use
|
||||
npm run generate-backend-client-mac
|
||||
```
|
||||
|
||||
Windmill available at `http://localhost/`
|
||||
3. Run your dev server with `npm run dev`
|
||||
4. Et voilà, windmill should be available at `http://localhost/`
|
||||
|
||||
### Backend + Frontend
|
||||
|
||||
See the [./frontend/README_DEV.md](./frontend/README_DEV.md) file for all
|
||||
running options.
|
||||
|
||||
1. Start a local Postgres database using for instance the `start-dev-db.sh` script which will make a database available at `postgres://postgres:changeme@localhost:5432/windmill`
|
||||
Then run the migrations using the following command:
|
||||
1. Create a Postgres Database for Windmill and create an admin role inside your
|
||||
Postgres setup. The easiest way to get a working db is to run
|
||||
```
|
||||
cargo install sqlx-cli
|
||||
env DATABASE_URL=<YOUR_DATABASE_URL> sqlx migrate run
|
||||
```
|
||||
This will also avoid compile time issue with sqlx's `query!` macro.
|
||||
2. (optional, linux only) Install [nsjail](https://github.com/google/nsjail) and have it accessible in
|
||||
This will also avoid compile time issue with sqlx's `query!` macro
|
||||
2. Install [nsjail](https://github.com/google/nsjail) and have it accessible in
|
||||
your PATH
|
||||
3. Install bun, deno and python3 (+ any languages you want to use), have the bins at `/usr/bin/bun`,`/usr/bin/deno`, and
|
||||
`/usr/local/bin/python3` or set the corresponding environment variables.
|
||||
4. (optional) Install the [lld linker](https://lld.llvm.org/)
|
||||
5. Go to `frontend/`:
|
||||
1. `npm install`, `npm run generate-backend-client` then `REMOTE=http://localhost:8000 npm run dev`
|
||||
3. Install deno and python3, have the bins at `/usr/bin/deno` and
|
||||
`/usr/local/bin/python3`
|
||||
4. Install [caddy](https://caddyserver.com)
|
||||
5. Install the [lld linker](https://lld.llvm.org/)
|
||||
6. Go to `frontend/`:
|
||||
1. `npm install`, `npm run generate-backend-client` then `npm run dev`
|
||||
2. You might need to set some extra heap space for the node runtime
|
||||
`export NODE_OPTIONS="--max-old-space-size=4096"`
|
||||
3. Create an empty `frontend/build` folder using `mkdir frontend/build`
|
||||
6. Go to `backend/`:
|
||||
1. `env DATABASE_URL=<YOUR_DATABASE_URL> RUST_LOG=info cargo run`
|
||||
2. You can specify any feature flag you want to enable, for example `cargo run --features python` to enable the python executor.
|
||||
7. Windmill should be available at `http://localhost:3000`
|
||||
3. In another shell `npm run build` otherwise the backend will not find the
|
||||
`frontend/build` folder and will not compile.
|
||||
4. In another shell `sudo caddy run --config Caddyfile`
|
||||
7. Go to `backend/`:
|
||||
`env DATABASE_URL=<DATABASE_URL_TO_YOUR_WINDMILL_DB> RUST_LOG=info cargo run`
|
||||
8. Et voilà, windmill should be available at `http://localhost/`
|
||||
|
||||
## Contributors
|
||||
|
||||
@@ -334,4 +429,4 @@ running options.
|
||||
|
||||
## Copyright
|
||||
|
||||
© 2023-2026 Windmill Labs, Inc.
|
||||
Windmill Labs, Inc 2023
|
||||
|
||||
@@ -1,27 +1,14 @@
|
||||
[build]
|
||||
incremental = true
|
||||
|
||||
[target.x86_64-unknown-linux-gnu]
|
||||
linker = "clang"
|
||||
rustflags = ["-C", "link-arg=-fuse-ld=mold"]
|
||||
|
||||
[target.aarch64-unknown-linux-gnu]
|
||||
linker = "clang"
|
||||
rustflags = ["-C", "link-arg=-fuse-ld=mold"]
|
||||
|
||||
[target.x86_64-apple-darwin]
|
||||
rustflags = [
|
||||
"-C", "link-arg=-undefined",
|
||||
"-C", "link-arg=dynamic_lookup",
|
||||
"-C", "link-args=-Wl,-rpath,$ORIGIN/"
|
||||
]
|
||||
|
||||
[target.aarch64-apple-darwin]
|
||||
rustflags = [
|
||||
"-C", "link-arg=-undefined",
|
||||
"-C", "link-arg=dynamic_lookup",
|
||||
"-C", "link-args=-Wl,-rpath,$ORIGIN/"
|
||||
]
|
||||
|
||||
[net]
|
||||
git-fetch-with-cli = true
|
||||
]
|
||||
7
backend/.gitignore
vendored
7
backend/.gitignore
vendored
@@ -5,9 +5,4 @@ oauth2.json
|
||||
tracing.folded
|
||||
heaptrack*
|
||||
index/
|
||||
windmill-api/openapi-*.*
|
||||
.duckdb/*
|
||||
*ee.rs
|
||||
generate_mcp_endpoints_tools/venv
|
||||
bacon.toml
|
||||
libwindmill_duckdb_ffi_internal.so
|
||||
windmill-api/openapi-*.*
|
||||
@@ -1 +0,0 @@
|
||||
!*ee.rs
|
||||
@@ -1,23 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT created_by FROM v2_job WHERE id = $1 AND workspace_id = $2",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "created_by",
|
||||
"type_info": "Varchar"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Uuid",
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "002d68d7c4437522a6dae95af007a356217bbae06b8453f0c32046f0cbf20dcb"
|
||||
}
|
||||
@@ -1,23 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT args as \"args: sqlx::types::Json<HashMap<String, Box<RawValue>>>\" FROM v2_job WHERE id = $1 AND workspace_id = $2",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "args: sqlx::types::Json<HashMap<String, Box<RawValue>>>",
|
||||
"type_info": "Jsonb"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Uuid",
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
true
|
||||
]
|
||||
},
|
||||
"hash": "0084c1246d1391d106da2e67a394eafc6695257632406ed9a2111dba1dd106c7"
|
||||
}
|
||||
@@ -1,15 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "INSERT INTO dependency_map (workspace_id, importer_path, importer_kind, imported_path, importer_node_id)\n SELECT $1, importer_path, importer_kind, imported_path, importer_node_id\n FROM dependency_map\n WHERE workspace_id = $2",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Varchar",
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "00b9f392a5cc07bd4ed14e3b69f96408e219d70015dd2f419fc87a440f070c64"
|
||||
}
|
||||
@@ -1,24 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT f.path\n FROM workspace_runnable_dependencies wru \n JOIN flow f\n ON wru.flow_path = f.path AND wru.workspace_id = f.workspace_id\n WHERE wru.runnable_path = $1 AND wru.runnable_is_flow = $2 AND wru.workspace_id = $3",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "path",
|
||||
"type_info": "Varchar"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text",
|
||||
"Bool",
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "00c0ae12b19ba495f307f0ce6b4833947c5b3fe45826fc5468e326d171d95236"
|
||||
}
|
||||
@@ -1,58 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT\n workspace.id AS \"id!\",\n workspace.name AS \"name!\",\n workspace.owner AS \"owner!\",\n workspace.deleted AS \"deleted!\",\n workspace.premium AS \"premium!\",\n workspace_settings.color AS \"color\",\n workspace.parent_workspace_id AS \"parent_workspace_id\"\n FROM workspace\n LEFT JOIN workspace_settings ON workspace.id = workspace_settings.workspace_id\n WHERE workspace.id = $1",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "id!",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 1,
|
||||
"name": "name!",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 2,
|
||||
"name": "owner!",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 3,
|
||||
"name": "deleted!",
|
||||
"type_info": "Bool"
|
||||
},
|
||||
{
|
||||
"ordinal": 4,
|
||||
"name": "premium!",
|
||||
"type_info": "Bool"
|
||||
},
|
||||
{
|
||||
"ordinal": 5,
|
||||
"name": "color",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 6,
|
||||
"name": "parent_workspace_id",
|
||||
"type_info": "Varchar"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
false,
|
||||
false,
|
||||
false,
|
||||
false,
|
||||
false,
|
||||
true,
|
||||
true
|
||||
]
|
||||
},
|
||||
"hash": "00c1dd0cfaf15aafdcfcabc1f123cebdf8d777f48e148bcb171fa15e8bf6f098"
|
||||
}
|
||||
@@ -1,24 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT data FROM app_bundles WHERE app_version_id = $1 AND file_type = $2 AND w_id = $3",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "data",
|
||||
"type_info": "Bytea"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Int8",
|
||||
"Text",
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "01050e7057f3d1971ad9e47ac83bf6a3c3c9f41689c3607f0b264437ae6b3324"
|
||||
}
|
||||
50
backend/.sqlx/query-011c7638eeeda710deb86a216a9e10df9c3e9458e85bcdde466b01011a1f2ac2.json
generated
Normal file
50
backend/.sqlx/query-011c7638eeeda710deb86a216a9e10df9c3e9458e85bcdde466b01011a1f2ac2.json
generated
Normal file
@@ -0,0 +1,50 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n SELECT\n path,\n is_flow,\n workspace_id,\n owner,\n email,\n trigger_config as \"trigger_config!: _\"\n FROM\n capture_config\n WHERE\n trigger_kind = 'postgres' AND\n last_client_ping > NOW() - INTERVAL '10 seconds' AND\n trigger_config IS NOT NULL AND\n (last_server_ping IS NULL OR last_server_ping < now() - interval '15 seconds')\n ",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "path",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 1,
|
||||
"name": "is_flow",
|
||||
"type_info": "Bool"
|
||||
},
|
||||
{
|
||||
"ordinal": 2,
|
||||
"name": "workspace_id",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 3,
|
||||
"name": "owner",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 4,
|
||||
"name": "email",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 5,
|
||||
"name": "trigger_config!: _",
|
||||
"type_info": "Jsonb"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": []
|
||||
},
|
||||
"nullable": [
|
||||
false,
|
||||
false,
|
||||
false,
|
||||
false,
|
||||
false,
|
||||
true
|
||||
]
|
||||
},
|
||||
"hash": "011c7638eeeda710deb86a216a9e10df9c3e9458e85bcdde466b01011a1f2ac2"
|
||||
}
|
||||
@@ -1,22 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT j.id\n FROM v2_job_queue q JOIN v2_job j USING (id) LEFT JOIN v2_job_runtime r USING (id) LEFT JOIN v2_job_status s USING (id)\n WHERE r.ping < now() - ($1 || ' seconds')::interval\n AND q.running = true AND j.kind NOT IN ('flow', 'flowpreview', 'flownode', 'singlestepflow') AND j.same_worker = false",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "id",
|
||||
"type_info": "Uuid"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "0186c1058f147e012b8120c342caf8688a6d1643747be3ec4f784c3029a59e52"
|
||||
}
|
||||
@@ -1,28 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT\n id As \"id!\",\n flow_status->'restarted_from'->'flow_job_id' AS \"restarted_from: Json<Uuid>\"\n FROM v2_job_status\n WHERE COALESCE((SELECT flow_innermost_root_job FROM v2_job WHERE id = $1), $1) = id",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "id!",
|
||||
"type_info": "Uuid"
|
||||
},
|
||||
{
|
||||
"ordinal": 1,
|
||||
"name": "restarted_from: Json<Uuid>",
|
||||
"type_info": "Jsonb"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Uuid"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
false,
|
||||
null
|
||||
]
|
||||
},
|
||||
"hash": "019100d178129340a7c35d60ab61f983c8a9cb810db4369554bf26c6b0d6003d"
|
||||
}
|
||||
@@ -1,12 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "INSERT INTO app_version (id, app_id, value, created_by, created_at)\n VALUES (3001, 3001, '{\"grid\": []}', 'admin', NOW())",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": []
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "01c040b04b487e86b7f4ff38b0faacf6af2c284ae446860113c82bc4e1da08ab"
|
||||
}
|
||||
@@ -1,23 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT value FROM variable WHERE path = $1 AND workspace_id = $2 AND is_secret = true",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "value",
|
||||
"type_info": "Varchar"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text",
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "020c031c3de6c85577e30421ada9d39a5a47ca1b6cf3dbfd6988aa0694d7364c"
|
||||
}
|
||||
@@ -1,15 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "DELETE FROM v2_job WHERE workspace_id = $1 AND id = ANY($2)",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text",
|
||||
"UuidArray"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "023555d33652d40fa381b1baaae6b319c4bac92cb2d90bb4ffd08e25f4a4d18b"
|
||||
}
|
||||
@@ -1,29 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n INSERT INTO native_trigger (\n external_id,\n workspace_id,\n service_name,\n script_path,\n is_flow,\n webhook_token_prefix,\n service_config\n ) VALUES (\n $1, $2, $3, $4, $5, $6, $7\n )\n ON CONFLICT (external_id, workspace_id, service_name)\n DO UPDATE SET script_path = $4, is_flow = $5, webhook_token_prefix = $6, service_config = $7, error = NULL, updated_at = NOW()\n ",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Varchar",
|
||||
"Varchar",
|
||||
{
|
||||
"Custom": {
|
||||
"name": "native_trigger_service",
|
||||
"kind": {
|
||||
"Enum": [
|
||||
"nextcloud"
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"Varchar",
|
||||
"Bool",
|
||||
"Varchar",
|
||||
"Jsonb"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "023cdbc77ea9e2c17a1aa92a5b9001f29e58e81b3f782887db6e0a627dd8ad75"
|
||||
}
|
||||
@@ -1,15 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "UPDATE asset SET workspace_id = $1 WHERE workspace_id = $2",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Varchar",
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "029b81eb00250eacded407b12bcfbab2b3f35354bdb9ef6e30281a4ff6235060"
|
||||
}
|
||||
@@ -1,23 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT flow_version.value AS \"value!: sqlx::types::Json<Box<sqlx::types::JsonRawValue>>\"\n FROM flow\n LEFT JOIN flow_version\n ON flow_version.id = flow.versions[array_upper(flow.versions, 1)]\n WHERE flow.path = $1 AND flow.workspace_id = $2",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "value!: sqlx::types::Json<Box<sqlx::types::JsonRawValue>>",
|
||||
"type_info": "Jsonb"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text",
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "02bf9763298f301d4fc75490c070a0663142d4d23a2df007361622b94d4783e1"
|
||||
}
|
||||
@@ -1,77 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT\n v2_job.permissioned_as_email,\n v2_job.created_by,\n v2_job.parent_job,\n v2_job.permissioned_as,\n v2_job.runnable_path,\n CASE WHEN v2_job.trigger_kind = 'schedule'::job_trigger_kind THEN v2_job.trigger END AS schedule_path,\n v2_job.flow_step_id,\n v2_job.flow_innermost_root_job,\n v2_job.root_job,\n v2_job_queue.scheduled_for AS \"scheduled_for: chrono::DateTime<chrono::Utc>\"\n FROM v2_job INNER JOIN v2_job_queue ON v2_job.id = v2_job_queue.id\n WHERE v2_job.id = $1 AND v2_job.workspace_id = $2",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "permissioned_as_email",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 1,
|
||||
"name": "created_by",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 2,
|
||||
"name": "parent_job",
|
||||
"type_info": "Uuid"
|
||||
},
|
||||
{
|
||||
"ordinal": 3,
|
||||
"name": "permissioned_as",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 4,
|
||||
"name": "runnable_path",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 5,
|
||||
"name": "schedule_path",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 6,
|
||||
"name": "flow_step_id",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 7,
|
||||
"name": "flow_innermost_root_job",
|
||||
"type_info": "Uuid"
|
||||
},
|
||||
{
|
||||
"ordinal": 8,
|
||||
"name": "root_job",
|
||||
"type_info": "Uuid"
|
||||
},
|
||||
{
|
||||
"ordinal": 9,
|
||||
"name": "scheduled_for: chrono::DateTime<chrono::Utc>",
|
||||
"type_info": "Timestamptz"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Uuid",
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
false,
|
||||
false,
|
||||
true,
|
||||
false,
|
||||
true,
|
||||
null,
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "02c945b5f18a56a826721f6884846d79167747742de236ce57f395561685adc0"
|
||||
}
|
||||
@@ -1,37 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n DELETE FROM asset\n WHERE (workspace_id, path, kind) IN (\n SELECT workspace_id, path, kind FROM (\n SELECT a.workspace_id, a.path, a.kind, a.usage_kind, ROW_NUMBER() OVER (\n PARTITION BY a.workspace_id, a.path, a.kind\n ORDER BY a.created_at DESC\n ) as rn,\n limits.max_n\n FROM asset a\n INNER JOIN (\n SELECT * FROM UNNEST(\n $1::varchar[], \n $2::varchar[], \n $3::asset_kind[],\n $4::int[]\n ) AS t(workspace_id, path, kind, max_n)\n ) limits\n ON a.workspace_id = limits.workspace_id \n AND a.path = limits.path \n AND a.kind = limits.kind\n WHERE a.usage_kind = 'job'\n ) ranked\n WHERE rn > max_n\n )",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"VarcharArray",
|
||||
"VarcharArray",
|
||||
{
|
||||
"Custom": {
|
||||
"name": "asset_kind[]",
|
||||
"kind": {
|
||||
"Array": {
|
||||
"Custom": {
|
||||
"name": "asset_kind",
|
||||
"kind": {
|
||||
"Enum": [
|
||||
"s3object",
|
||||
"resource",
|
||||
"variable",
|
||||
"ducklake",
|
||||
"datatable"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"Int4Array"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "02e526146f3584cd599dec708e1be48db3b0cd1c74adbfa2e4039377daa016f0"
|
||||
}
|
||||
@@ -1,22 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n SELECT jsonb_object_keys(ws.ducklake->'ducklakes') AS ducklake_name\n FROM workspace_settings ws\n WHERE ws.workspace_id = $1\n ",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "ducklake_name",
|
||||
"type_info": "Text"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
null
|
||||
]
|
||||
},
|
||||
"hash": "02ecdcc882931d5cbb2243e32805c8a1291a5106fff46ceba85fa27d50a0354c"
|
||||
}
|
||||
@@ -1,14 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "DELETE FROM v2_job_status WHERE id = ANY($1)",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"UuidArray"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "02fdd7b94e6b6c9bb7985dfeb2082655d08946206dcfb25158c10f78619cf7fc"
|
||||
}
|
||||
@@ -1,22 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n INSERT INTO postgres_trigger (\n path, script_path, is_flow, workspace_id, edited_by, email,\n postgres_resource_path, replication_slot_name, publication_name\n )\n VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)\n ",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Varchar",
|
||||
"Varchar",
|
||||
"Bool",
|
||||
"Varchar",
|
||||
"Varchar",
|
||||
"Varchar",
|
||||
"Varchar",
|
||||
"Varchar",
|
||||
"Varchar"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "0300afc35a880eef163dfdfd9d5299fac14562ee8595c792f3c30d042fa2d3eb"
|
||||
}
|
||||
26
backend/.sqlx/query-031d0d70b0aff52feaad487bddb74e5ef0aaa2505facbea8c764003dfc8fffb1.json
generated
Normal file
26
backend/.sqlx/query-031d0d70b0aff52feaad487bddb74e5ef0aaa2505facbea8c764003dfc8fffb1.json
generated
Normal file
@@ -0,0 +1,26 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "UPDATE capture_config SET last_server_ping = now(), error = $1 WHERE workspace_id = $2 AND path = $3 AND is_flow = $4 AND trigger_kind = 'websocket' AND server_id = $5 AND last_client_ping > NOW() - INTERVAL '10 seconds' RETURNING 1",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "?column?",
|
||||
"type_info": "Int4"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text",
|
||||
"Text",
|
||||
"Text",
|
||||
"Bool",
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
null
|
||||
]
|
||||
},
|
||||
"hash": "031d0d70b0aff52feaad487bddb74e5ef0aaa2505facbea8c764003dfc8fffb1"
|
||||
}
|
||||
@@ -1,19 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "UPDATE workspace_diff SET has_changes = true, exists_in_source = $5, exists_in_fork = $6\n WHERE path = $3 AND kind = $4 AND (\n (source_workspace_id = $1 AND fork_workspace_id = $2)\n OR (source_workspace_id = $2 AND fork_workspace_id =$1)\n )",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text",
|
||||
"Text",
|
||||
"Text",
|
||||
"Text",
|
||||
"Bool",
|
||||
"Bool"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "034a8519198daf30e0eb8a74ed92f896c83bb39e1cb52fe3c29c1a224c3859c2"
|
||||
}
|
||||
@@ -1,22 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "UPDATE schedule SET enabled = false WHERE workspace_id = $1 AND enabled = true RETURNING path",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "path",
|
||||
"type_info": "Varchar"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "03669873e4e3b22c737d5170821f677925474aad885bf1c0780bdb978225517e"
|
||||
}
|
||||
12
backend/.sqlx/query-036c84bb9ce72748956bc9c18fbe276444fab025a281dc4784596b0e31c1cb9d.json
generated
Normal file
12
backend/.sqlx/query-036c84bb9ce72748956bc9c18fbe276444fab025a281dc4784596b0e31c1cb9d.json
generated
Normal file
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "create index concurrently if not exists ix_job_workspace_id_created_at_new_9 ON v2_job (workspace_id, created_at DESC) where kind in ('dependencies', 'flowdependencies', 'appdependencies') AND parent_job IS NULL",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": []
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "036c84bb9ce72748956bc9c18fbe276444fab025a281dc4784596b0e31c1cb9d"
|
||||
}
|
||||
@@ -1,16 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n UPDATE workspace_settings\n SET auto_invite = COALESCE(auto_invite, '{}'::jsonb)\n || jsonb_build_object('instance_groups', $2::jsonb, 'instance_groups_roles', $3::jsonb)\n WHERE workspace_id = $1\n ",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text",
|
||||
"Jsonb",
|
||||
"Jsonb"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "03caaec9f41be87eac32e163d7a829e58ae993932e9bbad5aebab4770cc44d61"
|
||||
}
|
||||
@@ -1,16 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "INSERT INTO tutorial_progress (email, progress, skipped_all) VALUES ($2, $1::bigint::bit(64), $3) ON CONFLICT (email) DO UPDATE SET progress = EXCLUDED.progress, skipped_all = EXCLUDED.skipped_all",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Int8",
|
||||
"Varchar",
|
||||
"Bool"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "04362a55081f7a98bca8fe4db0669939da8944711037957664cc2989b239c9d1"
|
||||
}
|
||||
@@ -1,20 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT EXISTS(SELECT 1 FROM usr WHERE workspace_id = 'ws-with-auto-add' AND email = 'bob@example.com')",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "exists",
|
||||
"type_info": "Bool"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": []
|
||||
},
|
||||
"nullable": [
|
||||
null
|
||||
]
|
||||
},
|
||||
"hash": "045c5b21422113ce8592d1b645d4e513e0eff982c80d4ce490ee5381627f8d16"
|
||||
}
|
||||
@@ -1,20 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT COUNT(*) FROM v2_job",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "count",
|
||||
"type_info": "Int8"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": []
|
||||
},
|
||||
"nullable": [
|
||||
null
|
||||
]
|
||||
},
|
||||
"hash": "04ce5c530c80ae6f911dfe0dc9ed7d1a2e10342bbbc7f8486df0b73f5657a493"
|
||||
}
|
||||
@@ -1,15 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n DELETE FROM\n capture\n WHERE\n workspace_id = $1\n AND created_at <= (\n SELECT\n created_at\n FROM\n capture\n WHERE\n workspace_id = $1\n ORDER BY\n created_at DESC\n OFFSET $2\n LIMIT 1\n )\n ",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text",
|
||||
"Int8"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "0574df3e18f626dd8b3f83fbff8b0ee99cf8483a8fe66fa9311cb96e3f5a0ee2"
|
||||
}
|
||||
14
backend/.sqlx/query-05cb171b610bfb45f6228128a385cde8a5b86d7ca377a028004cc382e12faf41.json
generated
Normal file
14
backend/.sqlx/query-05cb171b610bfb45f6228128a385cde8a5b86d7ca377a028004cc382e12faf41.json
generated
Normal file
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "INSERT INTO concurrency_counter(concurrency_id, job_uuids) \n VALUES ($1, '{}'::jsonb)",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Varchar"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "05cb171b610bfb45f6228128a385cde8a5b86d7ca377a028004cc382e12faf41"
|
||||
}
|
||||
@@ -1,22 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT name FROM group_ WHERE workspace_id = $1 UNION SELECT name FROM instance_group ORDER BY name asc",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "name",
|
||||
"type_info": "Varchar"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
null
|
||||
]
|
||||
},
|
||||
"hash": "05f4663a0f58736e92fe7cbbef3c99a03bc74ab3be1bacdbbf3910a76a1beacc"
|
||||
}
|
||||
@@ -1,26 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "SELECT workspace_id, path FROM variable WHERE is_secret = true",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "workspace_id",
|
||||
"type_info": "Varchar"
|
||||
},
|
||||
{
|
||||
"ordinal": 1,
|
||||
"name": "path",
|
||||
"type_info": "Varchar"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": []
|
||||
},
|
||||
"nullable": [
|
||||
false,
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "0600f2a9179f83502c6b13e8e4284f85ca82636f274f5dce47da5a8320a60088"
|
||||
}
|
||||
@@ -1,33 +0,0 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n SELECT EXISTS(\n SELECT 1\n FROM native_trigger\n WHERE\n workspace_id = $1 AND\n service_name = $2 AND\n external_id = $3\n )\n ",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "exists",
|
||||
"type_info": "Bool"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text",
|
||||
{
|
||||
"Custom": {
|
||||
"name": "native_trigger_service",
|
||||
"kind": {
|
||||
"Enum": [
|
||||
"nextcloud"
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
null
|
||||
]
|
||||
},
|
||||
"hash": "06072cfe26abe58629623a8b38382b33947c5a5c702ce586e6e6ea51430380bf"
|
||||
}
|
||||
15
backend/.sqlx/query-0659bab15d4cccdb04c7a57e0e3bbb6bfebb8896601a27ddf5618d4eae678bc1.json
generated
Normal file
15
backend/.sqlx/query-0659bab15d4cccdb04c7a57e0e3bbb6bfebb8896601a27ddf5618d4eae678bc1.json
generated
Normal file
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "UPDATE usr SET workspace_id = $1 WHERE workspace_id = $2",
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Varchar",
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": []
|
||||
},
|
||||
"hash": "0659bab15d4cccdb04c7a57e0e3bbb6bfebb8896601a27ddf5618d4eae678bc1"
|
||||
}
|
||||
26
backend/.sqlx/query-070b8ad0b59f485fa5bf68082b060f5c3561c37e9c6f2834d234a862a475a6eb.json
generated
Normal file
26
backend/.sqlx/query-070b8ad0b59f485fa5bf68082b060f5c3561c37e9c6f2834d234a862a475a6eb.json
generated
Normal file
@@ -0,0 +1,26 @@
|
||||
{
|
||||
"db_name": "PostgreSQL",
|
||||
"query": "\n UPDATE \n gcp_trigger \n SET \n enabled = $1, \n email = $2, \n edited_by = $3, \n edited_at = now(), \n server_id = NULL, \n error = NULL\n WHERE \n path = $4 AND \n workspace_id = $5 \n RETURNING 1\n ",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "?column?",
|
||||
"type_info": "Int4"
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Bool",
|
||||
"Varchar",
|
||||
"Varchar",
|
||||
"Text",
|
||||
"Text"
|
||||
]
|
||||
},
|
||||
"nullable": [
|
||||
null
|
||||
]
|
||||
},
|
||||
"hash": "070b8ad0b59f485fa5bf68082b060f5c3561c37e9c6f2834d234a862a475a6eb"
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user