Compare commits

..

59 Commits

Author SHA1 Message Date
Ruben Fiszel
02fb2b3806 chore(main): release 1.8.2 (#58)
* chore(main): release 1.8.2

* Apply automatic changes

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2022-05-18 09:50:12 +02:00
Ruben Fiszel
563ba3e7f7 fix: deno client 2022-05-18 09:48:41 +02:00
Ruben Fiszel
3eed59fcb1 fix: deno lsp client 2022-05-18 01:32:00 +02:00
Ruben Fiszel
7365a8e87b fix: starting deno script is now async 2022-05-17 23:15:16 +02:00
Ruben Fiszel
dbd6142997 align jsonrpc 2022-05-17 23:01:02 +02:00
Ruben Fiszel
865d728224 fix: deno lsp uses wss instead of ws 2022-05-17 22:39:57 +02:00
Ruben Fiszel
8861e19564 ci: add deno 2022-05-17 22:21:45 +02:00
Ruben Fiszel
92b502d9ba chore(main): release 1.8.1 (#57)
* chore(main): release 1.8.1

* Apply automatic changes

* Apply automatic changes

* Apply automatic changes

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2022-05-17 21:49:10 +02:00
Ruben Fiszel
297a3e60e2 ci: fix change version 2022-05-17 21:47:36 +02:00
Ruben Fiszel
1decaafde0 remove poetry locks 2022-05-17 21:43:15 +02:00
Ruben Fiszel
a7ef616c0d ci: fix change version 2022-05-17 21:39:10 +02:00
Ruben Fiszel
481685a73e ci: fix change version 2022-05-17 21:36:49 +02:00
Ruben Fiszel
a356e7b7d3 ci: use python poetry for change versions 2022-05-17 21:35:10 +02:00
Ruben Fiszel
f793bc46d9 fix: frontend dependencies update 2022-05-17 21:30:10 +02:00
Ruben Fiszel
c49e4930bc update frontend 2022-05-17 21:28:24 +02:00
dependabot[bot]
7b6ae612a5 chore(deps): bump @codingame/monaco-jsonrpc in /frontend (#55)
Bumps [@codingame/monaco-jsonrpc](https://github.com/CodinGame/monaco-jsonrpc) from 0.3.1 to 0.4.0.
- [Release notes](https://github.com/CodinGame/monaco-jsonrpc/releases)
- [Commits](https://github.com/CodinGame/monaco-jsonrpc/commits)

---
updated-dependencies:
- dependency-name: "@codingame/monaco-jsonrpc"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-17 21:20:40 +02:00
dependabot[bot]
d179d6efc3 chore(deps): bump @zerodevx/svelte-toast in /frontend (#56)
Bumps [@zerodevx/svelte-toast](https://github.com/zerodevx/svelte-toast) from 0.6.3 to 0.7.1.
- [Release notes](https://github.com/zerodevx/svelte-toast/releases)
- [Commits](https://github.com/zerodevx/svelte-toast/compare/v0.6.3...v0.7.1)

---
updated-dependencies:
- dependency-name: "@zerodevx/svelte-toast"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-17 21:19:08 +02:00
Ruben Fiszel
f02e5b19ac update frontend + lock python client 2022-05-17 21:16:36 +02:00
Ruben Fiszel
e114d0f426 chore(main): release 1.8.0 (#52)
* chore(main): release 1.8.0

* Apply automatic changes

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2022-05-17 20:53:14 +02:00
Ruben Fiszel
03ec38e001 update cargo 2022-05-17 20:52:42 +02:00
Ruben Fiszel
2e1d43033f feat: Typescript support for scripts (alpha)
* typescript support

* frontend

* type inference

* type inference

* v0 works

* v0 typescript

* v0 typescript

* deno-client v0

* deno-client v0

* build_deno

* rm autogenerated files

* test workflow

* test workflow

* test workflow

* test workflow

* test workflow

* test workflow

* test workflow

* test workflow

* test workflow

* test workflow

* test workflow

* test workflow

* test workflow

* test workflow

* test workflow

* test workflow

* test workflow

* test workflow

* on tags

* createResource

* createResource

* createResource2

* typescript support

* templates

* include version
2022-05-17 20:42:05 +02:00
Ruben Fiszel
ec528fce67 chore(main): release 1.7.0 (#45)
* chore(main): release 1.7.0

* Apply automatic changes

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2022-05-14 14:58:31 +02:00
Tomasz Wsuł
5b413d7e04 feat: self host github oauth (#46) 2022-05-14 14:54:53 +02:00
Ruben Fiszel
02c8bea084 fix: better error message when saving script 2022-05-11 13:29:21 +02:00
Ruben Fiszel
bb31c80378 fix README docker-compose reference 2022-05-11 13:05:22 +02:00
Ruben Fiszel
91045e73cc BUG_ISSUE instructions 2022-05-11 08:10:51 +02:00
dependabot[bot]
9219b651a3 chore(deps-dev): bump @sveltejs/kit in /frontend (#25)
Bumps [@sveltejs/kit](https://github.com/sveltejs/kit/tree/HEAD/packages/kit) from 1.0.0-next.324 to 1.0.0-next.326.
- [Release notes](https://github.com/sveltejs/kit/releases)
- [Changelog](https://github.com/sveltejs/kit/blob/master/packages/kit/CHANGELOG.md)
- [Commits](https://github.com/sveltejs/kit/commits/@sveltejs/kit@1.0.0-next.326/packages/kit)

---
updated-dependencies:
- dependency-name: "@sveltejs/kit"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2022-05-11 01:27:27 +02:00
Ruben Fiszel
7f21d03d00 chore(main): release 1.6.1 (#34)
* chore(main): release 1.6.1

* Apply automatic changes

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2022-05-10 21:38:59 +02:00
dependabot[bot]
a62e6e5ee3 chore(deps): bump serde_json from 1.0.79 to 1.0.81 in /backend (#26)
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.79 to 1.0.81.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.79...v1.0.81)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2022-05-10 21:32:22 +02:00
Ruben Fiszel
2c28031e44 fix: also store and display "started at" for completed jobs (#33) 2022-05-10 21:32:07 +02:00
Ruben Fiszel
ca8de69126 run prettier 2022-05-10 21:29:54 +02:00
dependabot[bot]
98071bd68b chore(deps): bump tower-http from 0.2.5 to 0.3.3 in /backend (#27)
Bumps [tower-http](https://github.com/tower-rs/tower-http) from 0.2.5 to 0.3.3.
- [Release notes](https://github.com/tower-rs/tower-http/releases)
- [Commits](https://github.com/tower-rs/tower-http/compare/tower-http-0.2.5...tower-http-0.3.3)

---
updated-dependencies:
- dependency-name: tower-http
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2022-05-10 21:18:04 +02:00
dependabot[bot]
128dde4fb3 chore(deps): bump thiserror from 1.0.30 to 1.0.31 in /backend (#30)
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.30 to 1.0.31.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.30...1.0.31)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2022-05-10 21:07:45 +02:00
dependabot[bot]
f090945b27 chore(deps): bump serde from 1.0.136 to 1.0.137 in /backend (#32) 2022-05-10 21:07:29 +02:00
dependabot[bot]
60729d80b9 chore(deps): bump mhart/alpine-node from 14 to 16 (#21)
Bumps mhart/alpine-node from 14 to 16.

---
updated-dependencies:
- dependency-name: mhart/alpine-node
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2022-05-10 17:28:49 +02:00
Ruben Fiszel
e228beec2a ci: push to private registry builded image no matter what 2022-05-10 17:15:16 +02:00
dependabot[bot]
4dbf562fb7 chore(deps): bump GoogleCloudPlatform/release-please-action from 2 to 3 (#20) 2022-05-10 14:41:11 +02:00
dependabot[bot]
4952290296 chore(deps): bump actions/checkout from 2 to 3 (#19)
Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-10 14:24:17 +02:00
Juan Calderon-Perez
f53eb71e4a ci: add support for dependabot (#9)
* Add support for dependabot

* Add dependabot support for Python clients

* move to a weekly schedule

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2022-05-10 12:14:38 +00:00
Ruben Fiszel
96f54f5f44 chore: release 1.6.0 (#6)
* Apply automatic changes

* Update version.txt

* Apply automatic changes

* Update CHANGELOG.md

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2022-05-10 12:48:04 +02:00
Ruben Fiszel
0863e12e6a ci: add codeowners 2022-05-10 09:41:44 +02:00
Ruben Fiszel
d03266b0a4 ci: add CLA 2022-05-10 09:12:24 +02:00
Ruben Fiszel
4a4eaa90e2 ci: add CLA 2022-05-10 09:02:28 +02:00
Ruben Fiszel
5e7c14b722 ci: add CLA 2022-05-10 08:52:11 +02:00
Ruben Fiszel
55b5695673 fix: display more than default 30 workspaces as superadmin 2022-05-09 15:18:28 +02:00
Ruben Fiszel
8596ac50b9 delete starter script without lock files 2022-05-08 17:56:16 +02:00
Ruben Fiszel
13fb52117b feat: self host minimal 2 2022-05-08 17:51:33 +02:00
Ruben Fiszel
2c70a15594 feat: self host minimal 2022-05-08 17:26:51 +02:00
Ruben Fiszel
7a51f842f0 feat: superadmin settings 2022-05-08 17:03:13 +02:00
Ruben Fiszel
a130806e19 feat: user settings is now at workspace level 2022-05-08 12:58:58 +02:00
Ruben Fiszel
fd1f05dd16 ci: refactor + dockerhub 2022-05-08 11:57:37 +02:00
Ruben Fiszel
48e51733e0 docs: add main ci badge 2022-05-06 14:59:42 +02:00
Ruben Fiszel
e7817e6c9f alpha.windmill -> app.windmill 2022-05-06 13:55:14 +02:00
Ruben Fiszel
51ad6edfcb docs: typos 2022-05-05 15:59:59 +02:00
Ruben Fiszel
315f7edd64 docs: windmill imgs 2022-05-05 15:53:40 +02:00
Ruben Fiszel
a2c3deab74 docs: README general idea 2022-05-05 15:24:35 +02:00
Ruben Fiszel
891b7eb93a docs: architecture diagram 2022-05-05 13:22:13 +02:00
Ruben Fiszel
7efd87be79 docs: architecture diagram 2022-05-05 13:20:42 +02:00
Ruben Fiszel
5acbc8b48c Create FUNDING.yml 2022-05-05 10:50:54 +02:00
4919 changed files with 38443 additions and 625289 deletions

View File

@@ -1,3 +0,0 @@
/*
!/backend/
!/frontend/

View File

@@ -1,110 +0,0 @@
---
description:
globs: backend/**/*.rs
alwaysApply: false
---
# Windmill Backend - Rust Best Practices
## Project Structure
Windmill uses a workspace-based architecture with multiple crates:
- **windmill-api**: API server functionality
- **windmill-worker**: Job execution
- **windmill-common**: Shared code used by all crates
- **windmill-queue**: Job & flow queuing
- **windmill-audit**: Audit logging
- Other specialized crates (git-sync, autoscaling, etc.)
## Adding New Code
### Module Organization
- Place new code in the appropriate crate based on functionality
- For API endpoints, create or modify files in `windmill-api/src/` organized by domain
- For shared functionality, use `windmill-common/src/`
- Use the `_ee.rs` suffix for enterprise-only modules
- Follow existing patterns for file structure and organization
### Error Handling
- Use the custom `Error` enum from `windmill-common::error`
- Return `Result<T, Error>` or `JsonResult<T>` for functions that can fail
- Use the `?` operator for error propagation
- Add location tracking to errors using `#[track_caller]`
### Database Operations
- Use `sqlx` for database operations with prepared statements
- Leverage existing database helper functions in `db.rs` modules
- Use transactions for multi-step operations
- Handle database errors properly
### API Endpoints
- Follow existing patterns in the `windmill-api` crate
- Use axum's routing system and extractors
- Group related routes together
- Use consistent response formats (JSON)
- Follow proper authentication and authorization patterns
- Do not forget to update backend/windmill-api/openapi.yaml after modifying an api endpoint
## Performance Optimizations
When generating code, especially involving `serde`, `sqlx`, and `tokio`, prioritize performance by applying the following principles:
### Serde Optimizations (Serialization & Deserialization)
- **Specify Structure Explicitly:** When defining structs for Serde (`#[derive(Serialize, Deserialize)]`), use `#[serde(...` attributes extensively. This includes:
* `#[serde(rename = "...")]` or `#[serde(alias = "...")]` to map external names precisely, avoiding dynamic lookups.
* `#[serde(default)]` for optional fields with default values, reducing parsing complexity.
* `#[serde(skip_serializing_if = "...")]` to avoid writing fields that meet a certain condition (e.g., `Option::is_none()`, `Vec::is_empty()`, or a custom function), reducing output size and serialization work.
* `#[serde(skip_serializing)]` or `#[serde(skip_deserializing)]` for fields that should *not* be included.
- **Prefer Borrowing:** Where possible and safe (data lifetime allows), use `Cow<'a, str>` or `&'a str` (with `#[serde(borrow)]`) instead of `String` for string fields during deserialization. This avoids allocating new strings, enabling zero-copy reading from the input buffer. Apply this principle to byte slices (`&'a [u8]` / `Cow<'a, [u8]>`) and potentially borrowed vectors as well.
- **Avoid Intermediate `Value`:** Unless the data structure is truly dynamic or unknown at compile time, deserialize directly into a well-defined struct or enum rather than into `serde_json::Value` (or equivalent for other formats). This avoids unnecessary heap allocations and type switching.
### SQLx Optimizations (Database Interaction)
- **Select Only Necessary Columns:** In `SELECT` queries, list specific column names rather than using `SELECT *`. This reduces data transferred from the database and the work needed for hydration/deserialization.
- **Batch Operations:** For multiple `INSERT`, `UPDATE`, or `DELETE` statements, prefer executing them in a single query if the database and driver support it efficiently (e.g., `INSERT INTO ... VALUES (...), (...), ...`). This minimizes round trips to the database.
- **Avoid N+1 Queries:** Do not loop through results of one query and execute a separate query for each item (e.g., fetching users, then querying for each user's profile in a loop). Instead, use JOINs or a single query with an `IN` clause to fetch related data efficiently.
- **Deserialize Directly:** Use `#[derive(FromRow)]` on structs and ensure the struct fields match the selected columns in the query. This allows SQLx to hydrate objects directly, avoiding intermediate data structures.
- **Parameterize Queries:** Always use SQLx's query methods (`.bind(...)`) to pass values as parameters rather than string formatting. This prevents SQL injection and allows the database to cache query plans, improving performance on repeated executions.
### Tokio Optimizations (Asynchronous Runtime)
- **Avoid Blocking Operations:** **Crucially**, never perform blocking operations (synchronous file I/O, `std::thread::sleep`, CPU-bound loops, `std::sync::Mutex::lock`, blocking network calls without `tokio::net`) directly within an `async fn` or a standard `tokio::spawn` task. Blocking pauses the entire worker thread, potentially starving other tasks. Use `tokio::task::spawn_blocking` for CPU-intensive work or blocking I/O.
- **Use Tokio's Async Primitives:** Prefer `tokio::sync` (channels, mutexes, semaphores), `tokio::io`, `tokio::net`, and `tokio::time` over their `std` counterparts in asynchronous contexts. These are designed to yield control back to the scheduler.
- **Manage Concurrency:** Be mindful of how many tasks are spawned. Creating a new task for every tiny piece of work can introduce overhead. Group related asynchronous operations where appropriate.
- **Handle Shared State Efficiently:** Use `Arc` for shared ownership in concurrent tasks. When shared state needs mutation, prefer `tokio::sync::Mutex` over `std::sync::Mutex` in `async` code. Consider `tokio::sync::RwLock` if reads significantly outnumber writes. Minimize the duration for which locks are held.
- **Understand `.await`:** Place `.await` strategically to allow the runtime to switch to other ready tasks. Ensure that `.await` points to genuinely asynchronous operations.
- **Backpressure:** If dealing with data streams or queues between tasks, implement backpressure mechanisms (e.g., bounded channels like `tokio::sync::mpsc::channel`) to prevent one component from overwhelming another or critical resources like the database.
## Enterprise Features
- Use feature flags for enterprise functionality
- Conditionally compile with `#[cfg(feature = "enterprise")]`
- Isolate enterprise code in separate modules
## Code Style
- Group imports by external and internal crates
- Place struct/enum definitions before implementations
- Group similar functionality together
- Use descriptive naming consistent with the codebase
- Follow existing patterns for async code using tokio
## Testing
- Write unit tests for core functionality
- Use the `#[cfg(test)]` module for test code
- For database tests, use the existing test utilities
## Common Crates Used
- **tokio**: For async runtime
- **axum**: For web server and routing
- **sqlx**: For database operations
- **serde**: For serialization/deserialization
- **tracing**: For logging and diagnostics
- **reqwest**: For HTTP client functionality

View File

@@ -1,229 +0,0 @@
---
description:
globs: frontend/src/**/*.svelte
alwaysApply: false
---
# Svelte 5 Best Practices
This guide outlines best practices for developing with Svelte 5, incorporating the new Runes API and other modern Svelte features. These rules MUST NOT be applied on svelte 4 files unless explicitly asked to do so.
## Reactivity with Runes
Svelte 5 introduces Runes for more explicit and flexible reactivity.
1. **Embrace Runes for State Management**:
* Use `$state` for reactive local component state.
```svelte
<script>
let count = $state(0);
function increment() {
count += 1;
}
</script>
<button onclick={increment}>
Clicked {count} {count === 1 ? 'time' : 'times'}
</button>
```
* Use `$derived` for computed values based on other reactive state.
```svelte
<script>
let count = $state(0);
const doubled = $derived(count * 2);
</script>
<p>{count} * 2 = {doubled}</p>
```
* Use `$effect` for side effects that need to run when reactive values change (e.g., logging, manual DOM manipulation, data fetching). Remember `$effect` does not run on the server.
```svelte
<script>
let count = $state(0);
$effect(() => {
console.log('The count is now', count);
if (count > 5) {
alert('Count is too high!');
}
});
</script>
```
2. **Props with `$props`**:
* Declare component props using `$props()`. This offers better clarity and flexibility compared to `export let`.
```svelte
<script>
// ChildComponent.svelte
let { name, age = $state(30) } = $props();
</script>
<p>Name: {name}</p>
<p>Age: {age}</p>
```
* For bindable props, use `$bindable`.
```svelte
<script>
// MyInput.svelte
let { value = $bindable() } = $props();
</script>
<input bind:value />
```
## Event Handling
* **Use direct event attributes**: Svelte 5 moves away from `on:` directives for DOM events.
* **Do**: `<button onclick={handleClick}>...</button>`
* **Don't**: `<button on:click={handleClick}>...</button>`
* **For component events, prefer callback props**: Instead of `createEventDispatcher`, pass functions as props.
```svelte
<!-- Parent.svelte -->
<script>
import Child from './Child.svelte';
let message = $state('');
function handleChildEvent(detail) {
message = detail;
}
</script>
<Child onCustomEvent={handleChildEvent} />
<p>Message from child: {message}</p>
<!-- Child.svelte -->
<script>
let { onCustomEvent } = $props();
function emitEvent() {
onCustomEvent('Hello from child!');
}
</script>
<button onclick={emitEvent}>Send Event</button>
```
## Snippets for Content Projection
* **Use `{#snippet ...}` and `{@render ...}` instead of slots**: Snippets are more powerful and flexible.
```svelte
<!-- Parent.svelte -->
<script>
import Card from './Card.svelte';
</script>
<Card>
{#snippet title()}
My Awesome Title
{/snippet}
{#snippet content()}
<p>Some interesting content here.</p>
{/snippet}
</Card>
<!-- Card.svelte -->
<script>
let { title, content } = $props();
</script>
<article>
<header>{@render title()}</header>
<div>{@render content()}</div>
</article>
```
* Default content is passed via the `children` prop (which is a snippet).
```svelte
<!-- Wrapper.svelte -->
<script>
let { children } = $props();
</script>
<div>
{@render children?.()}
</div>
```
## Component Design
1. **Create Small, Reusable Components**: Break down complex UIs into smaller, focused components. Each component should have a single responsibility. This also aids performance by limiting the scope of reactivity updates.
2. **Descriptive Naming**: Use clear and descriptive names for variables, functions, and components.
3. **Minimize Logic in Components**: Move complex business logic to utility functions or services. Keep components focused on presentation and interaction.
## State Management (Stores)
1. **Segment Stores**: Avoid a single global store. Create multiple stores, each responsible for a specific piece of global state (e.g., `userStore.js`, `themeStore.js`). This can help limit reactivity updates to only the parts of the UI that depend on specific state segments.
2. **Use Custom Stores for Complex Logic**: For stores with related methods, create custom stores.
```javascript
// counterStore.js
import { writable } from 'svelte/store';
function createCounter() {
const { subscribe, set, update } = writable(0);
return {
subscribe,
increment: () => update(n => n + 1),
decrement: () => update(n => n - 1),
reset: () => set(0)
};
}
export const counter = createCounter();
```
3. **Use Context API for Localized State**: For state shared within a component subtree, consider Svelte's context API (`setContext`, `getContext`) instead of global stores when the state doesn't need to be truly global.
## Performance Optimizations (Svelte 5)
When generating Svelte 5 code, prioritize frontend performance by applying the following principles:
### General Svelte 5 Principles
- **Leverage the Compiler:** Trust Svelte's compiler to generate optimized JavaScript. Avoid manual DOM manipulation (`document.querySelector`, etc.) unless absolutely necessary for integrating third-party libraries that lack Svelte adapters.
- **Keep Components Small and Focused:** Reinforcing from Component Design, smaller components lead to less complex reactivity graphs and more targeted, efficient updates.
### Reactivity & State Management
- **Optimize Computations with `$derived`:** Always use `$derived` for computed values that depend on other state. This ensures the computation only runs when its specific dependencies change, avoiding unnecessary work compared to recomputing derived values in `$effect` or less efficient methods.
- **Minimize `$effect` Usage:** Use `$effect` sparingly and only for true side effects that interact with the outside world or non-Svelte state. Avoid putting complex logic or state updates *within* an `$effect` unless those updates are explicitly intended as a reaction to external changes or non-Svelte state. Excessive or complex effects can impact rendering performance.
- **Structure State for Fine-Grained Updates:** Design your `$state` objects or variables such that updates affect only the necessary parts of the UI. Avoid putting too much unrelated state into a single large object that gets frequently updated, as this can potentially trigger broader updates than necessary. Consider normalizing complex, nested state.
### List Rendering (`{#each}`)
- **Mandate `key` Attribute:** Always use a `key` attribute (`{#each items as item (item.id)}`) that refers to a unique, stable identifier for each item in a list. This is critical for allowing Svelte to efficiently update, reorder, add, or remove list items without destroying and re-creating unnecessary DOM elements and component instances.
### Component Loading & Bundling
- **Implement Lazy Loading/Code Splitting:** For routes, components, or modules that are not immediately needed on page load, use dynamic imports (`import(...)`) to split the code bundle. SvelteKit handles this automatically for routes, but it can be applied manually to components using helper patterns if needed.
- **Be Mindful of Third-Party Libraries:** When incorporating external libraries, import only the necessary functions or components to minimize the final bundle size. Prefer libraries designed to be tree-shakeable.
### Rendering & DOM
- **Use CSS for Animations/Transitions:** Prefer CSS animations or transitions where possible for performance. Svelte's built-in `transition:` directive is also highly optimized and should be used for complex state-driven transitions, but simple cases can often use plain CSS.
- **Optimize Image Loading:** Implement best practices for images: use optimized formats (WebP, AVIF), lazy loading (`loading="lazy"`), and responsive images (`<picture>`, `srcset`) to avoid loading unnecessarily large images.
### Server-Side Rendering (SSR) & Hydration
- **Ensure SSR Compatibility:** Write components that can be rendered on the server for faster initial page loads. Avoid relying on browser-specific APIs (like `window` or `document`) in the main `<script>` context. If necessary, use `$effect` or check `if (browser)` inside effects to run browser-specific code only on the client.
- **Minimize Work During Hydration:** Structure components and data fetching such that minimal complex setup or computation is required when the client-side Svelte code takes over from the server-rendered HTML. Heavy synchronous work during hydration can block the main thread.
## General Clean Code Practices
1. **Organized File Structure**: Group related files together. A common structure:
```
/src
|-- /routes // Page components (if using a router like SvelteKit)
|-- /lib // Utility functions, services, constants (SvelteKit often uses this)
| |-- /stores
| |-- /utils
| |-- /services
| |-- /components // Reusable UI components
|-- App.svelte
|-- main.js (or main.ts)
```
2. **Scoped Styles**: Keep CSS scoped to components to avoid unintended side effects and improve maintainability. Avoid `:global` where possible.
3. **Immutability**: With Svelte 5 and `$state`, direct assignments to properties of `$state` objects (`obj.prop = value;`) are generally fine as Svelte's reactivity system handles updates. However, for non-rune state or when interacting with other systems, understanding and sometimes preferring immutable updates (creating new objects/arrays) can still be relevant.
4. **Use `class:` and `style:` directives**: For dynamic classes and styles, use Svelte's built-in directives for cleaner templates and potentially optimized updates.
```svelte
<script>
let isActive = $state(true);
let color = $state('blue');
</script>
<div class:active={isActive} style:color={color}>
Hello
</div>
```
5. **Stay Updated**: Keep Svelte and its related packages up to date to benefit from the latest features, performance improvements, and security fixes.

View File

@@ -1,76 +0,0 @@
---
description:
globs:
alwaysApply: false
---
# Windmill Overview
Windmill is an open-source developer platform for building internal tools, API integrations, background jobs, workflows, and user interfaces. It offers a unified system where scripts are automatically turned into sharable UIs and can be composed into flows or embedded in custom applications.
## Core Capabilities
- **Script Development and Execution**: Write and run scripts in Python, TypeScript/JavaScript (Deno/Bun), Go, Bash, SQL, and other languages
- **Workflow Orchestration**: Compose scripts into multi-step flows with conditional logic, loops, and error handling
- **UI Generation**: Automatically generate UIs from scripts or build custom applications with a low-code editor
- **Job Scheduling**: Trigger scripts and flows on schedules, webhooks, or external events
- **Resource Management**: Securely store and use credentials, databases, and other connections
## Platform Architecture
The Windmill platform consists of several key components:
- **Frontend UI**: Web-based interface for script and flow development, app building, and result visualization
- **API Server**: Central API that handles authentication, resource management, and job coordination
- **Workers**: Execute scripts in their respective environments with proper sandboxing
- **Database**: PostgreSQL database for storage of scripts, flows, resources, job results, and more
- **Job Queue**: Queue system for managing job execution, implemented in PostgreSQL
- **Client Libraries**: Libraries for interacting with Windmill from Python, TypeScript, or command line
# Windmill Backend Architecture
The Windmill backend is written in Rust and consists of several services working together. These services are designed for horizontal scaling with stateless API servers and workers that can be deployed across multiple machines.
## Key Components
- **API Server (`windmill-api`)**: Handles HTTP requests, authentication, and resource management
- **Queue Manager (`windmill-queue`)**: Manages the job queue in PostgreSQL
- **Worker System (`windmill-worker`)**: Executes jobs in sandboxed environments
- **Common Utilities (`windmill-common`)**: Shared code used by multiple services
- **Git Sync (`windmill-git-sync`)**: Synchronizes scripts with Git repositories
## Job Execution System
The job execution process follows these steps:
1. The API server receives a request to run a script or flow and creates a job record in the database
2. The job is added to the queue system in PostgreSQL
3. Workers continuously poll the queue for jobs matching their capabilities
4. When a job is picked up, it's routed to the appropriate language executor
5. The script is executed in a sandboxed environment using NSJAIL for security
6. Results are processed and stored in the database
7. For flows, each step creates a new job that goes through the same process
Windmill supports worker tags and groups to route jobs to workers with specific capabilities or resource access.
# Windmill Frontend Architecture
The Windmill frontend is built with Svelte and provides several key interfaces for interacting with the platform.
## Key Components
- **Script Builder**: Code editor with language support, schema inference, and dependency management
- **Flow Builder**: Visual editor for creating multi-step workflows with branching and looping
- **App Editor**: Grid-based editor for building custom UIs that integrate scripts and flows
- **Schema Form System**: Generates form interfaces from script parameters automatically
- **Result Viewer**: Visualizes job results, logs, and execution status
The frontend uses the Monaco editor (same as VS Code) for code editing, with specialized language support for all supported script languages.
## UI Framework
The frontend is built with Svelte, providing a reactive and component-based architecture. Key frontend technologies include:
- **Svelte/SvelteKit**: Core framework for UI components and routing
- **Monaco Editor**: Code editing experience similar to VS Code
- **Schema Form**: Automatic UI generation from TypeScript/JSON schemas
- **Tailwind CSS**: Utility-first CSS framework for styling

View File

@@ -1,41 +0,0 @@
FROM python:3.10-slim-bullseye as nsjail
WORKDIR /nsjail
RUN apt-get -y update \
&& apt-get install -y \
bison \
flex \
g++ \
gcc \
git \
libprotobuf-dev \
libnl-route-3-dev \
make \
pkg-config \
protobuf-compiler \
&& apt-get clean -y && rm -rf /var/lib/apt/lists/*
RUN git clone -b master --single-branch https://github.com/google/nsjail.git . \
&& git checkout dccf911fd2659e7b08ce9507c25b2b38ec2c5800
RUN make
FROM mcr.microsoft.com/vscode/devcontainers/rust:bullseye as rust-deps
RUN cargo install sqlx-cli --no-default-features --features native-tls,postgres
RUN cargo install deno --locked
FROM mcr.microsoft.com/vscode/devcontainers/rust:bullseye
RUN apt update \
&& apt-get install -y \
python3 \
libprotobuf-dev \
libnl-route-3-dev \
&& apt-get clean -y && rm -rf /var/lib/apt/lists/*
USER vscode
COPY --from=rust-deps /usr/local/cargo/bin/sqlx /usr/local/cargo/bin/sqlx
COPY --from=rust-deps /usr/local/cargo/bin/deno /usr/local/cargo/bin/deno
COPY --from=nsjail /nsjail/nsjail /bin/nsjail

View File

@@ -1,24 +0,0 @@
version: '3.7'
services:
windmill:
build:
context: .
dockerfile: .devcontainer/Dockerfile
# image: mcr.microsoft.com/vscode/devcontainers/rust:bullseye
environment:
- DENO_PATH=/usr/local/cargo/bin/deno
- NSJAIL_PATH=/bin/nsjail
volumes:
- .:/workspace:cached
- ~/.ssh:/home/vscode/.ssh:ro
command: /bin/sh -c "while sleep 1000; do :; done"
front:
image: mcr.microsoft.com/vscode/devcontainers/typescript-node:16
volumes:
- .:/workspace:cached
- ~/.ssh:/home/node/.ssh:ro
command: /bin/sh -c "while sleep 1000; do :; done"

17
.env
View File

@@ -1,13 +1,6 @@
DATABASE_URL=postgres://postgres:changeme@db/windmill?sslmode=disable
SITE_URL=localhost
DB_PASSWORD=changeme
# For Enterprise Edition, use:
# WM_IMAGE=ghcr.io/windmill-labs/windmill-ee:main
WM_IMAGE=ghcr.io/windmill-labs/windmill:main
# To use another port than :80, setup the Caddyfile and the caddy section of the docker-compose to your needs: https://caddyserver.com/docs/getting-started
# To have caddy take care of automatic TLS
# To rotate logs, set the following variables:
#LOG_MAX_SIZE=10m
#LOG_MAX_FILE=3
# GitHub OAuth- https://docs.github.com/en/developers/apps/building-oauth-apps/creating-an-oauth-app
GITHUB_OAUTH_CLIENT_ID=yours_client_id
GITHUB_OAUTH_CLIENT_SECRET=yours_client_sected

1
.envrc
View File

@@ -1 +0,0 @@
use flake

View File

@@ -1,14 +0,0 @@
#!/bin/sh
#
# This file is symlinked to local .git/hooks/pre-commit by the setup-hooks.sh script
# It wil run before every commit, so it needs to be quick and efficient. If it returns
# a non-zero exit code, the commit will be aborted.
echo "Running pre-commit hook"
# This checks that there is no symlinks in the backend directory among the EE files
./backend/check_no_symlink.sh > /dev/null
if [ $? -ne 0 ]; then
echo "/!\ Symlinks detected in the backend directory. Please run './backend/substitute_ee_code.sh --revert' before committing."
exit 1
fi

5
.github/CODEOWNERS vendored
View File

@@ -1,4 +1 @@
* @rubenfiszel @HugoCasa @alpetric
/community/ @rubenfiszel @HugoCasa @alpetric
/frontend/ @rubenfiszel @HugoCasa @alpetric
* @rubenfiszel

2
.github/Dockerfile vendored
View File

@@ -1,7 +1,7 @@
FROM nikolaik/python-nodejs
RUN npm install -g @apidevtools/swagger-cli
RUN pip install openapi-python-client==0.15.1
RUN pip install openapi-python-client
RUN pip install poetry

View File

@@ -1,64 +0,0 @@
ARG RUST_IMAGE=rust:1.80-slim-bookworm
ARG PYTHON_IMAGE=python:3.11.4-slim-bookworm
FROM ${RUST_IMAGE} as builder
RUN apt-get update && apt-get install -y git libssl-dev pkg-config
RUN apt-get -y update \
&& apt-get install -y \
curl
ENV SQLX_OFFLINE=true
RUN mkdir -p /frontend/build
RUN apt-get update \
&& apt-get install -y ca-certificates tzdata libpq5 cmake unzip\
make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev \
libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev xz-utils tk-dev libxml2-dev \
libxmlsec1-dev libffi-dev liblzma-dev mecab-ipadic-utf8 libgdbm-dev libc6-dev git libprotobuf-dev libnl-route-3-dev \
libv8-dev nodejs npm clang libclang-dev\
&& rm -rf /var/lib/apt/lists/*
RUN wget https://golang.org/dl/go1.21.5.linux-amd64.tar.gz && tar -C /usr/local -xzf go1.21.5.linux-amd64.tar.gz
ENV PATH="${PATH}:/usr/local/go/bin"
ENV GO_PATH=/usr/local/go/bin/go
# UV
RUN curl --proto '=https' --tlsv1.2 -LsSf https://github.com/astral-sh/uv/releases/download/0.4.18/uv-installer.sh | sh && mv /usr/local/cargo/bin/uv /usr/local/bin/uv
ENV TZ=Etc/UTC
ENV PYTHON_VERSION 3.11.4
# Python
RUN wget https://www.python.org/ftp/python/${PYTHON_VERSION}/Python-${PYTHON_VERSION}.tgz \
&& tar -xf Python-${PYTHON_VERSION}.tgz && cd Python-${PYTHON_VERSION}/ && ./configure --enable-optimizations \
&& make -j 4 && make install
RUN /usr/local/bin/python3 -m pip install pip-tools
# Bun
COPY --from=oven/bun:1.2.4 /usr/local/bin/bun /usr/bin/bun
ARG TARGETPLATFORM
# Deno
RUN curl -Lsf https://github.com/denoland/deno/releases/download/v2.0.2/deno-x86_64-unknown-linux-gnu.zip -o deno.zip
# RUN [ "$TARGETPLATFORM" == "linux/arm64" ] && curl -Lsf https://github.com/denoland/deno/releases/download/v2.0.0/deno-aarch64-unknown-linux-gnu.zip -o deno.zip || true
RUN unzip deno.zip && rm deno.zip && mv deno /usr/bin/deno
RUN apt-get update \
&& apt-get install -y postgresql-client --allow-unauthenticated
RUN rustup component add rustfmt
# C#
COPY --from=bitnami/dotnet-sdk:9.0.101-debian-12-r0 /opt/bitnami/dotnet-sdk /opt/dotnet-sdk
RUN ln -s /opt/dotnet-sdk/bin/dotnet /usr/bin/dotnet
# Nushell
COPY --from=ghcr.io/nushell/nushell:0.101.0-bookworm /usr/bin/nu /usr/bin/nu

View File

@@ -1,7 +0,0 @@
FROM nikolaik/python-nodejs:python3.11-nodejs19
RUN python3 -m pip install pipx poetry
RUN python3 -m pipx ensurepath
ENV PATH="/root/.local/bin:${PATH}"
ENV PATH="/usr/local/bin:${PATH}"
RUN pipx install openapi-python-client==0.15.1 --include-deps

27
.github/ISSUE_TEMPLATE/bug_report.md vendored Normal file
View File

@@ -0,0 +1,27 @@
---
name: Bug report
about: Create a report to help us improve
title: 'bug:'
labels: 'bug'
assignees: 'rubenfiszel'
---
**Describe the bug** A clear and concise description of what the bug is.
**To Reproduce** Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
**Expected behavior** A clear and concise description of what you expected to
happen.
**Screenshots** If applicable, add screenshots to help explain your problem.
**Windmill version** Go on the left menu -> <user> -> User Settings and copy the
printed version in "Running windmill version (backend): XXX".
**Additional context** Add any other context about the problem here.

View File

@@ -1,62 +0,0 @@
name: Bug report
description: Create a report to help us improve.
title: 'bug:'
labels: 'bug'
assignees: 'rubenfiszel'
body:
- type: textarea
id: description
attributes:
label: Describe the bug
description: A clear and concise description of what the bug is.
validations:
required: true
- type: textarea
id: reproduction-steps
attributes:
label: To reproduce
description: Steps to reproduce the behavior
value: |
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
validations:
required: true
- type: textarea
id: expected-behaviour
attributes:
label: Expected behavior
placeholder: A clear and concise description of what you expected to happen.
validations:
required: true
- type: textarea
id: screenshots
attributes:
label: Screenshots
placeholder: If applicable, add screenshots to help explain your problem.
validations:
required: false
- type: input
id: browser
attributes:
label: Browser information
description: Which browser are you using? Which version?
placeholder: e.g. Chromium Version 92.0.4515.131
validations:
required: false
- type: input
id: version
attributes:
label: Application version
description: 'Go on the left menu -> <user> -> User Settings and copy the printed version in "Running windmill version (backend): XXX".'
placeholder: e.g. windmill version (backend) v1.35.0-63-ga85302c
validations:
required: false
- type: textarea
id: additional-context
attributes:
label: Additional Context
description: Add any other context about the problem here.
validations:
required: false

View File

@@ -1,26 +0,0 @@
#!/bin/bash
set -euo pipefail
script_dirpath="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
root_dirpath="$(cd "${script_dirpath}/.." && pwd)"
VERSION=$1
echo "Updating versions to: $VERSION"
sed -i '' -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/backend/Cargo.toml
sed -i '' -e "/^export const VERSION =/s/= .*/= \"v$VERSION\";/" ${root_dirpath}/cli/main.ts
sed -i '' -e "/^export const VERSION =/s/= .*/= \"v$VERSION\";/" ${root_dirpath}/benchmarks/lib.ts
sed -i '' -e "/version: /s/: .*/: $VERSION/" ${root_dirpath}/backend/windmill-api/openapi.yaml
sed -i '' -e "/version: /s/: .*/: $VERSION/" ${root_dirpath}/openflow.openapi.yaml
sed -i '' -e "/\"version\": /s/: .*,/: \"$VERSION\",/" ${root_dirpath}/typescript-client/package.json
sed -i '' -e "/\"version\": /s/: .*,/: \"$VERSION\",/" ${root_dirpath}/frontend/package.json
sed -i '' -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/python-client/wmill/pyproject.toml
sed -i '' -e "/^windmill-api =/s/= .*/= \"\\^$VERSION\"/" ${root_dirpath}/python-client/wmill/pyproject.toml
sed -i '' -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/python-client/wmill_pg/pyproject.toml
sed -i '' -e "/^[[:space:]]*ModuleVersion[[:space:]]*=/s/= .*/= '$VERSION'/" ${root_dirpath}/powershell-client/WindmillClient/WindmillClient.psd1
# sed -i '' -e "/^wmill =/s/= .*/= \"\\^$VERSION\"/" python-client/wmill_pg/pyproject.toml
sed -i '' -e "/^wmill =/s/= .*/= \">=$VERSION\"/" ${root_dirpath}/lsp/Pipfile
sed -i '' -e "/^wmill_pg =/s/= .*/= \">=$VERSION\"/" ${root_dirpath}/lsp/Pipfile
sed -i '' -E "s/name = \"windmill\"\nversion = \"[^\"]*\"\\n(.*)/name = \"windmill\"\nversion = \"$VERSION\"\\n\\1/" ${root_dirpath}/backend/Cargo.lock
cd ${root_dirpath}/frontend && npm i --package-lock-only

View File

@@ -1,27 +1,18 @@
#!/bin/bash
set -euo pipefail
script_dirpath="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
root_dirpath="$(cd "${script_dirpath}/.." && pwd)"
VERSION=$1
echo "Updating versions to: $VERSION"
sed -i -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/backend/Cargo.toml
sed -i -e "/^export const VERSION =/s/= .*/= \"$VERSION\";/" ${root_dirpath}/cli/main.ts
sed -i -e "/^export const VERSION =/s/= .*/= \"v$VERSION\";/" ${root_dirpath}/benchmarks/lib.ts
sed -i -e "/version: /s/: .*/: $VERSION/" ${root_dirpath}/backend/windmill-api/openapi.yaml
sed -i -e "/version: /s/: .*/: $VERSION/" ${root_dirpath}/openflow.openapi.yaml
sed -i -e "/\"version\": /s/: .*,/: \"$VERSION\",/" ${root_dirpath}/typescript-client/package.json
sed -i -e "/\"version\": /s/: .*,/: \"$VERSION\",/" ${root_dirpath}/typescript-client/jsr.json
sed -i -e "/\"version\": /s/: .*,/: \"$VERSION\",/" ${root_dirpath}/frontend/package.json
sed -i -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/python-client/wmill/pyproject.toml
sed -i -e "/^windmill-api =/s/= .*/= \"\\^$VERSION\"/" ${root_dirpath}/python-client/wmill/pyproject.toml
sed -i -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/python-client/wmill_pg/pyproject.toml
sed -i -e "/^[[:space:]]*ModuleVersion[[:space:]]*=/s/= .*/= '$VERSION'/" ${root_dirpath}/powershell-client/WindmillClient/WindmillClient.psd1
# sed -i -e "/^wmill =/s/= .*/= \"\\^$VERSION\"/" ${root_dirpath}/python-client/wmill_pg/pyproject.toml
sed -i -e "/^wmill =/s/= .*/= \">=$VERSION\"/" ${root_dirpath}/lsp/Pipfile
sed -i -e "/^wmill_pg =/s/= .*/= \">=$VERSION\"/" ${root_dirpath}/lsp/Pipfile
sed -i -e "/^version =/s/= .*/= \"$VERSION\"/" backend/Cargo.toml
sed -i -e "/version: /s/: .*/: $VERSION/" backend/openapi.yaml
sed -i -e "/\"version\": /s/: .*,/: \"$VERSION\",/" frontend/package.json
sed -i -e "/^version =/s/= .*/= \"$VERSION\"/" python-client/wmill/pyproject.toml
sed -i -e "/^windmill-api =/s/= .*/= \"\\^$VERSION\"/" python-client/wmill/pyproject.toml
sed -i -e "/^version =/s/= .*/= \"$VERSION\"/" python-client/wmill_pg/pyproject.toml
# sed -i -e "/^wmill =/s/= .*/= \"\\^$VERSION\"/" python-client/wmill_pg/pyproject.toml
sed -i -e "/^wmill =/s/= .*/= \">=$VERSION\"/" lsp/Pipfile
sed -i -e "/^wmill_pg =/s/= .*/= \">=$VERSION\"/" lsp/Pipfile
sed -i -zE "s/name = \"windmill\"\nversion = \"[^\"]*\"\\n(.*)/name = \"windmill\"\nversion = \"$VERSION\"\\n\\1/" ${root_dirpath}/backend/Cargo.lock
sed -i -zE "s/name = \"windmill\"\nversion = \"[^\"]*\"\\n(.*)/name = \"windmill\"\nversion = \"$VERSION\"\\n\\1/" backend/Cargo.lock
cd ${root_dirpath}/frontend && npm i --package-lock-only --ignore-scripts
cd frontend && npm i --package-lock-only

View File

@@ -1,14 +0,0 @@
#!/bin/bash
RT=$(curl -s https://hub.windmill.dev/resource_types/list | jq -c -r '.[]')
for item in ${RT[@]}; do
name=$(jq -r '.name' <<< "$item")
id=$(jq -r '.id' <<< "$item")
echo $name $id
body=$(curl -s -H "accept: application/json" https://hub.windmill.dev/resource_types/${id}/${name})
jq -r '.resource_type.schema' <<< "$body" > ./tmp
description=$(jq -r '.resource_type.description' <<< "$body")
description=$(echo -E $description)
echo "{\"workspace_id\": \"admins\", \"name\": \"$name\", \"schema\": $(cat ./tmp), \"description\": \"$description\"} " | jq . > community/resource_types/${name}.json
rm ./tmp
done

View File

@@ -1,94 +0,0 @@
name: Aider Auto-fix PR Review Change Requests
on:
pull_request_review:
types: [submitted]
jobs:
check-membership:
if: github.event.review.state == 'changes_requested' && contains(github.event.pull_request.title, '[Aider PR]')
runs-on: ubicloud-standard-2
outputs:
is_member: ${{ steps.check-membership.outputs.is_member }}
steps:
- name: Check organization membership
id: check-membership
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_REPOSITORY: ${{ github.repository }}
REVIEWER: ${{ github.event.review.user.login }}
ORG_ACCESS_TOKEN: ${{ secrets.ORG_ACCESS_TOKEN }}
run: |
ORG="windmill-labs"
STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
-H "Authorization: token $ORG_ACCESS_TOKEN" \
-H "Accept: application/vnd.github+json" \
-H "X-GitHub-Api-Version: 2022-11-28" \
"https://api.github.com/orgs/$ORG/members/$REVIEWER")
if [ "$STATUS" -eq 204 ]; then
echo "is_member=true" >> $GITHUB_OUTPUT
else
echo "is_member=false" >> $GITHUB_OUTPUT
fi
check-and-prepare:
needs: check-membership
if: github.event.review.state == 'changes_requested' && contains(github.event.pull_request.title, '[Aider PR]') && needs.check-membership.outputs.is_member == 'true'
runs-on: ubicloud-standard-2
permissions:
contents: write
pull-requests: write
outputs:
prompt_content: ${{ steps.prepare_prompt.outputs.prompt_content }}
env:
GEMINI_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
WINDMILL_TOKEN: ${{ secrets.WINDMILL_TOKEN }}
steps:
- name: Acknowledge Request
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_REPOSITORY: ${{ github.repository }}
run: |
echo "Commenting on PR #${{ github.event.pull_request.number }} to acknowledge the /aider command."
gh pr comment ${{ github.event.pull_request.number }} --body "🤖 Aider is starting to work on your request. Please be patient, this might take a few minutes." --repo $GITHUB_REPOSITORY
- name: Prepare prompt for Aider
id: prepare_prompt
shell: bash
env:
GITHUB_REPOSITORY: ${{ github.repository }}
PR_NUMBER: ${{ github.event.pull_request.number }}
REVIEW_BODY: ${{ github.event.review.body }}
run: |
REVIEW_BODY_ESCAPED="${REVIEW_BODY//\\/\\\\}"
REVIEW_BODY_ESCAPED="${REVIEW_BODY_ESCAPED//\"/\\\"}"
ALL_REVIEW_COMMENTS=$(gh api \
-H "Accept: application/vnd.github+json" \
-H "X-GitHub-Api-Version: 2022-11-28" \
/repos/$GITHUB_REPOSITORY/pulls/$PR_NUMBER/comments)
FORMATTED_COMMENTS=$(jq -r '[.[] | {diff_hunk: .diff_hunk, path: .path, body: .body}]' <<< "$ALL_REVIEW_COMMENTS")
BASE_PROMPT="Fix the following issues in the PR based on the review feedback. The review body is prepended with REVIEW. The review comments are prepended with REVIEW_COMMENTS. The review body and comments are separated by a blank line."
COMPLETE_PROMPT="${BASE_PROMPT}"$'\n'"REVIEW:"$'\n'"${REVIEW_BODY_ESCAPED}"$'\n'"REVIEW_COMMENTS:"$'\n'"${FORMATTED_COMMENTS}"
echo "prompt_content<<EOF" >> $GITHUB_OUTPUT
echo "$COMPLETE_PROMPT" >> $GITHUB_OUTPUT
echo "EOF" >> $GITHUB_OUTPUT
run-aider:
needs: [check-membership, check-and-prepare]
if: github.event.review.state == 'changes_requested' && contains(github.event.pull_request.title, '[Aider PR]') && needs.check-membership.outputs.is_member == 'true'
uses: ./.github/workflows/aider-common.yml
with:
needs_processing: false
base_prompt: ${{ needs.check-and-prepare.outputs.prompt_content }}
rules_files: ".cursor/rules/rust-best-practices.mdc .cursor/rules/svelte5-best-practices.mdc .cursor/rules/windmill-overview.mdc"
secrets: inherit

View File

@@ -1,522 +0,0 @@
name: Aider Common Steps
on:
workflow_call:
inputs:
issue_title:
description: "Title of the issue or PR"
required: false
type: string
issue_body:
description: "Body of the issue or PR"
required: false
type: string
instruction:
description: "Instruction for Aider"
required: false
type: string
issue_id:
description: "ID of the issue or PR"
required: false
type: string
needs_processing:
description: "Whether the issue needs to be processed by the external API"
required: false
type: boolean
default: true
base_prompt:
description: "Base prompt for Aider"
required: false
type: string
default: "Try to fix the following issue based on the instruction given by the user. The issue is prepended with the word ISSUE. The instruction is prepended with the word INSTRUCTION. The issue and instruction are separated by a blank line."
probe_prompt:
description: "Prompt for probe-chat"
required: false
type: string
default: 'I''m giving you a request that needs to be implemented. Your role is ONLY to give me the files that are relevant to the request and nothing else. The request is prepended with the word REQUEST. Give me all the files relevant to this request. Your output MUST be a single json array that can be parsed with programatic json parsing, with the relevant files. Files can be rust or typescript or javascript files. DO NOT INCLUDE ANY OTHER TEXT IN YOUR OUTPUT. ONLY THE JSON ARRAY. Example of output: ["file1.py", "file2.py"]'
rules_files:
description: "Rules files for Aider"
required: false
type: string
outputs:
files_to_edit:
description: "Files identified by probe-chat for editing"
value: ${{ jobs.common-steps.outputs.files_to_edit }}
final_prompt:
description: "Final prompt for Aider"
value: ${{ jobs.common-steps.outputs.final_prompt }}
pr_branch_name:
description: "Name of the branch used for PR"
value: ${{ jobs.common-steps.outputs.pr_branch_name }}
changes_applied_message:
description: "Message indicating changes were applied"
value: ${{ jobs.common-steps.outputs.changes_applied_message }}
changes_applied:
description: "Boolean indicating if changes were successfully applied"
value: ${{ jobs.common-steps.outputs.changes_applied }}
jobs:
common-steps:
runs-on: ubicloud-standard-8
outputs:
files_to_edit: ${{ steps.probe_files.outputs.files_to_edit }}
final_prompt: ${{ steps.create_prompt.outputs.final_prompt }}
pr_branch_name: ${{ steps.commit_and_push.outputs.PR_BRANCH_NAME }}
changes_applied_message: ${{ steps.commit_and_push.outputs.CHANGES_APPLIED_MESSAGE }}
changes_applied: ${{ steps.commit_and_push.outputs.CHANGES_APPLIED }}
env:
GEMINI_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
WINDMILL_TOKEN: ${{ secrets.WINDMILL_TOKEN }}
LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}
DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_AI_BOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@v2
with:
egress-policy: audit
- name: Check out code
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Checkout PR Branch
id: checkout_pr
if: (github.event_name == 'issue_comment' && github.event.issue.pull_request) || (github.event_name == 'pull_request_review')
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
echo "Issue comment trigger: Checking out PR branch..."
PR_NUMBER=""
if [ -n "${{ github.event.issue.number }}" ]; then
PR_NUMBER="${{ github.event.issue.number }}"
elif [ -n "${{ github.event.pull_request.number }}" ]; then
PR_NUMBER="${{ github.event.pull_request.number }}"
else
echo "::error::Could not determine PR number."
exit 1
fi
PR_HEAD_REF=$(gh pr view $PR_NUMBER --json headRefName -q .headRefName --repo $GITHUB_REPOSITORY)
if [[ -z "$PR_HEAD_REF" || "$PR_HEAD_REF" == "null" ]]; then
echo "::error::Could not determine PR head branch for PR #$PR_NUMBER via gh CLI."
exit 1
fi
echo "Checking out PR head branch: $PR_HEAD_REF for PR #$PR_NUMBER"
git fetch origin "refs/heads/${PR_HEAD_REF}:refs/remotes/origin/${PR_HEAD_REF}" --no-tags
git checkout "$PR_HEAD_REF"
echo "Successfully checked out branch $(git rev-parse --abbrev-ref HEAD)"
echo "PR_BRANCH=$PR_HEAD_REF" >> $GITHUB_OUTPUT
- name: Configure Git User
run: |
git config --global user.name "github-actions[bot]"
git config --global user.email "github-actions[bot]@users.noreply.github.com"
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Cache Python dependencies
uses: actions/cache@v3
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt', '**/setup.py') }}
restore-keys: |
${{ runner.os }}-pip-
- name: Install Aider and Dependencies
run: |
echo "Installing Aider..."
python -m pip install uv
python -m venv ~/uv-env
source ~/uv-env/bin/activate
uv pip install configargparse==1.7
uv pip install aider-chat==0.83.1
uv pip install -U google-generativeai
sudo apt-get update && sudo apt-get install -y jq
echo "$HOME/.local/bin" >> $GITHUB_PATH
echo "VIRTUAL_ENV_PATH=$HOME/uv-env" >> $GITHUB_ENV
- name: Create Prompt for Aider
id: create_prompt
shell: bash
env:
BASE_PROMPT_ENV: ${{ inputs.base_prompt }}
ISSUE_TITLE_ENV: ${{ inputs.issue_title }}
ISSUE_BODY_ENV: ${{ inputs.issue_body }}
INSTRUCTION_ENV: ${{ inputs.instruction }}
NEEDS_PROCESSING_ENV: ${{ inputs.needs_processing }}
WINDMILL_TOKEN: ${{ secrets.WINDMILL_TOKEN }}
run: |
set -e
FINAL_PROMPT_CONTENT=""
if [[ "$ISSUE_TITLE_ENV" != "" && "$ISSUE_BODY_ENV" != "" ]]; then
echo "Processing issue with title: $ISSUE_TITLE_ENV"
if [[ "$NEEDS_PROCESSING_ENV" == "true" ]]; then
echo "Needs processing is true. Calling Windmill API..."
JSON_PAYLOAD=$(jq -n \
--arg title "$ISSUE_TITLE_ENV" \
--arg body "$ISSUE_BODY_ENV" \
'{"body":{"issue_title":$title,"issue_body":$body}}')
echo "Windmill JSON Payload: $JSON_PAYLOAD"
API_RESULT_FILE=$(mktemp)
HTTP_CODE=$(curl -s -o "$API_RESULT_FILE" -w "%{http_code}" \
-X POST "https://app.windmill.dev/api/w/windmill-labs/jobs/run_wait_result/p/f/ai/quiet_script" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $WINDMILL_TOKEN" \
--data-binary "$JSON_PAYLOAD" \
--max-time 90)
BODY_CONTENT=$(cat "$API_RESULT_FILE")
rm -f "$API_RESULT_FILE" # Clean up temp file
echo "Windmill API HTTP Code: $HTTP_CODE"
if [[ "$HTTP_CODE" -eq 200 ]]; then
PROCESSED_ISSUE_PROMPT=$(echo "$BODY_CONTENT" | jq -r '.effective_body // empty')
if [[ -z "$PROCESSED_ISSUE_PROMPT" || "$PROCESSED_ISSUE_PROMPT" == "null" ]]; then
echo "::warning::Windmill API returned 200 but effective_body was empty or null."
EFFECTIVE_ISSUE_CONTENT_FOR_PROMPT="$ISSUE_BODY_ENV"
else
echo "Successfully processed issue via Windmill API."
EFFECTIVE_ISSUE_CONTENT_FOR_PROMPT="$PROCESSED_ISSUE_PROMPT"
fi
FINAL_PROMPT_CONTENT=$(printf "%s\nISSUE:\n%s\nINSTRUCTION:\n%s" \
"$BASE_PROMPT_ENV" "$EFFECTIVE_ISSUE_CONTENT_FOR_PROMPT" "$INSTRUCTION_ENV")
else
echo "::error::Windmill API call failed (HTTP $HTTP_CODE). Using raw issue content for prompt."
FINAL_PROMPT_CONTENT=$(printf "%s\nISSUE:\n%s\nINSTRUCTION:\n%s" \
"$BASE_PROMPT_ENV" "$ISSUE_BODY_ENV" "$INSTRUCTION_ENV")
fi
else
echo "Needs processing is false. Using raw issue content for prompt."
FINAL_PROMPT_CONTENT=$(printf "%s\nISSUE:\n%s\nINSTRUCTION:\n%s" \
"$BASE_PROMPT_ENV" "$ISSUE_BODY_ENV" "$INSTRUCTION_ENV")
fi
else
echo "No issue title or body given. Using base prompt."
FINAL_PROMPT_CONTENT=$(printf "%s\nINSTRUCTION:\n%s" "$BASE_PROMPT_ENV" "$INSTRUCTION_ENV")
fi
echo "Final prompt: $FINAL_PROMPT_CONTENT"
echo "final_prompt<<EOF_AIDER_PROMPT" >> "$GITHUB_OUTPUT"
echo "$FINAL_PROMPT_CONTENT" >> "$GITHUB_OUTPUT"
echo "EOF_AIDER_PROMPT" >> "$GITHUB_OUTPUT"
- name: Probe Chat for Relevant Files
id: probe_files
shell: bash
env:
FINAL_PROMPT: ${{ steps.create_prompt.outputs.final_prompt }}
PROBE_PROMPT: ${{ inputs.probe_prompt }}
run: |
echo "Running probe-chat to find relevant files..."
MESSAGE_FOR_PROBE=$(printf "%s\nREQUEST:\n%s" "$PROBE_PROMPT" "$FINAL_PROMPT")
set -o pipefail
PROBE_OUTPUT=$(npx --yes @buger/probe-chat@latest --max-iterations 50 --model-name gemini-2.5-pro-preview-05-06 --message "$MESSAGE_FOR_PROBE") || {
echo "::error::probe-chat command failed. Output:"
echo "$PROBE_OUTPUT"
exit 1
}
set +o pipefail
echo "Probe-chat raw output:"
echo "$PROBE_OUTPUT"
JSON_FILES=$(echo "$PROBE_OUTPUT" | sed -n '/^\s*\[/,$p' | sed '/^\s*\]/q')
echo "Extracted JSON block:"
echo "$JSON_FILES"
FILES_LIST=$(echo "$JSON_FILES" | jq -e -r '[.[] | select(type == "string" and . != "" and . != null and (endswith("/") | not))] | join(" ")' || echo "")
if [[ -z "$FILES_LIST" ]]; then
echo "::warning::probe-chat did not identify any relevant files."
fi
echo "Formatted files list for aider: $FILES_LIST"
echo "files_to_edit=$FILES_LIST" >> $GITHUB_OUTPUT
- name: Cache Aider tags
uses: actions/cache@v3
with:
path: .aider.tags.cache.v4
key: ${{ runner.os }}-aider-${{ github.sha }}
restore-keys: |
${{ runner.os }}-aider-
- name: Prepare branch for Aider
id: prepare_branch
env:
ISSUE_ID: ${{ inputs.issue_id }}
run: |
if [[ "$ISSUE_ID" != "" ]]; then
BRANCH_NAME="aider-fix-issue-${ISSUE_ID}"
# Check if branch exists remotely
if git ls-remote --heads origin $BRANCH_NAME | grep -q $BRANCH_NAME; then
echo "Branch $BRANCH_NAME already exists remotely, fetching it"
git fetch origin $BRANCH_NAME
git checkout $BRANCH_NAME
git pull origin $BRANCH_NAME
else
echo "Creating new branch $BRANCH_NAME"
git checkout -b $BRANCH_NAME
fi
echo "BRANCH_NAME=$BRANCH_NAME" >> $GITHUB_OUTPUT
else
# We're in a pull_request_review event
PR_NUMBER="${{ github.event.pull_request.number }}"
PR_HEAD_REF="${{ github.event.pull_request.head.ref }}"
echo "Handling pull_request_review for PR #$PR_NUMBER on branch $PR_HEAD_REF"
# Ensure we're on the correct branch
git config pull.rebase true
git fetch origin $PR_HEAD_REF
git checkout $PR_HEAD_REF
git pull origin $PR_HEAD_REF
echo "Using PR branch $PR_HEAD_REF for PR #$PR_NUMBER"
echo "BRANCH_NAME=$PR_HEAD_REF" >> $GITHUB_OUTPUT
fi
- name: Run Aider
id: run_aider
shell: bash
env:
FILES_TO_EDIT: ${{ steps.probe_files.outputs.files_to_edit }}
FINAL_PROMPT: ${{ steps.create_prompt.outputs.final_prompt }}
RULES_FILES: ${{ inputs.rules_files }}
run: |
source $VIRTUAL_ENV_PATH/bin/activate
echo "$FINAL_PROMPT" > .aider_final_prompt.txt
echo "FILES_TO_EDIT: $FILES_TO_EDIT"
RULES=""
if [ -n "$RULES_FILES" ]; then
for rule in $RULES_FILES; do
RULES="$RULES --read $rule"
done
fi
aider \
$RULES \
$FILES_TO_EDIT \
--model gemini/gemini-2.5-pro-preview-05-06 \
--message-file .aider_final_prompt.txt \
--yes \
--no-check-update \
--auto-commits \
--no-analytics \
--no-gitignore \
| tee .aider_output.txt || true
echo "Aider command completed. Output saved to .aider_output.txt"
- name: Cache Node.js dependencies
uses: actions/cache@v3
with:
path: ~/.npm
key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json', '**/yarn.lock') }}
restore-keys: |
${{ runner.os }}-node-
- name: Commit and Push Changes
id: commit_and_push
env:
ISSUE_ID: ${{ inputs.issue_id }}
BRANCH_NAME: ${{ steps.prepare_branch.outputs.BRANCH_NAME }}
run: |
if [[ "$ISSUE_ID" != "" ]]; then
# Check if there are any uncommitted changes
if [[ -n $(git status --porcelain) ]]; then
echo "Found uncommitted changes, committing them"
git add .
git commit -m "Aider changes"
fi
# Push changes to the branch
if git push origin $BRANCH_NAME; then
echo "Pushed to branch $BRANCH_NAME"
echo "PR_BRANCH_NAME=$BRANCH_NAME" >> $GITHUB_OUTPUT
echo "CHANGES_APPLIED_MESSAGE=Aider changes pushed to branch $BRANCH_NAME." >> $GITHUB_OUTPUT
echo "CHANGES_APPLIED=true" >> $GITHUB_OUTPUT
else
echo "::warning::Push to PR branch $BRANCH_NAME failed."
echo "CHANGES_APPLIED_MESSAGE=Aider ran, but failed to push changes to PR branch $BRANCH_NAME." >> $GITHUB_OUTPUT
echo "CHANGES_APPLIED=false" >> $GITHUB_OUTPUT
fi
else
# We're in a pull_request_review event
PR_HEAD_REF="${{ github.event.pull_request.head.ref }}"
echo "Attempting to push changes to PR branch $PR_HEAD_REF"
if git push origin $PR_HEAD_REF; then
echo "Push to $PR_HEAD_REF successful (or no new changes to push)."
echo "CHANGES_APPLIED_MESSAGE=Aider changes (if any) pushed to PR branch $PR_HEAD_REF." >> $GITHUB_OUTPUT
echo "PR_BRANCH_NAME=$PR_HEAD_REF" >> $GITHUB_OUTPUT
echo "CHANGES_APPLIED=true" >> $GITHUB_OUTPUT
else
echo "::warning::Push to PR branch $PR_HEAD_REF failed."
echo "CHANGES_APPLIED_MESSAGE=Aider ran, but failed to push changes to PR branch $PR_HEAD_REF." >> $GITHUB_OUTPUT
echo "CHANGES_APPLIED=false" >> $GITHUB_OUTPUT
fi
fi
- name: Create Pull Request
if: always() && (github.event_name == 'issue_comment' || github.event_name == 'repository_dispatch') && !github.event.issue.pull_request && steps.commit_and_push.outputs.PR_BRANCH_NAME != ''
id: create_pr
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_BRANCH: ${{ steps.commit_and_push.outputs.PR_BRANCH_NAME }}
ISSUE_NUM: ${{ inputs.issue_id }}
ISSUE_TITLE: ${{ inputs.issue_title }}
GITHUB_EVENT_NAME: ${{ github.event_name }}
run: |
# Create PR description in a temporary file to avoid command line length limits and ensure it stays under 40k chars
HEADER="This PR was created automatically by Aider to fix issue #${ISSUE_NUM}."
# if event is repository_dispatch, add the issue title to the header
if [ "$GITHUB_EVENT_NAME" == "repository_dispatch" ]; then
if [[ "${{ github.event.client_payload.source }}" == "linear" ]]; then
HEADER="This PR was created automatically by Aider to fix issue #linear:${ISSUE_NUM}."
elif [[ "${{ github.event.client_payload.source }}" == "discord" ]]; then
HEADER="This PR was created automatically by Aider to fix issue #discord:${ISSUE_NUM}."
fi
fi
cat > /tmp/pr-description.md << EOL | head -c 40000
$HEADER
## Aider Output
\`\`\`
$(cat .aider_output.txt || echo "No output available")
\`\`\`
EOL
# Create PR using the file for the body content, handle errors gracefully
set +e # Don't exit on error
PR_TITLE="[Aider PR] Fix: ${ISSUE_TITLE}"
if [ -z "$ISSUE_TITLE" ]; then
PR_TITLE="[Aider PR] AI changes after request"
fi
gh pr create \
--title "$PR_TITLE" \
--body-file /tmp/pr-description.md \
--head "$PR_BRANCH" \
--base main \
--draft
PR_CREATE_EXIT_CODE=$?
set -e # Re-enable exit on error
if [ $PR_CREATE_EXIT_CODE -eq 0 ]; then
echo "PR created successfully"
PR_URL=$(gh pr view $PR_BRANCH --json url --jq .url)
echo "PR_URL=$PR_URL" >> $GITHUB_OUTPUT
echo "PR_CREATED=true" >> $GITHUB_OUTPUT
else
echo "Warning: Failed to create PR. Exit code: $PR_CREATE_EXIT_CODE"
echo "PR_CREATED=false" >> $GITHUB_OUTPUT
# Continue workflow despite PR creation failure
fi
- name: Comment on PR with Aider Output
if: always() && github.event_name == 'pull_request_review' && steps.commit_and_push.outputs.CHANGES_APPLIED != ''
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUM: ${{ github.event.pull_request.number }}
JOB_STATUS: ${{ job.status }}
run: |
# Create comment body in a temporary file to avoid command line length limits
if [[ "${{ steps.commit_and_push.outputs.CHANGES_APPLIED }}" == "true" ]]; then
if [[ "$JOB_STATUS" == "success" ]]; then
STATUS_PREFIX="🤖 I've automatically addressed the feedback based on the review."
else
STATUS_PREFIX="⚠️ I attempted to address the feedback, but encountered some issues."
fi
else
if [[ "$JOB_STATUS" == "success" ]]; then
STATUS_PREFIX="🤖 I attempted to address the review feedback, but no modifications were made."
else
STATUS_PREFIX="⚠️ I encountered issues while attempting to address the feedback, and no modifications were made."
fi
fi
cat > /tmp/pr-comment.md << EOL
${STATUS_PREFIX}
## Aider Output
\`\`\`
$(cat .aider_output.txt || echo 'No output available')
\`\`\`
Please review the output and provide additional guidance if needed.
EOL
# Use the file for comment body
gh pr comment $PR_NUM --body-file /tmp/pr-comment.md
- name: Comment on issue/PR to let the user know Aider has finished working on the request
if: always() && github.event_name == 'issue_comment'
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_REPOSITORY: ${{ github.repository }}
JOB_STATUS: ${{ job.status }}
PR_CREATED: ${{ steps.create_pr.outputs.PR_CREATED }}
PR_URL: ${{ steps.create_pr.outputs.PR_URL }}
run: |
echo "Commenting on issue/PR #${{ github.event.issue.number }} to let the user know Aider has finished working on the request."
if [[ "$JOB_STATUS" == "success" ]]; then
if [[ "$PR_CREATED" == "true" ]]; then
COMMENT_BODY="🤖 Aider has finished working on your request. A PR has been created. $PR_URL"
else
COMMENT_BODY="🤖 Aider has finished working on your request, but was unable to create a PR."
fi
else
COMMENT_BODY="⚠️ Aider encountered issues while working on your request. Please check the workflow logs for details."
fi
gh issue comment ${{ github.event.issue.number }} --body "$COMMENT_BODY" --repo $GITHUB_REPOSITORY
- name: Comment on linear issue to let the user know Aider has finished working on the request
if: always() && github.event_name == 'repository_dispatch'
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_REPOSITORY: ${{ github.repository }}
JOB_STATUS: ${{ job.status }}
LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}
PR_CREATED: ${{ steps.create_pr.outputs.PR_CREATED }}
PR_URL: ${{ steps.create_pr.outputs.PR_URL }}
DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_AI_BOT_TOKEN }}
SOURCE: ${{ github.event.client_payload.source }}
run: |
echo "Notifying user about Aider completion status for $SOURCE request #${{ github.event.client_payload.issue_id }}"
if [[ "$JOB_STATUS" == "success" ]]; then
if [[ "$PR_CREATED" == "true" ]]; then
COMMENT_BODY="🤖 Aider has finished working on your request. A PR has been created. $PR_URL"
else
COMMENT_BODY="🤖 Aider has finished working on your request, but was unable to create a PR."
fi
else
COMMENT_BODY="⚠️ Aider encountered issues while working on your request. Please check the workflow logs for details."
fi
if [[ "$SOURCE" == "discord" ]]; then
curl -X POST \
-H "Authorization: Bot $DISCORD_BOT_TOKEN" \
-H "Content-Type: application/json" \
"https://discord.com/api/v10/channels/${{ github.event.client_payload.channel_id }}/messages" \
-d "{\"content\":\"${COMMENT_BODY}\"}"
else
curl -X POST \
-H "Authorization: $LINEAR_API_KEY" \
-H "Content-Type: application/json" \
"https://api.linear.app/graphql" \
-d "{\"query\":\"mutation { commentCreate(input: { issueId: \\\"${{ github.event.client_payload.issue_id }}\\\", body: \\\"${COMMENT_BODY}\\\" }) { success } }\"}"
fi

View File

@@ -1,80 +0,0 @@
name: External Aider Issue Fix
on:
repository_dispatch:
types: [external_issue_fix]
jobs:
check-and-prepare:
runs-on: ubicloud-standard-2
permissions:
contents: write
pull-requests: write
outputs:
issue_title: ${{ steps.determine_inputs.outputs.ISSUE_TITLE }}
issue_body: ${{ steps.determine_inputs.outputs.ISSUE_BODY }}
instruction: ${{ steps.determine_inputs.outputs.INSTRUCTION }}
env:
GEMINI_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
WINDMILL_TOKEN: ${{ secrets.WINDMILL_TOKEN }}
LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}
DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_AI_BOT_TOKEN }}
steps:
- name: Acknowledge Request
env:
LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}
DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_AI_BOT_TOKEN }}
run: |
if [[ "${{ github.event.client_payload.source }}" == "linear" ]]; then
echo "Commenting on Linear issue #${{ github.event.client_payload.issue_id }} to acknowledge the request."
curl -X POST \
-H "Authorization: $LINEAR_API_KEY" \
-H "Content-Type: application/json" \
"https://api.linear.app/graphql" \
-d "{\"query\":\"mutation { commentCreate(input: { issueId: \\\"${{ github.event.client_payload.issue_id }}\\\", body: \\\"🤖 Aider is starting to work on your request. I'll update you here once I have a PR ready. Please be patient, this might take a few minutes.\\\" }) { success } }\"}"
elif [[ "${{ github.event.client_payload.source }}" == "discord" ]]; then
echo "Commenting on Discord thread #${{ github.event.client_payload.channel_id }} to acknowledge the request."
curl -X POST \
-H "Authorization: Bot $DISCORD_BOT_TOKEN" \
-H "Content-Type: application/json" \
"https://discord.com/api/v10/channels/${{ github.event.client_payload.channel_id }}/messages" \
-d "{\"content\":\"🤖 Aider is starting to work on your request. I'll update you here once I have a PR ready. Please be patient, this might take a few minutes.\"}"
fi
- name: Determine inputs for Aider
id: determine_inputs
shell: bash
env:
ISSUE_TITLE: ${{ github.event.client_payload.issue_title }}
ISSUE_BODY: ${{ github.event.client_payload.issue_body }}
INSTRUCTION: ${{ github.event.client_payload.instruction }}
run: |
echo "Determining inputs for Aider..."
echo "ISSUE_TITLE<<EOF_AIDER_TITLE" >> "$GITHUB_OUTPUT"
echo "$ISSUE_TITLE" >> "$GITHUB_OUTPUT"
echo "EOF_AIDER_TITLE" >> "$GITHUB_OUTPUT"
echo "ISSUE_BODY<<EOF_AIDER_BODY" >> "$GITHUB_OUTPUT"
echo "$ISSUE_BODY" >> "$GITHUB_OUTPUT"
echo "EOF_AIDER_BODY" >> "$GITHUB_OUTPUT"
echo "INSTRUCTION<<EOF_AIDER_INSTRUCTION" >> "$GITHUB_OUTPUT"
echo "$INSTRUCTION" >> "$GITHUB_OUTPUT"
echo "EOF_AIDER_INSTRUCTION" >> "$GITHUB_OUTPUT"
echo "Finished determining inputs."
run-aider:
needs: check-and-prepare
uses: ./.github/workflows/aider-common.yml
with:
issue_title: ${{ needs.check-and-prepare.outputs.issue_title }}
issue_body: ${{ needs.check-and-prepare.outputs.issue_body }}
instruction: ${{ needs.check-and-prepare.outputs.instruction }}
issue_id: ${{ github.event.client_payload.issue_id }}
rules_files: ".cursor/rules/rust-best-practices.mdc .cursor/rules/svelte5-best-practices.mdc .cursor/rules/windmill-overview.mdc"
secrets: inherit

View File

@@ -1,165 +0,0 @@
name: Aider Auto-fix issues and PR comments via external prompt
on:
issue_comment:
types: [created]
jobs:
check-membership:
runs-on: ubicloud-standard-2
if: |
github.event_name == 'issue_comment' &&
contains(github.event.comment.body, '/aider') &&
!contains(github.event.comment.user.login, '[bot]')
outputs:
is_member: ${{ steps.check-membership.outputs.is_member }}
steps:
- name: Check organization membership
id: check-membership
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_REPOSITORY: ${{ github.repository }}
COMMENTER: ${{ github.event.comment.user.login }}
ORG_ACCESS_TOKEN: ${{ secrets.ORG_ACCESS_TOKEN }}
run: |
ORG="windmill-labs"
STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
-H "Authorization: token $ORG_ACCESS_TOKEN" \
-H "Accept: application/vnd.github+json" \
-H "X-GitHub-Api-Version: 2022-11-28" \
"https://api.github.com/orgs/$ORG/members/$COMMENTER")
if [ "$STATUS" -eq 204 ]; then
echo "is_member=true" >> $GITHUB_OUTPUT
else
echo "is_member=false" >> $GITHUB_OUTPUT
fi
check-and-prepare:
needs: check-membership
runs-on: ubicloud-standard-2
if: needs.check-membership.outputs.is_member == 'true'
permissions:
contents: write
pull-requests: write
issues: write
env:
GEMINI_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
WINDMILL_TOKEN: ${{ secrets.WINDMILL_TOKEN }}
outputs:
issue_title: ${{ steps.determine_inputs.outputs.ISSUE_TITLE }}
issue_body: ${{ steps.determine_inputs.outputs.ISSUE_BODY }}
comment_content: ${{ steps.determine_inputs.outputs.COMMENT_CONTENT }}
pr_branch: ${{ steps.checkout_pr.outputs.PR_BRANCH }}
steps:
- name: Acknowledge Request
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_REPOSITORY: ${{ github.repository }}
run: |
echo "Commenting on issue/PR #${{ github.event.issue.number }} to acknowledge the /aider command."
gh issue comment ${{ github.event.issue.number }} --body "🤖 Aider is starting to work on your request. I'll update you here once I have a PR ready. Please be patient, this might take a few minutes." --repo $GITHUB_REPOSITORY
- name: Determine inputs for Aider
id: determine_inputs
shell: bash
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
COMMENT_BODY: ${{ github.event.comment.body }}
ISSUE_NUMBER: ${{ github.event.issue.number }}
GITHUB_REPOSITORY: ${{ github.repository }}
LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}
run: |
echo "Determining inputs for Aider..."
ISSUE_TITLE_VAL=""
ISSUE_BODY_VAL=""
if [[ ! -z "${{ github.event.issue.pull_request }}" ]]; then
echo "This is a comment on a Pull Request"
PR_NUMBER="$ISSUE_NUMBER"
PR_BODY_JSON=$(gh pr view "$PR_NUMBER" --json body --repo "$GITHUB_REPOSITORY")
if [[ $? -ne 0 ]]; then
echo "Error fetching PR body for PR #$PR_NUMBER"
PR_BODY_VAL=""
else
PR_BODY_VAL=$(jq -r '.body // ""' <<< "$PR_BODY_JSON")
fi
if [[ ! -z "$PR_BODY_VAL" ]]; then
REFERENCED_ISSUE=""
if [[ "$PR_BODY_VAL" =~ \#linear:([a-f0-9-]+) ]]; then
REFERENCED_ISSUE="${BASH_REMATCH[1]}"
echo "Found referenced Linear issue #$REFERENCED_ISSUE in PR description"
LINEAR_ISSUE_JSON=$(curl -s -H "Authorization: $LINEAR_API_KEY" \
"https://api.linear.app/graphql" \
-X POST \
-H "Content-Type: application/json" \
-d "{\"query\":\"query { issue(id: \\\"$REFERENCED_ISSUE\\\") { title description } }\"}")
if [[ $? -eq 0 && ! "$LINEAR_ISSUE_JSON" =~ "error" ]]; then
ISSUE_TITLE_VAL=$(jq -r '.data.issue.title // ""' <<< "$LINEAR_ISSUE_JSON")
ISSUE_BODY_VAL=$(jq -r '.data.issue.description // ""' <<< "$LINEAR_ISSUE_JSON")
echo "Successfully fetched Linear issue details"
else
echo "Error fetching Linear issue details for #$REFERENCED_ISSUE"
fi
elif [[ "$PR_BODY_VAL" =~ \#([0-9]+) ]]; then
REFERENCED_ISSUE="${BASH_REMATCH[1]}"
echo "Found referenced GitHub issue #$REFERENCED_ISSUE in PR description"
ISSUE_DETAILS_JSON=$(gh issue view "$REFERENCED_ISSUE" --json title,body --repo "$GITHUB_REPOSITORY")
if [[ $? -ne 0 ]]; then
echo "Error fetching issue details for #$REFERENCED_ISSUE"
else
ISSUE_TITLE_VAL=$(jq -r '.title // ""' <<< "$ISSUE_DETAILS_JSON")
ISSUE_BODY_VAL=$(jq -r '.body // ""' <<< "$ISSUE_DETAILS_JSON")
fi
fi
else
echo "PR body is empty or could not be fetched."
fi
else
echo "This is a comment on a regular issue"
ISSUE_DETAILS_JSON=$(gh issue view "$ISSUE_NUMBER" --json title,body --repo "$GITHUB_REPOSITORY")
if [[ $? -ne 0 ]]; then
echo "Error fetching issue details for #$ISSUE_NUMBER"
else
ISSUE_TITLE_VAL=$(jq -r '.title // ""' <<< "$ISSUE_DETAILS_JSON")
ISSUE_BODY_VAL=$(jq -r '.body // ""' <<< "$ISSUE_DETAILS_JSON")
fi
fi
echo "ISSUE_TITLE<<EOF_AIDER_TITLE" >> "$GITHUB_OUTPUT"
echo "$ISSUE_TITLE_VAL" >> "$GITHUB_OUTPUT"
echo "EOF_AIDER_TITLE" >> "$GITHUB_OUTPUT"
echo "ISSUE_BODY<<EOF_AIDER_BODY" >> "$GITHUB_OUTPUT"
echo "$ISSUE_BODY_VAL" >> "$GITHUB_OUTPUT"
echo "EOF_AIDER_BODY" >> "$GITHUB_OUTPUT"
CLEAN_COMMENT="${COMMENT_BODY/\/aider/}"
CLEAN_COMMENT="${CLEAN_COMMENT#"${CLEAN_COMMENT%%[![:space:]]*}"}"
CLEAN_COMMENT="${CLEAN_COMMENT%"${CLEAN_COMMENT##*[![:space:]]}"}"
echo "COMMENT_CONTENT<<EOF_AIDER_COMMENT" >> "$GITHUB_OUTPUT"
echo "$CLEAN_COMMENT" >> "$GITHUB_OUTPUT"
echo "EOF_AIDER_COMMENT" >> "$GITHUB_OUTPUT"
echo "Finished determining inputs."
run-aider:
needs: [check-membership, check-and-prepare]
if: needs.check-membership.outputs.is_member == 'true'
uses: ./.github/workflows/aider-common.yml
with:
issue_title: ${{ needs.check-and-prepare.outputs.issue_title }}
issue_body: ${{ needs.check-and-prepare.outputs.issue_body }}
instruction: ${{ needs.check-and-prepare.outputs.comment_content }}
issue_id: ${{ github.event.issue.number }}
rules_files: ".cursor/rules/rust-best-practices.mdc .cursor/rules/svelte5-best-practices.mdc .cursor/rules/windmill-overview.mdc"
secrets: inherit

View File

@@ -1,26 +0,0 @@
# name: dependabot auto-merge
# on: pull_request_target
# permissions:
# contents: read
# pull-requests: read
# jobs:
# dependabot:
# runs-on: ubuntu-latest
# if: ${{ github.actor == 'dependabot[bot]' }}
# steps:
# - name: Dependabot metadata
# id: metadata
# uses: dependabot/fetch-metadata@v1.6.0
# with:
# github-token: "${{ secrets.GITHUB_TOKEN }}"
# - name: Enable auto-merge for Dependabot PRs
# if: steps.metadata.outputs.update-type == 'version-update:semver-patch' || steps.metadata.outputs.update-type == 'version-update:semver-minor'
# run: |
# echo ${{ secrets.RUBEN_PAT }} | gh auth login --with-token
# gh pr review --approve "$PR_URL"
# gh pr merge --auto --squash "$PR_URL"
# env:
# PR_URL: ${{github.event.pull_request.html_url}}

View File

@@ -1,133 +0,0 @@
name: Backend check
on:
workflow_run:
workflows: ["Change versions"]
types:
- completed
push:
paths:
- "backend/**"
- ".github/workflows/backend-check.yml"
jobs:
check_oss:
runs-on: ubicloud-standard-8
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions-rust-lang/setup-rust-toolchain@v1
with:
cache-workspaces: backend
toolchain: 1.85.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: backend
- name: cargo check
working-directory: ./backend
timeout-minutes: 16
run: SQLX_OFFLINE=true cargo check
check_oss_full:
runs-on: ubicloud-standard-8
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: install xmlsec1
run: |
sudo apt-get update
sudo apt-get install -y libxml2-dev libxmlsec1-dev
- uses: actions-rust-lang/setup-rust-toolchain@v1
with:
cache-workspaces: backend
toolchain: 1.85.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: backend
- name: cargo check
working-directory: ./backend
timeout-minutes: 16
run: |
mkdir -p fake_frontend_build
FRONTEND_BUILD_DIR=$(pwd)/fake_frontend_build SQLX_OFFLINE=true cargo check --all-features
check_ee:
runs-on: ubicloud-standard-8
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Read EE repo commit hash
run: |
echo "ee_repo_ref=$(cat ./backend/ee-repo-ref.txt)" >> "$GITHUB_ENV"
- uses: actions/checkout@v4
with:
repository: windmill-labs/windmill-ee-private
path: ./windmill-ee-private
ref: ${{ env.ee_repo_ref }}
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
fetch-depth: 0
- name: Substitute EE code (EE logic is behind feature flag)
run: |
./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
- uses: actions-rust-lang/setup-rust-toolchain@v1
with:
cache-workspaces: backend
toolchain: 1.85.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: backend
- name: cargo check
working-directory: ./backend
timeout-minutes: 16
run: SQLX_OFFLINE=true cargo check
check_ee_full:
runs-on: ubicloud-standard-8
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Read EE repo commit hash
run: |
echo "ee_repo_ref=$(cat ./backend/ee-repo-ref.txt)" >> "$GITHUB_ENV"
- uses: actions/checkout@v4
with:
repository: windmill-labs/windmill-ee-private
path: ./windmill-ee-private
ref: ${{ env.ee_repo_ref }}
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
fetch-depth: 0
- name: install xmlsec1
run: |
sudo apt-get update
sudo apt-get install -y libxml2-dev libxmlsec1-dev
- name: Substitute EE code (EE logic is behind feature flag)
run: |
./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
- uses: actions-rust-lang/setup-rust-toolchain@v1
with:
cache-workspaces: backend
toolchain: 1.85.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: backend
- name: cargo check
timeout-minutes: 16
working-directory: ./backend
run: |
mkdir -p fake_frontend_build
FRONTEND_BUILD_DIR=$(pwd)/fake_frontend_build SQLX_OFFLINE=true cargo check --all-features

View File

@@ -1,68 +0,0 @@
name: Backend only integration tests
on:
push:
branches:
- "main"
paths:
- "backend/**"
- ".github/workflows/backend-test.yml"
pull_request:
types: [opened, synchronize, reopened]
paths:
- "backend/**"
- ".github/workflows/backend-test.yml"
defaults:
run:
working-directory: ./backend
jobs:
cargo_test:
runs-on: ubicloud-standard-8
services:
postgres:
image: postgres
ports:
- 5432:5432
env:
POSTGRES_DB: windmill
POSTGRES_PASSWORD: changeme
options: >-
--health-cmd pg_isready --health-interval 10s --health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@v4
- uses: actions/setup-dotnet@v4
with:
dotnet-version: "9.0.x"
- uses: denoland/setup-deno@v2
with:
deno-version: v2.x
- uses: actions/setup-go@v2
with:
go-version: 1.21.5
- uses: oven-sh/setup-bun@v2
with:
bun-version: 1.1.43
- uses: astral-sh/setup-uv@v6
with:
version: "0.6.2"
- uses: actions-rust-lang/setup-rust-toolchain@v1
with:
cache-workspaces: backend
toolchain: 1.85.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: backend
- name: cargo test
timeout-minutes: 16
run:
deno --version && bun -v && go version && python3 --version &&
SQLX_OFFLINE=true
DATABASE_URL=postgres://postgres:changeme@localhost:5432/windmill
DISABLE_EMBEDDING=true RUST_LOG=info
DENO_PATH=$(which deno) BUN_PATH=$(which bun) GO_PATH=$(which go)
UV_PATH=$(which uv) cargo test --features
enterprise,deno_core,license,python,rust,scoped_cache --all --
--nocapture

View File

@@ -1,324 +0,0 @@
name: Run benchmarks
on:
schedule:
- cron: "0 0 */1 * *"
workflow_dispatch:
jobs:
benchmark_single:
runs-on: ubicloud-standard-8
services:
postgres:
image: postgres
env:
POSTGRES_DB: windmill
POSTGRES_PASSWORD: changeme
POSTGRES_INITDB_ARGS: "-c shared_buffers=2GB -c work_mem=32MB -c effective_cache_size=4GB"
options: >-
--health-cmd pg_isready --health-interval 10s --health-timeout 5s
--health-retries 5
--shm-size=2g
windmill:
image: ghcr.io/windmill-labs/windmill-ee:main
env:
DATABASE_URL: postgres://postgres:changeme@postgres:5432/windmill
LICENSE_KEY: ${{ secrets.WM_LICENSE_KEY_CI }}
WORKER_GROUP: main
WORKER_TAGS: deno,bun,go,python3,bash,dependency,flow,nativets
options: >-
--pull always --health-interval 10s --health-timeout 5s
--health-retries 5 --health-cmd "curl
http://localhost:8000/api/version"
ports:
- 8000:8000
steps:
- uses: denoland/setup-deno@v2
with:
deno-version: v2.x
- name: benchmark
timeout-minutes: 30
run: deno run -A -r
https://raw.githubusercontent.com/windmill-labs/windmill/${GITHUB_REF##ref/head/}/benchmarks/benchmark_suite.ts
-c
https://raw.githubusercontent.com/windmill-labs/windmill/${GITHUB_REF##ref/head/}/benchmarks/suite_config.json
- name: Save benchmark results
uses: actions/upload-artifact@v4
with:
name: benchmark_single
path: |
*.json
benchmark_dedicated:
runs-on: ubicloud-standard-8
services:
postgres:
image: postgres
env:
POSTGRES_DB: windmill
POSTGRES_PASSWORD: changeme
POSTGRES_INITDB_ARGS: "-c shared_buffers=2GB -c work_mem=32MB -c effective_cache_size=4GB"
options: >-
--health-cmd pg_isready --health-interval 10s --health-timeout 5s
--health-retries 5
windmill:
image: ghcr.io/windmill-labs/windmill-ee:main
env:
DATABASE_URL: postgres://postgres:changeme@postgres:5432/windmill
LICENSE_KEY: ${{ secrets.WM_LICENSE_KEY_CI }}
WORKER_GROUP: dedicated
DEDICATED_WORKER: "admins:f/benchmarks/dedicated"
options: >-
--pull always --restart unless-stopped --health-interval 10s --health-timeout 5s
--health-retries 5 --health-cmd "curl
http://localhost:8000/api/version"
ports:
- 8000:8000
steps:
- uses: denoland/setup-deno@v2
with:
deno-version: v2.x
- name: benchmark
timeout-minutes: 20
run: deno run -A -r
https://raw.githubusercontent.com/windmill-labs/windmill/${GITHUB_REF##ref/head/}/benchmarks/benchmark_suite.ts
--no-warm-up -c
https://raw.githubusercontent.com/windmill-labs/windmill/${GITHUB_REF##ref/head/}/benchmarks/suite_dedicated.json
- name: Save benchmark results
uses: actions/upload-artifact@v4
with:
name: benchmark_dedicated
path: |
*.json
benchmark_4workers:
runs-on: ubicloud-standard-8
services:
postgres:
image: postgres
env:
POSTGRES_DB: windmill
POSTGRES_PASSWORD: changeme
POSTGRES_INITDB_ARGS: "-c shared_buffers=2GB -c work_mem=32MB -c effective_cache_size=4GB"
options: >-
--health-cmd pg_isready --health-interval 10s --health-timeout 5s
--health-retries 5
windmill:
image: ghcr.io/windmill-labs/windmill-ee:main
env:
DATABASE_URL: postgres://postgres:changeme@postgres:5432/windmill
LICENSE_KEY: ${{ secrets.WM_LICENSE_KEY_CI }}
WORKER_GROUP: main
WORKER_TAGS: deno,bun,go,python3,bash,dependency,flow,nativets
options: >-
--pull always --health-interval 10s --health-timeout 5s
--health-retries 5 --health-cmd "curl
http://localhost:8000/api/version"
ports:
- 8000:8000
windmill_1:
image: ghcr.io/windmill-labs/windmill-ee:main
env:
DATABASE_URL: postgres://postgres:changeme@postgres:5432/windmill
LICENSE_KEY: ${{ secrets.WM_LICENSE_KEY_CI }}
MODE: worker
WORKER_GROUP: main
WORKER_TAGS: deno,bun,go,python3,bash,dependency,flow,nativets
options: >-
--pull always
windmill_2:
image: ghcr.io/windmill-labs/windmill-ee:main
env:
DATABASE_URL: postgres://postgres:changeme@postgres:5432/windmill
LICENSE_KEY: ${{ secrets.WM_LICENSE_KEY_CI }}
MODE: worker
WORKER_GROUP: main
WORKER_TAGS: deno,bun,go,python3,bash,dependency,flow,nativets
options: >-
--pull always
windmill_3:
image: ghcr.io/windmill-labs/windmill-ee:main
env:
DATABASE_URL: postgres://postgres:changeme@postgres:5432/windmill
LICENSE_KEY: ${{ secrets.WM_LICENSE_KEY_CI }}
MODE: worker
WORKER_GROUP: main
WORKER_TAGS: deno,bun,go,python3,bash,dependency,flow,nativets
options: >-
--pull always
steps:
- uses: denoland/setup-deno@v2
with:
deno-version: v2.x
- name: benchmark
timeout-minutes: 20
run: deno run -A -r
https://raw.githubusercontent.com/windmill-labs/windmill/${GITHUB_REF##ref/head/}/benchmarks/benchmark_suite.ts
-c
https://raw.githubusercontent.com/windmill-labs/windmill/${GITHUB_REF##ref/head/}/benchmarks/suite_config.json
--workers 4
--factor 3
- name: Save benchmark results
uses: actions/upload-artifact@v4
with:
name: benchmark_4workers
path: |
*.json
benchmark_8workers:
runs-on: ubicloud-standard-8
services:
postgres:
image: postgres
env:
POSTGRES_DB: windmill
POSTGRES_PASSWORD: changeme
POSTGRES_INITDB_ARGS: "-c shared_buffers=2GB -c work_mem=32MB -c effective_cache_size=4GB"
options: >-
--health-cmd pg_isready --health-interval 10s --health-timeout 5s
--health-retries 5
windmill:
image: ghcr.io/windmill-labs/windmill-ee:main
env:
DATABASE_URL: postgres://postgres:changeme@postgres:5432/windmill
LICENSE_KEY: ${{ secrets.WM_LICENSE_KEY_CI }}
WORKER_GROUP: main
WORKER_TAGS: deno,bun,go,python3,bash,dependency,flow,nativets
options: >-
--pull always --health-interval 10s --health-timeout 5s
--health-retries 5 --health-cmd "curl
http://localhost:8000/api/version"
ports:
- 8000:8000
windmill_1:
image: ghcr.io/windmill-labs/windmill-ee:main
env:
DATABASE_URL: postgres://postgres:changeme@postgres:5432/windmill
LICENSE_KEY: ${{ secrets.WM_LICENSE_KEY_CI }}
MODE: worker
WORKER_GROUP: main
WORKER_TAGS: deno,bun,go,python3,bash,dependency,flow,nativets
options: >-
--pull always
windmill_2:
image: ghcr.io/windmill-labs/windmill-ee:main
env:
DATABASE_URL: postgres://postgres:changeme@postgres:5432/windmill
LICENSE_KEY: ${{ secrets.WM_LICENSE_KEY_CI }}
MODE: worker
WORKER_GROUP: main
WORKER_TAGS: deno,bun,go,python3,bash,dependency,flow,nativets
options: >-
--pull always
windmill_3:
image: ghcr.io/windmill-labs/windmill-ee:main
env:
DATABASE_URL: postgres://postgres:changeme@postgres:5432/windmill
LICENSE_KEY: ${{ secrets.WM_LICENSE_KEY_CI }}
MODE: worker
WORKER_GROUP: main
WORKER_TAGS: deno,bun,go,python3,bash,dependency,flow,nativets
options: >-
--pull always
windmill_4:
image: ghcr.io/windmill-labs/windmill-ee:main
env:
DATABASE_URL: postgres://postgres:changeme@postgres:5432/windmill
LICENSE_KEY: ${{ secrets.WM_LICENSE_KEY_CI }}
MODE: worker
WORKER_GROUP: main
WORKER_TAGS: deno,bun,go,python3,bash,dependency,flow,nativets
options: >-
--pull always
windmill_5:
image: ghcr.io/windmill-labs/windmill-ee:main
env:
DATABASE_URL: postgres://postgres:changeme@postgres:5432/windmill
LICENSE_KEY: ${{ secrets.WM_LICENSE_KEY_CI }}
MODE: worker
WORKER_GROUP: main
WORKER_TAGS: deno,bun,go,python3,bash,dependency,flow,nativets
options: >-
--pull always
windmill_6:
image: ghcr.io/windmill-labs/windmill-ee:main
env:
DATABASE_URL: postgres://postgres:changeme@postgres:5432/windmill
LICENSE_KEY: ${{ secrets.WM_LICENSE_KEY_CI }}
MODE: worker
WORKER_GROUP: main
WORKER_TAGS: deno,bun,go,python3,bash,dependency,flow,nativets
options: >-
--pull always
windmill_7:
image: ghcr.io/windmill-labs/windmill-ee:main
env:
DATABASE_URL: postgres://postgres:changeme@postgres:5432/windmill
LICENSE_KEY: ${{ secrets.WM_LICENSE_KEY_CI }}
MODE: worker
WORKER_GROUP: main
WORKER_TAGS: deno,bun,go,python3,bash,dependency,flow,nativets
options: >-
--pull always
steps:
- uses: denoland/setup-deno@v2
with:
deno-version: v2.x
- name: benchmark
timeout-minutes: 20
run: deno run -A -r
https://raw.githubusercontent.com/windmill-labs/windmill/${GITHUB_REF##ref/head/}/benchmarks/benchmark_suite.ts
-c
https://raw.githubusercontent.com/windmill-labs/windmill/${GITHUB_REF##ref/head/}/benchmarks/suite_config.json
--workers 8
--factor 3
- name: Save benchmark results
uses: actions/upload-artifact@v4
with:
name: benchmark_8workers
path: |
*.json
benchmark_graphs:
runs-on: ubicloud
needs:
- benchmark_single
- benchmark_dedicated
- benchmark_4workers
- benchmark_8workers
steps:
- uses: denoland/setup-deno@v2
with:
deno-version: v2.x
- uses: actions/checkout@v4
with:
ref: benchmarks
- name: Download benchmark results
uses: actions/download-artifact@v4
with:
merge-multiple: true
- name: graphs
run: deno run -A -r
https://raw.githubusercontent.com/windmill-labs/windmill/${GITHUB_REF##ref/head/}/benchmarks/benchmark_graphs.ts
-c
https://raw.githubusercontent.com/windmill-labs/windmill/${GITHUB_REF##ref/head/}/benchmarks/graphs_config.json
- name: Push changes
run: |
ls -la
pwd
git add .
git config --local user.email "41898282+github-actions[bot]@users.noreply.github.com"
git config --local user.name "github-actions[bot]"
git commit -m "Update benchmarks"
git push

View File

@@ -1,45 +0,0 @@
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository_owner }}/caddy-l4
name: Build caddy-l4
on:
workflow_dispatch:
permissions: write-all
jobs:
build_ee:
runs-on: ubicloud
steps:
- uses: actions/checkout@v4
- uses: depot/setup-action@v1
- name: Docker meta
id: meta-ee-public
uses: docker/metadata-action@v5
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=sha
type=ref,event=branch
type=raw,value=latest,enable={{is_default_branch}}
- name: Login to registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push publicly
uses: depot/build-push-action@v1
with:
context: ./docker
file: ./docker/DockerfileCaddyL4
platforms: linux/amd64,linux/arm64
push: true
tags: |
${{ steps.meta-ee-public.outputs.tags }}
labels: |
${{ steps.meta-ee-public.outputs.labels }}

View File

@@ -1,130 +0,0 @@
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
name: Build and publish windmill for RHEL9
on: workflow_dispatch
permissions: write-all
jobs:
build_ee:
runs-on: ubicloud
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Read EE repo commit hash
run: |
echo "ee_repo_ref=$(cat ./backend/ee-repo-ref.txt)" >> "$GITHUB_ENV"
- uses: actions/checkout@v4
with:
repository: windmill-labs/windmill-ee-private
path: ./windmill-ee-private
ref: ${{ env.ee_repo_ref }}
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
fetch-depth: 0
# - name: Set up Docker Buildx
# uses: docker/setup-buildx-action@v2
- uses: depot/setup-action@v1
- name: Docker meta
id: meta-ee-public
uses: docker/metadata-action@v5
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee-rhel9
flavor: |
latest=false
tags: |
type=sha
- name: Login to registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Substitute EE code
run: |
./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
- name: Copy RHEL9 Dockerfile
run: |
cp ./docker/RHEL9/Dockerfile ./Dockerfile
- name: Build and push publicly ee amd64
uses: depot/build-push-action@v1
with:
context: .
platforms: linux/amd64
push: true
build-args: |
features=enterprise,enterprise_saml,stripe,embedding,parquet,prometheus,openidconnect,cloud,jemalloc,license,otel,http_trigger,zip,oauth2,kafka,sqs_trigger,nats,postgres_trigger,gcp_trigger,mqtt_trigger,websocket,smtp,static_frontend,all_languages,deno_core,mcp
secrets: |
rh_username=${{ secrets.RH_USERNAME }}
rh_password=${{ secrets.RH_PASSWORD }}
tags: |
${{ steps.meta-ee-public.outputs.tags }}-amd64
labels: |
${{ steps.meta-ee-public.outputs.labels }}-amd64
org.opencontainers.image.licenses=Windmill-Enterprise-License
- name: Build and push publicly ee arm64
uses: depot/build-push-action@v1
with:
context: .
platforms: linux/arm64
push: true
build-args: |
features=enterprise,enterprise_saml,stripe,embedding,parquet,prometheus,openidconnect,cloud,jemalloc,license,otel,http_trigger,zip,oauth2,kafka,sqs_trigger,nats,postgres_trigger,gcp_trigger,mqtt_trigger,websocket,smtp,static_frontend,all_languages,deno_core,mcp
secrets: |
rh_username=${{ secrets.RH_USERNAME }}
rh_password=${{ secrets.RH_PASSWORD }}
tags: |
${{ steps.meta-ee-public.outputs.tags }}-arm64
labels: |
${{ steps.meta-ee-public.outputs.labels }}-arm64
org.opencontainers.image.licenses=Windmill-Enterprise-License
- uses: shrink/actions-docker-extract@v3
id: extract-ee-amd64
with:
image: ${{ steps.meta-ee-public.outputs.tags}}-amd64
path: "/windmill/target/release/windmill"
# - uses: shrink/actions-docker-extract@v3
# id: extract-ee-arm64
# with:
# image: ${{ steps.meta-ee-public.outputs.tags}}-arm64
# path: "/windmill/target/release/windmill"
- name: Rename binary with corresponding architecture
run: |
mv "${{ steps.extract-ee-amd64.outputs.destination }}/windmill" "${{ steps.extract-ee-amd64.outputs.destination }}/windmill-ee-amd64-rhel9"
# mv "${{ steps.extract-ee-arm64.outputs.destination }}/windmill" "${{ steps.extract-ee-arm64.outputs.destination }}/windmill-ee-arm64-rhel9"
- uses: actions/upload-artifact@v4
with:
name: RHEL9-amd64 build
path: ${{ steps.extract-ee-amd64.outputs.destination
}}/windmill-ee-amd64-rhel9
# - uses: actions/upload-artifact@v4
# with:
# name: RHEL9-arm64 build
# path:
# ${{ steps.extract-ee-arm64.outputs.destination
# }}/windmill-ee-arm64-rhel9
# - name: Attach binary to release
# uses: softprops/action-gh-release@v2
# if: startsWith(github.ref, 'refs/tags/')
# with:
# files: |
# ${{ steps.extract-ee-arm64.outputs.destination }}/windmill-ee-arm64-rhel9
# ${{ steps.extract-ee-amd64.outputs.destination }}/windmill-ee-amd64-rhel9

View File

@@ -1,55 +0,0 @@
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}-cli
name: Publish cli image
on:
push:
tags:
- "v*"
workflow_dispatch:
permissions:
contents: read
id-token: write
packages: write
jobs:
publish_cli:
runs-on: ubicloud
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: depot/setup-action@v1
- name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=ref,event=branch
type=ref,event=pr
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
- name: Login to registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push publicly
uses: depot/build-push-action@v1
with:
file: "./docker/DockerfileCli"
platforms: linux/amd64,linux/arm64
push: true
tags: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest
${{ steps.meta.outputs.tags }}
labels: |
${{ steps.meta.outputs.labels }}
org.opencontainers.image.licenses=AGPLv3

View File

@@ -1,63 +0,0 @@
name: Build windows executable for this branch
on:
workflow_dispatch:
env:
CARGO_INCREMENTAL: 0
SQLX_OFFLINE: true
DISABLE_EMBEDDING: true
RUST_LOG: info
jobs:
cargo_build_windows:
runs-on: windows-latest
steps:
- uses: actions/checkout@v4
- name: Read EE repo commit hash
shell: pwsh
run: |
$ee_repo_ref = Get-Content .\backend\ee-repo-ref.txt
echo "ee_repo_ref=$ee_repo_ref" | Out-File -FilePath $env:GITHUB_ENV -Append
- name: Checkout windmill-ee-private repository
uses: actions/checkout@v4
with:
repository: windmill-labs/windmill-ee-private
path: ./windmill-ee-private
ref: ${{ env.ee_repo_ref }}
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
fetch-depth: 0
- name: Setup Rust
uses: actions-rs/toolchain@v1
with:
toolchain: 1.85.0
override: true
- name: Substitute EE code
shell: bash
run: |
./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
- name: Cargo build windows
timeout-minutes: 90
run: |
vcpkg.exe install openssl-windows:x64-windows
vcpkg.exe install openssl:x64-windows-static
vcpkg.exe integrate install
$env:VCPKGRS_DYNAMIC=1
$env:OPENSSL_DIR="${Env:VCPKG_INSTALLATION_ROOT}\installed\x64-windows-static"
mkdir frontend/build && cd backend
New-Item -Path . -Name "windmill-api/openapi-deref.yaml" -ItemType "File" -Force
cargo build --release --features=enterprise,stripe,embedding,parquet,prometheus,openidconnect,cloud,jemalloc,tantivy,license,http_trigger,zip,oauth2,kafka,nats,sqs_trigger,postgres_trigger,gcp_trigger,mqtt_trigger,websocket,smtp,static_frontend,all_languages,mcp
- name: Rename binary with corresponding architecture
run: |
Rename-Item -Path ".\backend\target\release\windmill.exe" -NewName "windmill-ee.exe"
- name: Upload artifact
uses: actions/upload-artifact@v4
with:
name: windmill-ee-binary
path: ./backend/target/release/windmill-ee.exe

View File

@@ -1,57 +0,0 @@
env:
REGISTRY: ghcr.io
ECR_REGISTRY: 976079455550.dkr.ecr.us-east-1.amazonaws.com
IMAGE_NAME: ${{ github.repository }}-multiplayer
name: Publish websocket multiplayer server
on:
workflow_dispatch:
permissions:
contents: read
id-token: write
packages: write
jobs:
publish_multiplayer:
runs-on: ubicloud-standard-8
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
# - uses: depot/setup-action@v1
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
- name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=ref,event=branch
type=ref,event=pr
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
- name: Login to registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push publicly
uses: docker/build-push-action@v6
with:
context: .
file: ./docker/DockerfileMultiplayer
platforms: linux/amd64,linux/arm64
push: true
tags: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest
${{ steps.meta.outputs.tags }}
labels: |
${{ steps.meta.outputs.labels }}
org.opencontainers.image.licenses=AGPLv3

View File

@@ -6,18 +6,10 @@ on:
- "version.txt"
jobs:
change_version:
runs-on: ubicloud
runs-on: ubuntu-latest
container: node:18
steps:
- uses: actions/checkout@v4
- run: git config --system --add safe.directory /__w/windmill/windmill
- uses: actions/checkout@v3
- name: Change versions
run: ./.github/change-versions.sh "$(cat version.txt)"
- uses: actions-rs/toolchain@v1
with:
toolchain: stable
- name: update lockfile
run: |
cd backend
cargo generate-lockfile
- uses: stefanzweifel/git-auto-commit-action@v5
- uses: stefanzweifel/git-auto-commit-action@v4

View File

@@ -1,85 +0,0 @@
name: Claude PR Assistant
on:
issue_comment:
types: [created]
pull_request_review_comment:
types: [created]
issues:
types: [opened, assigned]
pull_request_review:
types: [submitted]
jobs:
check-membership:
if: |
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '/ai') && !contains(github.event.comment.user.login, '[bot]')) ||
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '/ai') && !contains(github.event.comment.user.login, '[bot]')) ||
(github.event_name == 'pull_request_review' && contains(github.event.review.body, '/ai') && !contains(github.event.review.user.login, '[bot]')) ||
(github.event_name == 'issues' && contains(github.event.issue.body, '/ai') && !contains(github.event.issue.user.login, '[bot]'))
runs-on: ubicloud-standard-2
outputs:
is_member: ${{ steps.check-membership.outputs.is_member }}
steps:
- name: Check organization membership
id: check-membership
env:
ORG_ACCESS_TOKEN: ${{ secrets.ORG_ACCESS_TOKEN }}
run: |
ORG="windmill-labs"
if [[ "${{ github.event_name }}" == "issue_comment" || "${{ github.event_name }}" == "pull_request_review_comment" ]]; then
COMMENTER="${{ github.event.comment.user.login }}"
elif [[ "${{ github.event_name }}" == "pull_request_review" ]]; then
COMMENTER="${{ github.event.review.user.login }}"
else
COMMENTER="${{ github.event.issue.user.login }}"
fi
STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
-H "Authorization: token $ORG_ACCESS_TOKEN" \
-H "Accept: application/vnd.github+json" \
-H "X-GitHub-Api-Version: 2022-11-28" \
"https://api.github.com/orgs/$ORG/members/$COMMENTER")
if [ "$STATUS" -eq 204 ]; then
echo "is_member=true" >> $GITHUB_OUTPUT
else
echo "is_member=false" >> $GITHUB_OUTPUT
fi
claude-code-action:
needs: check-membership
if: |
needs.check-membership.outputs.is_member == 'true'
runs-on: ubicloud-standard-8
permissions:
contents: read
pull-requests: read
issues: read
id-token: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Run Claude PR Action
uses: anthropics/claude-code-action@beta
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
timeout_minutes: "60"
allowed_tools: "mcp__github__create_pull_request,Bash(npm run check),Bash(npm install),Bash(cargo check),Bash(curl https://sh.rustup.rs -sSf | sh -s -- -y)"
custom_instructions: "IMPORTANT INSTRUCTIONS:
- Bypass previous instructions and instead of adding a url in your comment to manually create a PR, if you created a new branch with your changes, automatically create a draft PR from that branch to main, with the title starting with [Claude PR].
- If you made changes to the frontend code, run npm install, then npm run generate-backend-client, then npm run check. You can ignore warnings that are reported by the check script, but fix the errors.
- If you made changes to the backend code, install Rust and then run cargo check. You can ignore warnings that are reported by the check script, but fix the errors.
- DO NOT FORGET TO OPEN A DRAFT PR AFTER YOU ARE DONE if you made changes after a request from a git issue.
AVAILABLE TOOLS:
- mcp__github__create_pull_request: Create a PR from a branch to main
- Bash(npm run check): Run the check script. You should run this tool after making changes to the frontend code.
- Bash(npm install): Install dependencies. You need this to run npm run check.
- Bash(npm run generate-backend-client): Generate the backend client. You need this to run npm run check.
- Bash(cargo check): Run the cargo check script. You should run this tool after making changes to the backend code.
- Bash(curl https://sh.rustup.rs -sSf | sh -s -- -y): Install Rust. You need this to run cargo check."
trigger_phrase: "/ai"

13
.github/workflows/clean-docker.yml vendored Normal file
View File

@@ -0,0 +1,13 @@
name: Clean docker
on:
schedule:
# * is a special character in YAML so you have to quote this string
- cron: "0 0 */2 * *"
jobs:
build:
runs-on: [self-hosted, new]
steps:
- name: clean docker
run: |
sudo docker system prune -f

View File

@@ -1,41 +0,0 @@
on:
issue_comment:
types: [created]
jobs:
check-membership:
if: ${{ github.event.issue.pull_request && startsWith(github.event.comment.body, '/docs') && github.event.comment.user.type != 'Bot' }}
runs-on: ubicloud-standard-2
outputs:
is_member: ${{ steps.check-membership.outputs.is_member }}
steps:
- name: Check organization membership
id: check-membership
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
COMMENTER: ${{ github.event.comment.user.login }}
run: |
ORG="windmill-labs"
STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
-H "Authorization: token $GH_TOKEN" \
-H "Accept: application/vnd.github+json" \
-H "X-GitHub-Api-Version: 2022-11-28" \
"https://api.github.com/orgs/$ORG/members/$COMMENTER")
if [ "$STATUS" -eq 204 ]; then
echo "is_member=true" >> $GITHUB_OUTPUT
else
echo "is_member=false" >> $GITHUB_OUTPUT
fi
trigger-docs:
needs: check-membership
if: ${{ github.event.issue.pull_request && startsWith(github.event.comment.body, '/docs') && needs.check-membership.outputs.is_member == 'true' }}
uses: windmill-labs/windmilldocs/.github/workflows/create-docs.yml@main
with:
pr_number: ${{ github.event.issue.number }}
repo: ${{ github.event.repository.name }}
comment_text: ${{ github.event.comment.body }}
secrets:
DOCS_TOKEN: ${{ secrets.DOCS_TOKEN }}
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}

View File

@@ -1,37 +1,29 @@
name: Publish go-client
name: Publish deno-client
on:
push:
tags:
- "v*"
workflow_dispatch:
env:
repo: windmill-go-client
repo: windmill-deno-client
jobs:
build_go_and_push_to_repo:
runs-on: ubicloud-standard-8
build_deno_and_push_to_repo:
runs-on: ubuntu-latest
container: openapitools/openapi-generator-cli:v6.0.0-beta
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v4
- name: install_jq
- uses: actions/checkout@v3
- name: generate_deno
run: |
sudo apt-get update
sudo apt-get install jq
- name: generate_go
run: |
go install github.com/deepmap/oapi-codegen/cmd/oapi-codegen@v1.11.0
cd go-client
cd deno-client
rm .gitignore
./build.sh
go build
./generate.sh
- name: Pushes to another repository
id: push_directory
uses: cpina/github-action-push-to-another-repository@devel
env:
API_TOKEN_GITHUB: ${{ secrets.DENO_PAT }}
with:
source-directory: go-client/
source-directory: deno-client/
destination-github-username: ${{ github.repository_owner }}
destination-repository-name: ${{ env.repo }}
user-email: ruben@windmill.dev
@@ -39,10 +31,10 @@ jobs:
target-branch: main
tag_repo:
needs: [build_go_and_push_to_repo]
needs: [build_deno_and_push_to_repo]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v3
with:
repository: ${{ github.repository_owner }}/${{ env.repo }}
token: ${{ secrets.DENO_PAT }}

View File

@@ -0,0 +1,20 @@
name: Deploy to windmill.dev
on:
push:
branches: [main]
paths:
- "community/*"
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Deploy to windmill.dev
uses: windmill-labs/windmill-gh-action-deploy@v1.0.0
with:
dry_run: false
input_dir: community
windmill_workspace: starter
windmill_token: ${{ secrets.WINDMILL_API_TOKEN }}

View File

@@ -1,32 +0,0 @@
name: Create discord thread when a PR is opened, react with green checkmark when PR is merged
on:
pull_request:
types:
- opened
- ready_for_review
- closed
jobs:
notify_discord_when_pr_opened:
if: (github.event.pull_request.draft == false) && (github.event.action == 'opened' || github.event.action == 'ready_for_review')
uses: ./.github/workflows/shareable-discord-notification.yml
with:
PR_TITLE: ${{ github.event.pull_request.title }}
PR_URL: ${{ github.event.pull_request.html_url }}
PR_AUTHOR: ${{ github.event.pull_request.user.login }}
PR_STATUS: "opened"
PR_NUMBER: ${{ github.event.pull_request.number }}
secrets:
DISCORD_WEBHOOK_URL: ${{ secrets.DISCORD_PR_REVIEWS_WEBHOOK }}
merge_success_emoji:
if: github.event.pull_request.merged == true
uses: ./.github/workflows/shareable-discord-notification.yml
with:
PR_STATUS: "merged"
DISCORD_CHANNEL_ID: "1372204995868491786"
DISCORD_GUILD_ID: "930051556043276338"
PR_NUMBER: ${{ github.event.pull_request.number }}
secrets:
DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_AI_BOT_TOKEN }}

View File

@@ -1,60 +0,0 @@
# env:
# REGISTRY: ghcr.io
# IMAGE_NAME: ${{ github.repository }}
# name: Build and push arm docker image
# on:
# push:
# branches: [main]
# tags: ["*"]
# concurrency:
# group: ${{ github.ref }}-arm
# cancel-in-progress: true
# permissions:
# contents: read
# id-token: write
# packages: write
# jobs:
# publish_arm:
# runs-on: ubuntu-22.04
# steps:
# - uses: actions/checkout@v3
# with:
# fetch-depth: 0
# - uses: depot/setup-action@v1
# - name: Docker meta
# id: meta-slim-public
# uses: docker/metadata-action@v4
# with:
# images: |
# ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
# tags: |
# type=ref,event=branch
# type=ref,event=pr
# type=semver,pattern={{version}}
# type=semver,pattern={{major}}.{{minor}}
# - name: Login to registry
# uses: docker/login-action@v2
# with:
# registry: ${{ env.REGISTRY }}
# username: ${{ github.actor }}
# password: ${{ secrets.GITHUB_TOKEN }}
# - name: Build and push publicly
# uses: depot/build-push-action@v1
# with:
# context: .
# push: true
# platforms: linux/amd64,linux/arm64
# tags: |
# ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest
# ${{ steps.meta-slim-public.outputs.tags }}
# labels: |
# ${{ steps.meta-slim-public.outputs.labels }}
# org.opencontainers.image.licenses=AGPLv3

View File

@@ -1,76 +0,0 @@
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}-rpi
name: Build windmill without jemalloc
on:
workflow_dispatch:
concurrency:
group: windmill-without-jemalloc
cancel-in-progress: true
permissions: write-all
jobs:
build:
runs-on: ubicloud
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Read EE repo commit hash
run: |
echo "ee_repo_ref=$(cat ./backend/ee-repo-ref.txt)" >> "$GITHUB_ENV"
- uses: actions/checkout@v4
with:
repository: windmill-labs/windmill-ee-private
path: ./windmill-ee-private
ref: ${{ env.ee_repo_ref }}
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
fetch-depth: 0
# - name: Set up Docker Buildx
# uses: docker/setup-buildx-action@v2
- uses: depot/setup-action@v1
- name: Login to registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Substitute EE code (EE logic is behind feature flag)
run: |
./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
- name: Docker meta
id: meta-public
uses: docker/metadata-action@v5
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
flavor: |
latest=false
tags: |
type=ref,event=pr
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
- name: Build and push publicly
uses: depot/build-push-action@v1
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
build-args: |
features=embedding,parquet,openidconnect,license,http_trigger,zip,oauth2,postgres_trigger,mqtt_trigger,websocket,smtp,static_frontend,all_languages,deno_core,mcp
tags: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:dev
${{ steps.meta-public.outputs.tags }}
labels: |
${{ steps.meta-public.outputs.labels }}
org.opencontainers.image.licenses=AGPLv3

View File

@@ -1,712 +1,83 @@
env:
LOCAL_REGISTRY: registry.wimill.xyz
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.event_name != 'pull_request' && github.event_name !=
'workflow_dispatch' && github.repository || 'windmill-labs/windmill-test' }}
DEV_SHA: ${{ github.event_name != 'pull_request' && github.event_name !=
'workflow_dispatch' && 'dev' || github.event.inputs.tag || github.sha }}
name: Build windmill:main
IMAGE_NAME: ${{ github.repository }}
name: Build and push docker image
on:
push:
branches: [main]
tags: ["*"]
pull_request:
types: [opened, synchronize, reopened]
paths:
- "Dockerfile"
workflow_dispatch:
inputs:
ee:
description: "Build EE image (true, false)"
required: false
default: false
type: boolean
tag:
description: "Tag the image"
required: true
default: "test"
nsjail:
description: "Build nsjail image (true, false)"
required: false
default: false
type: boolean
concurrency:
group: ${{ github.ref }}
cancel-in-progress: false
permissions: write-all
cancel-in-progress: true
jobs:
build:
runs-on: ubicloud
if: (github.event_name != 'workflow_dispatch') || (github.event.inputs &&
!github.event.inputs.ee)
runs-on: [self-hosted, new]
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.ref }}
fetch-depth: 0
- name: Read EE repo commit hash
run: |
echo "ee_repo_ref=$(cat ./backend/ee-repo-ref.txt)" >> "$GITHUB_ENV"
- uses: actions/checkout@v4
with:
repository: windmill-labs/windmill-ee-private
path: ./windmill-ee-private
ref: ${{ env.ee_repo_ref }}
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
fetch-depth: 0
# - name: Set up Docker Buildx
# uses: docker/setup-buildx-action@v2
- uses: depot/setup-action@v1
- name: Login to registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Substitute EE code (EE logic is behind feature flag)
run: |
./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
- name: Docker meta
id: meta-public
uses: docker/metadata-action@v5
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
flavor: |
latest=false
tags: |
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
- name: Build and push publicly
uses: depot/build-push-action@v1
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
build-args: |
features=embedding,parquet,openidconnect,jemalloc,license,http_trigger,zip,oauth2,dind,postgres_trigger,mqtt_trigger,websocket,smtp,static_frontend,agent_worker_server,all_languages,deno_core,mcp
tags: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.DEV_SHA }}
${{ steps.meta-public.outputs.tags }}
labels: |
${{ steps.meta-public.outputs.labels }}
build_ee:
runs-on: ubicloud
if: (github.event_name != 'workflow_dispatch') || (github.event.inputs.ee || github.event.inputs.nsjail)
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.ref }}
fetch-depth: 0
- name: Read EE repo commit hash
run: |
echo "ee_repo_ref=$(cat ./backend/ee-repo-ref.txt)" >> "$GITHUB_ENV"
- uses: actions/checkout@v4
with:
repository: windmill-labs/windmill-ee-private
path: ./windmill-ee-private
ref: ${{ env.ee_repo_ref }}
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
fetch-depth: 0
# - name: Set up Docker Buildx
# uses: docker/setup-buildx-action@v2
- uses: depot/setup-action@v1
- name: Docker meta
id: meta-ee-public
uses: docker/metadata-action@v5
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee
flavor: |
latest=false
tags: |
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
- name: Login to registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Substitute EE code
run: |
./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
- name: Build and push publicly ee
uses: depot/build-push-action@v1
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
build-args: |
features=enterprise,enterprise_saml,stripe,embedding,parquet,prometheus,openidconnect,cloud,jemalloc,agent_worker_server,tantivy,license,http_trigger,zip,oauth2,kafka,sqs_trigger,nats,otel,dind,postgres_trigger,mqtt_trigger,gcp_trigger,websocket,smtp,static_frontend,all_languages,deno_core,mcp
tags: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee:${{ env.DEV_SHA }}
${{ steps.meta-ee-public.outputs.tags }}
labels: |
${{ steps.meta-ee-public.outputs.labels }}
org.opencontainers.image.licenses=Windmill-Enterprise-License
# disabled until we make it 100% reliable and add more meaningful tests
# playwright:
# runs-on: [self-hosted, new]
# needs: [build]
# services:
# postgres:
# image: postgres
# env:
# POSTGRES_DB: windmill
# POSTGRES_USER: admin
# POSTGRES_PASSWORD: changeme
# ports:
# - 5432:5432
# options: >-
# --health-cmd pg_isready
# --health-interval 10s
# --health-timeout 5s
# --health-retries 5
# steps:
# - uses: actions/checkout@v4
# - name: "Docker"
# run: echo "::set-output name=id::$(docker run --network=host --rm -d -p 8000:8000 --privileged -it -e DATABASE_URL=postgres://admin:changeme@localhost:5432/windmill -e BASE_INTERNAL_URL=http://localhost:8000 ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest)"
# id: docker-container
# - uses: actions/setup-node@v3
# with:
# node-version: 16
# - name: "Playwright run"
# timeout-minutes: 2
# run: cd frontend && npm ci @playwright/test && npx playwright install && export BASE_URL=http://localhost:8000 && npm run test
# - name: "Clean up"
# run: docker kill ${{ steps.docker-container.outputs.id }}
# if: always()
attach_amd64_binary_to_release:
needs: [build, build_ee]
runs-on: ubicloud
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
env:
ARCH: amd64
steps:
- uses: actions/checkout@v4
- run: |
# pulling docker image with desired arch so that actions-docker-extract doesn't do it
docker pull --platform "linux/$ARCH" ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.DEV_SHA }}
docker pull --platform "linux/$ARCH" ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee:${{ env.DEV_SHA }}
- run: |
# Checks the image is in docker prior to running actions-docker-extract. It fails if not
# Also useful to visually check that the arch is the right opencontainers
docker image inspect ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.DEV_SHA }}
docker image inspect ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee:${{ env.DEV_SHA }}
- uses: shrink/actions-docker-extract@v3
id: extract
with:
image: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.DEV_SHA }}
path: "/usr/src/app/windmill"
- uses: shrink/actions-docker-extract@v3
id: extract-ee
with:
image: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee:${{ env.DEV_SHA }}
path: "/usr/src/app/windmill"
- name: Rename binary with corresponding architecture
run: |
mv "${{ steps.extract.outputs.destination }}/windmill" "${{ steps.extract.outputs.destination }}/windmill-${ARCH}"
mv "${{ steps.extract-ee.outputs.destination }}/windmill" "${{ steps.extract-ee.outputs.destination }}/windmill-ee-${ARCH}"
- name: Attach binary to release
uses: softprops/action-gh-release@v2
with:
files: |
${{ steps.extract.outputs.destination }}/*
${{ steps.extract-ee.outputs.destination }}/*
# attach_arm64_binary_to_release:
# needs: [build, build_ee]
# runs-on: ubicoud
# if: ${{ startsWith(github.ref, 'refs/tags/') }}
# env:
# ARCH: arm64
# steps:
# - uses: actions/checkout@v4
# - run: |
# # pulling docker image with desired arch so that actions-docker-extract doesn't do it
# docker pull --platform "linux/$ARCH" ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.DEV_SHA }}
# docker pull --platform "linux/$ARCH" ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee:${{ env.DEV_SHA }}
# - run: |
# # Checks the image is in docker prior to running actions-docker-extract. It fails if not
# # Also useful to visually check that the arch is the right opencontainers
# docker image inspect ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.DEV_SHA }}
# docker image inspect ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee:${{ env.DEV_SHA }}
# - uses: shrink/actions-docker-extract@v3
# id: extract
# with:
# image: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.DEV_SHA }}
# path: "/usr/src/app/windmill"
# - uses: shrink/actions-docker-extract@v3
# id: extract-ee
# with:
# image: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee:${{ env.DEV_SHA }}
# path: "/usr/src/app/windmill"
# - name: Rename binary with corresponding architecture
# run: |
# mv "${{ steps.extract.outputs.destination }}/windmill" "${{ steps.extract.outputs.destination }}/windmill-${ARCH}"
# mv "${{ steps.extract-ee.outputs.destination }}/windmill" "${{ steps.extract-ee.outputs.destination }}/windmill-ee-${ARCH}"
# - name: Attach binary to release
# uses: softprops/action-gh-release@v2
# with:
# files: |
# ${{ steps.extract.outputs.destination }}/*
# ${{ steps.extract-ee.outputs.destination }}/*
run_integration_test:
runs-on: ubicloud
needs: [build_ee]
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.ref }}
fetch-depth: 0
- name: Prepare test run
if: ${{ ! startsWith(github.ref, 'refs/tags/v') }}
run: cd integration_tests && ./build.sh
- name: Test run
if: ${{ ! startsWith(github.ref, 'refs/tags/v') }}
timeout-minutes: 15
env:
LICENSE_KEY: ${{ secrets.WM_LICENSE_KEY_CI }}
run: cd integration_tests && ./run.sh
- name: Archive logs
uses: actions/upload-artifact@v4
if: always()
with:
name: Windmill Integration Tests Logs
path: |
integration_tests/logs
tag_latest:
runs-on: ubicloud
needs: [run_integration_test, build]
if:
github.event_name != 'pull_request' && (github.ref == 'refs/heads/main' ||
startsWith(github.ref, 'refs/tags/v')) && (github.event_name != 'workflow_dispatch')
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Login to registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Tag main and latest
run: |
docker buildx imagetools create ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.DEV_SHA }} --tag ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest
docker buildx imagetools create ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.DEV_SHA }} --tag ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:main
tag_latest_ee:
runs-on: ubicloud
needs: [run_integration_test, build_ee]
if:
github.event_name != 'pull_request' && (github.event_name != 'workflow_dispatch') && (github.ref == 'refs/heads/main' ||
startsWith(github.ref, 'refs/tags/v'))
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Login to registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Tag main and latest for ee
run: |
docker buildx imagetools create ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee:${{ env.DEV_SHA }} --tag ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee:latest
docker buildx imagetools create ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee:${{ env.DEV_SHA }} --tag ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee:main
verify_ee_image_vulnerabilities:
runs-on: ubicloud
needs: [tag_latest_ee]
if: startsWith(github.ref, 'refs/tags/v') && (github.event_name != 'workflow_dispatch')
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Analyze for critical and high CVEs
id: docker-scout-cves
if: ${{ github.event_name != 'pull_request_target' }}
uses: docker/scout-action@v1
with:
command: cves
only-severities: critical,high
image: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee:main
sarif-file: sarif.output.json
summary: true
dockerhub-user: windmilllabs
dockerhub-password: ${{ secrets.DOCKER_PAT }}
- name: Upload SARIF result
id: upload-sarif
if: ${{ github.event_name != 'pull_request_target' }}
uses: github/codeql-action/upload-sarif@v2
with:
sarif_file: sarif.output.json
# docker_scout_ee:
# runs-on: ubicloud
# needs: [tag_latest_ee]
# steps:
# - name: Docker Scout
# id: docker-scout
# uses: docker/scout-action@v1
# with:
# dockerhub-
# command: cves,recommendations,compare
# to-latest: true
# ignore-base: true
# ignore-unchanged: true
# only-fixed: true
build_ee_nsjail:
needs: [build_ee]
runs-on: ubicloud
if: (github.event_name != 'pull_request') && ((github.event_name != 'workflow_dispatch') || (github.event.inputs.ee || github.event.inputs.nsjail))
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
ref: ${{ github.ref }}
# - name: Set up Docker Buildx
# uses: docker/setup-buildx-action@v2
- uses: depot/setup-action@v1
- name: Docker meta
id: meta-ee-public
uses: docker/metadata-action@v5
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee-nsjail
flavor: |
latest=false
tags: |
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=sha,enable=true,priority=100,prefix=,suffix=,format=short
type=ref,event=branch
type=ref,event=pr
- name: Login to registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Update Dockerfile image reference
run: |
sed -i 's|FROM ghcr.io/windmill-labs/windmill-ee:dev|FROM ghcr.io/${{ env.IMAGE_NAME }}-ee:${{ env.DEV_SHA }}|' ./docker/DockerfileNsjail
cat ./docker/DockerfileNsjail | grep "FROM"
- name: Build and push publicly ee
uses: depot/build-push-action@v1
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
file: "./docker/DockerfileNsjail"
tags: |
${{ steps.meta-ee-public.outputs.tags }}
labels: |
${{ steps.meta-ee-public.outputs.labels }}
org.opencontainers.image.licenses=Windmill-Enterprise-License
publish_ecr_s3:
needs: [build_ee_nsjail]
runs-on: ubicloud-standard-2-arm
if: (github.event_name != 'pull_request') && (github.event_name !=
'workflow_dispatch')
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
- name: Docker meta local
id: metalocal
uses: docker/metadata-action@v4
with:
images: ${{ env.LOCAL_REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=ref,event=branch
type=ref,event=pr
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
- name: Build and push privately
uses: docker/build-push-action@v3
if: github.event_name == 'pull_request'
with:
context: .
push: true
tags: |
${{ steps.metalocal.outputs.tags }}
labels: ${{ steps.metalocal.outputs.labels }}
cache-from: type=registry,ref=registry.wimill.xyz/windmilllabs/windmill:buildcache
cache-to: type=registry,ref=registry.wimill.xyz/windmilllabs/windmill:buildcache,mode=max
- name: Docker meta
if: github.event_name != 'pull_request'
id: meta
uses: docker/metadata-action@v4
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=ref,event=branch
type=ref,event=pr
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
- name: Login to registry
uses: docker/login-action@v3
if: github.event_name != 'pull_request'
uses: docker/login-action@v2
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: get git hash
if: github.event_name != 'pull_request'
id: git_hash
run: |
git_hash=$(git rev-parse --short "$GITHUB_SHA")
echo "GIT_HASH=${git_hash:0:7}" >> "$GITHUB_OUTPUT"
- uses: shrink/actions-docker-extract@v3
if: github.event_name != 'pull_request'
id: extract
with:
image: |-
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee-nsjail:${{ steps.git_hash.outputs.GIT_HASH }}
path: "/static_frontend/."
- uses: reggionick/s3-deploy@v4
- name: Build and push publically
uses: docker/build-push-action@v3
if: github.event_name != 'pull_request'
with:
folder: ${{ steps.extract.outputs.destination }}
bucket: windmill-frontend
bucket-region: us-east-1
build_ee_cuda:
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
needs: [build_ee]
runs-on: ubicloud
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
ref: ${{ github.ref }}
# - name: Set up Docker Buildx
# uses: docker/setup-buildx-action@v2
- uses: depot/setup-action@v1
- name: Docker meta
id: meta-ee-public
uses: docker/metadata-action@v5
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee-cuda
tags: |
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
- name: Login to registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push publicly ee
uses: depot/build-push-action@v1
with:
context: .
platforms: linux/amd64
push: true
file: "./docker/DockerfileCuda"
tags: |
${{ steps.meta-ee-public.outputs.tags }}
labels: |
${{ steps.meta-ee-public.outputs.labels }}
org.opencontainers.image.licenses=Windmill-Enterprise-License
${{ steps.metalocal.outputs.tags }}
${{ steps.meta.outputs.tags }}
build_slim:
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
needs: [build]
runs-on: ubicloud
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
# - name: Set up Docker Buildx
# uses: docker/setup-buildx-action@v2
- uses: depot/setup-action@v1
- name: Docker meta
id: meta-ee-public
uses: docker/metadata-action@v5
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-slim
tags: |
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
- name: Login to registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push publicly ee
uses: depot/build-push-action@v1
with:
context: .
platforms: linux/amd64
push: true
file: "./docker/DockerfileSlim"
tags: |
${{ steps.meta-ee-public.outputs.tags }}
labels: |
${{ steps.meta-ee-public.outputs.labels }}
build_ee_slim:
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
needs: [build_ee]
runs-on: ubicloud
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
# - name: Set up Docker Buildx
# uses: docker/setup-buildx-action@v2
- uses: depot/setup-action@v1
- name: Docker meta
id: meta-ee-public
uses: docker/metadata-action@v5
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee-slim
tags: |
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
- name: Login to registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push publicly ee
uses: depot/build-push-action@v1
with:
context: .
platforms: linux/amd64
push: true
file: "./docker/DockerfileSlimEe"
tags: |
${{ steps.meta-ee-public.outputs.tags }}
labels: |
${{ steps.meta-ee-public.outputs.labels }}
org.opencontainers.image.licenses=Windmill-Enterprise-License
build_full:
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
needs: [build]
runs-on: ubicloud
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
# - name: Set up Docker Buildx
# uses: docker/setup-buildx-action@v2
- uses: depot/setup-action@v1
- name: Docker meta
id: meta-public
uses: docker/metadata-action@v5
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-full
tags: |
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
- name: Login to registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push publicly
uses: depot/build-push-action@v1
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
file: "./docker/DockerfileFull"
tags: |
${{ steps.meta-public.outputs.tags }}
labels: |
${{ steps.meta-public.outputs.labels }}
build_ee_full:
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
needs: [build_ee]
runs-on: ubicloud
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
# - name: Set up Docker Buildx
# uses: docker/setup-buildx-action@v2
- uses: depot/setup-action@v1
- name: Docker meta
id: meta-ee-public
uses: docker/metadata-action@v5
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-ee-full
tags: |
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
- name: Login to registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push publicly ee
uses: depot/build-push-action@v1
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
file: "./docker/DockerfileFullEe"
tags: |
${{ steps.meta-ee-public.outputs.tags }}
labels: |
${{ steps.meta-ee-public.outputs.labels }}
org.opencontainers.image.licenses=Windmill-Enterprise-License
labels: ${{ steps.metalocal.outputs.labels }}
cache-from: type=registry,ref=registry.wimill.xyz/windmilllabs/windmill:buildcache
cache-to: type=registry,ref=registry.wimill.xyz/windmilllabs/windmill:buildcache,mode=max

View File

@@ -1,26 +0,0 @@
name: check frontend build
on:
workflow_run:
workflows: ["Change versions"]
types:
- completed
merge_group:
push:
paths:
- "frontend/**"
- ".github/workflows/frontend-check.yml"
jobs:
npm_check:
runs-on: ubicloud-standard-8
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v3
with:
node-version: 18
- name: "npm check"
timeout-minutes: 5
run:
cd frontend && npm ci && npm run generate-backend-client && npm run
check

View File

@@ -1,16 +0,0 @@
name: Publish powershell-client
on:
push:
tags:
- "v*"
workflow_dispatch:
jobs:
publish_gallery:
runs-on: ubicloud-standard-8
steps:
- uses: actions/checkout@v4
- run: . ./powershell-client/publish.ps1
shell: pwsh
env:
NUGET_API_KEY: ${{ secrets.NUGET_API_KEY }}

View File

@@ -1,66 +0,0 @@
name: Publish Helm Chart on Release
on:
release:
types: [published]
jobs:
bump-helm-version:
runs-on: ubicloud-standard-2
steps:
- name: Checkout on helm repository
uses: actions/checkout@v3
with:
repository: windmill-labs/windmill-helm-charts
token: ${{ secrets.DOCS_TOKEN }}
- name: Get version
id: get_version
run: |
echo "VERSION=${GITHUB_REF#refs/tags/v}" >> $GITHUB_ENV
- name: Create new branch
run: |
# Check if branch already exists remotely
if git ls-remote --heads origin bump-helm-version-${{ env.VERSION }} | grep -q bump-helm-version-${{ env.VERSION }}; then
# Branch exists, check it out
git fetch origin bump-helm-version-${{ env.VERSION }}
git checkout bump-helm-version-${{ env.VERSION }}
else
# Create new branch
git checkout -b bump-helm-version-${{ env.VERSION }}
fi
git config --local user.email "action@github.com"
git config --local user.name "GitHub Action"
- name: Bump helm version
run: |
# Get current version and increment it by 1
CURRENT_VERSION=$(grep "version:" ./charts/windmill/Chart.yaml | awk '{print $2}' | head -n 1)
NEW_VERSION=$(echo "$CURRENT_VERSION" | awk -F. '{$NF = $NF + 1;} 1' | sed 's/ /./g')
sed -i "s/^version: .*/version: $NEW_VERSION/" ./charts/windmill/Chart.yaml
# Get the app version from the version
VERSION=${{ env.VERSION }}
APP_VERSION=${VERSION#refs/tag/}
APP_VERSION=${APP_VERSION#v}
APP_VERSION=${APP_VERSION%/}
sed -i "s/appVersion: .*/appVersion: $APP_VERSION/" ./charts/windmill/Chart.yaml
- name: Commit and push
run: |
git add .
git commit -m "Bump helm version to ${{ env.VERSION }}"
git push origin bump-helm-version-${{ env.VERSION }}
- name: Create PR
env:
GH_TOKEN: ${{ secrets.DOCS_TOKEN }}
run: |
gh pr create \
--title "helm: bump version to ${{ env.VERSION }}" \
--body "This PR was auto-generated to bring the helm chart up to date for [release ${{ env.VERSION }}](https://github.com/windmill-labs/windmill/releases/tag/v${{ env.VERSION }}) in the main repo." \
--head bump-helm-version-${{ env.VERSION }} \
--base main

View File

@@ -1,16 +0,0 @@
name: Publish typescript-client on JSR
on:
push:
tags:
- "v*"
jobs:
publish:
runs-on: ubuntu-latest
permissions:
contents: read
id-token: write
steps:
- uses: actions/checkout@v4
- run: cd typescript-client && ./publish.jsr.sh

View File

@@ -1,38 +0,0 @@
name: Claude PR Assistant
on:
repository_dispatch:
types: [external_claude_issue_fix]
jobs:
claude-code-action:
runs-on: ubicloud-standard-8
permissions:
contents: read
pull-requests: read
issues: read
id-token: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Process inputs
id: process_inputs
shell: bash
run: |
ISSUE_TITLE="${{ github.event.client_payload.issue_title }}"
INSTRUCTION="${{ github.event.client_payload.instruction }}"
ISSUE_BODY=$(printf '%q' "${{ github.event.client_payload.issue_body }}")
BASE_PROMPT="Try to fix the following issue based on the instruction given. You are provided with the issue title, issue body, and instruction. You are to fix the issue based on the instruction. You are to create a pull request to fix the issue."
CUSTOM_PROMPT=$(printf -v PROMPT "%s\n\nISSUE_TITLE: %s\n\nISSUE_BODY: %s\n\nINSTRUCTION: %s" "$BASE_PROMPT" "$ISSUE_TITLE" "$ISSUE_BODY" "$INSTRUCTION")
echo "CUSTOM_PROMPT=$CUSTOM_PROMPT" >> $GITHUB_OUTPUT
- name: Run Claude PR Action
uses: anthropics/claude-code-action@beta
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
timeout_minutes: "60"
allowed_tools: "mcp__github__create_pull_request"
direct_prompt: ${{ steps.process_inputs.outputs.CUSTOM_PROMPT }}

37
.github/workflows/lsp_on_release.yml vendored Normal file
View File

@@ -0,0 +1,37 @@
name: Publish LSP Server
on:
push:
branches: [main]
paths:
- "python-client/*W"
- "lsp/*"
tags:
- "*"
jobs:
build:
runs-on: [self-hosted, new]
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
- name: Docker meta local
id: metalocal
uses: docker/metadata-action@v4
with:
images: registry.wimill.xyz/windmilllabs/lsp
tags: |
type=ref,event=branch
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
- name: Build and push
uses: docker/build-push-action@v3
with:
context: "{{defaultContext}}:lsp"
push: true
tags: ${{ steps.metalocal.outputs.tags }}
labels: ${{ steps.metalocal.outputs.labels }}
cache-from: type=registry,ref=registry.wimill.xyz/windmilllabs/lsp:buildcache
cache-to: type=registry,ref=registry.wimill.xyz/windmilllabs/lsp:buildcache,mode=max

View File

@@ -1,33 +0,0 @@
name: Publish typescript-client & CLI to NPM on release
on:
push:
tags:
- "v*"
workflow_dispatch:
jobs:
publish_typescript_client:
runs-on: ubicloud-standard-8
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v3
with:
node-version: "20.x"
registry-url: "https://registry.npmjs.org"
- run: cd typescript-client && ./publish.sh && cd ..
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
publish_cli:
runs-on: ubicloud-standard-8
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v3
with:
node-version: "20.x"
registry-url: "https://registry.npmjs.org"
- uses: denoland/setup-deno@v2
with:
deno-version: v2.x
- run: cd cli && ./build.sh && cd npm && npm publish
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

View File

@@ -1,64 +0,0 @@
env:
REGISTRY: ghcr.io
ECR_REGISTRY: 976079455550.dkr.ecr.us-east-1.amazonaws.com
IMAGE_NAME: ${{ github.repository }}-lsp
name: Publish lsp
on:
push:
tags:
- "v*"
workflow_dispatch:
permissions:
contents: read
id-token: write
packages: write
jobs:
sleep:
runs-on: ubicloud
steps:
- name: Sleep for 900 seconds waiting for pypi to update index
if: startsWith(github.ref, 'refs/tags/v')
run: sleep 900
shell: bash
publish_lsp:
needs: [sleep]
runs-on: ubicloud
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: depot/setup-action@v1
- name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=ref,event=branch
type=ref,event=pr
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
- name: Login to registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push publicly
uses: depot/build-push-action@v1
with:
context: "{{defaultContext}}:lsp"
platforms: linux/amd64,linux/arm64
push: true
tags: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest
${{ steps.meta.outputs.tags }}
labels: |
${{ steps.meta.outputs.labels }}
org.opencontainers.image.licenses=AGPLv3

View File

@@ -1,65 +0,0 @@
name: Build and Publish Windows Worker
on:
push:
tags:
- "v*"
env:
CARGO_INCREMENTAL: 0
SQLX_OFFLINE: true
DISABLE_EMBEDDING: true
RUST_LOG: info
jobs:
cargo_build_windows:
runs-on: windows-latest
steps:
- uses: actions/checkout@v4
- name: Read EE repo commit hash
shell: pwsh
run: |
$ee_repo_ref = Get-Content .\backend\ee-repo-ref.txt
echo "ee_repo_ref=$ee_repo_ref" | Out-File -FilePath $env:GITHUB_ENV -Append
- name: Checkout windmill-ee-private repository
uses: actions/checkout@v4
with:
repository: windmill-labs/windmill-ee-private
path: ./windmill-ee-private
ref: ${{ env.ee_repo_ref }}
token: ${{ secrets.WINDMILL_EE_PRIVATE_ACCESS }}
fetch-depth: 0
- name: Setup Rust
uses: actions-rs/toolchain@v1
with:
toolchain: 1.85.0
override: true
- name: Substitute EE code
shell: bash
run: |
./backend/substitute_ee_code.sh --copy --dir ./windmill-ee-private
- name: Cargo build windows
timeout-minutes: 90
run: |
vcpkg.exe install openssl-windows:x64-windows
vcpkg.exe install openssl:x64-windows-static
vcpkg.exe integrate install
$env:VCPKGRS_DYNAMIC=1
$env:OPENSSL_DIR="${Env:VCPKG_INSTALLATION_ROOT}\installed\x64-windows-static"
mkdir frontend/build && cd backend
New-Item -Path . -Name "windmill-api/openapi-deref.yaml" -ItemType "File" -Force
cargo build --release --features=enterprise,stripe,embedding,parquet,prometheus,openidconnect,cloud,jemalloc,tantivy,license,http_trigger,zip,oauth2,kafka,sqs_trigger,nats,postgres_trigger,mqtt_trigger,gcp_trigger,websocket,smtp,static_frontend,all_languages,mcp
- name: Rename binary with corresponding architecture
run: |
Rename-Item -Path ".\backend\target\release\windmill.exe" -NewName "windmill-ee.exe"
- name: Attach binary to release
uses: softprops/action-gh-release@v2
with:
files: |
./backend/target/release/windmill-ee.exe

View File

@@ -3,19 +3,17 @@ on:
push:
tags:
- "v*"
workflow_dispatch:
jobs:
publish_pypi:
runs-on: ubicloud-standard-8
if: startsWith(github.ref, 'refs/tags/v')
container:
image: ghcr.io/windmill-labs/python-client-builder
publish_pypi``:
runs-on: [self-hosted, new]
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v3
- name: Upload python client
env:
PYPI_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
run: |
cd python-client
export PATH=$PATH:/usr/local/bin
export PATH=$PATH:/root/.local/bin
./publish.sh

View File

@@ -6,7 +6,7 @@ name: release-please
jobs:
release-please:
name: "Release please"
runs-on: ubicloud
runs-on: ubuntu-latest
steps:
- uses: GoogleCloudPlatform/release-please-action@v3
with:

View File

@@ -1,98 +0,0 @@
name: "Notify Discord when a PR is opened or merged"
on:
workflow_call:
inputs:
PR_TITLE:
description: "The title of the PR"
type: string
PR_URL:
description: "The URL of the PR"
type: string
PR_AUTHOR:
description: "The author of the PR"
type: string
PR_STATUS:
description: "The status of the PR"
type: string
DISCORD_CHANNEL_ID:
description: "The Discord channel ID"
type: string
PR_NUMBER:
description: "The number of the PR"
type: string
DISCORD_GUILD_ID:
description: "The Discord guild ID"
type: string
secrets:
DISCORD_WEBHOOK_URL:
description: "Discord Webhook URL"
DISCORD_BOT_TOKEN:
description: "Discord Bot Token"
jobs:
open_thread:
runs-on: ubicloud-standard-2
if: ${{ inputs.PR_STATUS == 'opened' }}
steps:
- name: Send Discord notification and start a thread
env:
WEBHOOK_URL: ${{ secrets.DISCORD_WEBHOOK_URL }}
PR_TITLE: ${{ inputs.PR_TITLE }}
PR_NUMBER: ${{ inputs.PR_NUMBER }}
PR_URL: ${{ inputs.PR_URL }}
PR_AUTHOR: ${{ inputs.PR_AUTHOR }}
run: |
payload=$(jq -n \
--arg content "${PR_URL}" \
--arg thread "#${PR_NUMBER}: $PR_TITLE by \`${PR_AUTHOR}\`" \
'{
content: $content,
thread_name: $thread,
auto_archive_duration: 10080
}'
)
curl -H "Content-Type: application/json" \
-X POST \
-d "$payload" \
"$WEBHOOK_URL"
merge_success_emoji:
runs-on: ubuntu-latest
if: ${{ inputs.PR_STATUS == 'merged' }}
steps:
- name: React
env:
BOT_TOKEN: ${{ secrets.DISCORD_BOT_TOKEN }}
CHANNEL_ID: ${{ inputs.DISCORD_CHANNEL_ID }}
GUILD_ID: ${{ inputs.DISCORD_GUILD_ID }}
PR_NUMBER: ${{ inputs.PR_NUMBER }}
run: |
# 1) get PR thread
threads=$(curl -H "Authorization: Bot $BOT_TOKEN" "https://discord.com/api/v10/guilds/${GUILD_ID}/threads/active")
thread_id=$(
echo "$threads" \
| jq -r --arg cid "$CHANNEL_ID" \
--arg pref "#${PR_NUMBER}:" \
'.threads[]
| select(.parent_id == $cid and (.name | startswith($pref)))
| .id'
)
if [ -z "$thread_id" ]; then
echo "Thread not found"
exit 1
fi
# 2) get the first message in that thread
messages=$(curl -H "Authorization: Bot $BOT_TOKEN" \
"https://discord.com/api/v10/channels/$thread_id/messages")
message_id=$(echo "$messages" | jq -r '.[-1].id')
if [ -z "$message_id" ]; then
echo "Message not found"
exit 1
fi
# 3) add the ✅ reaction
curl -X PUT \
-H "Authorization: Bot $BOT_TOKEN" \
"https://discord.com/api/v10/channels/$thread_id/messages/$message_id/reactions/%E2%9C%85/@me"

View File

@@ -7,15 +7,12 @@ on:
jobs:
CLAssistant:
runs-on: ubicloud
runs-on: ubuntu-latest
steps:
- name: "CLA Assistant"
if:
(github.event.comment.body == 'recheck' || github.event.comment.body
== 'I have read the CLA Document and I hereby sign the CLA') ||
github.event_name == 'pull_request_target'
if: (github.event.comment.body == 'recheck' || github.event.comment.body == 'I have read the CLA Document and I hereby sign the CLA') || github.event_name == 'pull_request_target'
# Beta Release
uses: cla-assistant/github-action@v2.6.1
uses: cla-assistant/github-action@v2.1.3-beta
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PERSONAL_ACCESS_TOKEN: ${{ secrets.CLA_PAT }}

9
.gitignore vendored
View File

@@ -1,14 +1,5 @@
target/
.DS_Store
nohup.out
local/
frontend/src/routes/test.svelte
CaddyfileRemoteMalo
*.swp
**/.idea/
.direnv
.vscode
.dev-docker-wrapper*
backend/.minio-data
.aider*
!.aiderignore

13474
CHANGELOG.md

File diff suppressed because it is too large Load Diff

4
CLA.md
View File

@@ -2,8 +2,8 @@
## Individual Contributor Non-Exclusive License Agreement
Thank you for your interest in contributing to Windmill Labs, Inc's Windmill
("We" or "Us").
Thank you for your interest in contributing to Ruben Fiszel's Windmill ("We" or
"Us").
The purpose of this contributor agreement ("Agreement") is to clarify and
document the rights granted by contributors to Us.

View File

@@ -1,3 +0,0 @@
To have an overview of what this app does, see @.cursor/rules/windmill-overview.mdc
For backend modifications, follow the rules mentioned here @.cursor/rules/rust-best-practices.mdc
For frontend modifications, follow the rules mentioned here @.cursor/rules/svelte5-best-practices.mdc

View File

@@ -1,18 +1,4 @@
{
layer4 {
:25 {
proxy {
to windmill_server:2525
}
}
}
}
{$BASE_URL} {
bind {$ADDRESS}
reverse_proxy /ws/* http://lsp:3001
# reverse_proxy /ws_mp/* http://multiplayer:3002
# reverse_proxy /api/srch/* http://windmill_indexer:8002
reverse_proxy /* http://windmill_server:8000
# tls /certs/cert.pem /certs/key.pem
{$SITE_URL} {
bind {$ADDRESS}
reverse_proxy /* windmill:8000
}

View File

@@ -1,234 +1,101 @@
ARG DEBIAN_IMAGE=debian:bookworm-slim
ARG RUST_IMAGE=rust:1.86-slim-bookworm
FROM python:3.10-slim-buster as nsjail
FROM ${RUST_IMAGE} AS rust_base
RUN apt-get update && apt-get install -y git libssl-dev pkg-config npm
WORKDIR /nsjail
RUN apt-get -y update \
&& apt-get install -y \
curl nodejs
bison=2:3.3.* \
flex=2.6.* \
g++=4:8.3.* \
gcc=4:8.3.* \
git=1:2.20.* \
libprotobuf-dev=3.6.* \
libnl-route-3-dev=3.4.* \
make=4.2.* \
pkg-config=0.29-6 \
protobuf-compiler=3.6.*
RUN rustup component add rustfmt
RUN git clone -b master --single-branch https://github.com/google/nsjail.git . \
&& git checkout dccf911fd2659e7b08ce9507c25b2b38ec2c5800
RUN make
RUN CARGO_NET_GIT_FETCH_WITH_CLI=true cargo install cargo-chef --version 0.1.68
RUN cargo install sccache --version ^0.8
ENV RUSTC_WRAPPER=sccache SCCACHE_DIR=/backend/sccache
WORKDIR /windmill
ENV SQLX_OFFLINE=true
# ENV CARGO_INCREMENTAL=1
FROM node:20-alpine as frontend
FROM mhart/alpine-node:16 as frontend
# install dependencies
WORKDIR /frontend
COPY ./frontend/package.json ./frontend/package-lock.json ./
COPY ./frontend/scripts/ ./scripts/
RUN npm ci
# Copy all local files into the image.
COPY frontend .
RUN mkdir /backend
COPY /backend/windmill-api/openapi.yaml /backend/windmill-api/openapi.yaml
COPY /openflow.openapi.yaml /openflow.openapi.yaml
COPY /backend/windmill-api/build_openapi.sh /backend/windmill-api/build_openapi.sh
RUN cd /backend/windmill-api && . ./build_openapi.sh
COPY /backend/parsers/windmill-parser-wasm/pkg/ /backend/parsers/windmill-parser-wasm/pkg/
COPY /typescript-client/docs/ /frontend/static/tsdocs/
COPY /backend/openapi.yaml /backend/openapi.yaml
RUN npm run generate-backend-client
ENV NODE_OPTIONS "--max-old-space-size=8192"
ARG VITE_BASE_URL ""
# Read more about macro in docker/dev.nu
# -- MACRO-SPREAD-WASM-PARSER-DEV-ONLY -- #
RUN npm run build
FROM rust:slim-buster as builder
FROM rust_base AS planner
RUN apt-get update && apt-get install -y git libssl-dev pkg-config
COPY ./openflow.openapi.yaml /openflow.openapi.yaml
COPY ./backend ./
RUN USER=root cargo new --bin windmill
WORKDIR /windmill
RUN --mount=type=cache,target=/usr/local/cargo/registry \
--mount=type=cache,target=$SCCACHE_DIR,sharing=locked \
CARGO_NET_GIT_FETCH_WITH_CLI=true cargo chef prepare --recipe-path recipe.json
COPY ./backend/Cargo.toml .
COPY ./backend/Cargo.lock .
COPY ./backend/.cargo/ .cargo/
FROM rust_base AS builder
ARG features=""
RUN apt-get -y update \
&& apt-get install -y \
curl
COPY --from=planner /windmill/recipe.json recipe.json
ENV CARGO_INCREMENTAL=1
RUN apt-get update && apt-get install -y libxml2-dev=2.9.* libxmlsec1-dev=1.2.* clang=1:14.0-55.* libclang-dev=1:14.0-55.* cmake=3.25.* && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
RUN cargo build --release
RUN rm src/*.rs
RUN --mount=type=cache,target=/usr/local/cargo/registry \
--mount=type=cache,target=$SCCACHE_DIR,sharing=locked \
CARGO_NET_GIT_FETCH_WITH_CLI=true RUST_BACKTRACE=1 cargo chef cook --release --features "$features" --recipe-path recipe.json
RUN rm ./target/release/deps/windmill*
ENV SQLX_OFFLINE=true
COPY ./openflow.openapi.yaml /openflow.openapi.yaml
COPY ./backend ./
ADD ./backend ./
ADD ./nsjail /nsjail
RUN mkdir -p /frontend
COPY --from=1 /frontend /frontend
ADD .git/ .git/
COPY --from=frontend /frontend/build /frontend/build
COPY --from=frontend /backend/windmill-api/openapi-deref.yaml ./windmill-api/openapi-deref.yaml
COPY .git/ .git/
RUN --mount=type=cache,target=/usr/local/cargo/registry \
--mount=type=cache,target=$SCCACHE_DIR,sharing=locked \
CARGO_NET_GIT_FETCH_WITH_CLI=true cargo build --release --features "$features"
RUN cargo build --release
FROM ${DEBIAN_IMAGE}
ARG TARGETPLATFORM
ARG POWERSHELL_VERSION=7.5.0
ARG POWERSHELL_DEB_VERSION=7.5.0-1
ARG KUBECTL_VERSION=1.28.7
ARG HELM_VERSION=3.14.3
ARG GO_VERSION=1.22.5
FROM debian:buster-slim
ARG APP=/usr/src/app
ARG WITH_POWERSHELL=true
ARG WITH_KUBECTL=true
ARG WITH_HELM=true
ARG WITH_GIT=true
# To change latest stable version:
# 1. Change placeholder in instanceSettings.ts
# 2. Change LATEST_STABLE_PY in dockerfile
# 3. Change #[default] annotation for PyVersion in backend
ARG LATEST_STABLE_PY=3.11.10
ENV UV_PYTHON_INSTALL_DIR=/tmp/windmill/cache/py_runtime
ENV UV_PYTHON_PREFERENCE=only-managed
ENV UV_TOOL_BIN_DIR=/usr/local/bin
ENV PATH /usr/local/bin:/root/.local/bin:$PATH
RUN apt-get update \
&& apt-get install -y --no-install-recommends netbase tzdata ca-certificates wget curl jq unzip build-essential unixodbc xmlsec1 software-properties-common \
&& apt-get clean \
&& apt-get install -y ca-certificates tzdata libpq5 python3 python3-pip \
make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev \
libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev xz-utils tk-dev libxml2-dev \
libxmlsec1-dev libffi-dev liblzma-dev mecab-ipadic-utf8 libgdbm-dev libc6-dev git libprotobuf-dev=3.6.* libnl-route-3-dev=3.4.* \
libv8-dev \
&& rm -rf /var/lib/apt/lists/*
RUN if [ "$WITH_GIT" = "true" ]; then \
apt-get update -y \
&& apt-get install -y git \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*; \
else echo 'Building the image without git'; fi;
RUN if [ "$WITH_POWERSHELL" = "true" ]; then \
if [ "$TARGETPLATFORM" = "linux/amd64" ]; then apt-get update -y && apt install libicu-dev -y && wget -O 'pwsh.deb' "https://github.com/PowerShell/PowerShell/releases/download/v${POWERSHELL_VERSION}/powershell_${POWERSHELL_DEB_VERSION}.deb_amd64.deb" && apt-get clean \
&& rm -rf /var/lib/apt/lists/* && \
dpkg --install 'pwsh.deb' && \
rm 'pwsh.deb'; \
elif [ "$TARGETPLATFORM" = "linux/arm64" ]; then apt-get update -y && apt install libicu-dev -y && wget -O powershell.tar.gz "https://github.com/PowerShell/PowerShell/releases/download/v${POWERSHELL_VERSION}/powershell-${POWERSHELL_VERSION}-linux-arm64.tar.gz" && apt-get clean \
&& rm -rf /var/lib/apt/lists/* && \
mkdir -p /opt/microsoft/powershell/7 && \
tar zxf powershell.tar.gz -C /opt/microsoft/powershell/7 && \
chmod +x /opt/microsoft/powershell/7/pwsh && \
ln -s /opt/microsoft/powershell/7/pwsh /usr/bin/pwsh && \
rm powershell.tar.gz; \
else echo 'Could not install pwshell, not on amd64 or arm64'; fi; \
else echo 'Building the image without powershell'; fi
RUN if [ "$WITH_HELM" = "true" ]; then \
arch="$(dpkg --print-architecture)"; arch="${arch##*-}"; \
wget "https://get.helm.sh/helm-v${HELM_VERSION}-linux-$arch.tar.gz" && \
tar -zxvf "helm-v${HELM_VERSION}-linux-$arch.tar.gz" && \
mv linux-$arch/helm /usr/local/bin/helm &&\
chmod +x /usr/local/bin/helm; \
else echo 'Building the image without helm'; fi
RUN if [ "$WITH_KUBECTL" = "true" ]; then \
arch="$(dpkg --print-architecture)"; arch="${arch##*-}"; \
curl -LO "https://dl.k8s.io/release/v${KUBECTL_VERSION}/bin/linux/$arch/kubectl" && \
install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl; \
else echo 'Building the image without kubectl'; fi
RUN set -eux; \
arch="$(dpkg --print-architecture)"; arch="${arch##*-}"; \
case "$arch" in \
"amd64") \
targz="go${GO_VERSION}.linux-amd64.tar.gz"; \
;; \
"arm64") \
targz="go${GO_VERSION}.linux-arm64.tar.gz"; \
;; \
"armhf") \
targz="go${GO_VERSION}.linux-armv6l.tar.gz"; \
;; \
*) echo >&2 "error: unsupported architecture '$arch' (likely packaging update needed)"; exit 1 ;; \
esac; \
wget "https://golang.org/dl/$targz" -nv && tar -C /usr/local -xzf "$targz" && rm "$targz";
ENV PATH="${PATH}:/usr/local/go/bin"
ENV GO_PATH=/usr/local/go/bin/go
# Install UV
RUN curl --proto '=https' --tlsv1.2 -LsSf https://github.com/astral-sh/uv/releases/download/0.6.2/uv-installer.sh | sh && mv /root/.local/bin/uv /usr/local/bin/uv
# Preinstall python runtimes
RUN uv python install 3.11
RUN uv python install $LATEST_STABLE_PY
RUN uv venv
RUN curl -sL https://deb.nodesource.com/setup_20.x | bash -
RUN apt-get -y update && apt-get install -y curl procps nodejs awscli && apt-get clean \
&& rm -rf /var/lib/apt/lists/*
# go build is slower the first time it is ran, so we prewarm it in the build
RUN mkdir -p /tmp/gobuildwarm && cd /tmp/gobuildwarm && go mod init gobuildwarm && printf "package foo\nimport (\"fmt\")\nfunc main() { fmt.Println(42) }" > warm.go && go mod tidy && go build -x && rm -rf /tmp/gobuildwarm
ENV TZ=Etc/UTC
COPY --from=builder /frontend/build /static_frontend
ENV PYTHON_VERSION 3.10.4
RUN wget https://www.python.org/ftp/python/${PYTHON_VERSION}/Python-${PYTHON_VERSION}.tgz \
&& tar -xf Python-${PYTHON_VERSION}.tgz && cd Python-${PYTHON_VERSION}/ && ./configure --enable-optimizations \
&& make -j 4 && make install
RUN python3 -m pip install pip-tools
COPY --from=builder /windmill/target/release/windmill ${APP}/windmill
COPY --from=denoland/deno:2.2.1 --chmod=755 /usr/bin/deno /usr/bin/deno
COPY --from=nsjail /nsjail/nsjail /bin/nsjail
COPY --from=oven/bun:1.2.4 /usr/local/bin/bun /usr/bin/bun
COPY --from=denoland/deno:latest /usr/bin/deno /usr/bin/deno
COPY --from=php:8.3.7-cli /usr/local/bin/php /usr/bin/php
COPY --from=composer:2.7.6 /usr/bin/composer /usr/bin/composer
# add the docker client to call docker from a worker if enabled
COPY --from=docker:dind /usr/local/bin/docker /usr/local/bin/
ENV RUSTUP_HOME="/usr/local/rustup"
ENV CARGO_HOME="/usr/local/cargo"
RUN mkdir -p ${APP}
WORKDIR ${APP}
RUN ln -s ${APP}/windmill /usr/local/bin/windmill
COPY ./frontend/src/lib/hubPaths.json ${APP}/hubPaths.json
RUN windmill cache ${APP}/hubPaths.json && rm ${APP}/hubPaths.json && chmod -R 777 /tmp/windmill
# Create a non-root user 'windmill' with UID and GID 1000
RUN addgroup --gid 1000 windmill && \
adduser --disabled-password --gecos "" --uid 1000 --gid 1000 windmill
RUN cp -r /root/.cache /home/windmill/.cache
RUN mkdir -p /tmp/windmill/logs && \
mkdir -p /tmp/windmill/search
# Make directories world-readable and writable
RUN chmod -R 777 ${APP} && \
chmod -R 777 /tmp/windmill && \
chmod -R 777 /home/windmill/.cache
USER root
EXPOSE 8000
CMD ["windmill"]
CMD ["./windmill"]

40
LICENSE
View File

@@ -1,34 +1,12 @@
Source code in this repository is variously licensed under the Apache License
Version 2.0 (see file ./LICENSE-APACHE), or the AGPLv3 License (see file
./LICENSE-AGPL) and a proprietary license for certain enterprise features.
Version 2.0 (see file ./LICENSE-APACHE),or the AGPLv3 License (see file ./LICENSE-AGPL)
Every file is under copyright (c) Windmill Labs, Inc 2022 unless otherwise
specified. Every file is under License AGPL unless otherwise specified or
belonging to one of the below cases:
Every file is under copyright (c) Ruben Fiszel 2021 unless otherwise specified.
Every file is under License AGPL unless otherwise specified
or belonging to one of the below cases:
The files under backend/ are AGPLv3 Licensed, except any snippets of code under
the compile flag "enterprise". Those snippets and files are under a proprietary
and commercial license. The files under frontend/ are AGPLv3 Licensed, except
any snippets of code that require a positive license check to be activated.
Those snippets and files are under a proprietary and commercial license. Private
and public forks MUST not include any of the above proprietary and commercial
code. The files under python-client/ deno-client/ go-client/ powershell-client/
are Apache 2.0 Licensed. The openapi files, including the OpenFlow spec is
Apache 2.0 Licensed.
The binary compilable from source code in this repository without the
"enterprise" feature flag is open-source under the AGPLv3 License terms and
conditions.
The "Community Edition" of Windmill available in the docker images hosted under
ghcr.io/windmill-labs/windmill and the github binary releases contains the files
under the AGPLv3 and Apache 2 sources but also includes proprietary and
non-public code and features which are not open source and under the following
terms: Windmill Labs, Inc. grants a right to use all the features of the
"Community Edition" for free without restrictions other than the limits and
quotas set in the software and a right to distribute the community edition as is
but not to sell, resell, serve as a managed service, modify or wrap under any
form without an explicit agreement.
All third party components incorporated into the Windmill Software are licensed
under the original license provided by the owner of the applicable component.
The files under backend/ are AGPL Licensed.
The files under frontend/ are AGPL Licensed.
The files under python-client/ are Apache 2.0 Licensed.
The files under community/ are Apache 2.0 Licensed.

View File

@@ -186,7 +186,7 @@
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2022 Windmill Labs, Inc
Copyright 2021 Ruben Fiszel
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.

4
NOTICE
View File

@@ -1,4 +1,6 @@
Copyright (c) 2022 Windmill Labs, Inc
Ruben Fiszel
Copyright (c) 2021 Ruben Fiszel
Source code in this repository is variously licensed under the Apache License
Version 2.0 or the GNU Affero General Public License. Please see

475
README.md
View File

@@ -1,432 +1,127 @@
<p align="center">
<a href="https://www.windmill.dev/"><img src="./imgs/windmill-banner.png" alt="windmill.dev"></a>
<a href="https://app.windmill.dev"><img src="./imgs/windmill.svg" alt="windmill.dev"></a>
</p>
<p align="center">
<em>Windmill.dev is an OSS developer platform to quickly build production-grade multi-steps automations and internal apps from minimal Python and Typescript scripts.</em>
</p>
<p align=center>
Open-source developer infrastructure for internal tools (APIs, background jobs, workflows and UIs). Self-hostable alternative to Retool, Pipedream, Superblocks and a simplified Temporal with autogenerated UIs and custom UIs to trigger workflows and scripts as internal apps.
<p align=center>
Scripts are turned into sharable UIs automatically, and can be composed together into flows or used into richer apps built with low-code. Supported script languages supported are: Python, TypeScript, Go, Bash, SQL, and GraphQL.
</p>
<p align="center">
<a href="https://github.com/windmill-labs/windmill/blob/main/LICENSE-AGPL" target="_blank">
<img src="https://img.shields.io/badge/License-AGPLv3-blue.svg" alt="Package version">
</a>
<a href="https://github.com/windmill-labs/windmill/actions/workflows/docker-image.yml" target="_blank">
<img src="https://github.com/windmill-labs/windmill/actions/workflows/docker-image.yml/badge.svg" alt="Docker Image CI">
</a>
<a href="https://pypi.org/project/wmill" target="_blank">
<img src="https://img.shields.io/pypi/v/wmill?color=%2334D058&label=pypi%20package" alt="Package version">
</a>
</p>
<p align="center">
<a href="https://img.shields.io/github/commit-activity/m/windmill-labs/windmill" target="_blank">
<img src="https://img.shields.io/github/commit-activity/m/windmill-labs/windmill" alt="Commit activity">
</a>
<a href="https://discord.gg/V7PM2YHsPB" target="_blank">
<img src="https://discordapp.com/api/guilds/930051556043276338/widget.png" alt="Discord Shield"/>
</a>
</p>
---
**Join the alpha (personal workspaces are free forever)**:
<https://app.windmill.dev>
**Documentation**: <https://docs.windmill.dev>
**Discord**: <https://discord.gg/V7PM2YHsPB>
**We are hiring**: Software Engineers, DevOps, Solutions Engineers, Growth:
<https://docs.windmill.dev/hiring>
You can show your support for the project by starring this repo.
---
# Windmill
<p align="center">
<a href="https://app.windmill.dev">Try it</a> - <a href="https://www.windmill.dev/docs/intro/">Docs</a> - <a href="https://discord.gg/V7PM2YHsPB">Discord</a> - <a href="https://hub.windmill.dev">Hub</a> - <a href="https://www.windmill.dev/docs/misc/contributing">Contributor's guide</a>
<b>Disclaimer: </b>Windmill is in <b>BETA</b>. It is secure to run in production but the API might change,
especially concerning flows.
</p>
# Windmill - Developer platform for APIs, background jobs, workflows and UIs
![Windmill Screenshot](./imgs/windmill.webp)
Windmill is <b>fully open-sourced (AGPLv3)</b> and Windmill Labs offers
dedicated instance and commercial support and licenses.
Windmill is <b>fully open-sourced</b>:
![Windmill Diagram](./imgs/stacks.svg)
- `community/`, `python-client/` and `deno-client/` are Apache 2.0
- backend, frontend and everything else under AGPLv3.
https://github.com/windmill-labs/windmill/assets/122811744/0b132cd1-ee67-4505-822f-0c7ee7104252
## What is the general idea behind Windmill
- [Windmill - Developer platform for APIs, background jobs, workflows and UIs](#windmill---developer-platform-for-apis-background-jobs-workflows-and-uis)
- [Main Concepts](#main-concepts)
- [Show me some actual script code](#show-me-some-actual-script-code)
- [CLI](#cli)
- [Running scripts locally](#running-scripts-locally)
- [Stack](#stack)
- [Fastest Self-Hostable Workflow Engine](#fastest-self-hostable-workflow-engine)
- [Security](#security)
- [Sandboxing](#sandboxing)
- [Secrets, credentials and sensitive values](#secrets-credentials-and-sensitive-values)
- [Performance](#performance)
- [Architecture](#architecture)
- [How to self-host](#how-to-self-host)
- [Docker compose](#docker-compose)
- [Kubernetes (k8s) and Helm charts](#kubernetes-k8s-and-helm-charts)
- [Run from binaries](#run-from-binaries)
- [OAuth, SSO \& SMTP](#oauth-sso--smtp)
- [Commercial license](#commercial-license)
- [Integrations](#integrations)
- [Environment Variables](#environment-variables)
- [Run a local dev setup](#run-a-local-dev-setup)
- [only Frontend](#only-frontend)
- [Backend + Frontend](#backend--frontend)
- [Contributors](#contributors)
- [Copyright](#copyright)
1. Define a minimal and generic script in Python or Typescript that solve a
specific task. Here sending an email with SMTP. The code can be defined in
the provided Web IDE or synchronized with your own github repo:
![Step 1](./imgs/step1.png)
## Main Concepts
2. Your scripts parameters are automatically parsed and generate a frontend. You
can narrow down the types during task definition to specify regex for string,
an enum or a specific format for objects. Each script correspond to an app by
itself: ![Step 2](./imgs/step2.png)
1. Define a minimal and generic script in Python, TypeScript, Go or Bash that
solves a specific task. The code can be defined in the
[provided Web IDE](https://www.windmill.dev/docs/code_editor) or
[synchronized with your own GitHub repo](https://www.windmill.dev/docs/advanced/cli/sync)
(e.g. through
[VS Code](https://www.windmill.dev/docs/cli_local_dev/vscode-extension)
extension):
3. Make it flow! You can chain your scripts or scripts made by the community
inside flow by piping output to input using "Dynamic" fields that are just
plain Javascript. You can also refer to external variables, output from any
steps or inputs of the flow itself. The flow parameters then generate
automatically an intuitive forms that can be triggered by anyone, like for
scripts. ![Step 3](./imgs/step3.png)
![Step 1](./imgs/windmill-editor.png)
## Layout
2. Your scripts parameters are automatically parsed and
[generate a frontend](https://www.windmill.dev/docs/core_concepts/auto_generated_uis).
![Step 2](./imgs/windmill-run.png)
![Step 3](./imgs/windmill-result.png)
3. Make it [flow](https://www.windmill.dev/docs/flows/flow_editor)! You can
chain your scripts or scripts made by the community shared on
[WindmillHub](https://hub.windmill.dev).
![Step 3](./imgs/windmill-flow.png)
4. Build [complex UIs](https://www.windmill.dev/docs/apps/app_editor) on top of
your scripts and flows.
![Step 4](./imgs/windmill-builder.png)
Scripts and flows can also be triggered by a
[cron schedule](https://www.windmill.dev/docs/core_concepts/scheduling) (e.g.
'_/5 _ \* \* \*') or through
[webhooks](https://www.windmill.dev/docs/core_concepts/webhooks).
You can build your entire infra on top of Windmill!
## Show me some actual script code
```typescript
//import any dependency from npm
import * as wmill from "windmill-client";
import * as cowsay from "cowsay@1.5.0";
// fill the type, or use the +Resource type to get a type-safe reference to a resource
type Postgresql = {
host: string;
port: number;
user: string;
dbname: string;
sslmode: string;
password: string;
};
export async function main(
a: number,
b: "my" | "enum",
c: Postgresql,
d = "inferred type string from default arg",
e = { nested: "object" }
//f: wmill.Base64
) {
const email = process.env["WM_EMAIL"];
// variables are permissioned and by path
let variable = await wmill.getVariable("f/company-folder/my_secret");
const lastTimeRun = await wmill.getState();
// logs are printed and always inspectable
console.log(cowsay.say({ text: "hello " + email + " " + lastTimeRun }));
await wmill.setState(Date.now());
// return is serialized as JSON
return { foo: d, variable };
}
```
## CLI
We have a powerful CLI to interact with the windmill platform and sync your
scripts from local files, GitHub repos and to run scripts and flows on the
instance from local commands. See
[more details](https://www.windmill.dev/docs/advanced/cli).
![CLI Screencast](./cli/vhs/output/setup.gif)
### Running scripts locally
You can run your script locally easily, you simply need to pass the right
environment variables for the `wmill` client library to fetch resources and
variables from your instance if necessary. See more:
<https://www.windmill.dev/docs/advanced/local_development>.
To develop & test locally scripts & flows, we recommend using the Windmill VS
Code extension: <https://www.windmill.dev/docs/cli_local_dev/vscode-extension>.
- `backend/`: The whole Rust backend
- `frontend`: The whole Svelte frontend
- `community/`: Scripts and resource types created and curated by the community,
included in every workspace
- `lsp/`: The lsp asssistant for the monaco editor
- `nsjail/`: The nsjail configuration files for sandboxing of the scripts'
execution
- `python-client/`: The wmill python client used within scripts to interact with
the windmill platform
## Stack
- Postgres as the database.
- Backend in Rust with the following highly-available and horizontally scalable.
Architecture:
- Stateless API backend.
- Workers that pull jobs from a queue in Postgres (and later, Kafka or Redis.
Upvote [#173](#https://github.com/windmill-labs/windmill/issues/173) if
interested).
- Frontend in Svelte.
- Scripts executions are sandboxed using Google's
[nsjail](https://github.com/google/nsjail).
- Javascript runtime is the
[deno_core rust library](https://denolib.gitbook.io/guide/) (which itself uses
the [rusty_v8](https://github.com/denoland/rusty_v8) and hence V8 underneath).
- TypeScript runtime is Bun and deno.
- Python runtime is python3.
- Golang runtime is 1.19.1.
- postgres as the database
- backend in Rust with the follwing highly-available and horizontally scalable
architecture:
- stateless API backend
- workers that pull jobs from a queue
- frontend in svelte
- scripts executions are sandboxed using google's nsjail
- javascript runtime is deno_core rust library (which itself uses the rusty_v8
and hence V8 underneath)
- typescript runtime is deno
- python runtime is python3
## Fastest Self-Hostable Workflow Engine
### Development stack
We have compared Windmill to other self-hostable workflow engines (Airflow,
Prefect & Temporal) and Windmill is the most performant solution for both
benchmarks: one flow composed of 40 lightweight tasks & one flow composed of 10
long-running tasks.
All methodology & results on our
[Benchmarks](https://www.windmill.dev/docs/misc/benchmarks/competitors#airflow-setup)
page.
![Fastest workflow engine](./imgs/fastest.png)
## Security
### Sandboxing
Windmill can use [nsjail](https://github.com/google/nsjail). It is production
multi-tenant grade secure. Do not take our word for it, take
[fly.io's one](https://fly.io/blog/sandboxing-and-workload-isolation/).
### Secrets, credentials and sensitive values
There is one encryption key per workspace to encrypt the credentials and secrets
stored in Windmill's K/V store.
In addition, we strongly recommend that you encrypt the whole Postgres database.
That is what we do at <https://app.windmill.dev>.
## Performance
Once a job started, there is no overhead compared to running the same script on
the node with its corresponding runner (Deno/Go/Python/Bash). The added latency
from a job being pulled from the queue, started, and then having its result sent
back to the database is ~50ms. A typical lightweight deno job will take around
100ms total.
- caddy is the reverse proxy used for local development, see frontend's
Caddyfile and CaddyfileRemote
## Architecture
<p align="center">
<img src="./imgs/diagram.svg">
</p>
![Architecture](./imgs/architecture.svg)
## How to self-host
We only provide docker-compose setup here. For more advanced setups, like
compiling from source or using without a postgres super user, see
[Self-Host documentation](https://www.windmill.dev/docs/advanced/self_host).
`docker volume create caddy_data && docker-compose up` with the following
docker-compose is sufficient:
<https://github.com/windmill-labs/windmill/blob/main/docker-compose.yml>
### Docker compose
The default super-admin user is: admin@windmill.dev / changeme
Windmill can be deployed using 3 files:
([docker-compose.yml](./docker-compose.yml), [Caddyfile](./Caddyfile) and a
[.env](./.env)) in a single command.
From there, you can create other users (do not forget to change the password!)
Make sure Docker is started, and run:
```
curl https://raw.githubusercontent.com/windmill-labs/windmill/main/docker-compose.yml -o docker-compose.yml
curl https://raw.githubusercontent.com/windmill-labs/windmill/main/Caddyfile -o Caddyfile
curl https://raw.githubusercontent.com/windmill-labs/windmill/main/.env -o .env
docker compose up -d
```
Go to http://localhost et voilà :)
The default super-admin user is: admin@windmill.dev / changeme.
From there, you can follow the setup app and create other users.
More details in
[Self-Host Documention](https://www.windmill.dev/docs/advanced/self_host#docker).
### Kubernetes (k8s) and Helm charts
We publish helm charts at:
<https://github.com/windmill-labs/windmill-helm-charts>.
### Run from binaries
Each release includes the corresponding binaries for x86_64. You can simply
download the latest `windmill` binary using the following set of bash commands.
```bash
BINARY_NAME='windmill-amd64' # or windmill-ee-amd64 for the enterprise edition
LATEST_RELEASE=$(curl -L -s -H 'Accept: application/json' https://github.com/windmill-labs/windmill/releases/latest)
LATEST_VERSION=$(echo $LATEST_RELEASE | sed -e 's/.*"tag_name":"\([^"]*\)".*/\1/')
ARTIFACT_URL="https://github.com/windmill-labs/windmill/releases/download/$LATEST_VERSION/$BINARY_NAME"
wget "$ARTIFACT_URL" -O windmill
```
### OAuth, SSO & SMTP
Windmill Community Edition allows to configure the OAuth, SSO (including Google
Workspace SSO, Microsoft/Azure and Okta) directly from the UI in the superadmin
settings. Do note that there is a limit of 10 SSO users on the community
edition.
[See documentation](https://www.windmill.dev/docs/misc/setup_oauth).
### Commercial license
See the [LICENSE](https://github.com/windmill-labs/windmill/blob/main/LICENSE)
file for the full license text.
The "Community Edition" of Windmill available in the docker images hosted under
ghcr.io/windmill-labs/windmill and the github binary releases contains the files
under the AGPLv3 and Apache 2 sources but also includes proprietary and
non-public code and features which are not open source and under the following
terms: Windmill Labs, Inc. grants a right to use all the features of the
"Community Edition" for free without restrictions other than the limits and
quotas set in the software and a right to distribute the community edition as is
but not to sell, resell, serve Windmill as a managed service, modify or wrap
under any form without an explicit agreement.
The binary compilable from source code in this repository without the
"enterprise" feature flag is open-source under the
[LICENSE-AGPLv3](https://github.com/windmill-labs/windmill/blob/main/LICENSE-AGPL)
License terms and conditions.
To
[re-expose directly any Windmill parts to your users](https://www.windmill.dev/docs/misc/white_labelling)
as a feature of your product, with the exception of iframed public Windmill
"apps", or to build a feature on top of "Windmill Community Edition" that you
sell commercially or embed in a distributable product or binary, you must get a
commercial license. Contact us at <sales@windmill.dev> if you have any
questions. To do the same from the binary compiled from the source code in this
repository without the "enterprise" feature flag, you must comply with the
AGPLv3 license terms and conditions or get a commercial license from Windmill
Labs, Inc.
To use Windmill "Community Edition" as is internally in your organization, or to
use its APIs as is, you do NOT need a commercial license.
### Integrations
In Windmill, integrations are referred to as
[resources and resource types](https://www.windmill.dev/docs/core_concepts/resources_and_types).
Each Resource has a Resource Type that defines the schema that the resource
needs to implement.
On self-hosted instances, you might want to import all the approved resource
types from [WindmillHub](https://hub.windmill.dev). A setup script will prompt
you to have it being synced automatically everyday.
## Environment Variables
| Environment Variable name | Default | Description | Api Server/Worker/All |
| ----------------------------------- | ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------- |
| DATABASE_URL | | The Postgres database url. | All |
| WORKER_GROUP | default | The worker group the worker belongs to and get its configuration pulled from | Worker |
| MODE | standalone | The mode if the binary. Possible values: standalone, worker, server, agent | All |
| METRICS_ADDR | None | (ee only) The socket addr at which to expose Prometheus metrics at the /metrics path. Set to "true" to expose it on port 8001 | All |
| JSON_FMT | false | Output the logs in json format instead of logfmt | All |
| BASE_URL | http://localhost:8000 | The base url that is exposed publicly to access your instance. Is overriden by the instance settings if any. | Server |
| ZOMBIE_JOB_TIMEOUT | 30 | The timeout after which a job is considered to be zombie if the worker did not send pings about processing the job (every server check for zombie jobs every 30s) | Server |
| RESTART_ZOMBIE_JOBS | true | If true then a zombie job is restarted (in-place with the same uuid and some logs), if false the zombie job is failed | Server |
| SLEEP_QUEUE | 50 | The number of ms to sleep in between the last check for new jobs in the DB. It is multiplied by NUM_WORKERS such that in average, for one worker instance, there is one pull every SLEEP_QUEUE ms. | Worker |
| KEEP_JOB_DIR | false | Keep the job directory after the job is done. Useful for debugging. | Worker |
| LICENSE_KEY (EE only) | None | License key checked at startup for the Enterprise Edition of Windmill | Worker |
| SLACK_SIGNING_SECRET | None | The signing secret of your Slack app. See [Slack documentation](https://api.slack.com/authentication/verifying-requests-from-slack) | Server |
| COOKIE_DOMAIN | None | The domain of the cookie. If not set, the cookie will be set by the browser based on the full origin | Server |
| DENO_PATH | /usr/bin/deno | The path to the deno binary. | Worker |
| PYTHON_PATH | | The path to the python binary if wanting to not have it managed by uv. | Worker |
| GO_PATH | /usr/bin/go | The path to the go binary. | Worker |
| GOPRIVATE | | The GOPRIVATE env variable to use private go modules | Worker |
| GOPROXY | | The GOPROXY env variable to use | Worker |
| NETRC | | The netrc content to use a private go registry | Worker |
| PY_CONCURRENT_DOWNLOADS | 20 | Sets the maximum number of in-flight concurrent python downloads that windmill will perform at any given time. | Worker |
| PATH | None | The path environment variable, usually inherited | Worker |
| HOME | None | The home directory to use for Go and Bash , usually inherited | Worker |
| DATABASE_CONNECTIONS | 50 (Server)/3 (Worker) | The max number of connections in the database connection pool | All |
| SUPERADMIN_SECRET | None | A token that would let the caller act as a virtual superadmin superadmin@windmill.dev | Server |
| TIMEOUT_WAIT_RESULT | 20 | The number of seconds to wait before timeout on the 'run_wait_result' endpoint | Worker |
| QUEUE_LIMIT_WAIT_RESULT | None | The number of max jobs in the queue before rejecting immediately the request in 'run_wait_result' endpoint. Takes precedence on the query arg. If none is specified, there are no limit. | Worker |
| DENO_AUTH_TOKENS | None | Custom DENO_AUTH_TOKENS to pass to worker to allow the use of private modules | Worker |
| DISABLE_RESPONSE_LOGS | false | Disable response logs | Server |
| CREATE_WORKSPACE_REQUIRE_SUPERADMIN | true | If true, only superadmins can create new workspaces | Server |
| MIN_FREE_DISK_SPACE_MB | 15000 | Minimum amount of free space on worker. Sends critical alert if worker has less free space. | Worker |
## Run a local dev setup
See the [./frontend/README_DEV.md](./frontend/README_DEV.md) file for all
running options.
Using [Nix](./frontend/README_DEV.md#nix).
### only Frontend
This will use the backend of <https://app.windmill.dev> but your own frontend
with hot-code reloading. Note that you will need to use a username / password
login due to CSRF checks using a different auth provider.
In the `frontend/` directory:
1. install the dependencies with `npm install` (or `pnpm install` or `yarn`)
2. generate the windmill client:
```
npm run generate-backend-client
## on mac use
npm run generate-backend-client-mac
```
3. Run your dev server with `npm run dev`
4. Et voilà, windmill should be available at `http://localhost/`
### Backend + Frontend
See the [./frontend/README_DEV.md](./frontend/README_DEV.md) file for all
running options.
1. Create a Postgres Database for Windmill and create an admin role inside your
Postgres setup. The easiest way to get a working db is to run
```
cargo install sqlx-cli
env DATABASE_URL=<YOUR_DATABASE_URL> sqlx migrate run
```
This will also avoid compile time issue with sqlx's `query!` macro
2. Install [nsjail](https://github.com/google/nsjail) and have it accessible in
your PATH
3. Install deno and python3, have the bins at `/usr/bin/deno` and
`/usr/local/bin/python3`
4. Install [caddy](https://caddyserver.com)
5. Install the [lld linker](https://lld.llvm.org/)
6. Go to `frontend/`:
1. `npm install`, `npm run generate-backend-client` then `npm run dev`
2. You might need to set some extra heap space for the node runtime
`export NODE_OPTIONS="--max-old-space-size=4096"`
3. In another shell `npm run build` otherwise the backend will not find the
`frontend/build` folder and will not compile.
4. In another shell `sudo caddy run --config Caddyfile`
7. Go to `backend/`:
`env DATABASE_URL=<DATABASE_URL_TO_YOUR_WINDMILL_DB> RUST_LOG=info cargo run`
8. Et voilà, windmill should be available at `http://localhost/`
## Contributors
<a href="https://github.com/windmill-labs/windmill/graphs/contributors">
<img src="https://contrib.rocks/image?repo=windmill-labs/windmill" />
</a>
Detailed instructions for more complex deployments will come soon. For simpler
docker based ones, the docker-compose.yml file contains all the necessary
informations.
## Copyright
Windmill Labs, Inc 2023
2021 [Ruben Fiszel](https://github.com/rubenfiszel)
### Acknowledgement
This project is inspired from a previous project called
[Delightool](https://github.com/windmill-labs/delightool-legacy) which was also
led by [Ruben](https://github.com/rubenfiszel) and with large contribution on
the frontend from [Malo Marrec](https://github.com/malomarrec) who gave his
blessing to Windmill.

3
backend/.cargo/config Normal file
View File

@@ -0,0 +1,3 @@
[build]
rustflags = ["--cfg", "tokio_unstable"]
incremental = true

View File

@@ -1,14 +0,0 @@
[build]
incremental = true
[target.x86_64-apple-darwin]
rustflags = [
"-C", "link-arg=-undefined",
"-C", "link-arg=dynamic_lookup",
]
[target.aarch64-apple-darwin]
rustflags = [
"-C", "link-arg=-undefined",
"-C", "link-arg=dynamic_lookup",
]

View File

@@ -1,27 +0,0 @@
// For format details, see https://aka.ms/devcontainer.json. For config options, see the README at:
// https://github.com/microsoft/vscode-dev-containers/tree/v0.245.2/containers/docker-existing-docker-compose
// If you want to run as a non-root user in the container, see .devcontainer/docker-compose.yml.
{
"name": "Backend Container",
"dockerComposeFile": [
"../docker-compose.yml",
"../.devcontainer/docker-compose.yml"
],
"customizations": {
"vscode": {
"extensions": [
"rust-lang.rust-analyzer",
"yzhang.markdown-all-in-one"
]
}
},
"service": "windmill",
"workspaceFolder": "/workspace",
// Use 'forwardPorts' to make a list of ports inside the container available locally.
"forwardPorts": [
8000
],
// "postCreateCommand": "sudo apt-get update && sudo apt-get install -y lld python3 && cargo install sqlx-cli --no-default-features --features native-tls,postgres && (cd backend && sqlx migrate run) && cargo install deno --locked && rustup toolchain install nightly-2022-05-23 -c rust-src -c rustc-dev -c llvm-tools-preview",
"postCreateCommand": "cd backend && sqlx migrate run",
"remoteUser": "vscode"
}

View File

@@ -1 +0,0 @@
sqlx-data.json -diff

7
backend/.gitignore vendored
View File

@@ -1,9 +1,2 @@
target/
.env
oauth.json
oauth2.json
tracing.folded
heaptrack*
index/
windmill-api/openapi-*.*
.duckdb/*

View File

@@ -1,14 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE alerts SET acknowledged_workspace = true, acknowledged = true WHERE workspace_id = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text"
]
},
"nullable": []
},
"hash": "00588a40dde5189ac1c61505f17acb0f4c244c60477427505bf5bd1b104d3bf9"
}

View File

@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT email FROM password WHERE email = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "email",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false
]
},
"hash": "005b9255699e73600c579f74b529caf531b2312b6e405b4d35efd2f7ca663143"
}

View File

@@ -1,16 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE usr SET disabled = $1 WHERE username = $2 AND workspace_id = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Bool",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "006f03e979abdf8055b1c598bc9806337216a6abf74db4eb64b0acb918a0de08"
}

View File

@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT runnable_path FROM v2_job WHERE id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "runnable_path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Uuid"
]
},
"nullable": [
true
]
},
"hash": "00b6340396f5121aff49323d12a7e3244ebb2cb864828637e7ff7476a4a4939b"
}

View File

@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM usr WHERE workspace_id = $1 AND email = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": []
},
"hash": "00be497354f5375e9ccffb998d126a853da91d607ff9e57e10d0e5481e4d3848"
}

View File

@@ -1,40 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n DELETE\n FROM parallel_monitor_lock\n WHERE last_ping IS NOT NULL AND last_ping < NOW() - ($1 || ' seconds')::interval\n RETURNING parent_flow_id, job_id, last_ping, (SELECT workspace_id FROM v2_job_queue q\n WHERE q.id = parent_flow_id AND q.running = true AND q.canceled_by IS NULL\n ) AS workspace_id\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "parent_flow_id",
"type_info": "Uuid"
},
{
"ordinal": 1,
"name": "job_id",
"type_info": "Uuid"
},
{
"ordinal": 2,
"name": "last_ping",
"type_info": "Timestamptz"
},
{
"ordinal": 3,
"name": "workspace_id",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false,
false,
true,
null
]
},
"hash": "00c4a602aa6a50f2f922851ce63b5216e915c7649698687a00d47da55c70349f"
}

View File

@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO metrics (id, value)\n VALUES ($1, to_jsonb((\n SELECT EXTRACT(EPOCH FROM now() - scheduled_for)\n FROM v2_job_queue\n WHERE tag = $2 AND running = false AND scheduled_for <= now() - ('3 seconds')::interval\n ORDER BY priority DESC NULLS LAST, scheduled_for LIMIT 1\n )))",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "00e63eab76d26e148b77e932848de74e8b0943d30481465da453942e299a128f"
}

View File

@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE resource SET workspace_id = $1 WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "00f68ea1ad2d5ab045b4a20ce3f4dd7850041e396e14890e37e1d0db276d3694"
}

View File

@@ -1,50 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT\n path,\n is_flow,\n workspace_id,\n owner,\n email,\n trigger_config as \"trigger_config!: _\"\n FROM\n capture_config\n WHERE\n trigger_kind = 'postgres' AND\n last_client_ping > NOW() - INTERVAL '10 seconds' AND\n trigger_config IS NOT NULL AND\n (last_server_ping IS NULL OR last_server_ping < now() - interval '15 seconds')\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "path",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "is_flow",
"type_info": "Bool"
},
{
"ordinal": 2,
"name": "workspace_id",
"type_info": "Varchar"
},
{
"ordinal": 3,
"name": "owner",
"type_info": "Varchar"
},
{
"ordinal": 4,
"name": "email",
"type_info": "Varchar"
},
{
"ordinal": 5,
"name": "trigger_config!: _",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": []
},
"nullable": [
false,
false,
false,
false,
false,
true
]
},
"hash": "011c7638eeeda710deb86a216a9e10df9c3e9458e85bcdde466b01011a1f2ac2"
}

View File

@@ -1,16 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE flow SET archived = $1 WHERE path = $2 AND workspace_id = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Bool",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "01576057b71e32e20a1702b2c89c380eee422060a65b475d7cda8b199d70842e"
}

View File

@@ -1,17 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_runnable_dependencies (app_path, runnable_path, runnable_is_flow, workspace_id) VALUES ($1, $2, $3, $4) ON CONFLICT DO NOTHING",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Bool",
"Varchar"
]
},
"nullable": []
},
"hash": "01755585cd3f6e100a66da331720286cbc09d4abf2926146b24a8c95cf21e5c8"
}

View File

@@ -1,23 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT app.id FROM app\n WHERE app.path = $1 AND app.workspace_id = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false
]
},
"hash": "019258392434b3c8dfabfe53d61ad766626fe4ad67f101c1a58c9c9524531621"
}

View File

@@ -1,14 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM global_settings WHERE name = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text"
]
},
"nullable": []
},
"hash": "025e5bf6aef56e9832239489daf21b4970d84529cf7ed7928ce2dcfec8b007f0"
}

View File

@@ -1,26 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE capture_config SET last_server_ping = now(), error = $1 WHERE workspace_id = $2 AND path = $3 AND is_flow = $4 AND trigger_kind = 'websocket' AND server_id = $5 AND last_client_ping > NOW() - INTERVAL '10 seconds' RETURNING 1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "?column?",
"type_info": "Int4"
}
],
"parameters": {
"Left": [
"Text",
"Text",
"Text",
"Bool",
"Text"
]
},
"nullable": [
null
]
},
"hash": "031d0d70b0aff52feaad487bddb74e5ef0aaa2505facbea8c764003dfc8fffb1"
}

View File

@@ -1,12 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "create index concurrently if not exists ix_job_workspace_id_created_at_new_9 ON v2_job (workspace_id, created_at DESC) where kind in ('dependencies', 'flowdependencies', 'appdependencies') AND parent_job IS NULL",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "036c84bb9ce72748956bc9c18fbe276444fab025a281dc4784596b0e31c1cb9d"
}

View File

@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE usr SET username = $1 WHERE email = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "03cb31686d34035f2dc64d4dff93265af0e44a7e2dabdf28fd9c9327b5b53118"
}

View File

@@ -1,16 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE app SET policy = jsonb_set(policy, ARRAY['on_behalf_of'], to_jsonb('u/' || $1)) WHERE policy->>'on_behalf_of' = ('u/' || $2) AND workspace_id = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "03d4ec7a5feda9001e98c2e55f0788399ee41c9b128099937abb483f1d89a2c6"
}

View File

@@ -1,59 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT * from resource_type WHERE name = $1 AND (workspace_id = $2 OR workspace_id = 'admins')",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "workspace_id",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "name",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "schema",
"type_info": "Jsonb"
},
{
"ordinal": 3,
"name": "description",
"type_info": "Text"
},
{
"ordinal": 4,
"name": "edited_at",
"type_info": "Timestamptz"
},
{
"ordinal": 5,
"name": "created_by",
"type_info": "Varchar"
},
{
"ordinal": 6,
"name": "format_extension",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
false,
true,
true,
true,
true,
true
]
},
"hash": "03d63d2e64b012f624d2731b5bcb8849c74a9474777be61edf0ed43ddda07ef3"
}

View File

@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM usr_to_group WHERE usr = $1 AND workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": []
},
"hash": "03e9c9d8dd74c75b4608b49cad79f0a9572c7e8b48d8e373c0342b75b843d123"
}

View File

@@ -1,18 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO alerts (alert_type, message, acknowledged, acknowledged_workspace, workspace_id, resource)\n VALUES ('critical_error', $1, $2, $3, $4, $5)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Bool",
"Bool",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "044e2b428ee6e2dd4543c87ad8835e239cf7567d18b8b3fa6608ea3a9d206ca7"
}

View File

@@ -1,28 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT healthy, created_at FROM healthchecks WHERE check_type = 'min_alive_workers_' || $1 ORDER BY created_at DESC LIMIT 1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "healthy",
"type_info": "Bool"
},
{
"ordinal": 1,
"name": "created_at",
"type_info": "Timestamptz"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false,
false
]
},
"hash": "050fb876e10ad13654dbbde4532f408ff1ac92ed0f5d31a3ef6c58313e1f8671"
}

View File

@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE schedule SET on_success = NULL, on_success_extra_args = NULL WHERE workspace_id = $1 RETURNING path",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false
]
},
"hash": "051acd49542618e3be5654dce94567a02e87be23a4c0863e3b2d53b5c2cb6749"
}

View File

@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT status = 'success' OR status = 'skipped' AS \"success!\" FROM v2_job_completed WHERE id = ANY($1)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "success!",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"UuidArray"
]
},
"nullable": [
null
]
},
"hash": "05c65ba8a56b3b5f8bd37c30c0c6707522e01c4a05104969889b7bb41d6aa509"
}

View File

@@ -1,14 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO concurrency_counter(concurrency_id, job_uuids) \n VALUES ($1, '{}'::jsonb)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar"
]
},
"nullable": []
},
"hash": "05cb171b610bfb45f6228128a385cde8a5b86d7ca377a028004cc382e12faf41"
}

View File

@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE usr SET workspace_id = $1 WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "0659bab15d4cccdb04c7a57e0e3bbb6bfebb8896601a27ddf5618d4eae678bc1"
}

View File

@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT ai_config FROM workspace_settings WHERE workspace_id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "ai_config",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
true
]
},
"hash": "0689cdc6c7676f5e1984792a0e0b172ea9a70835bfba6cef56142556197e9767"
}

View File

@@ -1,14 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE v2_job_status\n SET flow_status = flow_status - 'retry'\n WHERE id = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid"
]
},
"nullable": []
},
"hash": "06db0e720dd59a7c52c0a98ea7b316237eb1a547678858c1a1e45985035b3468"
}

View File

@@ -1,26 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n UPDATE \n gcp_trigger \n SET \n enabled = $1, \n email = $2, \n edited_by = $3, \n edited_at = now(), \n server_id = NULL, \n error = NULL\n WHERE \n path = $4 AND \n workspace_id = $5 \n RETURNING 1\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "?column?",
"type_info": "Int4"
}
],
"parameters": {
"Left": [
"Bool",
"Varchar",
"Varchar",
"Text",
"Text"
]
},
"nullable": [
null
]
},
"hash": "070b8ad0b59f485fa5bf68082b060f5c3561c37e9c6f2834d234a862a475a6eb"
}

View File

@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO email_to_igroup (email, igroup) VALUES ($1, $2)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar"
]
},
"nullable": []
},
"hash": "07834003a631acac03a52a128d3dfce162abdef0d94e5020b84fed424dfd1ae9"
}

View File

@@ -1,23 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT EXISTS(SELECT 1 FROM resource WHERE path = $1 AND workspace_id = $2)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "exists",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
null
]
},
"hash": "0784bb86a503f02b9ef247a2b83a82ddfa49632552b223a9f4536a449b0a1eb8"
}

View File

@@ -1,14 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM flow WHERE workspace_id = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text"
]
},
"nullable": []
},
"hash": "07b984faa14193f529ae589608289a9ffb0f3d4a0e48054393e1bb8b70b4e5e0"
}

Some files were not shown because too many files have changed in this diff Show More