Compare commits

..

1 Commits

Author SHA1 Message Date
Alex Petric
2eb7360210 fix: add leaf_job to openapi spec of QueuedJob 2025-05-01 17:56:35 -04:00
65 changed files with 917 additions and 2653 deletions

View File

@@ -1,113 +0,0 @@
---
description:
globs: backend/**/*.rs
alwaysApply: false
---
---
description: Rust best practices for the Windmill backend, covering code organization, error handling, performance optimizations, and common patterns to follow when adding new code.
globs: **/*.rs
---
# Windmill Backend - Rust Best Practices
## Project Structure
Windmill uses a workspace-based architecture with multiple crates:
- **windmill-api**: API server functionality
- **windmill-worker**: Job execution
- **windmill-common**: Shared code used by all crates
- **windmill-queue**: Job & flow queuing
- **windmill-audit**: Audit logging
- Other specialized crates (git-sync, autoscaling, etc.)
## Adding New Code
### Module Organization
- Place new code in the appropriate crate based on functionality
- For API endpoints, create or modify files in `windmill-api/src/` organized by domain
- For shared functionality, use `windmill-common/src/`
- Use the `_ee.rs` suffix for enterprise-only modules
- Follow existing patterns for file structure and organization
### Error Handling
- Use the custom `Error` enum from `windmill-common::error`
- Return `Result<T, Error>` or `JsonResult<T>` for functions that can fail
- Use the `?` operator for error propagation
- Add location tracking to errors using `#[track_caller]`
### Database Operations
- Use `sqlx` for database operations with prepared statements
- Leverage existing database helper functions in `db.rs` modules
- Use transactions for multi-step operations
- Handle database errors properly
### API Endpoints
- Follow existing patterns in the `windmill-api` crate
- Use axum's routing system and extractors
- Group related routes together
- Use consistent response formats (JSON)
- Follow proper authentication and authorization patterns
## Performance Optimizations
When generating code, especially involving `serde`, `sqlx`, and `tokio`, prioritize performance by applying the following principles:
### Serde Optimizations (Serialization & Deserialization)
- **Specify Structure Explicitly:** When defining structs for Serde (`#[derive(Serialize, Deserialize)]`), use `#[serde(...` attributes extensively. This includes:
* `#[serde(rename = "...")]` or `#[serde(alias = "...")]` to map external names precisely, avoiding dynamic lookups.
* `#[serde(default)]` for optional fields with default values, reducing parsing complexity.
* `#[serde(skip_serializing_if = "...")]` to avoid writing fields that meet a certain condition (e.g., `Option::is_none()`, `Vec::is_empty()`, or a custom function), reducing output size and serialization work.
* `#[serde(skip_serializing)]` or `#[serde(skip_deserializing)]` for fields that should *not* be included.
- **Prefer Borrowing:** Where possible and safe (data lifetime allows), use `Cow<'a, str>` or `&'a str` (with `#[serde(borrow)]`) instead of `String` for string fields during deserialization. This avoids allocating new strings, enabling zero-copy reading from the input buffer. Apply this principle to byte slices (`&'a [u8]` / `Cow<'a, [u8]>`) and potentially borrowed vectors as well.
- **Avoid Intermediate `Value`:** Unless the data structure is truly dynamic or unknown at compile time, deserialize directly into a well-defined struct or enum rather than into `serde_json::Value` (or equivalent for other formats). This avoids unnecessary heap allocations and type switching.
### SQLx Optimizations (Database Interaction)
- **Select Only Necessary Columns:** In `SELECT` queries, list specific column names rather than using `SELECT *`. This reduces data transferred from the database and the work needed for hydration/deserialization.
- **Batch Operations:** For multiple `INSERT`, `UPDATE`, or `DELETE` statements, prefer executing them in a single query if the database and driver support it efficiently (e.g., `INSERT INTO ... VALUES (...), (...), ...`). This minimizes round trips to the database.
- **Avoid N+1 Queries:** Do not loop through results of one query and execute a separate query for each item (e.g., fetching users, then querying for each user's profile in a loop). Instead, use JOINs or a single query with an `IN` clause to fetch related data efficiently.
- **Deserialize Directly:** Use `#[derive(FromRow)]` on structs and ensure the struct fields match the selected columns in the query. This allows SQLx to hydrate objects directly, avoiding intermediate data structures.
- **Parameterize Queries:** Always use SQLx's query methods (`.bind(...)`) to pass values as parameters rather than string formatting. This prevents SQL injection and allows the database to cache query plans, improving performance on repeated executions.
### Tokio Optimizations (Asynchronous Runtime)
- **Avoid Blocking Operations:** **Crucially**, never perform blocking operations (synchronous file I/O, `std::thread::sleep`, CPU-bound loops, `std::sync::Mutex::lock`, blocking network calls without `tokio::net`) directly within an `async fn` or a standard `tokio::spawn` task. Blocking pauses the entire worker thread, potentially starving other tasks. Use `tokio::task::spawn_blocking` for CPU-intensive work or blocking I/O.
- **Use Tokio's Async Primitives:** Prefer `tokio::sync` (channels, mutexes, semaphores), `tokio::io`, `tokio::net`, and `tokio::time` over their `std` counterparts in asynchronous contexts. These are designed to yield control back to the scheduler.
- **Manage Concurrency:** Be mindful of how many tasks are spawned. Creating a new task for every tiny piece of work can introduce overhead. Group related asynchronous operations where appropriate.
- **Handle Shared State Efficiently:** Use `Arc` for shared ownership in concurrent tasks. When shared state needs mutation, prefer `tokio::sync::Mutex` over `std::sync::Mutex` in `async` code. Consider `tokio::sync::RwLock` if reads significantly outnumber writes. Minimize the duration for which locks are held.
- **Understand `.await`:** Place `.await` strategically to allow the runtime to switch to other ready tasks. Ensure that `.await` points to genuinely asynchronous operations.
- **Backpressure:** If dealing with data streams or queues between tasks, implement backpressure mechanisms (e.g., bounded channels like `tokio::sync::mpsc::channel`) to prevent one component from overwhelming another or critical resources like the database.
## Enterprise Features
- Use feature flags for enterprise functionality
- Conditionally compile with `#[cfg(feature = "enterprise")]`
- Isolate enterprise code in separate modules
## Code Style
- Group imports by external and internal crates
- Place struct/enum definitions before implementations
- Group similar functionality together
- Use descriptive naming consistent with the codebase
- Follow existing patterns for async code using tokio
## Testing
- Write unit tests for core functionality
- Use the `#[cfg(test)]` module for test code
- For database tests, use the existing test utilities
## Common Crates Used
- **tokio**: For async runtime
- **axum**: For web server and routing
- **sqlx**: For database operations
- **serde**: For serialization/deserialization
- **tracing**: For logging and diagnostics
- **reqwest**: For HTTP client functionality

View File

@@ -1,229 +0,0 @@
---
description:
globs: frontend/src/**/*.svelte
alwaysApply: false
---
# Svelte 5 Best Practices
This guide outlines best practices for developing with Svelte 5, incorporating the new Runes API and other modern Svelte features. They should be applied on every new files created, but not on existing svelte 4 files unless specifically asked to.
## Reactivity with Runes
Svelte 5 introduces Runes for more explicit and flexible reactivity.
1. **Embrace Runes for State Management**:
* Use `$state` for reactive local component state.
```svelte
<script>
let count = $state(0);
function increment() {
count += 1;
}
</script>
<button onclick={increment}>
Clicked {count} {count === 1 ? 'time' : 'times'}
</button>
```
* Use `$derived` for computed values based on other reactive state.
```svelte
<script>
let count = $state(0);
const doubled = $derived(count * 2);
</script>
<p>{count} * 2 = {doubled}</p>
```
* Use `$effect` for side effects that need to run when reactive values change (e.g., logging, manual DOM manipulation, data fetching). Remember `$effect` does not run on the server.
```svelte
<script>
let count = $state(0);
$effect(() => {
console.log('The count is now', count);
if (count > 5) {
alert('Count is too high!');
}
});
</script>
```
2. **Props with `$props`**:
* Declare component props using `$props()`. This offers better clarity and flexibility compared to `export let`.
```svelte
<script>
// ChildComponent.svelte
let { name, age = $state(30) } = $props();
</script>
<p>Name: {name}</p>
<p>Age: {age}</p>
```
* For bindable props, use `$bindable`.
```svelte
<script>
// MyInput.svelte
let { value = $bindable() } = $props();
</script>
<input bind:value />
```
## Event Handling
* **Use direct event attributes**: Svelte 5 moves away from `on:` directives for DOM events.
* **Do**: `<button onclick={handleClick}>...</button>`
* **Don't**: `<button on:click={handleClick}>...</button>`
* **For component events, prefer callback props**: Instead of `createEventDispatcher`, pass functions as props.
```svelte
<!-- Parent.svelte -->
<script>
import Child from './Child.svelte';
let message = $state('');
function handleChildEvent(detail) {
message = detail;
}
</script>
<Child onCustomEvent={handleChildEvent} />
<p>Message from child: {message}</p>
<!-- Child.svelte -->
<script>
let { onCustomEvent } = $props();
function emitEvent() {
onCustomEvent('Hello from child!');
}
</script>
<button onclick={emitEvent}>Send Event</button>
```
## Snippets for Content Projection
* **Use `{#snippet ...}` and `{@render ...}` instead of slots**: Snippets are more powerful and flexible.
```svelte
<!-- Parent.svelte -->
<script>
import Card from './Card.svelte';
</script>
<Card>
{#snippet title()}
My Awesome Title
{/snippet}
{#snippet content()}
<p>Some interesting content here.</p>
{/snippet}
</Card>
<!-- Card.svelte -->
<script>
let { title, content } = $props();
</script>
<article>
<header>{@render title()}</header>
<div>{@render content()}</div>
</article>
```
* Default content is passed via the `children` prop (which is a snippet).
```svelte
<!-- Wrapper.svelte -->
<script>
let { children } = $props();
</script>
<div>
{@render children?.()}
</div>
```
## Component Design
1. **Create Small, Reusable Components**: Break down complex UIs into smaller, focused components. Each component should have a single responsibility. This also aids performance by limiting the scope of reactivity updates.
2. **Descriptive Naming**: Use clear and descriptive names for variables, functions, and components.
3. **Minimize Logic in Components**: Move complex business logic to utility functions or services. Keep components focused on presentation and interaction.
## State Management (Stores)
1. **Segment Stores**: Avoid a single global store. Create multiple stores, each responsible for a specific piece of global state (e.g., `userStore.js`, `themeStore.js`). This can help limit reactivity updates to only the parts of the UI that depend on specific state segments.
2. **Use Custom Stores for Complex Logic**: For stores with related methods, create custom stores.
```javascript
// counterStore.js
import { writable } from 'svelte/store';
function createCounter() {
const { subscribe, set, update } = writable(0);
return {
subscribe,
increment: () => update(n => n + 1),
decrement: () => update(n => n - 1),
reset: () => set(0)
};
}
export const counter = createCounter();
```
3. **Use Context API for Localized State**: For state shared within a component subtree, consider Svelte's context API (`setContext`, `getContext`) instead of global stores when the state doesn't need to be truly global.
## Performance Optimizations (Svelte 5)
When generating Svelte 5 code, prioritize frontend performance by applying the following principles:
### General Svelte 5 Principles
- **Leverage the Compiler:** Trust Svelte's compiler to generate optimized JavaScript. Avoid manual DOM manipulation (`document.querySelector`, etc.) unless absolutely necessary for integrating third-party libraries that lack Svelte adapters.
- **Keep Components Small and Focused:** Reinforcing from Component Design, smaller components lead to less complex reactivity graphs and more targeted, efficient updates.
### Reactivity & State Management
- **Optimize Computations with `$derived`:** Always use `$derived` for computed values that depend on other state. This ensures the computation only runs when its specific dependencies change, avoiding unnecessary work compared to recomputing derived values in `$effect` or less efficient methods.
- **Minimize `$effect` Usage:** Use `$effect` sparingly and only for true side effects that interact with the outside world or non-Svelte state. Avoid putting complex logic or state updates *within* an `$effect` unless those updates are explicitly intended as a reaction to external changes or non-Svelte state. Excessive or complex effects can impact rendering performance.
- **Structure State for Fine-Grained Updates:** Design your `$state` objects or variables such that updates affect only the necessary parts of the UI. Avoid putting too much unrelated state into a single large object that gets frequently updated, as this can potentially trigger broader updates than necessary. Consider normalizing complex, nested state.
### List Rendering (`{#each}`)
- **Mandate `key` Attribute:** Always use a `key` attribute (`{#each items as item (item.id)}`) that refers to a unique, stable identifier for each item in a list. This is critical for allowing Svelte to efficiently update, reorder, add, or remove list items without destroying and re-creating unnecessary DOM elements and component instances.
### Component Loading & Bundling
- **Implement Lazy Loading/Code Splitting:** For routes, components, or modules that are not immediately needed on page load, use dynamic imports (`import(...)`) to split the code bundle. SvelteKit handles this automatically for routes, but it can be applied manually to components using helper patterns if needed.
- **Be Mindful of Third-Party Libraries:** When incorporating external libraries, import only the necessary functions or components to minimize the final bundle size. Prefer libraries designed to be tree-shakeable.
### Rendering & DOM
- **Use CSS for Animations/Transitions:** Prefer CSS animations or transitions where possible for performance. Svelte's built-in `transition:` directive is also highly optimized and should be used for complex state-driven transitions, but simple cases can often use plain CSS.
- **Optimize Image Loading:** Implement best practices for images: use optimized formats (WebP, AVIF), lazy loading (`loading="lazy"`), and responsive images (`<picture>`, `srcset`) to avoid loading unnecessarily large images.
### Server-Side Rendering (SSR) & Hydration
- **Ensure SSR Compatibility:** Write components that can be rendered on the server for faster initial page loads. Avoid relying on browser-specific APIs (like `window` or `document`) in the main `<script>` context. If necessary, use `$effect` or check `if (browser)` inside effects to run browser-specific code only on the client.
- **Minimize Work During Hydration:** Structure components and data fetching such that minimal complex setup or computation is required when the client-side Svelte code takes over from the server-rendered HTML. Heavy synchronous work during hydration can block the main thread.
## General Clean Code Practices
1. **Organized File Structure**: Group related files together. A common structure:
```
/src
|-- /routes // Page components (if using a router like SvelteKit)
|-- /lib // Utility functions, services, constants (SvelteKit often uses this)
| |-- /stores
| |-- /utils
| |-- /services
| |-- /components // Reusable UI components
|-- App.svelte
|-- main.js (or main.ts)
```
2. **Scoped Styles**: Keep CSS scoped to components to avoid unintended side effects and improve maintainability. Avoid `:global` where possible.
3. **Immutability**: With Svelte 5 and `$state`, direct assignments to properties of `$state` objects (`obj.prop = value;`) are generally fine as Svelte's reactivity system handles updates. However, for non-rune state or when interacting with other systems, understanding and sometimes preferring immutable updates (creating new objects/arrays) can still be relevant.
4. **Use `class:` and `style:` directives**: For dynamic classes and styles, use Svelte's built-in directives for cleaner templates and potentially optimized updates.
```svelte
<script>
let isActive = $state(true);
let color = $state('blue');
</script>
<div class:active={isActive} style:color={color}>
Hello
</div>
```
5. **Stay Updated**: Keep Svelte and its related packages up to date to benefit from the latest features, performance improvements, and security fixes.

View File

@@ -1,53 +1,5 @@
# Changelog
## [1.488.0](https://github.com/windmill-labs/windmill/compare/v1.487.0...v1.488.0) (2025-05-07)
### Features
* handle . in interpolated args ([0ac8e47](https://github.com/windmill-labs/windmill/commit/0ac8e477d6fb7c5a7699a198fce9d18a08aff68c))
### Bug Fixes
* fix azure object storage regression due to object_store regression ([df9f827](https://github.com/windmill-labs/windmill/commit/df9f827d103def27166a767044373bd0754285e2))
* performance and stability improvement to fetch last deployed script ([75d9924](https://github.com/windmill-labs/windmill/commit/75d992449c845fd11c9a317d401c405e7d78e1ec))
## [1.487.0](https://github.com/windmill-labs/windmill/compare/v1.486.1...v1.487.0) (2025-05-06)
### Features
* critical alert if disk near full ([#5549](https://github.com/windmill-labs/windmill/issues/5549)) ([4fd0561](https://github.com/windmill-labs/windmill/commit/4fd056123907337efb5f5669975b337973a124cc))
### Bug Fixes
* ansible in agent mode can use inventory.ini ([9bdd301](https://github.com/windmill-labs/windmill/commit/9bdd301f5296fbfb631df9ff9100e92e0984ff64))
## [1.486.1](https://github.com/windmill-labs/windmill/compare/v1.486.0...v1.486.1) (2025-05-04)
### Bug Fixes
* improve MultiSelectWrapper behavior ([36da8ae](https://github.com/windmill-labs/windmill/commit/36da8aec080742e13f23e1dee12b3954947f53dd))
## [1.486.0](https://github.com/windmill-labs/windmill/compare/v1.485.3...v1.486.0) (2025-05-01)
### Features
* add run now directly on schedule drawer and duplicate schedule option ([#5674](https://github.com/windmill-labs/windmill/issues/5674)) ([dfb947f](https://github.com/windmill-labs/windmill/commit/dfb947ff37c688f54a32de5aa3c5c3d142cb80f4))
* Database Manager ([#5586](https://github.com/windmill-labs/windmill/issues/5586)) ([41c15fc](https://github.com/windmill-labs/windmill/commit/41c15fc78aaf844c559d3d6c772e04ecce436e9d))
* Integrate MCP with hub ([#5685](https://github.com/windmill-labs/windmill/issues/5685)) ([ec701a9](https://github.com/windmill-labs/windmill/commit/ec701a9ee74c9d890b54234362392deca63a77c7))
### Bug Fixes
* Ai Chat: do not send tools if empty + respond even if tool fails ([#5692](https://github.com/windmill-labs/windmill/issues/5692)) ([9c55040](https://github.com/windmill-labs/windmill/commit/9c55040e47e76af8b7e2864b82fa30505545dcb5))
* do not track relative deps for scripts with raw defined deps from CLI ([#5696](https://github.com/windmill-labs/windmill/issues/5696)) ([7eb9d7d](https://github.com/windmill-labs/windmill/commit/7eb9d7d46cb48ae69a3fd3ff852a57abae450a3b))
* improve CLI file scanning performances ([0916978](https://github.com/windmill-labs/windmill/commit/09169784bd2d0ab7acf5f40dc86f36f1cae967b7))
## [1.485.3](https://github.com/windmill-labs/windmill/compare/v1.485.2...v1.485.3) (2025-04-29)

View File

@@ -363,7 +363,6 @@ you to have it being synced automatically everyday.
| DENO_AUTH_TOKENS | None | Custom DENO_AUTH_TOKENS to pass to worker to allow the use of private modules | Worker |
| DISABLE_RESPONSE_LOGS | false | Disable response logs | Server |
| CREATE_WORKSPACE_REQUIRE_SUPERADMIN | true | If true, only superadmins can create new workspaces | Server |
| MIN_FREE_DISK_SPACE_MB | 15000 | Minimum amount of free space on worker. Sends critical alert if worker has less free space. | Worker |
## Run a local dev setup

View File

@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE script SET ws_error_handler_muted = $3 WHERE workspace_id = $2 AND path = $1 AND created_at = (SELECT max(created_at) FROM script WHERE path = $1 AND workspace_id = $2)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Bool"
]
},
"nullable": []
},
"hash": "1182fe055306d7ea435d76b74d781e066915c8397e6bbc9e408ff3dda9fec27f"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO log_file (hostname, mode, worker_group, log_ts, file_path, ok_lines, err_lines, json_fmt)\n VALUES ($1, $2::text::LOG_MODE, $3, $4, $5, $6, $7, $8)\n ON CONFLICT (hostname, log_ts) DO UPDATE SET ok_lines = log_file.ok_lines + $6, err_lines = log_file.err_lines + $7",
"query": "INSERT INTO log_file (hostname, mode, worker_group, log_ts, file_path, ok_lines, err_lines, json_fmt) VALUES ($1, $2::text::LOG_MODE, $3, $4, $5, $6, $7, $8)",
"describe": {
"columns": [],
"parameters": {
@@ -17,5 +17,5 @@
},
"nullable": []
},
"hash": "92faee8714a45a403b623e04d789f7f99067a05e9dfe270223164db8a1df2e4b"
"hash": "33c1793e55b1127d88d2509aadd0eb04e042463200f237b4c2cb176612fa16fe"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT s.hash as hash, dm.deployment_msg as deployment_msg \n FROM script s LEFT JOIN deployment_metadata dm ON s.hash = dm.script_hash\n WHERE s.workspace_id = $1 AND s.path = $2\n ORDER by s.created_at DESC",
"query": "SELECT s.hash as hash, dm.deployment_msg as deployment_msg \n FROM script s LEFT JOIN deployment_metadata dm ON s.hash = dm.script_hash\n WHERE s.workspace_id = $1 AND s.path = $2\n ORDER by created_at DESC",
"describe": {
"columns": [
{
@@ -25,5 +25,5 @@
true
]
},
"hash": "726e956cfcd3ac7c07abeecdf92cf0996efe7fa7b671ac2b3b000ead0ea307de"
"hash": "362419eb262c83d6a98a0200b116e831ada60399fe5f55a56d930cc69aff2675"
}

View File

@@ -1,23 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT EXISTS(SELECT 1 FROM script WHERE path = $1 AND workspace_id = $2 AND archived = false ORDER BY created_at DESC LIMIT 1)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "exists",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
null
]
},
"hash": "4e07dcc1ba4396ada2f1080a400ad9fad00b1d77ea44c8639b7598c5953635ff"
}

View File

@@ -1,16 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE script \n SET ws_error_handler_muted = $3 \n WHERE ctid = (\n SELECT ctid FROM script\n WHERE path = $1 AND workspace_id = $2\n ORDER BY created_at DESC\n LIMIT 1\n )\n",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Bool"
]
},
"nullable": []
},
"hash": "848c8371eeb17ebd4b36a33f7d8a61eb8f07c54d291bb857ddd41a549cbc88dd"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT content FROM script WHERE path = $1 AND workspace_id = $2 AND archived = false ORDER BY created_at DESC LIMIT 1",
"query": "SELECT content FROM script WHERE path = $1 AND workspace_id = $2 AND\n created_at = (SELECT max(created_at) FROM script WHERE path = $1 AND archived = false AND workspace_id = $2)",
"describe": {
"columns": [
{
@@ -19,5 +19,5 @@
false
]
},
"hash": "b5860f6a7672a368d740dcd367a8d5ab98fa93e0382a57a698564695db6c40ac"
"hash": "a17260a1f1ee02e786690994d98c84ddf81e2eeb883f895c9cfc47e144d422cb"
}

View File

@@ -1,29 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT s.hash as hash, dm.deployment_msg as deployment_msg \n FROM script s LEFT JOIN deployment_metadata dm ON s.hash = dm.script_hash\n WHERE s.workspace_id = $1 AND s.path = $2\n ORDER by s.created_at DESC LIMIT 1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "hash",
"type_info": "Int8"
},
{
"ordinal": 1,
"name": "deployment_msg",
"type_info": "Text"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
true
]
},
"hash": "cf2a6ad6471a40b6298775cda9300aeecdd75503bed59d80cd62091d1642d1ec"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT EXISTS(SELECT 1 FROM script WHERE path = $1 AND workspace_id = $2 ORDER BY created_at DESC LIMIT 1)",
"query": "SELECT EXISTS(SELECT 1 FROM script WHERE path = $1 AND workspace_id = $2 AND\n created_at = (SELECT max(created_at) FROM script WHERE path = $1 AND workspace_id = $2))",
"describe": {
"columns": [
{
@@ -19,5 +19,5 @@
null
]
},
"hash": "2a49e5b5486b650d96f3e9038cba8a5f2e75d3b12ee4718452e82c7318b1bcf4"
"hash": "ea2b88dc050aec038641ea37399d68d4385c5bdc721d1351609f27ca45e4dbdc"
}

1524
backend/Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,6 @@
[package]
name = "windmill"
version = "1.488.0"
version = "1.485.3"
authors.workspace = true
edition.workspace = true
@@ -32,7 +32,7 @@ members = [
]
[workspace.package]
version = "1.488.0"
version = "1.485.3"
authors = ["Ruben Fiszel <ruben@windmill.dev>"]
edition = "2021"
@@ -96,9 +96,6 @@ java = ["windmill-worker/java"]
all_languages = [ "python", "deno_core", "rust", "mysql", "oracledb", "mssql", "bigquery", "csharp", "nu", "php", "java"]
[patch.crates-io]
object_store = { git = "https://github.com/apache/arrow-rs-object-store", rev = "36752c975d4f29e20b57c91f81a10872dcd48ae7" }
[dependencies]
anyhow.workspace = true
tokio.workspace = true
@@ -135,8 +132,6 @@ quote.workspace = true
memchr.workspace = true
v8 = { workspace = true, optional = true }
rustls.workspace = true
systemstat.workspace = true
size.workspace = true
[target.'cfg(not(target_env = "msvc"))'.dependencies]
tikv-jemallocator = { optional = true, workspace = true }
@@ -197,7 +192,7 @@ serde_json = { version = "^1", features = ["preserve_order", "raw_value"] }
uuid = { version = "^1", features = ["serde", "v4"] }
thiserror = "^2"
anyhow = "^1"
chrono = { version = "^0.4", features = ["serde"] }
chrono = { version = "=0.4.39", features = ["serde"] }
chrono-tz = "^0.10.1"
tracing = "^0"
tracing-subscriber = { version = "^0", features = ["env-filter", "json"] }
@@ -320,9 +315,9 @@ nix = { version = "0.27.1", features = ["process", "signal"] }
tinyvector = { git = "https://github.com/windmill-labs/tinyvector", rev = "20823b94c20f2b9093f318badd24026cf54dcc85" }
hf-hub = "0.3.2"
tokenizers = "0.14.1"
candle-core = "0.9.1"
candle-transformers = "0.9.1"
candle-nn = "0.9.1"
candle-core = "0.3.0"
candle-transformers = "0.3.0"
candle-nn = "0.3.0"
tiberius = { version = "0.12.3", default-features = false, features = ["rustls", "tds73", "chrono", "sql-browser-tokio"]}
pin-project = "1"
indexmap = { version = "2.2.5", features = ["serde"]}
@@ -336,8 +331,8 @@ async-nats = "0.38.0"
nkeys = "0.4.4"
nu-parser = { version = "0.101.0", default-features = false }
datafusion = "47.0.0"
object_store = { git = "https://github.com/apache/arrow-rs-object-store", rev = "36752c975d4f29e20b57c91f81a10872dcd48ae7", features = ["aws", "azure"] }
datafusion = "39.0.0"
object_store = { version = "0.10.0", features = ["aws", "azure"] }
openidconnect = { version = "4.0.0-rc.1" }
aws-config = "^1"
aws-sdk-sqs = "1.57.0"
@@ -360,6 +355,9 @@ bollard = "0.18.1"
tonic = { version = "=0.12.3", features = ["tls-native-roots"] }
byteorder = "1.5.0"
# todo remove
half = "=2.4.1"
tikv-jemallocator = { version = "0.5" }
tikv-jemalloc-sys = { version = "^0.5" }
tikv-jemalloc-ctl = { version = "^0.5" }
@@ -370,8 +368,6 @@ pin-project-lite = "^0"
tantivy = "0.22.0"
backon = "1.3.0"
systemstat = "0.2.4"
size = "0.5.0"
flume = { version = "0.11.1", features = ["async"] }

View File

@@ -1 +1 @@
868ccad87afb804fe22818ecd3d5a091199bcdbf
96a2129f585a1bc6567ef08bd27aaa4aca70543f

View File

@@ -1 +0,0 @@
DELETE FROM global_settings WHERE name = 'critical_alerts_on_db_oversize';

View File

@@ -1,3 +0,0 @@
INSERT INTO global_settings (name, value)
VALUES ('critical_alerts_on_db_oversize', '{}')
ON CONFLICT (name) DO NOTHING;

View File

@@ -1 +0,0 @@
-- Add down migration script here

View File

@@ -1,3 +0,0 @@
-- Add up migration script here
DROP INDEX IF EXISTS index_script_on_path_created_at;
CREATE INDEX IF NOT EXISTS index_script_on_path_created_at ON script (workspace_id, path, created_at DESC);

View File

@@ -142,8 +142,6 @@ pub static FULL_IMPORTS_MAP: PyMap = phf_map! {
"google.cloud.dns" => "google-cloud-dns",
"google.cloud.runtimeconfig" => "google-cloud-runtimeconfig",
"google.cloud.iot" => "google-cloud-iot",
"google.generativeai" => "google-generativeai",
"google.genai" => "google-genai",
// Azure
"azure.mgmt.hybridkubernetes" => "azure-mgmt-hybridkubernetes",
"azure.mgmt.sql" => "azure-mgmt-sql",

View File

@@ -8,12 +8,11 @@
use anyhow::Context;
use monitor::{
load_base_url, load_otel, reload_critical_alerts_on_db_oversize,
reload_delete_logs_periodically_setting, reload_indexer_config,
load_base_url, load_otel, reload_delete_logs_periodically_setting, reload_indexer_config,
reload_instance_python_version_setting, reload_maven_repos_setting,
reload_no_default_maven_setting, reload_nuget_config_setting,
reload_timeout_wait_result_setting, send_current_log_file_to_object_store,
send_logs_to_object_store, WORKERS_NAMES,
send_logs_to_object_store,
};
use rand::Rng;
use sqlx::postgres::PgListener;
@@ -34,16 +33,15 @@ use windmill_common::{
agent_workers::build_agent_http_client,
get_database_url,
global_settings::{
BASE_URL_SETTING, BUNFIG_INSTALL_SCOPES_SETTING, CRITICAL_ALERTS_ON_DB_OVERSIZE_SETTING,
CRITICAL_ALERT_MUTE_UI_SETTING, CRITICAL_ERROR_CHANNELS_SETTING, CUSTOM_TAGS_SETTING,
DEFAULT_TAGS_PER_WORKSPACE_SETTING, DEFAULT_TAGS_WORKSPACES_SETTING, EMAIL_DOMAIN_SETTING,
ENV_SETTINGS, EXPOSE_DEBUG_METRICS_SETTING, EXPOSE_METRICS_SETTING,
EXTRA_PIP_INDEX_URL_SETTING, HUB_BASE_URL_SETTING, INDEXER_SETTING,
INSTANCE_PYTHON_VERSION_SETTING, JOB_DEFAULT_TIMEOUT_SECS_SETTING, JWT_SECRET_SETTING,
KEEP_JOB_DIR_SETTING, LICENSE_KEY_SETTING, MAVEN_REPOS_SETTING,
MONITOR_LOGS_ON_OBJECT_STORE_SETTING, NO_DEFAULT_MAVEN_SETTING,
NPM_CONFIG_REGISTRY_SETTING, NUGET_CONFIG_SETTING, OAUTH_SETTING, OTEL_SETTING,
PIP_INDEX_URL_SETTING, REQUEST_SIZE_LIMIT_SETTING,
BASE_URL_SETTING, BUNFIG_INSTALL_SCOPES_SETTING, CRITICAL_ALERT_MUTE_UI_SETTING,
CRITICAL_ERROR_CHANNELS_SETTING, CUSTOM_TAGS_SETTING, DEFAULT_TAGS_PER_WORKSPACE_SETTING,
DEFAULT_TAGS_WORKSPACES_SETTING, EMAIL_DOMAIN_SETTING, ENV_SETTINGS,
EXPOSE_DEBUG_METRICS_SETTING, EXPOSE_METRICS_SETTING, EXTRA_PIP_INDEX_URL_SETTING,
HUB_BASE_URL_SETTING, INDEXER_SETTING, INSTANCE_PYTHON_VERSION_SETTING,
JOB_DEFAULT_TIMEOUT_SECS_SETTING, JWT_SECRET_SETTING, KEEP_JOB_DIR_SETTING,
LICENSE_KEY_SETTING, MAVEN_REPOS_SETTING, MONITOR_LOGS_ON_OBJECT_STORE_SETTING,
NO_DEFAULT_MAVEN_SETTING, NPM_CONFIG_REGISTRY_SETTING, NUGET_CONFIG_SETTING, OAUTH_SETTING,
OTEL_SETTING, PIP_INDEX_URL_SETTING, REQUEST_SIZE_LIMIT_SETTING,
REQUIRE_PREEXISTING_USER_FOR_OAUTH_SETTING, RETENTION_PERIOD_SECS_SETTING,
SAML_METADATA_SETTING, SCIM_TOKEN_SETTING, SMTP_SETTING, TEAMS_SETTING,
TIMEOUT_WAIT_RESULT_SETTING,
@@ -919,12 +917,6 @@ Windmill Community Edition {GIT_VERSION}
tracing::error!(error = %e, "Could not reload critical error emails setting");
}
},
CRITICAL_ALERTS_ON_DB_OVERSIZE_SETTING => {
if let Err(e) = reload_critical_alerts_on_db_oversize(&db).await {
tracing::error!(error = %e, "Could not reload critical alerts on db oversize setting");
}
},
JWT_SECRET_SETTING => {
if let Err(e) = reload_jwt_secret_setting(&db).await {
tracing::error!(error = %e, "Could not reload jwt secret setting");
@@ -1222,12 +1214,10 @@ pub async fn run_workers(
"Starting {num_workers} workers and SLEEP_QUEUE={}ms",
*windmill_worker::SLEEP_QUEUE
);
for i in 1..(num_workers + 1) {
let wk_conf = &workers[i as usize - 1];
let conn1 = wk_conf.conn.clone();
let worker_name = wk_conf.worker_name.clone();
WORKERS_NAMES.write().await.push(worker_name.clone());
let ip = ip.clone();
let rx = killpill_rxs.pop().unwrap();
let tx = tx.clone();

View File

@@ -13,7 +13,7 @@ use std::{
use chrono::{NaiveDateTime, Utc};
use futures::{stream::FuturesUnordered, StreamExt};
use serde::{de::DeserializeOwned, Deserialize};
use serde::de::DeserializeOwned;
use sqlx::{Pool, Postgres};
use tokio::{
join,
@@ -30,8 +30,6 @@ use windmill_api::{
#[cfg(feature = "enterprise")]
use windmill_common::ee::{jobs_waiting_alerts, worker_groups_alerts};
#[cfg(feature = "enterprise")]
use windmill_common::ee::low_disk_alerts;
#[cfg(feature = "oauth2")]
use windmill_common::global_settings::OAUTH_SETTING;
@@ -42,14 +40,13 @@ use windmill_common::{
error,
flow_status::{FlowStatus, FlowStatusModule},
global_settings::{
BASE_URL_SETTING, BUNFIG_INSTALL_SCOPES_SETTING, CRITICAL_ALERTS_ON_DB_OVERSIZE_SETTING,
CRITICAL_ALERT_MUTE_UI_SETTING, CRITICAL_ERROR_CHANNELS_SETTING,
DEFAULT_TAGS_PER_WORKSPACE_SETTING, DEFAULT_TAGS_WORKSPACES_SETTING,
EXPOSE_DEBUG_METRICS_SETTING, EXPOSE_METRICS_SETTING, EXTRA_PIP_INDEX_URL_SETTING,
HUB_BASE_URL_SETTING, INSTANCE_PYTHON_VERSION_SETTING, JOB_DEFAULT_TIMEOUT_SECS_SETTING,
JWT_SECRET_SETTING, KEEP_JOB_DIR_SETTING, LICENSE_KEY_SETTING,
MONITOR_LOGS_ON_OBJECT_STORE_SETTING, NPM_CONFIG_REGISTRY_SETTING, NUGET_CONFIG_SETTING,
OTEL_SETTING, PIP_INDEX_URL_SETTING, REQUEST_SIZE_LIMIT_SETTING,
BASE_URL_SETTING, BUNFIG_INSTALL_SCOPES_SETTING, CRITICAL_ALERT_MUTE_UI_SETTING,
CRITICAL_ERROR_CHANNELS_SETTING, DEFAULT_TAGS_PER_WORKSPACE_SETTING,
DEFAULT_TAGS_WORKSPACES_SETTING, EXPOSE_DEBUG_METRICS_SETTING, EXPOSE_METRICS_SETTING,
EXTRA_PIP_INDEX_URL_SETTING, HUB_BASE_URL_SETTING, INSTANCE_PYTHON_VERSION_SETTING,
JOB_DEFAULT_TIMEOUT_SECS_SETTING, JWT_SECRET_SETTING, KEEP_JOB_DIR_SETTING,
LICENSE_KEY_SETTING, MONITOR_LOGS_ON_OBJECT_STORE_SETTING, NPM_CONFIG_REGISTRY_SETTING,
NUGET_CONFIG_SETTING, OTEL_SETTING, PIP_INDEX_URL_SETTING, REQUEST_SIZE_LIMIT_SETTING,
REQUIRE_PREEXISTING_USER_FOR_OAUTH_SETTING, RETENTION_PERIOD_SECS_SETTING,
SAML_METADATA_SETTING, SCIM_TOKEN_SETTING, TIMEOUT_WAIT_RESULT_SETTING,
},
@@ -68,10 +65,10 @@ use windmill_common::{
DEFAULT_TAGS_WORKSPACES, INDEXER_CONFIG, SCRIPT_TOKEN_EXPIRY, SMTP_CONFIG, TMP_DIR,
WORKER_CONFIG, WORKER_GROUP,
},
KillpillSender, BASE_URL, CRITICAL_ALERTS_ON_DB_OVERSIZE, CRITICAL_ALERT_MUTE_UI_ENABLED,
CRITICAL_ERROR_CHANNELS, DB, DEFAULT_HUB_BASE_URL, HUB_BASE_URL, JOB_RETENTION_SECS,
METRICS_DEBUG_ENABLED, METRICS_ENABLED, MONITOR_LOGS_ON_OBJECT_STORE, OTEL_LOGS_ENABLED,
OTEL_METRICS_ENABLED, OTEL_TRACING_ENABLED, SERVICE_LOG_RETENTION_SECS,
KillpillSender, BASE_URL, CRITICAL_ALERT_MUTE_UI_ENABLED, CRITICAL_ERROR_CHANNELS, DB,
DEFAULT_HUB_BASE_URL, HUB_BASE_URL, JOB_RETENTION_SECS, METRICS_DEBUG_ENABLED, METRICS_ENABLED,
MONITOR_LOGS_ON_OBJECT_STORE, OTEL_LOGS_ENABLED, OTEL_METRICS_ENABLED, OTEL_TRACING_ENABLED,
SERVICE_LOG_RETENTION_SECS,
};
use windmill_queue::{cancel_job, MiniPulledJob, SameWorkerPayload};
use windmill_worker::{
@@ -137,7 +134,7 @@ lazy_static::lazy_static! {
.and_then(|x| x.parse::<bool>().ok())
.unwrap_or(false);
pub static ref WORKERS_NAMES: Arc<RwLock<Vec<String>>> = Arc::new(RwLock::new(Vec::new()));
static ref QUEUE_COUNT_TAGS: Arc<RwLock<Vec<String>>> = Arc::new(RwLock::new(Vec::new()));
static ref DISABLE_CONCURRENCY_LIMIT: bool = std::env::var("DISABLE_CONCURRENCY_LIMIT").is_ok_and(|s| s == "true");
@@ -186,12 +183,6 @@ pub async fn initial_load(
if server_mode {
if let Some(db) = conn.as_sql() {
load_require_preexisting_user(db).await;
if let Err(e) = reload_critical_alerts_on_db_oversize(db).await {
tracing::error!(
"Error reloading critical alerts on db oversize setting: {:?}",
e
)
}
}
}
@@ -647,9 +638,7 @@ async fn send_log_file_to_object_store(
let (ok_lines, err_lines) = read_log_counters(ts_str);
if let Some(db) = conn.as_sql() {
if let Err(e) = sqlx::query!("INSERT INTO log_file (hostname, mode, worker_group, log_ts, file_path, ok_lines, err_lines, json_fmt)
VALUES ($1, $2::text::LOG_MODE, $3, $4, $5, $6, $7, $8)
ON CONFLICT (hostname, log_ts) DO UPDATE SET ok_lines = log_file.ok_lines + $6, err_lines = log_file.err_lines + $7",
if let Err(e) = sqlx::query!("INSERT INTO log_file (hostname, mode, worker_group, log_ts, file_path, ok_lines, err_lines, json_fmt) VALUES ($1, $2::text::LOG_MODE, $3, $4, $5, $6, $7, $8)",
hostname, mode.to_string(), worker_group.clone(), ts, highest_file, ok_lines as i64, err_lines as i64, *JSON_FMT)
.execute(db)
.await {
@@ -1415,23 +1404,6 @@ pub async fn monitor_db(
}
};
let low_disk_alerts_f = async {
#[cfg(feature = "enterprise")]
if let Some(db) = conn.as_sql() {
low_disk_alerts(
&db,
server_mode,
_worker_mode,
WORKERS_NAMES.read().await.clone(),
)
.await;
}
#[cfg(not(feature = "enterprise"))]
{
()
}
};
let apply_autoscaling_f = async {
#[cfg(feature = "enterprise")]
if server_mode && !initial_load {
@@ -1454,7 +1426,6 @@ pub async fn monitor_db(
verify_license_key_f,
worker_groups_alerts_f,
jobs_waiting_alerts_f,
low_disk_alerts_f,
apply_autoscaling_f,
update_min_worker_version_f,
);
@@ -2210,37 +2181,6 @@ pub async fn reload_critical_error_channels_setting(conn: &DB) -> error::Result<
Ok(())
}
pub async fn reload_critical_alerts_on_db_oversize(conn: &DB) -> error::Result<()> {
#[derive(Deserialize)]
struct DBOversize {
enabled: bool,
value: f32,
}
let db_oversize_value =
load_value_from_global_settings(conn, CRITICAL_ALERTS_ON_DB_OVERSIZE_SETTING).await?;
let db_oversize = if let Some(q) = db_oversize_value {
match serde_json::from_value::<DBOversize>(q.clone()) {
Ok(DBOversize { enabled: true, value }) => Some(value),
Ok(_) => None,
Err(q) => {
tracing::error!(
"Could not parse critical_alerts_on_db_oversize setting, found: {:#?}",
&q
);
None
}
}
} else {
None
};
let mut l = CRITICAL_ALERTS_ON_DB_OVERSIZE.write().await;
*l = db_oversize;
Ok(())
}
async fn generate_and_save_jwt_secret(db: &DB) -> error::Result<String> {
let secret = rd_string(32);
sqlx::query!(

View File

@@ -16,20 +16,16 @@ use tokio::sync::RwLock;
#[cfg(feature = "enterprise")]
use tokio::time::{timeout, Duration};
#[cfg(feature = "python")]
use windmill_api_client::types::{CreateFlowBody, RawScript};
#[cfg(feature = "enterprise")]
use windmill_api_client::types::{EditSchedule, NewSchedule, ScriptArgs};
use windmill_api_client::types::{NewScript, ScriptLang as NewScriptLanguage};
use serde::Serialize;
#[cfg(feature = "deno_core")]
use windmill_common::flows::InputTransform;
use windmill_common::worker::WORKER_CONFIG;
use windmill_common::{
flow_status::{FlowStatus, FlowStatusModule, RestartedFrom},
flows::{FlowModule, FlowModuleValue, FlowValue},
flows::{FlowModule, FlowModuleValue, FlowValue, InputTransform},
jobs::{JobKind, JobPayload, RawCode},
jwt::JWT_SECRET,
scripts::{ScriptHash, ScriptLang},
@@ -145,7 +141,7 @@ impl ApiServer {
format!("http://localhost:{}", addr.port()),
));
_port_rx.await.expect("failed to receive port");
_port_rx.await.unwrap();
// clear the cache between tests
windmill_common::cache::clear();
@@ -171,7 +167,6 @@ impl ApiServer {
// Ok(())
// }
#[cfg(feature = "python")]
fn get_module(cjob: &CompletedJob, id: &str) -> Option<FlowStatusModule> {
cjob.flow_status.clone().and_then(|fs| {
find_module_in_vec(
@@ -181,7 +176,6 @@ fn get_module(cjob: &CompletedJob, id: &str) -> Option<FlowStatusModule> {
})
}
#[cfg(feature = "python")]
fn find_module_in_vec(modules: Vec<FlowStatusModule>, id: &str) -> Option<FlowStatusModule> {
modules.into_iter().find(|s| s.id() == id)
}
@@ -290,7 +284,6 @@ mod suspend_resume {
.unwrap()
}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base"))]
async fn test(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -373,7 +366,6 @@ mod suspend_resume {
);
}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base"))]
async fn cancel_from_job(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -399,7 +391,6 @@ mod suspend_resume {
);
}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base"))]
async fn cancel_after_suspend(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -573,7 +564,6 @@ def main(last, port):
.unwrap()
}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base"))]
async fn test_pass(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -619,7 +609,6 @@ def main(last, port):
assert_eq!(json!([3, 5, 7, 9]), result);
}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base"))]
async fn test_fail_step_zero(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -663,7 +652,6 @@ def main(last, port):
);
}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base"))]
async fn test_fail_step_one(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -705,7 +693,6 @@ def main(last, port):
.contains("index out of range"));
}
#[cfg(feature = "python")]
#[sqlx::test(fixtures("base"))]
async fn test_with_failure_module(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -782,7 +769,6 @@ def main(error, port):
}
}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base"))]
async fn test_iteration(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -841,7 +827,6 @@ async fn test_iteration(db: Pool<Postgres>) {
.contains("2"));
}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base"))]
async fn test_iteration_parallel(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -1120,7 +1105,6 @@ trait StreamFind: futures::Stream + Unpin + Sized {
impl<T: futures::Stream + Unpin + Sized> StreamFind for T {}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base"))]
async fn test_deno_flow(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -1239,7 +1223,6 @@ async fn test_deno_flow(db: Pool<Postgres>) {
}
}
#[cfg(feature = "python")]
#[sqlx::test(fixtures("base"))]
async fn test_identity(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -1277,7 +1260,6 @@ async fn test_identity(db: Pool<Postgres>) {
assert_eq!(result, serde_json::json!(42));
}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base"))]
async fn test_deno_flow_same_worker(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -1553,7 +1535,6 @@ async fn test_flow_result_by_id(db: Pool<Postgres>) {
assert_eq!(result, serde_json::json!([[42]]));
}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base"))]
async fn test_stop_after_if(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -1607,7 +1588,6 @@ async fn test_stop_after_if(db: Pool<Postgres>) {
assert_eq!(json!(-123), result);
}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base"))]
async fn test_stop_after_if_nested(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -1666,7 +1646,6 @@ async fn test_stop_after_if_nested(db: Pool<Postgres>) {
assert_eq!(json!([-123]), result);
}
#[cfg(all(feature = "deno_core", feature = "python"))]
#[sqlx::test(fixtures("base"))]
async fn test_python_flow(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -1724,7 +1703,6 @@ async fn test_python_flow(db: Pool<Postgres>) {
}
}
#[cfg(feature = "python")]
#[sqlx::test(fixtures("base"))]
async fn test_python_flow_2(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -1799,7 +1777,6 @@ func main(derp string) (string, error) {
assert_eq!(result, serde_json::json!("hello world"));
}
#[cfg(feature = "rust")]
#[sqlx::test(fixtures("base"))]
async fn test_rust_job(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -2055,7 +2032,6 @@ public class Main {
assert_eq!(job.json_result(), Some(json!("hello world")));
}
#[cfg(feature = "python")]
#[sqlx::test(fixtures("base"))]
async fn test_python_job(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -2089,7 +2065,6 @@ def main():
assert_eq!(result, serde_json::json!("hello world"));
}
#[cfg(feature = "python")]
#[sqlx::test(fixtures("base"))]
async fn test_python_job_heavy_dep(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -2126,7 +2101,6 @@ def main():
assert_eq!(result, serde_json::json!(3));
}
#[cfg(feature = "python")]
#[sqlx::test(fixtures("base"))]
async fn test_python_job_with_imports(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -2230,7 +2204,6 @@ export async function main(a: Date) {
assert_eq!(result, serde_json::json!("object"));
}
#[cfg(feature = "python")]
#[sqlx::test(fixtures("base"))]
async fn test_python_job_datetime_and_bytes(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -2266,7 +2239,6 @@ def main(a: datetime, b: bytes):
assert_eq!(result, serde_json::json!([true, true]));
}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base"))]
async fn test_empty_loop_1(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -2323,7 +2295,6 @@ async fn test_empty_loop_1(db: Pool<Postgres>) {
assert_eq!(result, serde_json::json!(0));
}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base"))]
async fn test_invalid_first_step(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -2404,7 +2375,6 @@ async fn test_empty_loop_2(db: Pool<Postgres>) {
assert_eq!(result, serde_json::json!([]));
}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base"))]
async fn test_step_after_loop(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -2528,7 +2498,6 @@ async fn test_branchone_simple(db: Pool<Postgres>) {
assert_eq!(result, serde_json::json!([1, 2]));
}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base"))]
async fn test_branchone_with_cond(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -2565,7 +2534,6 @@ async fn test_branchone_with_cond(db: Pool<Postgres>) {
assert_eq!(result, serde_json::json!([1, 3]));
}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base"))]
async fn test_branchall_sequential(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -2604,7 +2572,6 @@ async fn test_branchall_sequential(db: Pool<Postgres>) {
assert_eq!(result, serde_json::json!([[1, 2], [1, 3]]));
}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base"))]
async fn test_branchall_simple(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -2732,7 +2699,6 @@ async fn test_branchall_skip_failure(db: Pool<Postgres>) {
);
}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base"))]
async fn test_branchone_nested(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -2852,7 +2818,6 @@ async fn test_branchall_nested(db: Pool<Postgres>) {
);
}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base"))]
async fn test_failure_module(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -2965,7 +2930,6 @@ async fn test_failure_module(db: Pool<Postgres>) {
assert_eq!(json!({ "l": [0, 1, 2] }), result);
}
#[cfg(feature = "python")]
#[sqlx::test(fixtures("base"))]
async fn test_flow_lock_all(db: Pool<Postgres>) {
use futures::StreamExt;
@@ -3104,7 +3068,6 @@ async fn test_flow_lock_all(db: Pool<Postgres>) {
});
}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base"))]
async fn test_complex_flow_restart(db: Pool<Postgres>) {
@@ -3789,7 +3752,6 @@ export async function main() {
run_preview_relative_imports(&db, content, ScriptLang::Bun).await;
}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base", "relative_bun"))]
async fn test_nested_imports_bun(db: Pool<Postgres>) {
let content = r#"
@@ -3838,7 +3800,6 @@ export async function main() {
run_preview_relative_imports(&db, content, ScriptLang::Deno).await;
}
#[cfg(feature = "python")]
#[sqlx::test(fixtures("base", "relative_python"))]
async fn test_relative_imports_python(db: Pool<Postgres>) {
let content = r#"
@@ -3856,7 +3817,6 @@ def main():
run_preview_relative_imports(&db, content, ScriptLang::Python3).await;
}
#[cfg(feature = "python")]
#[sqlx::test(fixtures("base", "relative_python"))]
async fn test_nested_imports_python(db: Pool<Postgres>) {
let content = r#"
@@ -3872,7 +3832,6 @@ def main():
run_preview_relative_imports(&db, content, ScriptLang::Python3).await;
}
#[cfg(feature = "python")]
async fn assert_lockfile(
db: &Pool<Postgres>,
script_content: String,
@@ -3965,8 +3924,6 @@ async fn assert_lockfile(
)
.await;
}
#[cfg(feature = "python")]
#[sqlx::test(fixtures("base", "lockfile_python"))]
async fn test_requirements_python(db: Pool<Postgres>) {
let content = r#"
@@ -3992,8 +3949,6 @@ def main():
)
.await;
}
#[cfg(feature = "python")]
#[sqlx::test(fixtures("base", "lockfile_python"))]
async fn test_extra_requirements_python(db: Pool<Postgres>) {
{
@@ -4021,8 +3976,6 @@ def main():
.await;
}
}
#[cfg(feature = "python")]
#[sqlx::test(fixtures("base", "lockfile_python"))]
async fn test_extra_requirements_python2(db: Pool<Postgres>) {
let content = r#"
@@ -4045,7 +3998,6 @@ def main():
.await;
}
#[cfg(feature = "python")]
#[sqlx::test(fixtures("base", "lockfile_python"))]
async fn test_pins_python(db: Pool<Postgres>) {
let content = r#"
@@ -4329,7 +4281,6 @@ mod job_payload {
];
}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base", "hello"))]
async fn test_script_hash_payload(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -4490,7 +4441,6 @@ mod job_payload {
test_for_versions(VERSION_FLAGS.iter().cloned(), test).await;
}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base", "hello"))]
async fn test_flow_node_payload(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -4675,7 +4625,6 @@ mod job_payload {
test_for_versions(VERSION_FLAGS.iter().cloned(), test).await;
}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base", "hello"))]
async fn test_flow_payload(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -4718,7 +4667,6 @@ mod job_payload {
test_for_versions(VERSION_FLAGS.iter().cloned(), test).await;
}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base", "hello"))]
async fn test_flow_payload_with_preprocessor(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -4786,7 +4734,6 @@ mod job_payload {
test_for_versions(VERSION_FLAGS.iter().cloned(), test).await;
}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base", "hello"))]
async fn test_restarted_flow_payload(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -4839,7 +4786,6 @@ mod job_payload {
test_for_versions(VERSION_FLAGS.iter().cloned(), test).await;
}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base", "hello"))]
async fn test_raw_flow_payload(db: Pool<Postgres>) {
initialize_tracing().await;
@@ -4886,7 +4832,6 @@ mod job_payload {
test_for_versions(VERSION_FLAGS.iter().cloned(), test).await;
}
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base", "hello"))]
async fn test_raw_flow_payload_with_restarted_from(db: Pool<Postgres>) {
initialize_tracing().await;

View File

@@ -15,7 +15,7 @@ stripe = []
agent_worker_server = []
enterprise_saml = ["dep:samael", "dep:libxml"]
benchmark = []
embedding = ["dep:tinyvector", "dep:hf-hub", "dep:tokenizers", "dep:candle-core", "dep:candle-transformers", "dep:candle-nn"]
embedding = ["dep:tinyvector", "dep:hf-hub", "dep:tokenizers", "dep:candle-core", "dep:candle-transformers", "dep:candle-nn", "dep:half"]
parquet = ["dep:datafusion", "dep:object_store", "dep:url", "windmill-common/parquet", "windmill-worker/parquet"]
prometheus = ["windmill-common/prometheus", "windmill-queue/prometheus", "dep:prometheus", "windmill-worker/prometheus"]
openidconnect = ["dep:openidconnect"]
@@ -54,6 +54,7 @@ tokio-stream.workspace = true
anyhow.workspace = true
argon2.workspace = true
axum.workspace = true
half = { workspace = true, optional = true}
futures.workspace = true
git-version.workspace = true
tower.workspace = true

View File

@@ -1,7 +1,7 @@
openapi: "3.0.3"
info:
version: 1.488.0
version: 1.485.3
title: Windmill API
contact:
@@ -13430,6 +13430,8 @@ components:
type: string
raw_code:
type: string
leaf_job:
$ref: "#/components/schemas/LeafJob"
canceled:
type: boolean
canceled_by:
@@ -16729,4 +16731,30 @@ components:
channel_name:
type: string
description: Microsoft Teams channel name
minLength: 1
minLength: 1
LeafJob:
type: object
additionalProperties:
oneOf:
- type: object
properties:
SingleJob:
type: string
format: uuid
description: UUID of an individual job
required:
- SingleJob
additionalProperties: false
- type: object
properties:
ListJob:
type: array
items:
type: string
format: uuid
description: Array of UUIDs for multiple related jobs
required:
- ListJob
additionalProperties: false
description: Map of keys to JobResult, which can be either SingleJob (individual UUID) or ListJob (array of UUIDs)

View File

@@ -253,7 +253,7 @@ impl ModelInstance {
let token_ids = Tensor::new(&tokens[..], &Device::Cpu)?.unsqueeze(0)?;
let token_type_ids = token_ids.zeros_like()?;
let embedding = self.model.forward(&token_ids, &token_type_ids, None)?;
let embedding = self.model.forward(&token_ids, &token_type_ids)?;
let embedding = (embedding.sum(1)? / embedding.dim(1)? as f64)?;
let embedding = normalize_l2(&embedding)?;

View File

@@ -12,168 +12,27 @@ use rmcp::{
service::{RequestContext, RoleServer},
Error,
};
use serde::{Deserialize, Serialize};
use serde::Serialize;
use serde_json::Value;
use sql_builder::prelude::*;
use sqlx::FromRow;
use tokio::try_join;
use tokio_util::sync::CancellationToken;
use windmill_common::db::UserDB;
use windmill_common::scripts::Schema;
use windmill_common::worker::to_raw_value;
use windmill_common::{DB, HUB_BASE_URL};
use windmill_common::scripts::{get_full_hub_script_by_path, Schema};
use windmill_common::DB;
use crate::db::ApiAuthed;
use crate::jobs::{
run_wait_result_flow_by_path_internal, run_wait_result_script_by_path_internal, RunJobQuery,
};
use crate::HTTP_CLIENT;
use windmill_common::utils::{query_elems_from_hub, StripPath};
/// Transforms the path for workspace scripts/flows.
///
/// This function takes a path and a type string.
/// It then formats the transformed path with the type prefix.
/// This is used when listing, because we can't have names with slashes.
/// Because we replace slashes with underscores, we also need to escape underscores.
///
/// # Parameters
/// - `path`: The path to transform.
/// - `type_str`: The type of the item (script or flow).
///
/// # Returns
/// - `String`: The transformed path.
fn transform_path(path: &str, type_str: &str) -> String {
// Only apply special underscore escaping for paths starting with "f/"
let transformed = if path.starts_with("f/") {
let escaped_path = path.replace('_', "__");
escaped_path.replace('/', "_")
} else {
path.replace('/', "_")
};
// first letter of type_str is used as prefix, only one letter to avoid reaching 60 char name limit
format!("{}-{}", &type_str[..1], transformed)
}
fn convert_schema_to_schema_type(schema: Option<Schema>) -> SchemaType {
let schema_obj = if let Some(ref s) = schema {
match serde_json::from_str::<SchemaType>(s.0.get()) {
Ok(val) => val,
Err(_) => SchemaType::default(),
}
} else {
SchemaType::default()
};
schema_obj
}
trait ToolableItem {
fn get_path_or_id(&self) -> String;
fn get_summary(&self) -> &str;
fn get_description(&self) -> &str;
fn get_schema(&self) -> SchemaType;
fn is_hub(&self) -> bool;
fn item_type(&self) -> &'static str;
fn get_integration_type(&self) -> Option<String>;
}
impl ToolableItem for ScriptInfo {
fn get_path_or_id(&self) -> String {
transform_path(&self.path, "script")
}
fn get_summary(&self) -> &str {
self.summary.as_deref().unwrap_or("No summary")
}
fn get_description(&self) -> &str {
self.description.as_deref().unwrap_or("No description")
}
fn get_schema(&self) -> SchemaType {
convert_schema_to_schema_type(self.schema.clone())
}
fn is_hub(&self) -> bool {
false
}
fn item_type(&self) -> &'static str {
"script"
}
fn get_integration_type(&self) -> Option<String> {
None
}
}
impl ToolableItem for FlowInfo {
fn get_path_or_id(&self) -> String {
transform_path(&self.path, "flow")
}
fn get_summary(&self) -> &str {
self.summary.as_deref().unwrap_or("No summary")
}
fn get_description(&self) -> &str {
self.description.as_deref().unwrap_or("No description")
}
fn get_schema(&self) -> SchemaType {
convert_schema_to_schema_type(self.schema.clone())
}
fn is_hub(&self) -> bool {
false
}
fn item_type(&self) -> &'static str {
"flow"
}
fn get_integration_type(&self) -> Option<String> {
None
}
}
impl ToolableItem for HubScriptInfo {
fn get_path_or_id(&self) -> String {
let id = self.version_id;
let summary = self.summary.as_deref().unwrap_or("No summary");
format!("hs-{}-{}", id, summary.replace(" ", "_"))
}
fn get_summary(&self) -> &str {
self.summary.as_deref().unwrap_or("No summary")
}
fn get_description(&self) -> &str {
self.description.as_deref().unwrap_or("No description")
}
fn get_schema(&self) -> SchemaType {
match serde_json::from_value::<SchemaType>(self.schema.clone().unwrap_or_default()) {
Ok(schema_type) => schema_type,
Err(_) => SchemaType::default(),
}
}
fn is_hub(&self) -> bool {
true
}
fn item_type(&self) -> &'static str {
"script"
}
fn get_integration_type(&self) -> Option<String> {
self.app.clone()
}
}
use windmill_common::utils::StripPath;
#[derive(Clone)]
pub struct Runner {}
#[derive(Serialize, Deserialize, Debug)]
struct HubResponse {
asks: Vec<HubScriptInfo>,
}
#[derive(Serialize, Deserialize, Debug)]
struct HubScriptInfo {
version_id: u64,
summary: Option<String>,
description: Option<String>,
schema: Option<Value>,
app: Option<String>,
}
#[derive(Serialize, FromRow, Deserialize, Debug, Clone)]
#[derive(serde::Deserialize, serde::Serialize)]
struct SchemaType {
r#type: String,
properties: std::collections::HashMap<String, serde_json::Value>,
@@ -235,7 +94,7 @@ impl Runner {
authed: &ApiAuthed,
workspace_id: &str,
item_type: &str,
) -> Result<Option<Schema>, Error> {
) -> Result<ItemSchema, Error> {
let mut sqlb = SqlBuilder::select_from(&format!("{} as o", item_type));
sqlb.fields(&["o.schema"]);
sqlb.and_where("o.path = ?".bind(&path));
@@ -251,7 +110,7 @@ impl Runner {
.begin(authed)
.await
.map_err(|_e| Error::internal_error("failed to begin transaction", None))?;
let item = sqlx::query_as::<_, ItemSchema>(&sql)
let rows = sqlx::query_as::<_, ItemSchema>(&sql)
.fetch_one(&mut *tx)
.await
.map_err(|_e| {
@@ -261,29 +120,27 @@ impl Runner {
tx.commit()
.await
.map_err(|_e| Error::internal_error("failed to commit transaction", None))?;
Ok(item.schema)
Ok(rows)
}
/// Reverses the transformation of a path.
///
/// This function takes a transformed path and reverses the transformation applied by `transform_path`.
/// It checks if the path starts with "h" (indicating a Hub script) and removes the prefix if present.
/// It then determines the type of the item (script or flow) based on the prefix.
/// This is used in call_tool to get the original path, and the type of the item.
///
/// # Parameters
/// - `transformed_path`: The transformed path to reverse.
///
/// # Returns
/// - `Result<(&str, String, bool), String>`: A tuple containing the original path, the type of the item, and a boolean indicating if it's a Hub script.
/// - `Err(String)`: If the path is invalid.
fn reverse_transform(transformed_path: &str) -> Result<(&str, String, bool), String> {
let is_hub = transformed_path.starts_with("h");
let transformed_path = if is_hub {
transformed_path[1..].to_string()
fn transform_path(path: &str, type_str: &str) -> Result<String, String> {
if type_str != "script" && type_str != "flow" {
return Err(format!("Invalid type: {}", type_str));
}
// Only apply special underscore escaping for paths starting with "f/"
let transformed = if path.starts_with("f/") {
let escaped_path = path.replace('_', "__");
escaped_path.replace('/', "_")
} else {
transformed_path.to_string()
path.replace('/', "_")
};
// first letter of type_str is used as prefix, only one letter to avoid reaching 60 char name limit
Ok(format!("{}-{}", &type_str[..1], transformed))
}
fn reverse_transform(transformed_path: &str) -> Result<(&str, String), String> {
let type_str = if transformed_path.starts_with("s-") {
"script"
} else if transformed_path.starts_with("f-") {
@@ -300,10 +157,7 @@ impl Runner {
// Check if this path was previously transformed with special underscore handling
let is_special_path = mangled_path.starts_with("f_");
let original_path = if is_hub {
let parts = mangled_path.split("-").collect::<Vec<&str>>();
parts[0].to_string()
} else if is_special_path {
let original_path = if is_special_path {
const TEMP_PLACEHOLDER: &str = "@@UNDERSCORE@@";
let path_with_placeholder = mangled_path.replace("__", TEMP_PLACEHOLDER);
let path_with_slashes = path_with_placeholder.replace('_', "/");
@@ -312,7 +166,7 @@ impl Runner {
mangled_path.replacen('_', "/", 2)
};
Ok((type_str, original_path, is_hub))
Ok((type_str, original_path))
}
async fn inner_get_resources_types(
@@ -426,52 +280,6 @@ impl Runner {
Ok(rows)
}
async fn inner_get_scripts_from_hub(
db: &DB,
scope_integrations: Option<&str>,
) -> Result<Vec<HubScriptInfo>, Error> {
let query_params = Some(vec![
("limit", "100".to_string()),
("with_schema", "true".to_string()),
("apps", scope_integrations.unwrap_or("").to_string()),
]);
let url = format!("{}/scripts/top", *HUB_BASE_URL.read().await);
let (_status_code, _headers, response) =
query_elems_from_hub(&HTTP_CLIENT, &url, query_params, &db)
.await
.map_err(|e| {
tracing::error!("Failed to get items from hub: {}", e);
Error::internal_error(format!("Failed to get items from hub: {}", e), None)
})?;
let body_bytes = to_bytes(response, usize::MAX).await.map_err(|e| {
tracing::error!("Failed to read response body: {}", e);
Error::internal_error(format!("Failed to read response body: {}", e), None)
})?;
let body_str = String::from_utf8(body_bytes.to_vec()).map_err(|e| {
tracing::error!("Failed to decode response body: {}", e);
Error::internal_error(format!("Failed to decode response body: {}", e), None)
})?;
let hub_response: HubResponse = serde_json::from_str(&body_str).map_err(|e| {
tracing::error!("Failed to parse hub response: {}", e);
Error::internal_error(format!("Failed to parse hub response: {}", e), None)
})?;
Ok(hub_response.asks)
}
/// Transforms a value if it's an object.
///
/// This function takes a key and a value, and a schema object.
/// If the value is a string that starts with "$res:", it returns the value as is.
/// Otherwise, it checks if the key is defined in the schema and if it's an object type.
/// If it is, it transforms the value to a string. This is because some clients do not support object types.
/// # Parameters
/// - `key`: The key of the value to transform.
/// - `value`: The value to transform.
/// - `schema_obj`: The schema object.
///
/// # Returns
/// - `Value`: The transformed value.
fn transform_value_if_object(
key: &str,
value: &Value,
@@ -507,16 +315,6 @@ impl Runner {
value.clone()
}
/// Reverses the transformation of a key.
///
/// This function takes a transformed key and a schema object.
/// It then reverses the transformation applied by `apply_key_transformation`. This can be subject to collisions, but it's unlikely and is ok for our use case.
/// # Parameters
/// - `transformed_key`: The transformed key to reverse.
/// - `schema_obj`: The schema object.
///
/// # Returns
/// - `String`: The original key.
fn reverse_transform_key(transformed_key: &str, schema_obj: &Option<SchemaType>) -> String {
let schema_obj = match schema_obj {
Some(s) => s,
@@ -540,16 +338,6 @@ impl Runner {
transformed_key.to_string()
}
/// Applies a key transformation to a key.
///
/// This function takes a key and replaces spaces with underscores.
/// It also removes any characters that are not alphanumeric or underscores.
/// This is used when listing, because we can't have names with spaces or special characters in the schema properties.
/// # Parameters
/// - `key`: The key to transform.
///
/// # Returns
/// - `String`: The transformed key.
fn apply_key_transformation(key: &str) -> String {
key.replace(' ', "_")
.chars()
@@ -557,32 +345,18 @@ impl Runner {
.collect::<String>()
}
/// Transforms the schema for resources.
///
/// This function takes a schema and a database connection, and attempts to transform the schema for resources.
/// It replaces invalid characters in property keys with underscores and converts object properties to strings.
/// It also fetches resource type information and adds it to the description of resource properties.
///
/// # Parameters
/// - `schema`: The schema to transform.
/// - `user_db`: The database connection.
/// - `authed`: The authenticated user.
/// - `w_id`: The workspace ID.
/// - `resources_cache`: A mutable reference to the resources cache.
/// - `resources_types`: A reference to the resource types.
///
/// # Returns
/// - `Result<SchemaType, Error>`: The transformed schema.
/// - `Err(Error)`: If the transformation fails.
async fn transform_schema_for_resources(
schema: &SchemaType,
schema: &Schema,
user_db: &UserDB,
authed: &ApiAuthed,
w_id: &str,
resources_cache: &mut HashMap<String, Vec<ResourceInfo>>,
resources_types: &Vec<ResourceType>,
) -> Result<SchemaType, Error> {
let mut schema_obj: SchemaType = schema.clone();
let mut schema_obj: SchemaType = match serde_json::from_str(schema.0.get()) {
Ok(val) => val,
Err(_) => SchemaType::default(),
};
// replace invalid char in property key with underscore
let replacements: Vec<(String, String, serde_json::Value)> = schema_obj
@@ -715,133 +489,9 @@ impl Runner {
Ok(schema_obj)
}
/// Fetches the schema for a Hub script.
///
/// This function takes a script path and a database connection, and attempts to fetch the schema for the script.
/// It strips the path to remove any leading slashes, and then attempts to retrieve the full script using `get_full_hub_script_by_path`.
/// If successful, it converts the schema string to a `Schema` object.
/// If the schema cannot be converted, it logs a warning and returns `None`.
///
/// # Parameters
/// - `path`: The path of the script to fetch the schema for.
/// - `db`: The database connection.
///
/// # Returns
/// - `Ok(Option<Schema>)`: The schema if found, otherwise `None`.
/// - `Err(Error)`: If the request fails.
async fn get_hub_script_schema(path: &str, db: &DB) -> Result<Option<Schema>, Error> {
let strip_path = StripPath(path.to_string());
let res = get_full_hub_script_by_path(strip_path, &HTTP_CLIENT, Some(db))
.await
.map_err(|e| {
tracing::error!("Failed to get hub script: {}", e);
Error::internal_error(format!("Failed to get hub script: {}", e), None)
})?;
match serde_json::from_str::<Schema>(res.schema.get()) {
Ok(schema) => Ok(Some(schema)),
Err(e) => {
tracing::warn!("Failed to convert schema: {}", e);
Ok(None)
}
}
}
/// Creates a `Tool` from a `ToolableItem`.
///
/// This function takes an item that implements the `ToolableItem` trait and converts it into an RMCP `Tool`.
/// It handles both workspace scripts/flows and Hub scripts differently, depending on the item type.
///
/// # Parameters
/// - `item`: The item to convert to a `Tool`.
/// - `user_db`: The database connection.
/// - `authed`: The authenticated user.
/// - `workspace_id`: The workspace ID.
/// - `resources_cache`: A mutable reference to the resources cache.
/// - `resources_types`: A reference to the resource types.
///
/// # Returns
/// - `Ok(Tool)`: The created `Tool`.
async fn create_tool_from_item<T: ToolableItem>(
item: &T,
user_db: &UserDB,
authed: &ApiAuthed,
workspace_id: &str,
resources_cache: &mut HashMap<String, Vec<ResourceInfo>>,
resources_types: &Vec<ResourceType>,
) -> Result<Tool, Error> {
let is_hub = item.is_hub();
let path = item.get_path_or_id();
let item_type = item.item_type();
let description = format!(
"This is a {} named `{}` with the following description: `{}`.{}",
item_type,
item.get_summary(),
item.get_description(),
if is_hub {
format!(
" It is a tool used for the following app: {}",
item.get_integration_type()
.unwrap_or("No integration type".to_string())
)
} else {
"".to_string()
}
);
let schema_obj = Runner::transform_schema_for_resources(
&item.get_schema(),
user_db,
authed,
&workspace_id,
resources_cache,
&resources_types,
)
.await?;
let input_schema_map = match serde_json::to_value(schema_obj) {
Ok(Value::Object(map)) => map,
Ok(_) => {
tracing::warn!("Schema object for tool '{}' did not serialize to a JSON object, using empty schema.", path);
serde_json::Map::new()
}
Err(e) => {
tracing::error!(
"Failed to serialize schema object for tool '{}': {}. Using empty schema.",
path,
e
);
serde_json::Map::new()
}
};
Ok(Tool {
name: Cow::Owned(path),
description: Some(Cow::Owned(description)),
input_schema: Arc::new(input_schema_map),
annotations: None,
})
}
}
impl ServerHandler for Runner {
/// Handles the `CallTool` request from the MCP client.
///
/// This involves:
/// 1. Parsing arguments and extracting context (DB, Auth).
/// 2. Reversing the tool name (`request.name`) to get the original path and type using `reverse_transform`.
/// 3. Handling Hub scripts: If identified as a Hub script, searches the Hub for the actual script ID.
/// 4. Fetching the schema for the item (needed for argument transformation).
/// 5. Transforming incoming arguments:
/// - Reversing key transformations (e.g., `user_input` back to `user input`).
/// - Parsing stringified JSON objects back into JSON values based on schema type.
/// 6. Executing the corresponding script or flow using internal Windmill runners.
/// 7. Formatting the execution result into an RMCP `CallToolResult`.
///
/// # Parameters
/// - `request`: The `CallToolRequestParam` containing the tool name and arguments.
/// - `context`: The `RequestContext` providing access to workspace ID, DB connections, auth info.
///
/// # Returns
/// - `Ok(CallToolResult)`: On successful execution, containing the output.
/// - `Err(Error)`: If any step fails (parsing, DB access, execution, reversing transform, hub search).
async fn call_tool(
&self,
request: CallToolRequestParam,
@@ -870,17 +520,14 @@ impl ServerHandler for Runner {
.ok_or_else(|| Error::internal_error("UserDB not found", None))?;
let args = parse_args(request.arguments)?;
let (tool_type, path, is_hub) =
Runner::reverse_transform(&request.name).unwrap_or_default();
let (tool_type, path) = Runner::reverse_transform(&request.name).unwrap_or_default();
let item_schema = if is_hub {
Runner::get_hub_script_schema(&format!("hub/{}", path), db).await?
} else {
let item_info =
Runner::get_item_schema(&path, user_db, authed, &context.workspace_id, &tool_type)
.await?
};
.await?;
let schema_obj = if let Some(ref s) = item_schema {
let schema = item_info.schema;
let schema_obj = if let Some(ref s) = schema {
match serde_json::from_str::<SchemaType>(s.0.get()) {
Ok(val) => Some(val),
Err(e) => {
@@ -895,7 +542,7 @@ impl ServerHandler for Runner {
let push_args = if let Value::Object(map) = args.clone() {
let mut args_hash = HashMap::new();
for (k, v) in map {
// need to transform back the key without invalid characters to the original key
// need to transform back the key to the original key
let original_key = Runner::reverse_transform_key(&k, &schema_obj);
// object properties are transformed to string because some client does not support object, might change in the future
@@ -908,11 +555,7 @@ impl ServerHandler for Runner {
};
let w_id = context.workspace_id.clone();
let script_or_flow_path = if is_hub {
StripPath(format!("hub/{}", path))
} else {
StripPath(path)
};
let script_or_flow_path = StripPath(path);
let run_query = RunJobQuery::default();
let result = if tool_type == "script" {
@@ -960,31 +603,12 @@ impl ServerHandler for Runner {
}
}
/// Fetches available tools (scripts, flows, hub scripts) based on the user's scope.
///
/// - Determines scope (all, favorites, hub-specific) from auth token.
/// - Fetches relevant items (workspace scripts/flows, hub scripts) concurrently.
/// - Fetches resource type information needed for schema enrichment.
/// - Transforms each item into an RMCP `Tool` definition, including schema adjustments
/// (like resource description enrichment and object->string conversion).
///
/// # Parameters
/// - `_request`: Optional pagination parameters (currently ignored).
/// - `_context`: The `RequestContext` providing workspace ID, DB, auth.
///
/// # Returns
/// - `Ok(ListToolsResult)`: A list of `Tool` definitions. Pagination is not yet implemented.
/// - `Err(Error)`: If fetching data from DB or Hub fails.
async fn list_tools(
&self,
_request: Option<PaginatedRequestParam>,
mut _context: RequestContext<RoleServer>,
) -> Result<ListToolsResult, Error> {
let workspace_id = _context.workspace_id.clone();
let db = _context
.req_extensions
.get::<DB>()
.ok_or_else(|| Error::internal_error("DB not found", None))?;
let user_db = _context
.req_extensions
.get::<UserDB>()
@@ -993,27 +617,11 @@ impl ServerHandler for Runner {
.req_extensions
.get::<ApiAuthed>()
.ok_or_else(|| Error::internal_error("ApiAuthed not found", None))?;
let owned_scope = authed.scopes.as_ref().and_then(|scopes| {
scopes
.iter()
.find(|scope| scope.starts_with("mcp:") && !scope.contains("hub"))
});
let hub_scope = authed
let scope = authed
.scopes
.as_ref()
.and_then(|scopes| scopes.iter().find(|scope| scope.starts_with("mcp:hub")));
let scope_type = owned_scope.map_or("all", |scope| {
let parts = scope.split(":").collect::<Vec<&str>>();
parts[1]
});
let scope_integrations = hub_scope.and_then(|scope| {
let parts = scope.split(":").collect::<Vec<&str>>();
if parts.len() == 3 {
Some(parts[2])
} else {
None
}
});
.and_then(|scopes| scopes.iter().find(|scope| scope.starts_with("mcp:")));
let scope_type = scope.map_or("all", |scope| scope.split(":").last().unwrap_or("all"));
let scripts_fn = Runner::inner_get_items::<ScriptInfo>(
user_db,
@@ -1025,62 +633,84 @@ impl ServerHandler for Runner {
let flows_fn =
Runner::inner_get_items::<FlowInfo>(user_db, authed, &workspace_id, scope_type, "flow");
let resources_types_fn = Runner::inner_get_resources_types(user_db, authed, &workspace_id);
let hub_scripts_fn = Runner::inner_get_scripts_from_hub(db, scope_integrations.as_deref());
let (scripts, flows, resources_types, hub_scripts) = if scope_integrations.is_some() {
let (scripts, flows, resources_types, hub_scripts) =
try_join!(scripts_fn, flows_fn, resources_types_fn, hub_scripts_fn)?;
(scripts, flows, resources_types, hub_scripts)
} else {
let (scripts, flows, resources_types) =
try_join!(scripts_fn, flows_fn, resources_types_fn)?;
(scripts, flows, resources_types, vec![])
};
let (scripts, flows, resources_types) =
try_join!(scripts_fn, flows_fn, resources_types_fn)?;
let mut resources_cache: HashMap<String, Vec<ResourceInfo>> = HashMap::new();
let mut tools: Vec<Tool> = Vec::new();
let mut script_tools: Vec<Tool> = Vec::with_capacity(scripts.len());
for script in scripts {
tools.push(
Runner::create_tool_from_item(
&script,
let name = Runner::transform_path(&script.path, "script").unwrap_or_default();
let description = format!(
"This is a script named `{}` with the following description: `{}`.",
script.summary.as_deref().unwrap_or("No summary"),
script.description.as_deref().unwrap_or("No description")
);
let schema_obj = if let Some(schema) = script.schema {
Runner::transform_schema_for_resources(
&schema,
user_db,
authed,
&workspace_id,
&mut resources_cache,
&resources_types,
)
.await?,
);
.await?
} else {
SchemaType::default()
};
script_tools.push(Tool {
name: Cow::Owned(name),
description: Some(Cow::Owned(description)),
input_schema: {
let value = serde_json::to_value(schema_obj).unwrap_or_default();
if let serde_json::Value::Object(map) = value {
Arc::new(map)
} else {
Arc::new(serde_json::Map::new())
}
},
annotations: None,
});
}
let mut flow_tools: Vec<Tool> = Vec::with_capacity(flows.len());
for flow in flows {
tools.push(
Runner::create_tool_from_item(
&flow,
let name = Runner::transform_path(&flow.path, "flow").unwrap_or_default();
let description = format!(
"This is a flow named `{}` with the following description: `{}`.",
flow.summary.as_deref().unwrap_or("No summary"),
flow.description.as_deref().unwrap_or("No description")
);
let schema_obj = if let Some(schema) = flow.schema {
Runner::transform_schema_for_resources(
&schema,
user_db,
authed,
&workspace_id,
&mut resources_cache,
&resources_types,
)
.await?,
);
}
for hub_script in hub_scripts {
tools.push(
Runner::create_tool_from_item(
&hub_script,
user_db,
authed,
&workspace_id,
&mut resources_cache,
&resources_types,
)
.await?,
);
.await?
} else {
SchemaType::default()
};
flow_tools.push(Tool {
name: Cow::Owned(name),
description: Some(Cow::Owned(description)),
input_schema: {
let value = serde_json::to_value(schema_obj).unwrap_or_default();
if let serde_json::Value::Object(map) = value {
Arc::new(map)
} else {
Arc::new(serde_json::Map::new())
}
},
annotations: None,
});
}
let tools = [script_tools, flow_tools].concat();
Ok(ListToolsResult { tools, next_cursor: None })
}

View File

@@ -265,12 +265,9 @@ async fn list_scripts(
if lq.show_archived.unwrap_or(false) {
sqlb.and_where_eq(
"o.ctid",
"(SELECT ctid FROM script
WHERE path = o.path
AND workspace_id = ?
ORDER BY created_at DESC
LIMIT 1)"
"o.created_at",
"(select max(created_at) from script where o.path = path
AND workspace_id = ?)"
.bind(&w_id),
);
sqlb.and_where_eq("archived", true);
@@ -653,13 +650,8 @@ async fn create_script_internal<'c>(
) {
Some(String::new())
} else {
ns.lock.as_ref().and_then(|e| {
if e.is_empty() {
None
} else {
Some(e.to_string())
}
})
ns.lock
.and_then(|e| if e.is_empty() { None } else { Some(e) })
};
let needs_lock_gen = lock.is_none() && codebase.is_none();
@@ -685,32 +677,12 @@ async fn create_script_internal<'c>(
let (no_main_func, has_preprocessor) = match lang {
ScriptLang::Bun | ScriptLang::Bunnative | ScriptLang::Deno | ScriptLang::Nativets => {
let args = windmill_parser_ts::parse_deno_signature(&ns.content, true, true, None);
match args {
Ok(args) => (args.no_main_func, args.has_preprocessor),
Err(e) => {
tracing::warn!(
"Error parsing deno signature when deploying script {}: {:?}",
ns.path,
e
);
(None, None)
}
}
let args = windmill_parser_ts::parse_deno_signature(&ns.content, true, true, None)?;
(args.no_main_func, args.has_preprocessor)
}
ScriptLang::Python3 => {
let args = windmill_parser_py::parse_python_signature(&ns.content, None, true);
match args {
Ok(args) => (args.no_main_func, args.has_preprocessor),
Err(e) => {
tracing::warn!(
"Error parsing python signature when deploying script {}: {:?}",
ns.path,
e
);
(None, None)
}
}
let args = windmill_parser_py::parse_python_signature(&ns.content, None, true)?;
(args.no_main_func, args.has_preprocessor)
}
_ => (ns.no_main_func, ns.has_preprocessor),
};
@@ -929,7 +901,6 @@ async fn create_script_internal<'c>(
let permissioned_as2 = permissioned_as.clone();
let script_path2 = script_path.clone();
let parent_path = p_path_opt.clone();
let lock = ns.lock.clone();
let deployment_message = ns.deployment_message.clone();
let content = ns.content.clone();
let language = ns.language.clone();
@@ -949,7 +920,6 @@ async fn create_script_internal<'c>(
&authed2.email,
&authed2.username,
&permissioned_as2,
lock,
)
.await
{
@@ -1012,7 +982,7 @@ async fn get_script_by_path(
AND favorite.usr = $3
WHERE s.path = $1
AND s.workspace_id = $2
ORDER BY s.created_at DESC LIMIT 1",
AND s.created_at = (SELECT max(created_at) FROM script WHERE path = $1 AND workspace_id = $2)",
)
.bind(path)
.bind(w_id)
@@ -1021,7 +991,9 @@ async fn get_script_by_path(
.await?
} else {
sqlx::query_as::<_, ScriptWithStarred>(
"SELECT *, NULL as starred FROM script WHERE path = $1 AND workspace_id = $2 ORDER BY created_at DESC LIMIT 1",
"SELECT *, NULL as starred FROM script WHERE path = $1 AND workspace_id = $2 \
AND created_at = (SELECT max(created_at) FROM script WHERE path = $1 AND \
workspace_id = $2)",
)
.bind(path)
.bind(w_id)
@@ -1061,8 +1033,9 @@ async fn get_script_by_path_w_draft(
let script_o = sqlx::query_as::<_, ScriptWDraft>(
"SELECT hash, script.path, summary, description, content, language, kind, tag, schema, draft_only, envs, concurrent_limit, concurrency_time_window_s, cache_ttl, ws_error_handler_muted, draft.value as draft, dedicated_worker, priority, restart_unless_cancelled, delete_after_use, timeout, concurrency_key, visible_to_runner_only, no_main_func, has_preprocessor, on_behalf_of_email FROM script LEFT JOIN draft ON
script.path = draft.path AND script.workspace_id = draft.workspace_id AND draft.typ = 'script'
WHERE script.path = $1 AND script.workspace_id = $2
ORDER BY script.created_at DESC LIMIT 1",
WHERE script.path = $1 AND script.workspace_id = $2 \
AND script.created_at = (SELECT max(created_at) FROM script WHERE path = $1 AND \
workspace_id = $2)",
)
.bind(path)
.bind(w_id)
@@ -1084,7 +1057,7 @@ async fn get_script_history(
"SELECT s.hash as hash, dm.deployment_msg as deployment_msg
FROM script s LEFT JOIN deployment_metadata dm ON s.hash = dm.script_hash
WHERE s.workspace_id = $1 AND s.path = $2
ORDER by s.created_at DESC",
ORDER by created_at DESC",
w_id,
path.to_path(),
)
@@ -1112,7 +1085,7 @@ async fn get_latest_version(
"SELECT s.hash as hash, dm.deployment_msg as deployment_msg
FROM script s LEFT JOIN deployment_metadata dm ON s.hash = dm.script_hash
WHERE s.workspace_id = $1 AND s.path = $2
ORDER by s.created_at DESC LIMIT 1",
ORDER by created_at DESC",
w_id,
path.to_path(),
)
@@ -1208,15 +1181,7 @@ async fn toggle_workspace_error_handler(
match error_handler_maybe {
Some(_) => {
sqlx::query_scalar!(
"UPDATE script
SET ws_error_handler_muted = $3
WHERE ctid = (
SELECT ctid FROM script
WHERE path = $1 AND workspace_id = $2
ORDER BY created_at DESC
LIMIT 1
)
",
"UPDATE script SET ws_error_handler_muted = $3 WHERE workspace_id = $2 AND path = $1 AND created_at = (SELECT max(created_at) FROM script WHERE path = $1 AND workspace_id = $2)",
path.to_path(),
w_id,
req.muted,
@@ -1237,7 +1202,6 @@ async fn toggle_workspace_error_handler(
async fn get_tokened_raw_script_by_path(
Extension(user_db): Extension<UserDB>,
Extension(db): Extension<DB>,
Path((w_id, token, path)): Path<(String, String, StripPath)>,
Extension(cache): Extension<Arc<AuthCache>>,
) -> Result<String> {
@@ -1245,13 +1209,7 @@ async fn get_tokened_raw_script_by_path(
.get_authed(Some(w_id.clone()), &token)
.await
.ok_or_else(|| Error::NotAuthorized("Invalid token".to_string()))?;
return raw_script_by_path(
authed,
Extension(user_db),
Extension(db),
Path((w_id, path)),
)
.await;
return raw_script_by_path(authed, Extension(user_db), Path((w_id, path))).await;
}
async fn get_empty_ts_script_by_path() -> String {
@@ -1261,25 +1219,22 @@ async fn get_empty_ts_script_by_path() -> String {
async fn raw_script_by_path(
authed: ApiAuthed,
Extension(user_db): Extension<UserDB>,
Extension(db): Extension<DB>,
Path((w_id, path)): Path<(String, StripPath)>,
) -> Result<String> {
raw_script_by_path_internal(path, user_db, db, authed, w_id, false).await
raw_script_by_path_internal(path, user_db, authed, w_id, false).await
}
async fn raw_script_by_path_unpinned(
authed: ApiAuthed,
Extension(user_db): Extension<UserDB>,
Extension(db): Extension<DB>,
Path((w_id, path)): Path<(String, StripPath)>,
) -> Result<String> {
raw_script_by_path_internal(path, user_db, db, authed, w_id, true).await
raw_script_by_path_internal(path, user_db, authed, w_id, true).await
}
async fn raw_script_by_path_internal(
path: StripPath,
user_db: UserDB,
db: DB,
authed: ApiAuthed,
w_id: String,
unpin: bool,
@@ -1305,7 +1260,10 @@ async fn raw_script_by_path_internal(
let mut tx = user_db.begin(&authed).await?;
let content_o = sqlx::query_scalar!(
"SELECT content FROM script WHERE path = $1 AND workspace_id = $2 AND archived = false ORDER BY created_at DESC LIMIT 1",
"SELECT content FROM script WHERE path = $1 AND workspace_id = $2 \
AND
created_at = (SELECT max(created_at) FROM script WHERE path = $1 AND archived = false AND \
workspace_id = $2)",
path,
w_id
)
@@ -1313,22 +1271,6 @@ async fn raw_script_by_path_internal(
.await?;
tx.commit().await?;
if content_o.is_none() {
let exists = sqlx::query_scalar!(
"SELECT EXISTS(SELECT 1 FROM script WHERE path = $1 AND workspace_id = $2 AND archived = false ORDER BY created_at DESC LIMIT 1)",
path,
w_id
)
.fetch_one(&db)
.await?;
if exists.unwrap_or(false) {
return Err(Error::NotFound(format!(
"Script {path} not visible to {} but exists",
authed.username
)));
}
}
let content = not_found_if_none(content_o, "Script", path)?;
if unpin {
@@ -1345,7 +1287,8 @@ async fn exists_script_by_path(
let path = path.to_path();
let exists = sqlx::query_scalar!(
"SELECT EXISTS(SELECT 1 FROM script WHERE path = $1 AND workspace_id = $2 ORDER BY created_at DESC LIMIT 1)",
"SELECT EXISTS(SELECT 1 FROM script WHERE path = $1 AND workspace_id = $2 AND
created_at = (SELECT max(created_at) FROM script WHERE path = $1 AND workspace_id = $2))",
path,
w_id
)
@@ -1462,7 +1405,9 @@ pub async fn require_is_writer(authed: &ApiAuthed, path: &str, w_id: &str, db: D
path,
w_id,
db,
"SELECT extra_perms FROM script WHERE path = $1 AND workspace_id = $2 ORDER BY created_at DESC LIMIT 1",
"SELECT extra_perms FROM script WHERE path = $1 AND workspace_id = $2 \
AND created_at = (SELECT max(created_at) FROM script WHERE path = $1 AND \
workspace_id = $2)",
"script",
)
.await;

View File

@@ -33,8 +33,6 @@ impl<B> OnResponse<B> for MyOnResponse {
let status = response.status().as_u16();
if response.status().is_success() || response.status().is_redirection() {
tracing::info!(latency = latency, status = status, "response")
} else if response.status().as_u16() == 404 {
tracing::warn!(latency = latency, status = status, "response")
} else {
tracing::error!(latency = latency, status = status, "response")
}

View File

@@ -76,8 +76,6 @@ quick_cache.workspace = true
pin-project-lite.workspace = true
futures.workspace = true
tempfile.workspace = true
systemstat.workspace = true
size.workspace = true
opentelemetry-semantic-conventions = { workspace = true, optional = true }
opentelemetry-otlp = { workspace = true, optional = true }

View File

@@ -98,9 +98,3 @@ pub async fn worker_groups_alerts(_db: &DB) {}
#[cfg(feature = "enterprise")]
pub async fn jobs_waiting_alerts(_db: &DB) {}
#[cfg(feature = "enterprise")]
pub async fn low_disk_alerts(_db: &DB, _server_mode: bool, _worker_mode: bool, _workers: Vec<String>) {
// Implementation is not open source
}

View File

@@ -22,8 +22,7 @@ use crate::{
error::Error,
more_serde::{default_empty_string, default_id, default_null, default_true, is_default},
scripts::{Schema, ScriptHash, ScriptLang},
worker::{to_raw_value, Connection},
DB,
worker::{to_raw_value, Connection}, DB,
};
#[derive(Serialize, Deserialize, sqlx::FromRow)]

View File

@@ -37,7 +37,6 @@ pub const HUB_BASE_URL_SETTING: &str = "hub_base_url";
pub const HUB_ACCESSIBLE_URL_SETTING: &str = "hub_accessible_url";
pub const CRITICAL_ERROR_CHANNELS_SETTING: &str = "critical_error_channels";
pub const CRITICAL_ALERT_MUTE_UI_SETTING: &str = "critical_alert_mute_ui";
pub const CRITICAL_ALERTS_ON_DB_OVERSIZE_SETTING: &str = "critical_alerts_on_db_oversize";
pub const DEV_INSTANCE_SETTING: &str = "dev_instance";
pub const JWT_SECRET_SETTING: &str = "jwt_secret";
pub const EMAIL_DOMAIN_SETTING: &str = "email_domain";

View File

@@ -115,7 +115,6 @@ lazy_static::lazy_static! {
pub static ref CRITICAL_ERROR_CHANNELS: Arc<RwLock<Vec<CriticalErrorChannel>>> = Arc::new(RwLock::new(vec![]));
pub static ref CRITICAL_ALERTS_ON_DB_OVERSIZE: Arc<RwLock<Option<f32>>> = Arc::new(RwLock::new(None));
pub static ref JOB_RETENTION_SECS: Arc<RwLock<i64>> = Arc::new(RwLock::new(0));

View File

@@ -265,7 +265,7 @@ pub struct ScriptHistoryUpdate {
pub deployment_msg: Option<String>,
}
#[derive(Serialize, Deserialize, Debug, sqlx::Type, Clone)]
#[derive(Serialize, Deserialize, Debug, sqlx::Type)]
#[sqlx(transparent)]
#[serde(transparent)]
pub struct Schema(pub sqlx::types::Json<Box<serde_json::value::RawValue>>);

View File

@@ -2653,38 +2653,13 @@ fn interpolate_args(x: String, args: &PushArgs, workspace_id: &str) -> String {
let mut interpolated = workspaced.clone();
for cap in RE_ARG_TAG.captures_iter(&workspaced) {
let arg_name = cap.get(1).unwrap().as_str();
let arg_value = if arg_name.contains('.') {
let parts: Vec<&str> = arg_name.split('.').collect();
let root = parts[0];
let mut value = args
.args
.get(root)
.or(args.extra.as_ref().and_then(|x| x.get(root)))
.map(|x| x.get())
.unwrap_or_default().to_string();
for part in parts.iter().skip(1) {
if let Ok(obj) = serde_json::from_str::<serde_json::Value>(&value) {
value = obj.get(part)
.and_then(|v| Some(v.to_string()))
.unwrap_or_default()
.as_str().to_string();
} else {
value = "".to_string(); // Invalid JSON or missing field
break;
}
}
value.trim_matches('"').to_string()
} else {
args.args
.get(arg_name)
.or(args.extra.as_ref().and_then(|x| x.get(arg_name)))
.map(|x| x.get())
.unwrap_or_default()
.trim_matches('"')
.to_string()
};
tracing::error!("arg_value: {}", arg_value);
let arg_value = args
.args
.get(arg_name)
.or(args.extra.as_ref().and_then(|x| x.get(arg_name)))
.map(|x| x.get())
.unwrap_or_default()
.trim_matches('"');
interpolated =
interpolated.replace(format!("$args[{}]", arg_name).as_str(), &arg_value);
}
@@ -3252,7 +3227,7 @@ pub fn empty_result() -> Box<RawValue> {
// }
lazy_static::lazy_static! {
pub static ref RE_ARG_TAG: Regex = Regex::new(r#"\$args\[((?:\w+\.)*\w+)\]"#).unwrap();
pub static ref RE_ARG_TAG: Regex = Regex::new(r#"\$args\[(\w+)\]"#).unwrap();
}
// #[instrument(level = "trace", skip_all)]

View File

@@ -858,16 +858,18 @@ pub async fn handle_ansible_job(
let mut nsjail_extra_mounts = vec![];
if let Some(r) = reqs.as_ref() {
nsjail_extra_mounts = create_file_resources(
&job.id,
&job.workspace_id,
job_dir,
interpolated_args.as_ref(),
&r,
&client,
conn,
)
.await?;
if let Some(db) = conn.as_sql() {
nsjail_extra_mounts = create_file_resources(
&job.id,
&job.workspace_id,
job_dir,
interpolated_args.as_ref(),
&r,
&client,
db,
)
.await?;
}
for repo in &r.git_repos {
append_logs(
@@ -933,7 +935,8 @@ pub async fn handle_ansible_job(
if let Some(collections) = r.roles_and_collections.as_ref() {
let empty = String::new();
let (lockfile, logs) = req_lockfiles
let (lockfile, logs) =
req_lockfiles
.as_ref()
.map(|r| (&r.collections_and_roles, &r.collections_and_roles_logs))
.unwrap_or((collections, &empty));
@@ -1159,7 +1162,7 @@ async fn create_file_resources(
args: Option<&HashMap<String, Box<RawValue>>>,
r: &AnsibleRequirements,
client: &crate::AuthedClient,
conn: &Connection,
db: &sqlx::Pool<sqlx::Postgres>,
) -> error::Result<Vec<String>> {
let mut logs = String::new();
let mut nsjail_mounts: Vec<String> = vec![];
@@ -1229,7 +1232,7 @@ async fn create_file_resources(
file_res.target_path, file_res.resource_path
));
}
append_logs(job_id, w_id, logs, conn).await;
append_logs(job_id, w_id, logs, &Connection::Sql(db.clone())).await;
Ok(nsjail_mounts)
}

View File

@@ -42,7 +42,7 @@ use windmill_common::{
error::{self, Result},
get_latest_hash_for_path,
scripts::ScriptLang,
worker::{exists_in_cache, save_cache, to_raw_value, write_file, Connection, DISABLE_BUNDLING},
worker::{exists_in_cache, save_cache, write_file, Connection, DISABLE_BUNDLING},
DB,
};
@@ -111,7 +111,7 @@ pub async fn gen_bun_lockfile(
let mut empty_deps = false;
if let Some(raw_deps) = raw_deps.as_ref() {
if let Some(raw_deps) = raw_deps {
gen_bunfig(job_dir).await?;
write_file(job_dir, "package.json", raw_deps.as_str())?;
} else {
@@ -201,21 +201,10 @@ pub async fn gen_bun_lockfile(
}
if export_pkg {
let mut content;
let mut content = "".to_string();
{
let mut file = File::open(format!("{job_dir}/package.json")).await?;
let mut buf = String::default();
file.read_to_string(&mut buf).await?;
if raw_deps.is_some() {
let mut json_map: HashMap<String, Box<RawValue>> = serde_json::from_str(&buf)?;
json_map.insert(
"generatedFromPackageJson".to_string(),
to_raw_value(&"true".to_string()),
);
content = serde_json::to_string_pretty(&json_map)?;
} else {
content = buf;
}
file.read_to_string(&mut content).await?;
}
if !npm_mode {
#[cfg(any(target_os = "linux", target_os = "macos"))]

View File

@@ -77,7 +77,6 @@ use crate::{
start_child_process, OccupancyMetrics,
},
handle_child::handle_child,
worker_lockfiles::LOCKFILE_GENERATED_FROM_REQUIREMENTS_TXT,
worker_utils::ping_job_status,
AuthedClient, DISABLE_NSJAIL, DISABLE_NUSER, HOME_ENV, INSTANCE_PYTHON_VERSION, NSJAIL_PATH,
PATH_ENV, PIP_EXTRA_INDEX_URL, PIP_INDEX_URL, PROXY_ENVS, PY_INSTALL_DIR, TZ_ENV, UV_CACHE_DIR,
@@ -2304,16 +2303,8 @@ fn split_requirements(requirements: &str) -> Vec<&str> {
/// Check requirements/lockfile to figure out python version assigned to it.
fn get_pyv_from_requirements_lines(requirements_lines: &[&str]) -> PyVersion {
// If script is deployed we can try to parse first line to get assigned version
let index = if requirements_lines.get(0).map_or(false, |line| {
line.starts_with(LOCKFILE_GENERATED_FROM_REQUIREMENTS_TXT)
}) {
1
} else {
0
};
if let Some(v) = requirements_lines
.get(index)
.get(0)
.and_then(|line| PyVersion::parse_version(*line))
{
// We have valid assigned version, we use it

View File

@@ -65,17 +65,17 @@ pub async fn update_script_dependency_map(
relative_imports: Vec<String>,
) -> error::Result<()> {
let importer_kind = "script";
let mut tx = db.begin().await?;
tx = clear_dependency_parent_path(parent_path, script_path, w_id, importer_kind, tx).await?;
tx = clear_dependency_map_for_item(script_path, w_id, importer_kind, tx, &None).await?;
if !relative_imports.is_empty() {
let mut logs = "".to_string();
logs.push_str("\n--- RELATIVE IMPORTS ---\n\n");
logs.push_str(&relative_imports.join("\n"));
let mut tx = db.begin().await?;
tx =
clear_dependency_parent_path(parent_path, script_path, w_id, importer_kind, tx).await?;
tx = clear_dependency_map_for_item(script_path, w_id, importer_kind, tx, &None).await?;
tx = add_relative_imports_to_dependency_map(
script_path,
w_id,
@@ -86,10 +86,9 @@ pub async fn update_script_dependency_map(
None,
)
.await?;
tx.commit().await?;
append_logs(job_id, w_id, logs, &db.into()).await;
}
tx.commit().await?;
Ok(())
}
@@ -382,7 +381,6 @@ pub async fn handle_dependency_job(
&job.permissioned_as_email,
&job.created_by,
&job.permissioned_as,
None,
)
.await?;
@@ -435,42 +433,18 @@ pub async fn process_relative_imports(
permissioned_as_email: &str,
created_by: &str,
permissioned_as: &str,
lock: Option<String>,
) -> error::Result<()> {
let relative_imports = extract_relative_imports(&code, script_path, script_lang);
if let Some(relative_imports) = relative_imports {
if (script_lang.is_some_and(|v| v == ScriptLang::Bun)
&& lock
.as_ref()
.is_some_and(|v| v.contains("generatedFromPackageJson")))
|| (script_lang.is_some_and(|v| v == ScriptLang::Python3)
&& lock
.as_ref()
.is_some_and(|v| v.starts_with(LOCKFILE_GENERATED_FROM_REQUIREMENTS_TXT)))
{
// if the lock file is generated from a package.json/requirements.txt, we need to clear the dependency map
// because we do not want to have dependencies be recomputed automatically. Empty relative imports passed
// to update_script_dependency_map will clear the dependency map.
update_script_dependency_map(
&job_id.unwrap_or_else(|| Uuid::nil()),
db,
w_id,
&parent_path,
script_path,
vec![],
)
.await?;
} else {
update_script_dependency_map(
&job_id.unwrap_or_else(|| Uuid::nil()),
db,
w_id,
&parent_path,
script_path,
relative_imports,
)
.await?;
}
update_script_dependency_map(
&job_id.unwrap_or_else(|| Uuid::nil()),
db,
w_id,
&parent_path,
script_path,
relative_imports,
)
.await?;
let already_visited = args
.map(|x| {
x.get("already_visited")
@@ -2076,8 +2050,6 @@ async fn ansible_dep(
serde_json::to_string(&ansible_lockfile).map_err(|e| e.into())
}
pub const LOCKFILE_GENERATED_FROM_REQUIREMENTS_TXT: &str = "# from requirements.txt";
async fn capture_dependency_job(
job_id: &Uuid,
job_language: &ScriptLang,
@@ -2146,13 +2118,6 @@ async fn capture_dependency_job(
anns,
)
.await
.map(|res| {
if raw_deps {
format!("{}\n{}", LOCKFILE_GENERATED_FROM_REQUIREMENTS_TXT, res)
} else {
res
}
})
}
}
ScriptLang::Ansible => {

View File

@@ -2,7 +2,7 @@ import { sleep } from "https://deno.land/x/sleep@v1.2.1/mod.ts";
import * as windmill from "https://deno.land/x/windmill@v1.174.0/mod.ts";
import * as api from "https://deno.land/x/windmill@v1.174.0/windmill-api/index.ts";
export const VERSION = "v1.488.0";
export const VERSION = "v1.485.3";
export async function login(email: string, password: string): Promise<string> {
return await windmill.UserService.login({

View File

@@ -63,7 +63,7 @@ export {
// }
// });
export const VERSION = "1.488.0";
export const VERSION = "1.485.3";
const command = new Command()
.name("wmill")

View File

@@ -17,7 +17,6 @@
extensions = [
"rust-src" # for rust-analyzer
"rust-analyzer"
"rustfmt"
];
};
buildInputs = with pkgs; [
@@ -184,7 +183,6 @@
ANSIBLE_PLAYBOOK_PATH = "${pkgs.ansible}/bin/ansible-playbook";
ANSIBLE_GALAXY_PATH = "${pkgs.ansible}/bin/ansible-galaxy";
RUST_LOG = "debug";
SQLX_OFFLINE = "true";
};
packages.default = self.packages.${system}.windmill;
packages.windmill-client = pkgs.buildNpmPackage {

View File

@@ -1,12 +1,12 @@
{
"name": "windmill-components",
"version": "1.488.0",
"version": "1.485.3",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "windmill-components",
"version": "1.488.0",
"version": "1.485.3",
"hasInstallScript": true,
"license": "AGPL-3.0",
"dependencies": {

View File

@@ -1,6 +1,6 @@
{
"name": "windmill-components",
"version": "1.488.0",
"version": "1.485.3",
"scripts": {
"dev": "vite dev",
"build": "vite build",

View File

@@ -103,9 +103,7 @@
}}
disabled={!$enterpriseLicense || !$superadmin}
>
Save {#if !$superadmin}
<span class="text-2xs text-tertiary">superadmin only</span>
{/if}
Save {#if !$superadmin} <span class="text-2xs text-tertiary">superadmin only</span> {/if}
</Button>
<span class="text-2xs text-tertiary"

View File

@@ -63,11 +63,11 @@
const dispatch = createEventDispatcher()
$: inputCat = computeInputCat(
schema?.properties?.[argName]?.type,
schema?.properties?.[argName]?.format,
schema?.properties?.[argName]?.items?.type,
schema?.properties?.[argName]?.enum,
schema?.properties?.[argName]?.contentEncoding
schema?.properties?.[argName].type,
schema?.properties?.[argName].format,
schema?.properties?.[argName].items?.type,
schema?.properties?.[argName].enum,
schema?.properties?.[argName].contentEncoding
)
let propertyType = getPropertyType(arg)
@@ -275,7 +275,7 @@
function setDefaultCode() {
if (!arg?.value) {
monacoTemplate?.setCode(schema.properties?.[argName]?.default)
monacoTemplate?.setCode(schema.properties?.[argName].default)
}
}
@@ -287,7 +287,7 @@
}
$: updateFocused(focused)
$: schema?.properties?.[argName]?.default && setDefaultCode()
$: schema?.properties?.[argName].default && setDefaultCode()
let resourceTypes: string[] | undefined = undefined
@@ -321,10 +321,10 @@
simpleTooltip={headerTooltip}
simpleTooltipIconClass={headerTooltipIconClass}
SimpleTooltipIcon={HeaderTooltipIcon}
format={schema?.properties?.[argName]?.format}
contentEncoding={schema?.properties?.[argName]?.contentEncoding}
format={schema?.properties?.[argName].format}
contentEncoding={schema?.properties?.[argName].contentEncoding}
required={schema.required?.includes(argName)}
type={schema.properties?.[argName]?.type}
type={schema.properties?.[argName].type}
/>
{#if isStaticTemplate(inputCat)}

View File

@@ -334,15 +334,15 @@
latestKeyRenewalAttempt.result === 'success'
? 'text-green-600'
: isTrial
? 'text-yellow-600'
: 'text-red-600'
? 'text-yellow-600'
: 'text-red-600'
)}
>
{latestKeyRenewalAttempt.result === 'success'
? 'Latest key renewal succeeded'
: isTrial
? 'Latest key renewal ignored because in trial'
: 'Latest key renewal failed'}
? 'Latest key renewal ignored because in trial'
: 'Latest key renewal failed'}
on {attemptedAt}
</span>
</div>
@@ -928,30 +928,6 @@
{:else if setting.fieldType == 'object_store_config'}
<ObjectStoreConfigSettings bind:bucket_config={$values[setting.key]} />
<div class="mb-6"></div>
{:else if setting.fieldType == 'critical_alerts_on_db_oversize'}
{#if $values[setting.key]}
<div class="flex flex-row flex-wrap gap-2 p-0 items-center">
<div class="p-1">
<Toggle
disabled={!$enterpriseLicense}
bind:checked={$values[setting.key].enabled}
/>
</div>
{#if $values[setting.key].enabled}
<label class="block shrink min-w-0">
<input
type="number"
placeholder={setting.placeholder}
bind:value={$values[setting.key].value}
/>
</label>
<label class="block">
<span class="text-primary font-semibold text-sm">GB</span>
</label>
{/if}
</div>
<div class="mb-6"></div>
{/if}
{:else if setting.fieldType == 'number'}
<input
type="number"

View File

@@ -86,34 +86,6 @@ export async function main(s3: S3) {
return res.text()
})
}
`,
lang: 'bun',
argName: 's3'
},
azure_blob: {
code: `
import * as wmill from "windmill-client"
type S3 = object
export async function main(s3: S3) {
return fetch(process.env["BASE_URL"] + '/api/settings/test_object_storage_config', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Authorization: 'Bearer ' + process.env["WM_TOKEN"],
},
body: JSON.stringify({
type: "Azure",
...s3
}),
}).then(async (res) => {
if (!res.ok) {
throw new Error(await res.text())
}
return res.text()
})
}
`,
lang: 'bun',
argName: 's3'

View File

@@ -293,7 +293,7 @@
tooltip="This event is triggered when the script runs successfully."
items={Object.keys($runnableComponents).filter((_id) => _id !== id)}
bind:value={
() => hiddenInlineScript.script.recomputeIds ?? [],
() => hiddenInlineScript.script.recomputeIds,
(v) => {
if ($app.hiddenInlineScripts[hiddenInlineScript.index]) {
$app.hiddenInlineScripts[hiddenInlineScript.index].recomputeIds = v

View File

@@ -4,15 +4,12 @@
import MultiSelect from '$lib/components/multiselect/MultiSelectWrapper.svelte'
import { twMerge } from 'tailwind-merge'
export let items: string[]
export let value: string[] | undefined = undefined
export let title: string
export let tooltip: string
let { items, value = $bindable(), title, tooltip } = $props<{
items: string[]
value: string[] | undefined
title: string
tooltip: string
}>()
let width = $state(0)
let width = 0
const inputWidth = 280
</script>

View File

@@ -26,7 +26,6 @@ export interface Setting {
| 'license_key'
| 'object_store_config'
| 'critical_error_channels'
| 'critical_alerts_on_db_oversize'
| 'slack_connect'
| 'smtp_connect'
| 'indexer_rates'
@@ -354,15 +353,6 @@ export const settings: Record<string, Setting[]> = {
fieldType: 'smtp_connect',
storage: 'setting',
ee_only: ''
},
{
label: 'Alert on DB oversize',
key: 'critical_alerts_on_db_oversize',
description: 'Alert if DB grows more than specified size',
fieldType: 'critical_alerts_on_db_oversize',
placeholder: '100',
storage: 'setting',
ee_only: ''
}
],
'OTEL/Prom': [

View File

@@ -1,41 +1,30 @@
<script lang="ts">
// @ts-ignore
import Portal from '$lib/components/Portal.svelte'
import { createFloatingActions } from 'svelte-floating-ui'
import { tick } from 'svelte'
import { offset, flip, shift } from 'svelte-floating-ui/dom'
import MultiSelect from '$lib/components/multiselect/MultiSelect.svelte'
import DarkModeObserver from '../DarkModeObserver.svelte'
import { deepEqual } from 'fast-equals'
let {
items,
value = $bindable(),
placeholder = undefined,
target = undefined,
topPlacement = false,
allowUserOptions = undefined
} = $props<{
items: any[]
value?: string[]
placeholder?: string
target?: string | HTMLElement
topPlacement?: boolean
allowUserOptions?: boolean | 'append'
}>()
$effect.pre(() => { if (value === undefined) value = [] })
export let items: any[]
let propValue: string[] | undefined = []
export { propValue as value }
$: value = structuredClone(propValue)
export let placeholder: string | undefined = undefined
export let target: string | HTMLElement | undefined = undefined
export let topPlacement = false
export let allowUserOptions: boolean | 'append' | undefined = undefined
const [floatingRef, floatingContent] = createFloatingActions({
strategy: 'absolute',
placement: topPlacement ? 'top-start' : 'bottom-start',
middleware: [offset(5), flip(), shift()]
})
let outerDiv = $state<HTMLDivElement | undefined>(undefined)
let portalRef = $state<HTMLDivElement | undefined>(undefined)
let darkMode = $state(false)
let w = $state(0)
let open = $state(false)
let outerDiv: HTMLDivElement | undefined = undefined
let portalRef: HTMLDivElement | undefined = undefined
function moveOptionsToPortal() {
// Find ul element with class 'options' within the outerDiv
const ul = outerDiv?.querySelector('.options')
@@ -45,13 +34,17 @@
}
}
$effect(() => {
if (portalRef && outerDiv && (allowUserOptions || items?.length > 0)) {
tick().then(() => {
moveOptionsToPortal()
})
}
})
$: if (portalRef && outerDiv && (allowUserOptions || items?.length > 0)) {
tick().then(() => {
moveOptionsToPortal()
})
}
// bg-indigo-100 text-indigo-800 dark:bg-indigo-200 dark:text-indigo-900
let darkMode: boolean = false
let w = 0
let open: boolean = false
</script>
<DarkModeObserver bind:darkMode />
@@ -69,14 +62,10 @@
--sms-focus-border={'none'}
--sms-selected-bg={darkMode ? '#c7d2fe' : '#e0e7ff'}
--sms-selected-text-color={darkMode ? '#312e81' : '#3730a3'}
bind:selected={
() => [...value],
(newVal) => {
if (!deepEqual(value, newVal)) {
value = newVal
}
}
}
bind:selected={value}
on:change={() => {
propValue = value
}}
{placeholder}
options={items}
on:close={() => {
@@ -88,19 +77,15 @@
let:option
disableRemoveAll
>
<!-- needed because portal doesn't work for mouseup event en mobile -->
<!-- svelte-ignore a11y-no-static-element-interactions -->
<div
class="w-full text-sm"
role="option"
tabindex="0"
onmouseup={(e) => {
e.stopPropagation()
}}
onpointerdown={(e) => {
e.stopPropagation()
on:mouseup|stopPropagation
on:pointerdown|stopPropagation={(e) => {
let newe = new MouseEvent('mouseup')
e.target?.['parentElement']?.dispatchEvent(newe)
}}
aria-selected={value?.includes(option)}
>
{option}
</div>
@@ -108,17 +93,13 @@
</div>
<Portal {target} name="multi-select">
<div use:floatingContent class="z5000" hidden={!open}>
<!-- svelte-ignore a11y_click_events_have_key_events -->
<!-- svelte-ignore a11y_no_static_element_interactions -->
<!-- svelte-ignore a11y-click-events-have-key-events -->
<!-- svelte-ignore a11y-no-static-element-interactions -->
<div
bind:this={portalRef}
class="multiselect"
style={`min-width: ${w}px;`}
onclick={(e) => {
e.stopPropagation()
}}
role="listbox"
tabindex="0"
on:click|stopPropagation
></div>
</div>
</Portal>

View File

@@ -2,7 +2,7 @@
import TableCustom from '$lib/components/TableCustom.svelte'
import { displayDate, copyToClipboard } from '$lib/utils'
import type { TruncatedToken, NewToken } from '$lib/gen'
import { IntegrationService, UserService } from '$lib/gen'
import { UserService } from '$lib/gen'
import { Button } from '$lib/components/common'
import { Clipboard, Plus } from 'lucide-svelte'
import { workspaceStore, userWorkspaces, type UserWorkspace } from '$lib/stores'
@@ -12,7 +12,6 @@
import Toggle from '../Toggle.svelte'
import ClipboardPanel from '../details/ClipboardPanel.svelte'
import { sendUserToast } from '$lib/toast'
import MultiSelectWrapper from '../multiselect/MultiSelectWrapper.svelte'
// --- Props ---
interface Props {
@@ -38,14 +37,10 @@
let newToken = $state<string | undefined>(undefined)
let newTokenExpiration = $state<number | undefined>(undefined)
let newTokenWorkspace = $state<string | undefined>(defaultNewTokenWorkspace)
let newMcpApps = $state<string[]>([])
let displayCreateToken = $state(scopes != undefined)
let mcpCreationMode = $state(false)
let newMcpScope = $state('favorites')
let newMcpToken = $state<string | undefined>(undefined)
let loadingApps = $state(false)
let errorFetchApps = $state(false)
let allApps = $state<string[]>([])
function ensureCurrentWorkspaceIncluded(
workspacesList: UserWorkspace[],
@@ -72,14 +67,6 @@
listTokens()
})
$effect(() => {
if (mcpCreationMode) {
getAllApps()
} else {
newMcpApps = []
}
})
// --- Functions ---
async function createToken(mcpMode: boolean = false): Promise<void> {
try {
@@ -88,13 +75,7 @@
date = new Date(new Date().getTime() + newTokenExpiration * 1000)
}
let tokenScopes = scopes
if (mcpMode) {
tokenScopes = [`mcp:${newMcpScope}`]
if (newMcpApps.length > 0) {
tokenScopes.push(`mcp:hub:${newMcpApps.join(',')}`)
}
}
let tokenScopes = mcpMode ? [`mcp:${newMcpScope}`] : scopes
const createdToken = await UserService.createToken({
requestBody: {
@@ -126,7 +107,6 @@
newMcpToken = undefined
newToken = undefined
newTokenExpiration = undefined
newMcpApps = []
newTokenLabel = type === 'mcpUrl' ? 'MCP token' : undefined
}
@@ -161,26 +141,6 @@
tokenPage -= 1
listTokens()
}
async function getAllApps() {
if (allApps.length > 0) {
return
}
try {
loadingApps = true
allApps = (
await IntegrationService.listHubIntegrations({
kind: 'script'
})
).map((x) => x.name)
} catch (err) {
console.error('Hub is not available')
allApps = []
errorFetchApps = true
} finally {
loadingApps = false
}
}
</script>
<div class="grid grid-cols-2 pt-8 pb-1" class:pt-8={scopes == undefined}>
@@ -232,103 +192,66 @@
{#if displayCreateToken}
<div class="py-3 px-3 border rounded-md mb-6 bg-surface-secondary min-w-min">
<h3 class="pb-3 font-semibold">Add a new token</h3>
{#if showMcpMode}
<div class="mb-4 flex flex-row flex-shrink-0">
<Toggle
on:change={(e) => {
mcpCreationMode = e.detail
if (e.detail) {
newTokenLabel = 'MCP token'
newTokenExpiration = undefined
newTokenWorkspace = $workspaceStore
} else {
newTokenLabel = undefined
newTokenExpiration = undefined
newTokenWorkspace = defaultNewTokenWorkspace
}
}}
checked={mcpCreationMode}
options={{
right: 'Generate MCP URL',
rightTooltip:
'Generate a new MCP URL to make your scripts and flows available as tools through your LLM clients.',
rightDocumentationLink: 'https://www.windmill.dev/docs/core_concepts/mcp'
}}
size="xs"
/>
</div>
{/if}
{#if scopes != undefined}
<div class="mb-4">
<span class="block mb-1">Scope</span>
{#each scopes as scope}
<input disabled type="text" value={scope} class="mb-2 w-full" />
{/each}
</div>
{#each scopes as scope}
<div class="flex flex-col mb-4">
<label for="label">Scope</label>
<input disabled type="text" value={scope} />
</div>
{/each}
{/if}
<div class="grid grid-cols-1 md:grid-cols-2 gap-4">
{#if showMcpMode}
<Toggle
on:change={(e) => {
mcpCreationMode = e.detail
if (e.detail) {
newTokenLabel = 'MCP token'
newTokenExpiration = undefined
newTokenWorkspace = $workspaceStore
} else {
newTokenLabel = undefined
newTokenExpiration = undefined
newTokenWorkspace = defaultNewTokenWorkspace
}
}}
checked={mcpCreationMode}
options={{
right: 'Generate MCP URL',
rightTooltip:
'Generate a new MCP URL to make your scripts and flows available as tools through your LLM clients.',
rightDocumentationLink: 'https://www.windmill.dev/docs/core_concepts/mcp'
}}
class="mb-4"
size="xs"
/>
{/if}
<div class="flex flex-row flex-wrap gap-2 w-full justify-between">
{#if mcpCreationMode}
<div>
<span class="block mb-1">Scope</span>
<div class="flex flex-col">
<label for="label">Scope</label>
<ToggleButtonGroup bind:selected={newMcpScope} allowEmpty={false} let:item>
<ToggleButton
{item}
value="favorites"
label="Favorites only"
tooltip="Make only your favorite scripts and flows available as tools"
/>
<ToggleButton
{item}
value="all"
label="All scripts/flows"
tooltip="Make all your scripts and flows available as tools"
/>
<ToggleButton {item} value="favorites" label="Favorites Only" />
<ToggleButton {item} value="all" label="All Resources" />
</ToggleButtonGroup>
</div>
<div>
<span class="block mb-1">Hub scripts (optional)</span>
{#if loadingApps}
<div>Loading...</div>
{:else if errorFetchApps}
<div>Error fetching apps</div>
{:else}
<MultiSelectWrapper
items={allApps}
placeholder="Select apps"
bind:value={newMcpApps}
/>
{/if}
</div>
<div>
<span class="block mb-1">Workspace</span>
<select
bind:value={newTokenWorkspace}
disabled={workspaces.length === 1}
class="w-full"
>
<div class="flex flex-col">
<label for="label">Workspace</label>
<select bind:value={newTokenWorkspace} disabled={workspaces.length === 1}>
{#each workspaces as workspace}
<option value={workspace.id}>{workspace.name}</option>
{/each}
</select>
</div>
{/if}
<div>
<span class="block mb-1">Label <span class="text-xs text-tertiary">(optional)</span></span
>
<input type="text" bind:value={newTokenLabel} class="w-full" />
<div class="flex flex-col">
<label for="label">Label <span class="text-xs text-tertiary">(optional)</span></label>
<input type="text" bind:value={newTokenLabel} />
</div>
<div>
<span class="block mb-1"
>Expires In <span class="text-xs text-tertiary">(optional)</span></span
>
<select bind:value={newTokenExpiration} disabled={mcpCreationMode} class="w-full">
<div class="flex flex-col">
<label for="expires"
>Expires In &nbsp;<span class="text-xs text-tertiary">(optional)</span>
</label>
<select bind:value={newTokenExpiration} disabled={mcpCreationMode}>
<option value={undefined}>No expiration</option>
<option value={15 * 60}>15m</option>
<option value={30 * 60}>30m</option>
@@ -339,23 +262,15 @@
<option value={90 * 24 * 60 * 60}>90d</option>
</select>
</div>
</div>
<div class="mt-4 flex justify-end gap-2 flex-row">
<Button
on:click={() => {
mcpCreationMode = false
displayCreateToken = false
}}
>
Cancel
</Button>
<Button
on:click={() => createToken(mcpCreationMode)}
disabled={mcpCreationMode && newTokenWorkspace == undefined}
>
New token
</Button>
<div class="flex items-end">
<Button
btnClasses="!mt-2"
on:click={() => createToken(mcpCreationMode)}
disabled={mcpCreationMode && newTokenWorkspace === undefined}
>
New token
</Button>
</div>
</div>
</div>
{/if}

View File

@@ -133,7 +133,7 @@
} else {
await goto(rd ?? '/')
}
console.log('Workspace selected going to ' + (rd ? `rd: ${rd}` : 'home'))
console.log('Workspace selected, going to', rd)
} catch (e) {
console.error('Error going to', rd, e)
window.location.reload()

View File

@@ -163,7 +163,6 @@ class TestAgentWorkers(unittest.TestCase):
def test_create_agent_token(self):
token = self._agent_token
print(f"Agent token tests for token: {token}")
self.assertIsNotNone(token)
# JWT tokens have the format: jwt_agent_<prefix>_<token>

View File

@@ -415,5 +415,5 @@ class WindmillClient:
raise Exception(response.content.decode())
token = response.content.decode().strip('"')
print(f"Created agent token: {token}")
print(f"Created agent token: {token[:15]}...{token[-15:]}")
return token

View File

@@ -4,8 +4,8 @@ verify_ssl = true
name = "pypi"
[packages]
wmill = ">=1.488.0"
wmill_pg = ">=1.488.0"
wmill = ">=1.485.3"
wmill_pg = ">=1.485.3"
sendgrid = "*"
mysql-connector-python = "*"
pymongo = "*"

View File

@@ -1,7 +1,7 @@
openapi: "3.0.3"
info:
version: 1.488.0
version: 1.485.3
title: OpenFlow Spec
contact:
name: Ruben Fiszel

View File

@@ -12,7 +12,7 @@
RootModule = 'WindmillClient.psm1'
# Version number of this module.
ModuleVersion = '1.488.0'
ModuleVersion = '1.485.3'
# Supported PSEditions
# CompatiblePSEditions = @()

View File

@@ -1,6 +1,6 @@
[tool.poetry]
name = "wmill"
version = "1.488.0"
version = "1.485.3"
description = "A client library for accessing Windmill server wrapping the Windmill client API"
license = "Apache-2.0"
homepage = "https://windmill.dev"

View File

@@ -1,6 +1,6 @@
[tool.poetry]
name = "wmill-pg"
version = "1.488.0"
version = "1.485.3"
description = "An extension client for the wmill client library focused on pg"
license = "Apache-2.0"
homepage = "https://windmill.dev"

View File

@@ -1,6 +1,6 @@
{
"name": "@windmill/windmill",
"version": "1.488.0",
"version": "1.485.3",
"exports": "./src/index.ts",
"publish": {
"exclude": ["!src", "./s3Types.ts", "./client.ts"]

View File

@@ -1,7 +1,7 @@
{
"name": "windmill-client",
"description": "Windmill SDK client for browsers and Node.js",
"version": "1.488.0",
"version": "1.485.3",
"author": "Ruben Fiszel",
"license": "Apache 2.0",
"devDependencies": {

View File

@@ -1 +1 @@
1.488.0
1.485.3