Compare commits

..

51 Commits

Author SHA1 Message Date
HugoCasa
8468b2cb42 fix sqlx 2026-01-29 18:50:06 +01:00
hugocasa
60858d1e20 feat: workspace dedicated workers (#7741)
* feat: workspace dedicated workers

* ref

* chore: update ee-repo-ref to a18ac31062ac092cb9a5fc87629e217d97f4911d

This commit updates the EE repository reference after PR #398 was merged in windmill-ee-private.

Previous ee-repo-ref: 98cfe3fef764d9d815d326d5056c734a03689d33

New ee-repo-ref: a18ac31062ac092cb9a5fc87629e217d97f4911d

Automated by sync-ee-ref workflow.

* fix(frontend): workspace script in flow steps

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-01-29 17:32:48 +00:00
Ruben Fiszel
f45d9adf6a chore(main): release 1.621.2 (#7735)
* chore(main): release 1.621.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-01-29 17:19:39 +00:00
Ruben Fiszel
20357f41f5 fix(cli): revert findCodebase change that broke ../shared codebases (#7740)
* fix(cli): revert findCodebase relative_path check that broke ../shared codebases

The previous change added a check to ensure script paths start with the
codebase's relative_path. However, this broke cases where relative_path
uses parent directory references (e.g., "../shared") because:

1. path.join normalizes paths, so "/project/../shared/f/script.ts" becomes
   "/shared/f/script.ts"
2. FSFSElement strips the cwd prefix, resulting in "f/script.ts"
3. The check "f/script.ts".startsWith("../shared/") failed

The original behavior was correct - relative_path indicates where to find
codebase files, while includes/excludes patterns match against the normalized
paths that get passed during sync.

Fixes regression reported in #7729 comments.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test(cli): add preview test for codebase with imports

Tests that codebase bundling correctly includes imported modules,
which is the key functionality needed for ../shared codebases.
The test creates a helper module and a main script that imports
from it, then verifies the bundled script executes correctly.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 17:09:02 +00:00
centdix
fe4a230833 docs: add nuanced mutex selection guidance to Rust backend skill (#7737)
Add "Mutex Selection in Async Code" section explaining when to use
std::sync::Mutex vs tokio::sync::Mutex based on official Tokio docs.
std::sync::Mutex is preferred for data protection as it's faster;
tokio::sync::Mutex only needed when holding locks across .await points.

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 15:08:41 +00:00
centdix
d004aa8ec1 internal: add Rust backend coding skill and consolidate guidelines (#7731)
* feat: add Rust backend coding skill and consolidate guidelines

Create a dedicated skill at .claude/skills/rust-backend/SKILL.md that
provides comprehensive Rust coding guidelines adapted to Windmill patterns:
- Iterator chains, error handling, early returns, variable shadowing
- JSON handling with Box<RawValue>, Serde optimizations
- SQLx patterns (no SELECT *, batch operations, avoid N+1)
- Async/Tokio patterns (spawn_blocking, bounded channels)

Consolidate project context into backend/CLAUDE.md and remove the
redundant rust-best-practices.mdc file.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* add command plugins

* nit

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 12:02:27 +00:00
centdix
1aad20b7eb hide cred check (#7736) 2026-01-29 12:01:29 +00:00
wendrul
8bb6b6331b fix: do not quit indexer when receiving handoff during pull (#7659)
* fix: do not quit indexer when receiving handoff during pull

* update

* Add correct return type

* update ee-repo-ref [CI only]

* chore: update ee-repo-ref to c05572e93739e2697ab310d87efe2744cd0e1aaf

This commit updates the EE repository reference after PR #394 was merged in windmill-ee-private.

Previous ee-repo-ref: 4358aa9c5b3b38ba74d7ea52cafd49899d338a07

New ee-repo-ref: c05572e93739e2697ab310d87efe2744cd0e1aaf

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-01-29 11:43:26 +00:00
centdix
0089ebd4fb chore: handle empty strings in AI resource fields via serde deserializer (#7723)
* fix: handle empty strings in AI resource fields via serde deserializer

Add `empty_string_as_none` deserializer that converts empty strings to None
during deserialization. Applied to base_url, api_key, region, and AWS
credential fields in AIStandardResource and ProviderResource.

This fixes the "relative URL without a base" error when creating Anthropic
resources with empty base_url fields.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit

* nit

* nit

* cleaning

* cleaning

* cleaning

* cleaning

* fix: apply empty_string_as_none deserializer to api_key field

Consistent with other fields in ProviderResource, empty strings are now
deserialized as None for the api_key field.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 11:43:02 +00:00
centdix
f856f672d8 Allow wmill app dev to accept folder argument (#7718)
Enable running the dev command from any directory by specifying the
target .raw_app folder as an argument. Workspace resolution and
authentication still happen from the original cwd to find wmill.yaml.

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 11:42:51 +00:00
Ruben Fiszel
ebecd709af chore(main): release 1.621.1 (#7733)
* chore(main): release 1.621.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-01-29 09:25:42 +00:00
Ruben Fiszel
db74470ec3 fix: add 32MB memory limit to QuickJS runtime for flow expressions
QuickJS was missing an explicit memory limit, unlike deno_core which has
a 128MB heap limit. This adds a 32MB limit appropriate for lightweight
flow expression evaluation.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 09:17:10 +00:00
Ruben Fiszel
441da480f9 cli test nits 2026-01-29 08:44:16 +00:00
Ruben Fiszel
22a447591e chore(main): release 1.621.0 (#7732)
* chore(main): release 1.621.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-01-29 08:38:31 +00:00
Ruben Fiszel
799766264a nit cli tests 2026-01-29 08:38:07 +00:00
Ruben Fiszel
22cce51db5 fix: return null for non-existent step access in flow expressions
Previously, accessing a non-existent step via results.nonexistent would
throw an error. This fix makes both Deno Core and QuickJS return null
instead, enabling patterns like:

- results.nonexistent ?? 'default'
- results.nonexistent?.value ?? 'default'

The fix was applied to:
- js_eval.rs: handle_full_regex fast-path now uses .ok().flatten()
- js_eval_quickjs.rs: fallback path now uses .ok().unwrap_or(null)

Added flow engine test to verify the behavior.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 07:48:46 +00:00
Ruben Fiszel
5c20b37a53 feat: add QuickJS as alternative JS engine for flow expression evaluation (#7664)
* feat: add QuickJS as alternative JS engine for flow expression evaluation

Add rquickjs as an optional alternative to deno_core for evaluating
JavaScript expressions in flow transformations. QuickJS offers ~8-16x
faster startup times for simple expressions, making it ideal for
evaluating many small expressions in flows.

Key changes:
- Add new `quickjs` feature flag for windmill-worker
- Implement js_eval_quickjs.rs with true async Rust callbacks for
  variable(), resource(), and results.xxx access (no pre-fetching)
- Share expression transformation logic (replace_with_await,
  replace_with_await_result) between both implementations
- Add USE_QUICKJS_FOR_FLOW_EVAL env var to switch engines at runtime
- When only quickjs feature is enabled (no deno_core), QuickJS is
  automatically used
- Add comprehensive parity tests comparing QuickJS and deno_core output

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* all

* quickjs

* quickjs

* all

* all

* all

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 01:44:40 +00:00
Ruben Fiszel
45e0dd0b07 chore(main): release 1.620.1 (#7730)
* chore(main): release 1.620.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-01-28 22:58:56 +00:00
Ruben Fiszel
c59699acd7 fix: codebase preview in standalone mode
- Fix standalone bundle path lookup in worker to not add redundant file
  extension (the path already contains .tar/.esm suffixes from the API)
- Fix CLI preview tar bundle handling to preserve binary data correctly
  (was using btoa(blob.text()) which corrupted binary tar data)
- Add integration tests for script/flow preview commands covering:
  - Regular scripts (non-codebase)
  - Codebase scripts (CJS and ESM formats)
  - Codebase scripts with assets (tar bundles)
  - Flow preview

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 22:54:38 +00:00
Ruben Fiszel
f955496dc1 chore(main): release 1.620.0 (#7728)
* chore(main): release 1.620.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-01-28 20:00:23 +00:00
Ruben Fiszel
95cbb2c86c feat(cli): add script preview and flow preview commands (#7729)
- Add `wmill script preview <path> [--data <json>]` command to test scripts against remote workspace without deploying
- Add `wmill flow preview <path> [--data <json>]` command to test flows against remote workspace without deploying
- Support codebase scripts with automatic bundling via esbuild
- Add `--silent` flag to suppress logs and only output final result
- Fix `findCodebase` to properly check if path is within codebase relative_path before pattern matching

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 19:52:43 +00:00
Ruben Fiszel
eafee16bfc fix: cache git branch detection to avoid repeated execSync calls
Previously, getCurrentGitBranch() was called inside loops for every
file processed during sync pull/push operations. For workspaces with
1900+ files, this spawned thousands of git subprocesses, causing a ~2x
performance regression.

This fix caches the git branch at the start of:
- elementsToMap() for pull operations
- push() for push operations

Expected improvement: ~3.2s -> ~1.6s for large workspaces.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 19:09:03 +00:00
Ruben Fiszel
9be12bb607 chore(main): release 1.619.0 (#7722)
* chore(main): release 1.619.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-01-28 18:26:08 +00:00
Pyra
f50a866430 fix: nativets http tracing (#7716)
* fix: nativets http tracing

Signed-off-by: pyranota <pyra@duck.com>

* ee repo

Signed-off-by: pyranota <pyra@duck.com>

* nit

Signed-off-by: pyranota <pyra@duck.com>

* ee ref

Signed-off-by: pyranota <pyra@duck.com>

* ee repo

Signed-off-by: pyranota <pyra@duck.com>

* ee repo

Signed-off-by: pyranota <pyra@duck.com>

* fix

Signed-off-by: pyranota <pyra@duck.com>

* fix

Signed-off-by: pyranota <pyra@duck.com>

* fix v2

Signed-off-by: pyranota <pyra@duck.com>

* ee repo

Signed-off-by: pyranota <pyra@duck.com>

* chore: update ee-repo-ref to 5d841b358dd32130c9f34b54f59b96b5c322f213

This commit updates the EE repository reference after PR #396 was merged in windmill-ee-private.

Previous ee-repo-ref: 250723c698fceccbc66ae9a6c6c7c09e33465819

New ee-repo-ref: 5d841b358dd32130c9f34b54f59b96b5c322f213

Automated by sync-ee-ref workflow.

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-01-28 18:19:19 +00:00
hugocasa
d3d35d4cd8 fix(backend): leave job and audit history and archive workspace when changing workspace id (#7724) 2026-01-28 18:18:54 +00:00
wendrul
36dad2c7a2 fix: Raw apps deployment UI (and merge UI) (#7725) 2026-01-28 18:18:39 +00:00
centdix
82f378bcb4 fix: make api key optional (#7726) 2026-01-28 18:18:24 +00:00
Ruben Fiszel
116b9e7db3 fix(cli): handle symlinks in isMain() for Node.js
The dnt polyfill's import-meta-ponyfill doesn't resolve symlinks when
comparing process.argv[1] with import.meta.url. When npm creates a
symlink for the `wmill` bin (e.g., /usr/bin/wmill -> .../main.js),
the paths don't match and isMain() incorrectly returns false, causing
the CLI to silently exit without running.

This fix resolves symlinks using fs.realpathSync() before comparison,
ensuring the CLI works correctly when invoked via npm-installed symlinks.

Tested with Node.js 20 and 25.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 18:16:17 +00:00
Ruben Fiszel
b6abcc33a1 feat: enable tree-shaking for windmill-client
- Remove service re-exports from client.ts
- Build default export explicitly in index.ts
- Use unbundled ESM output
- Add sideEffects: false

Results: ~900 bytes vs 91KB for simple imports

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 18:02:14 +00:00
Ruben Fiszel
0f625580f3 fix: use tsc for clean .d.ts files instead of tsdown bundled types
tsdown's bundled .d.ts output uses namespace declarations that reference
forward-declared types, which breaks Monaco/ATA type acquisition.

Switch to:
- tsdown for JS bundles (ESM + CJS) with --no-dts
- tsc with emitDeclarationOnly for clean individual .d.ts files

This restores the type structure from 1.617.0 which worked correctly
with Monaco editor's automatic type acquisition.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 16:22:16 +00:00
Ruben Fiszel
a02938c80c fix: fix TypeScript default export for Monaco/ATA compatibility
tsdown generates "export { X as default }" which doesn't work properly
with Monaco's TypeScript type acquisition. This post-processes the .d.ts
files to use "export default X" instead.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 15:58:19 +00:00
Ruben Fiszel
971b3c8b4a chore(main): release 1.618.2 (#7721)
* chore(main): release 1.618.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-01-28 15:29:57 +00:00
Ruben Fiszel
e7ac7afe8e fix: add default export to typescript-client for ESM compatibility
Enables `import wmill from "windmill-client"` syntax which was previously
broken due to missing default export in the generated ESM bundle.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 15:23:42 +00:00
Ruben Fiszel
a1b10a2f52 chore(main): release 1.618.1 (#7720) 2026-01-28 15:13:55 +00:00
centdix
7cd51def2b fix: handle empty base_url and region strings in AI providers (#7719)
When creating an Anthropic resource with "standard platform", the resource
JSON may contain `"base_url": ""` rather than omitting the field. Serde
deserializes this as `Some("")`, which bypassed the fallback logic and
caused "relative URL without a base" errors.

Similarly, AWS Bedrock with an empty region string would produce an
invalid URL like `https://bedrock-runtime..amazonaws.com`.

Filter out empty strings when checking for custom base_url and region
values, allowing the default URLs to be used correctly.

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 15:08:30 +00:00
Diego Imbert
51dc166b13 Fix DB Manager refresh button (#7717) 2026-01-28 13:56:35 +00:00
Ruben Fiszel
a58dd287ee chore(main): release 1.618.0 (#7714)
* chore(main): release 1.618.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-01-28 12:41:54 +00:00
Diego Imbert
fd326f6b24 Fix superadmins frontend issue (#7712)
* Fix superadmins bug

* simpler
2026-01-28 12:35:00 +00:00
Mark Lai
07fb47e215 feat: typescript client esm build (#7709)
* feat: typescript client esm build

* fix: add --dts flag and restore tsconfig options for typescript client ESM build

- Add --dts flag to tsdown commands to generate declaration files
- Restore outDir in tsconfig.json for compatibility
- Restore forceConsistentCasingInFileNames for case-sensitive systems
- Update README_DEV.md to reflect new tsdown build process

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-28 11:30:57 +00:00
Guilhem
e37ab33b3f fix: fix flow viewer height (#7715) 2026-01-28 11:20:17 +00:00
Diego Imbert
b76d6e9be8 fix: fix annoying abort toasts (#7713)
* cancelableResource

* Fix cancelled toasts

* Revert "cancelableResource"

This reverts commit 9cf6df488f.
2026-01-28 11:08:09 +00:00
Ruben Fiszel
ed107891d9 chore(main): release 1.617.3 (#7707)
* chore(main): release 1.617.3

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-01-27 18:28:41 +00:00
centdix
705bc48131 fix(backend): include empty schemas in list_datatable_schemas endpoint (#7708)
The endpoint now returns all non-system schemas, including empty ones
without tables. This is useful for CLI and frontend features that need
to know about available schemas for autocompletion and app creation.

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 18:14:26 +00:00
Ruben Fiszel
564d8266dc fix: do not delete tokens on being promoted to superadmins 2026-01-27 18:13:48 +00:00
centdix
abe6cc49b9 fix(cli): make wmill app lint and wmill app generate-agents respect nonDottedPaths setting (#7706)
These commands were using folder suffix checks without first loading the
nonDottedPaths setting from wmill.yaml, causing them to fail when run
inside folders with non-dotted names (e.g., myapp__raw_app instead of
myapp.raw_app).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-27 17:58:03 +00:00
Ruben Fiszel
08aa6e4a4c chore(main): release 1.617.2 (#7702)
* chore(main): release 1.617.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-01-27 17:38:27 +00:00
hugocasa
4ef1616893 fix(backend): folder/group permissions workspace id change (#7703) 2026-01-27 17:31:35 +00:00
hugocasa
456dd478d8 fix: 404 triggers listing in CE (#7705) 2026-01-27 17:30:36 +00:00
centdix
720a7e56d1 fix(mcp): use computed base_internal_url instead of static default (#7701)
* fix(mcp): use computed base_internal_url instead of static default

Pass the actual base_internal_url (computed from the runtime port) to
the MCP backend instead of using the static BASE_INTERNAL_URL which
defaults to http://localhost:8000. This fixes internal API calls when
the server runs on a non-default port.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix

* remove BASE_INTERNAL_URL

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 17:01:24 +00:00
Guilhem
e9784cfa11 fix(frontend): bad overflow handling for flow schema in detail page (#7704)
* Fix bad overflow handling for flow schema

* nit
2026-01-27 17:00:50 +00:00
centdix
c548e52949 fix(cli): make wmill app new respects nonDottedPaths setting from wmill.yaml (#7700)
* update dnt

* fix nondottedpath settings
2026-01-27 15:14:42 +00:00
135 changed files with 11948 additions and 1626 deletions

View File

@@ -1,39 +1,4 @@
{
"hooks": {
"UserPromptSubmit": [
{
"hooks": [
{
"type": "command",
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/resolve-symlinks.sh",
"timeout": 30
}
]
}
],
"Stop": [
{
"hooks": [
{
"type": "command",
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/restore-symlinks.sh",
"timeout": 30
}
]
}
],
"SessionEnd": [
{
"hooks": [
{
"type": "command",
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/restore-symlinks.sh",
"timeout": 30
}
]
}
]
},
"permissions": {
"allow": [
"Bash(ls:*)",
@@ -93,8 +58,45 @@
]
},
"enableAllProjectMcpServers": true,
"hooks": {
"UserPromptSubmit": [
{
"hooks": [
{
"type": "command",
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/resolve-symlinks.sh",
"timeout": 30
}
]
}
],
"Stop": [
{
"hooks": [
{
"type": "command",
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/restore-symlinks.sh",
"timeout": 30
}
]
}
],
"SessionEnd": [
{
"hooks": [
{
"type": "command",
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/restore-symlinks.sh",
"timeout": 30
}
]
}
]
},
"enabledPlugins": {
"rust-analyzer-lsp@claude-plugins-official": true,
"typescript-lsp@claude-plugins-official": true
"typescript-lsp@claude-plugins-official": true,
"code-review@claude-plugins-official": true,
"commit-commands@claude-plugins-official": true
}
}

View File

@@ -0,0 +1,495 @@
---
name: rust-backend
description: Rust coding guidelines for the Windmill backend. Apply when writing or modifying Rust code in the backend directory.
---
# Rust Backend Coding Guidelines
Apply these patterns when writing or modifying Rust code in the `backend/` directory.
## Data Structure Design
Choose between `struct`, `enum`, or `newtype` based on domain needs:
- Use `enum` for state machines instead of boolean flags or loosely related fields
- Model invariants explicitly using types (e.g., `NonZeroU32`, `Duration`, custom enums)
- Consider ownership of each field:
- Use `&str` vs `String`, slices vs vectors
- Use `Arc<T>` when sharing across threads
- Use `Cow<'a, T>` for flexible ownership
```rust
// State machine with enum
enum JobState {
Pending { scheduled_for: DateTime<Utc> },
Running { started_at: DateTime<Utc>, worker: String },
Completed { result: JobResult, duration_ms: i64 },
Failed { error: String, retries: u32 },
}
// Avoid multiple booleans
struct Job {
is_pending: bool, // Don't do this
is_running: bool,
is_completed: bool,
}
```
## Impl Block Organization
Place `impl` blocks immediately below the struct/enum they modify. Group methods logically:
```rust
struct JobQueue {
jobs: Vec<Job>,
capacity: usize,
}
impl JobQueue {
// Constructors first
pub fn new(capacity: usize) -> Self { ... }
pub fn with_jobs(jobs: Vec<Job>) -> Self { ... }
// Getters
pub fn len(&self) -> usize { ... }
pub fn is_empty(&self) -> bool { ... }
// Mutation methods
pub fn push(&mut self, job: Job) -> Result<()> { ... }
pub fn pop(&mut self) -> Option<Job> { ... }
// Domain logic
pub fn next_scheduled(&self) -> Option<&Job> { ... }
}
```
## Iterator Chains Over For-Loops
Prefer functional iterator chains (`.filter().map().collect()`) over imperative for-loops:
```rust
// Preferred
let results: Vec<_> = items
.iter()
.filter(|item| item.is_valid())
.map(|item| item.transform())
.collect();
// Avoid
let mut results = Vec::new();
for item in items.iter() {
if item.is_valid() {
results.push(item.transform());
}
}
```
## Error Handling
Use the `Error` type from `windmill_common::error`. Return `Result<T, Error>` or `JsonResult<T>` for fallible functions:
```rust
use windmill_common::error::{Error, Result};
// Use ? operator for propagation
pub async fn get_job(db: &DB, id: Uuid) -> Result<Job> {
let job = sqlx::query_as!(Job, "SELECT ... WHERE id = $1", id)
.fetch_optional(db)
.await?
.ok_or_else(|| Error::NotFound("job not found".to_string()))?;
Ok(job)
}
```
Prefer `if let` for optional handling. Use `let...else` when early return makes code clearer:
```rust
let Some(config) = get_config() else {
return Err(Error::MissingConfig);
};
```
Never panic in library code. Reserve `.unwrap()` for cases with compile-time guarantees. Keep functions short to help lifetime inference and clarity.
## Early Returns
Return early to avoid deep nesting. Handle error cases and edge conditions first:
```rust
// Preferred - early returns
fn process_job(job: Option<Job>) -> Result<Output> {
let Some(job) = job else {
return Ok(Output::default());
};
if !job.is_valid() {
return Err(Error::InvalidJob);
}
if job.is_cached() {
return Ok(job.cached_result());
}
// Main logic at the end, not nested
execute_job(job)
}
// Avoid - deep nesting
fn process_job(job: Option<Job>) -> Result<Output> {
if let Some(job) = job {
if job.is_valid() {
if !job.is_cached() {
execute_job(job)
} else {
Ok(job.cached_result())
}
} else {
Err(Error::InvalidJob)
}
} else {
Ok(Output::default())
}
}
```
## Variable Shadowing
Shadow variables instead of creating new names with prefixes:
```rust
// Preferred
let data = fetch_raw_data();
let data = parse(data);
let data = validate(data)?;
// Avoid
let raw_data = fetch_raw_data();
let parsed_data = parse(raw_data);
let validated_data = validate(parsed_data)?;
```
## Minimal Comments
- No inline comments explaining obvious code
- No TODO/FIXME comments in committed code
- Doc comments (`///`) only on public items
- Let code be self-documenting through clear naming
## Type Safety
Use enums over boolean flags for clarity:
```rust
// Preferred
enum JobStatus {
Pending,
Running,
Completed,
}
// Avoid
struct Job {
is_running: bool,
is_completed: bool,
}
```
## Pattern Matching
Prefer explicit matching. Use wildcards strategically for fallback cases or ignored fields:
```rust
// Explicit matching preferred
match status {
JobStatus::Pending => handle_pending(),
JobStatus::Running => handle_running(),
JobStatus::Completed => handle_completed(),
}
// Wildcards OK for fallback
match result {
Ok(value) => process(value),
Err(_) => return default_value(),
}
// Wildcards OK for ignoring fields in destructuring
let Point { x, y, .. } = point;
```
## Destructuring in Function Signatures
Destructure structs directly in function parameters:
```rust
// Preferred
async fn process_job(
Extension(db): Extension<DB>,
Path((workspace, job_id)): Path<(String, Uuid)>,
Query(pagination): Query<Pagination>,
) -> Result<Json<Job>> {
// ...
}
// Avoid
async fn process_job(
db_ext: Extension<DB>,
path: Path<(String, Uuid)>,
query: Query<Pagination>,
) -> Result<Json<Job>> {
let Extension(db) = db_ext;
let Path((workspace, job_id)) = path;
// ...
}
```
## Trait Implementations
Use standard trait implementations to simplify conversions and reduce boilerplate:
```rust
// Implement From/Into for type conversions
impl From<DbJob> for ApiJob {
fn from(db: DbJob) -> Self {
ApiJob {
id: db.id,
status: db.status.into(),
}
}
}
// Use TryFrom for fallible conversions
impl TryFrom<String> for JobKind {
type Error = Error;
fn try_from(s: String) -> Result<Self, Self::Error> { ... }
}
```
Apply `derive` macros to reduce boilerplate:
```rust
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Job { ... }
```
## Module Structure
- Use `pub(crate)` instead of `pub` when possible; expose only what needs exposing
- Keep APIs small and expressive; avoid leaking internal types
- Organize code into modules reflecting ownership and domain boundaries
```rust
// Prefer restricted visibility
pub(crate) fn internal_helper() { ... }
// Only pub for external API
pub fn create_job(...) -> Result<Job> { ... }
```
## Code Navigation
Always use rust-analyzer LSP for:
- Go to definition
- Find references
- Type information
- Import resolution
Do not guess at module paths or type definitions.
## JSON Handling
Prefer `Box<serde_json::value::RawValue>` over `serde_json::Value` when:
- Storing JSON in the database (JSONB columns)
- Passing JSON through without modification
- The JSON structure doesn't need inspection
```rust
// Preferred - avoids parsing/serialization overhead
pub struct Job {
pub id: Uuid,
pub args: Option<Box<serde_json::value::RawValue>>,
}
// Only use Value when you need to inspect/modify JSON
let value: serde_json::Value = serde_json::from_str(&json)?;
if let Some(field) = value.get("field") {
// modify or inspect
}
```
## Serde Optimizations
Use serde attributes to optimize serialization:
```rust
#[derive(Serialize, Deserialize)]
pub struct Job {
#[serde(rename = "jobId")]
pub id: Uuid,
#[serde(default)]
pub priority: i32,
#[serde(skip_serializing_if = "Option::is_none")]
pub parent_job: Option<Uuid>,
#[serde(skip_serializing_if = "Vec::is_empty")]
pub tags: Vec<String>,
}
```
Prefer borrowing for zero-copy deserialization when lifetimes allow:
```rust
#[derive(Deserialize)]
pub struct JobInput<'a> {
#[serde(borrow)]
pub workspace_id: Cow<'a, str>,
#[serde(borrow)]
pub script_path: &'a str,
}
```
## SQLx Patterns
**Never use `SELECT *`** - always list columns explicitly. This is critical for backwards compatibility when workers run behind the API server version:
```rust
// Preferred - explicit columns
sqlx::query_as!(
Job,
"SELECT id, workspace_id, path, created_at FROM v2_job WHERE id = $1",
job_id
)
// Avoid - breaks when columns are added
sqlx::query_as!(Job, "SELECT * FROM v2_job WHERE id = $1", job_id)
```
Use batch operations to minimize round trips:
```rust
// Preferred - single query with multiple values
sqlx::query!(
"INSERT INTO job_logs (job_id, logs) VALUES ($1, $2), ($3, $4)",
id1, log1, id2, log2
)
// Avoid N+1 queries
for id in ids {
sqlx::query!("SELECT ... WHERE id = $1", id).fetch_one(db).await?;
}
// Preferred - single query with IN clause
sqlx::query!("SELECT ... WHERE id = ANY($1)", &ids[..]).fetch_all(db).await?
```
Use transactions for multi-step operations and parameterize all queries.
## Async & Tokio Patterns
Never block the async runtime. Use `spawn_blocking` for CPU-intensive or blocking I/O:
```rust
// Preferred - offload blocking work
let result = tokio::task::spawn_blocking(move || {
expensive_computation(&data)
}).await?;
// Avoid - blocks the runtime
let result = expensive_computation(&data); // Don't do this in async
```
Use tokio primitives for sleep and channels:
```rust
use tokio::sync::mpsc;
use tokio::time::sleep;
// Avoid in async contexts
use std::thread::sleep; // Blocks the runtime
```
Use bounded channels for backpressure:
```rust
// Preferred - bounded channel prevents overwhelming
let (tx, rx) = tokio::sync::mpsc::channel(100);
// Be careful with unbounded
let (tx, rx) = tokio::sync::mpsc::unbounded_channel();
```
## Mutex Selection in Async Code
**Prefer `std::sync::Mutex` (or `parking_lot::Mutex`) over `tokio::sync::Mutex`** for protecting data in async code. The async mutex is more expensive and only needed when holding locks across `.await` points.
```rust
// Preferred for data protection - std mutex is faster
use std::sync::Mutex;
struct Cache {
data: Mutex<HashMap<String, Value>>,
}
impl Cache {
fn get(&self, key: &str) -> Option<Value> {
self.data.lock().unwrap().get(key).cloned()
}
fn insert(&self, key: String, value: Value) {
self.data.lock().unwrap().insert(key, value);
}
}
```
**Use `tokio::sync::Mutex` only when you must hold the lock across `.await` points**, typically for IO resources like database connections:
```rust
use tokio::sync::Mutex;
use std::sync::Arc;
// Async mutex for IO resources held across await points
let conn = Arc::new(Mutex::new(db_connection));
async fn execute_query(conn: Arc<Mutex<DbConn>>, query: &str) {
let mut lock = conn.lock().await;
lock.execute(query).await; // Lock held across .await
}
```
**Common pattern**: Wrap `Arc<Mutex<...>>` in a struct with non-async methods that lock internally, keeping lock scope minimal:
```rust
struct SharedState {
inner: std::sync::Mutex<StateInner>,
}
impl SharedState {
fn update(&self, value: i32) {
self.inner.lock().unwrap().value = value;
}
fn get(&self) -> i32 {
self.inner.lock().unwrap().value
}
}
```
**Alternative for IO resources**: Spawn a dedicated task to manage the resource and communicate via message passing:
```rust
let (tx, mut rx) = tokio::sync::mpsc::channel(32);
tokio::spawn(async move {
while let Some(cmd) = rx.recv().await {
handle_io_command(&mut resource, cmd).await;
}
});
```
## Build & Tooling
Build speed tips:
- Use `cargo check` during rapid iteration over `cargo build`
- Minimize unnecessary dependencies and feature flags

View File

@@ -79,6 +79,17 @@ jobs:
with:
node-version: '20'
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Symlink Bun to /usr/bin/bun
run: sudo ln -sf $(which bun) /usr/bin/bun
- name: Symlink Node to /usr/bin/node
run: sudo ln -sf $(which node) /usr/bin/node
- name: Generate Windmill clients
working-directory: cli
run: |
@@ -125,6 +136,20 @@ jobs:
with:
node-version: '20'
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Get Bun and Node paths
id: runtime-paths
shell: pwsh
run: |
$bunPath = (Get-Command bun).Source
$nodePath = (Get-Command node).Source
echo "BUN_PATH=$bunPath" >> $env:GITHUB_OUTPUT
echo "NODE_BIN_PATH=$nodePath" >> $env:GITHUB_OUTPUT
- name: Generate Windmill clients
working-directory: cli
shell: bash
@@ -138,6 +163,8 @@ jobs:
env:
DATABASE_URL: postgres://postgres:changeme@localhost:5432
CI_MINIMAL_FEATURES: "true"
BUN_PATH: ${{ steps.runtime-paths.outputs.BUN_PATH }}
NODE_BIN_PATH: ${{ steps.runtime-paths.outputs.NODE_BIN_PATH }}
run: |
deno test --no-check --allow-all test/ `
--ignore=test/cargo_backend_example.test.ts

View File

@@ -67,7 +67,7 @@ jobs:
platforms: linux/amd64,linux/arm64
push: true
build-args: |
features=embedding,parquet,openidconnect,license,http_trigger,zip,oauth2,postgres_trigger,mqtt_trigger,websocket,smtp,static_frontend,all_languages,deno_core,mcp
features=embedding,parquet,openidconnect,license,http_trigger,zip,oauth2,postgres_trigger,mqtt_trigger,websocket,smtp,native_trigger,static_frontend,all_languages,deno_core,mcp
tags: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:dev
${{ steps.meta-public.outputs.tags }}

View File

@@ -97,7 +97,7 @@ jobs:
platforms: linux/amd64,linux/arm64
push: true
build-args: |
features=embedding,parquet,openidconnect,jemalloc,license,http_trigger,zip,oauth2,dind,postgres_trigger,mqtt_trigger,websocket,smtp,static_frontend,agent_worker_server,all_languages,deno_core,mcp,bedrock,private
features=embedding,parquet,openidconnect,jemalloc,license,http_trigger,zip,oauth2,dind,postgres_trigger,mqtt_trigger,websocket,smtp,native_trigger,static_frontend,agent_worker_server,all_languages,deno_core,mcp,bedrock,private
tags: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ env.DEV_SHA }}
${{ steps.meta-public.outputs.tags }}

View File

@@ -1,5 +1,116 @@
# Changelog
## [1.621.2](https://github.com/windmill-labs/windmill/compare/v1.621.1...v1.621.2) (2026-01-29)
### Bug Fixes
* **cli:** revert findCodebase change that broke ../shared codebases ([#7740](https://github.com/windmill-labs/windmill/issues/7740)) ([20357f4](https://github.com/windmill-labs/windmill/commit/20357f41f55ce246220ec56ef257ea7d6ac82e3a))
* do not quit indexer when receiving handoff during pull ([#7659](https://github.com/windmill-labs/windmill/issues/7659)) ([8bb6b63](https://github.com/windmill-labs/windmill/commit/8bb6b6331b74d43b1ecfa08d3393254f54a94f87))
## [1.621.1](https://github.com/windmill-labs/windmill/compare/v1.621.0...v1.621.1) (2026-01-29)
### Bug Fixes
* add 32MB memory limit to QuickJS runtime for flow expressions ([db74470](https://github.com/windmill-labs/windmill/commit/db74470ec355ae317a50f350133fc140d2921595))
## [1.621.0](https://github.com/windmill-labs/windmill/compare/v1.620.1...v1.621.0) (2026-01-29)
### Features
* add QuickJS as alternative JS engine for flow expression evaluation ([#7664](https://github.com/windmill-labs/windmill/issues/7664)) ([5c20b37](https://github.com/windmill-labs/windmill/commit/5c20b37a537bae09ce13ef133ac12fd5976d9c37))
### Bug Fixes
* return null for non-existent step access in flow expressions ([22cce51](https://github.com/windmill-labs/windmill/commit/22cce51db55fe2b08a4f38cbddf98c12c861542d))
## [1.620.1](https://github.com/windmill-labs/windmill/compare/v1.620.0...v1.620.1) (2026-01-28)
### Bug Fixes
* codebase preview in standalone mode ([c59699a](https://github.com/windmill-labs/windmill/commit/c59699acd73aa170d5bd65d903db6b635c19f9ad))
## [1.620.0](https://github.com/windmill-labs/windmill/compare/v1.619.0...v1.620.0) (2026-01-28)
### Features
* **cli:** add script preview and flow preview commands ([#7729](https://github.com/windmill-labs/windmill/issues/7729)) ([95cbb2c](https://github.com/windmill-labs/windmill/commit/95cbb2c86ce66abd8e5488400b2367a22237e8c7))
### Bug Fixes
* cache git branch detection to avoid repeated execSync calls ([eafee16](https://github.com/windmill-labs/windmill/commit/eafee16bfc66081a0d1d575020fc4e40c76feb8a))
## [1.619.0](https://github.com/windmill-labs/windmill/compare/v1.618.2...v1.619.0) (2026-01-28)
### Features
* enable tree-shaking for windmill-client ([b6abcc3](https://github.com/windmill-labs/windmill/commit/b6abcc33a121423faaa41d9eef5488df67686fe7))
### Bug Fixes
* **backend:** leave job and audit history and archive workspace when changing workspace id ([#7724](https://github.com/windmill-labs/windmill/issues/7724)) ([d3d35d4](https://github.com/windmill-labs/windmill/commit/d3d35d4cd86dc73a4e2e007f457bc945ccef8263))
* **cli:** handle symlinks in isMain() for Node.js ([116b9e7](https://github.com/windmill-labs/windmill/commit/116b9e7db38cd0a9ec2a5c5780a9004ff2015a02))
* fix TypeScript default export for Monaco/ATA compatibility ([a02938c](https://github.com/windmill-labs/windmill/commit/a02938c80c425b5964e815722be9919ea405234b))
* make api key optional ([#7726](https://github.com/windmill-labs/windmill/issues/7726)) ([82f378b](https://github.com/windmill-labs/windmill/commit/82f378bcb4d29f5c272c70564d30814542115fed))
* nativets http tracing ([#7716](https://github.com/windmill-labs/windmill/issues/7716)) ([f50a866](https://github.com/windmill-labs/windmill/commit/f50a866430da8f5f43cb3163ec116fe254407ef9))
* Raw apps deployment UI (and merge UI) ([#7725](https://github.com/windmill-labs/windmill/issues/7725)) ([36dad2c](https://github.com/windmill-labs/windmill/commit/36dad2c7a29e4880bdf0198611e07a4366b01edf))
* use tsc for clean .d.ts files instead of tsdown bundled types ([0f62558](https://github.com/windmill-labs/windmill/commit/0f625580f37e562240bdaa155e8b25e889bb680d))
## [1.618.2](https://github.com/windmill-labs/windmill/compare/v1.618.1...v1.618.2) (2026-01-28)
### Bug Fixes
* add default export to typescript-client for ESM compatibility ([e7ac7af](https://github.com/windmill-labs/windmill/commit/e7ac7afe8e2af7c30c225b2031a894bfcb1783c8))
## [1.618.1](https://github.com/windmill-labs/windmill/compare/v1.618.0...v1.618.1) (2026-01-28)
### Bug Fixes
* handle empty base_url and region strings in AI providers ([#7719](https://github.com/windmill-labs/windmill/issues/7719)) ([7cd51de](https://github.com/windmill-labs/windmill/commit/7cd51def2b89efc117f5add7c9f8d92caa1f782d))
## [1.618.0](https://github.com/windmill-labs/windmill/compare/v1.617.3...v1.618.0) (2026-01-28)
### Features
* typescript client esm build ([#7709](https://github.com/windmill-labs/windmill/issues/7709)) ([07fb47e](https://github.com/windmill-labs/windmill/commit/07fb47e215da2b36afb83529c6ae84bf8fa14ae6))
### Bug Fixes
* fix annoying abort toasts ([#7713](https://github.com/windmill-labs/windmill/issues/7713)) ([b76d6e9](https://github.com/windmill-labs/windmill/commit/b76d6e9be80030597e82cd5db5cb5267de3b2961))
* fix flow viewer height ([#7715](https://github.com/windmill-labs/windmill/issues/7715)) ([e37ab33](https://github.com/windmill-labs/windmill/commit/e37ab33b3f31a67da6238eb0827a46b9d0d831c8))
## [1.617.3](https://github.com/windmill-labs/windmill/compare/v1.617.2...v1.617.3) (2026-01-27)
### Bug Fixes
* **backend:** include empty schemas in list_datatable_schemas endpoint ([#7708](https://github.com/windmill-labs/windmill/issues/7708)) ([705bc48](https://github.com/windmill-labs/windmill/commit/705bc481312bcadc514d949b0b6cec6e95bdf856))
* **cli:** make `wmill app lint` and `wmill app generate-agents` respect nonDottedPaths setting ([#7706](https://github.com/windmill-labs/windmill/issues/7706)) ([abe6cc4](https://github.com/windmill-labs/windmill/commit/abe6cc49b93804b0706d97865c9bd5ff60f08906))
* do not delete tokens on being promoted to superadmins ([564d826](https://github.com/windmill-labs/windmill/commit/564d8266dcc87b0b63b09a99c5bf71ef64b64369))
## [1.617.2](https://github.com/windmill-labs/windmill/compare/v1.617.1...v1.617.2) (2026-01-27)
### Bug Fixes
* 404 triggers listing in CE ([#7705](https://github.com/windmill-labs/windmill/issues/7705)) ([456dd47](https://github.com/windmill-labs/windmill/commit/456dd478d83c1c57be2756bd8a201eb73fe43542))
* **backend:** folder/group permissions workspace id change ([#7703](https://github.com/windmill-labs/windmill/issues/7703)) ([4ef1616](https://github.com/windmill-labs/windmill/commit/4ef16168936d8f908a25a55965f9d7998ec68625))
* **cli:** make `wmill app new` respects nonDottedPaths setting from wmill.yaml ([#7700](https://github.com/windmill-labs/windmill/issues/7700)) ([c548e52](https://github.com/windmill-labs/windmill/commit/c548e529491a9547076af6b4567b9ce8909b07a5))
* **frontend:** bad overflow handling for flow schema in detail page ([#7704](https://github.com/windmill-labs/windmill/issues/7704)) ([e9784cf](https://github.com/windmill-labs/windmill/commit/e9784cfa11010d229f520558e6974b2f3dded6d9))
* **mcp:** use computed base_internal_url instead of static default ([#7701](https://github.com/windmill-labs/windmill/issues/7701)) ([720a7e5](https://github.com/windmill-labs/windmill/commit/720a7e56d1f86040173b3d49519a925bf649fb71))
## [1.617.1](https://github.com/windmill-labs/windmill/compare/v1.617.0...v1.617.1) (2026-01-27)

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT flow_version.value AS \"value!: sqlx::types::Json<Box<sqlx::types::JsonRawValue>>\" \n FROM flow \n LEFT JOIN flow_version \n ON flow_version.id = flow.versions[array_upper(flow.versions, 1)]\n WHERE flow.path = $1 AND flow.workspace_id = $2",
"query": "SELECT flow_version.value AS \"value!: sqlx::types::Json<Box<sqlx::types::JsonRawValue>>\"\n FROM flow\n LEFT JOIN flow_version\n ON flow_version.id = flow.versions[array_upper(flow.versions, 1)]\n WHERE flow.path = $1 AND flow.workspace_id = $2",
"describe": {
"columns": [
{
@@ -19,5 +19,5 @@
false
]
},
"hash": "bbce3e1eae78c48409d4204cd6cb3b9db088f6e51bea5e74a494c4e9f4c3b78e"
"hash": "02bf9763298f301d4fc75490c070a0663142d4d23a2df007361622b94d4783e1"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE folder SET workspace_id = $1 WHERE workspace_id = $2",
"query": "UPDATE mcp_oauth_server_code SET workspace_id = $1 WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
@@ -11,5 +11,5 @@
},
"nullable": []
},
"hash": "5876d8271adcb75752d518fc52ac192e162469a504338b4a0a37e8bcec114385"
"hash": "0cfb1528c3636dd1f43c41b91aa340862ed795f96870dd9ec999ea7e9373ec51"
}

View File

@@ -1,14 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM token WHERE email = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text"
]
},
"nullable": []
},
"hash": "192ddae8c3c82a8f099a4944483024d9826a328bf0416c22daf06fff5ced08f6"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE folder_permission_history SET workspace_id = $1 WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "3895fbafb771583c64cca560cf4c2d10811e6e07c38d621e86e90b34b709f99f"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE mcp_oauth_refresh_token SET workspace_id = $1 WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "452fe403cef5a62cb34b9996794c1607757eecba9d2d0326b09d6fcd4dc45c12"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE usr SET workspace_id = $1 WHERE workspace_id = $2",
"query": "UPDATE ai_agent_memory SET workspace_id = $1 WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
@@ -11,5 +11,5 @@
},
"nullable": []
},
"hash": "0659bab15d4cccdb04c7a57e0e3bbb6bfebb8896601a27ddf5618d4eae678bc1"
"hash": "4c8d3693059ce1e2bbc84d76b543830ed343a5c6a1fef780f477dfbed80300f3"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO folder SELECT name, $1, display_name, owners, extra_perms, summary, edited_at, created_by FROM folder WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "50b25537dcb799cc233dbb06c76798a860ce977954a856335bb62ae00f615659"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE audit SET workspace_id = $1 WHERE workspace_id = $2",
"query": "UPDATE flow_conversation SET workspace_id = $1 WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
@@ -11,5 +11,5 @@
},
"nullable": []
},
"hash": "0ba594244a366a31d9bed97a2d7b031d42c23463599d267d1712d1af1d26b321"
"hash": "642b6c2c55c19f554a0c4e8dcc9ddbeec6327a8d87d6a45d6c0823ec42c65639"
}

View File

@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE job_stats SET workspace_id = $1 WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "65121a4bfba70c2d7055a2b58c8520fddaef1bd9a3f041e851f9c136c73d34e7"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO flow\n (workspace_id, path, summary, description, archived, extra_perms, dependency_job, draft_only, tag, ws_error_handler_muted, dedicated_worker, timeout, visible_to_runner_only, on_behalf_of_email, concurrency_key, versions, value, schema, edited_by, edited_at, lock_error_logs)\n SELECT $1, path, summary, description, archived, extra_perms, dependency_job, draft_only, tag, ws_error_handler_muted, dedicated_worker, timeout, visible_to_runner_only, on_behalf_of_email, concurrency_key, versions, value, schema, edited_by, edited_at, lock_error_logs\n FROM flow WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "65c2ecb52cc777f17ffeb77be597ea87026bb4e56c0ccfb12f2feaf1a6124c86"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE v2_job SET workspace_id = $1\n WHERE workspace_id = $2\n AND id IN (SELECT id FROM v2_job_queue WHERE workspace_id = $1)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": []
},
"hash": "681bab96dce7249eda1d0d207513f0c4e8ba950bf79e9bd1cf2618a322be7858"
}

View File

@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE v2_job_completed SET workspace_id = $1 WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "6bdb3fcfe16fc40222dc7010a11026d4d4e0d381b31fe02da7d2667c0cdc1a85"
}

View File

@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT path FROM schedule WHERE workspace_id = $1 AND enabled = true",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "path",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false
]
},
"hash": "739501f637dac4e5f4843e3ebfc30aa94729cfdb307c7f56584db8edcfd48e42"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_settings SELECT $1, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, deploy_to, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, teams_command_script, teams_team_id, teams_team_name, git_app_installations, ducklake, slack_oauth_client_id, slack_oauth_client_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler FROM workspace_settings WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "7c45f8d05a10ccf538c1b63aa1337e6d0491a2e8d04fe87eddb5a573de00d125"
}

View File

@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE job_logs SET workspace_id = $1 WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "7e90578bb37b2923cd94c201f1818c8c5b0bd62c9555949dda2e6aba3e09a4f0"
}

View File

@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO flow \n (workspace_id, path, summary, description, archived, extra_perms, dependency_job, draft_only, tag, ws_error_handler_muted, dedicated_worker, timeout, visible_to_runner_only, on_behalf_of_email, concurrency_key, versions, value, schema, edited_by, edited_at) \n SELECT $1, path, summary, description, archived, extra_perms, dependency_job, draft_only, tag, ws_error_handler_muted, dedicated_worker, timeout, visible_to_runner_only, on_behalf_of_email, concurrency_key, versions, value, schema, edited_by, edited_at\n FROM flow WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "81a9d976ac5c1a78c83b95a1164995e78878a4a4ff6894a04e6626cdd98c24e4"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE workspace_diff SET source_workspace_id = $1 WHERE source_workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "81e87e212075159270ff6c2811e93f31e941e71b9addd1e915fe779f81a6ae43"
}

View File

@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace SELECT $1, $2, owner, false, premium FROM workspace WHERE id = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "82185eb02e03e3dd1a4b5a3f22c3b60169989703ee33d14ab348301885c9d745"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE workspace_key SET workspace_id = $1 WHERE workspace_id = $2",
"query": "UPDATE email_trigger SET workspace_id = $1 WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
@@ -11,5 +11,5 @@
},
"nullable": []
},
"hash": "0d0c379b1cd2eec15869dd0b1a31886a95d53096fdcb1cdb1e0eb282b54105dc"
"hash": "863581331dc7abd9ffa47c6977ee94d939a9e3ab3921605e5b0f4f7586e431c2"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO skip_workspace_diff_tally SELECT $1, added_at FROM skip_workspace_diff_tally WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "893c2dea38bfff42ad8cbd7afaf0280a8c1d6a7f1dc99f14c894899031534bea"
}

View File

@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE job_perms SET workspace_id = $1\n WHERE job_id IN (SELECT id FROM v2_job WHERE workspace_id = $1)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text"
]
},
"nullable": []
},
"hash": "8b8d29f3133228dbc5bdf1927e06dc1adfdaeed3f8dfc9d44887347f82f2d520"
}

View File

@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE workspace_settings SET workspace_id = $1 WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "93e6cdf58803f63d7d465f9d778c052c40dcdacade0f9b9921860160604f3763"
}

View File

@@ -0,0 +1,24 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO worker_ping (worker_instance, worker, ip, custom_tags, worker_group, dedicated_worker, dedicated_workers, wm_version, vcpus, memory, job_isolation) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11) ON CONFLICT (worker)\n DO UPDATE set ip = EXCLUDED.ip, custom_tags = EXCLUDED.custom_tags, worker_group = EXCLUDED.worker_group, dedicated_workers = EXCLUDED.dedicated_workers",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar",
"TextArray",
"Varchar",
"Varchar",
"TextArray",
"Varchar",
"Int8",
"Int8",
"Text"
]
},
"nullable": []
},
"hash": "97c61b6a9a5112ea484565236959a544511d5d501fb737da8110a8725b883465"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE workspace_integrations SET workspace_id = $1 WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "98034dc06e7478f4a828ea58cf9bbe728f4eabcd6a6e7ad7c8445efb6966e0c9"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO usr SELECT $1, username, email, is_admin, created_at, operator, disabled, role FROM usr WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "b3f791c8ef04f0aefd9b510d751dbe467e17e15d4a6512889009d850760502d9"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_key SELECT $1, kind, key FROM workspace_key WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "b928974804d710f37c4703a23c67440bcb4733ff706669abc787cd414c11f82e"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE token SET super_admin = $1 WHERE email = $2 AND label != 'session'",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Bool",
"Text"
]
},
"nullable": []
},
"hash": "be303445868662af7b7475f19dc630668a2776fc7a253a6e747653338d10dbd0"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE group_permission_history SET workspace_id = $1 WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "c9ffc9fcfe5e550af53fa6a4706876a09e38996471301d5c7fffaeb1d813d3bd"
}

View File

@@ -1,16 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace SELECT $1, $2, owner, deleted, premium FROM workspace WHERE id = $3",
"query": "UPDATE native_trigger SET workspace_id = $1 WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "8f0031533f1bf407bd5d8af4d364eaf00d4c38ee7ba75141b40fc9fcd2ffc0b8"
"hash": "e1e55ce8c28ac4d253c289e4c972ae00cfdcc3ef056625a5a5bef76377953278"
}

View File

@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE v2_job SET workspace_id = $1\n WHERE workspace_id = $2\n AND (id IN (SELECT id FROM v2_job_queue WHERE workspace_id = $1)\n OR id IN (SELECT id FROM v2_job_completed WHERE workspace_id = $1))",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": []
},
"hash": "e2f7a1a48bb026df5c7155b76e166cd5a6ca2ab25926f70095328d5ea2d2bc7c"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE workspace_dependencies SET workspace_id = $1 WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "e3491d953cb9f85a5e3a9b1a386dfde77f0387d1ed8f5defa34359147e9c0907"
}

View File

@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT\n nt.external_id,\n nt.workspace_id,\n nt.service_name AS \"service_name!: ServiceName\",\n nt.script_path,\n nt.is_flow,\n nt.webhook_token_prefix,\n nt.service_config,\n nt.error,\n nt.created_at,\n nt.updated_at\n FROM\n native_trigger nt\n WHERE\n nt.workspace_id = $1 AND\n nt.service_name = $2 AND\n (\n (nt.is_flow = false AND EXISTS (\n SELECT 1 FROM script s\n WHERE s.workspace_id = nt.workspace_id\n AND s.path = nt.script_path\n ))\n OR\n (nt.is_flow = true AND EXISTS (\n SELECT 1 FROM flow f\n WHERE f.workspace_id = nt.workspace_id\n AND f.path = nt.script_path\n ))\n )\n LIMIT $3\n OFFSET $4\n ",
"query": "\n SELECT\n nt.external_id,\n nt.workspace_id,\n nt.service_name AS \"service_name!: ServiceName\",\n nt.script_path,\n nt.is_flow,\n nt.webhook_token_prefix,\n nt.service_config,\n nt.error,\n nt.created_at,\n nt.updated_at\n FROM\n native_trigger nt\n WHERE\n nt.workspace_id = $1 AND\n nt.service_name = $2 AND\n ($5::text IS NULL OR nt.script_path = $5) AND\n ($6::bool IS NULL OR nt.is_flow = $6) AND\n (\n (nt.is_flow = false AND EXISTS (\n SELECT 1 FROM script s\n WHERE s.workspace_id = nt.workspace_id\n AND s.path = nt.script_path\n ))\n OR\n (nt.is_flow = true AND EXISTS (\n SELECT 1 FROM flow f\n WHERE f.workspace_id = nt.workspace_id\n AND f.path = nt.script_path\n ))\n )\n LIMIT $3\n OFFSET $4\n ",
"describe": {
"columns": [
{
@@ -77,7 +77,9 @@
}
},
"Int8",
"Int8"
"Int8",
"Text",
"Bool"
]
},
"nullable": [
@@ -93,5 +95,5 @@
false
]
},
"hash": "8701ec4c8a9cd5ec71093483516c864f6b5b26f644d75a35fa732346b013e270"
"hash": "ecab1af12a7afa685c056b9d0e526275203fc8ecddf83ca6d05c9fb77e46e7ee"
}

View File

@@ -1,23 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO worker_ping (worker_instance, worker, ip, custom_tags, worker_group, dedicated_worker, wm_version, vcpus, memory, job_isolation) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10) ON CONFLICT (worker)\n DO UPDATE set ip = EXCLUDED.ip, custom_tags = EXCLUDED.custom_tags, worker_group = EXCLUDED.worker_group",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar",
"TextArray",
"Varchar",
"Varchar",
"Varchar",
"Int8",
"Int8",
"Text"
]
},
"nullable": []
},
"hash": "ed90b9fcc57f530bab2d2d7426795bb358c91ebdd9dab50d7e3d2fdce63b947c"
}

View File

@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE workspace_diff SET fork_workspace_id = $1 WHERE fork_workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "f23a296f5e04dcf6f2cda808193e4ee91c14823639cbcdb77d92409fe5218c3c"
}

View File

@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT COUNT(*) FROM v2_job WHERE workspace_id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "count",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
null
]
},
"hash": "f863cb7ec416d774dc4c8170b2f5ec974b035405da38bf3e7764f7e22232b760"
}

View File

@@ -1,22 +1,64 @@
# Backend Development (Rust)
## Core Principles
## Project Structure
- Follow @rust-best-practices.mdc for detailed guidelines
- Database schema reference: @summarized_schema.txt
- The API routes prefixes are all listed in windmill-api/src/lib.rs
- This repository is the open source side of the project. The enterprise files (\*\_ee.rs) are in the `windmill-ee-private` folder (a sibling directory). Those files are symlinked into their corresponding locations within each crate's `src/` directory.
Windmill uses a workspace-based architecture with multiple crates:
## JSON Handling
- **windmill-api**: API server functionality
- **windmill-worker**: Job execution
- **windmill-common**: Shared code used by all crates
- **windmill-queue**: Job & flow queuing
- **windmill-audit**: Audit logging
- Other specialized crates (git-sync, autoscaling, etc.)
- **Prefer `Box<serde_json::value::RawValue>` over `serde_json::Value`** when possible, especially:
- When storing JSON in the database (JSONB columns)
- When passing JSON through without modification
- When the JSON structure doesn't need to be inspected or manipulated
- This avoids unnecessary parsing/serialization overhead and preserves the original JSON format
- Use `serde_json::Value` only when you need to inspect, modify, or construct JSON programmatically
## Key References
## Adding New Features
- Database schema: @summarized_schema.txt
- API route prefixes: `windmill-api/src/lib.rs`
1. Update database schema with migration if necessary
2. Update backend/windmill-api/openapi.yaml after modifying API endpoints
## Adding New Code
### Module Organization
- Place new code in the appropriate crate based on functionality
- For API endpoints, create or modify files in `windmill-api/src/` organized by domain
- For shared functionality, use `windmill-common/src/`
- Follow existing patterns for file structure and organization
### API Endpoints
- Follow existing patterns in the `windmill-api` crate
- Use axum's routing system and extractors
- Update `backend/windmill-api/openapi.yaml` after modifying API endpoints
### Database Changes
- Update database schema with migration if necessary
- Use `sqlx` for database operations with prepared statements
- Use transactions for multi-step operations
## Enterprise Features
- Enterprise files use the `*_ee.rs` suffix
- Enterprise source is in `windmill-ee-private` folder (sibling directory), symlinked into each crate's `src/`
- Use feature flags: `#[cfg(feature = "enterprise")]`
- Isolate enterprise code in separate modules
## Testing
- Write unit tests for core functionality
- Use the `#[cfg(test)]` module for test code
- For database tests, use the existing test utilities
## Common Crates
- **tokio**: Async runtime
- **axum**: Web server and routing
- **sqlx**: Database operations
- **serde**: Serialization/deserialization
- **tracing**: Logging and diagnostics
- **reqwest**: HTTP client
## Coding Guidelines
Detailed Rust coding patterns and best practices are provided by the `rust-backend` skill.

188
backend/Cargo.lock generated
View File

@@ -1117,9 +1117,9 @@ dependencies = [
[[package]]
name = "aws-smithy-async"
version = "1.2.7"
version = "1.2.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9ee19095c7c4dda59f1697d028ce704c24b2d33c6718790c7f1d5a3015b4107c"
checksum = "9330762ee48c6cecfad2cb37b1506c16c8e858c90638eda2b1a7272b56f88bd5"
dependencies = [
"futures-util",
"pin-project-lite",
@@ -1128,9 +1128,9 @@ dependencies = [
[[package]]
name = "aws-smithy-eventstream"
version = "0.60.14"
version = "0.60.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dc12f8b310e38cad85cf3bef45ad236f470717393c613266ce0a89512286b650"
checksum = "0810b22ae554f5076c3eabe1fe89b01aee61c354c575789f67e248e83c5f472b"
dependencies = [
"aws-smithy-types",
"bytes",
@@ -1161,9 +1161,9 @@ dependencies = [
[[package]]
name = "aws-smithy-http-client"
version = "1.1.5"
version = "1.1.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "59e62db736db19c488966c8d787f52e6270be565727236fd5579eaa301e7bc4a"
checksum = "ec918f18147cec121cb142a91b0038f66d99bbe903e585dccf871920e90b22ab"
dependencies = [
"aws-smithy-async",
"aws-smithy-runtime-api",
@@ -1200,18 +1200,18 @@ dependencies = [
[[package]]
name = "aws-smithy-observability"
version = "0.2.0"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ef1fcbefc7ece1d70dcce29e490f269695dfca2d2bacdeaf9e5c3f799e4e6a42"
checksum = "a700a7702874cd78b85fecdc9f64f3f72eb22fb713791cb445bcfd2a15bc1ecf"
dependencies = [
"aws-smithy-runtime-api",
]
[[package]]
name = "aws-smithy-query"
version = "0.60.9"
version = "0.60.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ae5d689cf437eae90460e944a58b5668530d433b4ff85789e69d2f2a556e057d"
checksum = "adc4a6cdc289a37be7fddb7f4365448187d62c603a40e6d46d13c68e5e81900f"
dependencies = [
"aws-smithy-types",
"urlencoding",
@@ -1243,9 +1243,9 @@ dependencies = [
[[package]]
name = "aws-smithy-runtime-api"
version = "1.10.0"
version = "1.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "efce7aaaf59ad53c5412f14fc19b2d5c6ab2c3ec688d272fd31f76ec12f44fb0"
checksum = "5c47b1e62accf759b01aba295e40479d1ba8fb77c2a54f0fed861c809ca49761"
dependencies = [
"aws-smithy-async",
"aws-smithy-types",
@@ -1260,9 +1260,9 @@ dependencies = [
[[package]]
name = "aws-smithy-types"
version = "1.3.6"
version = "1.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "65f172bcb02424eb94425db8aed1b6d583b5104d4d5ddddf22402c661a320048"
checksum = "c2d447863bdec38c899e5753a48c0abcf590f3ec629e257ad5a9ef8806ad7714"
dependencies = [
"base64-simd 0.8.0",
"bytes",
@@ -1842,7 +1842,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0686c856aa6aac0c4498f936d7d6a02df690f614c03e4d906d1018062b5c5e2c"
dependencies = [
"once_cell",
"proc-macro-crate",
"proc-macro-crate 3.4.0",
"proc-macro2",
"quote",
"syn 2.0.114",
@@ -2265,9 +2265,9 @@ dependencies = [
[[package]]
name = "clap"
version = "4.5.54"
version = "4.5.56"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c6e6ff9dcd79cff5cd969a17a545d79e84ab086e444102a591e288a8aa3ce394"
checksum = "a75ca66430e33a14957acc24c5077b503e7d374151b2b4b3a10c83b4ceb4be0e"
dependencies = [
"clap_builder",
"clap_derive",
@@ -2275,9 +2275,9 @@ dependencies = [
[[package]]
name = "clap_builder"
version = "4.5.54"
version = "4.5.56"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fa42cf4d2b7a41bc8f663a7cab4031ebafa1bf3875705bfaf8466dc60ab52c00"
checksum = "793207c7fa6300a0608d1080b858e5fdbe713cdc1c8db9fb17777d8a13e63df0"
dependencies = [
"anstream",
"anstyle",
@@ -2287,9 +2287,9 @@ dependencies = [
[[package]]
name = "clap_derive"
version = "4.5.49"
version = "4.5.55"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2a0b5487afeab2deb2ff4e03a807ad1a03ac532ff5a2cee5d86884440c7f7671"
checksum = "a92793da1a46a5f2a02a6f4c46c6496b28c43638adea8306fcb0caa1634f24e5"
dependencies = [
"heck 0.5.0",
"proc-macro2",
@@ -7036,9 +7036,9 @@ dependencies = [
[[package]]
name = "iana-time-zone"
version = "0.1.64"
version = "0.1.65"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "33e57f83510bb73707521ebaffa789ec8caf86f9657cad665b092b581d40e9fb"
checksum = "e31bc9ad994ba00e440a8aa5c9ef0ec67d5cb5e5cb0cc7f8b744a35b389cc470"
dependencies = [
"android_system_properties",
"core-foundation-sys",
@@ -8550,7 +8550,7 @@ dependencies = [
"darling 0.20.11",
"heck 0.5.0",
"num-bigint",
"proc-macro-crate",
"proc-macro-crate 3.4.0",
"proc-macro-error2",
"proc-macro2",
"quote",
@@ -9119,7 +9119,7 @@ version = "0.7.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ff32365de1b6743cb203b710788263c44a03de03802daf96092f2da4fe6ba4d7"
dependencies = [
"proc-macro-crate",
"proc-macro-crate 3.4.0",
"proc-macro2",
"quote",
"syn 2.0.114",
@@ -9356,9 +9356,9 @@ checksum = "7c87def4c32ab89d880effc9e097653c8da5d6ef28e6b539d313baaacfbafcbe"
[[package]]
name = "openssl-src"
version = "300.5.4+3.5.4"
version = "300.5.5+3.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a507b3792995dae9b0df8a1c1e3771e8418b7c2d9f0baeba32e6fe8b06c7cb72"
checksum = "3f1787d533e03597a7934fd0a765f0d28e94ecc5fb7789f8053b1e699a56f709"
dependencies = [
"cc",
]
@@ -10192,6 +10192,16 @@ dependencies = [
"elliptic-curve",
]
[[package]]
name = "proc-macro-crate"
version = "1.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f4c021e1093a56626774e81216a4ce732a735e5bad4868a03f3ed65ca0c3919"
dependencies = [
"once_cell",
"toml_edit 0.19.15",
]
[[package]]
name = "proc-macro-crate"
version = "3.4.0"
@@ -10895,6 +10905,12 @@ version = "0.8.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7a2d987857b319362043e95f5353c0535c1f58eec5336fdfcf626430af7def58"
[[package]]
name = "relative-path"
version = "1.9.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ba39f3699c378cd8970968dcbff9c43159ea4cfbd88d43c00b22f2ef10a435d2"
[[package]]
name = "rend"
version = "0.4.2"
@@ -11185,6 +11201,53 @@ dependencies = [
"serde_derive",
]
[[package]]
name = "rquickjs"
version = "0.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d16661bff09e9ed8e01094a188b463de45ec0693ade55b92ed54027d7ba7c40c"
dependencies = [
"rquickjs-core",
"rquickjs-macro",
]
[[package]]
name = "rquickjs-core"
version = "0.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6c8db6379e204ef84c0811e90e7cc3e3e4d7688701db68a00d14a6db6849087b"
dependencies = [
"async-lock",
"relative-path",
"rquickjs-sys",
]
[[package]]
name = "rquickjs-macro"
version = "0.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6041104330c019fcd936026ae05e2446f5e8a2abef329d924f25424b7052a2f3"
dependencies = [
"convert_case 0.6.0",
"fnv",
"ident_case",
"indexmap 2.11.1",
"proc-macro-crate 1.3.1",
"proc-macro2",
"quote",
"rquickjs-core",
"syn 2.0.114",
]
[[package]]
name = "rquickjs-sys"
version = "0.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4bc352c6b663604c3c186c000cfcc6c271f4b50bc135a285dd6d4f2a42f9790a"
dependencies = [
"cc",
]
[[package]]
name = "rsa"
version = "0.9.10"
@@ -14580,9 +14643,9 @@ checksum = "6af6ae20167a9ece4bcb41af5b80f8a1f1df981f6391189ce00fd257af04126a"
[[package]]
name = "typed-path"
version = "0.12.1"
version = "0.12.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e43ffa54726cdc9ea78392023ffe9fe9cf9ac779e1c6fcb0d23f9862e3879d20"
checksum = "3015e6ce46d5ad8751e4a772543a30c7511468070e98e64e20165f8f81155b64"
[[package]]
name = "typeid"
@@ -15403,7 +15466,7 @@ checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
[[package]]
name = "windmill"
version = "1.617.1"
version = "1.621.2"
dependencies = [
"anyhow",
"aws-sdk-config",
@@ -15466,7 +15529,7 @@ dependencies = [
[[package]]
name = "windmill-api"
version = "1.617.1"
version = "1.621.2"
dependencies = [
"anyhow",
"argon2",
@@ -15596,7 +15659,7 @@ dependencies = [
[[package]]
name = "windmill-api-client"
version = "1.617.1"
version = "1.621.2"
dependencies = [
"reqwest 0.12.28",
"serde",
@@ -15606,7 +15669,7 @@ dependencies = [
[[package]]
name = "windmill-audit"
version = "1.617.1"
version = "1.621.2"
dependencies = [
"chrono",
"lazy_static",
@@ -15620,7 +15683,7 @@ dependencies = [
[[package]]
name = "windmill-autoscaling"
version = "1.617.1"
version = "1.621.2"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -15639,7 +15702,7 @@ dependencies = [
[[package]]
name = "windmill-common"
version = "1.617.1"
version = "1.621.2"
dependencies = [
"anyhow",
"async-recursion",
@@ -15735,7 +15798,7 @@ dependencies = [
[[package]]
name = "windmill-git-sync"
version = "1.617.1"
version = "1.621.2"
dependencies = [
"regex",
"serde",
@@ -15750,7 +15813,7 @@ dependencies = [
[[package]]
name = "windmill-indexer"
version = "1.617.1"
version = "1.621.2"
dependencies = [
"anyhow",
"astral-tokio-tar",
@@ -15774,7 +15837,7 @@ dependencies = [
[[package]]
name = "windmill-macros"
version = "1.617.1"
version = "1.621.2"
dependencies = [
"itertools 0.14.0",
"lazy_static",
@@ -15790,7 +15853,7 @@ dependencies = [
[[package]]
name = "windmill-mcp"
version = "1.617.1"
version = "1.621.2"
dependencies = [
"anyhow",
"async-trait",
@@ -15810,7 +15873,7 @@ dependencies = [
[[package]]
name = "windmill-oauth"
version = "1.617.1"
version = "1.621.2"
dependencies = [
"anyhow",
"async-oauth2",
@@ -15834,7 +15897,7 @@ dependencies = [
[[package]]
name = "windmill-parser"
version = "1.617.1"
version = "1.621.2"
dependencies = [
"convert_case 0.6.0",
"serde",
@@ -15843,7 +15906,7 @@ dependencies = [
[[package]]
name = "windmill-parser-bash"
version = "1.617.1"
version = "1.621.2"
dependencies = [
"anyhow",
"lazy_static",
@@ -15855,7 +15918,7 @@ dependencies = [
[[package]]
name = "windmill-parser-csharp"
version = "1.617.1"
version = "1.621.2"
dependencies = [
"anyhow",
"serde_json",
@@ -15867,7 +15930,7 @@ dependencies = [
[[package]]
name = "windmill-parser-go"
version = "1.617.1"
version = "1.621.2"
dependencies = [
"anyhow",
"gosyn",
@@ -15879,7 +15942,7 @@ dependencies = [
[[package]]
name = "windmill-parser-graphql"
version = "1.617.1"
version = "1.621.2"
dependencies = [
"anyhow",
"lazy_static",
@@ -15891,7 +15954,7 @@ dependencies = [
[[package]]
name = "windmill-parser-java"
version = "1.617.1"
version = "1.621.2"
dependencies = [
"anyhow",
"serde_json",
@@ -15903,7 +15966,7 @@ dependencies = [
[[package]]
name = "windmill-parser-nu"
version = "1.617.1"
version = "1.621.2"
dependencies = [
"anyhow",
"nu-parser",
@@ -15914,7 +15977,7 @@ dependencies = [
[[package]]
name = "windmill-parser-php"
version = "1.617.1"
version = "1.621.2"
dependencies = [
"anyhow",
"itertools 0.14.0",
@@ -15925,7 +15988,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py"
version = "1.617.1"
version = "1.621.2"
dependencies = [
"anyhow",
"itertools 0.14.0",
@@ -15938,7 +16001,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py-imports"
version = "1.617.1"
version = "1.621.2"
dependencies = [
"anyhow",
"async-recursion",
@@ -15962,7 +16025,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ruby"
version = "1.617.1"
version = "1.621.2"
dependencies = [
"anyhow",
"lazy_static",
@@ -15976,7 +16039,7 @@ dependencies = [
[[package]]
name = "windmill-parser-rust"
version = "1.617.1"
version = "1.621.2"
dependencies = [
"anyhow",
"convert_case 0.6.0",
@@ -15993,7 +16056,7 @@ dependencies = [
[[package]]
name = "windmill-parser-sql"
version = "1.617.1"
version = "1.621.2"
dependencies = [
"anyhow",
"lazy_static",
@@ -16007,7 +16070,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ts"
version = "1.617.1"
version = "1.621.2"
dependencies = [
"anyhow",
"lazy_static",
@@ -16026,7 +16089,7 @@ dependencies = [
[[package]]
name = "windmill-parser-yaml"
version = "1.617.1"
version = "1.621.2"
dependencies = [
"anyhow",
"serde",
@@ -16037,7 +16100,7 @@ dependencies = [
[[package]]
name = "windmill-queue"
version = "1.617.1"
version = "1.621.2"
dependencies = [
"anyhow",
"async-recursion",
@@ -16074,7 +16137,7 @@ dependencies = [
[[package]]
name = "windmill-sql-datatype-parser-wasm"
version = "1.617.1"
version = "1.621.2"
dependencies = [
"wasm-bindgen",
"wasm-bindgen-test",
@@ -16084,7 +16147,7 @@ dependencies = [
[[package]]
name = "windmill-worker"
version = "1.617.1"
version = "1.621.2"
dependencies = [
"anyhow",
"async-once-cell",
@@ -16156,6 +16219,7 @@ dependencies = [
"regex",
"reqwest 0.13.1",
"reqwest-middleware",
"rquickjs",
"rust_decimal",
"rustls-pemfile 2.2.0",
"serde",
@@ -16984,18 +17048,18 @@ dependencies = [
[[package]]
name = "zerocopy"
version = "0.8.34"
version = "0.8.35"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "71ddd76bcebeed25db614f82bf31a9f4222d3fbba300e6fb6c00afa26cbd4d9d"
checksum = "fdea86ddd5568519879b8187e1cf04e24fce28f7fe046ceecbce472ff19a2572"
dependencies = [
"zerocopy-derive",
]
[[package]]
name = "zerocopy-derive"
version = "0.8.34"
version = "0.8.35"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d8187381b52e32220d50b255276aa16a084ec0a9017a0ca2152a1f55c539758d"
checksum = "0c15e1b46eff7c6c91195752e0eeed8ef040e391cdece7c25376957d5f15df22"
dependencies = [
"proc-macro2",
"quote",

View File

@@ -1,6 +1,6 @@
[package]
name = "windmill"
version = "1.617.1"
version = "1.621.2"
authors.workspace = true
edition.workspace = true
@@ -35,7 +35,7 @@ members = [
exclude = ["./windmill-duckdb-ffi-internal"]
[workspace.package]
version = "1.617.1"
version = "1.621.2"
authors = ["Ruben Fiszel <ruben@windmill.dev>"]
edition = "2021"
@@ -68,6 +68,7 @@ jemalloc = ["windmill-common/jemalloc", "dep:tikv-jemallocator", "dep:tikv-jemal
tantivy = ["dep:windmill-indexer", "windmill-api/tantivy", "windmill-indexer/enterprise", "windmill-indexer/parquet", "windmill-common/tantivy", "enterprise", "parquet"]
sqlx = ["windmill-worker/sqlx"]
deno_core = ["windmill-worker/deno_core", "windmill-api/deno_core", "dep:deno_core", "dep:v8"]
quickjs = ["windmill-worker/quickjs"]
deno_core_mac = ["deno_core", "windmill-worker/libffi_mac"]
kafka = ["windmill-api/kafka"]
nats = ["windmill-api/nats"]
@@ -391,8 +392,11 @@ nu-parser = { version = "0.101.0", default-features = false }
globset = "0.4.16"
croner = "2.2.0"
rmcp = { version = "^0", features = ["client", "transport-streamable-http-client", "transport-streamable-http-client-reqwest"] }
rquickjs = { version = "0.8", features = ["futures", "parallel", "macro"] }
process-wrap = { version = "8.2.1", features = ["tokio1"] }
systemstat = "0.2.4"
datafusion = "47.0.0"
object_store = { git = "https://github.com/apache/arrow-rs-object-store", rev = "36752c975d4f29e20b57c91f81a10872dcd48ae7", features = ["aws", "azure", "gcp"] }
openidconnect = { version = "4.0.0-rc.1" }
@@ -407,7 +411,6 @@ aws-sdk-sso = "=1.77.0"
aws-sdk-ssooidc = "=1.78.0"
rustls = "=0.23.35"
async-once-cell = "0.5.4"
systemstat = "0.2.4"
size = "0.5.0"
aws-smithy-types-convert = { version = "^0", features = ["convert-chrono"] }

View File

@@ -0,0 +1,370 @@
# QuickJS Migration - Potential Breaking Changes Analysis
## Summary
This document details the comprehensive investigation into potential breaking changes when migrating flow expressions from Deno Core (V8) to QuickJS.
## 1. Areas Already Tested (60+ Parity Tests)
The following areas have comprehensive parity tests in `js_eval_parity_tests.rs`:
- **Arithmetic operations**: +, -, *, /, %, **
- **Comparison operators**: ===, !==, >, <, >=, <=, ==, !=
- **Logical operators**: &&, ||, !, ??, ?.
- **Bitwise operators**: &, |, ^, ~, <<, >>, >>>
- **Object operations**: property access, spread, destructuring, Object.keys/values/entries
- **Array operations**: map, filter, reduce, find, some, every, slice, flat, etc.
- **String operations**: split, replace, includes, startsWith, trim, etc.
- **Template literals**: ${} interpolation
- **Optional chaining**: ?. for properties, methods, computed properties
- **Nullish coalescing**: ??
- **Try-catch blocks**
- **Arrow functions**
- **Destructuring**
- **Date operations** (with fixed dates)
- **JSON.parse/stringify**
- **Math functions**
- **Set and Map operations**
- **Regular expressions** (basic patterns)
- **flow_input, flow_env, previous_result access**
- **Error extraction logic** (from parallel results)
## 2. Potential Breaking Changes Identified
### 2.1 Number Handling Edge Cases (MEDIUM RISK)
**Implementation Difference:**
```rust
// QuickJS json_to_js:
if i >= i32::MIN as i64 && i <= i32::MAX as i64 {
Ok(Value::new_int(ctx.clone(), i as i32))
} else {
Ok(Value::new_float(ctx.clone(), i as f64))
}
```
**Potential Issues:**
- Numbers outside i32 range (-2147483648 to 2147483647) are stored as floats
- Large integers (between i32::MAX and 2^53) might lose precision
- **Timestamps** (e.g., 1704067200000) are typically in this range
**Test Case Needed:**
```javascript
// Numbers just above i32::MAX
2147483648 + 1 // i32::MAX + 2
9007199254740991 - 1 // Near MAX_SAFE_INTEGER
```
### 2.2 Object Property Order (LOW RISK)
**Implementation Difference:**
- QuickJS: `obj.props::<String, Value>()` iteration order
- V8: Guaranteed insertion order for string keys
**Potential Impact:**
- `Object.keys()`, `Object.values()`, `Object.entries()` order might differ
- Object spread `{...obj}` order might differ
**Mitigated by:**
- JSON comparison in tests normalizes order
- Most flow expressions don't depend on property order
### 2.3 Missing Browser/Deno APIs (MEDIUM RISK)
**APIs NOT available in QuickJS:**
- `atob()` / `btoa()` - Base64 encoding/decoding
- `TextEncoder` / `TextDecoder`
- `fetch()` (not relevant for expressions)
- `Blob`, `ArrayBuffer` (limited support)
- `Intl.*` - Internationalization APIs
- `console.log()` - No effect (not breaking, just no output)
**Expressions that would break:**
```javascript
atob("SGVsbG8=") // Would throw: atob is not defined
btoa("Hello") // Would throw: btoa is not defined
new TextEncoder().encode("test") // Would throw
"test".toLocaleUpperCase('tr-TR') // Might behave differently
```
### 2.4 Regular Expression Differences (LOW RISK)
**QuickJS RegExp limitations:**
- No `d` flag (indices)
- No lookbehind assertions `(?<=...)` and `(?<!...)`
- No named capture groups `(?<name>...)`
**Expressions that might break:**
```javascript
"test123".match(/(?<=test)\d+/) // Lookbehind not supported
/(?<name>\w+)/.exec("test")?.groups?.name // Named groups not supported
```
### 2.5 Prototype Method Availability (LOW RISK)
**Methods that might differ:**
- `Array.prototype.at()` - ES2022
- `String.prototype.at()` - ES2022
- `Object.hasOwn()` - ES2022
- `String.prototype.replaceAll()` - ES2021
**Test Case:**
```javascript
[1,2,3].at(-1) // Might not exist
"hello".at(-1) // Might not exist
```
### 2.6 NaN/Infinity/Special Values (LOW RISK)
**Implementation:**
```rust
// QuickJS js_to_json:
if let Some(n) = serde_json::Number::from_f64(f) {
return Ok(serde_json::Value::Number(n));
} else {
return Ok(serde_json::Value::Null); // NaN, Infinity -> null
}
```
Both engines convert NaN/Infinity to null in JSON, so this is consistent.
### 2.7 Fallback for Unsupported Types (LOW RISK)
**QuickJS fallback:**
```rust
// Fallback
Ok(serde_json::Value::String("[object]".to_string()))
```
Types that would trigger this:
- Symbol
- WeakMap/WeakRef
- Generator objects
- Custom objects with non-enumerable properties only
### 2.8 Date Object Timezone Handling (MEDIUM RISK)
**Potential Issue:**
- `new Date()` without arguments uses system time
- Timezone-dependent methods might vary
**Safe patterns (already tested):**
```javascript
new Date('2024-01-15T00:00:00.000Z').getUTCFullYear() // OK - UTC methods
Date.parse('2024-01-15T00:00:00.000Z') // OK - explicit timezone
```
**Risky patterns:**
```javascript
new Date().toLocaleDateString() // Timezone dependent
new Date().getHours() // Timezone dependent
```
## 3. Edge Cases NOT Currently Tested
### 3.1 Very Large Numbers
```javascript
9007199254740991 // MAX_SAFE_INTEGER
9007199254740992 // MAX_SAFE_INTEGER + 1 (loses precision)
2147483648 // i32::MAX + 1
```
### 3.2 Negative Zero
```javascript
-0 === 0 // true
Object.is(-0, 0) // false
1/-0 // -Infinity
```
### 3.3 Sparse Arrays
```javascript
const arr = [1, , 3] // Hole at index 1
arr.map(x => x * 2) // Holes might be handled differently
arr.filter(x => true) // Holes might be skipped or preserved
```
### 3.4 Unicode Edge Cases
```javascript
"🎉".length // 2 (surrogate pairs)
"🎉".split('') // Might differ
[..."🎉"] // Might differ
"café" === "café" // NFC vs NFD normalization
```
### 3.5 Prototype Chain
```javascript
const obj = Object.create({ inherited: 1 });
obj.own = 2;
Object.keys(obj) // Should only return ['own']
```
### 3.6 Getter/Setter Properties
```javascript
const obj = {
get prop() { return 42; },
set prop(v) { }
};
obj.prop // Should return 42
```
### 3.7 Circular References
```javascript
const obj = { a: 1 };
obj.self = obj;
JSON.stringify(obj) // Should throw in both
```
### 3.8 Array-like Objects
```javascript
const arrayLike = { 0: 'a', 1: 'b', length: 2 };
Array.from(arrayLike) // Should work in both
```
## 4. Recommended Additional Tests
### High Priority (Add to parity tests):
1. Large integers (i32 boundary, MAX_SAFE_INTEGER boundary)
2. `Array.prototype.at()` and `String.prototype.at()`
3. Sparse arrays with holes
4. Emoji/surrogate pair handling
5. Object property order verification
### Medium Priority:
1. Getter/setter access
2. Prototype chain behavior
3. Array-like object conversion
4. Error message format differences
### Low Priority (Unlikely to be used in expressions):
1. WeakMap/WeakSet
2. Generators
3. Symbols
4. Proxy edge cases
## 5. Known Safe Patterns
These patterns are safe to use and have been verified:
- All arithmetic and comparison operators
- All standard array methods (map, filter, reduce, etc.)
- All standard string methods
- Object spread and destructuring
- Optional chaining and nullish coalescing
- Template literals
- Arrow functions
- Try-catch blocks
- `flow_input`, `flow_env`, `previous_result`, `results` access
- JSON operations
- Date operations with UTC methods
- Regular expressions (basic patterns without lookbehind)
## 6. Test Coverage Summary
### Unit Parity Tests (114 tests in js_eval_parity_tests.rs)
- Basic arithmetic, comparison, logical, and bitwise operators
- Object operations: property access, spread, destructuring
- Array operations: map, filter, reduce, find, some, every, slice, flat, etc.
- String operations: all standard methods
- Template literals with complex expressions
- Optional chaining and nullish coalescing
- Set and Map operations
- JSON parse/stringify
- Date operations with UTC methods
- Error handling with try-catch
- Large integer handling (i32 boundaries, timestamps, MAX_SAFE_INTEGER)
- Unicode and special characters
- Type coercion
### Flow Engine Parity Tests (19 tests in flow_engine_parity.rs)
All tests pass with both Deno Core and QuickJS:
1. **Linear flow with input transforms** - `results.a.property` access
2. **For-loop with complex iterator** - `results.a.users.filter(...)`
3. **Branch conditions** - `results.a.status === 'premium' && results.a.score >= 90`
4. **Previous result aggregation** - `previous_result.value`, `results.a.value + results.b.value`
5. **Nested complexity** - Deep result access across loop iterations
6. **Parallel for-loops** - Multiple concurrent iterations
7. **Skip-if expressions** - Conditional step execution
8. **Object transformations** - Complex data manipulation
9. **Template literals** - `\`Status: ${results.a.status}\``
10. **Optional chaining** - `results.a.user?.name`, `results.a?.missing?.value ?? 'default'`
11. **Flow env access** - `flow_env.CONFIG.apiUrl`
12. **Combined flow_input and flow_env**
13. **Results optional chaining** - Deep optional chaining with results proxy
14. **Large integers** - Timestamps, i32 boundaries through results
15. **Unicode and emoji** - Strings with unicode through flow results
16. **Complex array operations** - Sort, filter/map chains, reduce through results
17. **Multiline expressions** - Multi-statement expressions with semicolons and return
18. **Spread operators** - `{...results.a.config}`, `[...results.a.tags]`
19. **Nested for-loop results access** - Accessing outer step results from inner loops
## 7. Conclusion
The QuickJS migration is **safe** for the vast majority of flow expressions. Comprehensive testing shows:
- **133 total parity tests pass** (114 unit + 19 flow engine)
- All tests pass with both Deno Core and QuickJS
- No behavioral differences detected in production-like scenarios
### ES2022+ Method Support (All SUPPORTED in both engines):
Tested and verified to work identically:
- `Array.prototype.at()` - ES2022 ✅
- `String.prototype.at()` - ES2022 ✅
- `Object.hasOwn()` - ES2022 ✅
- `String.prototype.replaceAll()` - ES2021 ✅
- `Array.prototype.findLast()` - ES2023 ✅
- `Array.prototype.findLastIndex()` - ES2023 ✅
- `Array.prototype.toSorted()` - ES2023 ✅
- `Array.prototype.toReversed()` - ES2023 ✅
- `Array.prototype.toSpliced()` - ES2023 ✅
- `Array.prototype.with()` - ES2023 ✅
- `Object.groupBy()` - ES2024 ✅
### Regex Feature Support (All SUPPORTED in both engines):
- Lookbehind assertions `(?<=...)`
- Negative lookbehind `(?<!...)`
- Named capture groups `(?<name>...)`
- `d` flag (indices) ✅
### Browser API Parity (Both engines return undefined):
These APIs are NOT available in either engine (consistent behavior):
- `atob` / `btoa` - Both return `typeof === "undefined"`
- `TextEncoder` / `TextDecoder` - Both return `typeof === "undefined"`
- `URL` / `URLSearchParams` - Both return `typeof === "undefined"`
### BREAKING CHANGE IDENTIFIED:
**Intl API** - ONLY breaking change found:
- Deno Core: `typeof Intl === "object"` (available)
- QuickJS: `typeof Intl === "undefined"` (NOT available)
Expressions using these will FAIL with QuickJS:
- `new Intl.NumberFormat('en-US').format(1234567.89)`
- `new Intl.DateTimeFormat('en-US').format(new Date())`
- `num.toLocaleString('de-DE')`
- `date.toLocaleDateString('fr-FR')`
**Mitigation**: Search production logs for `Intl` usage in flow expressions before migration.
### Recommendations:
1. ✅ Run the parity tests to verify current implementation (132 unit tests + 19 flow engine tests pass)
2. ✅ Add tests for edge cases (large integers, optional chaining, spread, multiline)
3. ✅ Test ES2022+ methods - All supported (Array.at, Object.hasOwn, etc.)
4. ✅ Test regex features - All supported (lookbehind, named groups)
5. ⚠️ **Search production for `Intl` usage** - Only confirmed breaking change
6. Run `USE_QUICKJS_FOR_FLOW_EVAL=1` in staging before full production rollout
7. Consider adding `Intl` polyfill to QuickJS if production usage is found
### Commands to Run Tests:
```bash
# Run all parity tests (132 tests)
cargo test --features deno_core,quickjs -p windmill-worker -- parity_
# Run flow engine tests with both engines (19 tests)
cargo test --features deno_core -p windmill --test flow_engine_parity
USE_QUICKJS_FOR_FLOW_EVAL=1 cargo test --features deno_core,quickjs -p windmill --test flow_engine_parity
```

View File

@@ -1 +1 @@
371efb2d7307f588c5ce00d63fd036751cc068f2
a18ac31062ac092cb9a5fc87629e217d97f4911d

View File

@@ -0,0 +1 @@
ALTER TABLE worker_ping DROP COLUMN IF EXISTS dedicated_workers;

View File

@@ -0,0 +1 @@
ALTER TABLE worker_ping ADD COLUMN IF NOT EXISTS dedicated_workers TEXT[];

View File

@@ -1,111 +0,0 @@
---
description:
globs: backend/**/*.rs
alwaysApply: false
---
# Windmill Backend - Rust Best Practices
## Project Structure
Windmill uses a workspace-based architecture with multiple crates:
- **windmill-api**: API server functionality
- **windmill-worker**: Job execution
- **windmill-common**: Shared code used by all crates
- **windmill-queue**: Job & flow queuing
- **windmill-audit**: Audit logging
- Other specialized crates (git-sync, autoscaling, etc.)
## Adding New Code
### Module Organization
- Place new code in the appropriate crate based on functionality
- For API endpoints, create or modify files in `windmill-api/src/` organized by domain
- For shared functionality, use `windmill-common/src/`
- Use the `_ee.rs` suffix for enterprise-only modules
- Follow existing patterns for file structure and organization
### Error Handling
- Use the custom `Error` enum from `windmill-common::error`
- Return `Result<T, Error>` or `JsonResult<T>` for functions that can fail
- Use the `?` operator for error propagation
- Add location tracking to errors using `#[track_caller]`
### Database Operations
- Use `sqlx` for database operations with prepared statements
- Leverage existing database helper functions in `db.rs` modules
- Use transactions for multi-step operations
- Handle database errors properly
### API Endpoints
- Follow existing patterns in the `windmill-api` crate
- Use axum's routing system and extractors
- Group related routes together
- Use consistent response formats (JSON)
- Follow proper authentication and authorization patterns
- Do not forget to update backend/windmill-api/openapi.yaml after modifying an api endpoint
## Performance Optimizations
When generating code, especially involving `serde`, `sqlx`, and `tokio`, prioritize performance by applying the following principles:
### Serde Optimizations (Serialization & Deserialization)
- **Specify Structure Explicitly:** When defining structs for Serde (`#[derive(Serialize, Deserialize)]`), use `#[serde(...` attributes extensively. This includes:
* `#[serde(rename = "...")]` or `#[serde(alias = "...")]` to map external names precisely, avoiding dynamic lookups.
* `#[serde(default)]` for optional fields with default values, reducing parsing complexity.
* `#[serde(skip_serializing_if = "...")]` to avoid writing fields that meet a certain condition (e.g., `Option::is_none()`, `Vec::is_empty()`, or a custom function), reducing output size and serialization work.
* `#[serde(skip_serializing)]` or `#[serde(skip_deserializing)]` for fields that should *not* be included.
- **Prefer Borrowing:** Where possible and safe (data lifetime allows), use `Cow<'a, str>` or `&'a str` (with `#[serde(borrow)]`) instead of `String` for string fields during deserialization. This avoids allocating new strings, enabling zero-copy reading from the input buffer. Apply this principle to byte slices (`&'a [u8]` / `Cow<'a, [u8]>`) and potentially borrowed vectors as well.
- **Avoid Intermediate `Value`:** Unless the data structure is truly dynamic or unknown at compile time, deserialize directly into a well-defined struct or enum rather than into `serde_json::Value` (or equivalent for other formats). This avoids unnecessary heap allocations and type switching.
### SQLx Optimizations (Database Interaction)
- **CRITICAL - Never Use `SELECT *` in Worker-Executed Queries:** For any query that can potentially be executed by workers, **always** explicitly list the specific columns you need instead of using `SELECT *`. This is essential for backwards compatibility: when workers are running behind the API server version (common in distributed deployments), adding new columns to database tables will cause outdated workers to fail when they try to deserialize rows with unexpected columns. Always use explicit column lists like `SELECT id, workspace_id, path, created_at FROM table` instead of `SELECT * FROM table`.
- **Select Only Necessary Columns:** In `SELECT` queries, list specific column names rather than using `SELECT *`. This reduces data transferred from the database and the work needed for hydration/deserialization.
- **Batch Operations:** For multiple `INSERT`, `UPDATE`, or `DELETE` statements, prefer executing them in a single query if the database and driver support it efficiently (e.g., `INSERT INTO ... VALUES (...), (...), ...`). This minimizes round trips to the database.
- **Avoid N+1 Queries:** Do not loop through results of one query and execute a separate query for each item (e.g., fetching users, then querying for each user's profile in a loop). Instead, use JOINs or a single query with an `IN` clause to fetch related data efficiently.
- **Deserialize Directly:** Use `#[derive(FromRow)]` on structs and ensure the struct fields match the selected columns in the query. This allows SQLx to hydrate objects directly, avoiding intermediate data structures.
- **Parameterize Queries:** Always use SQLx's query methods (`.bind(...)`) to pass values as parameters rather than string formatting. This prevents SQL injection and allows the database to cache query plans, improving performance on repeated executions.
### Tokio Optimizations (Asynchronous Runtime)
- **Avoid Blocking Operations:** **Crucially**, never perform blocking operations (synchronous file I/O, `std::thread::sleep`, CPU-bound loops, `std::sync::Mutex::lock`, blocking network calls without `tokio::net`) directly within an `async fn` or a standard `tokio::spawn` task. Blocking pauses the entire worker thread, potentially starving other tasks. Use `tokio::task::spawn_blocking` for CPU-intensive work or blocking I/O.
- **Use Tokio's Async Primitives:** Prefer `tokio::sync` (channels, mutexes, semaphores), `tokio::io`, `tokio::net`, and `tokio::time` over their `std` counterparts in asynchronous contexts. These are designed to yield control back to the scheduler.
- **Manage Concurrency:** Be mindful of how many tasks are spawned. Creating a new task for every tiny piece of work can introduce overhead. Group related asynchronous operations where appropriate.
- **Handle Shared State Efficiently:** Use `Arc` for shared ownership in concurrent tasks. When shared state needs mutation, prefer `tokio::sync::Mutex` over `std::sync::Mutex` in `async` code. Consider `tokio::sync::RwLock` if reads significantly outnumber writes. Minimize the duration for which locks are held.
- **Understand `.await`:** Place `.await` strategically to allow the runtime to switch to other ready tasks. Ensure that `.await` points to genuinely asynchronous operations.
- **Backpressure:** If dealing with data streams or queues between tasks, implement backpressure mechanisms (e.g., bounded channels like `tokio::sync::mpsc::channel`) to prevent one component from overwhelming another or critical resources like the database.
## Enterprise Features
- Use feature flags for enterprise functionality
- Conditionally compile with `#[cfg(feature = "enterprise")]`
- Isolate enterprise code in separate modules
## Code Style
- Group imports by external and internal crates
- Place struct/enum definitions before implementations
- Group similar functionality together
- Use descriptive naming consistent with the codebase
- Follow existing patterns for async code using tokio
## Testing
- Write unit tests for core functionality
- Use the `#[cfg(test)]` module for test code
- For database tests, use the existing test utilities
## Common Crates Used
- **tokio**: For async runtime
- **axum**: For web server and routing
- **sqlx**: For database operations
- **serde**: For serialization/deserialization
- **tracing**: For logging and diagnostics
- **reqwest**: For HTTP client functionality

View File

@@ -668,10 +668,11 @@ async fn windmill_main() -> anyhow::Result<()> {
// Load OTEL tracing proxy settings and initialize deno_telemetry if nativets tracing is enabled
// This must happen before any Deno runtime is created
#[cfg(all(feature = "private", feature = "enterprise", feature = "deno_core"))]
#[cfg(all(feature = "private", feature = "enterprise"))]
{
reload_otel_tracing_proxy_setting(&Connection::Sql(db.clone())).await;
#[cfg(feature = "deno_core")]
if windmill_worker::is_otel_tracing_proxy_enabled_for_lang(&ScriptLang::Nativets).await {
match windmill_worker::load_internal_otel_exporter().await {
Ok(()) => {
@@ -908,9 +909,11 @@ Windmill Community Edition {GIT_VERSION}
tracing::info!("Received killpill, aborting index initialization");
},
res = windmill_indexer::completed_runs_oss::init_index(&db) => {
let res = res?;
reader = Some(res.0);
writer = Some(res.1);
let res = res?;
if let Some(r) = res {
reader = Some(r.0);
writer = Some(r.1);
}
}
}
@@ -952,9 +955,11 @@ Windmill Community Edition {GIT_VERSION}
tracing::info!("Received killpill, aborting index initialization");
},
res = windmill_indexer::service_logs_oss::init_index(&db, killpill_tx.clone()) => {
let res = res?;
reader = Some(res.0);
writer = Some(res.1);
let res = res?;
if let Some(r) = res {
reader = Some(r.0);
writer = Some(r.1);
}
}
}
@@ -1572,34 +1577,17 @@ Windmill Community Edition {GIT_VERSION}
let otel_tracing_proxy_f = async {
#[cfg(all(feature = "private", feature = "enterprise"))]
{
// Start OTEL tracing proxy for HTTP request interception
// Only enabled when: setting is on, worker mode (not server), and single worker (to avoid race conditions)
if worker_mode
&& num_workers == 1
&& windmill_worker::OTEL_TRACING_PROXY_SETTINGS
.read()
.await
.enabled
{
if let Some(db) = conn.as_sql() {
tracing::info!(
"Starting jobs OTEL tracing (ports will be dynamically assigned)"
);
if let Err(e) =
windmill_worker::start_jobs_otel_tracing(db.clone(), otel_killpill_rx)
.await
{
tracing::error!("Jobs OTEL tracing error: {}", e);
}
}
} else if windmill_worker::OTEL_TRACING_PROXY_SETTINGS
.read()
if worker_mode {
if let Some(db) = conn.as_sql() {
if let Err(e) = windmill_worker::start_jobs_otel_tracing(
db.clone(),
otel_killpill_rx,
num_workers,
)
.await
.enabled
&& num_workers > 1
{
tracing::warn!("OTEL tracing proxy is enabled but num_workers > 1. Disabling to avoid race conditions. Set NUM_WORKERS=1 to enable.");
{
tracing::error!("Jobs OTEL tracing error: {}", e);
}
}
}

View File

@@ -56,8 +56,8 @@ use windmill_common::{
HUB_API_SECRET_SETTING, HUB_BASE_URL_SETTING, INSTANCE_PYTHON_VERSION_SETTING,
JOB_DEFAULT_TIMEOUT_SECS_SETTING, JWT_SECRET_SETTING, KEEP_JOB_DIR_SETTING,
LICENSE_KEY_SETTING, MONITOR_LOGS_ON_OBJECT_STORE_SETTING, NPM_CONFIG_REGISTRY_SETTING,
OTEL_TRACING_PROXY_SETTING, NUGET_CONFIG_SETTING, OTEL_SETTING, PIP_INDEX_URL_SETTING, POWERSHELL_REPO_PAT_SETTING,
POWERSHELL_REPO_URL_SETTING, REQUEST_SIZE_LIMIT_SETTING,
NUGET_CONFIG_SETTING, OTEL_SETTING, OTEL_TRACING_PROXY_SETTING, PIP_INDEX_URL_SETTING,
POWERSHELL_REPO_PAT_SETTING, POWERSHELL_REPO_URL_SETTING, REQUEST_SIZE_LIMIT_SETTING,
REQUIRE_PREEXISTING_USER_FOR_OAUTH_SETTING, RETENTION_PERIOD_SECS_SETTING,
SAML_METADATA_SETTING, SCIM_TOKEN_SETTING, TIMEOUT_WAIT_RESULT_SETTING,
},
@@ -84,10 +84,10 @@ use windmill_common::{
use windmill_common::{client::AuthedClient, global_settings::APP_WORKSPACED_ROUTE_SETTING};
use windmill_queue::{cancel_job, get_queued_job_v2, SameWorkerPayload};
use windmill_worker::{
handle_job_error, JobCompletedSender, OtelTracingProxySettings, SameWorkerSender, BUNFIG_INSTALL_SCOPES,
INSTANCE_PYTHON_VERSION, JOB_DEFAULT_TIMEOUT, KEEP_JOB_DIR, MAVEN_REPOS,
OTEL_TRACING_PROXY_SETTINGS, NO_DEFAULT_MAVEN, NPM_CONFIG_REGISTRY, NUGET_CONFIG, PIP_EXTRA_INDEX_URL,
PIP_INDEX_URL, POWERSHELL_REPO_PAT, POWERSHELL_REPO_URL,
handle_job_error, JobCompletedSender, OtelTracingProxySettings, SameWorkerSender,
BUNFIG_INSTALL_SCOPES, INSTANCE_PYTHON_VERSION, JOB_DEFAULT_TIMEOUT, KEEP_JOB_DIR, MAVEN_REPOS,
NO_DEFAULT_MAVEN, NPM_CONFIG_REGISTRY, NUGET_CONFIG, OTEL_TRACING_PROXY_SETTINGS,
PIP_EXTRA_INDEX_URL, PIP_INDEX_URL, POWERSHELL_REPO_PAT, POWERSHELL_REPO_URL,
};
#[cfg(feature = "parquet")]
@@ -246,6 +246,7 @@ pub async fn initial_load(
),
priority_tags_sorted: vec![],
dedicated_worker: None,
dedicated_workers: None,
init_bash: load_init_bash_from_env(),
periodic_script_bash: load_periodic_bash_script_from_env(),
periodic_script_interval_seconds: load_periodic_bash_script_interval_from_env(),
@@ -784,26 +785,24 @@ pub async fn load_keep_job_dir(conn: &Connection) {
pub async fn reload_otel_tracing_proxy_setting(conn: &Connection) {
match load_value_from_global_settings_with_conn(conn, OTEL_TRACING_PROXY_SETTING, true).await {
Ok(Some(settings)) => {
match serde_json::from_value::<OtelTracingProxySettings>(settings) {
Ok(new_settings) => {
let mut current = OTEL_TRACING_PROXY_SETTINGS.write().await;
if current.enabled != new_settings.enabled
|| current.enabled_languages != new_settings.enabled_languages
{
tracing::info!(
"OTEL tracing proxy settings changed: enabled={}, languages={:?}",
new_settings.enabled,
new_settings.enabled_languages
);
*current = new_settings;
}
}
Err(e) => {
tracing::error!("Error parsing OTEL tracing proxy settings: {e:#}");
Ok(Some(settings)) => match serde_json::from_value::<OtelTracingProxySettings>(settings) {
Ok(new_settings) => {
let mut current = OTEL_TRACING_PROXY_SETTINGS.write().await;
if current.enabled != new_settings.enabled
|| current.enabled_languages != new_settings.enabled_languages
{
tracing::info!(
"OTEL tracing proxy settings changed: enabled={}, languages={:?}",
new_settings.enabled,
new_settings.enabled_languages
);
*current = new_settings;
}
}
}
Err(e) => {
tracing::error!("Error parsing OTEL tracing proxy settings: {e:#}");
}
},
Err(e) => {
tracing::error!("Error loading OTEL tracing proxy setting: {e:#}");
}
@@ -985,7 +984,10 @@ pub async fn delete_expired_items(db: &DB) -> () {
);
}
}
Err(e) => tracing::error!("Error deleting expired MCP OAuth authorization codes: {:?}", e),
Err(e) => tracing::error!(
"Error deleting expired MCP OAuth authorization codes: {:?}",
e
),
}
let job_retention_secs = *JOB_RETENTION_SECS.read().await;
@@ -1866,7 +1868,13 @@ pub async fn monitor_db(
let update_min_worker_version_f = async {
#[cfg(not(feature = "test_job_debouncing"))]
windmill_common::min_version::update_min_version(conn, _worker_mode, WORKERS_NAMES.read().await.clone(), initial_load).await;
windmill_common::min_version::update_min_version(
conn,
_worker_mode,
WORKERS_NAMES.read().await.clone(),
initial_load,
)
.await;
};
// Run every 5 minutes (10 iterations * 30s = 5 minutes)
@@ -2060,10 +2068,12 @@ pub async fn reload_worker_config(db: &DB, tx: KillpillSender, kill_if_change: b
} else {
let wc = WORKER_CONFIG.read().await;
let config = config.unwrap();
if *wc != config || config.dedicated_worker.is_some() {
let has_dedicated = config.dedicated_worker.is_some() || config.dedicated_workers.as_ref().is_some_and(|dws| !dws.is_empty());
if *wc != config || has_dedicated {
if kill_if_change {
if config.dedicated_worker.is_some()
if has_dedicated
|| (*wc).dedicated_worker != config.dedicated_worker
|| (*wc).dedicated_workers != config.dedicated_workers
{
tracing::info!("Dedicated worker config changed, sending killpill. Expecting to be restarted by supervisor.");
let _ = tx.send();

View File

@@ -995,6 +995,7 @@ TABLE: worker_ping
- custom_tags (text[])
- worker_group (character)
- dedicated_worker (character)
- dedicated_workers (text[])
- wm_version (character)
- current_job_id (uuid)
- current_job_workspace_id (character)

File diff suppressed because it is too large Load Diff

View File

@@ -1,7 +1,7 @@
openapi: "3.0.3"
info:
version: 1.617.1
version: 1.621.2
title: Windmill API
contact:
@@ -5751,6 +5751,14 @@ paths:
If true, the description field will be omitted from the response.
schema:
type: boolean
- name: dedicated_worker
in: query
description: |
(default regardless)
If true, show only scripts with dedicated_worker enabled.
If false, show only scripts with dedicated_worker disabled.
schema:
type: boolean
responses:
"200":
@@ -7178,6 +7186,14 @@ paths:
If true, the description field will be omitted from the response.
schema:
type: boolean
- name: dedicated_worker
in: query
description: |
(default regardless)
If true, show only flows with dedicated_worker enabled.
If false, show only flows with dedicated_worker disabled.
schema:
type: boolean
responses:
"200":
description: All flow
@@ -7703,16 +7719,19 @@ paths:
schema:
type: boolean
/w/{workspace}/apps/get_data/{version}/{path}:
/w/{workspace}/apps/get_data/v/{secretWithExtension}:
get:
summary: get app by path
summary: get raw app data by
operationId: getRawAppData
tags:
- raw_app
- app
parameters:
- $ref: "#/components/parameters/WorkspaceId"
- $ref: "#/components/parameters/VersionId"
- $ref: "#/components/parameters/ScriptPath"
- name: secretWithExtension
in: path
required: true
schema:
type: string
responses:
"200":
description: app details
@@ -12247,6 +12266,16 @@ paths:
$ref: "#/components/schemas/NativeServiceName"
- $ref: "#/components/parameters/Page"
- $ref: "#/components/parameters/PerPage"
- name: path
description: filter by script path
in: query
schema:
type: string
- name: is_flow
description: filter by is_flow
in: query
schema:
type: boolean
responses:
"200":
description: native triggers list
@@ -22678,4 +22707,4 @@ components:
required:
- id
- name
- path
- path

View File

@@ -16,7 +16,7 @@ use serde::{Deserialize, Serialize};
use serde_json::{json, value::RawValue};
use std::collections::HashMap;
use windmill_audit::{audit_oss::audit_log, ActionKind};
use windmill_common::ai_providers::{AIProvider, ProviderConfig, ProviderModel};
use windmill_common::ai_providers::{empty_string_as_none, AIProvider, ProviderConfig, ProviderModel};
use windmill_common::error::{to_anyhow, Error, Result};
use windmill_common::utils::configure_client;
use windmill_common::variables::get_variable_or_self;
@@ -143,15 +143,17 @@ enum AnthropicPlatform {
#[derive(Deserialize, Debug)]
struct AIStandardResource {
#[serde(alias = "baseUrl")]
#[serde(alias = "baseUrl", default, deserialize_with = "empty_string_as_none")]
base_url: Option<String>,
#[serde(alias = "apiKey")]
#[serde(alias = "apiKey", default, deserialize_with = "empty_string_as_none")]
api_key: Option<String>,
#[serde(default, deserialize_with = "empty_string_as_none")]
organization_id: Option<String>,
#[serde(default, deserialize_with = "empty_string_as_none")]
region: Option<String>,
#[serde(alias = "awsAccessKeyId")]
#[serde(alias = "awsAccessKeyId", default, deserialize_with = "empty_string_as_none")]
aws_access_key_id: Option<String>,
#[serde(alias = "awsSecretAccessKey")]
#[serde(alias = "awsSecretAccessKey", default, deserialize_with = "empty_string_as_none")]
aws_secret_access_key: Option<String>,
/// Platform for Anthropic API (standard or google_vertex_ai)
#[serde(default)]
@@ -207,9 +209,14 @@ impl AIRequestConfig {
AIResource::Standard(resource) => {
let region = resource.region.clone();
let platform = resource.platform.clone();
let base_url = provider
.get_base_url(resource.base_url, resource.region, db)
.await?;
// Skip get_base_url for Bedrock - it uses SDK directly, not HTTP
let base_url = if matches!(provider, AIProvider::AWSBedrock) {
String::new()
} else {
provider
.get_base_url(resource.base_url, db)
.await?
};
let api_key = if let Some(api_key) = resource.api_key {
Some(get_variable_or_self(api_key, db, w_id).await?)
} else {
@@ -251,7 +258,7 @@ impl AIRequestConfig {
None
};
let token = Self::get_token_using_oauth(resource, db, w_id).await?;
let base_url = provider.get_base_url(None, None, db).await?;
let base_url = provider.get_base_url(None, db).await?;
(
None,
@@ -578,7 +585,7 @@ async fn global_proxy(
return Err(Error::BadRequest("API key is required".to_string()));
};
let base_url = provider.get_base_url(None, None, &db).await?;
let base_url = provider.get_base_url(None, &db).await?;
let url = format!("{}/{}", base_url, ai_path);
@@ -745,7 +752,7 @@ async fn proxy(
let region = request_config
.region
.as_deref()
.ok_or_else(|| Error::internal_err("AWS region must be set for Bedrock"))?;
.unwrap_or(windmill_common::ai_providers::USE_ENV_REGION);
// Audit log before making the SDK request
let mut tx = db.begin().await?;

View File

@@ -191,6 +191,9 @@ async fn list_flows(
if !lq.include_draft_only.unwrap_or(false) || authed.is_operator {
sqlb.and_where("o.draft_only IS NOT TRUE");
}
if let Some(dw) = &lq.dedicated_worker {
sqlb.and_where_eq("dedicated_worker", dw);
}
if lq.with_deployment_msg.unwrap_or(false) {
sqlb.join("deployment_metadata dm")

View File

@@ -419,7 +419,7 @@ pub async fn run_server(
if server_mode || mcp_mode {
use mcp::add_www_authenticate_header;
let (mcp_router, mcp_cancellation_token) =
setup_mcp_server(db.clone(), user_db).await?;
setup_mcp_server(db.clone(), user_db, _base_internal_url.clone()).await?;
// Apply middleware: auth check inside WWW-Authenticate wrapper so 401s get the header
let mcp_router = mcp_router
.route_layer(from_extractor::<ApiAuthed>())

View File

@@ -75,11 +75,12 @@ impl McpAuth for ApiAuthed {
pub struct WindmillBackend {
pub db: DB,
pub user_db: UserDB,
pub base_internal_url: String,
}
impl WindmillBackend {
pub fn new(db: DB, user_db: UserDB) -> Self {
Self { db, user_db }
pub fn new(db: DB, user_db: UserDB, base_internal_url: String) -> Self {
Self { db, user_db, base_internal_url }
}
}
@@ -355,9 +356,7 @@ impl McpBackend for WindmillBackend {
let query_string = build_query_string(args_map, &endpoint_tool.query_params_schema);
let full_url = format!(
"{}/api{}{}",
windmill_common::BASE_INTERNAL_URL.as_str(),
path_template,
query_string
self.base_internal_url, path_template, query_string
);
// Prepare request body
@@ -462,11 +461,12 @@ pub async fn add_www_authenticate_header(
pub async fn setup_mcp_server(
db: DB,
user_db: UserDB,
base_internal_url: String,
) -> anyhow::Result<(Router, CancellationToken)> {
let cancellation_token = CancellationToken::new();
let session_manager = Arc::new(LocalSessionManager::default());
let backend = WindmillBackend::new(db, user_db);
let backend = WindmillBackend::new(db, user_db, base_internal_url);
let runner = Runner::new(backend);
let service_config = StreamableHttpServerConfig {

View File

@@ -43,6 +43,8 @@ async fn require_is_writer_on_runnable(
pub struct ListQuery {
pub page: Option<usize>,
pub per_page: Option<usize>,
pub path: Option<String>,
pub is_flow: Option<bool>,
}
#[derive(Debug, Serialize)]
@@ -502,6 +504,8 @@ async fn list_native_triggers_handler<T: External>(
service_name,
query.page,
query.per_page,
query.path.as_deref(),
query.is_flow,
)
.await?;
tx.commit().await?;

View File

@@ -820,6 +820,8 @@ pub async fn list_native_triggers<'c, E: sqlx::Executor<'c, Database = Postgres>
service_name: ServiceName,
page: Option<usize>,
per_page: Option<usize>,
path: Option<&str>,
is_flow: Option<bool>,
) -> Result<Vec<NativeTrigger>> {
let offset = (page.unwrap_or(0) * per_page.unwrap_or(100)) as i64;
let limit = per_page.unwrap_or(100) as i64;
@@ -843,6 +845,8 @@ pub async fn list_native_triggers<'c, E: sqlx::Executor<'c, Database = Postgres>
WHERE
nt.workspace_id = $1 AND
nt.service_name = $2 AND
($5::text IS NULL OR nt.script_path = $5) AND
($6::bool IS NULL OR nt.is_flow = $6) AND
(
(nt.is_flow = false AND EXISTS (
SELECT 1 FROM script s
@@ -862,7 +866,9 @@ pub async fn list_native_triggers<'c, E: sqlx::Executor<'c, Database = Postgres>
workspace_id,
service_name as ServiceName,
limit,
offset
offset,
path,
is_flow
)
.fetch_all(db)
.await?;

View File

@@ -194,7 +194,7 @@ pub async fn sync_workspace_triggers<T: External>(
);
let windmill_triggers =
list_native_triggers(db, workspace_id, T::SERVICE_NAME, None, None).await?;
list_native_triggers(db, workspace_id, T::SERVICE_NAME, None, None, None, None).await?;
if windmill_triggers.is_empty() {
tracing::info!(

View File

@@ -371,6 +371,9 @@ async fn list_scripts(
if let Some(it) = &lq.is_template {
sqlb.and_where_eq("is_template", it);
}
if let Some(dw) = &lq.dedicated_worker {
sqlb.and_where_eq("dedicated_worker", dw);
}
if authed.is_operator {
sqlb.and_where_eq("kind", quote("script"));
} else if let Some(lowercased_kinds) = lowercased_kinds {

View File

@@ -32,7 +32,7 @@ impl TriggerCrud for EmailTrigger {
const SUPPORTS_TEST_CONNECTION: bool = false;
const ROUTE_PREFIX: &'static str = "/email_triggers";
const DEPLOYMENT_NAME: &'static str = "";
const IS_CLOUD_HOSTED: bool = false;
const IS_ALLOWED_ON_CLOUD: bool = false;
fn get_deployed_object(path: String) -> DeployedObject {
DeployedObject::EmailTrigger { path }

View File

@@ -762,7 +762,7 @@ pub fn generate_trigger_routers() -> Router {
);
}
#[cfg(all(feature = "enterprise", feature = "smtp", feature = "private"))]
#[cfg(all(feature = "smtp", feature = "private"))]
{
use crate::triggers::email::EmailTrigger;

View File

@@ -11,7 +11,7 @@ pub enum HandlerAction {
// Future variants can be added here (e.g., Script, Flow, etc.)
}
#[cfg(all(feature = "smtp", feature = "enterprise", feature = "private"))]
#[cfg(all(feature = "smtp", feature = "private"))]
pub mod email;
#[cfg(all(feature = "gcp_trigger", feature = "enterprise", feature = "private"))]
pub mod gcp;

View File

@@ -1603,7 +1603,7 @@ async fn update_user(
require_super_admin(&db, &authed.email).await?;
let mut tx = db.begin().await?;
let mut revoke_tokens = false;
let mut new_super_admin: Option<bool> = None;
if let Some(sa) = eu.is_super_admin {
sqlx::query_scalar!(
"UPDATE password SET super_admin = $1 WHERE email = $2",
@@ -1612,7 +1612,7 @@ async fn update_user(
)
.execute(&mut *tx)
.await?;
revoke_tokens = true;
new_super_admin = Some(sa);
}
if let Some(dv) = eu.is_devops {
@@ -1623,13 +1623,33 @@ async fn update_user(
)
.execute(&mut *tx)
.await?;
revoke_tokens = true;
// If super_admin wasn't explicitly set, we still need to refresh tokens
if new_super_admin.is_none() {
new_super_admin = sqlx::query_scalar!(
"SELECT super_admin FROM password WHERE email = $1",
&email_to_update
)
.fetch_optional(&mut *tx)
.await?;
}
}
if revoke_tokens {
sqlx::query!("DELETE FROM token WHERE email = $1", &email_to_update)
.execute(&mut *tx)
.await?;
if let Some(sa) = new_super_admin {
// Delete session tokens to force re-login with new privileges
sqlx::query!(
"DELETE FROM token WHERE email = $1 AND label = 'session'",
&email_to_update
)
.execute(&mut *tx)
.await?;
// Update super_admin flag on non-session tokens (webhooks, API tokens, etc.)
sqlx::query!(
"UPDATE token SET super_admin = $1 WHERE email = $2 AND label != 'session'",
sa,
&email_to_update
)
.execute(&mut *tx)
.await?;
}
if let Some(n) = eu.name {

View File

@@ -1336,31 +1336,55 @@ async fn get_datatable_schema(db: &DB, w_id: &str, datatable_name: &str) -> Resu
}
});
// Query the schema information
let rows = client
// First, get all non-system schemas (including empty ones)
let schema_rows = client
.query(
r#"
SELECT
nsp.nspname::text AS table_schema,
c.table_name::text,
c.column_name::text,
c.udt_name::text,
c.is_nullable::text,
c.column_default::text
FROM information_schema.columns c
JOIN pg_namespace nsp ON c.table_schema = nsp.nspname
WHERE nsp.nspname NOT IN ('information_schema', 'pg_toast', 'pg_catalog')
AND c.table_name IS NOT NULL
ORDER BY c.table_schema, c.table_name, c.ordinal_position
SELECT nspname::text AS schema_name
FROM pg_namespace
WHERE nspname NOT IN ('information_schema', 'pg_toast', 'pg_catalog')
AND nspname NOT LIKE 'pg_%'
ORDER BY nspname
"#,
&[],
)
.await
.map_err(|e| Error::internal_err(format!("Failed to query schema: {}", e)))?;
.map_err(|e| Error::internal_err(format!("Failed to query schemas: {}", e)))?;
// Build hierarchical structure: schema -> table -> column -> compact_type
let mut schema_map: SchemaMap = HashMap::new();
// Collect schema names and initialize map
let schema_names: Vec<String> = schema_rows
.iter()
.map(|row| {
let name: String = row.get(0);
schema_map.entry(name.clone()).or_default();
name
})
.collect();
// Query column information only for the schemas we found
let rows = client
.query(
r#"
SELECT
table_schema::text,
table_name::text,
column_name::text,
udt_name::text,
is_nullable::text,
column_default::text
FROM information_schema.columns
WHERE table_schema = ANY($1)
AND table_name IS NOT NULL
ORDER BY table_schema, table_name, ordinal_position
"#,
&[&schema_names],
)
.await
.map_err(|e| Error::internal_err(format!("Failed to query columns: {}", e)))?;
for row in rows {
let table_schema: String = row.get(0);
let table_name: String = row.get(1);
@@ -3593,18 +3617,18 @@ async fn edit_workspace(
Ok(format!("Updated workspace {}", &w_id))
}
async fn archive_workspace(
Extension(db): Extension<DB>,
Path(w_id): Path<String>,
authed: ApiAuthed,
) -> Result<String> {
require_admin(authed.is_admin, &authed.username)?;
/// Archive a workspace: disable schedules, cancel jobs, and mark as deleted.
/// Returns (schedules_disabled_count, jobs_canceled_count).
pub(crate) async fn archive_workspace_impl(
db: &DB,
w_id: &str,
username: &str,
) -> Result<(usize, usize)> {
// Step 1: Disable all schedules and clear their queued jobs
let mut tx = db.begin().await?;
let disabled_schedules = sqlx::query_scalar!(
"UPDATE schedule SET enabled = false WHERE workspace_id = $1 AND enabled = true RETURNING path",
&w_id
w_id
)
.fetch_all(&mut *tx)
.await?;
@@ -3618,15 +3642,20 @@ async fn archive_workspace(
// Clear all schedule-related jobs using the existing clear_schedule function
for schedule_path in &disabled_schedules {
crate::schedule::clear_schedule(&mut tx, schedule_path, &w_id).await?;
crate::schedule::clear_schedule(&mut tx, schedule_path, w_id).await?;
}
// Mark workspace as archived
sqlx::query!("UPDATE workspace SET deleted = true WHERE id = $1", w_id)
.execute(&mut *tx)
.await?;
tx.commit().await?;
// Step 2: Get all remaining queued jobs for this workspace (non-schedule jobs)
let jobs_to_cancel =
sqlx::query_scalar!("SELECT id FROM v2_job_queue WHERE workspace_id = $1", &w_id)
.fetch_all(&db)
sqlx::query_scalar!("SELECT id FROM v2_job_queue WHERE workspace_id = $1", w_id)
.fetch_all(db)
.await?;
let jobs_count = jobs_to_cancel.len();
@@ -3640,9 +3669,9 @@ async fn archive_workspace(
let canceled_count = if !jobs_to_cancel.is_empty() {
let axum::Json(canceled_jobs) = crate::jobs::cancel_jobs(
jobs_to_cancel,
&db,
&authed.username,
&w_id,
db,
username,
w_id,
false, // force_cancel
)
.await?;
@@ -3654,12 +3683,21 @@ async fn archive_workspace(
0
};
// Step 4: Archive the workspace
let mut tx = db.begin().await?;
sqlx::query!("UPDATE workspace SET deleted = true WHERE id = $1", &w_id)
.execute(&mut *tx)
.await?;
Ok((schedules_count, canceled_count))
}
async fn archive_workspace(
Extension(db): Extension<DB>,
Path(w_id): Path<String>,
authed: ApiAuthed,
) -> Result<String> {
require_admin(authed.is_admin, &authed.username)?;
let (schedules_count, canceled_count) =
archive_workspace_impl(&db, &w_id, &authed.username).await?;
// Audit log
let mut tx = db.begin().await?;
let mut audit_params = HashMap::new();
audit_params.insert("disabled_schedules", schedules_count.to_string());
audit_params.insert("canceled_jobs", canceled_count.to_string());

View File

@@ -810,7 +810,7 @@ pub(crate) async fn tarball_workspace(
for service_name in ServiceName::iter() {
let native_triggers =
list_native_triggers(&mut *tx, &w_id, service_name, None, None).await?;
list_native_triggers(&mut *tx, &w_id, service_name, None, None, None, None).await?;
for trigger in native_triggers {
let trigger_str = &to_string_without_metadata(

View File

@@ -1,6 +1,11 @@
use std::collections::HashMap;
use crate::db::ApiAuthed;
use crate::workspaces::{check_w_id_conflict, CREATE_WORKSPACE_REQUIRE_SUPERADMIN, WM_FORK_PREFIX};
use crate::workspaces::{
archive_workspace_impl, check_w_id_conflict, CREATE_WORKSPACE_REQUIRE_SUPERADMIN,
WM_FORK_PREFIX,
};
use crate::{db::DB, utils::require_super_admin};
use axum::extract::Query;
@@ -10,6 +15,7 @@ use axum::{
};
use sqlx::{Postgres, Transaction};
use tracing::info;
use windmill_audit::audit_oss::audit_log;
use windmill_audit::ActionKind;
@@ -20,6 +26,7 @@ use windmill_common::{
error::{Error, Result},
utils::require_admin,
};
use windmill_queue::schedule::{get_schedule_opt, push_scheduled_job};
use serde::Deserialize;
@@ -47,31 +54,19 @@ pub(crate) async fn change_workspace_id(
require_admin(authed.is_admin, &authed.username)?;
}
// Check total job count before attempting migration
let job_count = sqlx::query_scalar!(
"SELECT COUNT(*) FROM v2_job WHERE workspace_id = $1",
&old_id
)
.fetch_one(&db)
.await?
.unwrap_or(0);
if job_count > 100_000 {
return Err(Error::BadRequest(
format!(
"Workspace has {} jobs which exceeds the 100k limit for direct migration. Please use the Windmill CLI to migrate jobs instead: `wmill jobs pull/push`.",
job_count
)
));
}
let mut tx = db.begin().await?;
check_w_id_conflict(&mut tx, &rw.new_id).await?;
// duplicate workspace with new id name
info!(
"Changing workspace id from {} to {} (move and archive approach)",
old_id, rw.new_id
);
// Create new workspace with new id and name
info!("Creating new workspace row");
sqlx::query!(
"INSERT INTO workspace SELECT $1, $2, owner, deleted, premium FROM workspace WHERE id = $3",
"INSERT INTO workspace SELECT $1, $2, owner, false, premium FROM workspace WHERE id = $3",
&rw.new_id,
&rw.new_name,
&old_id
@@ -79,6 +74,44 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
// Duplicate workspace settings (keep copy in old workspace for reference)
info!("Duplicating workspace_settings table");
sqlx::query!(
"INSERT INTO workspace_settings SELECT $1, slack_team_id, slack_name, slack_command_script, slack_email, customer_id, plan, webhook, deploy_to, ai_config, large_file_storage, git_sync, default_app, default_scripts, deploy_ui, mute_critical_alerts, color, operator_settings, teams_command_script, teams_team_id, teams_team_name, git_app_installations, ducklake, slack_oauth_client_id, slack_oauth_client_secret, datatable, teams_team_guid, auto_invite, error_handler, success_handler FROM workspace_settings WHERE workspace_id = $2",
&rw.new_id,
&old_id
)
.execute(&mut *tx)
.await?;
info!("Duplicating workspace_key table");
sqlx::query!(
"INSERT INTO workspace_key SELECT $1, kind, key FROM workspace_key WHERE workspace_id = $2",
&rw.new_id,
&old_id
)
.execute(&mut *tx)
.await?;
info!("Updating workspace_env table");
sqlx::query!(
"UPDATE workspace_env SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
&old_id
)
.execute(&mut *tx)
.await?;
info!("Updating workspace_invite table");
sqlx::query!(
"UPDATE workspace_invite SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
&old_id
)
.execute(&mut *tx)
.await?;
info!("Updating account table");
sqlx::query!(
"UPDATE account SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
@@ -87,6 +120,7 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
info!("Updating app table");
sqlx::query!(
"UPDATE app SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
@@ -95,14 +129,7 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
sqlx::query!(
"UPDATE audit SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
&old_id
)
.execute(&mut *tx)
.await?;
info!("Updating capture table");
sqlx::query!(
"UPDATE capture SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
@@ -111,6 +138,7 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
info!("Updating capture_config table");
sqlx::query!(
"UPDATE capture_config SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
@@ -119,6 +147,7 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
info!("Updating http_trigger table");
sqlx::query!(
"UPDATE http_trigger SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
@@ -127,6 +156,7 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
info!("Updating websocket_trigger table");
sqlx::query!(
"UPDATE websocket_trigger SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
@@ -135,6 +165,7 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
info!("Updating kafka_trigger table");
sqlx::query!(
"UPDATE kafka_trigger SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
@@ -143,6 +174,7 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
info!("Updating nats_trigger table");
sqlx::query!(
"UPDATE nats_trigger SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
@@ -151,6 +183,7 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
info!("Updating postgres_trigger table");
sqlx::query!(
"UPDATE postgres_trigger SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
@@ -159,6 +192,7 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
info!("Updating mqtt_trigger table");
sqlx::query!(
"UPDATE mqtt_trigger SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
@@ -167,6 +201,7 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
info!("Updating gcp_trigger table");
sqlx::query!(
"UPDATE gcp_trigger SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
@@ -175,6 +210,7 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
info!("Updating sqs_trigger table");
sqlx::query!(
"UPDATE sqs_trigger SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
@@ -183,14 +219,25 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
info!("Updating email_trigger table");
sqlx::query!(
"UPDATE v2_job_completed SET workspace_id = $1 WHERE workspace_id = $2",
"UPDATE email_trigger SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
&old_id
)
.execute(&mut *tx)
.await?;
info!("Updating native_trigger table");
sqlx::query!(
"UPDATE native_trigger SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
&old_id
)
.execute(&mut *tx)
.await?;
info!("Updating dependency_map table");
sqlx::query!(
"UPDATE dependency_map SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
@@ -199,6 +246,7 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
info!("Updating deployment_metadata table");
sqlx::query!(
"UPDATE deployment_metadata SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
@@ -207,6 +255,7 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
info!("Updating draft table");
sqlx::query!(
"UPDATE draft SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
@@ -215,6 +264,7 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
info!("Updating favorite table");
sqlx::query!(
"UPDATE favorite SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
@@ -223,10 +273,12 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
// Duplicate flow table rows (FK constraint requires insert then delete)
info!("Duplicating flow table rows");
sqlx::query!(
"INSERT INTO flow
(workspace_id, path, summary, description, archived, extra_perms, dependency_job, draft_only, tag, ws_error_handler_muted, dedicated_worker, timeout, visible_to_runner_only, on_behalf_of_email, concurrency_key, versions, value, schema, edited_by, edited_at)
SELECT $1, path, summary, description, archived, extra_perms, dependency_job, draft_only, tag, ws_error_handler_muted, dedicated_worker, timeout, visible_to_runner_only, on_behalf_of_email, concurrency_key, versions, value, schema, edited_by, edited_at
"INSERT INTO flow
(workspace_id, path, summary, description, archived, extra_perms, dependency_job, draft_only, tag, ws_error_handler_muted, dedicated_worker, timeout, visible_to_runner_only, on_behalf_of_email, concurrency_key, versions, value, schema, edited_by, edited_at, lock_error_logs)
SELECT $1, path, summary, description, archived, extra_perms, dependency_job, draft_only, tag, ws_error_handler_muted, dedicated_worker, timeout, visible_to_runner_only, on_behalf_of_email, concurrency_key, versions, value, schema, edited_by, edited_at, lock_error_logs
FROM flow WHERE workspace_id = $2",
&rw.new_id,
&old_id
@@ -234,6 +286,7 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
info!("Updating flow_version table");
sqlx::query!(
"UPDATE flow_version SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
@@ -242,6 +295,7 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
info!("Updating workspace_runnable_dependencies table");
sqlx::query!(
"UPDATE workspace_runnable_dependencies SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
@@ -250,6 +304,86 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
info!("Updating workspace_dependencies table");
sqlx::query!(
"UPDATE workspace_dependencies SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
&old_id
)
.execute(&mut *tx)
.await?;
info!("Updating workspace_diff table");
sqlx::query!(
"UPDATE workspace_diff SET source_workspace_id = $1 WHERE source_workspace_id = $2",
&rw.new_id,
&old_id
)
.execute(&mut *tx)
.await?;
sqlx::query!(
"UPDATE workspace_diff SET fork_workspace_id = $1 WHERE fork_workspace_id = $2",
&rw.new_id,
&old_id
)
.execute(&mut *tx)
.await?;
info!("Updating workspace_integrations table");
sqlx::query!(
"UPDATE workspace_integrations SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
&old_id
)
.execute(&mut *tx)
.await?;
info!("Updating ai_agent_memory table");
sqlx::query!(
"UPDATE ai_agent_memory SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
&old_id
)
.execute(&mut *tx)
.await?;
info!("Updating flow_conversation table");
sqlx::query!(
"UPDATE flow_conversation SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
&old_id
)
.execute(&mut *tx)
.await?;
info!("Updating mcp_oauth_refresh_token table");
sqlx::query!(
"UPDATE mcp_oauth_refresh_token SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
&old_id
)
.execute(&mut *tx)
.await?;
info!("Updating mcp_oauth_server_code table");
sqlx::query!(
"UPDATE mcp_oauth_server_code SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
&old_id
)
.execute(&mut *tx)
.await?;
info!("Duplicating skip_workspace_diff_tally table");
sqlx::query!(
"INSERT INTO skip_workspace_diff_tally SELECT $1, added_at FROM skip_workspace_diff_tally WHERE workspace_id = $2",
&rw.new_id,
&old_id
)
.execute(&mut *tx)
.await?;
info!("Updating asset table");
sqlx::query!(
"UPDATE asset SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
@@ -258,6 +392,7 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
info!("Updating flow_node table");
sqlx::query!(
"UPDATE flow_node SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
@@ -266,11 +401,13 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
info!("Deleting old flow rows");
sqlx::query!("DELETE FROM flow WHERE workspace_id = $1", &old_id)
.execute(&mut *tx)
.await?;
// have to duplicate group_ with new workspace id because of foreign key constraint
// Duplicate group_ with new workspace id (FK constraint)
info!("Duplicating group_ table rows");
sqlx::query!(
"INSERT INTO group_ SELECT $1, name, summary, extra_perms FROM group_ WHERE workspace_id = $2",
&rw.new_id,
@@ -279,6 +416,7 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
info!("Updating usr_to_group table");
sqlx::query!(
"UPDATE usr_to_group SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
@@ -287,19 +425,45 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
// then delete old group_
sqlx::query!("DELETE FROM group_ WHERE workspace_id = $1", &old_id)
.execute(&mut *tx)
.await?;
info!("Updating group_permission_history table");
sqlx::query!(
"UPDATE folder SET workspace_id = $1 WHERE workspace_id = $2",
"UPDATE group_permission_history SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
&old_id
)
.execute(&mut *tx)
.await?;
info!("Deleting old group_ rows");
sqlx::query!("DELETE FROM group_ WHERE workspace_id = $1", &old_id)
.execute(&mut *tx)
.await?;
// Duplicate folders with new workspace id (FK constraint)
info!("Duplicating folder table rows");
sqlx::query!(
"INSERT INTO folder SELECT name, $1, display_name, owners, extra_perms, summary, edited_at, created_by FROM folder WHERE workspace_id = $2",
&rw.new_id,
&old_id
)
.execute(&mut *tx)
.await?;
info!("Updating folder_permission_history table");
sqlx::query!(
"UPDATE folder_permission_history SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
&old_id
)
.execute(&mut *tx)
.await?;
info!("Deleting old folder rows");
sqlx::query!("DELETE FROM folder WHERE workspace_id = $1", &old_id)
.execute(&mut *tx)
.await?;
info!("Updating input table");
sqlx::query!(
"UPDATE input SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
@@ -308,22 +472,27 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
sqlx::query!(
"UPDATE job_logs SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
// Get enabled schedules and clear their queued jobs BEFORE moving jobs
// This way we don't need to filter out scheduled jobs - they're already removed
let enabled_schedule_paths: Vec<String> = sqlx::query_scalar!(
"SELECT path FROM schedule WHERE workspace_id = $1 AND enabled = true",
&old_id
)
.execute(&mut *tx)
.fetch_all(&mut *tx)
.await?;
sqlx::query!(
"UPDATE job_stats SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
&old_id
)
.execute(&mut *tx)
.await?;
info!(
"Found {} enabled schedules, clearing their queued jobs",
enabled_schedule_paths.len()
);
for schedule_path in &enabled_schedule_paths {
crate::schedule::clear_schedule(&mut tx, schedule_path, &old_id).await?;
}
// Move queued jobs (not running) to new workspace using skip lock
// Scheduled jobs were already cleared above, so no need to filter them
info!("Moving v2_job_queue entries to new workspace");
sqlx::query!(
"UPDATE v2_job_queue SET workspace_id = $1
WHERE id IN (
@@ -339,17 +508,27 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
info!("Updating v2_job table for moved queue entries");
sqlx::query!(
"UPDATE v2_job SET workspace_id = $1
WHERE workspace_id = $2
AND (id IN (SELECT id FROM v2_job_queue WHERE workspace_id = $1)
OR id IN (SELECT id FROM v2_job_completed WHERE workspace_id = $1))",
AND id IN (SELECT id FROM v2_job_queue WHERE workspace_id = $1)",
&rw.new_id,
&old_id
)
.execute(&mut *tx)
.await?;
info!("Updating job_perms table for migrated jobs");
sqlx::query!(
"UPDATE job_perms SET workspace_id = $1
WHERE job_id IN (SELECT id FROM v2_job WHERE workspace_id = $1)",
&rw.new_id
)
.execute(&mut *tx)
.await?;
info!("Updating raw_app table");
sqlx::query!(
"UPDATE raw_app SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
@@ -358,6 +537,7 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
info!("Updating resource table");
sqlx::query!(
"UPDATE resource SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
@@ -366,6 +546,7 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
info!("Updating resource_type table");
sqlx::query!(
"UPDATE resource_type SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
@@ -374,6 +555,7 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
info!("Updating schedule table");
sqlx::query!(
"UPDATE schedule SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
@@ -382,6 +564,7 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
info!("Updating script table");
sqlx::query!(
"UPDATE script SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
@@ -390,6 +573,7 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
info!("Updating token table");
sqlx::query!(
"UPDATE token SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
@@ -398,6 +582,7 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
info!("Updating usage table");
sqlx::query!(
"UPDATE usage SET id = $1 WHERE id = $2 AND is_workspace = true",
&rw.new_id,
@@ -406,14 +591,16 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
info!("Duplicating usr table");
sqlx::query!(
"UPDATE usr SET workspace_id = $1 WHERE workspace_id = $2",
"INSERT INTO usr SELECT $1, username, email, is_admin, created_at, operator, disabled, role FROM usr WHERE workspace_id = $2",
&rw.new_id,
&old_id
)
.execute(&mut *tx)
.await?;
info!("Updating variable table");
sqlx::query!(
"UPDATE variable SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
@@ -422,43 +609,18 @@ pub(crate) async fn change_workspace_id(
.execute(&mut *tx)
.await?;
sqlx::query!(
"UPDATE workspace_env SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
&old_id
)
.execute(&mut *tx)
.await?;
sqlx::query!(
"UPDATE workspace_invite SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
&old_id
)
.execute(&mut *tx)
.await?;
sqlx::query!(
"UPDATE workspace_key SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
&old_id
)
.execute(&mut *tx)
.await?;
sqlx::query!(
"UPDATE workspace_settings SET workspace_id = $1 WHERE workspace_id = $2",
&rw.new_id,
&old_id
)
.execute(&mut *tx)
.await?;
// delete old workspace
sqlx::query!("DELETE FROM workspace WHERE id = $1", &old_id)
.execute(&mut *tx)
.await?;
// Re-push jobs for enabled schedules in the new workspace (same transaction)
info!(
"Re-pushing jobs for {} enabled schedules",
enabled_schedule_paths.len()
);
for schedule_path in &enabled_schedule_paths {
if let Some(schedule) = get_schedule_opt(&mut *tx, &rw.new_id, schedule_path).await? {
tx = push_scheduled_job(&db, tx, &schedule, None, None).await?;
}
}
// Audit log in the same transaction as the workspace changes
audit_log(
&mut *tx,
&authed,
@@ -466,13 +628,30 @@ pub(crate) async fn change_workspace_id(
ActionKind::Update,
&rw.new_id,
Some(&authed.email),
None,
Some(
[("old_workspace_id", old_id.as_str())]
.into_iter()
.collect::<HashMap<&str, &str>>(),
),
)
.await?;
tx.commit().await?;
// Archive old workspace: disable schedules, cancel remaining jobs, set deleted=true
// Note: schedules were already moved to new workspace, so this will find 0 schedules
info!("Archiving old workspace");
let (_schedules_count, canceled_count) =
archive_workspace_impl(&db, &old_id, &authed.username).await?;
info!(
"Workspace id change completed: moved {} to {}, archived old workspace",
old_id, rw.new_id
);
Ok(format!(
"updated workspace from {} to {}",
&old_id, &rw.new_id
"Moved workspace from {} to {}, archived old workspace (canceled {} remaining jobs)",
&old_id, &rw.new_id, canceled_count
))
}

View File

@@ -4,7 +4,17 @@
use crate::db::DB;
use crate::error::{Error, Result};
use serde::{Deserialize, Serialize};
use serde::{Deserialize, Deserializer, Serialize};
/// Deserializes an Option<String> where empty strings become None.
/// Use with `#[serde(default, deserialize_with = "empty_string_as_none")]`
pub fn empty_string_as_none<'de, D>(deserializer: D) -> std::result::Result<Option<String>, D::Error>
where
D: Deserializer<'de>,
{
let opt = Option::<String>::deserialize(deserializer)?;
Ok(opt.filter(|s| !s.is_empty()))
}
lazy_static::lazy_static! {
static ref OPENAI_AZURE_BASE_PATH: Option<String> = std::env::var("OPENAI_AZURE_BASE_PATH").ok();
@@ -13,6 +23,10 @@ lazy_static::lazy_static! {
pub const OPENAI_BASE_URL: &str = "https://api.openai.com/v1";
pub const GOOGLE_AI_BASE_URL: &str = "https://generativelanguage.googleapis.com/v1beta";
/// Empty string signals BedrockClient::from_env() to use the region from AWS environment/config
/// (e.g., AWS_REGION or AWS_DEFAULT_REGION env vars, or ~/.aws/config)
pub const USE_ENV_REGION: &str = "";
#[derive(Serialize, Deserialize, Debug, Eq, PartialEq, Hash, Clone)]
#[serde(rename_all = "lowercase")]
pub enum AIProvider {
@@ -36,10 +50,8 @@ impl AIProvider {
pub async fn get_base_url(
&self,
resource_base_url: Option<String>,
region: Option<String>,
db: &DB,
) -> Result<String> {
// If a base URL is provided in the resource, use it
if let Some(base_url) = resource_base_url {
return Ok(base_url);
}
@@ -74,31 +86,14 @@ impl AIProvider {
AIProvider::TogetherAI => Ok("https://api.together.xyz/v1".to_string()),
AIProvider::Anthropic => Ok("https://api.anthropic.com/v1".to_string()),
AIProvider::Mistral => Ok("https://api.mistral.ai/v1".to_string()),
p @ (AIProvider::CustomAI | AIProvider::AzureOpenAI) => {
if let Some(base_url) = resource_base_url {
Ok(base_url)
} else {
Err(Error::BadRequest(format!(
"{:?} provider requires a base URL in the resource",
p
)))
}
}
p @ (AIProvider::CustomAI | AIProvider::AzureOpenAI) => Err(Error::BadRequest(
format!("{:?} provider requires a base URL in the resource", p),
)),
AIProvider::AWSBedrock => {
#[cfg(feature = "bedrock")]
{
Ok(format!(
"https://bedrock-runtime.{}.amazonaws.com",
region.unwrap_or_else(|| "us-east-1".to_string())
))
}
#[cfg(not(feature = "bedrock"))]
{
let _ = region;
Err(Error::BadRequest(
"AWS Bedrock support is not enabled. Build with 'bedrock' feature.".to_string()
))
}
// AWS Bedrock uses the SDK directly, not HTTP base URL
Err(Error::internal_err(
"AWS Bedrock uses SDK directly, not HTTP base URL".to_string(),
))
}
}
}

View File

@@ -1146,6 +1146,7 @@ pub struct ListFlowQuery {
pub starred_only: Option<bool>,
pub include_draft_only: Option<bool>,
pub with_deployment_msg: Option<bool>,
pub dedicated_worker: Option<bool>,
}
pub fn add_virtual_items_if_necessary(modules: &mut Vec<FlowModule>) {

View File

@@ -152,8 +152,6 @@ lazy_static::lazy_static! {
pub static ref BASE_URL: Arc<RwLock<String>> = Arc::new(RwLock::new("".to_string()));
pub static ref IS_READY: std::sync::atomic::AtomicBool = std::sync::atomic::AtomicBool::new(false);
pub static ref BASE_INTERNAL_URL: String = std::env::var("BASE_INTERNAL_URL").unwrap_or("http://localhost:8000".to_string());
pub static ref HUB_BASE_URL: Arc<RwLock<String>> = Arc::new(RwLock::new(DEFAULT_HUB_BASE_URL.to_string()));

View File

@@ -687,6 +687,7 @@ pub struct ListScriptQuery {
pub with_deployment_msg: Option<bool>,
#[serde(default, deserialize_with = "from_seq")]
pub languages: Option<Vec<ScriptLang>>,
pub dedicated_worker: Option<bool>,
}
fn from_seq<'de, D>(deserializer: D) -> Result<Option<Vec<ScriptLang>>, D::Error>

View File

@@ -35,7 +35,7 @@ use crate::{
indexer::TantivyIndexerSettings,
server::Smtp,
utils::{merge_nested_raw_values_to_array, merge_raw_values_to_array},
KillpillSender, BASE_INTERNAL_URL, DB,
KillpillSender, DB,
};
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Default)]
@@ -219,6 +219,7 @@ lazy_static::lazy_static! {
worker_tags: Default::default(),
priority_tags_sorted: Default::default(),
dedicated_worker: Default::default(),
dedicated_workers: Default::default(),
cache_clear: Default::default(),
init_bash: Default::default(),
periodic_script_bash: Default::default(),
@@ -274,6 +275,8 @@ pub const ROOT_CACHE_NOMOUNT_DIR: &str = concatcp!(TMP_DIR, "/cache_nomount/");
pub static MIN_VERSION_IS_LATEST: AtomicBool = AtomicBool::new(false);
const DEFAULT_BASE_INTERNAL_URL: &str = "http://localhost:8000";
#[derive(Clone)]
pub struct HttpClient {
pub client: ClientWithMiddleware,
@@ -298,7 +301,7 @@ impl HttpClient {
let base_url = self
.base_internal_url
.clone()
.unwrap_or(BASE_INTERNAL_URL.clone().to_owned());
.unwrap_or(DEFAULT_BASE_INTERNAL_URL.to_owned());
let response_builder = self.client.post(format!("{}{}", base_url, url)).json(body);
@@ -327,7 +330,7 @@ impl HttpClient {
let base_url = self
.base_internal_url
.clone()
.unwrap_or(BASE_INTERNAL_URL.clone().to_owned());
.unwrap_or(DEFAULT_BASE_INTERNAL_URL.to_owned());
let response = self
.client
@@ -1226,7 +1229,6 @@ pub fn get_windmill_memory_usage() -> Option<i64> {
}
}
#[derive(Serialize, Deserialize)]
pub enum PingType {
Initial,
@@ -1242,6 +1244,7 @@ pub struct Ping {
pub ip: Option<String>,
pub tags: Option<Vec<String>>,
pub dw: Option<String>,
pub dws: Option<Vec<String>>,
pub version: Option<String>,
pub vcpus: Option<i64>,
pub memory: Option<i64>,
@@ -1297,6 +1300,7 @@ pub async fn update_ping_http(
&insert_ping.ip.unwrap(),
insert_ping.tags.unwrap_or_default().as_slice(),
insert_ping.dw,
insert_ping.dws.as_deref(),
&insert_ping.version.unwrap(),
insert_ping.vcpus,
insert_ping.memory,
@@ -1427,6 +1431,7 @@ pub async fn insert_ping_query(
ip: &str,
tags: &[String],
dw: Option<String>,
dws: Option<&[String]>,
version: &str,
vcpus: Option<i64>,
memory: Option<i64>,
@@ -1434,14 +1439,15 @@ pub async fn insert_ping_query(
db: &DB,
) -> anyhow::Result<()> {
sqlx::query!(
"INSERT INTO worker_ping (worker_instance, worker, ip, custom_tags, worker_group, dedicated_worker, wm_version, vcpus, memory, job_isolation) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10) ON CONFLICT (worker)
DO UPDATE set ip = EXCLUDED.ip, custom_tags = EXCLUDED.custom_tags, worker_group = EXCLUDED.worker_group",
"INSERT INTO worker_ping (worker_instance, worker, ip, custom_tags, worker_group, dedicated_worker, dedicated_workers, wm_version, vcpus, memory, job_isolation) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11) ON CONFLICT (worker)
DO UPDATE set ip = EXCLUDED.ip, custom_tags = EXCLUDED.custom_tags, worker_group = EXCLUDED.worker_group, dedicated_workers = EXCLUDED.dedicated_workers",
worker_instance,
worker_name,
ip,
tags,
worker_group,
dw,
dws,
version,
vcpus,
memory,
@@ -1613,6 +1619,30 @@ pub async fn load_worker_config(
}
})
.transpose()?;
// Parse dedicated_workers (multiple dedicated workers)
let dedicated_workers = config
.dedicated_workers
.map(|workers| {
workers
.into_iter()
.map(|x| {
let splitted = x.split(':').to_owned().collect_vec();
if splitted.len() != 2 {
return Err(anyhow::anyhow!(
"Invalid dedicated_workers format. Got {x}, expects <workspace_id>:<path>"
));
}
let workspace = splitted[0];
let script_path = splitted[1];
Ok(WorkspacedPath {
workspace_id: workspace.to_string(),
path: script_path.to_string(),
})
})
.collect::<Result<Vec<_>, _>>()
})
.transpose()?;
if *WORKER_GROUP == "default" && dedicated_worker.is_none() {
let mut all_tags = config
.worker_tags
@@ -1646,7 +1676,18 @@ pub async fn load_worker_config(
let worker_tags = config
.worker_tags
.or_else(|| {
if let Some(ref dedicated_worker) = dedicated_worker.as_ref() {
// Check for multiple dedicated workers first
if let Some(ref dws) = dedicated_workers.as_ref() {
let mut dedi_tags: Vec<String> = dws
.iter()
.map(|dw| format!("{}:{}", dw.workspace_id, dw.path))
.collect();
if std::env::var("ADD_FLOW_TAG").is_ok() {
dedi_tags.push("flow".to_string());
}
Some(dedi_tags)
} else if let Some(ref dedicated_worker) = dedicated_worker.as_ref() {
// Fallback to single dedicated worker for backward compatibility
let mut dedi_tags = vec![format!(
"{}:{}",
dedicated_worker.workspace_id, dedicated_worker.path
@@ -1742,6 +1783,7 @@ pub async fn load_worker_config(
worker_tags,
priority_tags_sorted,
dedicated_worker,
dedicated_workers,
init_bash: config
.init_bash
.or_else(|| load_init_bash_from_env())
@@ -1835,6 +1877,7 @@ pub struct WorkerConfigOpt {
pub worker_tags: Option<Vec<String>>,
pub priority_tags: Option<HashMap<String, u8>>,
pub dedicated_worker: Option<String>,
pub dedicated_workers: Option<Vec<String>>,
pub init_bash: Option<String>,
pub periodic_script_bash: Option<String>,
pub periodic_script_interval_seconds: Option<u64>,
@@ -1851,6 +1894,7 @@ impl Default for WorkerConfigOpt {
worker_tags: Default::default(),
priority_tags: Default::default(),
dedicated_worker: Default::default(),
dedicated_workers: Default::default(),
init_bash: Default::default(),
periodic_script_bash: Default::default(),
periodic_script_interval_seconds: Default::default(),
@@ -1868,6 +1912,7 @@ pub struct WorkerConfig {
pub worker_tags: Vec<String>,
pub priority_tags_sorted: Vec<PriorityTags>,
pub dedicated_worker: Option<WorkspacedPath>,
pub dedicated_workers: Option<Vec<WorkspacedPath>>,
pub init_bash: Option<String>,
pub periodic_script_bash: Option<String>,
pub periodic_script_interval_seconds: Option<u64>,
@@ -1879,8 +1924,8 @@ pub struct WorkerConfig {
impl std::fmt::Debug for WorkerConfig {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "WorkerConfig {{ worker_tags: {:?}, priority_tags_sorted: {:?}, dedicated_worker: {:?}, init_bash: {:?}, periodic_script_bash: {:?}, periodic_script_interval_seconds: {:?}, cache_clear: {:?}, additional_python_paths: {:?}, pip_local_dependencies: {:?}, env_vars: {:?} }}",
self.worker_tags, self.priority_tags_sorted, self.dedicated_worker, self.init_bash, self.periodic_script_bash, self.periodic_script_interval_seconds, self.cache_clear, self.additional_python_paths, self.pip_local_dependencies, self.env_vars.iter().map(|(k, v)| format!("{}: {}{} ({} chars)", k, &v[..3.min(v.len())], "***", v.len())).collect::<Vec<String>>().join(", "))
write!(f, "WorkerConfig {{ worker_tags: {:?}, priority_tags_sorted: {:?}, dedicated_worker: {:?}, dedicated_workers: {:?}, init_bash: {:?}, periodic_script_bash: {:?}, periodic_script_interval_seconds: {:?}, cache_clear: {:?}, additional_python_paths: {:?}, pip_local_dependencies: {:?}, env_vars: {:?} }}",
self.worker_tags, self.priority_tags_sorted, self.dedicated_worker, self.dedicated_workers, self.init_bash, self.periodic_script_bash, self.periodic_script_interval_seconds, self.cache_clear, self.additional_python_paths, self.pip_local_dependencies, self.env_vars.iter().map(|(k, v)| format!("{}: {}{} ({} chars)", k, &v[..3.min(v.len())], "***", v.len())).collect::<Vec<String>>().join(", "))
}
}

View File

@@ -18,7 +18,7 @@ pub struct IndexReader;
pub struct IndexWriter;
#[cfg(not(feature = "private"))]
pub async fn init_index(_db: &Pool<Postgres>) -> Result<(IndexReader, IndexWriter), Error> {
pub async fn init_index(_db: &Pool<Postgres>) -> Result<Option<(IndexReader, IndexWriter)>, Error> {
Err(anyhow!("Cannot initialize index: not in EE").into())
}

View File

@@ -22,7 +22,7 @@ pub struct ServiceLogIndexWriter;
pub async fn init_index(
_db: &Pool<Postgres>,
mut _killpill_tx: KillpillSender,
) -> Result<(ServiceLogIndexReader, ServiceLogIndexWriter), Error> {
) -> Result<Option<(ServiceLogIndexReader, ServiceLogIndexWriter)>, Error> {
Err(anyhow!("Cannot initialize index: not in EE").into())
}

View File

@@ -22,7 +22,8 @@ flow_testing = []
cloud = []
sqlx = []
deno_core = ["dep:deno_fetch", "dep:deno_webidl", "dep:deno_web", "dep:deno_net", "dep:deno_console", "dep:deno_url", "dep:deno_core",
"dep:deno_ast", "dep:deno_tls", "dep:deno_permissions", "dep:deno_io", "dep:deno_runtime", "dep:deno_telemetry", "dep:deno_error", "dep:winapi", "dep:rustls-pemfile"]
"dep:deno_ast", "dep:deno_tls", "dep:deno_permissions", "dep:deno_io", "dep:deno_runtime", "dep:deno_telemetry", "dep:deno_error", "dep:winapi", "dep:rustls-pemfile",
"quickjs"]
libffi_mac = ["dep:libffi-sys"]
otel = ["windmill-common/otel", "dep:opentelemetry", "dep:tracing-opentelemetry"]
dind = ["dep:bollard"]
@@ -36,6 +37,7 @@ nu = ["dep:windmill-parser-nu"]
java = ["dep:windmill-parser-java"]
ruby = ["dep:windmill-parser-ruby"]
duckdb = ["dep:libloading"]
quickjs = ["dep:rquickjs"]
bedrock = ["dep:aws-sdk-bedrockruntime", "windmill-common/bedrock"]
[dependencies]
@@ -145,6 +147,7 @@ prost.workspace = true
axum.workspace = true
bollard = { workspace = true, optional = true }
oracle = { workspace = true, optional = true }
rquickjs = { workspace = true, optional = true }
hudsucker.workspace = true
hyper-http-proxy.workspace = true
hyper-tls.workspace = true

View File

@@ -51,10 +51,9 @@ impl BedrockQueryBuilder {
) -> Result<ParsedResponse, Error> {
let bedrock_client = if !api_key.is_empty() {
BedrockClient::from_bearer_token(api_key.to_string(), region).await?
} else if let (Some(access_key_id), Some(secret_access_key)) = (
aws_access_key_id.filter(|s| !s.is_empty()),
aws_secret_access_key.filter(|s| !s.is_empty()),
) {
} else if let (Some(access_key_id), Some(secret_access_key)) =
(aws_access_key_id, aws_secret_access_key)
{
BedrockClient::from_credentials(
access_key_id.to_string(),
secret_access_key.to_string(),

View File

@@ -14,7 +14,11 @@ pub struct McpToolSource {
pub resource_path: String,
}
use windmill_common::{
ai_providers::AIProvider, db::DB, error::Error, flow_status::AgentAction, flows::FlowModule,
ai_providers::{empty_string_as_none, AIProvider},
db::DB,
error::Error,
flow_status::AgentAction,
flows::FlowModule,
s3_helpers::S3Object,
};
use windmill_parser::Typ;
@@ -162,17 +166,18 @@ pub enum AnthropicPlatform {
#[derive(Deserialize, Debug)]
pub struct ProviderResource {
#[serde(alias = "apiKey")]
pub api_key: String,
#[serde(alias = "baseUrl")]
#[serde(alias = "apiKey", default, deserialize_with = "empty_string_as_none")]
pub api_key: Option<String>,
#[serde(alias = "baseUrl", default, deserialize_with = "empty_string_as_none")]
pub base_url: Option<String>,
#[allow(dead_code)]
#[serde(default, deserialize_with = "empty_string_as_none")]
pub region: Option<String>,
#[allow(dead_code)]
#[serde(alias = "awsAccessKeyId")]
#[serde(alias = "awsAccessKeyId", default, deserialize_with = "empty_string_as_none")]
pub aws_access_key_id: Option<String>,
#[allow(dead_code)]
#[serde(alias = "awsSecretAccessKey")]
#[serde(alias = "awsSecretAccessKey", default, deserialize_with = "empty_string_as_none")]
pub aws_secret_access_key: Option<String>,
/// Platform for Anthropic API (standard or google_vertex_ai)
#[serde(default)]
@@ -187,8 +192,8 @@ pub struct ProviderWithResource {
}
impl ProviderWithResource {
pub fn get_api_key(&self) -> &str {
&self.resource.api_key
pub fn get_api_key(&self) -> Option<&str> {
self.resource.api_key.as_deref()
}
pub fn get_model(&self) -> &str {
@@ -199,7 +204,6 @@ impl ProviderWithResource {
self.kind
.get_base_url(
self.resource.base_url.clone(),
self.resource.region.clone(),
db,
)
.await

View File

@@ -21,7 +21,7 @@ use windmill_mcp::McpClient;
#[cfg(not(feature = "mcp"))]
use crate::ai::tools::McpClientStub as McpClient;
use windmill_common::{
ai_providers::AIProvider,
ai_providers::{AIProvider},
cache,
client::AuthedClient,
db::DB,
@@ -410,9 +410,14 @@ pub async fn run_agent(
has_websearch: bool,
) -> error::Result<Box<RawValue>> {
let output_type = args.output_type.as_ref().unwrap_or(&OutputType::Text);
let base_url = args.provider.get_base_url(db).await?;
let api_key = args.provider.get_api_key();
// Skip get_base_url for Bedrock - it uses SDK directly, not HTTP
let base_url = if args.provider.kind == AIProvider::AWSBedrock {
String::new()
} else {
args.provider.get_base_url(db).await?
};
let api_key = args.provider.get_api_key().unwrap_or("");
// Create the query builder for the provider
let query_builder = create_query_builder(&args.provider);
@@ -660,12 +665,7 @@ pub async fn run_agent(
let parsed = if args.provider.kind == AIProvider::AWSBedrock {
#[cfg(feature = "bedrock")]
{
let region = args.provider.get_region();
let Some(region) = region else {
return Err(Error::internal_err(
"AWS Bedrock region is required".to_string(),
));
};
let region = args.provider.get_region().unwrap_or(windmill_common::ai_providers::USE_ENV_REGION);
// Use Bedrock SDK via dedicated query builder
crate::ai::providers::bedrock::BedrockQueryBuilder::default()
.execute_request(

View File

@@ -635,10 +635,9 @@ async fn pull_codebase(w_id: &str, id: &str, job_dir: &str) -> Result<PulledCode
&& object_store.is_none()
{
let bun_cache_path = format!(
"{}/{}.{}",
"{}/{}",
*windmill_common::worker::ROOT_STANDALONE_BUNDLE_DIR,
path,
if is_tar { "tar" } else { "js" }
path
);
if std::fs::metadata(&bun_cache_path).is_ok() {
tracing::info!("loading {bun_cache_path} from standalone bundle cache");
@@ -1328,22 +1327,6 @@ try {{
let stream_notifier = StreamNotifier::new(conn, job);
// Set job context for OTEL tracing (EE only)
#[cfg(all(feature = "private", feature = "enterprise"))]
{
let tracing_enabled =
crate::worker::is_otel_tracing_proxy_enabled_for_lang(&ScriptLang::Nativets)
.await;
tracing::debug!(
"nativets job {}: OTEL tracing enabled={}",
job.id,
tracing_enabled
);
if tracing_enabled {
crate::otel_tracing_proxy_ee::set_current_job_context(job.id).await;
}
}
let result = crate::js_eval::eval_fetch_timeout(
env_code,
inner_content.clone(),

View File

@@ -533,6 +533,7 @@ pub async fn update_worker_ping_for_failed_init_script(
ip: None,
tags: None,
dw: None,
dws: None,
jobs_executed: None,
occupancy_rate: None,
occupancy_rate_15s: None,

View File

@@ -29,6 +29,8 @@ use deno_web::{BlobStore, TimersPermission};
#[cfg(feature = "deno_core")]
use itertools::Itertools;
use lazy_static::lazy_static;
#[cfg(feature = "quickjs")]
use once_cell::sync::Lazy;
use regex::Regex;
use serde_json::value::RawValue;
use sqlx::types::Json;
@@ -324,9 +326,18 @@ async fn handle_full_regex(
};
let result = if obj_name == "results" {
authed_client
.get_result_by_id(&by_id.flow_job.to_string(), obj_key, query)
// Use .ok() to match deno_core op_get_id behavior: return null for non-existent steps
// instead of throwing an error
let res = authed_client
.get_result_by_id::<Option<Box<RawValue>>>(&by_id.flow_job.to_string(), obj_key, query)
.await
.ok()
.flatten();
match res {
Some(v) => Ok(v),
None => serde_json::value::to_raw_value(&serde_json::Value::Null)
.map_err(|e| anyhow::anyhow!("Failed to serialize null: {}", e)),
}
} else if obj_name == "flow_env" {
authed_client
.get_flow_env_by_flow_job_id(&by_id.flow_job.to_string(), obj_key, query)
@@ -393,6 +404,35 @@ pub async fn eval_timeout(
}
}
// Use QuickJS if enabled and either deno_core is not available or USE_QUICKJS env var is set
#[cfg(all(feature = "quickjs", not(feature = "deno_core")))]
{
return crate::js_eval_quickjs::eval_timeout_quickjs(
expr,
transform_context,
flow_input,
flow_env,
authed_client,
by_id,
ctx,
)
.await;
}
#[cfg(all(feature = "quickjs", feature = "deno_core"))]
if *USE_QUICKJS {
return crate::js_eval_quickjs::eval_timeout_quickjs(
expr,
transform_context,
flow_input,
flow_env,
authed_client,
by_id,
ctx,
)
.await;
}
#[cfg(not(feature = "deno_core"))]
{
#[allow(unreachable_code)]
@@ -522,8 +562,8 @@ pub async fn eval_timeout(
}
}
#[cfg(feature = "deno_core")]
fn replace_with_await(expr: String, fn_name: &str) -> String {
#[cfg(any(feature = "deno_core", feature = "quickjs"))]
pub fn replace_with_await(expr: String, fn_name: &str) -> String {
let sep = format!("{}(", fn_name);
let mut split = expr.split(&sep);
let mut s = split.next().unwrap_or_else(|| "").to_string();
@@ -545,12 +585,16 @@ lazy_static! {
Regex::new(r"^(https?)://(([^:@\s]+):([^:@\s]+)@)?([^:@\s]+)(:(\d+))?$").unwrap();
}
#[cfg(feature = "deno_core")]
fn replace_with_await_result(expr: String) -> String {
#[cfg(feature = "quickjs")]
#[allow(dead_code)] // Only used when both quickjs and deno_core features are enabled
static USE_QUICKJS: Lazy<bool> = Lazy::new(|| std::env::var("USE_QUICKJS_FOR_FLOW_EVAL").is_ok());
#[cfg(any(feature = "deno_core", feature = "quickjs"))]
pub fn replace_with_await_result(expr: String) -> String {
RE.replace_all(&expr, "(await $r)").to_string()
}
#[cfg(feature = "deno_core")]
#[cfg(any(feature = "deno_core", feature = "quickjs"))]
fn add_closing_bracket(s: &str) -> String {
let mut s = s.to_string();
let mut level = 1;
@@ -1135,7 +1179,9 @@ pub async fn eval_fetch_timeout(
// We call the function exposed by runtime.js since we can't dynamically import ext: modules.
#[cfg(all(feature = "private", feature = "enterprise"))]
if crate::DENO_OTEL_INITIALIZED.load(std::sync::atomic::Ordering::SeqCst) {
if let Err(e) = js_runtime.execute_script("<otel_bootstrap>", "globalThis.__bootstrapOtel()") {
if let Err(e) =
js_runtime.execute_script("<otel_bootstrap>", "globalThis.__bootstrapOtel()")
{
tracing::warn!("Failed to bootstrap OTEL telemetry: {}", e);
}
}
@@ -1331,6 +1377,26 @@ async fn eval_fetch(
.context("failed to load module")?;
let main_override = script_entrypoint_override.unwrap_or("main".to_string());
// Inject parent trace context using enterSpan with a duck-typed span object.
// Uses job_id as trace_id so all spans are linked to the job.
// span_id is a placeholder - it gets overwritten by the OTLP handler with the real parent span_id.
#[cfg(all(feature = "private", feature = "enterprise"))]
let otel_context_inject = if crate::DENO_OTEL_INITIALIZED.load(std::sync::atomic::Ordering::SeqCst) {
let trace_id = job_id.as_simple().to_string();
format!(
r#"globalThis.__enterSpan?.({{
isRecording: () => true,
spanContext: () => ({{ traceId: "{trace_id}", spanId: "ffffffffffffffff", traceFlags: 1 }})
}});"#
)
} else {
String::new()
};
#[cfg(not(all(feature = "private", feature = "enterprise")))]
let otel_context_inject = "";
let script = js_runtime
.execute_script(
"<anon>",
@@ -1343,7 +1409,7 @@ function isAsyncIterable(obj) {{
function processStreamIterative(res) {{
const iterator = res[Symbol.asyncIterator]();
function processLoop() {{
return new Promise(function(resolve) {{
function step() {{
@@ -1363,10 +1429,12 @@ function processStreamIterative(res) {{
step();
}});
}}
return processLoop();
}}
{otel_context_inject}
let args = Deno.core.ops.op_get_static_args().map(JSON.parse)
import("file:///eval.ts").then((module) => module.{main_override}(...args))
.then(res => {{

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,797 @@
/*
* Author: Ruben Fiszel
* Copyright: Windmill Labs, Inc 2022
* This file and its contents are licensed under the AGPLv3 License.
* Please see the included NOTICE for copyright information and
* LICENSE-AGPL for a copy of the license.
*/
//! QuickJS-based JavaScript expression evaluation for flow transformations.
//!
//! This module provides an alternative to deno_core for evaluating arbitrary JavaScript
//! expressions in flow transformations. QuickJS offers significantly faster startup times
//! (~200μs vs ~3ms for V8), making it ideal for evaluating many small expressions.
//!
//! ## Performance Characteristics (release mode benchmarks)
//! - **Simple expressions**: ~238μs (QuickJS) vs ~3.05ms (deno_core) = **~13x faster**
//! - **Complex expressions**: ~192μs (QuickJS) vs ~3.09ms (deno_core) = **~16x faster**
//! - **Memory**: ~2.5% of V8's footprint
//!
//! For flow expression evaluation where startup time dominates, QuickJS is significantly
//! faster overall despite being slower for long-running CPU-intensive code.
//!
//! ## Async Operations
//! This implementation uses true async Rust callbacks (similar to deno_core's ops) for
//! `variable()`, `resource()`, and `results.xxx` access. The async functions use
//! rquickjs's `Async<MutFn<...>>` wrapper which returns JavaScript Promises that are
//! resolved when the Rust async operations complete. No pre-fetching is required.
use std::collections::HashMap;
use std::sync::Arc;
use rquickjs::{
async_with,
prelude::{Async, Func, MutFn},
AsyncContext, AsyncRuntime, CatchResultExt, FromJs, IntoJs, Object, Value,
};
use serde_json::value::RawValue;
use windmill_common::client::AuthedClient;
use windmill_common::flow_status::JobResult;
use crate::js_eval::{replace_with_await, replace_with_await_result, IdContext};
/// Shared state for async operations within QuickJS
#[derive(Clone)]
struct AsyncOpState {
client: AuthedClient,
}
/// Evaluates a JavaScript expression using QuickJS runtime.
///
/// This function provides the same interface as `eval_timeout` but uses QuickJS
/// instead of deno_core/V8 for potentially faster startup times.
///
/// Unlike deno_core, this uses true async Rust callbacks for `variable()`,
/// `resource()`, and `results.xxx` access - no pre-fetching required.
pub async fn eval_timeout_quickjs(
expr: String,
transform_context: HashMap<String, Arc<Box<RawValue>>>,
flow_input: Option<mappable_rc::Marc<HashMap<String, Box<RawValue>>>>,
flow_env: Option<&HashMap<String, Box<RawValue>>>,
authed_client: Option<&AuthedClient>,
by_id: Option<&IdContext>,
ctx: Option<Vec<(String, String)>>,
) -> anyhow::Result<Box<RawValue>> {
let expr = expr.trim().to_string();
tracing::debug!(
"evaluating js eval (quickjs): {} with context {:?}",
expr,
transform_context
);
// Clone data for the blocking task
let by_id_clone = by_id.cloned();
let flow_input_clone = flow_input.clone();
let flow_env_clone = flow_env.cloned();
let authed_client_clone = authed_client.cloned();
// Determine which context keys are actually used in the expression
let p_ids = by_id.map(|x| {
[
format!("results.{}", x.previous_id),
format!("results?.{}", x.previous_id),
format!("results[\"{}\"]", x.previous_id),
format!("results?.[\"{}\"]", x.previous_id),
]
});
let mut context_keys: Vec<String> = transform_context
.keys()
.filter(|x| expr.contains(&x.to_string()))
.cloned()
.collect();
if !context_keys.contains(&"previous_result".to_string())
&& (p_ids.is_some() && p_ids.as_ref().unwrap().iter().any(|x| expr.contains(x)))
|| expr.contains("error")
{
context_keys.push("previous_result".to_string());
}
let has_flow_input = expr.contains("flow_input");
if has_flow_input {
context_keys.push("flow_input".to_string())
}
// Filter transform_context to only include used keys
let filtered_context: HashMap<String, Arc<Box<RawValue>>> = transform_context
.into_iter()
.filter(|(k, _)| context_keys.contains(k))
.collect();
let expr_clone = expr.clone();
// Run the QuickJS evaluation with a timeout
tokio::time::timeout(
std::time::Duration::from_millis(10000),
tokio::task::spawn_blocking(move || {
// Create a new tokio runtime for async operations within the blocking context
let rt = tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()?;
rt.block_on(async move {
eval_quickjs_inner(
&expr_clone,
filtered_context,
flow_input_clone,
flow_env_clone,
authed_client_clone,
by_id_clone,
ctx,
context_keys,
)
.await
})
}),
)
.await
.map_err(|_| {
anyhow::anyhow!(
"The expression evaluation `{expr}` took too long to execute (>10000ms)"
)
})??
}
/// Memory limit for QuickJS runtime (32MB).
/// This is much smaller than deno_core's 128MB limit since flow expressions
/// should be lightweight transformations, not memory-intensive operations.
const QUICKJS_MEMORY_LIMIT: usize = 32 * 1024 * 1024;
async fn eval_quickjs_inner(
expr: &str,
transform_context: HashMap<String, Arc<Box<RawValue>>>,
flow_input: Option<mappable_rc::Marc<HashMap<String, Box<RawValue>>>>,
flow_env: Option<HashMap<String, Box<RawValue>>>,
authed_client: Option<AuthedClient>,
by_id: Option<IdContext>,
extra_ctx: Option<Vec<(String, String)>>,
context_keys: Vec<String>,
) -> anyhow::Result<Box<RawValue>> {
let runtime = AsyncRuntime::new()?;
runtime.set_memory_limit(QUICKJS_MEMORY_LIMIT).await;
let context = AsyncContext::full(&runtime).await?;
// Create shared state for async ops if we have a client
let op_state = authed_client.map(|client| Arc::new(AsyncOpState { client }));
let op_state_clone = op_state.clone();
let by_id_clone = by_id.clone();
// Transform expression to add await for variable/resource/results access
let expr_with_funcs = ["variable", "resource"]
.into_iter()
.fold(expr.to_string(), replace_with_await);
let transformed_expr = replace_with_await_result(expr_with_funcs);
async_with!(context => |ctx| {
let globals = ctx.globals();
// Set up context variables
for key in &context_keys {
if key == "flow_input" {
if let Some(ref fi) = flow_input {
let json_str = serde_json::to_string(fi.as_ref())?;
let val: serde_json::Value = serde_json::from_str(&json_str)?;
let js_val = json_to_js(&ctx, &val)?;
globals.set(key.as_str(), js_val)?;
} else {
globals.set(key.as_str(), Value::new_null(ctx.clone()))?;
}
} else if let Some(raw_val) = transform_context.get(key) {
let val: serde_json::Value = serde_json::from_str(raw_val.get())?;
let js_val = json_to_js(&ctx, &val)?;
globals.set(key.as_str(), js_val)?;
}
}
// Set up flow_env if referenced
if expr.contains("flow_env") {
if let Some(ref fe) = flow_env {
let obj = Object::new(ctx.clone())?;
for (k, v) in fe {
let val: serde_json::Value = serde_json::from_str(v.get())?;
let js_val = json_to_js(&ctx, &val)?;
obj.set(k.as_str(), js_val)?;
}
globals.set("flow_env", obj)?;
} else {
globals.set("flow_env", Object::new(ctx.clone())?)?;
}
}
// Set up additional context variables
if let Some(ctx_vars) = extra_ctx {
for (k, v) in ctx_vars {
globals.set(k.as_str(), v.as_str())?;
}
}
// Set up error extraction if needed
if expr.contains("error") && context_keys.contains(&"previous_result".to_string()) {
let error_setup = r#"
let error = previous_result?.error;
if (!error) {
if (Array.isArray(previous_result)) {
const errors = previous_result.filter(item => item && typeof item === 'object' && 'error' in item);
if (errors.length === 1) {
error = errors[0].error;
} else if (errors.length > 1) {
error = {
name: 'MultipleErrors',
message: errors.map(({ error: e }, i) => `[${e.step_id || i}] ${e.message || e.name}`).join('; '),
errors: previous_result
};
} else {
error = {
name: 'MultipleErrors',
message: "Could not parse errors",
errors: previous_result
};
}
} else {
if (previous_result) {
error = { name: 'UnknownError', message: 'Could not parse the error', error: previous_result };
} else {
error = { name: 'UnknownError', message: 'No error found' };
}
}
}
"#;
ctx.eval::<(), _>(error_setup).catch(&ctx).map_err(quickjs_error_to_anyhow)?;
}
// Set up async functions if we have a client
if let Some(ref state) = op_state_clone {
setup_async_ops(&ctx, &globals, state.clone())?;
} else {
// Set up stub functions that throw errors
setup_stub_functions(&ctx, &globals)?;
}
// Set up results proxy if we have by_id context
if let Some(ref by_id) = by_id_clone {
setup_results_proxy(&ctx, &globals, by_id, op_state_clone.clone())?;
}
// Determine if we need to add return statement
let code = if should_add_return_quickjs(&transformed_expr) {
format!("(async function() {{ return {}; }})()", transformed_expr)
} else {
format!("(async function() {{ {} }})()", transformed_expr)
};
// Evaluate the expression (returns a Promise)
let promise: rquickjs::Promise = ctx.eval(code).catch(&ctx).map_err(quickjs_error_to_anyhow)?;
// Await the promise
let result: Value = promise.into_future().await.catch(&ctx).map_err(quickjs_error_to_anyhow)?;
// Convert result to JSON
let json_result = js_to_json(&ctx, &result)?;
let json_str = serde_json::to_string(&json_result)?;
Ok(windmill_common::worker::to_raw_value(&serde_json::from_str::<serde_json::Value>(&json_str)?))
})
.await
}
/// Set up async variable() and resource() functions using true Rust async callbacks.
///
/// This uses rquickjs's `Async<MutFn<...>>` wrapper to create JavaScript functions that
/// return Promises. The Promises are resolved by spawned Rust async operations.
fn setup_async_ops<'js>(
ctx: &rquickjs::Ctx<'js>,
globals: &Object<'js>,
state: Arc<AsyncOpState>,
) -> anyhow::Result<()> {
// Error prefix - must match the JavaScript side
const ERR_PREFIX: &str = "\x00__WINDMILL_ERR__\x00";
// Create variable() function with true async Rust callback
// Returns a JSON string that JavaScript will parse
let state_for_var = state.clone();
globals.set(
"__fetchVariable",
Func::from(Async(MutFn::new(move |path: String| {
let client = state_for_var.client.clone();
async move {
match client.get_variable_value(&path).await {
Ok(value) => value,
Err(e) => format!("{}{}", ERR_PREFIX, e),
}
}
}))),
)?;
// Create resource() function - returns JSON string
let state_for_res = state.clone();
globals.set(
"__fetchResource",
Func::from(Async(MutFn::new(move |path: String| {
let client = state_for_res.client.clone();
async move {
match client
.get_resource_value_interpolated::<serde_json::Value>(&path, None)
.await
{
Ok(value) => serde_json::to_string(&value).unwrap_or_else(|_| "null".to_string()),
Err(e) => format!("{}{}", ERR_PREFIX, e),
}
}
}))),
)?;
// Create JavaScript wrappers that parse the JSON results
// We use a unique prefix that's extremely unlikely to appear in real data
let wrapper_code = r#"
const __ERR_PREFIX = '\x00__WINDMILL_ERR__\x00';
async function variable(path) {
const result = await __fetchVariable(path);
if (typeof result === 'string' && result.startsWith(__ERR_PREFIX)) {
throw new Error(result.substring(__ERR_PREFIX.length));
}
return result;
}
async function resource(path) {
const result = await __fetchResource(path);
if (typeof result === 'string' && result.startsWith(__ERR_PREFIX)) {
throw new Error(result.substring(__ERR_PREFIX.length));
}
return JSON.parse(result);
}
"#;
ctx.eval::<(), _>(wrapper_code)
.catch(ctx)
.map_err(quickjs_error_to_anyhow)?;
Ok(())
}
/// Set up stub functions that throw errors when no client is available
fn setup_stub_functions<'js>(
ctx: &rquickjs::Ctx<'js>,
_globals: &Object<'js>,
) -> anyhow::Result<()> {
let setup_code = r#"
function variable(path) {
return Promise.reject(new Error(`variable() is not available without an authenticated client`));
}
function resource(path) {
return Promise.reject(new Error(`resource() is not available without an authenticated client`));
}
"#;
ctx.eval::<(), _>(setup_code)
.catch(ctx)
.map_err(quickjs_error_to_anyhow)?;
Ok(())
}
/// Set up the `results` Proxy object with dynamic access to step results.
///
/// Uses async Rust callbacks to fetch results on-demand when accessed.
fn setup_results_proxy<'js>(
ctx: &rquickjs::Ctx<'js>,
globals: &Object<'js>,
by_id: &IdContext,
op_state: Option<Arc<AsyncOpState>>,
) -> anyhow::Result<()> {
// Store previous_id for the shortcut optimization
globals.set("__previous_id", by_id.previous_id.clone())?;
// Create async __getResult function that fetches step results via Rust
if let Some(state) = op_state {
let by_id_for_result = by_id.clone();
globals.set(
"__fetchResult",
Func::from(Async(MutFn::new(move |step_id: String| {
let client = state.client.clone();
let by_id = by_id_for_result.clone();
let step_id_clone = step_id.clone();
// Look up the job ID(s) for this step from the local cache
let job_result = by_id.steps_results.get(&step_id).cloned();
let flow_job_id = by_id.flow_job.to_string();
async move {
const ERR_PREFIX: &str = "\x00__WINDMILL_ERR__\x00";
let result: Result<serde_json::Value, String> = match job_result {
Some(jr) => {
// Found in local cache, fetch result by job ID
match jr {
JobResult::SingleJob(job_id) => {
client
.get_completed_job_result::<serde_json::Value>(&job_id.to_string(), None)
.await
.map_err(|e| format!("Failed to fetch result for step '{}': {}", step_id_clone, e))
}
JobResult::ListJob(job_ids) => {
let futs = job_ids.iter().map(|job_id| {
let client = client.clone();
let job_id_str = job_id.to_string();
async move {
client
.get_completed_job_result::<serde_json::Value>(&job_id_str, None)
.await
}
});
let results: Vec<_> = futures::future::join_all(futs).await;
let collected: Result<Vec<_>, _> = results.into_iter().collect();
collected
.map(serde_json::Value::Array)
.map_err(|e| format!("Failed to fetch results for step '{}': {}", step_id_clone, e))
}
}
}
None => {
// Not in local cache, fallback to querying by flow_job_id and step_id
// This happens for branch modules that need to access parent flow step results
// Use .ok() to match deno_core behavior: return null for non-existent steps
// instead of throwing an error
Ok(client
.get_result_by_id::<serde_json::Value>(&flow_job_id, &step_id_clone, None)
.await
.ok() // Swallow errors, convert to Option
.unwrap_or(serde_json::Value::Null)) // None -> null
}
};
match result {
Ok(value) => serde_json::to_string(&value).unwrap_or_else(|_| "null".to_string()),
Err(e) => format!("{}{}", ERR_PREFIX, e),
}
}
}))),
)?;
// Create JavaScript wrapper that parses the JSON result
let wrapper_code = r#"
const __RESULT_ERR_PREFIX = '\x00__WINDMILL_ERR__\x00';
async function __getResult(stepId) {
const result = await __fetchResult(stepId);
if (typeof result === 'string' && result.startsWith(__RESULT_ERR_PREFIX)) {
throw new Error(result.substring(__RESULT_ERR_PREFIX.length));
}
return JSON.parse(result);
}
"#;
ctx.eval::<(), _>(wrapper_code)
.catch(ctx)
.map_err(quickjs_error_to_anyhow)?;
} else {
// No client - stub function that rejects
let stub_code = r#"
function __getResult(stepId) {
return Promise.reject(new Error('Result fetching not available without authenticated client'));
}
"#;
ctx.eval::<(), _>(stub_code)
.catch(ctx)
.map_err(quickjs_error_to_anyhow)?;
}
// Create the results proxy that calls __getResult for on-demand fetching
// Matches deno_core behavior: always try to fetch, let backend handle unknown step IDs
let proxy_setup = r#"
const results = new Proxy({}, {
get: function(target, name, receiver) {
// Handle symbol properties (like Symbol.toStringTag)
if (typeof name === 'symbol') {
return undefined;
}
// Check if it's the previous_id and previous_result exists
if (name === __previous_id && typeof previous_result !== 'undefined') {
return Promise.resolve(previous_result);
}
// Always try to fetch - let Rust/backend handle unknown step IDs
// This matches deno_core behavior
return __getResult(name);
}
});
"#;
ctx.eval::<(), _>(proxy_setup)
.catch(ctx)
.map_err(quickjs_error_to_anyhow)?;
Ok(())
}
/// Convert a serde_json::Value to a QuickJS Value
fn json_to_js<'js>(
ctx: &rquickjs::Ctx<'js>,
val: &serde_json::Value,
) -> rquickjs::Result<Value<'js>> {
match val {
serde_json::Value::Null => Ok(Value::new_null(ctx.clone())),
serde_json::Value::Bool(b) => Ok(Value::new_bool(ctx.clone(), *b)),
serde_json::Value::Number(n) => {
if let Some(i) = n.as_i64() {
if i >= i32::MIN as i64 && i <= i32::MAX as i64 {
Ok(Value::new_int(ctx.clone(), i as i32))
} else {
Ok(Value::new_float(ctx.clone(), i as f64))
}
} else if let Some(f) = n.as_f64() {
Ok(Value::new_float(ctx.clone(), f))
} else {
Ok(Value::new_float(ctx.clone(), 0.0))
}
}
serde_json::Value::String(s) => s.clone().into_js(ctx),
serde_json::Value::Array(arr) => {
let js_arr = rquickjs::Array::new(ctx.clone())?;
for (i, item) in arr.iter().enumerate() {
js_arr.set(i, json_to_js(ctx, item)?)?;
}
Ok(js_arr.into_value())
}
serde_json::Value::Object(obj) => {
let js_obj = Object::new(ctx.clone())?;
for (k, v) in obj {
js_obj.set(k.as_str(), json_to_js(ctx, v)?)?;
}
Ok(js_obj.into_value())
}
}
}
/// Convert a QuickJS Value to a serde_json::Value
fn js_to_json<'js>(
ctx: &rquickjs::Ctx<'js>,
val: &Value<'js>,
) -> anyhow::Result<serde_json::Value> {
if val.is_null() || val.is_undefined() {
return Ok(serde_json::Value::Null);
}
if let Some(b) = val.as_bool() {
return Ok(serde_json::Value::Bool(b));
}
if let Some(i) = val.as_int() {
return Ok(serde_json::Value::Number(i.into()));
}
if let Some(f) = val.as_float() {
// Check if this float represents an exact integer
// This preserves integer formatting for values like timestamps
if f.fract() == 0.0 && f.abs() <= (i64::MAX as f64) {
let i = f as i64;
// Verify the conversion is exact (for very large numbers)
if (i as f64) == f {
return Ok(serde_json::Value::Number(i.into()));
}
}
if let Some(n) = serde_json::Number::from_f64(f) {
return Ok(serde_json::Value::Number(n));
} else {
return Ok(serde_json::Value::Null);
}
}
if let Ok(s) = String::from_js(ctx, val.clone()) {
return Ok(serde_json::Value::String(s));
}
if let Ok(arr) = rquickjs::Array::from_js(ctx, val.clone()) {
let mut json_arr = Vec::new();
for i in 0..arr.len() {
if let Ok(item) = arr.get::<Value>(i) {
json_arr.push(js_to_json(ctx, &item)?);
}
}
return Ok(serde_json::Value::Array(json_arr));
}
if let Ok(obj) = Object::from_js(ctx, val.clone()) {
let mut json_obj = serde_json::Map::new();
for res in obj.props::<String, Value>() {
if let Ok((k, v)) = res {
json_obj.insert(k, js_to_json(ctx, &v)?);
}
}
return Ok(serde_json::Value::Object(json_obj));
}
// Fallback
Ok(serde_json::Value::String("[object]".to_string()))
}
/// Determines if we should prepend "return" to the expression
fn should_add_return_quickjs(expr: &str) -> bool {
let trimmed = expr.trim();
if trimmed.is_empty() {
return true;
}
if trimmed.starts_with("return ") || trimmed.starts_with("return;") || trimmed == "return" {
return false;
}
let statement_prefixes = [
"const ", "let ", "var ", "if ", "if(", "for ", "for(", "while ", "while(", "switch ",
"switch(", "try ", "try{", "throw ", "function ", "class ", "async ", "await ",
];
for prefix in &statement_prefixes {
if trimmed.starts_with(prefix) {
return false;
}
}
if contains_semicolon_outside_strings(trimmed) {
return false;
}
true
}
fn contains_semicolon_outside_strings(expr: &str) -> bool {
let mut in_single_quote = false;
let mut in_double_quote = false;
let mut in_template = false;
let mut prev_char = '\0';
for ch in expr.chars() {
match ch {
'\'' if prev_char != '\\' && !in_double_quote && !in_template => {
in_single_quote = !in_single_quote;
}
'"' if prev_char != '\\' && !in_single_quote && !in_template => {
in_double_quote = !in_double_quote;
}
'`' if prev_char != '\\' && !in_single_quote && !in_double_quote => {
in_template = !in_template;
}
';' if !in_single_quote && !in_double_quote && !in_template => {
return true;
}
_ => {}
}
prev_char = ch;
}
false
}
fn quickjs_error_to_anyhow(err: rquickjs::CaughtError<'_>) -> anyhow::Error {
anyhow::anyhow!("QuickJS evaluation error: {}", err)
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
use windmill_common::worker::to_raw_value;
#[tokio::test]
async fn test_eval_quickjs_simple() -> anyhow::Result<()> {
let mut env = HashMap::new();
env.insert("x".to_string(), Arc::new(to_raw_value(&json!(5))));
env.insert("y".to_string(), Arc::new(to_raw_value(&json!(3))));
let result =
eval_timeout_quickjs("x + y".to_string(), env, None, None, None, None, None).await?;
assert_eq!(result.get(), "8");
Ok(())
}
#[tokio::test]
async fn test_eval_quickjs_object_access() -> anyhow::Result<()> {
let mut env = HashMap::new();
env.insert(
"params".to_string(),
Arc::new(to_raw_value(&json!({"test": 42, "nested": {"value": 100}}))),
);
let result = eval_timeout_quickjs(
"params.test".to_string(),
env.clone(),
None,
None,
None,
None,
None,
)
.await?;
assert_eq!(result.get(), "42");
let result2 = eval_timeout_quickjs(
"params.nested.value".to_string(),
env,
None,
None,
None,
None,
None,
)
.await?;
assert_eq!(result2.get(), "100");
Ok(())
}
#[tokio::test]
async fn test_eval_quickjs_array() -> anyhow::Result<()> {
let mut env = HashMap::new();
env.insert(
"arr".to_string(),
Arc::new(to_raw_value(&json!([1, 2, 3, 4, 5]))),
);
let result = eval_timeout_quickjs(
"arr.map(x => x * 2)".to_string(),
env,
None,
None,
None,
None,
None,
)
.await?;
assert_eq!(result.get(), "[2,4,6,8,10]");
Ok(())
}
#[tokio::test]
async fn test_eval_quickjs_flow_input() -> anyhow::Result<()> {
let mut flow_input = HashMap::new();
flow_input.insert("name".to_string(), to_raw_value(&json!("test")));
flow_input.insert("count".to_string(), to_raw_value(&json!(10)));
let result = eval_timeout_quickjs(
"flow_input.name".to_string(),
HashMap::new(),
Some(mappable_rc::Marc::new(flow_input)),
None,
None,
None,
None,
)
.await?;
assert_eq!(result.get(), "\"test\"");
Ok(())
}
#[test]
fn test_should_add_return_quickjs() {
assert!(should_add_return_quickjs("5"));
assert!(should_add_return_quickjs("x + y"));
assert!(should_add_return_quickjs("foo()"));
assert!(!should_add_return_quickjs("return 5"));
assert!(!should_add_return_quickjs("return x + y"));
assert!(!should_add_return_quickjs("const x = 5"));
assert!(!should_add_return_quickjs("let y = 10"));
assert!(!should_add_return_quickjs("if (x > 5) { return x; }"));
assert!(!should_add_return_quickjs("let x = 5; x + 1"));
}
}

View File

@@ -42,6 +42,10 @@ pub mod job_logger;
pub mod job_logger_ee;
mod job_logger_oss;
mod js_eval;
#[cfg(feature = "quickjs")]
pub mod js_eval_quickjs;
#[cfg(test)]
mod js_eval_parity_tests;
pub mod memory_common;
#[cfg(feature = "private")]
pub mod memory_ee;
@@ -86,7 +90,7 @@ pub use otel_tracing_proxy_ee::{
set_current_job_context, start_jobs_otel_tracing, TRACING_PROXY_PORT,
};
#[cfg(all(feature = "private", feature = "enterprise", feature = "deno_core"))]
pub use otel_tracing_proxy_ee::{load_internal_otel_exporter, DENO_OTEL_INITIALIZED};
pub use otel_tracing_proxy_ee::{load_internal_otel_exporter, DENO_OTEL_INITIALIZED, OTLP_COLLECTOR_PORT};
pub use result_processor::handle_job_error;

View File

@@ -52,9 +52,12 @@ Object.assign(globalThis, {
// Expose bootstrapOtel globally so it can be called from Rust after runtime creation.
// We use dynamic import so deno_telemetry isn't loaded during snapshot creation.
// Config: [tracingEnabled, metricsEnabled, consoleConfig, deterministic]
// consoleConfig: 0=ignore, 1=capture, 2=replace
globalThis.__bootstrapOtel = () => {
import("ext:deno_telemetry/telemetry.ts").then(({ bootstrap }) => {
bootstrap([1, 0, 0, 0]);
import("ext:deno_telemetry/telemetry.ts").then(({ bootstrap, enterSpan }) => {
bootstrap([1, 0, 1, 0]);
// Expose enterSpan for setting parent trace context
globalThis.__enterSpan = enterSpan;
});
};

View File

@@ -675,7 +675,6 @@ async fn get_otel_tracing_proxy_envs() -> anyhow::Result<Vec<(&'static str, Stri
])
}
#[cfg(windows)]
lazy_static::lazy_static! {
pub static ref SYSTEM_ROOT: String = std::env::var("SystemRoot").unwrap_or_else(|_| "C:\\Windows".to_string());
@@ -1530,7 +1529,10 @@ pub async fn run_worker(
let mut occupancy_metrics = OccupancyMetrics::new(start_time);
let mut jobs_executed = 0;
let is_dedicated_worker: bool = WORKER_CONFIG.read().await.dedicated_worker.is_some();
let is_dedicated_worker: bool = {
let config = WORKER_CONFIG.read().await;
config.dedicated_worker.is_some() || config.dedicated_workers.as_ref().is_some_and(|dws| !dws.is_empty())
};
#[cfg(feature = "benchmark")]
let benchmark_jobs: i32 = std::env::var("BENCHMARK_JOBS")
@@ -1632,9 +1634,9 @@ pub async fn run_worker(
// Option<JoinHandle<()>>,
#[cfg(all(feature = "private", feature = "enterprise"))]
let (dedicated_workers, is_flow_worker, dedicated_handles): (
let (dedicated_workers, dedicated_flow_paths, dedicated_handles): (
HashMap<String, Sender<DedicatedWorkerJob>>,
bool,
HashSet<String>,
Vec<JoinHandle<()>>,
) = match conn {
Connection::Sql(pool) => {
@@ -1649,15 +1651,15 @@ pub async fn run_worker(
)
.await
}
Connection::Http(_) => (HashMap::new(), false, vec![]),
Connection::Http(_) => (HashMap::new(), HashSet::new(), vec![]),
};
#[cfg(any(not(feature = "private"), not(feature = "enterprise")))]
let (dedicated_workers, is_flow_worker, dedicated_handles): (
let (dedicated_workers, dedicated_flow_paths, dedicated_handles): (
HashMap<String, Sender<DedicatedWorkerJob>>,
bool,
HashSet<String>,
Vec<JoinHandle<()>>,
) = (HashMap::new(), false, vec![]);
) = (HashMap::new(), HashSet::new(), vec![]);
if i_worker == 1 {
if let Err(e) = queue_init_bash_maybe(conn, same_worker_tx.clone(), &worker_name).await {
@@ -2031,30 +2033,32 @@ pub async fn run_worker(
JobKind::Script | JobKind::Preview | JobKind::FlowScript
) {
if !dedicated_workers.is_empty() {
let key_o = if is_flow_worker {
job.flow_step_id.as_ref().map(|x| x.to_string())
// Try flow path + step_id combinations for flow jobs, otherwise use runnable_path
let dedicated_worker_tx = if let Some(step_id) = job.flow_step_id.as_ref() {
dedicated_flow_paths.iter().find_map(|flow_path| {
let key = format!("{}:{}", flow_path, step_id);
dedicated_workers.get(&key)
})
} else {
job.runnable_path.as_ref().map(|x| x.to_string())
job.runnable_path.as_ref().and_then(|path| dedicated_workers.get(path))
};
if let Some(key) = key_o {
if let Some(dedicated_worker_tx) = dedicated_workers.get(&key) {
let dedicated_job = DedicatedWorkerJob {
job: Arc::new(job.job()),
flow_runners: None,
done_tx: None,
};
if let Err(e) = dedicated_worker_tx.send(dedicated_job).await {
tracing::info!("failed to send jobs to dedicated workers. Likely dedicated worker has been shut down. This is normal: {e:?}");
}
#[cfg(feature = "benchmark")]
{
add_time!(bench, "sent to dedicated worker");
infos.add_iter(bench, true);
}
continue;
if let Some(dedicated_worker_tx) = dedicated_worker_tx {
let dedicated_job = DedicatedWorkerJob {
job: Arc::new(job.job()),
flow_runners: None,
done_tx: None,
};
if let Err(e) = dedicated_worker_tx.send(dedicated_job).await {
tracing::info!("failed to send jobs to dedicated workers. Likely dedicated worker has been shut down. This is normal: {e:?}");
}
#[cfg(feature = "benchmark")]
{
add_time!(bench, "sent to dedicated worker");
infos.add_iter(bench, true);
}
continue;
}
}
@@ -2704,19 +2708,6 @@ async fn do_nativets(
let stream_notifier = StreamNotifier::new(conn, job);
// Set job context for OTEL tracing (EE only)
#[cfg(all(feature = "private", feature = "enterprise"))]
{
let tracing_enabled = is_otel_tracing_proxy_enabled_for_lang(&ScriptLang::Nativets).await;
tracing::debug!(
"nativets job {}: OTEL tracing enabled={}",
job.id, tracing_enabled
);
if tracing_enabled {
crate::otel_tracing_proxy_ee::set_current_job_context(job.id).await;
}
}
Ok(eval_fetch_timeout(
env_code,
code.clone(),
@@ -3133,6 +3124,14 @@ pub async fn handle_queued_job(
_ => None,
});
// Set job context for OTEL tracing before entering handle_code_execution_job's span
#[cfg(all(feature = "private", feature = "enterprise"))]
if matches!(job.script_lang, Some(ScriptLang::Nativets) | Some(ScriptLang::Bunnative))
&& is_otel_tracing_proxy_enabled_for_lang(&ScriptLang::Nativets).await
{
crate::otel_tracing_proxy_ee::set_current_job_context(job.id).await;
}
// Box::pin to move large future to heap
let r = Box::pin(handle_code_execution_job(
job.as_ref(),

View File

@@ -133,6 +133,7 @@ async fn update_worker_ping_full_inner(
ip: None,
tags: Some(tags.to_vec()),
dw: None,
dws: None,
jobs_executed: Some(jobs_executed),
occupancy_rate: Some(occupancy_rate),
occupancy_rate_15s: Some(occupancy_rate_15s.unwrap_or(0.0)),
@@ -159,13 +160,19 @@ pub async fn insert_ping(
ip: &str,
db: &Connection,
) -> anyhow::Result<()> {
let (tags, dw) = {
let (tags, dw, dws) = {
let wc = WORKER_CONFIG.read().await.clone();
(
wc.worker_tags,
wc.dedicated_worker
.as_ref()
.map(|x| format!("{}:{}", x.workspace_id, x.path)),
wc.dedicated_workers.as_ref().map(|workers| {
workers
.iter()
.map(|x| format!("{}:{}", x.workspace_id, x.path))
.collect::<Vec<_>>()
}),
)
};
@@ -190,6 +197,7 @@ pub async fn insert_ping(
ip,
tags.as_slice(),
dw,
dws.as_deref(),
windmill_common::utils::GIT_VERSION,
vcpus,
memory,
@@ -210,6 +218,7 @@ pub async fn insert_ping(
ip: Some(ip.to_string()),
tags: Some(tags.to_vec()),
dw: dw,
dws: dws,
jobs_executed: None,
occupancy_rate: None,
occupancy_rate_15s: None,
@@ -282,6 +291,7 @@ pub async fn update_worker_ping_from_job(
ip: None,
tags: None,
dw: None,
dws: None,
version: None,
vcpus: None,
memory: None,

View File

@@ -2,7 +2,7 @@ import { sleep } from "https://deno.land/x/sleep@v1.2.1/mod.ts";
import * as windmill from "https://deno.land/x/windmill@v1.174.0/mod.ts";
import * as api from "https://deno.land/x/windmill@v1.174.0/windmill-api/index.ts";
export const VERSION = "v1.617.1";
export const VERSION = "v1.621.2";
export async function login(email: string, password: string): Promise<string> {
return await windmill.UserService.login({

40
cli/deno.lock generated
View File

@@ -2,8 +2,10 @@
"version": "5",
"specifiers": {
"jsr:@david/code-block-writer@^13.0.2": "13.0.2",
"jsr:@david/code-block-writer@^13.0.3": "13.0.3",
"jsr:@deno/cache-dir@~0.10.3": "0.10.3",
"jsr:@deno/dnt@0.41.3": "0.41.3",
"jsr:@deno/dnt@0.42.3": "0.42.3",
"jsr:@deno/dnt@~0.41.3": "0.41.3",
"jsr:@deno/graph@~0.73.1": "0.73.1",
"jsr:@std/assert@0.223": "0.223.0",
@@ -20,7 +22,7 @@
"jsr:@std/fmt@~0.225.4": "0.225.6",
"jsr:@std/fs@*": "1.0.22",
"jsr:@std/fs@0.223": "0.223.0",
"jsr:@std/fs@1": "1.0.20",
"jsr:@std/fs@1": "1.0.22",
"jsr:@std/fs@^1.0.11": "1.0.22",
"jsr:@std/fs@^1.0.21": "1.0.22",
"jsr:@std/fs@~0.229.3": "0.229.3",
@@ -32,7 +34,7 @@
"jsr:@std/log@*": "0.224.14",
"jsr:@std/path@*": "1.1.4",
"jsr:@std/path@0.223": "0.223.0",
"jsr:@std/path@1": "1.1.3",
"jsr:@std/path@1": "1.1.4",
"jsr:@std/path@1.0.0-rc.1": "1.0.0-rc.1",
"jsr:@std/path@1.0.0-rc.2": "1.0.0-rc.2",
"jsr:@std/path@^1.1.3": "1.1.4",
@@ -42,7 +44,9 @@
"jsr:@std/yaml@*": "1.0.10",
"jsr:@std/yaml@^1.0.10": "1.0.10",
"jsr:@ts-morph/bootstrap@0.24": "0.24.0",
"jsr:@ts-morph/bootstrap@0.27": "0.27.0",
"jsr:@ts-morph/common@0.24": "0.24.0",
"jsr:@ts-morph/common@0.27": "0.27.0",
"jsr:@windmill-labs/cliffy-ansi@1.0.0-rc.5": "1.0.0-rc.5",
"jsr:@windmill-labs/cliffy-command@1.0.0-rc.5": "1.0.0-rc.5",
"jsr:@windmill-labs/cliffy-flags@1.0.0-rc.5": "1.0.0-rc.5",
@@ -90,6 +94,9 @@
"@david/code-block-writer@13.0.2": {
"integrity": "14dd3baaafa3a2dea8bf7dfbcddeccaa13e583da2d21d666c01dc6d681cd74ad"
},
"@david/code-block-writer@13.0.3": {
"integrity": "f98c77d320f5957899a61bfb7a9bead7c6d83ad1515daee92dbacc861e13bb7f"
},
"@deno/cache-dir@0.10.3": {
"integrity": "eb022f84ecc49c91d9d98131c6e6b118ff63a29e343624d058646b9d50404776",
"dependencies": [
@@ -103,12 +110,22 @@
"@deno/dnt@0.41.3": {
"integrity": "b2ef2c8a5111eef86cb5bfcae103d6a2938e8e649e2461634a7befb7fc59d6d2",
"dependencies": [
"jsr:@david/code-block-writer",
"jsr:@david/code-block-writer@^13.0.2",
"jsr:@deno/cache-dir",
"jsr:@std/fmt@1",
"jsr:@std/fs@1",
"jsr:@std/path@1",
"jsr:@ts-morph/bootstrap"
"jsr:@ts-morph/bootstrap@0.24"
]
},
"@deno/dnt@0.42.3": {
"integrity": "62a917a0492f3c8af002dce90605bb0d41f7d29debc06aca40dba72ab65d8ae3",
"dependencies": [
"jsr:@david/code-block-writer@^13.0.3",
"jsr:@std/fmt@1",
"jsr:@std/fs@1",
"jsr:@std/path@1",
"jsr:@ts-morph/bootstrap@0.27"
]
},
"@deno/graph@0.73.1": {
@@ -236,7 +253,13 @@
"@ts-morph/bootstrap@0.24.0": {
"integrity": "a826a2ef7fa8a7c3f1042df2c034d20744d94da2ee32bf29275bcd4dffd3c060",
"dependencies": [
"jsr:@ts-morph/common"
"jsr:@ts-morph/common@0.24"
]
},
"@ts-morph/bootstrap@0.27.0": {
"integrity": "b8d7bc8f7942ce853dde4161b28f9aa96769cef3d8eebafb379a81800b9e2448",
"dependencies": [
"jsr:@ts-morph/common@0.27"
]
},
"@ts-morph/common@0.24.0": {
@@ -246,6 +269,13 @@
"jsr:@std/path@~0.225.2"
]
},
"@ts-morph/common@0.27.0": {
"integrity": "c7b73592d78ce8479b356fd4f3d6ec3c460d77753a8680ff196effea7a939052",
"dependencies": [
"jsr:@std/fs@1",
"jsr:@std/path@1"
]
},
"@windmill-labs/cliffy-ansi@1.0.0-rc.5": {
"integrity": "1109cbcb0c415b57779352f708f5969b8c645f56bc555cbafc6ea5e0c6a360a4",
"dependencies": [

View File

@@ -1,5 +1,5 @@
// ex. scripts/build_npm.ts
import { build, emptyDir } from "jsr:@deno/dnt@0.41.3";
import { build, emptyDir } from "jsr:@deno/dnt@0.42.3";
import { VERSION } from "./src/main.ts";
await emptyDir("./npm");

View File

@@ -39,54 +39,12 @@ import { regenerateAgentDocs } from "./generate_agents.ts";
import {
getFolderSuffix,
hasFolderSuffix,
setNonDottedPaths,
loadNonDottedPathsSetting,
} from "../../utils/resource_folders.ts";
const DEFAULT_PORT = 4000;
const DEFAULT_HOST = "localhost";
/**
* Search for wmill.yaml by traversing upward from the current directory.
* Unlike the standard findWmillYaml() in conf.ts, this does not stop at
* the git root - it continues searching until the filesystem root.
* This is needed for `app dev` which runs from inside a raw_app folder
* that may be deeply nested within a larger git repository.
*/
async function findAndLoadNonDottedPathsSetting(): Promise<void> {
let currentDir = process.cwd();
while (true) {
const wmillYamlPath = path.join(currentDir, "wmill.yaml");
if (fs.existsSync(wmillYamlPath)) {
try {
const config = await yamlParseFile(wmillYamlPath) as {
nonDottedPaths?: boolean;
};
setNonDottedPaths(config?.nonDottedPaths ?? false);
log.debug(
`Found wmill.yaml at ${wmillYamlPath}, nonDottedPaths=${
config?.nonDottedPaths ?? false
}`,
);
} catch (e) {
log.debug(`Failed to parse wmill.yaml at ${wmillYamlPath}: ${e}`);
}
return;
}
// Check if we've reached the filesystem root
const parentDir = path.dirname(currentDir);
if (parentDir === currentDir) {
// Reached filesystem root without finding wmill.yaml
log.debug("No wmill.yaml found, using default dotted paths");
return;
}
currentDir = parentDir;
}
}
// HTML template with live reload and SQL migration modal
const createHTML = (jsPath: string, cssPath: string) => `
<!DOCTYPE html>
@@ -348,39 +306,53 @@ interface DevOptions extends GlobalOptions {
open?: boolean;
}
async function dev(opts: DevOptions) {
async function dev(opts: DevOptions, appFolder?: string) {
GLOBAL_CONFIG_OPT.noCdToRoot = true;
// Search for wmill.yaml by traversing upward (without git root constraint)
// to initialize nonDottedPaths setting before using folder suffix functions
await findAndLoadNonDottedPathsSetting();
await loadNonDottedPathsSetting();
// Validate that we're in a .raw_app folder
const cwd = process.cwd();
const currentDirName = path.basename(cwd);
// Resolve target directory from argument or use current directory
const originalCwd = process.cwd();
let targetDir = originalCwd;
if (!hasFolderSuffix(currentDirName, "raw_app")) {
if (appFolder) {
targetDir = path.isAbsolute(appFolder)
? appFolder
: path.join(originalCwd, appFolder);
if (!fs.existsSync(targetDir)) {
log.error(colors.red(`Error: Directory not found: ${targetDir}`));
Deno.exit(1);
}
}
// Validate that target is a .raw_app folder
const targetDirName = path.basename(targetDir);
if (!hasFolderSuffix(targetDirName, "raw_app")) {
log.error(
colors.red(
`Error: The dev command must be run inside a ${
getFolderSuffix("raw_app")
} folder.\n` +
`Current directory: ${currentDirName}\n` +
`Target directory: ${targetDirName}\n` +
`Please navigate to a folder ending with '${
getFolderSuffix("raw_app")
}' before running this command.`,
}' or specify one as argument.`,
),
);
Deno.exit(1);
}
// Check for raw_app.yaml
const rawAppPath = path.join(cwd, "raw_app.yaml");
// Check for raw_app.yaml in target directory
const rawAppPath = path.join(targetDir, "raw_app.yaml");
if (!fs.existsSync(rawAppPath)) {
log.error(
colors.red(
`Error: raw_app.yaml not found in current directory.\n` +
`The dev command must be run in a ${
`Error: raw_app.yaml not found in ${targetDir}.\n` +
`The dev command requires a ${
getFolderSuffix("raw_app")
} folder containing a raw_app.yaml file.`,
),
@@ -388,11 +360,16 @@ async function dev(opts: DevOptions) {
Deno.exit(1);
}
// Resolve workspace and authenticate
// Resolve workspace and authenticate (from original cwd to find wmill.yaml)
const workspace = await resolveWorkspace(opts);
await requireLogin(opts);
const workspaceId = workspace.workspaceId;
// Change to target directory for the rest of the command
if (appFolder) {
process.chdir(targetDir);
}
// Load app path from raw_app.yaml
const rawApp = (await yamlParseFile(rawAppPath)) as any;
const appPath = rawApp?.custom_path ?? "u/unknown/newapp";
@@ -1331,6 +1308,7 @@ const command = new Command()
.description(
"Start a development server for building apps with live reload and hot module replacement",
)
.arguments("[app_folder:string]")
.option(
"--port <port:number>",
"Port to run the dev server on (will find next available port if occupied)",

View File

@@ -7,6 +7,11 @@ import { DataTableSchema } from "../../../gen/types.gen.ts";
import { generateAgentsDocumentation } from "../sync/sync.ts";
import path from "node:path";
import * as fs from "node:fs";
import {
getFolderSuffix,
hasFolderSuffix,
loadNonDottedPathsSetting,
} from "../../utils/resource_folders.ts";
interface GenerateAgentsOptions extends GlobalOptions {
output?: string;
@@ -230,14 +235,17 @@ async function generateAgents(
: path.join(cwd, appFolder);
}
// Ensure we're in a .raw_app folder or targeting one
// Load nonDottedPaths setting before using folder suffix functions
await loadNonDottedPathsSetting();
// Ensure we're in a raw_app folder or targeting one
const dirName = path.basename(targetDir);
if (!dirName.endsWith(".raw_app")) {
// Check if current directory is a .raw_app folder
if (!path.basename(cwd).endsWith(".raw_app") && !appFolder) {
if (!hasFolderSuffix(dirName, "raw_app")) {
// Check if current directory is a raw_app folder
if (!hasFolderSuffix(path.basename(cwd), "raw_app") && !appFolder) {
log.error(
colors.red(
"Error: Must be run inside a .raw_app folder or specify one as argument."
`Error: Must be run inside a ${getFolderSuffix("raw_app")} folder or specify one as argument.`
)
);
log.info(colors.gray("Usage: wmill app generate-agents [app_folder]"));

View File

@@ -10,6 +10,7 @@ import { loadRunnablesFromBackend } from "./raw_apps.ts";
import {
getFolderSuffix,
hasFolderSuffix,
loadNonDottedPathsSetting,
} from "../../utils/resource_folders.ts";
interface LintOptions extends GlobalOptions {
@@ -200,6 +201,9 @@ async function lintRawApp(
* Main lint command
*/
async function lint(opts: LintOptions, appFolder?: string) {
// Load nonDottedPaths setting before using folder suffix functions
await loadNonDottedPathsSetting();
const targetDir = appFolder ?? process.cwd();
log.info(colors.bold.blue(`\n🔍 Linting raw app: ${targetDir}\n`));

View File

@@ -14,7 +14,10 @@ import { resolveWorkspace } from "../../core/context.ts";
import { requireLogin } from "../../core/auth.ts";
import * as wmill from "../../../gen/services.gen.ts";
import path from "node:path";
import { buildFolderPath } from "../../utils/resource_folders.ts";
import {
buildFolderPath,
loadNonDottedPathsSetting,
} from "../../utils/resource_folders.ts";
// Framework templates - adapted from frontend/src/routes/(root)/(logged)/apps_raw/add/templates.ts
const reactIndex = `
@@ -472,6 +475,9 @@ CREATE SCHEMA IF NOT EXISTS ${schemaName};
);
}
// Load nonDottedPaths setting from wmill.yaml before creating folder
await loadNonDottedPathsSetting();
// Create the directory structure - preserve full path (e.g., f/foobar/x/y becomes f/foobar/x/y.raw_app)
const folderName = buildFolderPath(appPath, "raw_app");
const appDir = path.join(Deno.cwd(), folderName);

View File

@@ -193,6 +193,78 @@ async function run(
log.info(JSON.stringify(jobInfo.result ?? {}, null, 2));
}
async function preview(
opts: GlobalOptions & {
data?: string;
silent: boolean;
},
flowPath: string
) {
const workspace = await resolveWorkspace(opts);
await requireLogin(opts);
// Normalize path - ensure it's a directory path to a .flow folder
if (!flowPath.endsWith(".flow") && !flowPath.endsWith(".flow" + SEP)) {
// Check if it's a flow.yaml file
if (flowPath.endsWith("flow.yaml") || flowPath.endsWith("flow.json")) {
flowPath = flowPath.substring(0, flowPath.lastIndexOf(SEP));
} else {
throw new Error(
"Flow path must be a .flow directory or a flow.yaml file"
);
}
}
if (!flowPath.endsWith(SEP)) {
flowPath += SEP;
}
// Read and parse the flow definition
const localFlow = (await yamlParseFile(flowPath + "flow.yaml")) as FlowFile;
// Replace inline scripts with their actual content
await replaceInlineScripts(
localFlow.value.modules,
async (path: string) => await Deno.readTextFile(flowPath + path),
log,
flowPath,
SEP
);
const input = opts.data ? await resolve(opts.data) : {};
if (!opts.silent) {
log.info(colors.yellow(`Running flow preview for ${flowPath}...`));
}
log.debug(`Flow value: ${JSON.stringify(localFlow.value, null, 2)}`);
// Run the flow preview
let result;
try {
result = await wmill.runFlowPreviewAndWaitResult({
workspace: workspace.workspaceId,
requestBody: {
value: localFlow.value,
path: flowPath.substring(0, flowPath.indexOf(".flow")).replaceAll(SEP, "/"),
args: input,
},
});
} catch (e: any) {
if (e.body) {
log.error(`Flow preview failed: ${JSON.stringify(e.body)}`);
}
throw e;
}
if (opts.silent) {
console.log(JSON.stringify(result, null, 2));
} else {
log.info(colors.bold.underline.green("Flow preview completed"));
log.info(JSON.stringify(result, null, 2));
}
}
async function generateLocks(
opts: GlobalOptions & {
yes?: boolean;
@@ -315,6 +387,20 @@ const command = new Command()
"Do not ouput anything other then the final output. Useful for scripting."
)
.action(run as any)
.command(
"preview",
"preview a local flow without deploying it. Runs the flow definition from local files."
)
.arguments("<flow_path:string>")
.option(
"-d --data <data:string>",
"Inputs specified as a JSON string or a file using @<filename> or stdin using @-."
)
.option(
"-s --silent",
"Do not output anything other then the final output. Useful for scripting."
)
.action(preview as any)
.command(
"generate-locks",
"re-generate the lock files of all inline scripts of all updated flows"

View File

@@ -1090,6 +1090,212 @@ async function generateMetadata(
}
}
async function preview(
opts: GlobalOptions & {
data?: string;
silent: boolean;
} & SyncOptions,
filePath: string
) {
opts = await mergeConfigWithConfigFile(opts);
const workspace = await resolveWorkspace(opts);
await requireLogin(opts);
if (!validatePath(filePath)) {
return;
}
const fstat = await Deno.stat(filePath);
if (!fstat.isFile) {
throw new Error("file path must refer to a file.");
}
if (filePath.endsWith(".script.json") || filePath.endsWith(".script.yaml")) {
throw Error(
"Cannot preview a script metadata file, point to the script content file instead (.py, .ts, .go, .sh)"
);
}
const codebases = await listSyncCodebases(opts);
const language = inferContentTypeFromFilePath(filePath, opts?.defaultTs);
const content = await Deno.readTextFile(filePath);
const input = opts.data ? await resolve(opts.data) : {};
// Check if this is a codebase script
const codebase =
language == "bun" ? findCodebase(filePath, codebases) : undefined;
let bundledContent: string | Blob | undefined = undefined;
let isTar = false;
if (codebase) {
if (codebase.customBundler) {
if (!opts.silent) {
log.info(`Using custom bundler ${codebase.customBundler} for preview`);
}
bundledContent = execSync(codebase.customBundler + " " + filePath, {
maxBuffer: 1024 * 1024 * 50,
}).toString();
} else {
const esbuild = await import("npm:esbuild@0.24.2");
if (!opts.silent) {
log.info(`Bundling ${filePath} for preview...`);
}
const startTime = performance.now();
const format = codebase.format ?? "cjs";
const out = await esbuild.build({
entryPoints: [filePath],
format: format,
bundle: true,
write: false,
external: codebase.external,
inject: codebase.inject,
define: codebase.define,
loader: codebase.loader ?? { ".node": "file" },
outdir: "/",
platform: "node",
packages: "bundle",
target: format == "cjs" ? "node20.15.1" : "esnext",
banner: codebase.banner,
});
const endTime = performance.now();
bundledContent = out.outputFiles[0].text;
// Handle multiple output files (create tarball)
if (out.outputFiles.length > 1) {
const archiveNpm = await import("npm:@ayonli/jsext/archive");
if (!opts.silent) {
log.info(`Creating tarball for multiple output files...`);
}
const tarball = new archiveNpm.Tarball();
const mainPath = filePath.split(SEP).pop()?.split(".")[0] + ".js";
const mainContent =
out.outputFiles.find((file: OutputFile) => file.path == "/" + mainPath)?.text ?? "";
tarball.append(new File([mainContent], "main.js", { type: "text/plain" }));
for (const file of out.outputFiles) {
if (file.path == "/" + mainPath) continue;
// deno-lint-ignore no-explicit-any
const fil = new File([file.contents as any], file.path.substring(1));
tarball.append(fil);
}
bundledContent = await streamToBlob(tarball.stream());
isTar = true;
} else if (Array.isArray(codebase.assets) && codebase.assets.length > 0) {
// Handle assets
const archiveNpm = await import("npm:@ayonli/jsext/archive");
if (!opts.silent) {
log.info(`Adding assets to tarball...`);
}
const tarball = new archiveNpm.Tarball();
tarball.append(new File([bundledContent], "main.js", { type: "text/plain" }));
for (const asset of codebase.assets) {
const data = fs.readFileSync(asset.from);
const blob = new Blob([data], { type: "text/plain" });
const file = new File([blob], asset.to);
tarball.append(file);
}
bundledContent = await streamToBlob(tarball.stream());
isTar = true;
}
if (!opts.silent) {
const size = typeof bundledContent === "string" ? bundledContent.length : bundledContent.size;
log.info(
`Bundled ${filePath}: ${(size / 1024).toFixed(0)}kB (${(
endTime - startTime
).toFixed(0)}ms)`
);
}
}
}
if (!opts.silent) {
log.info(colors.yellow(`Running preview for ${filePath}...`));
}
// For codebase scripts with bundles, we need to use a multipart form upload
if (bundledContent) {
const form = new FormData();
const previewPayload = {
content: content, // Pass the original content (frontend does this too)
path: filePath.substring(0, filePath.indexOf(".")).replaceAll(SEP, "/"),
args: input,
language: language,
kind: isTar ? "tarbundle" : "bundle",
format: codebase?.format ?? "cjs",
};
form.append("preview", JSON.stringify(previewPayload));
form.append(
"file",
typeof bundledContent === "string"
? new Blob([bundledContent], { type: "application/javascript" })
: bundledContent
);
const url =
workspace.remote +
"api/w/" +
workspace.workspaceId +
"/jobs/run/preview_bundle";
const response = await fetch(url, {
method: "POST",
headers: { Authorization: `Bearer ${workspace.token}` },
body: form,
});
if (!response.ok) {
throw new Error(
`Preview failed: ${response.status} - ${response.statusText} - ${await response.text()}`
);
}
const jobId = await response.text();
if (!opts.silent) {
await track_job(workspace.workspaceId, jobId);
}
// Wait for the job to complete and get the result
while (true) {
try {
const completedJob = await wmill.getCompletedJob({
workspace: workspace.workspaceId,
id: jobId,
});
const result = completedJob.result ?? {};
if (opts.silent) {
console.log(JSON.stringify(result, null, 2));
} else {
log.info(JSON.stringify(result, null, 2));
}
break;
} catch {
await new Promise((resolve) => setTimeout(resolve, 100));
}
}
} else {
// For regular scripts, use the standard preview API
const result = await wmill.runScriptPreviewAndWaitResult({
workspace: workspace.workspaceId,
requestBody: {
content,
path: filePath.substring(0, filePath.indexOf(".")).replaceAll(SEP, "/"),
args: input,
language: language as any,
},
});
if (opts.silent) {
console.log(JSON.stringify(result, null, 2));
} else {
log.info(colors.bold.underline.green("Preview completed"));
log.info(JSON.stringify(result, null, 2));
}
}
}
const command = new Command()
.description("script related commands")
.option("--show-archived", "Enable archived scripts in output")
@@ -1114,6 +1320,20 @@ const command = new Command()
"Do not output anything other then the final output. Useful for scripting."
)
.action(run as any)
.command(
"preview",
"preview a local script without deploying it. Supports both regular and codebase scripts."
)
.arguments("<path:file>")
.option(
"-d --data <data:file>",
"Inputs specified as a JSON string or a file using @<filename> or stdin using @-."
)
.option(
"-s --silent",
"Do not output anything other than the final output. Useful for scripting."
)
.action(preview as any)
.command("bootstrap", "create a new script")
.arguments("<path:file> <language:string>")
.option("--summary <summary:string>", "script summary")

View File

@@ -1283,6 +1283,8 @@ export async function elementsToMap(
): Promise<{ [key: string]: string }> {
const map: { [key: string]: string } = {};
const processedBasePaths = new Set<string>();
// Cache git branch at the start to avoid repeated execSync calls per file
const cachedBranch = branchOverride ?? getCurrentGitBranch() ?? undefined;
for await (const entry of readDirRecursiveWithIgnore(ignore, els)) {
// console.log("FOO", entry.path, entry.ignored, entry.isDirectory)
if (entry.isDirectory || entry.ignored) {
@@ -1383,7 +1385,7 @@ export async function elementsToMap(
// Handle branch-specific files - skip files for other branches
if (specificItems && isBranchSpecificFile(path)) {
if (!isCurrentBranchFile(path, branchOverride)) {
if (!isCurrentBranchFile(path, cachedBranch)) {
// Skip branch-specific files for other branches
continue;
}
@@ -1418,9 +1420,9 @@ export async function elementsToMap(
}
// Handle branch-specific path mapping after all filtering
if (isCurrentBranchFile(path, branchOverride)) {
if (cachedBranch && isCurrentBranchFile(path, cachedBranch)) {
// This is a branch-specific file for current branch
const currentBranch = branchOverride || getCurrentGitBranch()!;
const currentBranch = cachedBranch;
const basePath = fromBranchSpecificPath(path, currentBranch);
// Only use branch-specific files if the item type IS configured as branch-specific
@@ -1952,6 +1954,7 @@ export async function pull(
} catch {
// ignore
}
const zipFile = await downloadZip(
workspace,
opts.plainSecrets,
@@ -1976,9 +1979,11 @@ export async function pull(
resourceTypeToFormatExtension,
true,
);
const local = !opts.stateful
? await FSFSElement(Deno.cwd(), codebases, true)
: await FSFSElement(path.join(Deno.cwd(), ".wmill"), [], true);
const changes = await compareDynFSElement(
remote,
local,
@@ -2675,6 +2680,8 @@ export async function push(
// Create a pool of workers that processes items as they become available
const pool = new Set();
const queue = [...groupedChangesArray];
// Cache git branch at the start to avoid repeated execSync calls per change
const cachedBranchForPush = opts.branch || (isGitRepository() ? getCurrentGitBranch() : null);
while (queue.length > 0 || pool.size > 0) {
// Fill the pool until we reach parallelizationFactor
@@ -2759,7 +2766,7 @@ export async function push(
// For branch-specific resources, push to the base path on the workspace server
// This ensures branch-specific files are stored with their base names in the workspace
let serverPath = resourceFilePath;
const currentBranch = opts.branch || (isGitRepository() ? getCurrentGitBranch() : null);
const currentBranch = cachedBranchForPush;
if (currentBranch && isBranchSpecificFile(resourceFilePath)) {
serverPath = fromBranchSpecificPath(

View File

@@ -5,6 +5,13 @@ import {
esMain,
log,
} from "../deps.ts";
// Node.js-specific imports for symlink resolution in isMain()
// These are only used in Node.js, not Deno
// dnt-shim-ignore
import { realpathSync } from "node:fs";
// dnt-shim-ignore
import { fileURLToPath } from "node:url";
import flow from "./commands/flow/flow.ts";
import app from "./commands/app/app.ts";
import script from "./commands/script/script.ts";
@@ -70,7 +77,7 @@ export {
// }
// });
export const VERSION = "1.617.1";
export const VERSION = "1.621.2";
// Re-exported from constants.ts to maintain backwards compatibility
export { WM_FORK_PREFIX } from "./core/constants.ts";
@@ -239,8 +246,23 @@ function isMain() {
}
return isMain;
} else {
//@ts-ignore
return esMain.default(import.meta);
// For Node.js, we need to handle symlinks properly.
// The dnt polyfill doesn't resolve symlinks when comparing process.argv[1]
// with import.meta.url, so `wmill` symlink doesn't match the real file path.
// We resolve symlinks manually to get accurate comparison.
try {
const scriptPath = process.argv[1];
if (!scriptPath) return false;
const realScriptPath = realpathSync(scriptPath);
const modulePath = fileURLToPath(import.meta.url);
return realScriptPath === modulePath;
} catch {
// Fallback to esMain if something fails
//@ts-ignore
return esMain.default(import.meta);
}
}
}
if (isMain()) {

Some files were not shown because too many files have changed in this diff Show More