Files
windmill/backend/parsers/windmill-parser-wasm/tests/wasm.rs
Ruben Fiszel 31d6660d56 feat: script module mode with CLI sync, preview, and WAC UI improvements (#8380)
* feat: add script module mode with folder model for Bun and Python

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add missing modules field to RawCode in bun_executor

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* sqlx

* feat: enrich WAC templates with checkpoint and replay semantics

Add prominent comments explaining that all computation must happen
inside task/step/taskScript or it will be replayed on resume/retry.
Clarify that waitForApproval does not hold a worker and that
approve/reject URLs are available in the timeline step details.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(cli): script module sync idempotency, per-module hash tracking, and preview support

- Fix pull→push idempotency: use `??` instead of `||` for module lock
  field so empty strings are preserved (matches API's `lock: ""`)
- Add per-module hash tracking in wmill-lock.yaml following the flow
  inline script pattern (SCRIPT_TOP_HASH + per-module subpath hashes)
- Selective module lock regeneration: only regenerate locks for modules
  whose content actually changed, not all modules
- Use unfiltered rawWorkspaceDependencies for module hashes to match
  what updateModuleLocks passes to fetchScriptLock
- Show changed module names in stale script output for clarity
- Add module support to `script preview` command: read modules from
  __mod/ folder and pass them in the preview API request
- Add preview tests for taskScript pattern (flat and folder layout)
- Update test assertion for module stale detection output

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(frontend): WAC UI improvements — reorder templates, module tab rename, import consolidation

- Reorder WAC template buttons: TypeScript before Python in
  ScriptBuilder, CreateActionsScript, and CreateActionsFlow
- Remove dropdown items from +Script button (simplify to direct link)
- Move "Import Workflow-as-Code" to +Flow dropdown with dedicated drawer
- Add module tab rename: pencil icon on hover opens popover with
  validation, fixed-width icon container prevents layout shift

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: remaining module-mode changes from working branch

- Backend parser updates for WAC detection
- CLI sync/types updates for raw app path and module support
- Frontend UI polish (Dev.svelte, ScriptRow, script hash page)
- Test fixture updates

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(cli): add test for module modification detection in generate-metadata

Verifies that modifying a single module file re-triggers stale
detection and only the changed module is listed, not all modules.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backend): critical fixes from PR review

- Fix hardcoded dev path in bun_executor.rs WAC v2 wrapper — use
  "windmill-client" import instead of absolute filesystem path
- Fix missed no_main_func → auto_kind rename in parser TS test
- Add modules column to clone_script SQL (windmill-common and
  windmill-api-workspaces) so cloned scripts retain their modules
- Add modules: None to RawCode structs in worker tests
- Restore complete sqlx cache (merge main's cache + our new queries)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backend): fix clone warning treated as error in CI

Change `.clone()` on double reference to `*k` dereference in
scripts.rs hash implementation. Update sqlx cache with new query
hashes from modified clone_script SQL.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(frontend): use published parser wasm versions for CI build

The local file:// paths for windmill-parser-wasm-py and
windmill-parser-wasm-ts don't exist in the Cloudflare Pages build
environment. Revert to published npm versions (1.655.0).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(frontend): update parser wasm packages to 1.657.2

Use newly published windmill-parser-wasm-ts and windmill-parser-wasm-py
v1.657.2 which include auto_kind/WAC detection changes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(frontend): regenerate package-lock.json for npm ci compatibility

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(frontend): use main's lockfile as base, update only parser wasm packages

Regenerating package-lock.json from scratch pulled different dependency
versions causing svelte-check type errors. Instead, start from main's
lockfile and only update the two changed packages.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backend): add modules column to fetch_script_for_update query

The Script<SR> struct has a modules field (FromRow), but
fetch_script_for_update didn't SELECT modules, causing a runtime
error "no column found for name: modules" when the worker processed
dependency jobs. This was the root cause of the relock_skip test
timeout.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backend): fix script module execution for Python and Bun

- Fix modules not passed through job queue: inject _MODULES into
  PushArgs.extra when pushing Code jobs so worker can extract them
- Fix Python module imports: use relative imports (from .helper)
  and add sys.path.insert for module directory in wrapper
- Fix Python tests: use relative imports and empty lock to prevent
  pip from resolving module names as packages
- Add local file check in Bun loader for module resolution
- Ignore Bun module test (bundle mode loader integration tracked
  separately)
- Add missing modules column to fetch_script_for_update query

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backend): remove unnecessary empty lock in Python module tests

Relative imports (from .helper) are not parsed as pip packages,
so the empty lock workaround is not needed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backend): fix module execution for Python and Bun — all tests pass

Python modules:
- Use relative imports (from .helper import greet) since scripts run
  as packages
- Add sys.path.insert for module directory in wrapper to ensure local
  modules take precedence over pip packages with same name

Bun modules:
- Use bundled output (./out/main.js) as wrapper import when modules
  are present — the bundled output has module content inlined by
  Bun.build, avoiding runtime loader resolution issues
- Add local file check in loader.bun.js onResolve to short-circuit
  API URL resolution for module files on disk

Job queue:
- Inject _MODULES into PushArgs.extra when pushing Code jobs so
  the worker can extract them at execution time

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: address PR review — simplify, fix correctness, remove dead code

Critical fixes:
- Replace all CLI `no_main_func` references with `auto_kind` (string)
  to match the backend migration and API changes
- Remove duplicated `compute_python_module_dir` in worker.rs, use
  the canonical version from python_executor.rs

High priority:
- Auto-create `__init__.py` in intermediate directories for nested
  Python modules so imports like `from .utils.math import add` work
  without users manually creating __init__.py files
- Remove redundant `sys_path_insert` — relative imports use Python's
  package system, not sys.path

Medium:
- Fix lock file base name extraction: use regex to strip only the
  final extension (`.replace(/\.[^.]+$/, '')`) instead of `indexOf(".")`
  which breaks for files like `helper.test.ts`

Simplification:
- Remove dead `{#if false}` Popover block in ScriptEditor.svelte
- Guard loader.bun.js local file check to only run for relative paths
  (matching the Windows loader pattern)
- Add clarifying comment on Bun dual mechanism (build + run phases)
- Add maintenance comment on manual Hash impl for NewScript

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: final review fixes — stale cleanup, baseName, auto_kind export

- Fix sync.ts baseName extraction using indexOf(".") → regex
  (same fix as script.ts/metadata.ts, missed this instance)
- Add stale module file cleanup in writeModulesToDisk: removes files
  from __mod/ that are no longer in the modules map before writing,
  fixing the pull→push cycle that couldn't delete modules
- Log warning when _MODULES serialization fails in job push instead
  of silently dropping modules
- Use strict equality (===) for auto_kind comparison
- Exclude auto_kind from workspace export — it is auto-detected by
  the parser at deploy time from script content

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): remove auto_kind from push, comparison, and metadata

auto_kind is auto-detected by the parser at deploy time, so the CLI
should not send it, compare it, or write it to script.yaml.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove erroneously added backend/backend/.sqlx directory

Duplicate .sqlx cache was committed at the wrong nested path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address PR review feedback + fix CI dead_code warning

Frontend (ScriptEditor.svelte):
- Fix switchToMain() missing lastSyncedCode update — prevents stale
  code sync on external changes while editing a module tab
- Fix formatAction saving module code to main script's localStorage
  draft — now saves main code when on a module tab
- Fix non-null assertion on inferModuleLang in renameModule — fall
  back to original language instead of force unwrap
- Remove redundant activeModuleTab truthy check in runTest

CLI (script.ts):
- Clean up empty directories after removing stale module files in
  writeModulesToDisk

Backend:
- Add path traversal guard in write_module_files — reject module
  paths containing ".."
- Fix dead_code warning on auto_kind field in workspace export struct

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(frontend): improve auto_kind UX + address review findings

- Rename "Include without main function" toggle to "Include library
  scripts" in script list (ItemsList.svelte)
- Update NoMainFuncBadge: "No main" → "Library" with clearer tooltip
- Filter module file extensions by main script language — Python
  scripts only allow .py modules, TypeScript only .ts, etc.
- Split flushModuleState into flushModuleContent (no UI side-effect)
  and flushModuleState (flush + reset tab), reducing duplication
- Dynamic placeholder and hint text in add module popover based on
  main script language

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 01:20:09 +00:00

456 lines
14 KiB
Rust

use serde_json::json;
use wasm_bindgen_test::wasm_bindgen_test;
use windmill_parser::{Arg, MainArgSignature, ObjectProperty, ObjectType, Typ};
use windmill_parser_bash::parse_powershell_sig;
use windmill_parser_ts::{parse_deno_signature, parse_expr_for_ids, parse_expr_for_imports};
#[allow(dead_code)]
#[wasm_bindgen_test]
fn test_parse_deno_sig() -> anyhow::Result<()> {
let code = "
export function main(test1?: string, test2: string = \"burkina\",
test3: wmill.Resource<'postgres'>, b64: Base64, ls: Base64[],
email: Email, literal: \"test\", literal_union: \"test\" | \"test2\",
opt_type?: string | null, opt_type_union: string | null, opt_type_union_union2: string | undefined,
min_object: {a: string, b: number},
literals_with_undefined: \"foo\" | \"bar\" | undefined,
dyn_select: DynSelect_foo) {
console.log(42)
}
";
assert_eq!(
parse_deno_signature(code, false, false, None)?,
MainArgSignature {
star_args: false,
star_kwargs: false,
args: vec![
Arg {
otyp: None,
name: "test1".to_string(),
typ: Typ::Str(None),
default: None,
has_default: true,
oidx: None
},
Arg {
otyp: None,
name: "test2".to_string(),
typ: Typ::Str(None),
default: Some(json!("burkina")),
has_default: true,
oidx: None
},
Arg {
otyp: None,
name: "test3".to_string(),
typ: Typ::Resource("postgres".to_string()),
default: None,
has_default: false,
oidx: None
},
Arg {
otyp: None,
name: "b64".to_string(),
typ: Typ::Bytes,
default: None,
has_default: false,
oidx: None
},
Arg {
otyp: None,
name: "ls".to_string(),
typ: Typ::List(Box::new(Typ::Bytes)),
default: None,
has_default: false,
oidx: None
},
Arg {
otyp: None,
name: "email".to_string(),
typ: Typ::Email,
default: None,
has_default: false,
oidx: None
},
Arg {
otyp: None,
name: "literal".to_string(),
typ: Typ::Str(Some(vec!["test".to_string()])),
default: None,
has_default: false,
oidx: None
},
Arg {
otyp: None,
name: "literal_union".to_string(),
typ: Typ::Str(Some(vec!["test".to_string(), "test2".to_string()])),
default: None,
has_default: false,
oidx: None
},
Arg {
otyp: None,
name: "opt_type".to_string(),
typ: Typ::Str(None),
default: None,
has_default: true,
oidx: None
},
Arg {
otyp: None,
name: "opt_type_union".to_string(),
typ: Typ::Str(None),
default: None,
has_default: true,
oidx: None
},
Arg {
otyp: None,
name: "opt_type_union_union2".to_string(),
typ: Typ::Str(None),
default: None,
has_default: true,
oidx: None
},
Arg {
otyp: None,
name: "min_object".to_string(),
typ: Typ::Object(ObjectType::new(None, Some(vec![
ObjectProperty { key: "a".to_string(), typ: Box::new(Typ::Str(None)) },
ObjectProperty { key: "b".to_string(), typ: Box::new(Typ::Float) }
]))),
default: None,
has_default: false,
oidx: None
},
Arg {
otyp: None,
name: "literals_with_undefined".to_string(),
typ: Typ::Str(Some(vec!["foo".to_string(), "bar".to_string()])),
default: None,
has_default: true,
oidx: None
},
Arg {
otyp: None,
name: "dyn_select".to_string(),
typ: Typ::DynSelect("foo".to_string()),
default: None,
has_default: false,
oidx: None
}
],
auto_kind: None,
has_preprocessor: Some(false)
}
);
Ok(())
}
#[allow(dead_code)]
#[wasm_bindgen_test]
fn test_parse_deno_sig_implicit_types() -> anyhow::Result<()> {
let code = "
export function main(test2 = \"burkina\",
bool = true,
float = 4.2,
int = 42,
ls = [\"test\"],
min_object = {a: \"test\", b: 42}) {
console.log(42)
}
";
assert_eq!(
parse_deno_signature(code, false, false, None)?,
MainArgSignature {
star_args: false,
star_kwargs: false,
args: vec![
Arg {
otyp: None,
name: "test2".to_string(),
typ: Typ::Str(None),
default: Some(json!("burkina")),
has_default: true,
oidx: None
},
Arg {
otyp: None,
name: "bool".to_string(),
typ: Typ::Bool,
default: Some(json!(true)),
has_default: true,
oidx: None
},
Arg {
otyp: None,
name: "float".to_string(),
typ: Typ::Float,
default: Some(json!(4.2)),
has_default: true,
oidx: None
},
Arg {
otyp: None,
name: "int".to_string(),
typ: Typ::Int,
default: Some(json!(42)),
has_default: true,
oidx: None
},
Arg {
otyp: None,
name: "ls".to_string(),
typ: Typ::List(Box::new(Typ::Str(None))),
default: Some(json!(["test"])),
has_default: true,
oidx: None
},
Arg {
otyp: None,
name: "min_object".to_string(),
typ: Typ::Object(ObjectType::new(None, Some(vec![
ObjectProperty { key: "a".to_string(), typ: Box::new(Typ::Str(None)) },
ObjectProperty { key: "b".to_string(), typ: Box::new(Typ::Int) }
]))),
default: Some(json!({"a": "test", "b": 42})),
has_default: true,
oidx: None
}
],
auto_kind: None,
has_preprocessor: Some(false)
}
);
Ok(())
}
#[allow(dead_code)]
#[wasm_bindgen_test]
fn test_parse_deno_types() -> anyhow::Result<()> {
let code = "
type FooBar = {
a: string,
b: number,
}
export function main(foo: FooBar, {a, b}: FooBar, {c, d}: FooBar = {a: \"foo\", b: 42}) {
}
";
assert_eq!(
parse_deno_signature(code, false, false, None)?,
MainArgSignature {
star_args: false,
star_kwargs: false,
args: vec![
Arg {
name: "foo".to_string(),
otyp: None,
typ: Typ::Resource("foo_bar".to_string()),
default: None,
has_default: false,
oidx: None
},
Arg {
name: "anon1".to_string(),
otyp: None,
typ: Typ::Resource("foo_bar".to_string()),
default: None,
has_default: false,
oidx: None
},
Arg {
name: "anon2".to_string(),
otyp: None,
typ: Typ::Resource("foo_bar".to_string()),
default: Some(json!({"a": "foo", "b": 42})),
has_default: true,
oidx: None
}
],
auto_kind: None,
has_preprocessor: Some(false)
}
);
Ok(())
}
#[allow(dead_code)]
#[wasm_bindgen_test]
fn test_parse_enum_list() -> anyhow::Result<()> {
let code = "
export function main(foo: (\"foo\" | \"bar\")[]) {
}
";
assert_eq!(
parse_deno_signature(code, false, false, None)?,
MainArgSignature {
star_args: false,
star_kwargs: false,
args: vec![Arg {
name: "foo".to_string(),
otyp: None,
typ: Typ::List(Box::new(Typ::Str(Some(vec![
"foo".to_string(),
"bar".to_string()
])))),
default: None,
has_default: false,
oidx: None
}],
auto_kind: None,
has_preprocessor: Some(false)
}
);
Ok(())
}
#[allow(dead_code)]
#[wasm_bindgen_test]
fn test_parse_extract_ident() -> anyhow::Result<()> {
let code = "
let foo = 3;
bar
baroof.foob.ar
foobar[barfoo.x]
";
assert_eq!(
parse_expr_for_ids(code)?,
vec![
("baroof".to_string(), "foob".to_string()),
("barfoo".to_string(), "x".to_string())
]
);
Ok(())
}
#[allow(dead_code)]
#[wasm_bindgen_test]
fn test_parse_imports() -> anyhow::Result<()> {
let code = "
import * as foo from '@foo/bar';
import { bar } from \"./bar\";
import { bar } from \"bar/foo/d\";
import { bar as baroof } from \"bar\";
";
let mut l = parse_expr_for_imports(code, false)?;
l.sort();
assert_eq!(
l,
vec![
"./bar".to_string(),
"@foo/bar".to_string(),
"bar".to_string(),
"bar/foo/d".to_string()
]
);
Ok(())
}
#[allow(dead_code)]
#[wasm_bindgen_test]
fn test_parse_imports_dts() -> anyhow::Result<()> {
let code = "
export type foo = number
";
let mut l = parse_expr_for_imports(code, false)?;
l.sort();
assert_eq!(l, vec![] as Vec<String>);
Ok(())
}
#[allow(dead_code)]
#[wasm_bindgen_test]
fn test_parse_powershell_sig() -> anyhow::Result<()> {
let code = "
param($test_none, [string]$test_string [int]$test_int, [decimal]$test_decimal, [double]$test_double, [single]$test_single, [datetime]$test_datetime_lower, [DateTime]$test_datetime_upper)
Write-Output 'Testing...'
";
assert_eq!(
parse_powershell_sig(code)?,
MainArgSignature {
star_args: false,
star_kwargs: false,
args: vec![
Arg {
otyp: None,
name: "test_none".to_string(),
typ: Typ::Str(None),
default: None,
has_default: false,
oidx: None
},
Arg {
otyp: None,
name: "test_string".to_string(),
typ: Typ::Str(None),
default: None,
has_default: false,
oidx: None
},
Arg {
otyp: None,
name: "test_int".to_string(),
typ: Typ::Int,
default: None,
has_default: false,
oidx: None
},
Arg {
otyp: None,
name: "test_decimal".to_string(),
typ: Typ::Float,
default: None,
has_default: false,
oidx: None
},
Arg {
otyp: None,
name: "test_double".to_string(),
typ: Typ::Float,
default: None,
has_default: false,
oidx: None
},
Arg {
otyp: None,
name: "test_single".to_string(),
typ: Typ::Float,
default: None,
has_default: false,
oidx: None
},
Arg {
otyp: None,
name: "test_datetime_lower".to_string(),
typ: Typ::Datetime,
default: None,
has_default: false,
oidx: None
},
Arg {
otyp: None,
name: "test_datetime_upper".to_string(),
typ: Typ::Datetime,
default: None,
has_default: false,
oidx: None
}
],
auto_kind: None,
has_preprocessor: None
}
);
Ok(())
}