Files
Ruben Fiszel c57c769dea feat: add CI test scripts with auto-trigger on deploy (#8736)
* feat: add CI test scripts with auto-trigger on deploy

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: fix annotation parser early return and handle renames correctly

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: move CI test results to top of script/flow detail pages

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: improve CI test results spacing, icon, and remove pass label

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: support one-line annotation and use script/path format

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: move CI test trigger logic to EE

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: move CI badge next to New badge and add deduplicated CI summary

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add CI test e2e tests and fix nullable column annotations

Add integration tests for CI test annotation parsing (creates/removes
ci_test_reference rows) and the CI test results API (single + batch
endpoints). Add backend test for auto-trigger on deploy (private+python).

Fix sqlx LEFT JOIN LATERAL nullable column annotations in
get_ci_test_results and get_ci_test_results_batch queries — sqlx
cannot infer nullability from LATERAL subqueries, causing runtime
decode errors when no matching job exists.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix build/sqlx

* fix

* feat: CI test improvements and templates

- Fix windmill-dep-map/private feature propagation in worker, api-scripts,
  and api-flows Cargo.toml so CI test triggers actually fire in EE mode
- Clone ci_test_reference rows during workspace fork
- Add polling to CiTestResults component (refetch every 3s while running)
- Add running state and auto-refresh to ForkWorkspaceBanner CI summary
- Add yellow "CI test" badge on script list rows and detail page
- Fix Library badge border color (remove indigo border override)
- Add CI Test TypeScript and CI Test Python templates in ScriptBuilder
- Update sqlx offline cache
- Add debug tracing for CI test trigger in worker_lockfiles

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add missing children prop to WorkspaceDeployLayout

Fixes svelte-fast-check type error when passing named snippets as
children content inside the component tag.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address PR review feedback

- Remove empty wrapper divs around CiTestResults, move mb-4 into component
- Add batch endpoint size cap (max 200 items)
- Add ON DELETE CASCADE to ci_test_reference workspace FK (new migration)
- Downgrade CI test trigger logs from info to debug
- Fix false-positive polling: only treat status='running' as running,
  not null status (CiTestResults, CompareWorkspaces, ForkWorkspaceBanner)
- Fix test numbering in integration tests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to latest EE commit

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to d9d68c2406df0b59f413ea0b2cb24780a9817d04

This commit updates the EE repository reference after PR #516 was merged in windmill-ee-private.

Previous ee-repo-ref: d7ccd9b86da99ec056a0e8708e3637d64290387a

New ee-repo-ref: d9d68c2406df0b59f413ea0b2cb24780a9817d04

Automated by sync-ee-ref workflow.

* fix: treat queued jobs (job_id set, null status) as running

Jobs that have been pushed but not yet picked up by a worker have a
job_id but null status. Treat these as 'running' to avoid showing
misleading 'pass' badges or '0 passing'. Tests that were never
triggered (no job_id, null status) remain neutral/hidden.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: hugocasa <hugo@casademont.ch>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-04-09 17:21:36 +00:00

201 lines
6.6 KiB
Rust

use serde_json::json;
use sqlx::{Pool, Postgres};
use windmill_test_utils::*;
fn script_url(port: u16, endpoint: &str, path: &str) -> String {
format!("http://localhost:{port}/api/w/test-workspace/scripts/{endpoint}/{path}")
}
fn client() -> reqwest::Client {
reqwest::Client::new()
}
fn authed(builder: reqwest::RequestBuilder) -> reqwest::RequestBuilder {
builder.header("Authorization", "Bearer SECRET_TOKEN")
}
fn new_script(path: &str, content: &str) -> serde_json::Value {
json!({
"path": path,
"summary": "",
"description": "",
"content": content,
"language": "deno",
"schema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {},
"required": []
}
})
}
/// Test 1: Creating a script with a CI test annotation inserts rows into ci_test_reference,
/// and deploying a new version without the annotation removes them.
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_ci_test_annotation_creates_and_removes_references(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let base = format!("http://localhost:{port}/api/w/test-workspace/scripts");
// Create target script (no annotation)
let resp = authed(client().post(format!("{base}/create")))
.json(&new_script(
"u/test-user/target_script",
"export async function main() { return 42; }",
))
.send()
.await?;
assert_eq!(resp.status(), 201, "create target: {}", resp.text().await?);
// Create test script with CI annotation
let resp = authed(client().post(format!("{base}/create")))
.json(&new_script(
"u/test-user/ci_test_script",
"// test: script/u/test-user/target_script\nexport async function main() { return 'test passed'; }",
))
.send()
.await?;
assert_eq!(resp.status(), 201, "create test: {}", resp.text().await?);
// Verify ci_test_reference row exists
let refs = sqlx::query!(
"SELECT test_script_path, tested_item_path, tested_item_kind \
FROM ci_test_reference WHERE workspace_id = 'test-workspace'"
)
.fetch_all(&db)
.await?;
assert_eq!(refs.len(), 1, "expected 1 ci_test_reference row");
assert_eq!(refs[0].test_script_path, "u/test-user/ci_test_script");
assert_eq!(refs[0].tested_item_path, "u/test-user/target_script");
assert_eq!(refs[0].tested_item_kind, "script");
// Get the hash of the test script (needed as parent_hash for the update)
let resp = authed(client().get(script_url(port, "get/p", "u/test-user/ci_test_script")))
.send()
.await?;
let body = resp.json::<serde_json::Value>().await?;
let hash = body["hash"].as_str().unwrap().to_string();
// Create new version of test script WITHOUT annotation
let mut updated = new_script(
"u/test-user/ci_test_script",
"export async function main() { return 'no longer a test'; }",
);
updated["parent_hash"] = json!(hash);
let resp = authed(client().post(format!("{base}/create")))
.json(&updated)
.send()
.await?;
assert_eq!(
resp.status(),
201,
"remove annotation: {}",
resp.text().await?
);
// Verify ci_test_reference row was deleted
let refs = sqlx::query!(
"SELECT test_script_path FROM ci_test_reference \
WHERE workspace_id = 'test-workspace'"
)
.fetch_all(&db)
.await?;
assert_eq!(
refs.len(),
0,
"ci_test_reference should be empty after removing annotation"
);
Ok(())
}
/// Test 2: The CI test results API returns test references (with null job info when
/// no CI test job has run yet), and the batch endpoint aggregates correctly.
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_ci_test_results_api(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let base = format!("http://localhost:{port}/api/w/test-workspace/scripts");
// Create target script
let resp = authed(client().post(format!("{base}/create")))
.json(&new_script(
"u/test-user/target_for_results",
"export async function main() { return 1; }",
))
.send()
.await?;
assert_eq!(resp.status(), 201, "create target: {}", resp.text().await?);
// Create test script with annotation
let resp = authed(client().post(format!("{base}/create")))
.json(&new_script(
"u/test-user/test_for_results",
"// test: script/u/test-user/target_for_results\nexport async function main() { return true; }",
))
.send()
.await?;
assert_eq!(resp.status(), 201, "create test: {}", resp.text().await?);
// --- Single item endpoint ---
let resp = authed(client().get(script_url(
port,
"ci_test_results/script",
"u/test-user/target_for_results",
)))
.send()
.await?;
assert_eq!(resp.status(), 200);
let results = resp.json::<Vec<serde_json::Value>>().await?;
assert_eq!(results.len(), 1, "expected 1 CI test result");
assert_eq!(
results[0]["test_script_path"],
"u/test-user/test_for_results"
);
// No CI test job has been triggered yet
assert!(results[0]["job_id"].is_null());
assert!(results[0]["status"].is_null());
// --- Batch endpoint ---
let resp = authed(client().post(format!(
"http://localhost:{port}/api/w/test-workspace/scripts/ci_test_results_batch"
)))
.json(&json!({
"items": [
{"path": "u/test-user/target_for_results", "kind": "script"},
{"path": "u/test-user/nonexistent", "kind": "script"}
]
}))
.send()
.await?;
assert_eq!(resp.status(), 200);
let batch = resp
.json::<serde_json::Map<String, serde_json::Value>>()
.await?;
// Target should have one test
let key = "script:u/test-user/target_for_results";
assert!(batch.contains_key(key), "missing key {key}");
let target_results = batch[key].as_array().unwrap();
assert_eq!(target_results.len(), 1);
assert_eq!(
target_results[0]["test_script_path"],
"u/test-user/test_for_results"
);
// Nonexistent should have empty array
let key = "script:u/test-user/nonexistent";
assert!(batch.contains_key(key), "missing key {key}");
assert_eq!(batch[key].as_array().unwrap().len(), 0);
Ok(())
}