worker-batch-pull-write
7 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Diego Imbert
|
fda68a72e5 |
feat: Support .ducklake() and .datatable() in agent workers (#8697)
* Update clients to check for agent workers * fixes * typescript uses 127.0.0.1 * Refresh system prompts * fix: check both localhost and 127.0.0.1 in workerHasInternalServer detection Both Python and TypeScript clients now check for both hostnames to avoid silent breakage if BASE_INTERNAL_URL uses one or the other. Also adds return type annotation to the Python method. Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com> * refresh system prompts * nit localhost regex boundary * fix: use provider.language instead of undefined bare language in sqlUtils The language variable was referenced as a bare identifier in the fetch calls, resolving to undefined at runtime instead of reading from provider.language. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
a46aa641f9 |
feat: add R language support (#8263)
* feat: add R language support Add R as a new supported scripting language in Windmill, following the same pattern used for Ruby. Includes: - Backend: ScriptLang::Rlang enum variant, DB migration, tree-sitter-r parser crate with tests, WASM parser binding, R executor with NSJail sandboxing, job dispatch and signature parsing - Frontend: language picker, R icon, syntax highlighting, editor bar insertions (Sys.getenv, get_variable, get_resource), schema inference, init code template, BETA badge - CLI: .r extension mapping, sync support, bootstrap template R scripts use `main <- function(...)` syntax, jsonlite for JSON serialization, and system curl for the Windmill client helper. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add R package resolution and installation Parse library()/require() calls from R scripts to extract dependencies. Resolve versions from CRAN, cache lockfiles in pip_resolution_cache, and install packages to a shared R library cache. The run step sets R_LIBS_USER so installed packages are available to the script. - Parser: parse_r_requirements() extracts package names from AST - Executor: resolve() generates lockfile, install() installs from CRAN - Worker lockfiles: wire up R resolve for dependency jobs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add nsjail sandboxing for R resolve and install phases Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: fix R get_variable/get_resource and add sandbox annotation + e2e tests Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: fix R arg inference with JS fallback parser and get_variable/get_resource Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix flake * nsjail * nits * fix: R install improvements - suppress verbose output, flat lockfile logging, Dockerfile R support, rlimits - Suppress renv verbose output during resolve and install (controlled by #verbose annotation) - Filter renv from install list (already loaded, causes noisy restart message) - Log compact "resolved N packages" instead of full renv.lock JSON - Add R (r-base, r-cran-renv) to DockerfileFull and DockerfileFullEe - Use disable_rl for nsjail install config (R compiles from source) - Reduce default concurrency from 20 to 5 - Add rlang to openflow.openapi.yaml - Fix MainArgSignature (no_main_func -> auto_kind) after main merge Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * final * fix: remove accidental R install from multiplayer Dockerfile Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: remove R from Windows build and DockerfileExtra Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: rename R migration to avoid timestamp collision with trigger_filter_logic Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * all * fix: R install improvements - suppress verbose output, flat lockfile logging, Dockerfile R support, rlimits Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: add clear error when Rscript binary is missing Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: fix type errors in R fallback parser, use format! in wrap(), add R system prompts Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: pyranota <pyra@duck.com> |
||
|
Ruben Fiszel
|
d578e40101 |
feat: add selfApproval option to WAC + inline approval buttons (#8440)
* feat: add selfApproval option to WAC waitForApproval + inline approval buttons Add self-approval configuration to WAC workflows and inline approve/reject buttons in WorkflowTimeline. - TS SDK: add selfApproval option to waitForApproval() - Python SDK: add self_approval param to wait_for_approval() - Backend: store approval_conditions in flow_status for WAC, enforce self-approval checks on resume endpoints - Frontend: show Approve/Reject buttons in timeline with form support (EE), gated by user permissions Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: revert sqlx query change + regenerate system prompts - Revert get_suspended_flow_info to use original sqlx::query_as! with COALESCE to avoid sqlx offline cache mismatch in CI - Detect WAC by checking if FlowStatus parsing fails + suspend > 0 - Re-fetch flow_status column separately for WAC approval conditions - Regenerate auto-generated system prompt files for SDK changes Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: use resume URLs for WAC inline approval buttons - Backend generates HMAC-signed resume/cancel URLs when creating WAC approval, stores them in timeline entry and approval meta - Frontend uses anonymous resume endpoint (like classic flows) with fallback to resumeSuspendedFlowAsOwner for admins - Buttons show for everyone when URLs are present; server-side self_approval_disabled check enforces restrictions - Show warning for admins/owners when self-approval is disabled - selfApproval: false requires EE (errors at dispatch on CE) - self_approval_disabled check moved outside user_auth_required gate so it works independently - WAC detection no longer requires task import Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add resume_suspended and approval_info endpoints - New approval_token DB table for token-based approval access - New POST /jobs_u/flow/resume_suspended/{job_id} endpoint: - OptAuthed: works with login or approval_token - Checks approval_conditions (self_approval, groups, auth) - Admins/owners bypass rules - New GET /jobs_u/flow/approval_info/{job_id} endpoint: - Returns form, rules, can_approve status - HMAC anonymous endpoint now bypasses all approval_conditions (secret = full capability) - getResumeUrls approvalPage URL now uses token format - WAC approval dispatch generates and stores approval tokens - Mark resumeSuspendedFlowAsOwner as legacy Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: simplify frontend to use resume_suspended endpoint - OpenAPI spec updated with resume_suspended and approval_info endpoints - WorkflowTimeline: removed URL parsing, now calls single resumeSuspended endpoint for both approve and reject - Buttons show for any logged-in user viewing the job (backend enforces authorization rules) - Kept self-approval warning for admins Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: stateless approval tokens, new approval page, FlowStatusWaitingForEvents update - Replace DB-stored approval tokens with stateless HMAC derivation: token = HMAC(workspace_key, job_id + "approval_token") Verifiable without DB lookup, not reversible to resume secret - Drop approval_token migration (no DB table needed) - FlowStatusWaitingForEvents: use resumeSuspended endpoint instead of URL parsing + resumeSuspendedFlowAsOwner - New approval page route /approve/{ws}/{job}?token= that uses approval_info and resume_suspended endpoints - Old approval page route kept for back-compat Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: match old approval page content in new approval page - Add FlowMetadata, JobArgs, FlowGraphV2, DisplayResult - Add approvers with tooltips, flow arguments section - Add admin self-approval bypass warning - Add "Open run details" link - Fetch full job alongside approval_info for all UI data Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: filter _MODULES from args, show 'workflow' for WAC approvals Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: remove deno template from approval/prompt SuspendDrawer Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: approval page form display + hide deno from approval script picker - Fix form schema rendering on new approval page by wrapping flat WAC form schemas in { properties, order } for SchemaForm - Hide deno from the approval step language picker in flow editor Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove deno from canHaveApproval in script_helpers.ts The insert menu uses canHaveApproval() from script_helpers.ts via FlowInputsQuick, not the displayLang function in FlowInputs.svelte. Revert the unnecessary FlowInputs.svelte change. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: return form schema and description in approval_info for classic flows The approval_info endpoint was returning None for form_schema on classic flows. Now fetches raw_flow to get suspend.resume_form schema, hide_cancel, and the step's completed result for description. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: inline Login component on approval page instead of redirect Show the Login component directly on the approval page when authentication is required. On successful login, reloads user and approval info without navigating away. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: show resume buttons for all users, not just owners The resume_suspended endpoint handles authorization server-side, so the frontend should always show the buttons. Remove isOwner gate and the "cannot resume" message. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: prevent layout shift on resume by removing spinner from cancel button Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: prevent resume button expansion by using disabled instead of loading The loading prop adds a Loader2 spinner that expands the button width. Use disabled={loading} instead to prevent layout shift. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: approval page login redirects back with full page reload Set rd to the full URL (starts with http) so Login.redirectUser() uses window.location.href instead of goto(), triggering a full page reload after login. This ensures the approval page re-fetches data as an authenticated user. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: fetch flow definition from flow_version when raw_flow is null Deployed flows don't store raw_flow on the job. Fall back to flow_version table using runnable_id to get suspend settings (form schema, hide_cancel) for the approval_info endpoint. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: show specific reasons when user cannot approve Display whether denial is due to self-approval being disabled, required group membership, or both. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: support both nested and flat form schema in waitForApproval Users can now pass either: waitForApproval({ form: { schema: { name: { type: "string" } } } }) or: waitForApproval({ form: { name: { type: "string" } } }) Both WorkflowTimeline and approval page handle both formats. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: convert sqlx query macros to non-macro for CI offline cache Replace sqlx::query! and sqlx::query_scalar! with sqlx::query and sqlx::query_as to avoid SQLX_OFFLINE cache misses in CI. Also remove unused LogIn import from approval page. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: suppress dead code warning + unused isOwner variable - Add #[allow(dead_code)] to without_flow method (CI -D warnings) - Rename isOwner to _isOwner in FlowStatusWaitingForEvents (unused) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: security and robustness fixes from PR review - Add workspace_id verification in resume_suspended to prevent cross-workspace approval (#3) - Fix token leakage: use relative path for login redirect instead of full URL with token (#4) - Handle getJob failure independently from approval_info so the page works for unauthenticated users (#7) - Clear error state on successful data load (#13) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address review feedback — shared token gen, rand resume_id, UX - Move generate_approval_token to windmill-common::variables (shared between windmill-api and windmill-worker, eliminates duplicate HMAC) - Use rand::random::<u32>() for resume_id instead of DefaultHasher - Stop polling after approve/reject on approval page - Add cancelLoading state to WorkflowTimeline Reject button Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
749964e326 |
ci: add freshness check for auto-generated system prompts (#8338)
* ci: add freshness check for auto-generated system prompts Add a CI workflow and script to verify system_prompts/auto-generated/ stays in sync with its source files (SDKs, schemas, CLI commands, etc). Also remove the hardcoded CLI version from generated output to avoid unnecessary churn on every release. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * imports * imports Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * regenerate system prompts after rebase on main Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
centdix
|
b40cf80fdd |
fix: optimize flow lock generation and add rt.d.ts guidance for TS resource types (#8295)
Instruct AI to pass specific flow folder path to `wmill flow generate-locks` instead of running it on all flows. Also add guidance for TypeScript language files to check `rt.d.ts` for available resource types before using them. Re-ran generate.py to propagate changes to all auto-generated files. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
centdix
|
a7ce5484b8 |
feat(local-dev): create Claude skills when doing wmill init (#7699)
* use skills * add prompts * update system prompts * generate skills on init * add prompts in cli * better for raw apps * nit * test pipeline draft * better * yaml for triggers and schedules * cleaning * better * add descriptions to ai agent fileds * adjust * better openapi * better * nit * feat: add typed provider and memory schemas for ai agent in openapi Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: improve zod validation errors with dynamic schema extraction Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * regen * fix * cleaning * refactor: deduplicate skill descriptions in generate_skills_ts_export Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * cleaning --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev> |
||
|
centdix
|
31e002ad41 |
feat(ai): standardize and improve system prompts (#7346)
* init * test in frontend * copy files * use in cli * better * add desc to sdks * better * fix ts parsing * add docs to ts client * add docs to python client * use script prompt in frontend * regen * use in flow * rm * use in cli, create AGENTS.md instead of cursor rules * remove apply * better * better * simplify cli * more docs * cleaning * update readme * generate cli file * better folder names * fix ts * fix multiline |