Files
windmill/backend/windmill-worker/src/agent_workers.rs
Diego Imbert 9bbab3321e feat: Data tables (#7226)
* data tables settings ui

* install runed

* zod 4 fixes

* use new toJSONSchema

* Migrate ducklake catalogs to more generic custom instance databases

* fix compilation

* Safety conversion for old duckdb ffi

* data tables settings

* ts client basis

* inline run works

* datatables work

* Revert "datatables work"

This reverts commit 6e1588d59e.

* datatables work (without leaking pg credentials)

* println

* separate sqlUtils.ts

* nit

* Separate custom instance db Select and Wizard components

* nit

* nit wording

* add tags to custom instance dbs

* error when trying to use ducklake as datatable or opposite

* show status in dropdown

* data table instance setup works

* sqk function for ducklake

* factorize logic

* fix temp reactivity

* Data table assetexplore

* Migrate S3 permissions to modal

* Revert "Migrate S3 permissions to modal"

This reverts commit 0631d03cb0.

* nit query -> fetch

* Custom instance setup new look

* run_language_executor separate fn

* run_inline param

* nit wording

* Better typed client

* Data tables display as assets in frontend

* asset db icon

* nit

* cleaner errors

* nit

* Fix sed calls in mac

* run_inline_script_preview in python client

* basic python datatable client

* datatable and datalake parser in python

* ducklake client python

* nit fix

* Fix migration producing NULL instead of {} when no custom databases

* merge conflict fail

* python ducklake client arg fix

* parse or infer sql types in ts client

* ts asset parser, detect datatable & ducklake R/W

* fix sql repl for other read ops than select

* export type SqlTemplateFunction

* rename list_custom_instance_pg_databases

* typecheck datatable and ducklake name in Typescript

* Fix typecheck datatable and ducklake in TS

* declare module overriding instead of extending

* infer_sql_type in python client

* SqlQuery object in python

* fix merge conflicts

* update const_format

* CI fix

* factor out to var_identifiers

* sqlx prepare

* unnecessary security (admin is required)

* clearer comment

* ee repo ref

* nit snake case

* claude step 1: detect var declarations

* move detect_sql_access_type to common mod

* claude step 2: detect when saved vars are queried

* Revert "claude step 2: detect when saved vars are queried"

This reverts commit 1e1f930568.

* Revert "claude step 1: detect var declarations"

This reverts commit f866f4819d.

* remove ducklake/datatable and default

* detect data table assigns in var_identifiers

* Python parser successfully infers R/W/RW from ducklake / datatable

* still register ducklake/datatable if not used as unknown R/W

* Go to settings button in Assets Dropdown on not found

* nit

* sqlx prepare fail

* manual fix, somehow sqlx prepare won't do it

* fix frontend ci

* ee repo ref

* ducklake_user doesnt exist in unit tests

* nit fix

* ui nit

* nit

* nit missing clone

* fork ducklakes and datatables

* fix surface hover bug

* stupid mistake

* better deeply reactive mutable derived

* Ducklake picker

* Editor bar data tables

* DuckDB supports datatables

* datatable in duckdb asset parser

* duckdb asset parser var_identifiers

* Revert "duckdb asset parser var_identifiers"

This reverts commit 88068b1a77.

* sqlx prepare

* Box pin in test_workflow_as_code to fix stack overflow

* go to settings button

* ee repo ref

* fix compilation

* wording nit
2025-12-05 23:08:58 +00:00

82 lines
2.1 KiB
Rust

use reqwest::header::HeaderMap;
use uuid::Uuid;
use windmill_common::{
agent_workers::QueueInitJob, worker::HttpClient, workspaces::DucklakeWithConnData,
};
use windmill_queue::{JobAndPerms, JobCompleted};
pub async fn queue_init_job(client: &HttpClient, content: &str) -> anyhow::Result<Uuid> {
client
.post(
"/api/agent_workers/queue_init_job",
None,
&QueueInitJob { content: content.to_string() },
)
.await
.and_then(|x: String| Uuid::parse_str(&x).map_err(|e| anyhow::anyhow!(e)))
}
pub async fn queue_periodic_job(client: &HttpClient, content: &str) -> anyhow::Result<Uuid> {
client
.post(
"/api/agent_workers/queue_periodic_job",
None,
&QueueInitJob { content: content.to_string() },
)
.await
.and_then(|x: String| Uuid::parse_str(&x).map_err(|e| anyhow::anyhow!(e)))
}
pub async fn pull_job(
client: &HttpClient,
headers: Option<HeaderMap>,
body: Option<bool>,
) -> anyhow::Result<Option<JobAndPerms>> {
client
.post("/api/agent_workers/pull_job", headers, &body)
.await
}
pub async fn send_result(client: &HttpClient, jc: JobCompleted) -> anyhow::Result<String> {
client
.post(
&format!(
"/api/w/{}/agent_workers/send_result/{}",
jc.job.workspace_id, jc.job.id
),
None,
&jc,
)
.await
}
#[allow(dead_code)]
pub async fn get_ducklake_from_agent_http(
client: &HttpClient,
name: &str,
w_id: &str,
) -> anyhow::Result<DucklakeWithConnData> {
client
.get(&format!(
"/api/w/{}/agent_workers/get_ducklake/{}",
w_id, &name
))
.await
}
#[allow(dead_code)]
pub async fn get_datatable_resource_from_agent_http(
client: &HttpClient,
name: &str,
w_id: &str,
) -> anyhow::Result<serde_json::Value> {
client
.get(&format!(
"/api/w/{}/agent_workers/get_datatable_resource/{}",
w_id, &name
))
.await
}
pub const UPDATE_PING_URL: &str = "/api/agent_workers/update_ping";