Commit Graph

1124 Commits

Author SHA1 Message Date
HugoCasa
0c5abc1f2f presigned 2025-04-10 22:37:19 +02:00
Ruben Fiszel
1f1164199b nit 2025-04-10 01:00:29 +00:00
Ruben Fiszel
63fa499015 feat: agent workers v2 using http (#5588) 2025-04-10 00:56:11 +00:00
HugoCasa
b4f8c88c19 fix: mssql ca_cert deserializing (#5587) 2025-04-08 17:12:20 +02:00
Piyush मिश्र
1007f37f4b function takes 13 arguments but 14 arguments were supplied (#5577) 2025-04-06 20:38:39 +00:00
Piyush मिश्र
1f3e7d9029 fix: Dynamic select does not work with tag //native (#5576)
closes #5490
2025-04-05 20:43:53 +00:00
pyranota
f3ed108e43 nit: replace KJQXZ with more meaningful notation (#5547)
* nit: replace `KJQXZ` with more meaningful notation

Originally this string is located in all places, where modification is needed in order to add new language support

* relative -> related

* revert shebang in substitue.sh

* remove '}'
2025-04-02 21:03:04 +00:00
Alexander Petric
e5519f79aa feat: mssql support cert configuration (#5559)
* feat: mssql support cert configuration

* use tokio async

* using jobdir

* use std
2025-04-02 20:10:51 +00:00
Alexander Petric
3fd7a5ce9c fix: aad_token can be empty string (#5557)
* fix: mssql aad_token can be empty string

* simplify deserializer
2025-04-02 19:26:32 +02:00
Alexander Petric
dd30692617 fix: mssql aad_token can be empty string (#5556) 2025-04-02 18:51:02 +02:00
Alexander Petric
c230e2aed9 feat: ms sql aad authentication support (#5539)
* feat: ms sql aad authentication support

* format

* format
2025-03-31 04:05:08 +00:00
pyranota
c020af5a98 nit: fix windows build (#5531)
rror[E0425]: cannot find function `pull_from_tar` in module `crate::global_cache`
    --> windmill-worker\src\common.rs:1214:43
     |
1214 |                 Some(crate::global_cache::pull_from_tar(
     |                                           ^^^^^^^^^^^^^ not found in `crate::global_cache`
     |
note: found an item that was configured out
    --> windmill-worker\src\global_cache.rs:87:14
     |
87   | pub async fn pull_from_tar(
     |              ^^^^^^^^^^^^^
note: the item is gated here
    --> windmill-worker\src\global_cache.rs:86:1
     |
86   | #[cfg(all(feature = "enterprise", feature = "parquet", unix))]
     | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-03-28 17:17:23 +01:00
Ruben Fiszel
2e74595a37 nit, add scheduled_for in logs 2025-03-27 21:28:21 +01:00
pyranota
59740c0478 feat: add java support (#5458)
* feat: add nu (nushell) support

* add worker tests

* deactivate tables and non-any types below top-level

full support will come in V1
for V0 it's better to keep things minimal and simple

* add syntax highlighting

used python's grammar, since nushell isn't supported by monaco nor svelte-highlights

for V1 nu will get it`s own grammar

* add logo

* partially implement plugin support

* change logo + ability to deploy + nsjail draft

* static variables + get_resource + get_variable

* lsp/dev.nu + initial nu lsp (not working yet)

* make it work with nsjail

* nullguard

* Much more flexible signature parsing and better error-messages

* add init script

* rename nulsp to nu

* install nu to dockerfile

* fix merge

* implement Default for MainArgSignature

* stage NU_CACHE_DIR

* improve dockerfiles

* dev.nu for parser-wasm + flake.nix

* update code for windows

* add nushell to flake

* upload Cargo.lock

* make build.sh work on nixos

* build wasm cli parsers

* add docs to README_DEV.md

* add helper script docker/dev.nu

* improve docker/dev.nu

* fix windows

* commit frontend/package(lock).json

* update cargo.lock

* correctly update cargo.lock

* remove lsp

* update flake.nix to include svelte server and nushell

* Revert base.sql to main

* remove PLUGIN_USE_RE

* make CARGO_PATH private

* add nu to cli

* Change flags to build wasm-nu-parser

* remove flake.nix from parser-wasm

* update wasm-build target

* remove unused import

* add cli support for nu

* update github workflows

* wasm-build 0.17 -> 0.19

* update build script

* update cargo.lock

* Fix typographical error

* start working on java

* do java boilerplate

* implement parser for java

* update Cargo.lock

* update ENV_SETTINGS

* use published nu parser

* update package.lock

* java is S3 + Caching enabled

* install nsjail backup

* commit v0

* fix nsjail

* v0.1

* rewrite parser in tree-sitter

* implement parser from scratch

* polishing

* change init script to match new parser

* fix imports

* fix cli build

* fix cli build

* refactor install phase

* implement .valid.windmill atomic verification

* implement java init functionality

* remove quick-xml

* fix windows not recognizing 'mvn'

* create empty settings.xml if there is no config provided

* clean up

* change default settings.xml

* change classpath format for windows

* docs to helper

* java copy bin cache instead of symlink

* remove comments

* merge

* fix package.json

* fix package.json 2

* minor fixing

* migrate to Coursier

* update misc

* Http(s) Proxy + CA certs

* remove unused .wasm

* make requirements insensitive to spaces

* update handle_child refs

* rework save_cache for directories

* fix s3 bug

* compile .wasm for cli

* remove uuid import

* fix compilation

* use reference

* fix zero-dep failure

* removing unsafe stuff

* remove unneeded imports

* revert: we still need winapi

* remove nix store from nsjail

* do not create cache_nomount

* add java to dnt

* remove duplicated dependency in init script

* fix typos

* fix CI

* use published parser
2025-03-27 15:12:55 +01:00
Ruben Fiszel
8129672d9e fix: clean job dirs between flow locks 2025-03-26 23:23:04 +01:00
Ruben Fiszel
5573d88695 perf: cache workspace premium check 2025-03-25 02:00:51 +01:00
Diego Imbert
e868fe2bf5 feat: list references upon renaming a script or a flow (#5487)
* Refactored flow_workspace_runnables to more generic workspace_runnable_dependencies

* list flows referencing an item upon renaming it

* Refactor with two exclusive columns to avoid breaking FK constraints

* Show apps depending on item upon renaming

* sqlx prepare

* list-disc instead of •

* on delete and on update cascade

* displayPathChangedWarning oneOf check instead noneOf

* combine migrations + add "on update cascade" to flow fk

* unique index on app dependencies to avoid duplicates

* create new workspace_runnable_dependencies instead of renaming old table

* Add "looking for references" loading msg

* Revert "create new workspace_runnable_dependencies instead of renaming old table"

This reverts commit 015c38ca8f.

* flow_workspace_runnables view for backwards compatibility

* Add warning for script imports on rename

* support import dependency tracking in deno

* number of using scripts / flows / apps tooltip

* forgot sqlx prepare

* delete app-related rows in down migration
2025-03-24 22:08:19 +01:00
Ruben Fiszel
3edca4bc91 perf: optimize number of queries needed for job run (#5504)
* optPerf

* update sqlx

* update sqlx
2025-03-24 20:12:26 +01:00
wendrul
38ee0183aa feat: unsafe parameters for sql queries (table names, column names) (#5488)
* Make schema validation struct

Schema Validation rules that are constructed from the schema or from the
MainArgSig(TODO).

* Make other validator builder

* Fail dependency job like with lockfile failing for schema validator

* Add last types + tests

* Remove unused dependency

* fix typos

* Migration ID was colliding with another, changed it manually

* Add Oneof + other fixes

* fix: cache for querying scripts correclty handles ScriptMetadata

* Add cache for schema validation from main arg sig

* Prepare sqlx

* Remove default features

* Feature flags

* WIP: unsafe sql params for sql langauges

* Fix down migration table name

* cleanup: put validation logic inside a function

* Refactor to cache the should_validate boolean

Changed the schemavalidators cache to take in an
Option<SchemaValidator>, effectively storing the `should_validate_schema` information.

Also pass the schema when avaialble to construct the schema validator

* Add other job kinds to u8 cache key just in case

* Change sql languages to all get arguments as Values instead of RawValue

* Only cache if not preview

* Add last sql languages and some CI fixes

* Rename after typo on `sanitized`

* Finish rename

* Remove unused import

* Fix wrong test

* Add newly published regex parser version

* Remove default features from cargo.toml

* Change to a cleaner syntax for the interpolated args

* Update republished parser
2025-03-20 20:02:25 +01:00
pyranota
a3faea16e7 feat: add nu-lang support (#5217)
* feat: add nu (nushell) support

* add worker tests

* deactivate tables and non-any types below top-level

full support will come in V1
for V0 it's better to keep things minimal and simple

* add syntax highlighting

used python's grammar, since nushell isn't supported by monaco nor svelte-highlights

for V1 nu will get it`s own grammar

* add logo

* partially implement plugin support

* change logo + ability to deploy + nsjail draft

* static variables + get_resource + get_variable

* lsp/dev.nu + initial nu lsp (not working yet)

* make it work with nsjail

* nullguard

* Much more flexible signature parsing and better error-messages

* add init script

* rename nulsp to nu

* install nu to dockerfile

* fix merge

* implement Default for MainArgSignature

* stage NU_CACHE_DIR

* improve dockerfiles

* dev.nu for parser-wasm + flake.nix

* update code for windows

* add nushell to flake

* upload Cargo.lock

* make build.sh work on nixos

* build wasm cli parsers

* add docs to README_DEV.md

* add helper script docker/dev.nu

* improve docker/dev.nu

* fix windows

* commit frontend/package(lock).json

* update cargo.lock

* correctly update cargo.lock

* remove lsp

* update flake.nix to include svelte server and nushell

* Revert base.sql to main

* remove PLUGIN_USE_RE

* make CARGO_PATH private

* add nu to cli

* Change flags to build wasm-nu-parser

* remove flake.nix from parser-wasm

* update wasm-build target

* remove unused import

* add cli support for nu

* update github workflows

* wasm-build 0.17 -> 0.19

* update build script

* update cargo.lock

* Fix typographical error

* update Cargo.lock

* update ENV_SETTINGS

* use published nu parser

* update package.lock

* rewrite parser in tree-sitter

* implement parser from scratch

* polishing

* change init script to match new parser

* fix imports

* fix cli build

* fix cli build

* merge

* update wasm

* use MiniPulledJob

* update cli

* change cli wasm schema

* change cli

* update deno.json

* make wasm modules load lazily

* regenerate parsers

* remove leftover

* update cargo.lock

* clean up dnt.ts

* add docs to cli/test.nu

* add schema validation option

* add Nu to try_validate_schema

* reference frontend to new parser version
2025-03-20 16:45:21 +01:00
Ruben Fiszel
a6cf327f74 fix: improve memory metrics graph 2025-03-20 09:54:55 +01:00
wendrul
6634c82e20 feat: backend arg schema validation (#5455)
* Make schema validation struct

Schema Validation rules that are constructed from the schema or from the
MainArgSig(TODO).

* Make other validator builder

* Fail dependency job like with lockfile failing for schema validator

* Add last types + tests

* Remove unused dependency

* fix typos

* Migration ID was colliding with another, changed it manually

* Add Oneof + other fixes

* fix: cache for querying scripts correclty handles ScriptMetadata

* Add cache for schema validation from main arg sig

* Prepare sqlx

* Remove default features

* Feature flags

* Fix down migration table name

* cleanup: put validation logic inside a function

* Refactor to cache the should_validate boolean

Changed the schemavalidators cache to take in an
Option<SchemaValidator>, effectively storing the `should_validate_schema` information.

Also pass the schema when avaialble to construct the schema validator

* Add other job kinds to u8 cache key just in case

* Only cache if not preview
2025-03-19 19:41:10 +01:00
Ruben Fiszel
fb532e0b85 nit move away from using views 2025-03-19 10:48:43 +01:00
Ruben Fiszel
880db319e8 feat: filter by worker + backend perf opt (#5489)
* all

* all

* all

* refactor

* sqlx

* ref

* impr

* fix

* fix

* fix

* ee private
2025-03-19 09:55:53 +01:00
Ruben Fiszel
eb2ffee4ec pg nits 2025-03-14 12:13:12 +01:00
Ruben Fiszel
8e95bc3972 fix: avoid lock contention for native workers on cached connection (#5481) 2025-03-14 11:58:03 +01:00
Ruben Fiszel
d58ca9b395 fix: prevent monitoring task to die without sending killpill (#5472) 2025-03-12 11:51:53 +01:00
Ruben Fiszel
6aab95f1f9 display more complete error for failing to untar 2025-03-06 13:24:18 +01:00
Ruben Fiszel
fc987c7147 allow ce to run codebase bundles preview 2025-03-04 22:21:57 +01:00
pyranota
ebb58e0dc7 fix(python): windows worker fails to install 3.10 (#5409)
```
execution error:\nFind python error: error: Failed to inspect Python interpreter from managed installations at `C:\\tmp\\windmill\\cache\\py_runtime\\cpython-3.10.16-windows-x86_64-none\\python.exe`\n  Caused by: Querying Python at `C:\\tmp\\windmill\\cache\\py_runtime\\cpython-3.10.16-windows-x86_64-none\\python.exe` failed with exit status exit code: 1\n\n[stderr]\nFatal Python error: _Py_HashRandomization_Init: failed to get random numbers to initialize Python\nPython runtime state: preinitialized\n"
```
2025-03-04 20:05:45 +01:00
Ruben Fiszel
9abaf6cec3 local codebase in home 2025-03-04 19:30:13 +01:00
Ruben Fiszel
b9c56f50b9 fix codebase & bundles on EE 2025-03-04 18:53:06 +01:00
Ruben Fiszel
9095ca7b5b add standalone bundle support on CE 2025-03-04 18:09:12 +01:00
Ruben Fiszel
3546d23f88 all 2025-03-04 15:25:45 +01:00
pyranota
2e7910c0d3 nit: print more information in frontend if something goes wrong with uv (#5411)
* nit: print to frontend logs if there is fallback to system python

* print to logs if fallback
2025-03-04 00:36:11 +01:00
HugoCasa
46c784574a feat: app s3 input anonymous delete (#5401)
* feat: app s3 input anonymous delete

* tmp ee ref

* fix build
2025-02-27 16:28:56 +01:00
HugoCasa
7bf9e25ede feat: track workspace runnables used in flows (#5369)
* feat: track workspace runnables used in flows

* track script hash

* weird

* do it with lock

* Revert "feat: add support for | None and Optional in python (#5361)"

This reverts commit 9736355d5f.

* Revert "Revert "feat: add support for | None and Optional in python (#5361)""

This reverts commit bb8f709894.

* update openapi

* delete old in lock_modules + don't track hub scripts
2025-02-27 10:01:12 +01:00
HugoCasa
cb559d6083 fix: disable bundling using env var (#5396) 2025-02-26 21:41:49 +01:00
pyranota
25f9369d5e fix dedicated worker multipython (#5338)
* fix dedicated worker multipython

* fix typo

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* fix dedublicate logic

* fix ellipsis complaints

* more factorization

* pass slice instead of ref to vector

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-02-25 17:22:03 +01:00
Ruben Fiszel
c8a959691c feat: limit the number of times a job can be restarted (3) after loss of pings 2025-02-25 16:35:29 +01:00
Ruben Fiszel
d45c1f69d4 fix: handle better optional chaining operator 2025-02-24 20:08:13 +01:00
Ruben Fiszel
8806870b1b fix: add LOCALAPPDATA env variable to python execution on windows 2025-02-24 18:36:26 +01:00
Ruben Fiszel
b8e6d0da79 feat: make flow lock deployment error visible in UI 2025-02-22 15:02:16 +01:00
Ruben Fiszel
20547493e0 add PRINT_NATIVE_ERRORS env variable 2025-02-22 11:06:42 +01:00
Ruben Fiszel
c6dbd239b4 update deno_core implementation to latest 2.2.1 2025-02-22 11:04:54 +01:00
Ruben Fiszel
9ea6776ec1 initialize an uv venv globally but skip it for python executor 2025-02-22 01:53:14 +01:00
Ruben Fiszel
e92a46b088 fix: init_script do not need to use nsjail even in nsjail mode 2025-02-22 01:04:33 +01:00
Ruben Fiszel
911fcf2012 improve error message to run executable 2025-02-21 18:09:25 +01:00
Ruben Fiszel
6b31a6dee5 fix ansible for nsjail 2025-02-21 17:06:15 +01:00
pyranota
5740679627 fix windows build (#5336) 2025-02-20 09:16:43 +01:00