Commit Graph

1640 Commits

Author SHA1 Message Date
Lucas Abel
390ee3113b fix: invalid null comparison while inserting flow node (#4815) 2024-11-28 17:29:16 +01:00
Lucas Abel
77937a494a fix: invalid jsonb comparison while inserting flow node (#4814) 2024-11-28 14:43:54 +01:00
pyranota
a853c8d6a5 Remove flock (#4809) 2024-11-28 12:42:25 +01:00
Ruben Fiszel
c998102cf3 simplify logic to check for license key on workers 2024-11-27 18:51:06 +01:00
Lucas Abel
2911775d0a feat(backend): store flow inner modules into flow_node table (#4778) 2024-11-27 16:08:31 +01:00
pyranota
34609a7d8c Fix tar prefix (#4781)
* Fix tar prefix

* Use TAR_PIP(PYTHON_311)_CACHE_DIR instead

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2024-11-27 15:54:47 +01:00
Lucas Abel
d86ea30221 enable is_simple optimization for FlowScript (#4799)
* backend: remove some allocations in `worker_flow.rs`

* feat(backend): enable `is_simple` optimization for `FlowScript`

* fix: fix node retrieval for `flowscript` inner step
2024-11-27 10:06:19 +01:00
Lucas Abel
150ca33f0d fix: disable flow_version_lite in dedicated workers (#4800) 2024-11-27 10:02:44 +01:00
pyranota
e876ae4f42 fix(python): Merge to site-packages packages with same name (#4793)
* (python) Initially implement merge

Iterate overall all python paths and if same folder has same name multiple times,
then merge the content and put to <job_dir>/site-packages

Solves problem with imports for some dependencies.

Default layout (/windmill/cache/):

dep==x.y.z
└── X
   └── A
dep-ext==x.y.z
└── X
   └── B

In this case python would be confused with finding B module.

This function will convert it to (/<job_id>):

site-packages
└── X
   ├── A
   └── B

This way python has no problems with finding correct module

* Better error-handling

* Optimize and fix bug

* Update backend/windmill-worker/src/python_executor.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-worker/src/python_executor.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Polish

* Add path to PathBuf

* Add & before PathBuf

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2024-11-26 18:24:26 +01:00
Lucas Abel
e4784e89da feat(backend): implement flow scripts (#4748)
* feat(backend): implement flow scripts

* conditionally resolve reduced flow value

* conditionally use flow scripts based on the min worker version

* grant all to `flow_{version_lite,node}`

* do not silently discard errors

* env: add flow script feature flag

* flow script on cloud automatically

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-11-25 23:20:51 +01:00
Ruben Fiszel
dd5081592f email quotas only apply to top-level job 2024-11-25 16:20:55 +01:00
Ruben Fiszel
6768e5bbbd fix: retry on inserting completed job (#4784)
* fix: retry on pushing next scheduled job of schedule

* all

* improve error handling

* update sqlx
2024-11-25 01:33:44 +01:00
Ruben Fiszel
89456f502f remove rsmq 2024-11-24 11:04:54 +01:00
Ruben Fiszel
495d4487bc fix: expose DISABLE_DENO_LOCK 2024-11-23 08:39:57 +01:00
Lucas Abel
e6c7fe7027 backend: rework raw values fetching logic (#4770) 2024-11-21 13:59:42 +01:00
Lucas Abel
97457a5679 feat(backend): move some static fields out of job tables (#4689)
* feat(backend): move some static fields out of job tables

* backend: re-use `flow_value` if any and avoid clone

* update minimal version to 1.427.0
2024-11-19 11:27:55 +01:00
wendrul
5e10782779 feat: Add mode (permissions) option to files in ansible (#4724)
* Add chmod option to files in ansible

* Add 0 prefix if string

* Rename chmod to mode
2024-11-18 17:02:38 +01:00
HugoCasa
17d893315b fix: revert bool to text support in pg (#4727) 2024-11-16 10:01:16 +01:00
HugoCasa
d999e91583 unbreak nullable inputs in pg (#4726) 2024-11-15 22:27:28 +01:00
HugoCasa
9ad1afc8f2 unbreak ::text for json in pg (#4722)
* unbreak ::text for json in pg

* better error handling
2024-11-15 18:35:21 +01:00
Ruben Fiszel
b868e446fc improve error messages on windows 2024-11-15 09:43:54 +01:00
Ruben Fiszel
029462bc57 nits logs 2024-11-15 01:15:51 +01:00
Ruben Fiszel
089826e5b5 delete venv folder if pip install didn't succeed 2024-11-15 01:11:17 +01:00
pyranota
f240d1322a feat: Handle pip install by uv (#4517)
* feat: Handle `pip install` by `uv`

Dirty and untested, but already something working

* Integrate with NSJAIL and prepare fallbacks

* Refactor fallback
no_uv disable compile and install
where no_uv_install and no_uv_compile are a bit more specific

* Remove `--disable-pip-version-check`
Reason:
   warning: pip's `--disable-pip-version-check` has no effect

* Fix backend compilation error

* Pip fallback overwrite UV's cache

* Initially refactor cache (No S3)

* Support S3

* Remove unused import

* Handle flags for NSJAIL

* Return deleted flag

* Update Dockerfile

* Update docker-image.yml

* Update docker-image.yml

* Add --link-mode=copy and remove -v

* Fix NSJAIL INDEX_URL

* Fix flock and windows

* Update python_executor.rs

* Remove line from Dockerfile

We dont need it and to trigger build

* fixing for windows

* Dont pin python to specific version

* Change TMP for windows

* Revert docker-image.yml

* Disable UV for ansible

Will be enabled later.
Needs proper testing and its better to split onto 2 PRs with first modifying python and second ansible

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Alexander Petric <petric.al@gmail.com>
2024-11-15 00:12:56 +01:00
Lucas Abel
4fdca87de9 nit: cleanup raw_flow usage (#4707)
* nit: cleanup `raw_flow` usage

* nit: refactor two queries into one
2024-11-14 12:23:27 +01:00
Lucas Abel
2d572695ef nit: generalize usage of has_failure_module (#4706) 2024-11-14 08:49:12 +01:00
Ruben Fiszel
84bd06e906 fix: fix intempestive expr type change in flow transform 2024-11-14 00:24:02 +01:00
Ruben Fiszel
4b8f3580a4 add LOGS_TO_STDOUT to ee 2024-11-13 23:30:52 +01:00
Ruben Fiszel
ce01fa1677 split more ee log logic 2024-11-13 18:49:41 +01:00
Ruben Fiszel
50861fccc0 fix: support multiple pip-extra-index-url with commas 2024-11-12 23:31:00 +01:00
Ruben Fiszel
8c6f5a320b propagate exit code in error result when relevant 2024-11-08 16:37:58 +01:00
Ruben Fiszel
8bc9a021a8 fix(bash): correctly propagate exit errors 2024-11-08 16:33:01 +01:00
HugoCasa
dc8bd6d2b5 feat: http custom routes for static assets (#4666)
* feat: http custom routes for static assets

* update ee ref + fix build

* fix build

* nit
2024-11-07 20:16:14 +01:00
Ruben Fiszel
771d740701 fix: clarify error messages when job timeout or cancelled with more details 2024-11-06 19:18:34 +01:00
Ruben Fiszel
7c0d9901e0 fix main 2024-11-06 16:34:09 +01:00
HugoCasa
7578cebaf9 fix: display logs in native mode when script fails (#4655)
* fix: display logs in native mode when script fails

* oups

* oups
2024-11-06 16:26:23 +01:00
Alexander Petric
693b7a4fd4 feat(oauth): add snowflake oauth support (#4622)
* feat(oauth): add snowflake oauth support

* fixes

* fix keypair auth

* avoid loop when changing settings

* including account id doc link in the settings ui

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2024-11-04 23:35:27 +01:00
Guilhem
22ab51e991 feat(frontend): improve and simplify scheduled poll flows (#4560) 2024-11-04 23:18:09 +01:00
Ruben Fiszel
e11cc13c92 fix: apply NO_PROXY and HTTP_PROXY, HTTPS_PROXY more consistently 2024-11-02 13:57:30 +01:00
Ruben Fiszel
05ac6ba094 fix(prometheus): fix incorrect worker_busy set to 1 2024-11-01 16:31:32 +01:00
Ruben Fiszel
557b949efe fix(s3): align s3 handler additional creds providers 2024-11-01 00:51:28 +01:00
pyranota
ef37928865 Add rollback for multipython (#4610) 2024-10-31 14:27:34 +01:00
HugoCasa
921e3217c6 improve array params handling in postgres (#4605) 2024-10-30 14:44:42 +01:00
Ruben Fiszel
692b0568fc fix(bun): handle bun lockfile created with windows (#4602) 2024-10-29 14:09:38 +01:00
Ruben Fiszel
71f84dee08 handle better empty tag overrides 2024-10-29 09:01:02 +01:00
Ruben Fiszel
99400ebbf3 improve test this step for flow tag overide 2024-10-29 08:19:17 +01:00
Ruben Fiszel
ab38215c86 fix(backend): in flows, workspace scripts should use their set tags instead of the default one 2024-10-29 07:29:39 +01:00
Ruben Fiszel
85de44cb93 feat: autoscaling v0 (#4593)
* all

* all

all

all

all

all

all

all

all

all

* all

* all

* nits

* nits
2024-10-28 00:47:19 +01:00
Ruben Fiszel
2cbf027653 fix: do not update created_at of scripts on lockfile generation 2024-10-25 17:44:41 +02:00
wendrul
058d48ccab Make file_resources that are filled with a variable in ansible (#4584)
* Make file_resources that are filled with a variable in ansible

* Rename file_resources section to files

* Add variable example to init code
2024-10-25 14:18:28 +02:00