Commit Graph

1656 Commits

Author SHA1 Message Date
pyranota
b54c9ee657 feat: Replace pip-compile with uv (#4460)
* Update `shell.nix`

- Replace pip-compile with uv packages
- Pin rust version
- Add var to trigger windmill print more info in stdout

* Replace `pip-compile` with `uv` (dirty + untested)

* Fix arguments passed to uv

Some of the flags are included by default in UV and can be safely removed:
- --resolver=backtracking
- --no-emit-index-url

Also uv does not support `--pip-args` and suggests to directly pass args to uv.

* Remove extra `dbg!`

* Replace 'pip-compile' with 'uv' in Dockerfile

* Add fallback option to `pip-compile` (Disabled)

* Add `uv` to `docker/DockerfileSlim*`

* Add `get_annotation_python` and rename `get_annotation` to `get_annotation_ts`

* Add option to fallback to pip-compile

Put `# no_uv` on top the file for specific python script

Or set `USE_PIP_COMPILE` variable to `true`

* Put back `pip-tools` into shell.nix

* Make sure lockfile resolves again if `#no_uv` used

Add #no_uv to the end of requirements (requirements.in)
That way if something breaks for customer, then they put #no_uv and new lockfile will be resolved

* Put `pip install pip-tools` in original spot

* Fix compilation error

* Fix EE compilation error

error[E0658]: attributes on expressions are experimental
   --> windmill-worker/src/python_executor.rs:144:5
    |
144 |     #[cfg(feature = "enterprise")]
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: see issue #15701 <https://github.com/rust-lang/rust/issues/15701> for more information

* Add `no_cache` annotation

Will force recalculation of lockfile
And block uv from using cached values

* Target uv cache to /tmp/windmill/cache

* Prohibit uv from managing python

* Add uv to DockerfileBackendTests

* Pin uv version to 0.4.18 in Dockerfiles

* Dont put `#no_uv` in requirements.in

Instead postfix hash for requirements.in with `-no_uv`

* Push Warning to logs if fallbacked to pip-compile

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2024-10-03 16:00:35 +02:00
Alexander Petric
f5c4727274 feat(worker): support workers to run natively on windows (#4446)
* minimal code change to get windmill worker on windows for bun and python + rustfmt

* adding support for powershell

* compiling error on unix

* rust linting comments

* comments hugo: PSModulePath

* comments ruben, refactor to simplify

* adding build workflow

* editing workflow

* editing workflow

* editing workflow

* editing workflow

* editing workflow

* skip migration env, ee fixes

* improvements powershell

* testing windows runner

* testing windows runner

* testing windows runner

* testing windows runner

* testing windows runner

* install postgres on runner

* install postgres on runner

* install postgres on runner

* install postgres on runner

* install postgres on runner

* install postgres on runner

* install postgres on runner

* install postgres on runner

* killing process tree in windows

* sqlx_offline

* install openssl for github windows runner

* used pre-installed openssl

* used pre-installed openssl

* build ee

* build ee

* build ee

* build ee

* adding commented out steps for artifact publishing

* build on tag matchinv v* pattern

* ren instead of mv on Windows

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* fix merging issue

* gate imports for windows

* fixing default cargo home path...

* fixing default cargo home path...

* comments ruben

* make pwsh default modules loading more robust on unix (#4448)

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: HugoCasa <hugo@casademont.ch>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2024-10-03 15:59:12 +02:00
Ruben Fiszel
1d8e361612 chore(main): release 1.404.1 (#4474)
* chore(main): release 1.404.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-10-03 15:13:39 +02:00
Ruben Fiszel
b69bbceedb chore(main): release 1.404.0 (#4468)
* chore(main): release 1.404.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-10-03 14:53:39 +02:00
Ruben Fiszel
0476f98231 chore(main): release 1.403.1 (#4467)
* chore(main): release 1.403.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-10-01 16:07:57 +02:00
Ruben Fiszel
73ab8e1653 fix: fix new instance db setup 2024-10-01 16:03:58 +02:00
Ruben Fiszel
85130dc791 chore(main): release 1.403.0 (#4459)
* chore(main): release 1.403.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-10-01 12:38:44 +02:00
HugoCasa
0df169e3f9 feat: flow step skipping (#4461) 2024-09-30 16:30:59 +02:00
Ruben Fiszel
630ae5d425 fix: skip one migration to avoid using md5 for azure support 2024-09-30 14:02:31 +02:00
HugoCasa
c600a62d28 send stats when renewing key if last >24h (#4430)
* feat: send stats when renewing key if last >24h

* nits

* fix: sqlx

* nit

* renewal reason

* stats reason

* update ee ref

* Update ee-repo-ref.txt

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2024-09-30 12:23:07 +02:00
Ruben Fiszel
8764849580 chore(main): release 1.402.3 (#4458)
* chore(main): release 1.402.3

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-09-30 09:55:50 +02:00
Ruben Fiszel
c3bf207d2f chore(main): release 1.402.1 (#4456)
* chore(main): release 1.402.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-09-28 16:10:20 +02:00
Ruben Fiszel
cfaea38e92 improve benchmarking tools (#4450) 2024-09-27 10:48:04 +02:00
Ruben Fiszel
3fac66a386 chore(main): release 1.402.0 (#4447)
* chore(main): release 1.402.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-09-26 17:04:02 +02:00
Ruben Fiszel
bca4099d1b add suspended flows to cli queues 2024-09-26 16:52:09 +02:00
Ruben Fiszel
8e0eb3dd15 improve list queue api 2024-09-26 14:31:17 +02:00
Ruben Fiszel
9f91b1995a feat(cli): add queues, workers and worker-groups commands (#4439)
* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all
2024-09-26 11:27:37 +02:00
Ruben Fiszel
c2ae4fc799 chore(main): release 1.401.0 (#4444)
* chore(main): release 1.401.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-09-25 18:14:05 +02:00
Ruben Fiszel
9909b8989e chore(main): release 1.400.0 (#4438)
* chore(main): release 1.400.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-09-25 16:26:07 +02:00
wendrul
398a09b741 feat: add static variable and resources support to ansible (#4435)
* Add static variable and resources to ansible

* Add wasm parser to cli

* Fix pulling of resources, rs and playbooks

* Remove unused import, add publish pkg-yaml

* Update package.json

* Update generate-metadata

* Fix null fields being serialized in script metadata

* npm install
2024-09-25 15:05:53 +02:00
Ruben Fiszel
fbd0071a54 chore(main): release 1.399.0 (#4436)
* chore(main): release 1.399.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-09-25 14:42:36 +02:00
HugoCasa
590321fd3c feat: add tag filtering to external JWT authentication (#4425)
* feat: tag filtering jwt ext auth

* move tags to scopes

* fix symlink

* update ee ref
2024-09-25 11:21:19 +02:00
pyranota
ad69c08228 fix get_job_update returning error (#4427)
When get_progress specified to true, but there is no record of progress for given job
Endpoint will fail and return error.

Nothing cricial but should be fixed.
2024-09-24 10:24:26 +02:00
Ruben Fiszel
99c24109de chore(main): release 1.398.1 (#4424)
* chore(main): release 1.398.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-09-23 18:04:41 +02:00
Ruben Fiszel
5c6c0ff3cc chore(main): release 1.398.0 (#4412)
* chore(main): release 1.398.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-09-23 17:28:18 +02:00
wendrul
a500994cc2 fix: various improvements for ansible (#4419)
* Improve error messages, fix dependency job issue

* Add nsjail support for ansible

* Remove invalid comment syntax

* Change nsjail config and ansible controller config to be ephemeral

* Move ansible cfg to later to overwrite possible files named ansible.cfg

* Fix ansible.cfg env var

* Fix tmp dir issue
2024-09-23 16:44:37 +02:00
Ruben Fiszel
9c506a83cb critical channels build on non ee 2024-09-21 22:54:15 +02:00
Ruben Fiszel
50a6f789fa fix: migrate smtp instance settings to global settings (#4416)
* all

* all

* all

* fix

* update ee private

* all
2024-09-21 01:42:15 +02:00
HugoCasa
70fa78c10d fix: allow no body in job requests (#4413) 2024-09-20 18:26:50 +02:00
HugoCasa
304dac3447 feat: http routing (#4339)
* feat: http routing

* all

* feat: improve UI

* final stuff

* fix: sqlx

* fix: nit

* fix: nits

* fix: error handler display

* fix: routes panel perms

* all

* fix: improve ability to paste from macos in vscode extension

* fix lock-write for deno

* all

* cleaning

* fix

* cli preprocessor

* nits

* nits

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-09-20 15:38:38 +02:00
Ruben Fiszel
815f34b2af chore(main): release 1.397.4 (#4410)
* chore(main): release 1.397.4

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-09-20 14:53:16 +02:00
Ruben Fiszel
9e9562e1a4 chore(main): release 1.397.3 (#4409)
* chore(main): release 1.397.3

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-09-20 11:38:40 +02:00
Ruben Fiszel
aec6972e29 chore(main): release 1.397.2 (#4408)
* chore(main): release 1.397.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-09-20 09:52:35 +02:00
Ruben Fiszel
a20ee6096a chore(main): release 1.397.1 (#4406)
* chore(main): release 1.397.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-09-20 09:17:15 +02:00
Ruben Fiszel
63f70f67d7 chore(main): release 1.397.0 (#4405)
* chore(main): release 1.397.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-09-19 01:40:02 +02:00
wendrul
6855b8da9a feat: ansible playbook support (#4399)
* Ansible execution and parsing

* Working collections and pip dependencies

* Remove unused vars

* File resources logic for ansible

* Make frontend for ansible and resource file

* Format file

* Change naming of file resource and inventory fields

* Add autocomplete for file extension resource creation

* Add endpoint to list file formats for resource types

* Add CLI functionality to pull/push file resources

* Add beta tag to ansible

* Add Full image dockerfiles containing ansible

* Prepare sqlx

* Update cargo.lock

* Update ansible path

* Remove unused imports

* Add back import removed by rust-analyzer

* Improve ansible init code

* Prepare sqlx

* Change dockerfile to make the full windmill image

* Remove old dockerfile

* Improve autocomplete file resource extension select

* Remove comment

* Validate file extension

* Add icons to text file resources

* Remove editability of file resource types

* Remove unused import

* Missing space

* Add yaml parser
2024-09-19 00:21:15 +02:00
Ruben Fiszel
5973e427c3 chore(main): release 1.396.1 (#4402)
* chore(main): release 1.396.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-09-18 14:36:40 +02:00
Ruben Fiszel
1f2d203f11 chore(main): release 1.396.0 (#4401)
* chore(main): release 1.396.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-09-18 11:27:13 +02:00
pyranota
d6d4756b7a feat: Allow setProgress and getProgress from within the script (#4400)
* Allow setting progress explicitly from script body.

This feature exposes:
 * `getProgress`
 * `setProgress`
 * `incProgress`

API in TypeScript client (python is coming soon).

NOTE: Progress cannot be out of range 0..100 and cannot decrease.

With exposed APIs there is also UI changes, so progress can be shown for individual jobs as well.

For optimization reasons, jobs start to ask for progress only after N-seconds of execution.

* feat: Add `shell.nix`

If you dont have anything but nix, dont worry, run nix-shell in root, or activate with direnv and get all needed dependencies

NOTE: You will still need docker

* feat: Add `dev.nu` to typescript client

Little helper function, allowing developer to work on ts client easier.

To use:
  `./dev.nu watch`

Now add import of windmill in body of your script and `//nobundle` on top of the file

Edit ts client in your favourite editor and hit save. Script will do the rest.

* Cleanup files

* Fix: Failed to deserialize query string: missing field `get_progress`

* perf: Implement non-naive polling mechanism for getting job progress

* Add independant delay for getProgress

Problem in `TestJobLoader`:

There should be 2 delays:

    One until we find our first progress (every 5s)
    Once we found our first progress, we can do it every second

* nit: Use `query_scalar!` instead of `query_as`

* Fix: Sql error, no rows returned by a query that expected to return at least one row

* refactor: Remove global CSS for JobProgressBar

* Change UI for progress of flow subjobs

* Replace `Step 1` with `Running` in ProgressBar for individual jobs

* Remove `incProgress`

incProgress is not very usefull and error-prone

* perf: Set metric only for jobs that are actually using it

(https://github.com/windmill-labs/windmill/pull/4373#discussion_r1759843773)

* Offload registering progress from clients to server

* Add `jobId?` argument to typescript-client's `setProgress` and `getProgress`

Allows to set progress of other jobs and flows,
if jobId specified, than flow id will be inferred automatically.

Could be used by SDK.

* Add `Error::MetricNotFound` for better error handling

* Fix: Make `JobProgressBar` display in red when failed

* Add persistant progress bar

Now you can reload the page after job is done and progress will be still there

* Allow succeeded individual job's progress bar stick to 100%

* Add python support 

* nit: Remove usage of undefined variable in python-client

* Add `async` in ts client (for error handling)

* nit(frontend): Remove unused import

* Dont load JobProgressBar when it is not needed

* nit: npm check fix

* cargo sqlx prepare

* fix sqlx

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-09-18 01:16:19 +02:00
Ruben Fiszel
a24365237c rm legacy hubScripts 2024-09-17 17:33:52 +02:00
Ruben Fiszel
c661ac8696 chore(main): release 1.395.0 (#4394)
* chore(main): release 1.395.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-09-17 10:41:26 +02:00
Ruben Fiszel
d57b139df2 fix: cannot create duplicate apps and raw_apps 2024-09-16 21:15:29 +02:00
HugoCasa
de78f6c192 feat: failed key renewal alert + renew on start if no recent renewal (#4387)
* feat: failed key renewal alert + renew on start if no recent renewal

* Update ee-repo-ref.txt
2024-09-16 16:54:35 +02:00
Ruben Fiszel
692c1dcb93 chore(main): release 1.394.6 (#4393)
* chore(main): release 1.394.6

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-09-15 21:07:50 +02:00
Ruben Fiszel
396258f637 fix: improve first time setup experience 2024-09-15 21:02:36 +02:00
Ruben Fiszel
2e80107e03 chore(main): release 1.394.5 (#4385)
* chore(main): release 1.394.5

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-09-13 23:39:12 +02:00
HugoCasa
af85d4936d fix: queue metrics graph (#4388) 2024-09-13 23:34:48 +02:00
Ruben Fiszel
51cf420272 fix: parquet renderer display number of rows" (#4389)
* all

* all

* all

* nocheck
2024-09-13 23:34:36 +02:00
Ruben Fiszel
e81c8384e4 chore(main): release 1.394.4 (#4375)
* chore(main): release 1.394.4

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-09-13 08:52:34 +02:00
HugoCasa
ff08b5a9b5 fix: tighten inputs for granular kinds (#4379)
* fix(backend): acls perms

* nit
2024-09-12 14:40:22 +02:00