* s3 proxy works with get (no auth yet)
* nit
* support s3:// syntax
* Support s3:// syntax and fix vite api proxy normalizing double slashes in URI
* s3 checks authed
* nit
* PUT works
* delete file works
* Derive the JWT signature from the backend
* Authorize s3 correctly (JWT signature is never sent in cleartext)
* convert object store error to wmill error for correct status code
* stash
* fix
* POST first request proxy works
* s3 put for duckdb
* factor out direct proxy code
* Fix Issue with backend proxy and wrong signature due to Host header mismatch
* Add _default_ syntax to solve URI normalization issues with signing
* restricted to user paths toggle
* user path restriction works !
* change restriction to allow
* fix
* factor out code
* better permissions UX in object storage settings
* Revert to restrict_to_user_paths
* check permissions in old s3 api
* DuckDB now uses S3 Proxy and no longer needs LFS query
* implement todo
* fix hardcoded w_id
* s3 proxy size limit
* s3_proxy is ee
* nit
* add Google Cloud Storage as option to secondary storage
* GCS secret in duckdb
* fix toolchain compile
* Remove user permissions for v0
* fix ci 2
* fix CI OSS
* fix missing feature flag
* fix unused warning
* integration test fails bc rustc 1.85.0
* ee ref
* fix ci ...
* update ee ref
* Make schema validation struct
Schema Validation rules that are constructed from the schema or from the
MainArgSig(TODO).
* Make other validator builder
* Fail dependency job like with lockfile failing for schema validator
* Add last types + tests
* Remove unused dependency
* fix typos
* Migration ID was colliding with another, changed it manually
* Add Oneof + other fixes
* fix: cache for querying scripts correclty handles ScriptMetadata
* Add cache for schema validation from main arg sig
* Prepare sqlx
* Remove default features
* Feature flags
* Fix down migration table name
* cleanup: put validation logic inside a function
* Refactor to cache the should_validate boolean
Changed the schemavalidators cache to take in an
Option<SchemaValidator>, effectively storing the `should_validate_schema` information.
Also pass the schema when avaialble to construct the schema validator
* Add other job kinds to u8 cache key just in case
* Only cache if not preview
* feat: Handle `pip install` by `uv`
Dirty and untested, but already something working
* Integrate with NSJAIL and prepare fallbacks
* Refactor fallback
no_uv disable compile and install
where no_uv_install and no_uv_compile are a bit more specific
* Remove `--disable-pip-version-check`
Reason:
warning: pip's `--disable-pip-version-check` has no effect
* Fix backend compilation error
* Pip fallback overwrite UV's cache
* Initially refactor cache (No S3)
* Support S3
* Remove unused import
* Handle flags for NSJAIL
* Return deleted flag
* Remove verbose mode and enable link-mode=copy
* Granural migration of lockfiles
Before i realized we dont need it :)
* Initial draft (not-working)
* Add fallback
* Fix bug preventing uv from installing deps
'\n' - Love it
* Add verbosity indicator
* Iterate on feature
- Added instance python version
- Rework logic
* Fix EE build error
error[E0599]: no method named `iter` found for tuple `(PyVersion, std::vec::Vec<std::string::String>)` in the current scope
* Support S3
* Support NSJAIL
* Refactor `get_python`
* Make NSJAIL work [Unsafe]
config file missed /proc mount causing install phase to fail
* Trigger CI
* Clean up
* Make Actions build it
* Trigger CI #2
* Update Dockerfile and clean up
* Change fallbacks
now there is only no_uv and NOUV
* Expose INSTANCE_PYTHON_VERSION through env variable
* Change namings
* Include py-version to requirements.in
Also add comments and make code much cleaner
* Use const for python installation dir
It was hardcoded before
* Pin preinstalled version
* Update python_executor.rs
* Up to date branch
* Create PYCACHE dirs
TODO: PY_TAR_DIRS
* Fix after merge
* Make it safer
* Implement USE_SYSTEM_PYTHON
* Implement latest_stable option
* Load INSTANCE_PYTHON_VERSION on startup
* Check for multiple annotations used
* Fix Latest Stable button not pressed if selected
* Proper error handling for conflict on multiple annotations
* Fix merge conflicts
* Preinstall 3.11 and Latest Stable
* Preinstall latest stable in non-blocking manner
* Fix Warning
* Gate preinstall logic behind "python" feature
* Handle raw_deps properly
* Make it work with nsjail
* Revert docker-image.yml
* Revert Dockerfile
* Cleanup + Fixing
* Add windows support
---------
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
* Allow setting progress explicitly from script body.
This feature exposes:
* `getProgress`
* `setProgress`
* `incProgress`
API in TypeScript client (python is coming soon).
NOTE: Progress cannot be out of range 0..100 and cannot decrease.
With exposed APIs there is also UI changes, so progress can be shown for individual jobs as well.
For optimization reasons, jobs start to ask for progress only after N-seconds of execution.
* feat: Add `shell.nix`
If you dont have anything but nix, dont worry, run nix-shell in root, or activate with direnv and get all needed dependencies
NOTE: You will still need docker
* feat: Add `dev.nu` to typescript client
Little helper function, allowing developer to work on ts client easier.
To use:
`./dev.nu watch`
Now add import of windmill in body of your script and `//nobundle` on top of the file
Edit ts client in your favourite editor and hit save. Script will do the rest.
* Cleanup files
* Fix: Failed to deserialize query string: missing field `get_progress`
* perf: Implement non-naive polling mechanism for getting job progress
* Add independant delay for getProgress
Problem in `TestJobLoader`:
There should be 2 delays:
One until we find our first progress (every 5s)
Once we found our first progress, we can do it every second
* nit: Use `query_scalar!` instead of `query_as`
* Fix: Sql error, no rows returned by a query that expected to return at least one row
* refactor: Remove global CSS for JobProgressBar
* Change UI for progress of flow subjobs
* Replace `Step 1` with `Running` in ProgressBar for individual jobs
* Remove `incProgress`
incProgress is not very usefull and error-prone
* perf: Set metric only for jobs that are actually using it
(https://github.com/windmill-labs/windmill/pull/4373#discussion_r1759843773)
* Offload registering progress from clients to server
* Add `jobId?` argument to typescript-client's `setProgress` and `getProgress`
Allows to set progress of other jobs and flows,
if jobId specified, than flow id will be inferred automatically.
Could be used by SDK.
* Add `Error::MetricNotFound` for better error handling
* Fix: Make `JobProgressBar` display in red when failed
* Add persistant progress bar
Now you can reload the page after job is done and progress will be still there
* Allow succeeded individual job's progress bar stick to 100%
* Add python support
* nit: Remove usage of undefined variable in python-client
* Add `async` in ts client (for error handling)
* nit(frontend): Remove unused import
* Dont load JobProgressBar when it is not needed
* nit: npm check fix
* cargo sqlx prepare
* fix sqlx
---------
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
* fix: can cancel only if can disable schedule + stop worker on force cancel + cancel job parents
* fix: sqlx build
* fix: flow reschedule check + soft cancel only direct parent
* fix: sqlx prepare
* fix: only soft cancel parent if force cancel and job is a flow step
* backend auth user validation
* Approval page suggesting login
* BE impl to support groups
* Add FE for setting suspend on flow step
* handle admin as a specific case
* restrict feature to entperprise only
* group names as static input transforms
* Suspend settings with tabs
* Update enterprise edition check
* cleanup
---------
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
