Commit Graph

19 Commits

Author SHA1 Message Date
Ruben Fiszel
4829f447ed fix: add checkpoint.json mount to python nsjail config for WAC v2 (#8421)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 08:17:08 +00:00
hugocasa
65a92d9899 fix: set nsjail time_limit from job timeout so configured defaults are respected (#8389)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-16 12:06:31 +00:00
Pyra
95df7b9a6a feat: otel REST tracing (#7571) 2026-01-20 13:38:02 +00:00
Ruben Fiszel
e7e27259ee nsjail skip_setsid for pg propagation + faster exit 2025-07-13 11:48:01 +00:00
pyranota
c60f8dacf1 feat(python): add custom_wheels directory to PYTHONPATH (#5169)
* fix(python): fix uv can't find ssl certificates

- Add `PY_NATIVE_CERT` flag, forces UV to use native tls
- Rename `PIP_INDEX_CERT` to `PY_INDEX_CERT`
- Rename `PIP_TRUSTED_HOST` to `PY_TRUSTED_HOST`

For backwards compatibility PIP* variables are still accessible

* feat(python): add `custom_wheels` directory to PYTHONPATH

Add global directory by path `<CACHE_DIR>/python_xyz/custom_wheels`

For example for scripts running python 3.11, in every execution
`<CACHE_DIR>/python_311/custom_wheels` will be accessible and all wheels placed there could be imported and used.

This is usefull for preinstalling wheels before runtime

* Make it work with Nsjail

* Rework and make custom_wheels optional

* Remove `create_dir_all` from imports

* Use sync version of metadata

* Rename `custom_wheels` to `global-site-packages`
2025-01-29 19:13:47 +01:00
pyranota
e47dd697f9 feat(python): Multiple runtime versions (#4579)
* feat: Handle `pip install` by `uv`

Dirty and untested, but already something working

* Integrate with NSJAIL and prepare fallbacks

* Refactor fallback
no_uv disable compile and install
where no_uv_install and no_uv_compile are a bit more specific

* Remove `--disable-pip-version-check`
Reason:
   warning: pip's `--disable-pip-version-check` has no effect

* Fix backend compilation error

* Pip fallback overwrite UV's cache

* Initially refactor cache (No S3)

* Support S3

* Remove unused import

* Handle flags for NSJAIL

* Return deleted flag

* Remove verbose mode and enable link-mode=copy

* Granural migration of lockfiles

Before i realized we dont need it :)

* Initial draft (not-working)

* Add fallback

* Fix bug preventing uv from installing deps

'\n' - Love it

* Add verbosity indicator

* Iterate on feature
- Added instance python version
- Rework logic

* Fix EE build error
error[E0599]: no method named `iter` found for tuple `(PyVersion, std::vec::Vec<std::string::String>)` in the current scope

* Support S3

* Support NSJAIL

* Refactor `get_python`

* Make NSJAIL work [Unsafe]

config file missed /proc mount causing install phase to fail

* Trigger CI

* Clean up

* Make Actions build it

* Trigger CI #2

* Update Dockerfile and clean up

* Change fallbacks
now there is only no_uv and NOUV

* Expose INSTANCE_PYTHON_VERSION through env variable

* Change namings

* Include py-version to requirements.in

Also add comments and make code much cleaner

* Use const for python installation dir

It was hardcoded before

* Pin preinstalled version

* Update python_executor.rs

* Up to date branch

* Create PYCACHE dirs

TODO: PY_TAR_DIRS

* Fix after merge

* Make it safer

* Implement USE_SYSTEM_PYTHON

* Implement latest_stable option

* Load INSTANCE_PYTHON_VERSION on startup

* Check for multiple annotations used

* Fix Latest Stable button not pressed if selected

* Proper error handling for conflict on multiple annotations

* Fix merge conflicts

* Preinstall 3.11 and Latest Stable

* Preinstall latest stable in non-blocking manner

* Fix Warning

* Gate preinstall logic behind "python" feature

* Handle raw_deps properly

* Make it work with nsjail

* Revert docker-image.yml

* Revert Dockerfile

* Cleanup + Fixing

* Add windows support

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-01-24 11:42:48 +01:00
Ruben Fiszel
b603f4a6ba allow python multiprocessing on nsjail 2024-10-11 01:40:09 +02:00
HugoCasa
6fb936cdb6 fix: allow preprocessor to write to args.json on nsjail (#4455) 2024-09-27 21:12:38 +02:00
Ruben Fiszel
afed03565f make nsjsail mode compatible with arm64 by making lib64 non mandatory 2024-06-28 15:11:41 +02:00
Ruben Fiszel
ea92fa1154 remove nltk_data binding 2023-08-03 11:30:36 +02:00
Ruben Fiszel
904d74187e feat(backend): allow relative imports for python 2023-03-28 20:27:40 +02:00
Ruben Fiszel
5250532043 relax nsjail limits 2023-01-19 14:51:43 +01:00
Ruben Fiszel
13300be1ac relax nsjail limits 2023-01-19 14:35:06 +01:00
Ruben Fiszel
3ad5bfc192 feat: add relative imports for python scripts (#1075)
* all

* all
2023-01-09 13:49:30 +01:00
Ruben Fiszel
65a535eb86 add urandom and all /etc to nsjail 2022-11-25 19:05:41 +01:00
Ruben Fiszel
dbe1ada3a9 nits on nsjail config 2022-11-23 14:00:49 +01:00
Ruben Fiszel
0a8fd771b0 feat: supercache extended to all version 2022-11-18 20:17:23 +01:00
Ruben Fiszel
61c915488c ci: trim down docker main docker image 2022-11-07 13:37:37 +01:00
Kai Jellinghaus
6263e90735 restructure the entire backend layout using workspaces (#815) 2022-10-29 11:58:06 +02:00