Commit Graph

578 Commits

Author SHA1 Message Date
Alexander Petric
aa37f643e7 feat: git sync improvements (#6182)
* init checkpoint

* ui second pass...

* round 1 backend + saving settings + detecting changes...

* checkpoint

* fix openapi

* saving + correct wmill.yaml diff

* cli refactor

* cli and tests refactor done

* cli multi workspace support

* cli support skip core types to align with ui

* new test framework

* sqlx

* openapi spec

* frontend

* sync + settings changes

* some fixes

* some fixes

* security: Remove hardcoded EE license key, use environment variable only

- Remove hardcoded license key from containerized test backend
- Environment variable EE_LICENSE_KEY now required for EE features
- License key no longer stored in database during tests

* sqlx

* tests

* fixing tests

* fix tests

* checkpoint

* checkpoint

* cli build

* frontend - cli exchange

* settings match

* ee repo ref

* npm check

* openapi

* tests

* checkpoint

* cli + tests

* reset to preview on changes

* merge issue ee

* cleanup

* hubscript

* simplifications

* ee repo ref

* cli fixes

* fix sync and add tests

* extra test

* git sync settings / key change aware

* ee-repo ref

* ee-repo ref

* ee repo ref

* ee ref

* review 1

* ee ref

* Update frontend/src/lib/components/PullGitRepoPopover.svelte

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update frontend/src/routes/(root)/(logged)/workspace_settings/+page.svelte

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* ee ref

* remove extra includes from ui

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-07-15 16:25:31 +00:00
HugoCasa
328ef605ad feat: triggers error handler and retry (#6138)
* feat: http triggers error handling and retries

* feat: handle sync http triggers

* feat: triggers retry and error handling

* feat: frontend and preprocessor

* fix build

* minimum allow dead code

* check workspace tags in schedules

* fix flow preprocesor + hub urls

* update ee ref

* nit

* nit

* final nits

* schedule UI nits

* migrate workspace settings to svelte 5
2025-07-11 18:55:23 +02:00
Diego Imbert
433341b295 feat: assets as a primary concept (#6125)
* assets migration

* parse assets (duckdb)

* iterate on assets

* S3 object Preview

* remove pagination

* filterText

* better occurence list

* tweak

* assets in JobPreview

* clone impl

* AssetsDetectedBadge

* improve DbManagerButton + asset dropdown button

* edit resource btn

* warning when incorrect resource

* +Resource in DuckDB

* +S3 Object editor bar

* nit fix rename

* flow asset badge

* More Generic OnChange

* Highlight assets used in modules

* Show occurence count in flow

* Better UX, avoid moving parts

* nit

* Asset nodes

* move to dedicated Asset ctx

* fix layoutNodes not handling first assetsMap

* explore asset btn in flow asset node

* correct offset

* single computeAssetNodes function

* Fix y positioning of nodes with assets

* resource editor

* write mode node (ui)

* accessType in ctx + fix insert button positioning

* right positioning when mixing read and write nodes

* right positioning when mixing R and W assets

* Better layout fix algorithm

* listAssetsByUsage and asset nodes on transitive usages

* refactor + remove linkAssets

* Refactor to allow for custom R/W modes

* AssetsDropdownButton in flow script editor

* R/W/RW selection and changes node pos in flow

* layoutNodes doesnt need recompute now

* fix wrong assumption that nodes recompute when assets change

* r/w/rw multi toggle

* MultiToggle cool animation + clearable

* rename + 1px nit

* remove mini toggle button group, use ToggleButtonGroup

* Combinator parser that detects R / W asset context

* nit fix missing flex-1

* missing order by

* better ui indication for access type

* special x offset case when only one asset node for clarity

* parse getResource in TS with swc ecma parser

* support load and write s3 detection in TS

* Python asset parser

* support wmill api calls without special $res: or s3:// syntax

* detect out of context asset uris python

* do not use access type override when not ambiguous in flow graph

* parse_assets match case in rust

* AsRef<str> refactor

* From impl

* Save flow assets

* Save script asset usages + fixes + save fallback access types

* asset sub icon

* max total asset node width to avoid overlap

* small refactor

* don't parse comments in duckdb assets

* fix assets clearing on parse error

* fix script asset save in wrong place

* load initial asset fallback access types

* support variables

* ui fixes

* Support S3Object as URI in TS client

* support new syntax in python client

* Support +S3Object in EditorBar for TS and python

* Reduce resource requests in assets page

* import windmill client when necessary

* update s3Types.d.ts

* nit fix

* Show input resources and s3 objects as assets

* improve asset icons

* DarkModeObserver refactor

* asset page tabs

* Moved resource variables and s3object pages to assets tabs

* fetch resource usages

* Get variables usages

* move assets usage dropdown to component

* Revert "move assets usage dropdown to component"

This reverts commit 622ea4ab12.

* Revert "Get variables usages"

This reverts commit b11ced4e29.

* Revert "fetch resource usages"

This reverts commit aa5187ad4b.

* Revert "Moved resource variables and s3object pages to assets tabs"

This reverts commit 4430487be4.

* Revert "asset page tabs"

This reverts commit dacc2f0da5.

* move assets usage dropdown to component

* asset icon in asset pages

* tooltip

* details

* Storage selector in S3 File Picker

* make edge less opaque

* Refactor computeAssetNodes to separate in and out nodes

* AssetsOverflowedNode

* nits

* fix assets not being parsed in flows sometimes

* show asset kind and resource_type

* ui nits

* support res:// in duckdb

* add banner for old deployments

* Fix permissionning

* fix broken disable /enable all

* assets page view permission for operators

* Disable ExploreAssetButton for operators

* asset kind as subtitle

* do not spam getResource in assets page. prob. revert fail

* update assets page on workspace change

* reload storage names on ws change

* delete assets on archive / deletion

* sqlx prepare

* missing update when updating user

* add indexes on asset

* better message

* missing loadInit: false

* dead code

* use transaction

* typo

* update package.json

* update package.json

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-07-11 09:56:53 +00:00
wendrul
835645643e feat: Better tracing for audit logs, including a graph to visualize them (#6078)
* Migrate audit log page to svelte 5

* Add email and span cols to audit table

* Add token_prefixs to audit logs (into AuditAuthorable trait)

* Add audit logs graph (wip)

* Add audit span on push and jwt

* Unify same job audit into the same audit span

* Improve the graph visually

* Fix typo

* functioning graph with svelte issue

* Fix leak

* feat: migrate AuditLogsTable from DataTable to VirtualList for performance

- Replace DataTable component with VirtualList for handling thousands of rows
- Migrate to Svelte 5 runes ($props, $bindable, $derived, $state)
- Implement flattenLogs() for virtual scrolling with grouped date headers
- Add sticky indices and dynamic height calculation
- Update parent component to use callback prop pattern instead of events
- Preserve all existing functionality: filtering, selection, pagination
- Follows RunsTable.svelte implementation pattern

Resolves performance issues when displaying large audit log datasets.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

* Fix remaining virtual list issues

* WIP graph

* Fix chart styling

* Fix npm check

* Fix missing audit_span arguments

* Update sqlx

* use varchar 255 for email as in other tables

* Remove syntax inconsistency

* Match struct with ee crate

* Update ee-repo-ref.txt

* Update worker_flow.rs

* Remove redefinition of trait to prevent shadowing

* Re add trait on oss but only when no `private` flag

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-07-03 21:02:45 +00:00
claude[bot]
13716c7df3 chore: update croner dependency to version 2.2.0 (#6119)
Update croner dependency from 2.0.6 to 2.2.0 as requested in #6118.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-07-03 09:08:25 +00:00
windmill-internal-app[bot]
c51e128920 fix: add support for GCS object storage (#6083)
* AI: Updates to files (run 15923369101)

* all

* ee ref

* unneeded

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: HugoCasa <hugo@casademont.ch>
2025-06-30 20:11:57 +00:00
Ruben Fiszel
d15b889fb4 warn after long list_jobs duration 2025-06-28 19:02:34 +02:00
Ruben Fiszel
b3b6c53430 remove WM_OIDC_JWT from context vars 2025-06-20 12:12:38 +02:00
dieriba
aba8c01d7f feat: windmill http triggers and webhooks to openapi spec (#5918)
* add sum and description to http routes

* add to openapi spec

* add subsection

* add collaspable and action button

* added path rendering,

* migrate logic to backend

* remover server and handle different format generation

* add filter for http route and document generated

* remove print

* handled webhook and integrated server component

* done

* add download and copy

* nits

* update .sqlx

* remove .vscode

* fix npm check

* add summary description and fix toggle

* added security handling and nits

* update .sqlx

* nits and rename key

* remove unused code

* update ref

* nits rename var

* nits

* nits

* nits

* nits

* nits

* add token generation for cURL command

* create token

* update label and remove section

* update repo ref

* clean

* let brower start download before cleaning up resource

* handle sync/async for webhook

* format fix

* nits

* nits

* reset sum and description
2025-06-17 19:49:00 +02:00
Ruben Fiszel
ed6d018253 warn for raw scripts 2025-06-16 15:16:56 +02:00
Ruben Fiszel
126b610561 introduce QUIET_LOGS 2025-06-13 11:25:27 +02:00
Alexander Petric
1551dc8af2 fix: fix regex that extract workspaces from custom tags (#5876) 2025-06-05 18:43:59 +02:00
dieriba
f2526571a3 feat: ssh repl like direct to workers hosts machine (#5809)
* feat(backend): add repl for worker

* feat(frontend): add repl to interact with worker and missing packages

* nits

* Update backend/windmill-worker/src/result_processor.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-worker/src/result_processor.rs

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>

* Update frontend/src/lib/components/WorkerRepl.svelte

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>

* fix typo

* fix cd

* nits

* refactoring

* feat: handle other agent worker to also access repl feature, fix bugs

* update repo ref

* nits and match function args

* nits

* typo

* remove struct AgentWorkerData

* update repo ref

* nits

* impl new for authed client and revert to async

* update ref

* updage ee repo ref

* fix missing method/imports small bugs

* update ee repo ref

* update .sqlx

* test

* clean wait for interactive_shell future

* free call stack

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
2025-06-03 18:41:14 +02:00
Diego Imbert
0e316239dd EE Refactor (#5844)
* app compiles with every ee substituted

* Replace all oss files content

* Revert "Replace all oss files content"

This reverts commit ea4017d59f.

* delete all ee

* hide all _ee files under private flag

* hide every oss stuff when private flag set

* pub use *

* gitignore and substitute script

* pub mod for ee needed for ee repo

* small mistakes

* remove oidc_oss impl

* ee ref (temp)

* ee ref

* fix --all-features selecting private in OSS CI

* ee repo ref

* allow unused
2025-06-02 22:12:33 +02:00
Diego Imbert
fdefd4be93 feat: duckdb sql lang support (#5761) 2025-05-27 15:52:57 +02:00
Ruben Fiszel
5b96bccedd feat: add aws oidc support for instance s3 storage (#5810)
* backend

* iterate

* all

* all

* all

* iterate

* revert

* all

* add tracing to get of authed client

* all

* all

* lal

* all

* update

* fix

* push

* all

* all

* revert

* frontend

* fix checks

* avoid deadlock

* safer

* fix

* fix
2025-05-25 14:03:38 +02:00
pyranota
26222539e6 feat(python): inline script metadata (PEP 723) (#5712)
* make resolver

* more updates

* fix build

* fix raw_dependencies job type

* compat with http agent workers

* refactor

* rename

* more refactor

* cleanup

* more tests

* fix s3

* small fixes

* more fixing

* fix endpoint

* nit: update comment

* update ee ref

* update ee ref

* update ee ref

* implement safer `list_available_python_versions`

* add tracing to get of authed client

* internal: Trigger claude when commenting with /aider (#5783)

* add claude instructions files

* call claude too when using aider

* fix

* add draft for linear claude integration

* fix build

* update ee ref

* ignore versions <=3.9

* fix windows build

* correct versions filter

* fix windows build (this time for real)

* inject error to debug CI

* update CI

* undo debug of CI

* fix tests

* remove outdated comment

* update ee repo ref

* Update ee-repo-ref.txt

* Update backend/parsers/windmill-parser-py-imports/src/lib.rs

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>

* Update InstanceSetting.svelte

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: centdix <40307056+centdix@users.noreply.github.com>
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
2025-05-24 02:13:47 +02:00
HugoCasa
30edcdfe0e fix: workspace preprocessor improvements (#5784)
* add tracing to get of authed client

* internal: Trigger claude when commenting with /aider (#5783)

* add claude instructions files

* call claude too when using aider

* fix

* add draft for linear claude integration

* fix: workspace preprocessor fixes

* tmp ee ref

* fix build

* update ee ref

* fix: hub script preprocessor handling

* fix build

* good ref

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: centdix <40307056+centdix@users.noreply.github.com>
2025-05-21 17:49:21 +02:00
Ruben Fiszel
ba4c89e7db nit 2025-05-20 12:55:13 +02:00
Ruben Fiszel
d223b0b12e add log on s3 loading 2025-05-20 10:24:55 +02:00
pyranota
07c2ff5668 feat(python): add annotation to skip result post-processing (#5769)
* feat(python): add annotation to skip result post-processing

Typically windmill will replace all NaN, Infinity and -Infinity from resulting string.
We do it because JSON specification does not support these types as well as DB.

However it will substitute also the cases when any of those words are used within the string.
E.g. script returning "To Infinity and Beyond" will be postprocessed to "To null and Beyond".
Current behaviour is done for the performance sake and now can be disabled with `#skip_result_postprocessing` annotation.

* add comments

* remove extra comments
2025-05-19 22:31:46 +02:00
Ruben Fiszel
5b26498e09 add more duration warnings to logs 2025-05-14 10:13:14 +02:00
Diego Imbert
c7886ea07a feat: sql jobs outputting to s3 + streaming for high-number of rows (#5704)
* stream to s3 boilerplate

* S3 works with new syntax

* snowflake s3 streaming support

* postgres s3 support

* fix postgres stream format

* mysql s3 streaming

* mssql s3 streaming

* new s3 mode syntax

* optional folder param

* rename folder to prefix

* json_stream_arr_values

* cargo toml rollback

* convert_ndjson with datafusion

* format conversion kinda works

* Fixed not finishing the datafusion writer

* support for pg and mssql

* fix file ext

* bigquery conversion and works with s3 streaming

* fix s3 flag parser

* snowflake s3 streaming support

* factor out duplicate code

* remove anyhow

* Err case for parse s3 mode

* Send error to mpsc

* bigquery s3 streaming fix for huge queries

* remove extra stuff

* snowflake s3 streaming support

* small regex mistake

* cfg(not(feature = "parquet"))

* fix CI (unused import)

* error handling fix (graphite)
2025-05-13 10:19:44 +02:00
dieriba
54b4815df2 nits: use generic deserializer for empty data (#5727)
* use generic deserializer for empty data

* update repo ref

* fix import
2025-05-12 11:00:55 +02:00
Ruben Fiszel
a3c76fb10c fix: improve error display of nativets exceptions 2025-05-11 11:35:31 +02:00
Ruben Fiszel
7c69959853 fix: improve agents workers handling of WHITELIST_ENVS 2025-05-11 08:38:46 +02:00
HugoCasa
254c3cf8ef feat: preprocessor refactor (#5629)
* feat: preprocessor refactor

* nit

* oneOf kind support + new capture format + fix build

* fix build

* fix build

* update preprocessor templates

* cache runnable version info

* update ee ref

* update hub scripts && ee ref
2025-05-09 09:50:01 +02:00
dieriba
242a565428 feat: raise error if end early in flow (#5653)
* update front

* handle err_msg display if enabled when stop condition is met

* nits

* Update backend/windmill-worker/src/worker_flow.rs

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>

* Update openflow.openapi.yaml

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update worker_flow.rs

* Update worker_flow.rs

* update flow test

* update condition

* remove println

* nits

* fix flow test

* nits and missing property on struct definition

* nits

* fix unsaved state

* update .sqlx

* cleanup sqlx

---------

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-05-07 18:23:40 +02:00
Ruben Fiszel
afc9cf5940 update deps versions (#5703) 2025-05-05 16:19:39 +02:00
pyranota
4fd0561239 feat: critical alert if disk near full (#5549)
* feat: critical alert if disk near full

* update logic to cover edge-case

* update logic

* windows support

* Update README.md

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* make it periodically

* add extra safety

* respect killpills

* do not check every 3 secs

* move to monitor.rs

* rework

* alter readme

* extending functionality

* fix worker_mode

* fix compilation

* fix typo

* make use of AI suggestion

* update ee-repo-ref

* update ee ref

* logs are CE

* update ee repo ref

* remove systemstat from worker crate

* fix comp error + sync cargo.lock

* more comptime fixing

* Update ee-repo-ref.txt

* fix compilation error

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-05-05 08:54:48 +02:00
centdix
ec701a9ee7 feat: Integrate MCP with hub (#5685)
* draft

* working draft

* tool to list scripts

* working sse server

* add proxy + parse query params

* working mcp that runs script

* remove useless dependencies

* log context

* update crate

* mcp no proxy

* use custom sdk

* integrate in windmill api

* draft

* put rmcp sdk here

* do not use mcp as crate

* use extensions for get scripts

* remove unused crate

* list actual scripts

* fix

* give schema in list tools

* cleaning + take workspace id from context

* implement calling the script with tool

* cleaning + fix ctrl-c

* make post path a param

* cleaning

* better name for tools

* fix error with tool name

* cleaning

* draft cleaning

* more cleaning

* list script based on settings

* fix query

* add params in openapi

* use rmcp fork from git

* remove files

* cleaning + fix query

* remove settings and use favorite by default

* add flows in tools list

* handle running flows

* remove frontend for mcp settings

* handle resource args

* send list of resource in shema

* handle mcp url setting with token scope

* cleaning

* avoid calling list tools in call tool

* apply scope to flows

* cleaning

* cleaning

* cleaning

* cleaning

* format files

* fix typos

* remove log

* add back missing dispatch

* fix transform for resource-obj + put every resource in description

* transform obj to string

* cleaner code

* better frontend

* cleaner code

* cleaner logic

* add parentheses just in case

* add func to fetch hub scripts

* fix typos

* working fetch and run hub script

* also fetch flows from hub

* improvments

* merge create tool logic

* add integrations in description

* cleaning

* cleaning

* small fix

* get schema for flow

* filter tools fetch by token scope

* remove hub flows

* remove prints

* add hub script integration choice

* higher limits

* cleaning

* fix merge

* better naming for hub scripts

* no workspace for hub

* alow multiple app in one token

* plural

* fix

* cleaning

* add documentation

* fix bad code

* use id directly

* cleaner bindings

* fix disabled condition

* add cancel button + reset apps if not hub token

* reset mcp apps

* combine all/favorites + hub

* small fix
2025-05-01 19:31:56 +02:00
Ruben Fiszel
60018aadf6 fix: improve agent workers for deployed scripts 2025-04-29 19:35:52 +02:00
centdix
1625524311 feat: Add MCP only mode (#5661)
* add mcp mode

* fix flag usage

* change mcp path

* do not migrate db in mcp mode

* handle null values
2025-04-25 14:22:45 +02:00
wendrul
fdd1642ce1 feat: Ansible improvements (vault, roles and git repos) (#5655)
* Ansible vault + roles

* Clone arbitrary repos

* Fix cloning logic after merge

* Make function for cloning without history any commit

* Cloning repos and lockfile on the commit

* Improve error messages

* Create lockfile for roles and collections

* Simplify ansible ssh identity interface

* Ansible vault password: pass just a variable instead of 2 step approach

* Lock lockfiles for roles and collections

* fix typo

* Change git ssh identity section name

* Rename variable

* Update init script for ansible

* Suppress error when no roles

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-04-24 16:04:28 +02:00
Ruben Fiszel
3c68fef2ab ui code builder v0 (at secret path) (#4964) 2025-04-20 01:19:02 +02:00
dieriba
5b123b01a1 feat: handle different aws auth resource type (#5637)
* feat: oidc auth with sqs

* rafctor

* update: add id token struct

* update has_expired function

* fix duration

* fix typo

* Update backend/windmill-common/src/auth.rs

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>

* Update backend/windmill-common/src/auth.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* update ref

* update sqlx

* update resource type

* update repo ref

* Update settings.json

* add new resource type

* update repo ref and clear resource path on resource type change

* update .sqlx

* add missing key

---------

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-04-17 21:51:18 +02:00
dieriba
34b307b2be feat: oidc support for sqs trigger (#5614)
* feat: oidc auth with sqs

* rafctor

* update: add id token struct

* update has_expired function

* fix duration

* fix typo

* Update backend/windmill-common/src/auth.rs

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>

* Update backend/windmill-common/src/auth.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* update ref

* update sqlx

* update resource type

* update repo ref

* Update settings.json

---------

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-04-17 15:36:17 +02:00
Ruben Fiszel
fd47cd6016 improve perf of setting api roles 2025-04-12 14:47:48 +00:00
Ruben Fiszel
12ba15c928 improve benchmarks 2025-04-12 11:09:21 +00:00
HugoCasa
b9e879618b feat: signed s3 objects (#5593)
* feat: accept signed s3 objects for s3 file keys in apps + sign endpoint and helpers

* Update backend/windmill-api/openapi.yaml

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* fix build

* Update python-client/wmill/wmill/client.py

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>

* nti

* fix build

* fix build

* presigned

* Update frontend/src/lib/components/apps/editor/settingsPanel/InputsSpecEditor.svelte

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update frontend/src/lib/components/apps/editor/settingsPanel/InputsSpecEditor.svelte

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* fix build

* b

* fix sqlx

* nit

* typo

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
2025-04-11 18:03:40 +02:00
dieriba
6339775404 feat: add gcp trigger (#5501)
* update: add migration for gcp_trigger table, add cli method gcp and add gcp_trigger args for github ci

* update: add gcp module and routing it in lib.rs

* update: added gcp type to TRIGGER KIND type in db, add new feature condition to try_get_fn and added gcp_trigger feature in cargo file

* update: add ui for gcp trigger, added missing triggers type for job kin, add gcp trigger in deploy to functionallity

* adding google cloud crate

* update: handle pull and push delivery

* update: handle pull and push delivery

* update: handle push event, changed front ui

* update: ui

* fix: capture for gcp

* update: automatically manage pub sub subscription and refactoring

* capture done

* update cli types

* fix: wrong func argument and type openapi

* fix: missing import

* update .sqlx

* update: svg color and remove pulse button for capture push gcp

* update: handle deployto

* update script helper and link to hub gcp script/flow

* update gcp representation

* update: add confirmation modal

* add unique index to mitigate same subscriber id, fix `zombie worker`, update test connection logic

* Update frontend/src/lib/components/triggers/gcp/GcpTriggerEditorInner.svelte

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update cli/gen/services.gen.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* nits

* update repo ref

* fix feature function

* update dependencies

* update .sqlx

* fix missing import

* update: handle existing subscription and creating new one or update it

* fix ee build

* handle existing config properly

* update .sqlx

* update .sqlx

* remove unused

* update documentation link fix ci

* use on instance of empty_string_as_none fn

* update refo ref

* updat script_helper

* update ui

* update migration

* add missing arguments

* fix and nits

* update repo ref

* remove unused

* nits

* add doc links

* nits

* Update frontend/src/lib/components/triggers/gcp/GcpTriggerPanel.svelte

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update frontend/src/lib/components/triggers/gcp/GcpTriggerEditorInner.svelte

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* use trigger path as default route path

* update .sqlx

* update repo ref

* update ref

* update route path

* update types

* fix deploy to

* nits

* update repo ref

* update .sqlx

* Update frontend/src/lib/components/triggers/gcp/GcpTriggerEditorConfigSection.svelte

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* nits

* nits

* nits

* remove dupliacte fn and update repo ref

* remove unused import

* fix missing import

* fix: wong name var

* update script helper and template script

* Update frontend/src/lib/script_helpers.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update ee-repo-ref.txt

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: HugoCasa <hugo@casademont.ch>
2025-04-10 20:39:01 +00:00
Ruben Fiszel
63fa499015 feat: agent workers v2 using http (#5588) 2025-04-10 00:56:11 +00:00
Ruben Fiszel
1bb402bcfc fix benchmarks 2025-04-03 21:17:13 +00:00
HugoCasa
2c5e58cf1a feat: make azure a standalone AI provider (#5558)
* feat: make azure a standalone AI provider

* oups

* nit

* fix: openai/azure oauth

* nit

* nits
2025-04-03 19:26:56 +00:00
pyranota
f3ed108e43 nit: replace KJQXZ with more meaningful notation (#5547)
* nit: replace `KJQXZ` with more meaningful notation

Originally this string is located in all places, where modification is needed in order to add new language support

* relative -> related

* revert shebang in substitue.sh

* remove '}'
2025-04-02 21:03:04 +00:00
pyranota
59740c0478 feat: add java support (#5458)
* feat: add nu (nushell) support

* add worker tests

* deactivate tables and non-any types below top-level

full support will come in V1
for V0 it's better to keep things minimal and simple

* add syntax highlighting

used python's grammar, since nushell isn't supported by monaco nor svelte-highlights

for V1 nu will get it`s own grammar

* add logo

* partially implement plugin support

* change logo + ability to deploy + nsjail draft

* static variables + get_resource + get_variable

* lsp/dev.nu + initial nu lsp (not working yet)

* make it work with nsjail

* nullguard

* Much more flexible signature parsing and better error-messages

* add init script

* rename nulsp to nu

* install nu to dockerfile

* fix merge

* implement Default for MainArgSignature

* stage NU_CACHE_DIR

* improve dockerfiles

* dev.nu for parser-wasm + flake.nix

* update code for windows

* add nushell to flake

* upload Cargo.lock

* make build.sh work on nixos

* build wasm cli parsers

* add docs to README_DEV.md

* add helper script docker/dev.nu

* improve docker/dev.nu

* fix windows

* commit frontend/package(lock).json

* update cargo.lock

* correctly update cargo.lock

* remove lsp

* update flake.nix to include svelte server and nushell

* Revert base.sql to main

* remove PLUGIN_USE_RE

* make CARGO_PATH private

* add nu to cli

* Change flags to build wasm-nu-parser

* remove flake.nix from parser-wasm

* update wasm-build target

* remove unused import

* add cli support for nu

* update github workflows

* wasm-build 0.17 -> 0.19

* update build script

* update cargo.lock

* Fix typographical error

* start working on java

* do java boilerplate

* implement parser for java

* update Cargo.lock

* update ENV_SETTINGS

* use published nu parser

* update package.lock

* java is S3 + Caching enabled

* install nsjail backup

* commit v0

* fix nsjail

* v0.1

* rewrite parser in tree-sitter

* implement parser from scratch

* polishing

* change init script to match new parser

* fix imports

* fix cli build

* fix cli build

* refactor install phase

* implement .valid.windmill atomic verification

* implement java init functionality

* remove quick-xml

* fix windows not recognizing 'mvn'

* create empty settings.xml if there is no config provided

* clean up

* change default settings.xml

* change classpath format for windows

* docs to helper

* java copy bin cache instead of symlink

* remove comments

* merge

* fix package.json

* fix package.json 2

* minor fixing

* migrate to Coursier

* update misc

* Http(s) Proxy + CA certs

* remove unused .wasm

* make requirements insensitive to spaces

* update handle_child refs

* rework save_cache for directories

* fix s3 bug

* compile .wasm for cli

* remove uuid import

* fix compilation

* use reference

* fix zero-dep failure

* removing unsafe stuff

* remove unneeded imports

* revert: we still need winapi

* remove nix store from nsjail

* do not create cache_nomount

* add java to dnt

* remove duplicated dependency in init script

* fix typos

* fix CI

* use published parser
2025-03-27 15:12:55 +01:00
dieriba
4c6f60010f feat: add description option to schedule page (#5500)
* add description option to schedule page

* add description property to schedule trigger type

* fix ci

* update: use textare for description and add down migration

* perf: use jsonb instead of json, and use query_as macro sqlx instead of function variant

* update .sqlx

* nits

* fix: missing comma in description
2025-03-27 09:47:34 +00:00
dieriba
e920101107 feat: api key/basic/hmac auth for http triggers (#5476)
* feat: add webhook configuration in front and start handling logic in back

* feat: add generic implementation

* update: add stripe provider working on auth

* merge main

* handling new webhook, add generic implem

* feat: add auth method and api key and handling custom authentication

* nits

* added generic implem for webhook provider

* feat: added resource webhook_auth to handle http route auth, refactored backend code

* update sqlx

* fix: zoom and twitch challenge request event, call verify function

* removing webhook name to make it more general, properly added custom auth option alongside windmill_auth option

* update .sqlx

* update deply to for http routes

* update .sqlx

* add different auth method for http routes

* add custom script method and custom signature fixed bug with webhook auth

* update auth options

* update .sqlx

* add new auth method

* update .sqlx

* refacto function and fix frontend bug

* fix typo, update .sqlx and handle pototential error

* nits

* nits

* add script template flow

* Update frontend/src/lib/components/triggers/http/utils.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* nits

* update: remove wrong trigger type and use right icon for sqs trigger

* nits

* update: .sqlx

* update: reunite flow and script logic for script placeholder value replacement

* Update backend/windmill-api/src/http_trigger_auth.rs

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>

* delete vite config

* remove darks mode observer

* update: add new option to capture panel for http route

* nits

* revert

* nits

* typo

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
Co-authored-by: HugoCasa <hugo@casademont.ch>
2025-03-27 08:43:42 +00:00
Ruben Fiszel
5573d88695 perf: cache workspace premium check 2025-03-25 02:00:51 +01:00
Ruben Fiszel
3edca4bc91 perf: optimize number of queries needed for job run (#5504)
* optPerf

* update sqlx

* update sqlx
2025-03-24 20:12:26 +01:00