all

Bumps [aws-config](https://github.com/smithy-lang/smithy-rs) from 0.57.2 to 1.0.0.
- [Release notes](https://github.com/smithy-lang/smithy-rs/releases)
- [Changelog](https://github.com/smithy-lang/smithy-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/smithy-lang/smithy-rs/commits)

---
updated-dependencies:
- dependency-name: aws-config
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Guillaume Bouvignies <guillaume@windmill.dev>

CHANGELOG.md

@@ -1,6 +1,273 @@
 # Changelog
 
 
+## [1.216.0](https://github.com/windmill-labs/windmill/compare/v1.215.0...v1.216.0) (2023-11-26)
+
+
+### Features
+
+* add early return for flows ([dbaef0a](https://github.com/windmill-labs/windmill/commit/dbaef0aa5f22c55b691f6205ae053c155a0e025c))
+* fill pg resource from string ([#2699](https://github.com/windmill-labs/windmill/issues/2699)) ([f1cabb4](https://github.com/windmill-labs/windmill/commit/f1cabb40f6cbec05b105563dad58da048d9187ec))
+* **frontend:** add currency format + add enum autocomplete + fix run… ([#2670](https://github.com/windmill-labs/windmill/issues/2670)) ([0c0f43d](https://github.com/windmill-labs/windmill/commit/0c0f43dd3ac0541d889046c0c6df8dbf78650b9c))
+* **frontend:** add stat card ([#2687](https://github.com/windmill-labs/windmill/issues/2687)) ([81ef24b](https://github.com/windmill-labs/windmill/commit/81ef24b3cec18f86e407b343b4d72ac566d268e1))
+* **python:** Update return type for 'get_resource' function ([#2695](https://github.com/windmill-labs/windmill/issues/2695)) ([603e7ff](https://github.com/windmill-labs/windmill/commit/603e7ff67f5f68d291db37173b89325d5de911a1))
+
+
+### Bug Fixes
+
+* expand enum type narrowing to forms ([94e9b80](https://github.com/windmill-labs/windmill/commit/94e9b80e1993a217415118bf48b8575cd8363746))
+* **frontend:** correctly handle when result is undefined ([#2693](https://github.com/windmill-labs/windmill/issues/2693)) ([f4aa768](https://github.com/windmill-labs/windmill/commit/f4aa76897ea0f261c1e47976f517058601c479f9))
+* improve multiselect from form ([5fa653d](https://github.com/windmill-labs/windmill/commit/5fa653d154c6e8697119796a025bdc380f68de9a))
+* improve resource pages ([16be230](https://github.com/windmill-labs/windmill/commit/16be2300ff66cad8b740856a6435d1c1e53bce99))
+* lighten monaco editor workers ([#2690](https://github.com/windmill-labs/windmill/issues/2690)) ([4c42836](https://github.com/windmill-labs/windmill/commit/4c42836cfcad63da023a658213f03b45c925efb2))
+* Priority tags FE buggy when missing from config ([#2702](https://github.com/windmill-labs/windmill/issues/2702)) ([e8b1f22](https://github.com/windmill-labs/windmill/commit/e8b1f220dd299922a32b04451756f2b7fa735d5d))
+* S3 bucket browser small improvements and fixes ([#2700](https://github.com/windmill-labs/windmill/issues/2700)) ([55e34d8](https://github.com/windmill-labs/windmill/commit/55e34d8cdd64362615f2c1bc2698c0538c287784))
+
+## [1.215.0](https://github.com/windmill-labs/windmill/compare/v1.214.1...v1.215.0) (2023-11-23)
+
+
+### Features
+
+* query embeddings from s3 ([#2683](https://github.com/windmill-labs/windmill/issues/2683)) ([e3f2b43](https://github.com/windmill-labs/windmill/commit/e3f2b43748e8e0853dd0dbae87cba37b950dc76e))
+
+
+### Bug Fixes
+
+* bigquery schema and date inputs ([#2688](https://github.com/windmill-labs/windmill/issues/2688)) ([f5e098d](https://github.com/windmill-labs/windmill/commit/f5e098d03b467002a8deab4d2c519945c01458d4))
+* improve wmill go client ([cfd3da4](https://github.com/windmill-labs/windmill/commit/cfd3da41efe4e89e5d8672c08433442b05d11f37))
+
+## [1.214.1](https://github.com/windmill-labs/windmill/compare/v1.214.0...v1.214.1) (2023-11-23)
+
+
+### Features
+
+* **python:** Quality of Life improvements to Python client ([#2686](https://github.com/windmill-labs/windmill/issues/2686)) ([1c2abcd](https://github.com/windmill-labs/windmill/commit/1c2abcda231b2d10e7a358a4e4b7973785cb6199))
+
+
+### Bug Fixes
+
+* add image base64 source kinds ([7ae84fc](https://github.com/windmill-labs/windmill/commit/7ae84fce5014d97bf17803e2ccefafee009b28bb))
+* fix backend build ([45ee1d7](https://github.com/windmill-labs/windmill/commit/45ee1d770345f03c03cc48a3ddcc130ca3655a4c))
+* **frontend:** disable active interaction to avoid broken state ([#2675](https://github.com/windmill-labs/windmill/issues/2675)) ([bfdb559](https://github.com/windmill-labs/windmill/commit/bfdb559b47786e37f135ca345c55828cc70e49e9))
+* **frontend:** improve tutorial ux ([#2677](https://github.com/windmill-labs/windmill/issues/2677)) ([7fe3bca](https://github.com/windmill-labs/windmill/commit/7fe3bca624fa3b434e3fbe9b33b213449044accc))
+* **frontend:** use popover for schedule for later on the runs page ([#2678](https://github.com/windmill-labs/windmill/issues/2678)) ([31fbc58](https://github.com/windmill-labs/windmill/commit/31fbc5867cc72befea5733811148126289ce8bb9))
+* graphql variables ([#2682](https://github.com/windmill-labs/windmill/issues/2682)) ([217e694](https://github.com/windmill-labs/windmill/commit/217e69498fa5958e83c595003b2a086528e51388))
+* relax tags constraints ([8f8ea22](https://github.com/windmill-labs/windmill/commit/8f8ea227c850f9d01f913660547b6a6394c6e142))
+* Various fixes and improvements for Windmill S3 capabilities ([#2674](https://github.com/windmill-labs/windmill/issues/2674)) ([c42c54e](https://github.com/windmill-labs/windmill/commit/c42c54e69b5acdcbe44a0c26ae7fbaa2189e6b5f))
+
+## [1.214.0](https://github.com/windmill-labs/windmill/compare/v1.213.0...v1.214.0) (2023-11-22)
+
+
+### Features
+
+* hub path scripts + nested inputs glue ([#2668](https://github.com/windmill-labs/windmill/issues/2668)) ([ad199af](https://github.com/windmill-labs/windmill/commit/ad199afd06814540bb7d36669709902be56eeb8a))
+* **python:** Refactor Windmill Python client (remove `windmill-api`) ([#2665](https://github.com/windmill-labs/windmill/issues/2665)) ([37ffdae](https://github.com/windmill-labs/windmill/commit/37ffdaed60fb750f1466b440353e1d8409eaea90))
+
+
+### Bug Fixes
+
+* fix flow primary schedule clearing ([3ebe6d7](https://github.com/windmill-labs/windmill/commit/3ebe6d7a620e37fd6c81bcf4c8713a48eb081f81))
+
+## [1.213.0](https://github.com/windmill-labs/windmill/compare/v1.212.0...v1.213.0) (2023-11-21)
+
+
+### Features
+
+* code completion UI + other nits ([#2657](https://github.com/windmill-labs/windmill/issues/2657)) ([6d426b4](https://github.com/windmill-labs/windmill/commit/6d426b4ec49d4749a89c86dc5eb1f11bf705ca26))
+* Expanding an s3object result now opens the S3 file browser ([#2656](https://github.com/windmill-labs/windmill/issues/2656)) ([baac93f](https://github.com/windmill-labs/windmill/commit/baac93f40140ee37548a273885c028a8e6500b6d))
+
+
+### Bug Fixes
+
+* ask to return value ([#2659](https://github.com/windmill-labs/windmill/issues/2659)) ([de1e1f5](https://github.com/windmill-labs/windmill/commit/de1e1f545d4cd42f46d9af9a0349af86acf1901c))
+* fix embedded approval step timeouts ([51ce2f8](https://github.com/windmill-labs/windmill/commit/51ce2f8cb308da285dab0dc433bf596caa5eeed0))
+* fix error handling for list of errors ([c1bb97d](https://github.com/windmill-labs/windmill/commit/c1bb97d990810c9b3d909c9a045f0ce84be6c25e))
+* set session code completion to enabled by default ([#2664](https://github.com/windmill-labs/windmill/issues/2664)) ([ded0bb8](https://github.com/windmill-labs/windmill/commit/ded0bb890bb54ef80c857395474e692865ee4717))
+
+## [1.212.0](https://github.com/windmill-labs/windmill/compare/v1.211.0...v1.212.0) (2023-11-20)
+
+
+### Features
+
+* S3 file picker as a drawer ([#2640](https://github.com/windmill-labs/windmill/issues/2640)) ([624b4d0](https://github.com/windmill-labs/windmill/commit/624b4d0e9898dddcce3cb2ce989ce1f9e4736061))
+* test an iteration ([c2598b3](https://github.com/windmill-labs/windmill/commit/c2598b330450f9885f7d10e2b5baa54d6ef88cc5))
+* upgrade to gpt-4-turbo ([#2655](https://github.com/windmill-labs/windmill/issues/2655)) ([8ea98c2](https://github.com/windmill-labs/windmill/commit/8ea98c2c8d636209954a267116eb03ab13217ef8))
+
+## [1.211.0](https://github.com/windmill-labs/windmill/compare/v1.210.1...v1.211.0) (2023-11-20)
+
+
+### Features
+
+* agent mode v0 ([cd260e7](https://github.com/windmill-labs/windmill/commit/cd260e7062802ab39115025577a2456c66435899))
+* agent mode v0 ([47ad8d6](https://github.com/windmill-labs/windmill/commit/47ad8d6013e7144a5d018f43c827b171228b5f42))
+* ai regex ([19907e4](https://github.com/windmill-labs/windmill/commit/19907e4012e88a8fd28a5f0564a7ea48ec12020c))
+* custom error messages for forms ([2f15ebc](https://github.com/windmill-labs/windmill/commit/2f15ebc5f9628b0d26ec08b3527ddc96c6d42ba0))
+* **python:** Add functionality and resiliency to wmill python client ([#2650](https://github.com/windmill-labs/windmill/issues/2650)) ([c7a30f7](https://github.com/windmill-labs/windmill/commit/c7a30f7c9db26252f0ee69e1276ddccc0d52acb3))
+
+
+### Bug Fixes
+
+* add refresh button to item picker ([4882d94](https://github.com/windmill-labs/windmill/commit/4882d94dfe18c156662fe483e1dd4f5d3e3be3af))
+* fix base64 assignment (file input) ([70f1210](https://github.com/windmill-labs/windmill/commit/70f121035edd94b3940f530a5603b1ff4bf03839))
+* main broken tests ([#2652](https://github.com/windmill-labs/windmill/issues/2652)) ([c90f7f1](https://github.com/windmill-labs/windmill/commit/c90f7f167e0b9c20008a62eeeef7819e24fc3da9))
+* token expiry is equal to timeout ([b667317](https://github.com/windmill-labs/windmill/commit/b667317d44f37bb50f24e356a1a3d231ebe8b4b4))
+
+## [1.210.1](https://github.com/windmill-labs/windmill/compare/v1.210.0...v1.210.1) (2023-11-18)
+
+
+### Bug Fixes
+
+* add toggle to invites ([36379d6](https://github.com/windmill-labs/windmill/commit/36379d6db05de170e0237b12e767b6d6f4a6f2ef))
+* auto-invite all instead of by domain ([c07e905](https://github.com/windmill-labs/windmill/commit/c07e9056f1872cae70b8c3bebdbbf47daeee7ac5))
+* flow copilot arg types ([#2648](https://github.com/windmill-labs/windmill/issues/2648)) ([08c14e5](https://github.com/windmill-labs/windmill/commit/08c14e51c792fe65d4f993379eff0e5c8a75215b))
+* improve error message for unauthorized variables/resources ([5a72ca9](https://github.com/windmill-labs/windmill/commit/5a72ca9b24c5c4e9fe94c7865b9145283aceff53))
+* leave workspace + instance api ([ee243de](https://github.com/windmill-labs/windmill/commit/ee243dedc6df28a64f15e0b274b7fa96f6428474))
+* make wmill compatible with python 3.7 ([6ae1a69](https://github.com/windmill-labs/windmill/commit/6ae1a69b75fe039586956064880c274a21fc5970))
+* migrate old state env variable to new env variable ([2692737](https://github.com/windmill-labs/windmill/commit/2692737418ed601c0a36c368f59ffb8d10d9ad38))
+
+## [1.210.0](https://github.com/windmill-labs/windmill/compare/v1.209.0...v1.210.0) (2023-11-17)
+
+
+### Features
+
+* truncate logs when necessary ([05d3fca](https://github.com/windmill-labs/windmill/commit/05d3fcad269adebd2a701da8a49cdc692f26b18f))
+* truncate logs when necessary ([4d92745](https://github.com/windmill-labs/windmill/commit/4d927457867a3c828b5d09f43fc905984769e29f))
+
+
+### Bug Fixes
+
+* remove stats payload logging ([#2644](https://github.com/windmill-labs/windmill/issues/2644)) ([cc978c2](https://github.com/windmill-labs/windmill/commit/cc978c230fe65d4f93b369abff96ee3354d3a899))
+* restore flow local storage ([526b14c](https://github.com/windmill-labs/windmill/commit/526b14cca3ca5eb3fc2c60e404b057da05b84b4c))
+
+## [1.209.0](https://github.com/windmill-labs/windmill/compare/v1.208.0...v1.209.0) (2023-11-17)
+
+
+### Features
+
+* add prompt history ([#2638](https://github.com/windmill-labs/windmill/issues/2638)) ([c2557a2](https://github.com/windmill-labs/windmill/commit/c2557a270f89d4bc055d03833bb90b20161e9de0))
+* collect vcpus and accounts usage ([#2635](https://github.com/windmill-labs/windmill/issues/2635)) ([8183c38](https://github.com/windmill-labs/windmill/commit/8183c38d05da220e036ec36a8ac5cc6f6e004d56))
+* dedicated workers for flows ([#2637](https://github.com/windmill-labs/windmill/issues/2637)) ([b13cc58](https://github.com/windmill-labs/windmill/commit/b13cc58315ea07d67d98bffb907e2cbd0c1d6b7d))
+* **frontend:** add filter by resource on Audit Log page ([#2628](https://github.com/windmill-labs/windmill/issues/2628)) ([1d1258f](https://github.com/windmill-labs/windmill/commit/1d1258ff5873023e66e727a87d6ecb04d1a77e70))
+* **frontend:** Add optional dropdown trigger ([#2642](https://github.com/windmill-labs/windmill/issues/2642)) ([998843b](https://github.com/windmill-labs/windmill/commit/998843ba62a9b154c7a4903bef5b6fc2dc3fafe5))
+
+
+### Bug Fixes
+
+* improve suspended flow handling when missing next step ([316d5f3](https://github.com/windmill-labs/windmill/commit/316d5f344f409ca564f92e32d6a04068a316c012))
+* nested schemas can use required ([33dfcfb](https://github.com/windmill-labs/windmill/commit/33dfcfbdc74d611e457e54cc8ddb9152bf365560))
+* support non root imports in bun ([9ff428f](https://github.com/windmill-labs/windmill/commit/9ff428f210dd1dea6cff664aaade1411317587c5))
+* www-url-encoded body ([9c7bb5e](https://github.com/windmill-labs/windmill/commit/9c7bb5ecd9f169d627d98cc1a494d0e5010f0166))
+* x overflow scroll + add clear history btn ([#2641](https://github.com/windmill-labs/windmill/issues/2641)) ([1aef2aa](https://github.com/windmill-labs/windmill/commit/1aef2aae9027010346ce352e6a6993fb175bc904))
+
+## [1.208.0](https://github.com/windmill-labs/windmill/compare/v1.207.0...v1.208.0) (2023-11-15)
+
+
+### Features
+
+* Add Python SDK capabilities to generate Polars and DuckDB connection settings to S3 ([#2625](https://github.com/windmill-labs/windmill/issues/2625)) ([9009365](https://github.com/windmill-labs/windmill/commit/90093656b49f78d550012ff2e8627fc43b8e26ea))
+* add yaml for import/export/diff ([#2631](https://github.com/windmill-labs/windmill/issues/2631)) ([34abe01](https://github.com/windmill-labs/windmill/commit/34abe01c2f0d9d7055ed17cb2dd34acf9d5e2e88))
+* sync team plans ([#2619](https://github.com/windmill-labs/windmill/issues/2619)) ([f38498b](https://github.com/windmill-labs/windmill/commit/f38498b725cb475d8b4c04641d3fa9870ed23adc))
+
+
+### Bug Fixes
+
+* Resource secret heuristic is wrong ([#2632](https://github.com/windmill-labs/windmill/issues/2632)) ([0d6c1ec](https://github.com/windmill-labs/windmill/commit/0d6c1ec064d83d3394791cef776bcd33ef3f570f))
+* stripe ops ee only ([#2634](https://github.com/windmill-labs/windmill/issues/2634)) ([e3cb2e6](https://github.com/windmill-labs/windmill/commit/e3cb2e66ba41b156e2833ce54b1b153eba20785e))
+
+## [1.207.0](https://github.com/windmill-labs/windmill/compare/v1.206.0...v1.207.0) (2023-11-15)
+
+
+### Features
+
+* cache postgres connection ([#2621](https://github.com/windmill-labs/windmill/issues/2621)) ([ff8b9b0](https://github.com/windmill-labs/windmill/commit/ff8b9b03848bf44303bb3dc7d04572823fef28f8))
+
+
+### Bug Fixes
+
+* ai gen minor fixes ([#2626](https://github.com/windmill-labs/windmill/issues/2626)) ([eafde10](https://github.com/windmill-labs/windmill/commit/eafde1099532caacd23af19fd1405f71e6f7f723))
+* copy component only if empty selection ([9fd8a31](https://github.com/windmill-labs/windmill/commit/9fd8a31e1c3b515022e7967d5195c9b38a1e4cc3))
+* false not undefined in result ([#2620](https://github.com/windmill-labs/windmill/issues/2620)) ([f185eba](https://github.com/windmill-labs/windmill/commit/f185eba49cc84fc0bcfcfd00716999c11212db44))
+* fix delete schedule clear jobs ([244b85e](https://github.com/windmill-labs/windmill/commit/244b85e859e74e8e509c48c22dc9f543f11dd1f2))
+* fix diff editor error ([cbeefa8](https://github.com/windmill-labs/windmill/commit/cbeefa807a25e06dc8052e5c0984fe9c883da460))
+* **frontend:** Fix Step viewer ([#2627](https://github.com/windmill-labs/windmill/issues/2627)) ([1daccdb](https://github.com/windmill-labs/windmill/commit/1daccdbdb104f4d6e619ed0bdf28ef9ecc15bebb))
+
+## [1.206.0](https://github.com/windmill-labs/windmill/compare/v1.205.0...v1.206.0) (2023-11-12)
+
+
+### Features
+
+* Add Authentik SSO support ([#2614](https://github.com/windmill-labs/windmill/issues/2614)) ([ce38a43](https://github.com/windmill-labs/windmill/commit/ce38a4322e13b35660f16d4a3ca66224487dd27b))
+* add dedicated worker support for deno ([528f6fa](https://github.com/windmill-labs/windmill/commit/528f6fa9107d3b93c36fd5418ca95b7f2b701445))
+* add support for requiring users to pre-exist ([eb5067a](https://github.com/windmill-labs/windmill/commit/eb5067aee5fd0c64614c967ef4b2af2398e8807c))
+
+
+### Bug Fixes
+
+* add windmill loading screen ([8151b01](https://github.com/windmill-labs/windmill/commit/8151b01013383a83ce8aca8c2f918ead159b4273))
+* fix selected component switch tab + copyComponent keep layout ([6a819cc](https://github.com/windmill-labs/windmill/commit/6a819cc51211cba7009d985caddb9ae6f0ec8f48))
+* handle console logs in dedicated workers ([fa3efd3](https://github.com/windmill-labs/windmill/commit/fa3efd3f608a754f0c271da557a244ff6b359cfd))
+* keep dark theme on login redirect ([7bbe3cf](https://github.com/windmill-labs/windmill/commit/7bbe3cf288ec480e56cc33a5d3248c00567e137f))
+* update monaco ([#2616](https://github.com/windmill-labs/windmill/issues/2616)) ([f809172](https://github.com/windmill-labs/windmill/commit/f809172e3da88594d7e5a3ed8a12116e804ca6ae))
+
+## [1.205.0](https://github.com/windmill-labs/windmill/compare/v1.204.1...v1.205.0) (2023-11-11)
+
+
+### Features
+
+* exit dedicated worker upon new deployment ([2038fe6](https://github.com/windmill-labs/windmill/commit/2038fe68ce2ffeeed34c4cfcb44c46ad08f8e732))
+* exit dedicated worker upon new deployment ([4661dd2](https://github.com/windmill-labs/windmill/commit/4661dd2cea343572099ca66459d3db5f3c7879c8))
+
+
+### Bug Fixes
+
+* improve display of optimized one-loops + dependency jobs of dedicated workers happen on dedicated worker ([f998c72](https://github.com/windmill-labs/windmill/commit/f998c7229cd0b27d906a79f99ee7c6e0da1d1810))
+* Invalid config for workers does not panic ([#2612](https://github.com/windmill-labs/windmill/issues/2612)) ([aee35d6](https://github.com/windmill-labs/windmill/commit/aee35d6d511d16130fb64ae4dd2e28757e99f79a))
+* support digit prefixed script in python ([b9e41d0](https://github.com/windmill-labs/windmill/commit/b9e41d066da801ac43276c48af3f100aa70c5a5d))
+
+## [1.204.1](https://github.com/windmill-labs/windmill/compare/v1.204.0...v1.204.1) (2023-11-10)
+
+
+### Bug Fixes
+
+* fix custom components ([4136442](https://github.com/windmill-labs/windmill/commit/41364421ea2ed5980bada139261760bbb6ee8e31))
+* **frontend:** fix login icons + add Okta ([#2609](https://github.com/windmill-labs/windmill/issues/2609)) ([e22f373](https://github.com/windmill-labs/windmill/commit/e22f3738d512b4d7657acc8d4ddf280039acbe56))
+* optimize single step iterative forloops ([#2596](https://github.com/windmill-labs/windmill/issues/2596)) ([88e3648](https://github.com/windmill-labs/windmill/commit/88e3648ee413286769b72acc02a4af6173fa6bac))
+
+## [1.204.0](https://github.com/windmill-labs/windmill/compare/v1.203.0...v1.204.0) (2023-11-10)
+
+
+### Features
+
+* add sql server ([#2604](https://github.com/windmill-labs/windmill/issues/2604)) ([577e130](https://github.com/windmill-labs/windmill/commit/577e1300b93773ab038b067574928d92cae69275))
+* add support for custom sso logins ([0ccf706](https://github.com/windmill-labs/windmill/commit/0ccf706fa28ba615b887ae9c930089be45f14b13))
+* **frontend:** add confirmation modal when deleting a user in instance settings ([#2608](https://github.com/windmill-labs/windmill/issues/2608)) ([a99edf7](https://github.com/windmill-labs/windmill/commit/a99edf7764f1a46809387f851fd9acdb1057810a))
+* **frontend:** Buttons refactor ([#2545](https://github.com/windmill-labs/windmill/issues/2545)) ([fe35c7a](https://github.com/windmill-labs/windmill/commit/fe35c7ad3cf5cb9d8ebcd2a0723533200034fa74))
+* **frontend:** draft script/flow can only access the edit page until… ([#2607](https://github.com/windmill-labs/windmill/issues/2607)) ([adad712](https://github.com/windmill-labs/windmill/commit/adad71266269c17d27ca1bbe8ebe5505b89eb855))
+
+
+### Bug Fixes
+
+* `iter` args conflicts with external variables named iter ([#2605](https://github.com/windmill-labs/windmill/issues/2605)) ([cb01703](https://github.com/windmill-labs/windmill/commit/cb01703a08f4f63362af98594eec4d08e3f25c04))
+
+## [1.203.0](https://github.com/windmill-labs/windmill/compare/v1.202.1...v1.203.0) (2023-11-09)
+
+
+### Features
+
+* add support for custom components in react or vanilla JS ([#2603](https://github.com/windmill-labs/windmill/issues/2603)) ([28c9fdc](https://github.com/windmill-labs/windmill/commit/28c9fdc4f209bcc099f741a448cd3af0797acd53))
+* **frontend:** add a way to customise the link's label ([#2591](https://github.com/windmill-labs/windmill/issues/2591)) ([72854b5](https://github.com/windmill-labs/windmill/commit/72854b55b9db5c7c2ec3cbf65b0ed851ca7eb29a))
+* **frontend:** Migrate flow advanced settings to new layout ([#2589](https://github.com/windmill-labs/windmill/issues/2589)) ([55e3a95](https://github.com/windmill-labs/windmill/commit/55e3a9561899127ba647ff87d32cf010f2aefb90))
+
+
+### Bug Fixes
+
+* Fix error handler token injection ([#2598](https://github.com/windmill-labs/windmill/issues/2598)) ([aefa43d](https://github.com/windmill-labs/windmill/commit/aefa43dcafe929d8939dd2ee5ba94633759204a7))
+* frontend build ([#2593](https://github.com/windmill-labs/windmill/issues/2593)) ([a87b15f](https://github.com/windmill-labs/windmill/commit/a87b15f2c56c19e6f901da69240b3c127ece3b50))
+* Frontend workspace error handler args ([#2597](https://github.com/windmill-labs/windmill/issues/2597)) ([fc87413](https://github.com/windmill-labs/windmill/commit/fc874132c029a7fb2571fe5c296c836b451b351a))
+* **frontend:** correctly clear result when removing all options in a multi select ([#2600](https://github.com/windmill-labs/windmill/issues/2600)) ([ed24838](https://github.com/windmill-labs/windmill/commit/ed24838b46916f8415afcfab3e9700d2ffad9a63))
+* Pythong script in dedicated worker fails with more than 1 arg ([#2588](https://github.com/windmill-labs/windmill/issues/2588)) ([0d846b3](https://github.com/windmill-labs/windmill/commit/0d846b310d8f1ade8a01607d49c6e50ba417f54e))
+* s3 snippets arg inputs update ([#2592](https://github.com/windmill-labs/windmill/issues/2592)) ([16a5fb9](https://github.com/windmill-labs/windmill/commit/16a5fb9e8662afdf84c7e87dbe7a8db7d7f09563))
+
 ## [1.202.1](https://github.com/windmill-labs/windmill/compare/v1.202.0...v1.202.1) (2023-11-07)
 
 

README.md

@@ -1,14 +1,11 @@
 <p align="center">
   <a href="https://www.windmill.dev/"><img src="./imgs/windmill-banner.png" alt="windmill.dev"></a>
 </p>
-<p align="center">
-    <em>.</em>
-</p>
 <p align=center>
-Open-source developer infrastructure for internal tools. Self-hostable alternative to Airplane, Pipedream, Superblocks and a simplified Temporal with autogenerated UIsm and custom UIs to trigger workflows and scripts as internal apps.
+Open-source developer infrastructure for internal tools (APIs, background jobs, workflows and UIs). Self-hostable alternative to Airplane, Pipedream, Superblocks and a simplified Temporal with autogenerated UIsm and custom UIs to trigger workflows and scripts as internal apps.
 
 <p align=center>
-Scripts are turned into UIs and no-code modules, no-code modules can be composed into very rich flows, and script and flows can be triggered from internal UIs made with a low-code builder. The script languages supported are: Python, TypeScript, Go, Bash, SQL. Scripts can be generated by an AI assistant powered by OpenAI.
+Scripts are turned into UIs and no-code modules, no-code modules can be composed into rich flows, and script and flows can be triggered from internal UIs made with a low-code builder. Supported script languages supported are: Python, TypeScript, Go, Bash, SQL.
 </p>
 
 <p align="center">
@@ -36,7 +33,7 @@ Scripts are turned into UIs and no-code modules, no-code modules can be composed
   <a href="https://app.windmill.dev">Try it</a> - <a href="https://www.windmill.dev/docs/intro/">Docs</a> - <a href="https://discord.gg/V7PM2YHsPB">Discord</a> - <a href="https://hub.windmill.dev">Hub</a> - <a href="https://www.windmill.dev/docs/misc/contributing">Contributor's guide</a>
 </p>
 
-# Windmill - Turn scripts into workflows and UIs that you can share and run at scale
+# Windmill - Developer platform for APIs, background jobs, workflows and UIs
 
 Windmill is <b>fully open-sourced (AGPLv3)</b> and Windmill Labs offers
 dedicated instance and commercial support and licenses.
@@ -45,12 +42,13 @@ dedicated instance and commercial support and licenses.
 
 https://github.com/windmill-labs/windmill/assets/122811744/0b132cd1-ee67-4505-822f-0c7ee7104252
 
-- [Windmill - Turn scripts into workflows and UIs that you can share and run at scale](#windmill---turn-scripts-into-workflows-and-uis-that-you-can-share-and-run-at-scale)
+- [Windmill - Developer platform for APIs, background jobs, workflows and UIs](#windmill---developer-platform-for-apis-background-jobs-workflows-and-uis)
   - [Main Concepts](#main-concepts)
   - [Show me some actual script code](#show-me-some-actual-script-code)
   - [CLI](#cli)
     - [Running scripts locally](#running-scripts-locally)
   - [Stack](#stack)
+  - [Fastest Self-Hostable Workflow Engine](#fastest-self-hostable-workflow-engine)
   - [Security](#security)
     - [Sandboxing](#sandboxing)
     - [Secrets, credentials and sensitive values](#secrets-credentials-and-sensitive-values)
@@ -59,12 +57,10 @@ https://github.com/windmill-labs/windmill/assets/122811744/0b132cd1-ee67-4505-82
   - [How to self-host](#how-to-self-host)
     - [Docker compose](#docker-compose)
     - [Kubernetes (k8s) and Helm charts](#kubernetes-k8s-and-helm-charts)
-    - [Postgres without superuser](#postgres-without-superuser)
+    - [Run from binaries](#run-from-binaries)
+    - [OAuth, SSO \& SMTP](#oauth-sso--smtp)
     - [Commercial license](#commercial-license)
-    - [OAuth for self-hosting](#oauth-for-self-hosting)
-    - [smtp for self-hosting](#smtp-for-self-hosting)
-    - [Resource types](#resource-types)
-    - [Manually fetch latest Windmill binary](#manually-fetch-latest-windmill-binary)
+    - [Integrations](#integrations)
   - [Environment Variables](#environment-variables)
   - [Run a local dev setup](#run-a-local-dev-setup)
     - [only Frontend](#only-frontend)
@@ -75,22 +71,28 @@ https://github.com/windmill-labs/windmill/assets/122811744/0b132cd1-ee67-4505-82
 ## Main Concepts
 
 1. Define a minimal and generic script in Python, TypeScript, Go or Bash that
-   solves a specific task. Here sending a POST request. The code can be defined
-   in the provided Web IDE or synchronized with your own github repo:
+   solves a specific task. The code can be defined
+   in the [provided Web IDE](https://www.windmill.dev/docs/code_editor) or [synchronized with your own GitHub repo](https://www.windmill.dev/docs/advanced/cli/sync) (e.g. through [VS Code](https://www.windmill.dev/docs/cli_local_dev/vscode-extension) extension):
+
    ![Step 1](./imgs/windmill-editor.png)
 
-2. Your scripts parameters are automatically parsed and generate a frontend.
-   ![Step 2](./imgs/windmill-run.png) ![Step 3](./imgs/windmill-result.png)
+2. Your scripts parameters are automatically parsed and [generate a frontend](https://www.windmill.dev/docs/core_concepts/auto_generated_uis).
 
-3. Make it flow! You can chain your scripts or scripts made by the community
+  ![Step 2](./imgs/windmill-run.png)
+  
+  ![Step 3](./imgs/windmill-result.png)
+
+3. Make it [flow](https://www.windmill.dev/docs/flows/flow_editor)! You can chain your scripts or scripts made by the community
    shared on [WindmillHub](https://hub.windmill.dev).
+
    ![Step 3](./imgs/windmill-flow.png)
 
-4. Build complex UIs on top of your scripts and flows.
+4. Build [complex UIs](https://www.windmill.dev/docs/apps/app_editor) on top of your scripts and flows.
+
    ![Step 4](./imgs/windmill-builder.png)
 
-Scripts and flows can also be triggered by a cron schedule '_/5 _ \* \* \*' or
-through webhooks.
+Scripts and flows can also be triggered by a [cron schedule](https://www.windmill.dev/docs/core_concepts/scheduling) (e.g. '_/5 _ \* \* \*') or
+through [webhooks](https://www.windmill.dev/docs/core_concepts/webhooks).
 
 You can build your entire infra on top of Windmill!
 
@@ -146,7 +148,9 @@ instance from local commands. See
 You can run your script locally easily, you simply need to pass the right
 environment variables for the `wmill` client library to fetch resources and
 variables from your instance if necessary. See more:
-<https://www.windmill.dev/docs/advanced/local_development/>.
+<https://www.windmill.dev/docs/advanced/local_development>.
+
+To develop & test locally scripts & flows, we recommend using the Windmill VS Code extension: <https://www.windmill.dev/docs/cli_local_dev/vscode-extension>.
 
 ## Stack
 
@@ -167,6 +171,14 @@ variables from your instance if necessary. See more:
 - python runtime is python3
 - golang runtime is 1.19.1
 
+## Fastest Self-Hostable Workflow Engine
+
+We have compared Windmill to other self-hostable workflow engines (Airflow, Prefect & Temporal) and Windmill is the most performant solution for both benchmarks: one flow composed of 40 lightweight tasks & one flow composed of 10 long-running tasks.
+
+All methodology & results on our [Benchmarks](https://www.windmill.dev/docs/misc/benchmarks/competitors#airflow-setup) page.
+
+![Fastest workflow engine](./imgs/fastest.png)
+
 ## Security
 
 ### Sandboxing
@@ -204,10 +216,14 @@ back to the database is ~50ms. A typical lightweight deno job will take around
 
 We only provide docker-compose setup here. For more advanced setups, like
 compiling from source or using without a postgres super user, see
-[documentation](https://www.windmill.dev/docs/advanced/self_host).
+[Self-Host documentation](https://www.windmill.dev/docs/advanced/self_host).
 
 ### Docker compose
 
+Windmill can be deployed using 3 files: ([docker-compose.yml](./docker-compose.yml), [Caddyfile](./Caddyfile) and a [.env](./.env)) in a single command.
+
+Make sure Docker is started, and run:
+
 ```
 curl https://raw.githubusercontent.com/windmill-labs/windmill/main/docker-compose.yml -o docker-compose.yml
 curl https://raw.githubusercontent.com/windmill-labs/windmill/main/Caddyfile -o Caddyfile
@@ -222,6 +238,8 @@ The default super-admin user is: admin@windmill.dev / changeme.
 
 From there, you can follow the setup app and create other users.
 
+More details in [Self-Host Documention](https://www.windmill.dev/docs/advanced/self_host#docker).
+
 ### Kubernetes (k8s) and Helm charts
 
 We publish helm charts at:
@@ -247,14 +265,16 @@ Workspace SSO, Microsoft/Azure and Okta) directly from the UI in the superadmin
 settings. Do note that there is a limit of 50 SSO users on the community
 edition.
 
+[See documentation](https://www.windmill.dev/docs/misc/setup_oauth).
+
 ### Commercial license
 
-To self-host Windmill, you must respect the terms of the AGPLv3 license which
+To self-host Windmill, you must respect the terms of the [AGPLv3 license](https://www.gnu.org/licenses/agpl-3.0.en.html) which
 you do not need to worry about for personal uses. For business uses, you should
-be fine if you do not re-expose it in any way Windmill to your users and are
+be fine if you do not re-expose Windmill in any way to your users and are
 comfortable with AGPLv3.
 
-To re-expose any Windmill parts to your users as a feature of your product, or
+To [re-expose any Windmill parts to your users](https://www.windmill.dev/docs/misc/white_labelling) as a feature of your product, or
 to build a feature on top of Windmill, to comply with AGPLv3 your product must
 be AGPLv3 or you must get a commercial license. Contact us at
 <ruben@windmill.dev> if you have any doubts.
@@ -264,9 +284,11 @@ your current infrastructure to Windmill, support with tight SLA, and our global
 cache sync for high-performance/no dependency cache miss of cluster from 10+
 nodes to 200+ nodes.
 
-### Resource types
+### Integrations
 
-You will also want to import all the approved resource types from
+In Windmill, integrations are referred to as [resources and resource types](https://www.windmill.dev/docs/core_concepts/resources_and_types). Each Resource has a Resource Type that defines the schema that the resource needs to implement.
+
+On self-hosted instances, you might want to import all the approved resource types from
 [WindmillHub](https://hub.windmill.dev). A setup script will prompt you to have
 it being synced automatically everyday.
 
@@ -341,7 +363,7 @@ it being synced automatically everyday.
 | SMTP_PASSWORD                                 | None                                       | password for the smtp server to send invite emails                                                                                                                                                 | Server                |
 | SMTP_TLS_IMPLICIT                             | false                                      | https://docs.rs/mail-send/latest/mail_send/struct.SmtpClientBuilder.html#method.implicit_tlsemails                                                                                                 | Server                |
 | CREATE_WORKSPACE_REQUIRE_SUPERADMIN           | true                                       | If true, only superadmin can create workspaces                                                                                                                                                     | Server                |
-| GLOBAL_ERROR_HANDLER_PATH_IN_ADMINS_WORKSPACE | None                                       | Path to a script to run when a root job fails. The script will be run in and from the admins workspace                                                                                             | Server                |
+| GLOBAL_ERROR_HANDLER_PATH_IN_ADMINS_WORKSPACE | None                                       | Path to a script or flow to run when a root job fails. The path needs to be prefixed with either `script/` or `flow/` to indicate the kind of error handler being used (assuming `script/` by default). The error handler will be run in and from the admins workspace                                                                                             | Server                |
 | WHITELIST_ENVS                                | None                                       | List of envs variables, separated by a ',' that are whitelisted as being safe to passthrough the workers                                                                                           | Worker                |
 | SAML_METADATA                                 | None                                       | SAML Metadata URL to enable SAML SSO (EE only)                                                                                                                                                     | Server                |
 | SECRET_SALT                                   | None                                       | Secret Salt used for encryption and decryption of secrets. If defined, the secrets will not be decryptable unless the right salt is passed in, which is the case for the workers and the server    | Server + Worker       |

backend/.sqlx/query-00be497354f5375e9ccffb998d126a853da91d607ff9e57e10d0e5481e4d3848.json

@@ -0,0 +1,15 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "DELETE FROM usr WHERE workspace_id = $1 AND email = $2",
+  "describe": {
+    "columns": [],
+    "parameters": {
+      "Left": [
+        "Text",
+        "Text"
+      ]
+    },
+    "nullable": []
+  },
+  "hash": "00be497354f5375e9ccffb998d126a853da91d607ff9e57e10d0e5481e4d3848"
+}

backend/.sqlx/query-0360207b5fb2a7f877c2608566454f40f7cbbcd20bcb84e5968ac3e21b6ea0f6.json

@@ -1,6 +1,6 @@
 {
   "db_name": "PostgreSQL",
-  "query": "INSERT INTO workspace_invite\n        (workspace_id, email, is_admin, operator)\n        SELECT $1::text, email, false, $3 FROM password WHERE email LIKE CONCAT('%', $2::text) AND NOT EXISTS (\n            SELECT 1 FROM usr WHERE workspace_id = $1::text AND email = password.email\n        )\n        ON CONFLICT DO NOTHING",
+  "query": "INSERT INTO workspace_invite\n        (workspace_id, email, is_admin, operator)\n        SELECT $1::text, email, false, $3 FROM password WHERE ($2::text = '*' OR email LIKE CONCAT('%', $2::text)) AND NOT EXISTS (\n            SELECT 1 FROM usr WHERE workspace_id = $1::text AND email = password.email\n        )\n        ON CONFLICT DO NOTHING",
   "describe": {
     "columns": [],
     "parameters": {
@@ -12,5 +12,5 @@
     },
     "nullable": []
   },
-  "hash": "d4bf68b4c2bfd11db84787e41e92c13612fda7352e841f3cdef45ea762588f04"
+  "hash": "0360207b5fb2a7f877c2608566454f40f7cbbcd20bcb84e5968ac3e21b6ea0f6"
 }

backend/.sqlx/query-108e4c505168381b51ad298b5294aa73e84d35bb68a5911985139bbf94c1d231.json

@@ -0,0 +1,23 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "SELECT extra_perms from resource WHERE path = $1 AND workspace_id = $2",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "extra_perms",
+        "type_info": "Jsonb"
+      }
+    ],
+    "parameters": {
+      "Left": [
+        "Text",
+        "Text"
+      ]
+    },
+    "nullable": [
+      false
+    ]
+  },
+  "hash": "108e4c505168381b51ad298b5294aa73e84d35bb68a5911985139bbf94c1d231"
+}

backend/.sqlx/query-11ae5e1aade2bc9a78af2d956ece73766ae3330319619a68383b508a76f2d19c.json

@@ -42,12 +42,14 @@
                 "bash",
                 "postgresql",
                 "nativets",
+                "Nativets",
                 "bun",
                 "mysql",
                 "bigquery",
                 "snowflake",
                 "graphql",
-                "powershell"
+                "powershell",
+                "mssql"
               ]
             }
           }

backend/.sqlx/query-1289e7278d2a289bfaa53f00e0b6dceb195df0fb43a8ac03bc8b35939fc941dd.json

@@ -27,6 +27,11 @@
         "ordinal": 4,
         "name": "premium",
         "type_info": "Bool"
+      },
+      {
+        "ordinal": 5,
+        "name": "is_overquota",
+        "type_info": "Bool"
       }
     ],
     "parameters": {
@@ -40,6 +45,7 @@
       false,
       false,
       false,
+      false,
       false
     ]
   },

backend/.sqlx/query-1730f39fd1793d45fbb41b21389c61296a3ff7489ae12f52a19f9543173ac597.json

@@ -82,6 +82,11 @@
         "ordinal": 15,
         "name": "error_handler_muted_on_cancel",
         "type_info": "Bool"
+      },
+      {
+        "ordinal": 16,
+        "name": "large_file_storage",
+        "type_info": "Jsonb"
       }
     ],
     "parameters": {
@@ -105,7 +110,8 @@
       true,
       false,
       true,
-      false
+      false,
+      true
     ]
   },
   "hash": "1730f39fd1793d45fbb41b21389c61296a3ff7489ae12f52a19f9543173ac597"

backend/.sqlx/query-18ef1ffa6591dfac2ac77659d84b3d755f487e87f3f43ccc90fa7a6068c84b32.json

@@ -0,0 +1,14 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "DELETE FROM capture WHERE workspace_id = $1",
+  "describe": {
+    "columns": [],
+    "parameters": {
+      "Left": [
+        "Text"
+      ]
+    },
+    "nullable": []
+  },
+  "hash": "18ef1ffa6591dfac2ac77659d84b3d755f487e87f3f43ccc90fa7a6068c84b32"
+}

backend/.sqlx/query-254776959f60455a00a2c29a1783ee6b27c836197df6454d3bb5c59441c34f44.json

@@ -67,12 +67,14 @@
                 "bash",
                 "postgresql",
                 "nativets",
+                "Nativets",
                 "bun",
                 "mysql",
                 "bigquery",
                 "snowflake",
                 "graphql",
-                "powershell"
+                "powershell",
+                "mssql"
               ]
             }
           }

backend/.sqlx/query-2ff397ad0aeabdc15776025f75c2d1390ce5ac700b0bf71481d31747c8f2e870.json

@@ -0,0 +1,14 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "UPDATE workspace_settings SET large_file_storage = NULL WHERE workspace_id = $1",
+  "describe": {
+    "columns": [],
+    "parameters": {
+      "Left": [
+        "Text"
+      ]
+    },
+    "nullable": []
+  },
+  "hash": "2ff397ad0aeabdc15776025f75c2d1390ce5ac700b0bf71481d31747c8f2e870"
+}

backend/.sqlx/query-3a534b4fc36171efaa7c647f48320b59bbc414cfb92e960c174dd63fc180e187.json

@@ -1,12 +1,17 @@
 {
   "db_name": "PostgreSQL",
-  "query": "SELECT tag from flow WHERE path = $1 and workspace_id = $2",
+  "query": "SELECT tag, dedicated_worker from flow WHERE path = $1 and workspace_id = $2",
   "describe": {
     "columns": [
       {
         "ordinal": 0,
         "name": "tag",
         "type_info": "Varchar"
+      },
+      {
+        "ordinal": 1,
+        "name": "dedicated_worker",
+        "type_info": "Bool"
       }
     ],
     "parameters": {
@@ -16,8 +21,9 @@
       ]
     },
     "nullable": [
+      true,
       true
     ]
   },
-  "hash": "238a59ebc80619504e6dc41c3c24f4ce27786997f380e520d27e169023b28d89"
+  "hash": "3a534b4fc36171efaa7c647f48320b59bbc414cfb92e960c174dd63fc180e187"
 }

backend/.sqlx/query-4999995f4459da32a0e2fc7115eae97fb49b0080075fb01e0863d164ba9a5f13.json

@@ -0,0 +1,40 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "SELECT premium, usage.usage as \"usage?\", workspace_settings.customer_id, workspace_settings.plan FROM workspace LEFT JOIN workspace_settings ON workspace_settings.workspace_id = $1 LEFT JOIN usage ON usage.id = $1 AND month_ = EXTRACT(YEAR FROM current_date) * 12 + EXTRACT(MONTH FROM current_date) AND usage.is_workspace IS true WHERE workspace.id = $1",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "premium",
+        "type_info": "Bool"
+      },
+      {
+        "ordinal": 1,
+        "name": "usage?",
+        "type_info": "Int4"
+      },
+      {
+        "ordinal": 2,
+        "name": "customer_id",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 3,
+        "name": "plan",
+        "type_info": "Varchar"
+      }
+    ],
+    "parameters": {
+      "Left": [
+        "Text"
+      ]
+    },
+    "nullable": [
+      false,
+      false,
+      true,
+      true
+    ]
+  },
+  "hash": "4999995f4459da32a0e2fc7115eae97fb49b0080075fb01e0863d164ba9a5f13"
+}

backend/.sqlx/query-49e57089ff611b69f173c7d9706dd6d297c053b685ee0b3885c8c29fb2ecc84c.json

@@ -0,0 +1,14 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "UPDATE schedule SET on_recovery = NULL, on_recovery_extra_args = NULL, on_recovery_times = NULL WHERE workspace_id = $1",
+  "describe": {
+    "columns": [],
+    "parameters": {
+      "Left": [
+        "Text"
+      ]
+    },
+    "nullable": []
+  },
+  "hash": "49e57089ff611b69f173c7d9706dd6d297c053b685ee0b3885c8c29fb2ecc84c"
+}

backend/.sqlx/query-4c970f10d345bcdcf956dcbfa22b6e80888e511fb4787cb1a7976878abed1d30.json

@@ -1,17 +1,16 @@
 {
   "db_name": "PostgreSQL",
-  "query": "SELECT is_flow FROM schedule WHERE path = $1 AND workspace_id = $2",
+  "query": "SELECT premium FROM workspace WHERE workspace.id = $1",
   "describe": {
     "columns": [
       {
         "ordinal": 0,
-        "name": "is_flow",
+        "name": "premium",
         "type_info": "Bool"
       }
     ],
     "parameters": {
       "Left": [
-        "Text",
         "Text"
       ]
     },
@@ -19,5 +18,5 @@
       false
     ]
   },
-  "hash": "56839d3aec6c0177d14589aedda8d5c431d841b6d5d0d99ce3836bb42d4d83d9"
+  "hash": "4c970f10d345bcdcf956dcbfa22b6e80888e511fb4787cb1a7976878abed1d30"
 }

backend/.sqlx/query-52ad1773a9f5a1b6be1bb16af408f051f27d3f21b615c468b85aa75b64d943db.json

@@ -28,12 +28,14 @@
                 "bash",
                 "postgresql",
                 "nativets",
+                "Nativets",
                 "bun",
                 "mysql",
                 "bigquery",
                 "snowflake",
                 "graphql",
-                "powershell"
+                "powershell",
+                "mssql"
               ]
             }
           }

backend/.sqlx/query-53a70f4bd6ded970841c6064352c69b019ff84059d9c06a0536eda45d468397c.json

@@ -0,0 +1,14 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "DELETE FROM draft WHERE workspace_id = $1",
+  "describe": {
+    "columns": [],
+    "parameters": {
+      "Left": [
+        "Text"
+      ]
+    },
+    "nullable": []
+  },
+  "hash": "53a70f4bd6ded970841c6064352c69b019ff84059d9c06a0536eda45d468397c"
+}

backend/.sqlx/query-5445083864b2b092b012e894bff7630a1d7b9deb8d33e9f909061f351f96844e.json

@@ -82,6 +82,11 @@
         "ordinal": 15,
         "name": "error_handler_muted_on_cancel",
         "type_info": "Bool"
+      },
+      {
+        "ordinal": 16,
+        "name": "large_file_storage",
+        "type_info": "Jsonb"
       }
     ],
     "parameters": {
@@ -105,7 +110,8 @@
       true,
       false,
       true,
-      false
+      false,
+      true
     ]
   },
   "hash": "5445083864b2b092b012e894bff7630a1d7b9deb8d33e9f909061f351f96844e"

backend/.sqlx/query-5e4e0740e1cff4b2a7c620da3758604a47f81b505fa9a713efa3519b050b546a.json

@@ -0,0 +1,22 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "SELECT large_file_storage FROM workspace_settings WHERE workspace_id = $1",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "large_file_storage",
+        "type_info": "Jsonb"
+      }
+    ],
+    "parameters": {
+      "Left": [
+        "Text"
+      ]
+    },
+    "nullable": [
+      true
+    ]
+  },
+  "hash": "5e4e0740e1cff4b2a7c620da3758604a47f81b505fa9a713efa3519b050b546a"
+}

backend/.sqlx/query-620ddf29c5e867079df4c2aa6e80bccb19beeb9ddfa308ca97f254cd5ba8157e.json

@@ -60,12 +60,14 @@
                 "bash",
                 "postgresql",
                 "nativets",
+                "Nativets",
                 "bun",
                 "mysql",
                 "bigquery",
                 "snowflake",
                 "graphql",
-                "powershell"
+                "powershell",
+                "mssql"
               ]
             }
           }

backend/.sqlx/query-67624d479cc293b0306398f9e855cf38a72dd3d7d75b01e93cfd3ac4875b6e37.json

@@ -0,0 +1,20 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "SELECT COUNT(*) FROM password",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "count",
+        "type_info": "Int8"
+      }
+    ],
+    "parameters": {
+      "Left": []
+    },
+    "nullable": [
+      null
+    ]
+  },
+  "hash": "67624d479cc293b0306398f9e855cf38a72dd3d7d75b01e93cfd3ac4875b6e37"
+}

backend/.sqlx/query-6d4f3f071d9800f3330a809c34368d34b4563e17132037e2d2295e82cbc4e629.json

@@ -0,0 +1,14 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "UPDATE schedule SET ws_error_handler_muted = false, on_failure = NULL, on_failure_extra_args = NULL, on_failure_times = NULL, on_failure_exact = NULL WHERE workspace_id = $1",
+  "describe": {
+    "columns": [],
+    "parameters": {
+      "Left": [
+        "Text"
+      ]
+    },
+    "nullable": []
+  },
+  "hash": "6d4f3f071d9800f3330a809c34368d34b4563e17132037e2d2295e82cbc4e629"
+}

backend/.sqlx/query-6dbf0275e52d937a114ba305c6646d313f6ef05062ac55f7397caf8c02ccec5e.json

@@ -1,37 +0,0 @@
-{
-  "db_name": "PostgreSQL",
-  "query": "DELETE FROM queue WHERE schedule_path = $1 AND running = false AND job_kind = $2 AND workspace_id = $3",
-  "describe": {
-    "columns": [],
-    "parameters": {
-      "Left": [
-        "Text",
-        {
-          "Custom": {
-            "name": "job_kind",
-            "kind": {
-              "Enum": [
-                "script",
-                "preview",
-                "flow",
-                "dependencies",
-                "flowpreview",
-                "script_hub",
-                "identity",
-                "flowdependencies",
-                "http",
-                "graphql",
-                "postgresql",
-                "noop",
-                "appdependencies"
-              ]
-            }
-          }
-        },
-        "Text"
-      ]
-    },
-    "nullable": []
-  },
-  "hash": "6dbf0275e52d937a114ba305c6646d313f6ef05062ac55f7397caf8c02ccec5e"
-}

backend/.sqlx/query-6e816fdba0d281d5147ddc5d4d438a3209b811324dba4ac661938e0523210682.json

@@ -0,0 +1,28 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "SELECT premium, is_overquota FROM workspace WHERE id = $1",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "premium",
+        "type_info": "Bool"
+      },
+      {
+        "ordinal": 1,
+        "name": "is_overquota",
+        "type_info": "Bool"
+      }
+    ],
+    "parameters": {
+      "Left": [
+        "Text"
+      ]
+    },
+    "nullable": [
+      false,
+      false
+    ]
+  },
+  "hash": "6e816fdba0d281d5147ddc5d4d438a3209b811324dba4ac661938e0523210682"
+}

backend/.sqlx/query-7813908c080e239fd6b9df3711fd704d6bb5d5686a02b72de2cb61e3c127cb54.json

@@ -0,0 +1,23 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "SELECT extra_perms from folder WHERE name = $1 AND workspace_id = $2",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "extra_perms",
+        "type_info": "Jsonb"
+      }
+    ],
+    "parameters": {
+      "Left": [
+        "Text",
+        "Text"
+      ]
+    },
+    "nullable": [
+      false
+    ]
+  },
+  "hash": "7813908c080e239fd6b9df3711fd704d6bb5d5686a02b72de2cb61e3c127cb54"
+}

backend/.sqlx/query-7d3180d119da0f6215a571118439469569c31e0234bb073af33f2cd4d6ca71d4.json

@@ -0,0 +1,14 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "DELETE FROM queue WHERE workspace_id = $1",
+  "describe": {
+    "columns": [],
+    "parameters": {
+      "Left": [
+        "Text"
+      ]
+    },
+    "nullable": []
+  },
+  "hash": "7d3180d119da0f6215a571118439469569c31e0234bb073af33f2cd4d6ca71d4"
+}

backend/.sqlx/query-8c5e13df05fca96685deb44631b285568cb4f7d5b8b6a95f1f669c59f0355752.json

@@ -1,6 +1,6 @@
 {
   "db_name": "PostgreSQL",
-  "query": "SELECT workspace_id, auto_invite_operator FROM workspace_settings WHERE auto_invite_domain = $1",
+  "query": "SELECT workspace_id, auto_invite_operator FROM workspace_settings WHERE auto_invite_domain = $1 OR auto_invite_domain = '*'",
   "describe": {
     "columns": [
       {
@@ -24,5 +24,5 @@
       true
     ]
   },
-  "hash": "9db64c9ff790d8c833c1e831a87803c32103ce9de68cc9c08d6f56cc988d7e37"
+  "hash": "8c5e13df05fca96685deb44631b285568cb4f7d5b8b6a95f1f669c59f0355752"
 }

backend/.sqlx/query-8e0679c2b1bd451691fe5c69a2841ddc9f211311316ec6b9d4699b2c70997a19.json

@@ -0,0 +1,35 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "SELECT tag, dedicated_worker, value->>'early_return' as early_return from flow WHERE path = $1 and workspace_id = $2",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "tag",
+        "type_info": "Varchar"
+      },
+      {
+        "ordinal": 1,
+        "name": "dedicated_worker",
+        "type_info": "Bool"
+      },
+      {
+        "ordinal": 2,
+        "name": "early_return",
+        "type_info": "Text"
+      }
+    ],
+    "parameters": {
+      "Left": [
+        "Text",
+        "Text"
+      ]
+    },
+    "nullable": [
+      true,
+      true,
+      null
+    ]
+  },
+  "hash": "8e0679c2b1bd451691fe5c69a2841ddc9f211311316ec6b9d4699b2c70997a19"
+}

backend/.sqlx/query-98c8e2c58b38beedcb474251c2554a2f1b768ec7ed1a419313f4842f26425e5d.json

@@ -0,0 +1,14 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "INSERT INTO folder (workspace_id, name, display_name, owners, extra_perms) VALUES ($1, 'app_custom', 'App Custom Components', ARRAY[]::TEXT[], '{\"g/all\": false}') ON CONFLICT DO NOTHING",
+  "describe": {
+    "columns": [],
+    "parameters": {
+      "Left": [
+        "Varchar"
+      ]
+    },
+    "nullable": []
+  },
+  "hash": "98c8e2c58b38beedcb474251c2554a2f1b768ec7ed1a419313f4842f26425e5d"
+}

backend/.sqlx/query-9d3556319411a27a875bf6cf0e5eda837cc63e4d8be912c0b5bfeea4a0c8db2e.json

@@ -40,12 +40,14 @@
                 "bash",
                 "postgresql",
                 "nativets",
+                "Nativets",
                 "bun",
                 "mysql",
                 "bigquery",
                 "snowflake",
                 "graphql",
-                "powershell"
+                "powershell",
+                "mssql"
               ]
             }
           }

backend/.sqlx/query-a90924854cbd98bb16a2ed02e7966b0726e11ddd7511f1d503b1df29dee71419.json

@@ -0,0 +1,23 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "SELECT extra_perms from variable WHERE path = $1 AND workspace_id = $2",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "extra_perms",
+        "type_info": "Jsonb"
+      }
+    ],
+    "parameters": {
+      "Left": [
+        "Text",
+        "Text"
+      ]
+    },
+    "nullable": [
+      false
+    ]
+  },
+  "hash": "a90924854cbd98bb16a2ed02e7966b0726e11ddd7511f1d503b1df29dee71419"
+}

backend/.sqlx/query-a96dd57b127a1adbdca13867a76d7fd9f9a9c76b02bcebd2830d928821f3abc0.json

@@ -1,16 +1,15 @@
 {
   "db_name": "PostgreSQL",
-  "query": "UPDATE queue SET canceled = true, canceled_reason = 'http connection broke', canceled_by = $1 WHERE id = $2 AND workspace_id = $3",
+  "query": "UPDATE queue SET canceled = true, canceled_reason = 'http connection broke', canceled_by = queue.created_by WHERE id = $1 AND workspace_id = $2",
   "describe": {
     "columns": [],
     "parameters": {
       "Left": [
-        "Varchar",
         "Uuid",
         "Text"
       ]
     },
     "nullable": []
   },
-  "hash": "460a3c3161899e172d2d20004e12ad3f331c73f260d099749a9e8fb85ae5614e"
+  "hash": "a96dd57b127a1adbdca13867a76d7fd9f9a9c76b02bcebd2830d928821f3abc0"
 }

backend/.sqlx/query-a9915d3c72a278ff08fbe6571c2a4fc993507e255004107345b7151322e063de.json

@@ -46,12 +46,14 @@
                 "bash",
                 "postgresql",
                 "nativets",
+                "Nativets",
                 "bun",
                 "mysql",
                 "bigquery",
                 "snowflake",
                 "graphql",
-                "powershell"
+                "powershell",
+                "mssql"
               ]
             }
           }

backend/.sqlx/query-ac01e45d3335015f53f3d63fe159e631efb65c3d326b6b6ae8361a2116bff145.json

@@ -1,6 +1,6 @@
 {
   "db_name": "PostgreSQL",
-  "query": "\n                    SELECT content FROM script WHERE path = $1 AND workspace_id = $2\n                    AND created_at = (SELECT max(created_at) FROM script WHERE path = $1 AND \n                    workspace_id = $2)\n                    ",
+  "query": "\n                    SELECT content FROM script WHERE path = $1 AND workspace_id = $2\n                    AND created_at = (SELECT max(created_at) FROM script WHERE path = $1 AND\n                    workspace_id = $2)\n                    ",
   "describe": {
     "columns": [
       {
@@ -19,5 +19,5 @@
       false
     ]
   },
-  "hash": "d581bfb507d3a875f07677187717a469ec6ab1db1585835c961409612d81ec7d"
+  "hash": "ac01e45d3335015f53f3d63fe159e631efb65c3d326b6b6ae8361a2116bff145"
 }

backend/.sqlx/query-ade89de6e8527c543b182229f1febeb2513ad58b03ab526df148582264fb3a44.json

@@ -0,0 +1,15 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "DELETE FROM queue WHERE schedule_path = $1 AND running = false AND workspace_id = $2",
+  "describe": {
+    "columns": [],
+    "parameters": {
+      "Left": [
+        "Text",
+        "Text"
+      ]
+    },
+    "nullable": []
+  },
+  "hash": "ade89de6e8527c543b182229f1febeb2513ad58b03ab526df148582264fb3a44"
+}

backend/.sqlx/query-b6074eb9369de071f79165bf506fbc44f1688035e1928b4c3649d875187b1e1d.json

@@ -42,12 +42,14 @@
                 "bash",
                 "postgresql",
                 "nativets",
+                "Nativets",
                 "bun",
                 "mysql",
                 "bigquery",
                 "snowflake",
                 "graphql",
-                "powershell"
+                "powershell",
+                "mssql"
               ]
             }
           }

backend/.sqlx/query-bf27083af3514e5fae868364518096d1cf4a042c23526dc134c3b2d27904e954.json

@@ -0,0 +1,15 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "UPDATE workspace_settings SET large_file_storage = $1 WHERE workspace_id = $2",
+  "describe": {
+    "columns": [],
+    "parameters": {
+      "Left": [
+        "Jsonb",
+        "Text"
+      ]
+    },
+    "nullable": []
+  },
+  "hash": "bf27083af3514e5fae868364518096d1cf4a042c23526dc134c3b2d27904e954"
+}

backend/.sqlx/query-c10f6b544af2a4a3f7e172a6bb919d5133e633b47942406ddf9834d17950df30.json

@@ -0,0 +1,14 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "UPDATE config SET config = config WHERE name = $1",
+  "describe": {
+    "columns": [],
+    "parameters": {
+      "Left": [
+        "Text"
+      ]
+    },
+    "nullable": []
+  },
+  "hash": "c10f6b544af2a4a3f7e172a6bb919d5133e633b47942406ddf9834d17950df30"
+}

backend/.sqlx/query-c3da506f47dfb42434debf6e005319147c7513193e853c38cfb7c071d61a3d2e.json

@@ -1,6 +1,6 @@
 {
   "db_name": "PostgreSQL",
-  "query": "INSERT INTO flow (workspace_id, path, summary, description, value, edited_by, edited_at, schema, dependency_job, draft_only, tag) VALUES ($1, $2, $3, $4, $5, $6, now(), $7::text::json, NULL, $8, $9)",
+  "query": "INSERT INTO flow (workspace_id, path, summary, description, value, edited_by, edited_at, schema, dependency_job, draft_only, tag, dedicated_worker) VALUES ($1, $2, $3, $4, $5, $6, now(), $7::text::json, NULL, $8, $9, $10)",
   "describe": {
     "columns": [],
     "parameters": {
@@ -13,10 +13,11 @@
         "Varchar",
         "Text",
         "Bool",
-        "Varchar"
+        "Varchar",
+        "Bool"
       ]
     },
     "nullable": []
   },
-  "hash": "4757d024f8d8e7b56e5cb648a1bbafe773db2d4086e9b0dbb2bbe229d221135d"
+  "hash": "c3da506f47dfb42434debf6e005319147c7513193e853c38cfb7c071d61a3d2e"
 }

backend/.sqlx/query-c7febee7146a28481ce36f3570bceb38751a6527827525a582d9f8804981f552.json

@@ -42,12 +42,14 @@
                 "bash",
                 "postgresql",
                 "nativets",
+                "Nativets",
                 "bun",
                 "mysql",
                 "bigquery",
                 "snowflake",
                 "graphql",
-                "powershell"
+                "powershell",
+                "mssql"
               ]
             }
           }

backend/.sqlx/query-cdfc8e9af4414b7ee7525bc74d9caad74b5479c64337c401a7a9550a3434cdfa.json

@@ -1,6 +1,6 @@
 {
   "db_name": "PostgreSQL",
-  "query": "SELECT COUNT(*) FROM worker_ping WHERE ping_at > NOW() - INTERVAL '5 minutes'",
+  "query": "SELECT COUNT(*) FROM worker_ping WHERE ping_at > NOW() - INTERVAL '2 minutes'",
   "describe": {
     "columns": [
       {
@@ -16,5 +16,5 @@
       null
     ]
   },
-  "hash": "1c67cce8cdf6e56752643d6382ebaab76e24581009ca5e8d8b5ce3f199b689ab"
+  "hash": "cdfc8e9af4414b7ee7525bc74d9caad74b5479c64337c401a7a9550a3434cdfa"
 }

backend/.sqlx/query-ce4733a0460cbef39fb50f4ca9746944d4cd5057d27d09de9c5dc494e3123ab4.json

@@ -1,12 +1,17 @@
 {
   "db_name": "PostgreSQL",
-  "query": "UPDATE queue SET canceled = true,  canceled_by = $2, scheduled_for = now(), suspend = 0 WHERE workspace_id = $1 AND schedule_path IS NULL RETURNING id",
+  "query": "UPDATE queue SET canceled = true,  canceled_by = $2, scheduled_for = now(), suspend = 0 WHERE workspace_id = $1 AND schedule_path IS NULL RETURNING id, running",
   "describe": {
     "columns": [
       {
         "ordinal": 0,
         "name": "id",
         "type_info": "Uuid"
+      },
+      {
+        "ordinal": 1,
+        "name": "running",
+        "type_info": "Bool"
       }
     ],
     "parameters": {
@@ -16,8 +21,9 @@
       ]
     },
     "nullable": [
+      false,
       false
     ]
   },
-  "hash": "e25829ed40f35a0f59a9838d030a7f888ca88441e069e73a52d75450aecae70e"
+  "hash": "ce4733a0460cbef39fb50f4ca9746944d4cd5057d27d09de9c5dc494e3123ab4"
 }

backend/.sqlx/query-d48070ef889723f408ed6c1a97b5ab9f0e98340cd33c6e29824f6030f0089071.json

@@ -37,12 +37,14 @@
                 "bash",
                 "postgresql",
                 "nativets",
+                "Nativets",
                 "bun",
                 "mysql",
                 "bigquery",
                 "snowflake",
                 "graphql",
-                "powershell"
+                "powershell",
+                "mssql"
               ]
             }
           }

backend/.sqlx/query-d970a0b07a2b5840d1feb1baacb834dbaf91c633d3e7e1e29c8eb7eedc53e888.json

@@ -1,6 +1,6 @@
 {
   "db_name": "PostgreSQL",
-  "query": "INSERT INTO workspace_invite\n            (workspace_id, email, is_admin, operator)\n            VALUES ($1, $2, $3, $4)",
+  "query": "INSERT INTO workspace_invite\n            (workspace_id, email, is_admin, operator)\n            VALUES ($1, $2, $3, $4) ON CONFLICT (workspace_id, email)\n            DO UPDATE SET is_admin = $3, operator = $4",
   "describe": {
     "columns": [],
     "parameters": {
@@ -13,5 +13,5 @@
     },
     "nullable": []
   },
-  "hash": "be7a99a5bb6858323ca61dd51077010f51ba58ae76b9a413339255024dcb524d"
+  "hash": "d970a0b07a2b5840d1feb1baacb834dbaf91c633d3e7e1e29c8eb7eedc53e888"
 }

backend/.sqlx/query-de06474de29c6c2cadb99787f5d627678a5285011d7b1a516d4e240a5810ddba.json

@@ -1,6 +1,6 @@
 {
   "db_name": "PostgreSQL",
-  "query": "UPDATE flow SET path = $1, summary = $2, description = $3, value = $4, edited_by = $5, edited_at = now(), schema = $6::text::json, dependency_job = NULL, draft_only = NULL, tag = $9 WHERE path = $7 AND workspace_id = $8",
+  "query": "UPDATE flow SET path = $1, summary = $2, description = $3, value = $4, edited_by = $5, edited_at = now(), schema = $6::text::json, dependency_job = NULL, draft_only = NULL, tag = $9, dedicated_worker = $10\n        WHERE path = $7 AND workspace_id = $8",
   "describe": {
     "columns": [],
     "parameters": {
@@ -13,10 +13,11 @@
         "Text",
         "Text",
         "Text",
-        "Varchar"
+        "Varchar",
+        "Bool"
       ]
     },
     "nullable": []
   },
-  "hash": "07486bff9344f8c8906b8120ca66c79ab7ac5e0685a1465e8d140686dc1df247"
+  "hash": "de06474de29c6c2cadb99787f5d627678a5285011d7b1a516d4e240a5810ddba"
 }

backend/.sqlx/query-e70835de2b70a01a2e85f44f3cb80f41288d8e68b762de80ae724ed69ef1908c.json

@@ -0,0 +1,26 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "SELECT author.count as author_count, operator.count as operator_count FROM (SELECT count(*)::INT FROM usr where usr.operator IS false) as author, (SELECT count(*)::INT FROM usr where usr.operator IS true) as operator",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "author_count",
+        "type_info": "Int4"
+      },
+      {
+        "ordinal": 1,
+        "name": "operator_count",
+        "type_info": "Int4"
+      }
+    ],
+    "parameters": {
+      "Left": []
+    },
+    "nullable": [
+      null,
+      null
+    ]
+  },
+  "hash": "e70835de2b70a01a2e85f44f3cb80f41288d8e68b762de80ae724ed69ef1908c"
+}

backend/.sqlx/query-e92536776a1960fa739bbb01d5a8d1b451ff2689a95442bdf820c65f0a8412a5.json

@@ -0,0 +1,23 @@
+{
+  "db_name": "PostgreSQL",
+  "query": "SELECT value->>'js' FROM resource\n        WHERE path = $1 AND workspace_id = $2",
+  "describe": {
+    "columns": [
+      {
+        "ordinal": 0,
+        "name": "?column?",
+        "type_info": "Text"
+      }
+    ],
+    "parameters": {
+      "Left": [
+        "Text",
+        "Text"
+      ]
+    },
+    "nullable": [
+      null
+    ]
+  },
+  "hash": "e92536776a1960fa739bbb01d5a8d1b451ff2689a95442bdf820c65f0a8412a5"
+}

backend/.sqlx/query-f7f2598ac824e9b5719b96b1d1873ad728f7f786c7f447df6cac3e65d5a53bf1.json

@@ -27,6 +27,11 @@
         "ordinal": 4,
         "name": "premium",
         "type_info": "Bool"
+      },
+      {
+        "ordinal": 5,
+        "name": "is_overquota",
+        "type_info": "Bool"
       }
     ],
     "parameters": {
@@ -39,6 +44,7 @@
       false,
       false,
       false,
+      false,
       false
     ]
   },

backend/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "windmill"
-version = "1.202.1"
+version = "1.216.0"
 authors.workspace = true
 edition.workspace = true
 
@@ -21,7 +21,7 @@ members = [
 ]
 
 [workspace.package]
-version = "1.202.1"
+version = "1.216.0"
 authors = ["Ruben Fiszel <ruben@windmill.dev>"]
 edition = "2021"
 
@@ -97,7 +97,7 @@ headers = "^0"
 hyper = { version = "^0", features = ["full"] }
 tokio = { version = "^1", features = ["full", "tracing"] }
 tower = "^0"
-tower-http = { version = "^0", features = ["trace", "cors"] }
+tower-http = { version = "^0.4", features = ["trace", "cors"] }
 tower-cookies = "^0"
 serde = "^1"
 serde_json = { version = "^1", features = ["preserve_order", "raw_value"] }
@@ -172,9 +172,10 @@ serde_derive = "1.0.147"
 const_format = { version = "0.2", features = ["rust_1_64", "rust_1_51"] }
 dyn-iter = "0.2.0"
 rsa = "0.7.2"
-async-stripe = { version = "0.14", features = [
+async-stripe = { version = "0.25.2", features = [
     "runtime-tokio-hyper",
     "checkout",
+    "billing",
 ] }
 async_zip = { version = "0.0.11", features = ["full"] }
 once_cell = "1.17.1"
@@ -203,3 +204,4 @@ tokenizers = "0.14.1"
 candle-core = "0.3.0"
 candle-transformers = "0.3.0"
 candle-nn = "0.3.0"
+tiberius = { version = "0.12.2", default-features = false, features = ["rustls", "tds73", "chrono"] }

backend/migrations/20231108200232_add_custom_components.down.sql

@@ -0,0 +1 @@
+-- Add down migration script here

backend/migrations/20231108200232_add_custom_components.up.sql

@@ -0,0 +1,2 @@
+-- Add up migration script here
+INSERT INTO folder (workspace_id, name, display_name, owners, extra_perms) SELECT id, 'app_custom', 'App Custom Components', ARRAY[]::TEXT[], '{"g/all": false}' FROM workspace ON CONFLICT DO NOTHING;

backend/migrations/20231109101743_add_mssql.down.sql

@@ -0,0 +1 @@
+-- Add down migration script here

backend/migrations/20231109101743_add_mssql.up.sql

@@ -0,0 +1,4 @@
+-- Add up migration script here
+ALTER TYPE SCRIPT_LANG ADD VALUE IF NOT EXISTS 'mssql';
+-- Add up migration script here
+UPDATE config set config = '{"worker_tags": ["nativets", "postgresql", "mysql", "graphql", "snowflake", "bigquery", "mssql"]}'::jsonb where name = 'worker__native' and config = '{"worker_tags": ["nativets", "postgresql", "mysql", "graphql", "snowflake", "bigquery"]}'::jsonb ;

backend/migrations/20231113194945_add_dedicated_workers_to_flows.down.sql

@@ -0,0 +1 @@
+-- Add down migration script here

backend/migrations/20231113194945_add_dedicated_workers_to_flows.up.sql

@@ -0,0 +1,2 @@
+-- Add up migration script here
+ALTER TABLE flow ADD COLUMN dedicated_worker BOOLEAN;

backend/migrations/20231114161526_add_overquota.down.sql

@@ -0,0 +1 @@
+-- Add down migration script here

backend/migrations/20231114161526_add_overquota.up.sql

@@ -0,0 +1,2 @@
+-- Add up migration script here
+alter table workspace add column is_overquota boolean not null default false;

backend/migrations/20231115101254_workspace_s3_bucket_settings.down.sql

@@ -0,0 +1,2 @@
+-- Add down migration script here
+ALTER TABLE workspace_settings DROP COLUMN large_file_storage;

backend/migrations/20231115101254_workspace_s3_bucket_settings.up.sql

@@ -0,0 +1,2 @@
+-- Add up migration script here
+ALTER TABLE workspace_settings ADD COLUMN IF NOT EXISTS large_file_storage JSONB;

backend/migrations/20231122100516_relax_tag_varchars.down.sql

@@ -0,0 +1 @@
+-- Add down migration script here

backend/migrations/20231122100516_relax_tag_varchars.up.sql

@@ -0,0 +1,3 @@
+-- Add up migration script here
+ALTER TABLE queue ALTER COLUMN tag TYPE varchar(255);
+ALTER TABLE completed_job ALTER COLUMN tag TYPE varchar(255);

backend/parsers/windmill-parser-py-imports/src/lib.rs

@@ -44,7 +44,9 @@ static PYTHON_IMPORTS_REPLACEMENT: phf::Map<&'static str, &'static str> = phf_ma
     "tabula" => "tabula-py",
     "shapefile" => "pyshp",
     "sklearn" => "scikit-learn",
-    "umap" => "umap-learn"
+    "umap" => "umap-learn",
+    "cv2" => "opencv-python",
+    "atlassian" => "atlassian-python-api"
 };
 
 fn replace_import(x: String) -> String {
@@ -166,7 +168,7 @@ pub async fn parse_python_imports(
                 let code = sqlx::query_scalar!(
                     r#"
                     SELECT content FROM script WHERE path = $1 AND workspace_id = $2
-                    AND created_at = (SELECT max(created_at) FROM script WHERE path = $1 AND 
+                    AND created_at = (SELECT max(created_at) FROM script WHERE path = $1 AND
                     workspace_id = $2)
                     "#,
                     &rpath,

backend/parsers/windmill-parser-sql/src/lib.rs

@@ -47,6 +47,16 @@ pub fn parse_snowflake_sig(code: &str) -> anyhow::Result<MainArgSignature> {
     }
 }
 
+pub fn parse_mssql_sig(code: &str) -> anyhow::Result<MainArgSignature> {
+    let parsed = parse_mssql_file(&code)?;
+    if let Some(x) = parsed {
+        let args = x;
+        Ok(MainArgSignature { star_args: false, star_kwargs: false, args })
+    } else {
+        Err(anyhow!("Error parsing sql".to_string()))
+    }
+}
+
 lazy_static::lazy_static! {
     static ref RE_CODE_PGSQL: Regex = Regex::new(r#"(?m)\$(\d+)(?:::(\w+(?:\[\])?))?"#).unwrap();
 
@@ -60,6 +70,9 @@ lazy_static::lazy_static! {
 
     static ref RE_ARG_SNOWFLAKE: Regex = Regex::new(r#"(?m)^-- \? (\w+) \((\w+)\)(?: ?\= ?(.+))? *[\r\n$]"#).unwrap();
 
+
+    static ref RE_ARG_MSSQL: Regex = Regex::new(r#"(?m)^-- @(?:P|p)\d+ (\w+) \((\w+)\)(?: ?\= ?(.+))? *[\r\n$]"#).unwrap();
+
 }
 
 fn parse_mysql_file(code: &str) -> anyhow::Result<Option<Vec<Arg>>> {
@@ -200,6 +213,36 @@ fn parse_snowflake_file(code: &str) -> anyhow::Result<Option<Vec<Arg>>> {
     Ok(Some(args))
 }
 
+fn parse_mssql_file(code: &str) -> anyhow::Result<Option<Vec<Arg>>> {
+    let mut args: Vec<Arg> = vec![];
+
+    for cap in RE_ARG_MSSQL.captures_iter(code) {
+        let name = cap.get(1).map(|x| x.as_str().to_string()).unwrap();
+        let typ = cap
+            .get(2)
+            .map(|x| x.as_str().to_string().to_lowercase())
+            .unwrap();
+        let default = cap.get(3).map(|x| x.as_str().to_string());
+        let has_default = default.is_some();
+        let parsed_typ = parse_mssql_typ(typ.as_str());
+
+        let parsed_default = default.and_then(|x| match parsed_typ {
+            Typ::Int => x.parse::<i64>().ok().map(|x| json!(x)),
+            Typ::Float => x.parse::<f64>().ok().map(|x| json!(x)),
+            _ => Some(json!(x)),
+        });
+        args.push(Arg {
+            name,
+            typ: parsed_typ,
+            default: parsed_default,
+            otyp: Some(typ),
+            has_default,
+        });
+    }
+
+    Ok(Some(args))
+}
+
 pub fn parse_mysql_typ(typ: &str) -> Typ {
     match typ {
         "varchar" | "char" | "binary" | "varbinary" | "blob" | "text" | "enum" | "set" => {
@@ -269,6 +312,20 @@ pub fn parse_snowflake_typ(typ: &str) -> Typ {
     }
 }
 
+pub fn parse_mssql_typ(typ: &str) -> Typ {
+    match typ {
+        "char" | "text" | "varchar" | "nchar" | "nvarchar" | "ntext" => Typ::Str(None),
+        "binary" | "varbinary" | "image" => Typ::Bytes,
+        "date" | "datetime2" | "datetime" | "datetimeoffset" | "smalldatetime" | "time" => {
+            Typ::Datetime
+        }
+        "bigint" | "int" | "tinyint" | "smallint" => Typ::Int,
+        "float" | "real" | "numeric" | "decimal" => Typ::Float,
+        "bit" => Typ::Bool,
+        _ => Typ::Str(None),
+    }
+}
+
 #[cfg(test)]
 mod tests {
 

backend/parsers/windmill-parser-wasm/pkg/package.json

@@ -3,7 +3,7 @@
   "collaborators": [
     "Ruben Fiszel <ruben@windmill.dev>"
   ],
-  "version": "1.190.0",
+  "version": "1.202.1",
   "files": [
     "windmill_parser_wasm_bg.wasm",
     "windmill_parser_wasm.js",

backend/parsers/windmill-parser-wasm/pkg/windmill_parser_wasm.d.ts

@@ -54,6 +54,11 @@ export function parse_snowflake(code: string): string;
 * @param {string} code
 * @returns {string}
 */
+export function parse_mssql(code: string): string;
+/**
+* @param {string} code
+* @returns {string}
+*/
 export function parse_graphql(code: string): string;
 
 export type InitInput = RequestInfo | URL | Response | BufferSource | WebAssembly.Module;
@@ -70,6 +75,7 @@ export interface InitOutput {
   readonly parse_mysql: (a: number, b: number, c: number) => void;
   readonly parse_bigquery: (a: number, b: number, c: number) => void;
   readonly parse_snowflake: (a: number, b: number, c: number) => void;
+  readonly parse_mssql: (a: number, b: number, c: number) => void;
   readonly parse_graphql: (a: number, b: number, c: number) => void;
   readonly __wbindgen_malloc: (a: number, b: number) => number;
   readonly __wbindgen_realloc: (a: number, b: number, c: number, d: number) => number;

backend/parsers/windmill-parser-wasm/pkg/windmill_parser_wasm.js

@@ -97,15 +97,6 @@ function getInt32Memory0() {
     return cachedInt32Memory0;
 }
 
-function addHeapObject(obj) {
-    if (heap_next === heap.length) heap.push(heap.length + 1);
-    const idx = heap_next;
-    heap_next = heap[idx];
-
-    heap[idx] = obj;
-    return idx;
-}
-
 const cachedTextDecoder = (typeof TextDecoder !== 'undefined' ? new TextDecoder('utf-8', { ignoreBOM: true, fatal: true }) : { decode: () => { throw Error('TextDecoder not available') } } );
 
 if (typeof TextDecoder !== 'undefined') { cachedTextDecoder.decode(); };
@@ -115,6 +106,15 @@ function getStringFromWasm0(ptr, len) {
     return cachedTextDecoder.decode(getUint8Memory0().subarray(ptr, ptr + len));
 }
 
+function addHeapObject(obj) {
+    if (heap_next === heap.length) heap.push(heap.length + 1);
+    const idx = heap_next;
+    heap_next = heap[idx];
+
+    heap[idx] = obj;
+    return idx;
+}
+
 let cachedFloat64Memory0 = null;
 
 function getFloat64Memory0() {
@@ -427,6 +427,29 @@ export function parse_snowflake(code) {
     }
 }
 
+/**
+* @param {string} code
+* @returns {string}
+*/
+export function parse_mssql(code) {
+    let deferred2_0;
+    let deferred2_1;
+    try {
+        const retptr = wasm.__wbindgen_add_to_stack_pointer(-16);
+        const ptr0 = passStringToWasm0(code, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
+        const len0 = WASM_VECTOR_LEN;
+        wasm.parse_mssql(retptr, ptr0, len0);
+        var r0 = getInt32Memory0()[retptr / 4 + 0];
+        var r1 = getInt32Memory0()[retptr / 4 + 1];
+        deferred2_0 = r0;
+        deferred2_1 = r1;
+        return getStringFromWasm0(r0, r1);
+    } finally {
+        wasm.__wbindgen_add_to_stack_pointer(16);
+        wasm.__wbindgen_free(deferred2_0, deferred2_1, 1);
+    }
+}
+
 /**
 * @param {string} code
 * @returns {string}
@@ -503,6 +526,14 @@ function __wbg_get_imports() {
         getInt32Memory0()[arg0 / 4 + 1] = len1;
         getInt32Memory0()[arg0 / 4 + 0] = ptr1;
     };
+    imports.wbg.__wbg_eval_f742bccbdcf34b02 = function(arg0, arg1) {
+        const ret = eval(getStringFromWasm0(arg0, arg1));
+        return addHeapObject(ret);
+    };
+    imports.wbg.__wbindgen_error_new = function(arg0, arg1) {
+        const ret = new Error(getStringFromWasm0(arg0, arg1));
+        return addHeapObject(ret);
+    };
     imports.wbg.__wbindgen_boolean_get = function(arg0) {
         const v = getObject(arg0);
         const ret = typeof(v) === 'boolean' ? (v ? 1 : 0) : 2;
@@ -520,14 +551,6 @@ function __wbg_get_imports() {
         const ret = getObject(arg0) === getObject(arg1);
         return ret;
     };
-    imports.wbg.__wbindgen_bigint_from_u64 = function(arg0) {
-        const ret = BigInt.asUintN(64, arg0);
-        return addHeapObject(ret);
-    };
-    imports.wbg.__wbindgen_error_new = function(arg0, arg1) {
-        const ret = new Error(getStringFromWasm0(arg0, arg1));
-        return addHeapObject(ret);
-    };
     imports.wbg.__wbindgen_number_get = function(arg0, arg1) {
         const obj = getObject(arg1);
         const ret = typeof(obj) === 'number' ? obj : undefined;
@@ -543,19 +566,19 @@ function __wbg_get_imports() {
         const ret = getObject(arg0) in getObject(arg1);
         return ret;
     };
-    imports.wbg.__wbg_eval_d090ffdbb4b6658b = function(arg0, arg1) {
-        const ret = eval(getStringFromWasm0(arg0, arg1));
+    imports.wbg.__wbindgen_bigint_from_u64 = function(arg0) {
+        const ret = BigInt.asUintN(64, arg0);
         return addHeapObject(ret);
     };
     imports.wbg.__wbindgen_jsval_loose_eq = function(arg0, arg1) {
         const ret = getObject(arg0) == getObject(arg1);
         return ret;
     };
-    imports.wbg.__wbg_get_44be0491f933a435 = function(arg0, arg1) {
+    imports.wbg.__wbg_get_4a9aa5157afeb382 = function(arg0, arg1) {
         const ret = getObject(arg0)[arg1 >>> 0];
         return addHeapObject(ret);
     };
-    imports.wbg.__wbg_length_fff51ee6522a1a18 = function(arg0) {
+    imports.wbg.__wbg_length_cace2e0b3ddc0502 = function(arg0) {
         const ret = getObject(arg0).length;
         return ret;
     };
@@ -563,76 +586,76 @@ function __wbg_get_imports() {
         const ret = typeof(getObject(arg0)) === 'function';
         return ret;
     };
-    imports.wbg.__wbg_next_526fc47e980da008 = function(arg0) {
+    imports.wbg.__wbg_next_15da6a3df9290720 = function(arg0) {
         const ret = getObject(arg0).next;
         return addHeapObject(ret);
     };
-    imports.wbg.__wbg_next_ddb3312ca1c4e32a = function() { return handleError(function (arg0) {
+    imports.wbg.__wbg_next_1989a20442400aaa = function() { return handleError(function (arg0) {
         const ret = getObject(arg0).next();
         return addHeapObject(ret);
     }, arguments) };
-    imports.wbg.__wbg_done_5c1f01fb660d73b5 = function(arg0) {
+    imports.wbg.__wbg_done_bc26bf4ada718266 = function(arg0) {
         const ret = getObject(arg0).done;
         return ret;
     };
-    imports.wbg.__wbg_value_1695675138684bd5 = function(arg0) {
+    imports.wbg.__wbg_value_0570714ff7d75f35 = function(arg0) {
         const ret = getObject(arg0).value;
         return addHeapObject(ret);
     };
-    imports.wbg.__wbg_iterator_97f0c81209c6c35a = function() {
+    imports.wbg.__wbg_iterator_7ee1a391d310f8e4 = function() {
         const ret = Symbol.iterator;
         return addHeapObject(ret);
     };
-    imports.wbg.__wbg_get_97b561fb56f034b5 = function() { return handleError(function (arg0, arg1) {
+    imports.wbg.__wbg_get_2aff440840bb6202 = function() { return handleError(function (arg0, arg1) {
         const ret = Reflect.get(getObject(arg0), getObject(arg1));
         return addHeapObject(ret);
     }, arguments) };
-    imports.wbg.__wbg_call_cb65541d95d71282 = function() { return handleError(function (arg0, arg1) {
+    imports.wbg.__wbg_call_669127b9d730c650 = function() { return handleError(function (arg0, arg1) {
         const ret = getObject(arg0).call(getObject(arg1));
         return addHeapObject(ret);
     }, arguments) };
-    imports.wbg.__wbg_isArray_4c24b343cb13cfb1 = function(arg0) {
+    imports.wbg.__wbg_isArray_38525be7442aa21e = function(arg0) {
         const ret = Array.isArray(getObject(arg0));
         return ret;
     };
-    imports.wbg.__wbg_instanceof_ArrayBuffer_39ac22089b74fddb = function(arg0) {
+    imports.wbg.__wbg_instanceof_ArrayBuffer_c7cc317e5c29cc0d = function(arg0) {
         let result;
         try {
             result = getObject(arg0) instanceof ArrayBuffer;
-        } catch {
+        } catch (_) {
             result = false;
         }
         const ret = result;
         return ret;
     };
-    imports.wbg.__wbg_isSafeInteger_bb8e18dd21c97288 = function(arg0) {
+    imports.wbg.__wbg_isSafeInteger_c38b0a16d0c7cef7 = function(arg0) {
         const ret = Number.isSafeInteger(getObject(arg0));
         return ret;
     };
-    imports.wbg.__wbg_entries_e51f29c7bba0c054 = function(arg0) {
+    imports.wbg.__wbg_entries_6d727b73ee02b7ce = function(arg0) {
         const ret = Object.entries(getObject(arg0));
         return addHeapObject(ret);
     };
-    imports.wbg.__wbg_buffer_085ec1f694018c4f = function(arg0) {
+    imports.wbg.__wbg_buffer_344d9b41efe96da7 = function(arg0) {
         const ret = getObject(arg0).buffer;
         return addHeapObject(ret);
     };
-    imports.wbg.__wbg_new_8125e318e6245eed = function(arg0) {
+    imports.wbg.__wbg_new_d8a000788389a31e = function(arg0) {
         const ret = new Uint8Array(getObject(arg0));
         return addHeapObject(ret);
     };
-    imports.wbg.__wbg_set_5cf90238115182c3 = function(arg0, arg1, arg2) {
+    imports.wbg.__wbg_set_dcfd613a3420f908 = function(arg0, arg1, arg2) {
         getObject(arg0).set(getObject(arg1), arg2 >>> 0);
     };
-    imports.wbg.__wbg_length_72e2208bbc0efc61 = function(arg0) {
+    imports.wbg.__wbg_length_a5587d6cd79ab197 = function(arg0) {
         const ret = getObject(arg0).length;
         return ret;
     };
-    imports.wbg.__wbg_instanceof_Uint8Array_d8d9cb2b8e8ac1d4 = function(arg0) {
+    imports.wbg.__wbg_instanceof_Uint8Array_19e6f142a5e7e1e1 = function(arg0) {
         let result;
         try {
             result = getObject(arg0) instanceof Uint8Array;
-        } catch {
+        } catch (_) {
             result = false;
         }
         const ret = result;

backend/parsers/windmill-parser-wasm/pkg/windmill_parser_wasm_bg.wasm.d.ts

@@ -11,6 +11,7 @@ export function parse_sql(a: number, b: number, c: number): void;
 export function parse_mysql(a: number, b: number, c: number): void;
 export function parse_bigquery(a: number, b: number, c: number): void;
 export function parse_snowflake(a: number, b: number, c: number): void;
+export function parse_mssql(a: number, b: number, c: number): void;
 export function parse_graphql(a: number, b: number, c: number): void;
 export function __wbindgen_malloc(a: number, b: number): number;
 export function __wbindgen_realloc(a: number, b: number, c: number, d: number): number;

backend/parsers/windmill-parser-wasm/src/lib.rs

@@ -67,6 +67,11 @@ pub fn parse_snowflake(code: &str) -> String {
     wrap_sig(windmill_parser_sql::parse_snowflake_sig(code))
 }
 
+#[wasm_bindgen]
+pub fn parse_mssql(code: &str) -> String {
+    wrap_sig(windmill_parser_sql::parse_mssql_sig(code))
+}
+
 #[wasm_bindgen]
 pub fn parse_graphql(code: &str) -> String {
     wrap_sig(windmill_parser_graphql::parse_graphql_sig(code))

backend/src/main.rs

@@ -9,7 +9,6 @@
 use gethostname::gethostname;
 use git_version::git_version;
 use rand::Rng;
-use serde::Deserialize;
 use sqlx::{postgres::PgListener, Pool, Postgres};
 use std::{
     net::{IpAddr, Ipv4Addr, SocketAddr},
@@ -26,10 +25,11 @@ use windmill_common::{
         BASE_URL_SETTING, CUSTOM_TAGS_SETTING, DISABLE_STATS_SETTING, ENV_SETTINGS,
         EXPOSE_DEBUG_METRICS_SETTING, EXPOSE_METRICS_SETTING, EXTRA_PIP_INDEX_URL_SETTING,
         KEEP_JOB_DIR_SETTING, LICENSE_KEY_SETTING, NPM_CONFIG_REGISTRY_SETTING, OAUTH_SETTING,
-        REQUEST_SIZE_LIMIT_SETTING, RETENTION_PERIOD_SECS_SETTING,
+        REQUEST_SIZE_LIMIT_SETTING, REQUIRE_PREEXISTING_USER_FOR_OAUTH_SETTING,
+        RETENTION_PERIOD_SECS_SETTING,
     },
     stats::schedule_stats,
-    utils::rd_string,
+    utils::{rd_string, Mode},
     worker::{reload_custom_tags_setting, WORKER_GROUP},
     DB, METRICS_ADDR, METRICS_ENABLED,
 };
@@ -41,9 +41,10 @@ use windmill_worker::{
 };
 
 use crate::monitor::{
-    initial_load, load_keep_job_dir, monitor_db, monitor_pool, reload_base_url_setting,
-    reload_extra_pip_index_url_setting, reload_license_key, reload_npm_config_registry_setting,
-    reload_retention_period_setting, reload_server_config, reload_worker_config,
+    initial_load, load_keep_job_dir, load_require_preexisting_user, monitor_db, monitor_pool,
+    reload_base_url_setting, reload_extra_pip_index_url_setting, reload_license_key,
+    reload_npm_config_registry_setting, reload_retention_period_setting, reload_server_config,
+    reload_worker_config,
 };
 
 const GIT_VERSION: &str = git_version!(args = ["--tag", "--always"], fallback = "unknown-version");
@@ -57,14 +58,6 @@ mod monitor;
 #[cfg(feature = "pg_embed")]
 mod pg_embed;
 
-#[derive(Deserialize, PartialEq)]
-#[serde(rename_all = "lowercase")]
-pub enum Mode {
-    Worker,
-    Server,
-    Standalone,
-}
-
 #[tokio::main]
 async fn main() -> anyhow::Result<()> {
     dotenv::dotenv().ok();
@@ -104,6 +97,15 @@ async fn main() -> anyhow::Result<()> {
             } else if &x == "worker" {
                 tracing::info!("Binary is in 'worker' mode");
                 Mode::Worker
+            } else if &x == "agent" {
+                tracing::info!("Binary is in 'agent' mode");
+                if std::env::var("BASE_INTERNAL_URL").is_err() {
+                    panic!("BASE_INTERNAL_URL is required in agent mode")
+                }
+                if std::env::var("JOB_TOKEN").is_err() {
+                    tracing::warn!("JOB_TOKEN is not passed, hence workers will still create one ephemeral token per job and the DATABASE_URL need to be of a role that can INSERT into the token table")
+                }
+                Mode::Agent
             } else {
                 if &x != "standalone" {
                     tracing::error!("mode not recognized, defaulting to standalone: {x}");
@@ -137,7 +139,7 @@ async fn main() -> anyhow::Result<()> {
         .ok()
         .and_then(|x| x.parse::<bool>().ok())
         .unwrap_or(false)
-        && mode != Mode::Worker;
+        && (mode == Mode::Server || mode == Mode::Standalone);
 
     let server_bind_address: IpAddr = if server_mode {
         std::env::var("SERVER_BIND_ADDR")
@@ -198,31 +200,36 @@ async fn main() -> anyhow::Result<()> {
         None
     };
 
-    let last_mig_version =
-        sqlx::query_scalar!("select version from _sqlx_migrations order by version desc limit 1;")
-            .fetch_optional(&db)
-            .await
-            .ok()
-            .flatten();
+    let is_agent = mode == Mode::Agent;
 
-    tracing::info!(
+    if !is_agent {
+        let last_mig_version = sqlx::query_scalar!(
+            "select version from _sqlx_migrations order by version desc limit 1;"
+        )
+        .fetch_optional(&db)
+        .await
+        .ok()
+        .flatten();
+
+        tracing::info!(
         "Last migration version: {last_mig_version:?}. Starting potential migration of the db if first connection on a new windmill version (can take a while depending on the migration) ...",
     );
 
-    // migration code to avoid break
-    windmill_api::migrate_db(&db).await?;
+        // migration code to avoid break
+        windmill_api::migrate_db(&db).await?;
 
-    let last_mig_version =
-        sqlx::query_scalar!("select version from _sqlx_migrations order by version desc limit 1;")
-            .fetch_optional(&db)
-            .await
-            .ok()
-            .flatten();
-
-    tracing::info!(
-        "Completed potential migration of the db. Last migration version: {last_mig_version:?}",
-    );
+        let last_mig_version = sqlx::query_scalar!(
+            "select version from _sqlx_migrations order by version desc limit 1;"
+        )
+        .fetch_optional(&db)
+        .await
+        .ok()
+        .flatten();
 
+        tracing::info!(
+            "Completed potential migration of the db. Last migration version: {last_mig_version:?}",
+        );
+    }
     let (tx, rx) = tokio::sync::broadcast::channel::<()>(3);
     let shutdown_signal = windmill_common::shutdown_signal(tx.clone(), rx.resubscribe());
 
@@ -255,8 +262,18 @@ Windmill Community Edition {GIT_VERSION}
             port_var.unwrap_or(0)
         };
 
+        let default_base_internal_url = format!("http://localhost:{}", port.to_string());
         // since it's only on server mode, the port is statically defined
-        let base_internal_url: String = format!("http://localhost:{}", port.to_string());
+        let base_internal_url: String = if let Ok(base_url) = std::env::var("BASE_INTERNAL_URL") {
+            if !is_agent {
+                tracing::warn!("BASE_INTERNAL_URL is now unecessary and ignored unless the mode is 'agent', you can remove it.");
+                default_base_internal_url.clone()
+            } else {
+                base_url
+            }
+        } else {
+            default_base_internal_url.clone()
+        };
 
         initial_load(&db, tx.clone(), worker_mode, server_mode).await;
 
@@ -264,14 +281,10 @@ Windmill Community Edition {GIT_VERSION}
 
         monitor_pool(&db).await;
 
-        if std::env::var("BASE_INTERNAL_URL").is_ok() {
-            tracing::warn!("BASE_INTERNAL_URL is now unecessary and ignored, you can remove it.");
-        }
-
         let addr = SocketAddr::from((server_bind_address, port));
 
         let rsmq2 = rsmq.clone();
-        let (port_tx, port_rx) = tokio::sync::oneshot::channel::<u16>();
+        let (base_internal_tx, base_internal_rx) = tokio::sync::oneshot::channel::<String>();
 
         DirBuilder::new()
             .recursive(true)
@@ -280,21 +293,28 @@ Windmill Community Edition {GIT_VERSION}
             .expect("could not create initial server dir");
 
         let server_f = async {
-            windmill_api::run_server(
-                db.clone(),
-                rsmq2,
-                addr,
-                rx.resubscribe(),
-                port_tx,
-                server_mode,
-            )
-            .await?;
+            if !is_agent {
+                windmill_api::run_server(
+                    db.clone(),
+                    rsmq2,
+                    addr,
+                    rx.resubscribe(),
+                    base_internal_tx,
+                    server_mode,
+                )
+                .await?;
+            } else {
+                base_internal_tx
+                    .send(base_internal_url.clone())
+                    .map_err(|e| {
+                        anyhow::anyhow!("Could not send base_internal_url to agent: {e}")
+                    })?;
+            }
             Ok(()) as anyhow::Result<()>
         };
 
         let workers_f = async {
-            let port = port_rx.await?;
-            let base_internal_url: String = format!("http://localhost:{}", port.to_string());
+            let base_internal_url = base_internal_rx.await?;
             if worker_mode {
                 run_workers(
                     db.clone(),
@@ -387,6 +407,9 @@ Windmill Community Edition {GIT_VERSION}
                                                 },
                                                 KEEP_JOB_DIR_SETTING => {
                                                     load_keep_job_dir(&db).await;
+                                                },
+                                                REQUIRE_PREEXISTING_USER_FOR_OAUTH_SETTING => {
+                                                    load_require_preexisting_user(&db).await;
                                                 }
                                                 EXPOSE_METRICS_SETTING | EXPOSE_DEBUG_METRICS_SETTING => {
                                                     if n.payload() != EXPOSE_DEBUG_METRICS_SETTING || worker_mode {
@@ -455,10 +478,8 @@ Windmill Community Edition {GIT_VERSION}
             Ok(()) as anyhow::Result<()>
         };
 
-        if mode == Mode::Server || mode == Mode::Standalone {
-            let instance_name = rd_string(8);
-            schedule_stats(&db, instance_name, &HTTP_CLIENT).await;
-        }
+        let instance_name = rd_string(8);
+        schedule_stats(instance_name, mode, &db, &HTTP_CLIENT).await;
 
         futures::try_join!(shutdown_signal, server_f, metrics_f, workers_f, monitor_f)?;
     } else {

backend/src/monitor.rs

@@ -24,9 +24,10 @@ use windmill_common::{
         BASE_URL_SETTING, EXPOSE_DEBUG_METRICS_SETTING, EXPOSE_METRICS_SETTING,
         EXTRA_PIP_INDEX_URL_SETTING, KEEP_JOB_DIR_SETTING, LICENSE_KEY_SETTING,
         NPM_CONFIG_REGISTRY_SETTING, OAUTH_SETTING, REQUEST_SIZE_LIMIT_SETTING,
-        RETENTION_PERIOD_SECS_SETTING,
+        REQUIRE_PREEXISTING_USER_FOR_OAUTH_SETTING, RETENTION_PERIOD_SECS_SETTING,
     },
     jobs::{JobKind, QueuedJob},
+    oauth2::REQUIRE_PREEXISTING_USER_FOR_OAUTH,
     server::load_server_config,
     users::truncate_token,
     worker::{load_worker_config, reload_custom_tags_setting, SERVER_CONFIG, WORKER_CONFIG},
@@ -92,6 +93,10 @@ pub async fn initial_load(
         tracing::error!("Error loading expose debug metrics: {e}");
     }
 
+    if server_mode {
+        load_require_preexisting_user(db).await;
+    }
+
     if worker_mode {
         load_keep_job_dir(db).await;
     }
@@ -163,13 +168,13 @@ pub async fn load_metrics_debug_enabled(db: &DB) -> error::Result<()> {
     Ok(())
 }
 pub async fn load_keep_job_dir(db: &DB) {
-    let metrics_enabled = sqlx::query_scalar!(
+    let value = sqlx::query_scalar!(
         "SELECT value FROM global_settings WHERE name = $1",
         KEEP_JOB_DIR_SETTING
     )
     .fetch_optional(db)
     .await;
-    match metrics_enabled {
+    match value {
         Ok(Some(serde_json::Value::Bool(t))) => KEEP_JOB_DIR.store(t, Ordering::Relaxed),
         Err(e) => {
             tracing::error!("Error loading keep job dir metrics: {e}");
@@ -178,6 +183,24 @@ pub async fn load_keep_job_dir(db: &DB) {
     };
 }
 
+pub async fn load_require_preexisting_user(db: &DB) {
+    let value = sqlx::query_scalar!(
+        "SELECT value FROM global_settings WHERE name = $1",
+        REQUIRE_PREEXISTING_USER_FOR_OAUTH_SETTING
+    )
+    .fetch_optional(db)
+    .await;
+    match value {
+        Ok(Some(serde_json::Value::Bool(t))) => {
+            REQUIRE_PREEXISTING_USER_FOR_OAUTH.store(t, Ordering::Relaxed)
+        }
+        Err(e) => {
+            tracing::error!("Error loading keep job dir metrics: {e}");
+        }
+        _ => (),
+    };
+}
+
 pub async fn delete_expired_items(db: &DB) -> () {
     let tokens_deleted_r: std::result::Result<Vec<String>, _> = sqlx::query_scalar(
         "DELETE FROM token WHERE expiration <= now()
@@ -525,15 +548,17 @@ pub async fn reload_worker_config(
     tx: tokio::sync::broadcast::Sender<()>,
     kill_if_change: bool,
 ) {
-    let config = load_worker_config(&db).await;
+    let config = load_worker_config(&db, tx.clone()).await;
     if let Err(e) = config {
         tracing::error!("Error reloading worker config: {:?}", e)
     } else {
         let wc = WORKER_CONFIG.read().await;
         let config = config.unwrap();
-        if *wc != config {
+        if *wc != config || config.dedicated_worker.is_some() {
             if kill_if_change {
-                if (*wc).dedicated_worker != config.dedicated_worker {
+                if config.dedicated_worker.is_some()
+                    || (*wc).dedicated_worker != config.dedicated_worker
+                {
                     tracing::info!("Dedicated worker config changed, sending killpill. Expecting to be restarted by supervisor.");
                     let _ = tx.send(());
                 }

backend/tests/worker.rs

@@ -124,7 +124,7 @@ impl ApiServer {
 
         let addr = sock.local_addr().unwrap();
         drop(sock);
-        let (port_tx, _port_rx) = tokio::sync::oneshot::channel::<u16>();
+        let (port_tx, _port_rx) = tokio::sync::oneshot::channel::<String>();
 
         let task = tokio::task::spawn(windmill_api::run_server(
             db.clone(),
@@ -961,7 +961,7 @@ fn spawn_test_worker(
     tokio::sync::broadcast::Sender<()>,
     tokio::task::JoinHandle<()>,
 ) {
-    for x in [windmill_worker::LOCK_CACHE_DIR] {
+    for x in [windmill_worker::LOCK_CACHE_DIR, windmill_worker::GO_BIN_CACHE_DIR] {
         std::fs::DirBuilder::new()
             .recursive(true)
             .create(x)
@@ -2578,6 +2578,7 @@ async fn test_flow_lock_all(db: Pool<Postgres>) {
                     tag: None,
                     ws_error_handler_muted: None,
                     priority: None,
+                    dedicated_worker: None,
                 },
                 draft_only: None,
             },

backend/windmill-api/Cargo.toml

@@ -81,4 +81,9 @@ hf-hub.workspace = true
 tokenizers.workspace = true
 candle-core.workspace = true
 candle-transformers.workspace = true
-candle-nn.workspace = true
+candle-nn.workspace = true
+aws-sdk-s3 = "0.36.0"
+aws-config = "1.0.0"
+polars = { version = "0.35.2", features = ["lazy", "parquet", "aws", "csv", "dtype-full"] }
+polars-io = { version = "0.35.2", features = ["csv"] }
+object_store = { version = "0.8.0", features = ["aws"] }

backend/windmill-api/openapi.yaml

@@ -1,7 +1,7 @@
 openapi: "3.0.3"
 
 info:
-  version: 1.202.1
+  version: 1.216.0
   title: Windmill API
 
   contact:
@@ -731,6 +731,20 @@ paths:
               schema:
                 type: string
 
+  /users/leave_instance:
+    post:
+      summary: leave instance
+      operationId: leaveInstance
+      tags:
+        - user
+      responses:
+        "200":
+          description: status
+          content:
+            text/plain:
+              schema:
+                type: string
+
   /users/usage:
     get:
       summary: get current usage outside of premium workspaces
@@ -827,22 +841,6 @@ paths:
               schema:
                 $ref: "#/components/schemas/User"
 
-  /w/{workspace}/users/leave_workspace:
-    post:
-      summary: leave workspace
-      operationId: leaveWorkspace
-      tags:
-        - user
-      parameters:
-        - $ref: "#/components/parameters/WorkspaceId"
-      responses:
-        "200":
-          description: status
-          content:
-            text/plain:
-              schema:
-                type: string
-
   /users/accept_invite:
     post:
       summary: accept invite to workspace
@@ -1051,6 +1049,22 @@ paths:
               schema:
                 type: string
 
+  /w/{workspace}/workspaces/leave:
+    post:
+      summary: leave workspace
+      operationId: leaveWorkspace
+      tags:
+        - workspace
+      parameters:
+        - $ref: "#/components/parameters/WorkspaceId"
+      responses:
+        "200":
+          description: status
+          content:
+            text/plain:
+              schema:
+                type: string
+
   /w/{workspace}/users/whois/{username}:
     get:
       summary: whois
@@ -1176,6 +1190,8 @@ paths:
                     $ref: "#/components/schemas/ScriptArgs"
                   error_handler_muted_on_cancel:
                     type: boolean
+                  large_file_storage:
+                    $ref: "#/components/schemas/LargeFileStorage"
                 required:
                   - code_completion_enabled
 
@@ -1199,6 +1215,23 @@ paths:
                   deploy_to:
                     type: string
 
+  /w/{workspace}/workspaces/is_premium:
+    get:
+      summary: get if workspace is premium
+      operationId: getIsPremium
+      tags:
+        - workspace
+      parameters:
+        - $ref: "#/components/parameters/WorkspaceId"
+
+      responses:
+        "200":
+          description: status
+          content:
+            application/json:
+              schema:
+                type: boolean
+
   /w/{workspace}/workspaces/premium_info:
     get:
       summary: get premium info
@@ -1220,6 +1253,8 @@ paths:
                     type: boolean
                   usage:
                     type: number
+                  seats:
+                    type: number
                 required:
                   - premium
 
@@ -1328,6 +1363,8 @@ paths:
               properties:
                 operator:
                   type: boolean
+                invite_all:
+                  type: boolean
 
       responses:
         "200":
@@ -1450,6 +1487,49 @@ paths:
               schema:
                 type: string
 
+  /w/{workspace}/workspaces/edit_large_file_storage_config:
+    post:
+      summary: edit large file storage settings
+      operationId: editLargeFileStorageConfig
+      tags:
+        - workspace
+      parameters:
+        - $ref: "#/components/parameters/WorkspaceId"
+      requestBody:
+        description: LargeFileStorage info
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              properties:
+                large_file_storage:
+                  $ref: "#/components/schemas/LargeFileStorage"
+
+      responses:
+        "200":
+          description: status
+          content:
+            application/json:
+              schema: {}
+
+  /w/{workspace}/workspaces/get_large_file_storage_config:
+    get:
+      summary: get large file storage config
+      operationId: getLargeFileStorageConfig
+      tags:
+        - workspace
+      parameters:
+        - $ref: "#/components/parameters/WorkspaceId"
+
+      responses:
+        "200":
+          description: status
+          content:
+            application/json:
+              schema:
+                $ref: "#/components/schemas/LargeFileStorage"
+
   /w/{workspace}/users/list:
     get:
       summary: list users
@@ -5332,7 +5412,6 @@ paths:
               required:
                 - handler_type
                 - override_existing
-                - path
       responses:
         "201":
           description: default error handler set
@@ -6213,6 +6292,189 @@ paths:
                 type: string
                 format: uuid
 
+  /w/{workspace}/job_helpers/duckdb_connection_settings:
+    post:
+      summary:
+        Converts an S3 resource to the set of instructions necessary to connect
+        DuckDB to an S3 bucket
+      operationId: duckdbConnectionSettings
+      tags:
+        - helpers
+      parameters:
+        - $ref: "#/components/parameters/WorkspaceId"
+      requestBody:
+        description: S3 resource to connect to
+        required: true
+        content:
+          application/json:
+            schema:
+              s3_resource:
+                $ref: "#/components/schemas/S3Resource"
+      responses:
+        "200":
+          description: Connection settings
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  connection_settings_str:
+                    type: string
+  /w/{workspace}/job_helpers/polars_connection_settings:
+    post:
+      summary:
+        Converts an S3 resource to the set of arguments necessary to connect
+        Polars to an S3 bucket
+      operationId: polarsConnectionSettings
+      tags:
+        - helpers
+      parameters:
+        - $ref: "#/components/parameters/WorkspaceId"
+      requestBody:
+        description: S3 resource to connect to
+        required: true
+        content:
+          application/json:
+            schema:
+              s3_resource:
+                $ref: "#/components/schemas/S3Resource"
+      responses:
+        "200":
+          description: Connection settings
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  endpoint_url:
+                    type: string
+                  key:
+                    type: string
+                  secret:
+                    type: string
+                  use_ssl:
+                    type: boolean
+                  cache_regions:
+                    type: boolean
+                  client_kwargs:
+                    $ref: "#/components/schemas/PolarsClientKwargs"
+                required:
+                  - endpoint_url
+                  - use_ssl
+                  - cache_regions
+                  - client_kwargs
+
+  /w/{workspace}/job_helpers/test_connection:
+    get:
+      summary: Test connection to the workspace datasets storage
+      operationId: datasetStorageTestConnection
+      tags:
+        - helpers
+      parameters:
+        - $ref: "#/components/parameters/WorkspaceId"
+      responses:
+        "200":
+          description: Connection settings
+          content:
+            application/json:
+              schema: {}
+
+  /w/{workspace}/job_helpers/list_stored_files:
+    get:
+      summary: List the file keys available in the worspace files storage (S3)
+      operationId: listStoredFiles
+      tags:
+        - helpers
+      parameters:
+        - $ref: "#/components/parameters/WorkspaceId"
+        - name: max_keys
+          in: query
+          required: true
+          schema:
+            type: integer
+        - name: marker
+          in: query
+          schema:
+            type: string
+      responses:
+        "200":
+          description: List of file keys
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  next_marker:
+                    type: string
+                  windmill_large_files:
+                    type: array
+                    items:
+                      $ref: "#/components/schemas/WindmillLargeFile"
+                required:
+                  - windmill_large_files
+
+  /w/{workspace}/job_helpers/load_file_metadata:
+    get:
+      summary: Load metadata of the file
+      operationId: loadFileMetadata
+      tags:
+        - helpers
+      parameters:
+        - $ref: "#/components/parameters/WorkspaceId"
+        - name: file_key
+          in: query
+          required: true
+          schema:
+            type: string
+      responses:
+        "200":
+          description: FileMetadata
+          content:
+            application/json:
+              schema:
+                $ref: "#/components/schemas/WindmillFileMetadata"
+
+  /w/{workspace}/job_helpers/load_file_preview:
+    get:
+      summary: Load a preview of the file
+      operationId: loadFilePreview
+      tags:
+        - helpers
+      parameters:
+        - $ref: "#/components/parameters/WorkspaceId"
+        - name: file_key
+          in: query
+          required: true
+          schema:
+            type: string
+        - name: file_size_in_bytes
+          in: query
+          schema:
+            type: integer
+        - name: file_mime_type
+          in: query
+          schema:
+            type: string
+        - name: csv_separator
+          in: query
+          schema:
+            type: string
+        - name: read_bytes_from
+          in: query
+          schema:
+            type: integer
+        - name: read_bytes_length
+          in: query
+          schema:
+            type: integer
+      responses:
+        "200":
+          description: FilePreview
+          content:
+            application/json:
+              schema:
+                $ref: "#/components/schemas/WindmillFilePreview"
+
 components:
   securitySchemes:
     bearerAuth:
@@ -6588,6 +6850,7 @@ components:
               mysql,
               bigquery,
               snowflake,
+              mssql,
               graphql,
               nativets,
               bun,
@@ -6669,6 +6932,7 @@ components:
               mysql,
               bigquery,
               snowflake,
+              mssql,
               graphql,
               nativets,
               bun,
@@ -6858,6 +7122,7 @@ components:
               mysql,
               bigquery,
               snowflake,
+              mssql,
               graphql,
               nativets,
               bun,
@@ -6965,6 +7230,7 @@ components:
               mysql,
               bigquery,
               snowflake,
+              mssql,
               graphql,
               nativets,
               bun,
@@ -7479,6 +7745,7 @@ components:
               mysql,
               bigquery,
               snowflake,
+              mssql,
               graphql,
               nativets,
               bun,
@@ -7958,6 +8225,8 @@ components:
           type: boolean
         priority:
           type: integer
+        dedicated_worker:
+          type: boolean
       required:
         - path
         - edited_by
@@ -7978,6 +8247,8 @@ components:
               type: boolean
             priority:
               type: integer
+            dedicated_worker:
+              type: boolean
           required:
             - path
 
@@ -8183,3 +8454,81 @@ components:
       schema:
         type: string
         enum: [script, failure, trigger, approval]
+
+    PolarsClientKwargs:
+      type: object
+      properties:
+        region_name:
+          type: string
+      required:
+        - region_name
+
+    LargeFileStorage:
+      type: object
+      properties:
+        type:
+          type: string
+          enum: ["S3Storage"]
+        s3_resource_path:
+          type: string
+
+    WindmillLargeFile:
+      type: object
+      properties:
+        s3:
+          type: string
+      required:
+        - s3
+
+    WindmillFileMetadata:
+      type: object
+      properties:
+        mime_type:
+          type: string
+        size_in_bytes:
+          type: integer
+        last_modified:
+          type: string
+          format: date-time
+        expires:
+          type: string
+          format: date-time
+        version_id:
+          type: string
+
+    WindmillFilePreview:
+      type: object
+      properties:
+        msg:
+          type: string
+        content:
+          type: string
+        content_type:
+          type: string
+          enum: ["RawText", "Csv", "Parquet", "Unknown"]
+      required:
+        - content_type
+
+    S3Resource:
+      type: object
+      properties:
+        bucket:
+          type: string
+        region:
+          type: string
+        endPoint:
+          type: string
+        useSSL:
+          type: boolean
+        accessKey:
+          type: string
+        secretKey:
+          type: string
+        pathStyle:
+          type: boolean
+      required:
+        - bucket
+        - region
+        - endPoint
+        - useSSL
+        - pathStyle

backend/windmill-api/src/embeddings.rs

@@ -1,6 +1,6 @@
 use std::{collections::HashMap, path::PathBuf, sync::Arc};
 
-use anyhow::{self, Error, Result};
+use anyhow::{anyhow, Error, Result};
 use axum::{
     extract::{Path, Query},
     routing::get,
@@ -228,14 +228,31 @@ impl EmbeddingsDb {
         self.db
             .create_collection("resource_types".to_string(), 384, Distance::Cosine)?;
 
-        let response = http_get_from_hub(
-            &HTTP_CLIENT,
-            "https://hub.windmill.dev/scripts/embeddings",
-            false,
-            None,
-            pg_db,
-        )
-        .await?;
+        let response = HTTP_CLIENT
+            .get("https://bucket.windmillhub.com/embeddings/scripts_embeddings.json")
+            .send()
+            .await;
+        let response =
+            if response.is_err() || response.as_ref().unwrap().error_for_status_ref().is_err() {
+                tracing::warn!("Failed to get scripts embeddings from bucket, trying hub...");
+                http_get_from_hub(
+                    &HTTP_CLIENT,
+                    "https://hub.windmill.dev/scripts/embeddings",
+                    false,
+                    None,
+                    pg_db,
+                )
+                .await?
+            } else {
+                response.unwrap()
+            };
+        if response.error_for_status_ref().is_err() {
+            return Err(anyhow!(
+                "Failed to get scripts embeddings from hub with error code: {}",
+                response.status()
+            ));
+        }
+
         let hub_scripts = response.json::<Vec<HubScript>>().await?;
 
         for script in &hub_scripts {
@@ -257,14 +274,31 @@ impl EmbeddingsDb {
             self.db.insert_into_collection("scripts", embedding)?;
         }
 
-        let response = http_get_from_hub(
-            &HTTP_CLIENT,
-            "https://hub.windmill.dev/resource_types/embeddings",
-            false,
-            None,
-            pg_db,
-        )
-        .await?;
+        let response = HTTP_CLIENT
+            .get("https://bucket.windmillhub.com/embeddings/resource_types_embeddings.json")
+            .send()
+            .await;
+        let response = if response.is_err()
+            || response.as_ref().unwrap().error_for_status_ref().is_err()
+        {
+            tracing::warn!("Failed to get resource types embeddings from bucket, trying hub...");
+            http_get_from_hub(
+                &HTTP_CLIENT,
+                "https://hub.windmill.dev/resource_types/embeddings",
+                false,
+                None,
+                pg_db,
+            )
+            .await?
+        } else {
+            response.unwrap()
+        };
+        if response.error_for_status_ref().is_err() {
+            return Err(anyhow!(
+                "Failed to get resource types embeddings from hub with error code: {}",
+                response.status()
+            ));
+        }
         let hub_resource_types = response.json::<Vec<HubResourceType>>().await?;
 
         let resource_types: Vec<ResourceType> =

backend/windmill-api/src/flows.rs

@@ -315,7 +315,7 @@ async fn create_flow(
 
     sqlx::query!(
         "INSERT INTO flow (workspace_id, path, summary, description, value, edited_by, edited_at, \
-         schema, dependency_job, draft_only, tag) VALUES ($1, $2, $3, $4, $5, $6, now(), $7::text::json, NULL, $8, $9)",
+         schema, dependency_job, draft_only, tag, dedicated_worker) VALUES ($1, $2, $3, $4, $5, $6, now(), $7::text::json, NULL, $8, $9, $10)",
         w_id,
         nf.path,
         nf.summary,
@@ -324,7 +324,8 @@ async fn create_flow(
         &authed.username,
         nf.schema.and_then(|x| serde_json::to_string(&x.0).ok()),
         nf.draft_only,
-        nf.tag
+        nf.tag,
+        nf.dedicated_worker
     )
     .execute(&mut tx)
     .await?;
@@ -358,7 +359,10 @@ async fn create_flow(
         &db,
         tx,
         &w_id,
-        JobPayload::FlowDependencies { path: nf.path.clone() },
+        JobPayload::FlowDependencies {
+            path: nf.path.clone(),
+            dedicated_worker: nf.dedicated_worker,
+        },
         PushArgs::empty(),
         &authed.username,
         &authed.email,
@@ -472,7 +476,8 @@ async fn update_flow(
     let old_dep_job = not_found_if_none(old_dep_job, "Flow", flow_path)?;
     sqlx::query!(
         "UPDATE flow SET path = $1, summary = $2, description = $3, value = $4, edited_by = $5, \
-         edited_at = now(), schema = $6::text::json, dependency_job = NULL, draft_only = NULL, tag = $9 WHERE path = $7 AND workspace_id = $8",
+         edited_at = now(), schema = $6::text::json, dependency_job = NULL, draft_only = NULL, tag = $9, dedicated_worker = $10
+        WHERE path = $7 AND workspace_id = $8",
         nf.path,
         nf.summary,
         nf.description.unwrap_or_else(String::new),
@@ -481,7 +486,8 @@ async fn update_flow(
         schema.and_then(|x| serde_json::to_string(&x).ok()),
         flow_path,
         w_id,
-        nf.tag
+        nf.tag,
+        nf.dedicated_worker
     )
     .execute(&mut tx)
     .await?;
@@ -514,11 +520,12 @@ async fn update_flow(
     .await?;
 
     if let Some(schedule) = schedule {
+        clear_schedule(tx.transaction_mut(), &flow_path, &w_id).await?;
         schedulables.push(schedule);
     }
 
     for schedule in schedulables.into_iter() {
-        clear_schedule(tx.transaction_mut(), &schedule.path, true, &w_id).await?;
+        clear_schedule(tx.transaction_mut(), &schedule.path, &w_id).await?;
 
         if schedule.enabled {
             tx = push_scheduled_job(&db, tx, schedule).await?;
@@ -564,7 +571,10 @@ async fn update_flow(
         &db,
         tx,
         &w_id,
-        JobPayload::FlowDependencies { path: nf.path.clone() },
+        JobPayload::FlowDependencies {
+            path: nf.path.clone(),
+            dedicated_worker: nf.dedicated_worker,
+        },
         PushArgs::empty(),
         &authed.username,
         &authed.email,
@@ -636,8 +646,12 @@ pub struct FlowWDraft {
     pub draft: Option<serde_json::Value>,
     #[serde(skip_serializing_if = "Option::is_none")]
     pub draft_only: Option<bool>,
+    #[serde(skip_serializing_if = "Option::is_none")]
     pub tag: Option<String>,
+    #[serde(skip_serializing_if = "Option::is_none")]
     pub ws_error_handler_muted: Option<bool>,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub dedicated_worker: Option<bool>,
 }
 
 async fn get_flow_by_path_w_draft(
@@ -649,7 +663,8 @@ async fn get_flow_by_path_w_draft(
     let mut tx = user_db.begin(&authed).await?;
 
     let flow_o = sqlx::query_as::<_, FlowWDraft>(
-        "SELECT flow.path, flow.summary, flow,description, flow.schema, flow.value, flow.extra_perms, flow.draft_only, flow.ws_error_handler_muted, draft.value as draft, flow.tag FROM flow
+        "SELECT flow.path, flow.summary, flow,description, flow.schema, flow.value, flow.extra_perms, flow.draft_only, flow.ws_error_handler_muted, flow.dedicated_worker, draft.value as draft, flow.tag
+         FROM flow
         LEFT JOIN draft ON 
         flow.path = draft.path AND draft.workspace_id = $2 AND draft.typ = 'flow' 
         WHERE flow.path = $1 AND flow.workspace_id = $2",
@@ -882,8 +897,8 @@ mod tests {
             concurrency_time_window_s: None,
             skip_expr: None,
             cache_ttl: None,
-            ws_error_handler_muted: None,
             priority: None,
+            early_return: None,
         };
         let expect = serde_json::json!({
           "modules": [

backend/windmill-api/src/groups.rs

@@ -204,9 +204,9 @@ async fn _check_nb_of_groups(db: &DB) -> Result<()> {
     let nb_groups = sqlx::query_scalar!("SELECT COUNT(*) FROM group_ WHERE name != 'all' AND name != 'error_handler' AND name != 'slack'",)
         .fetch_one(db)
         .await?;
-    if nb_groups.unwrap_or(0) >= 5 {
+    if nb_groups.unwrap_or(0) >= 3 {
         return Err(Error::BadRequest(
-            "You have reached the maximum number of groups (5 outside of native groups 'all', 'slack' and 'error_handler') without an enterprise license"
+            "You have reached the maximum number of groups (3 outside of native groups 'all', 'slack' and 'error_handler') without an enterprise license"
                 .to_string(),
         ));
     }

backend/windmill-api/src/job_helpers.rs

@@ -0,0 +1,728 @@
+use std::cmp;
+
+use crate::{
+    db::DB, resources::transform_json_value, users::Tokened, workspaces::LargeFileStorage,
+};
+use aws_sdk_s3::config::{Credentials, Region};
+use axum::{
+    extract::{Path, Query},
+    routing::{get, post},
+    Extension, Json, Router,
+};
+use hyper::http;
+use object_store::ClientConfigKey;
+use polars::{
+    io::{
+        cloud::{AmazonS3ConfigKey, CloudOptions},
+        SerReader,
+    },
+    lazy::{
+        dsl::col,
+        frame::{LazyFrame, ScanArgsParquet},
+    },
+    prelude::CsvReader,
+};
+use serde::{Deserialize, Serialize};
+use tower_http::cors::{Any, CorsLayer};
+use windmill_common::{db::UserDB, error};
+
+use crate::db::ApiAuthed;
+
+pub fn workspaced_service() -> Router {
+    let cors = CorsLayer::new()
+        .allow_methods([http::Method::GET, http::Method::POST])
+        .allow_headers([http::header::CONTENT_TYPE, http::header::AUTHORIZATION])
+        .allow_origin(Any);
+
+    Router::new()
+        .route(
+            "/duckdb_connection_settings",
+            post(duckdb_connection_settings).layer(cors.clone()),
+        )
+        .route(
+            "/polars_connection_settings",
+            post(polars_connection_settings).layer(cors.clone()),
+        )
+        .route("/test_connection", get(test_connection).layer(cors.clone()))
+        .route(
+            "/list_stored_files",
+            get(list_stored_files).layer(cors.clone()),
+        )
+        .route(
+            "/load_file_metadata",
+            get(load_file_metadata).layer(cors.clone()),
+        )
+        .route(
+            "/load_file_preview",
+            get(load_file_preview).layer(cors.clone()),
+        )
+}
+
+#[derive(Debug, Deserialize, Clone)]
+struct S3Resource {
+    #[serde(rename = "bucket")]
+    bucket: String,
+    region: String,
+    #[serde(rename = "endPoint")]
+    endpoint: String,
+    #[serde(rename = "useSSL")]
+    use_ssl: bool,
+    #[serde(rename = "accessKey")]
+    access_key: Option<String>,
+    #[serde(rename = "secretKey")]
+    secret_key: Option<String>,
+    #[serde(rename = "pathStyle")]
+    path_style: bool,
+}
+
+#[derive(Deserialize)]
+struct DuckdbConnectionSettingsQuery {
+    s3_resource: S3Resource,
+}
+#[derive(Serialize)]
+struct DuckdbConnectionSettingsResponse {
+    connection_settings_str: String,
+}
+
+async fn duckdb_connection_settings(
+    Path(_w_id): Path<String>,
+    Json(query): Json<DuckdbConnectionSettingsQuery>,
+) -> error::JsonResult<DuckdbConnectionSettingsResponse> {
+    let mut duckdb_settings: String = String::new();
+
+    let s3_resource = query.s3_resource;
+    duckdb_settings.push_str(format!("SET home_directory='./';\n").as_str()); // TODO: make this configurable maybe, or point to a temporary folder
+    duckdb_settings.push_str(format!("INSTALL 'httpfs';\n").as_str());
+    if s3_resource.path_style {
+        duckdb_settings.push_str(format!("SET s3_url_style='path';\n").as_str());
+    }
+    duckdb_settings.push_str(format!("SET s3_region='{}';\n", s3_resource.region).as_str());
+    duckdb_settings.push_str(format!("SET s3_endpoint='{}';\n", s3_resource.endpoint).as_str());
+    if !s3_resource.use_ssl {
+        duckdb_settings.push_str(format!("SET s3_use_ssl=0;\n").as_str()); // default is true for DuckDB
+    }
+    if let Some(access_key_id) = s3_resource.access_key {
+        duckdb_settings.push_str(format!("SET s3_access_key_id='{}';\n", access_key_id).as_str());
+    }
+    if let Some(secret_access_key) = s3_resource.secret_key {
+        duckdb_settings
+            .push_str(format!("SET s3_secret_access_key='{}';\n", secret_access_key).as_str());
+    }
+
+    let response = DuckdbConnectionSettingsResponse { connection_settings_str: duckdb_settings };
+    return Ok(Json(response));
+}
+
+#[derive(Deserialize)]
+struct PolarsConnectionSettingsQuery {
+    s3_resource: S3Resource,
+}
+
+#[derive(Serialize)]
+struct PolarsConnectionSettingsResponse {
+    endpoint_url: String,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    key: Option<String>,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    secret: Option<String>,
+    use_ssl: bool,
+    cache_regions: bool,
+    client_kwargs: PolarsConnectionSettings,
+}
+
+#[derive(Serialize)]
+struct PolarsConnectionSettings {
+    pub region_name: String,
+}
+
+async fn polars_connection_settings(
+    Path(_w_id): Path<String>,
+    Json(query): Json<PolarsConnectionSettingsQuery>,
+) -> error::JsonResult<PolarsConnectionSettingsResponse> {
+    let s3_resource = query.s3_resource;
+
+    let response = PolarsConnectionSettingsResponse {
+        endpoint_url: render_endpoint(&s3_resource),
+        key: s3_resource.access_key,
+        secret: s3_resource.secret_key,
+        use_ssl: s3_resource.use_ssl,
+        cache_regions: false,
+        client_kwargs: PolarsConnectionSettings { region_name: s3_resource.region },
+    };
+
+    return Ok(Json(response));
+}
+
+#[derive(Serialize, Deserialize, Clone)]
+struct WindmillLargeFile {
+    s3: String,
+}
+
+async fn test_connection(
+    authed: ApiAuthed,
+    Extension(user_db): Extension<UserDB>,
+    Extension(db): Extension<DB>,
+    Tokened { token }: Tokened,
+    Path(w_id): Path<String>,
+) -> error::JsonResult<()> {
+    let s3_resource_opt = get_workspace_s3_resource(&authed, &user_db, &db, &token, &w_id).await?;
+    if s3_resource_opt.is_none() {
+        return Err(error::Error::NotFound(
+            "No datasets storage resource defined at the workspace level".to_string(),
+        ));
+    }
+
+    let s3_resource = s3_resource_opt.ok_or(error::Error::InternalErr(
+        "No files storage resource defined at the workspace level".to_string(),
+    ))?;
+    let s3_client = build_s3_client(&s3_resource);
+    s3_client
+        .list_objects()
+        .bucket(s3_resource.bucket)
+        .max_keys(1)
+        .send()
+        .await
+        .map_err(|err| error::Error::InternalErr(err.to_string()))?;
+    return Ok(Json(()));
+}
+
+#[derive(Deserialize)]
+struct ListStoredFilesQuery {
+    pub max_keys: i32,
+    pub marker: Option<String>,
+}
+
+#[derive(Serialize)]
+struct ListStoredDatasetsResponse {
+    windmill_large_files: Vec<WindmillLargeFile>,
+    pub next_marker: Option<String>,
+}
+
+async fn list_stored_files(
+    authed: ApiAuthed,
+    Extension(user_db): Extension<UserDB>,
+    Extension(db): Extension<DB>,
+    Tokened { token }: Tokened,
+    Path(w_id): Path<String>,
+    Query(query): Query<ListStoredFilesQuery>,
+) -> error::JsonResult<ListStoredDatasetsResponse> {
+    let s3_resource_opt = get_workspace_s3_resource(&authed, &user_db, &db, &token, &w_id).await?;
+
+    let s3_resource = s3_resource_opt.ok_or(error::Error::InternalErr(
+        "No files storage resource defined at the workspace level".to_string(),
+    ))?;
+    let s3_client = build_s3_client(&s3_resource);
+
+    let s3_bucket = s3_resource.bucket;
+
+    let list_object_query = s3_client
+        .list_objects()
+        .bucket(s3_bucket.clone())
+        .max_keys(query.max_keys)
+        .set_marker(query.marker);
+
+    let bucket_objects = list_object_query
+        .send()
+        .await
+        .map_err(|err| error::Error::InternalErr(err.to_string()))?;
+
+    let stored_datasets = bucket_objects
+        .contents()
+        .iter()
+        .filter(|object| object.key().is_some())
+        .map(|object| object.key())
+        .map(Option::unwrap)
+        .map(&str::to_string)
+        .map(|object_key| WindmillLargeFile { s3: object_key.clone() })
+        .collect::<Vec<WindmillLargeFile>>();
+
+    #[cfg(not(feature = "enterprise"))]
+    if stored_datasets.len() > 20 {
+        return Err(error::Error::ExecutionErr(
+            "The workspace s3 bucket contains more than 20 files. Consider upgrading to Windmill Enterprise Edition to continue to use this feature."
+                .to_string(),
+        ));
+    }
+
+    let next_marker = if bucket_objects.is_truncated() {
+        if bucket_objects.next_marker().is_some() {
+            // some S3 providers returns the next marker for us. If that's the case just re-use it
+            bucket_objects.next_marker().map(|v| v.to_owned())
+        } else {
+            // others, like AWS, doesn't and implicitly expect users to return the last key of the current page
+            stored_datasets.last().map(|v| v.s3.clone())
+        }
+    } else {
+        None
+    };
+
+    return Ok(Json(ListStoredDatasetsResponse {
+        windmill_large_files: stored_datasets,
+        next_marker: next_marker,
+    }));
+}
+
+#[derive(Deserialize)]
+struct LoadFileMetadataQuery {
+    pub file_key: String,
+}
+
+#[derive(Serialize)]
+struct LoadFileMetadataResponse {
+    pub mime_type: Option<String>,
+    pub size_in_bytes: Option<i64>,
+    pub last_modified: Option<chrono::DateTime<chrono::Utc>>,
+    pub expires: Option<chrono::DateTime<chrono::Utc>>,
+    pub version_id: Option<String>,
+}
+
+#[derive(Deserialize)]
+struct LoadFilePreviewQuery {
+    pub file_key: String,
+
+    // The two options below are requested from s3 with an additional query is not set
+    pub file_size_in_bytes: Option<i64>,
+    pub file_mime_type: Option<String>,
+
+    // For CSV files, the separator needs to be specify
+    pub csv_separator: Option<String>,
+
+    // Specify the content length to be read. Both will be taken into account when reading files, except for:
+    // - CSVs: only the length will be taken into account, a CSV file larger than this will be truncated.
+    //   Note that truncated CSV files might not be valid CSV files anymore, and therefore the preview might fail
+    // - Parquet files: Parquet files are lazy-loaded. Therefore none of those params will be taken into account
+    pub read_bytes_from: i64,
+    pub read_bytes_length: i64,
+}
+
+#[derive(Serialize)]
+struct LoadFilePreviewResponse {
+    pub content: Option<String>,
+    pub content_type: WindmillContentType,
+    pub msg: Option<String>,
+}
+
+#[derive(Serialize)]
+enum WindmillContentType {
+    RawText,
+    Csv,
+    Parquet,
+    Unknown,
+}
+
+async fn load_file_metadata(
+    authed: ApiAuthed,
+    Extension(user_db): Extension<UserDB>,
+    Extension(db): Extension<DB>,
+    Tokened { token }: Tokened,
+    Path(w_id): Path<String>,
+    Query(query): Query<LoadFileMetadataQuery>,
+) -> error::JsonResult<LoadFileMetadataResponse> {
+    let file_key = query.file_key.clone();
+    let s3_resource_opt = get_workspace_s3_resource(&authed, &user_db, &db, &token, &w_id).await?;
+
+    let s3_resource = s3_resource_opt.ok_or(error::Error::InternalErr(
+        "No files storage resource defined at the workspace level".to_string(),
+    ))?;
+    let s3_client = build_s3_client(&s3_resource);
+
+    let s3_bucket = s3_resource.bucket.clone();
+    let s3_object_metadata = s3_client
+        .head_object()
+        .bucket(&s3_bucket)
+        .key(&file_key)
+        .send()
+        .await
+        .map_err(|err| error::Error::InternalErr(err.to_string()))?;
+
+    let response = LoadFileMetadataResponse {
+        mime_type: s3_object_metadata.content_type().map(&str::to_string),
+        size_in_bytes: Some(s3_object_metadata.content_length()),
+        last_modified: s3_object_metadata
+            .last_modified()
+            .map(|dt| chrono::DateTime::from_timestamp(dt.secs(), dt.subsec_nanos()))
+            .flatten(),
+        expires: s3_object_metadata
+            .expires()
+            .map(|dt| chrono::DateTime::from_timestamp(dt.secs(), dt.subsec_nanos()))
+            .flatten(),
+        version_id: s3_object_metadata.version_id().map(&str::to_string),
+    };
+    return Ok(Json(response));
+}
+
+async fn load_file_preview(
+    authed: ApiAuthed,
+    Extension(user_db): Extension<UserDB>,
+    Extension(db): Extension<DB>,
+    Tokened { token }: Tokened,
+    Path(w_id): Path<String>,
+    Query(query): Query<LoadFilePreviewQuery>,
+) -> error::JsonResult<LoadFilePreviewResponse> {
+    // query validation
+    if query.read_bytes_length > 8 * 1024 * 1024 {
+        return Err(error::Error::BadRequest(
+            "Cannot load file bigger than 8MB".to_string(),
+        ));
+    }
+
+    let file_key = query.file_key.clone();
+    let s3_resource_opt = get_workspace_s3_resource(&authed, &user_db, &db, &token, &w_id).await?;
+
+    let s3_resource = s3_resource_opt.ok_or(error::Error::InternalErr(
+        "No files storage resource defined at the workspace level".to_string(),
+    ))?;
+    let s3_client = build_s3_client(&s3_resource);
+
+    let s3_bucket = s3_resource.bucket.clone();
+
+    // if content length is provided in the request, use it, otherwise get it from s3
+    let (s3_object_mime_type, s3_object_content_length) =
+        if query.file_size_in_bytes.is_none() || query.file_mime_type.is_none() {
+            let s3_object_metadata = s3_client
+                .head_object()
+                .bucket(&s3_bucket)
+                .key(&file_key)
+                .send()
+                .await
+                .map_err(|err| error::Error::InternalErr(err.to_string()))?;
+            (
+                s3_object_metadata.content_type().map(|v| v.to_owned()),
+                s3_object_metadata.content_length(),
+            )
+        } else {
+            (
+                query.file_mime_type.clone(),
+                query.file_size_in_bytes.unwrap(),
+            )
+        };
+
+    let file_chunk_length = cmp::min(
+        query.read_bytes_length,
+        s3_object_content_length - query.read_bytes_from,
+    );
+
+    let content_type: WindmillContentType;
+    let content_preview = match s3_object_mime_type.as_deref() {
+        Some("text/csv") => {
+            content_type = WindmillContentType::Csv;
+            read_s3_csv_object_head(
+                &s3_client,
+                &s3_bucket,
+                &file_key,
+                file_chunk_length,
+                query.csv_separator,
+            )
+            .await
+        }
+        Some(mt)
+            if mt.starts_with("text/")
+                || mt == "application/json"
+                || mt == "application/x-yaml" =>
+        {
+            content_type = WindmillContentType::RawText;
+            read_s3_text_object_head(
+                &s3_client,
+                &s3_bucket,
+                &file_key,
+                query.read_bytes_from,
+                file_chunk_length,
+            )
+            .await
+        }
+        mt_opt => {
+            // sometimes S3 doesn't infer the content type on upload. Guess it from the file extension
+            if file_key.to_lowercase().ends_with(".parquet") {
+                content_type = WindmillContentType::Parquet;
+                read_s3_parquet_object_head(&s3_resource, &file_key).await
+            } else if file_key.to_lowercase().ends_with(".csv") {
+                content_type = WindmillContentType::Csv;
+                read_s3_csv_object_head(
+                    &s3_client,
+                    &s3_bucket,
+                    &file_key,
+                    file_chunk_length,
+                    query.csv_separator,
+                )
+                .await
+            } else {
+                content_type = WindmillContentType::Unknown;
+                let msg = match mt_opt {
+                    Some(mt) => {
+                        format!("Preview is not available for content of type '{}'", mt).to_string()
+                    }
+                    None => "Preview is not available. Content type is unknown or not supported"
+                        .to_string(),
+                };
+                Err(error::Error::ExecutionErr(msg))
+            }
+        }
+    };
+
+    let response: LoadFilePreviewResponse = match content_preview {
+        Ok(content) => LoadFilePreviewResponse {
+            content_type: content_type,
+            content: Some(content),
+            msg: None,
+        },
+
+        Err(err) => LoadFilePreviewResponse {
+            content_type: content_type,
+            content: None,
+            msg: Some(err.to_string()),
+        },
+    };
+
+    return Ok(Json(response));
+}
+
+async fn get_workspace_s3_resource<'c>(
+    authed: &ApiAuthed,
+    user_db: &UserDB,
+    db: &DB,
+    token: &str,
+    w_id: &str,
+) -> error::Result<Option<S3Resource>> {
+    let mut tx = user_db.clone().begin(authed).await?;
+    let raw_lfs_opt = sqlx::query_scalar!(
+        "SELECT large_file_storage FROM workspace_settings WHERE workspace_id = $1",
+        w_id
+    )
+    .fetch_optional(&mut *tx)
+    .await?
+    .flatten();
+    tx.commit().await?;
+
+    if raw_lfs_opt.is_none() {
+        return Ok(None);
+    }
+
+    let large_file_storage = serde_json::from_value::<LargeFileStorage>(
+        raw_lfs_opt.unwrap(),
+    )
+    .map_err(|err| {
+        tracing::error!(
+            "Value stored in large_file_storage column is invalid and could not be deserialized: {}",
+            err
+        );
+        error::Error::InternalErr(
+            "Could not deserialize LargeFileStorage value found in database".to_string(),
+        )
+    })?;
+    let s3_lfs = match large_file_storage {
+        LargeFileStorage::S3Storage(s3_lfs) => s3_lfs,
+    };
+
+    let resource_path_json_value = serde_json::to_value(s3_lfs.s3_resource_path)
+        .map_err(|err| error::Error::InternalErr(err.to_string()))?;
+    let interpolated_value = transform_json_value(
+        authed,
+        user_db,
+        db,
+        &w_id,
+        resource_path_json_value,
+        &Option::None,
+        token,
+    )
+    .await?;
+
+    let s3_resource = serde_json::from_value::<S3Resource>(interpolated_value)
+        .map_err(|err| error::Error::InternalErr(err.to_string()))?;
+    return Ok(Some(s3_resource));
+}
+
+fn build_s3_client(s3_resource_ref: &S3Resource) -> aws_sdk_s3::Client {
+    let s3_resource = s3_resource_ref.clone();
+
+    let endpoint = render_endpoint(&s3_resource);
+    let mut s3_config_builder = aws_sdk_s3::Config::builder()
+        .endpoint_url(endpoint)
+        .region(Region::new(s3_resource.region));
+    if s3_resource.access_key.is_some() {
+        s3_config_builder = s3_config_builder.credentials_provider(Credentials::new(
+            s3_resource.access_key.unwrap_or_default(),
+            s3_resource.secret_key.unwrap_or_default(),
+            None,
+            None,
+            "s3_storage",
+        ));
+    }
+    if s3_resource.path_style {
+        s3_config_builder = s3_config_builder.force_path_style(true);
+    }
+    let s3_config = s3_config_builder.build();
+    return aws_sdk_s3::Client::from_conf(s3_config);
+}
+
+fn build_polars_s3_config(s3_resource_ref: &S3Resource) -> CloudOptions {
+    let s3_resource = s3_resource_ref.to_owned();
+    let mut s3_configs: Vec<(AmazonS3ConfigKey, String)> = vec![
+        (AmazonS3ConfigKey::Region, s3_resource.region),
+        (AmazonS3ConfigKey::Bucket, s3_resource.bucket),
+        (
+            AmazonS3ConfigKey::Endpoint,
+            render_endpoint(s3_resource_ref),
+        ),
+        (
+            AmazonS3ConfigKey::Client(ClientConfigKey::AllowHttp),
+            (!s3_resource.use_ssl).to_string(),
+        ),
+        (
+            AmazonS3ConfigKey::VirtualHostedStyleRequest,
+            (!s3_resource.path_style).to_string(),
+        ),
+    ];
+    if let Some(access_key) = s3_resource.access_key {
+        s3_configs.push((AmazonS3ConfigKey::AccessKeyId, access_key));
+    }
+    if let Some(secret_key) = s3_resource.secret_key {
+        s3_configs.push((AmazonS3ConfigKey::SecretAccessKey, secret_key));
+    }
+    return CloudOptions::default().with_aws(s3_configs);
+}
+
+fn render_endpoint(s3_resource: &S3Resource) -> String {
+    if s3_resource.endpoint.starts_with("http://") || s3_resource.endpoint.starts_with("https://") {
+        s3_resource.endpoint.clone()
+    } else if s3_resource.use_ssl {
+        format!("https://{}", s3_resource.endpoint)
+    } else {
+        format!("http://{}", s3_resource.endpoint)
+    }
+}
+
+async fn read_s3_text_object_head(
+    s3_client: &aws_sdk_s3::Client,
+    s3_bucket: &str,
+    file_key: &str,
+    from_char: i64,
+    length: i64,
+) -> error::Result<String> {
+    let s3_object = s3_client
+        .get_object()
+        .range(format!("bytes={}-{}", from_char, length).to_string())
+        .bucket(s3_bucket)
+        .key(file_key)
+        .send()
+        .await
+        .map_err(|err| {
+            tracing::warn!("Error fetching text file from S3: {:?}", err);
+            error::Error::InternalErr(err.to_string())
+        })?;
+
+    let payload = s3_object
+        .body
+        .collect()
+        .await
+        .map_err(|err| {
+            tracing::warn!(
+                "Error reading raw text file {}. Error was: {:?}",
+                file_key,
+                err
+            );
+            error::Error::InternalErr("File encoding is not supported".to_string())
+        })?
+        .into_bytes()
+        .to_vec();
+
+    let file_header_str = String::from_utf8(payload).map_err(|err| {
+        tracing::warn!(
+            "Encoding of file {} unsupported. Error was: {:?}",
+            file_key,
+            err
+        );
+        error::Error::InternalErr("File encoding is not supported".to_string())
+    })?;
+    return Ok(file_header_str);
+}
+
+async fn read_s3_parquet_object_head(
+    s3_resource_ref: &S3Resource,
+    file_key: &str,
+) -> error::Result<String> {
+    let s3_cloud_config = build_polars_s3_config(s3_resource_ref);
+
+    let args: ScanArgsParquet = ScanArgsParquet {
+        n_rows: Some(1),
+        cache: false,
+        parallel: polars::io::parquet::ParallelStrategy::Auto,
+        rechunk: false,
+        row_count: None,
+        low_memory: false,
+        use_statistics: false,
+        hive_partitioning: false,
+        cloud_options: Some(s3_cloud_config),
+    };
+
+    let file_key_clone = file_key.to_string();
+    let s3_bucket_clone = s3_resource_ref.bucket.to_string();
+    let polars_df_result = tokio::task::spawn_blocking(move || {
+        let s3_file_key = format!("s3://{}/{}", s3_bucket_clone, file_key_clone);
+        let lzdf_result = LazyFrame::scan_parquet(s3_file_key, args);
+        match lzdf_result {
+            Err(err) => {
+                tracing::warn!("Error fetching parquet file from S3: {:?}", err);
+                return Err(error::Error::InternalErr(err.to_string()));
+            }
+            Ok(lzdf) => {
+                let df = lzdf
+                    .select(&[col("*")])
+                    .limit(10) // for now read only first 10 lines
+                    .collect()
+                    .map_err(|err| error::Error::InternalErr(err.to_string()))?;
+                return Ok(format!("{:?}", df).to_string());
+            }
+        }
+    })
+    .await
+    .map_err(|err| error::Error::InternalErr(err.to_string()))?;
+
+    return polars_df_result;
+}
+
+async fn read_s3_csv_object_head(
+    s3_client: &aws_sdk_s3::Client,
+    s3_bucket: &str,
+    file_key: &str,
+    length: i64,
+    separator: Option<String>,
+) -> error::Result<String> {
+    let separator_final = if let Some(separator_char) = separator {
+        if separator_char.len() != 1 {
+            return Err(error::Error::BadRequest(
+                "Separator must be a single character".to_string(),
+            ));
+        }
+        separator_char.as_bytes()[0]
+    } else {
+        ",".as_bytes()[0] // polars uses the comma as default, doing the same here
+    };
+
+    let s3_object = s3_client
+        .get_object()
+        .bucket(s3_bucket)
+        .range(format!("bytes=0-{}", length).to_string())
+        .key(file_key)
+        .send()
+        .await
+        .map_err(|err| error::Error::InternalErr(err.to_string()))?;
+
+    // TODO: polars does not seem to support lazy csv reader, unfortunately. We can implement it ourselves if needed
+    // Right now it's fine b/c we limit the download from AWS to 32MB. We should recomment users to use parquet
+    // for larger files
+    let file_content_bytes = s3_object
+        .body
+        .collect()
+        .await
+        .map_err(|err| error::Error::InternalErr(err.to_string()))?
+        .into_bytes();
+    let cursor = std::io::Cursor::new(file_content_bytes);
+
+    let csv_df = CsvReader::new(cursor)
+        .with_n_rows(Some(10)) // for now read only first 10 lines
+        .with_separator(separator_final)
+        .finish()
+        .map_err(|err| error::Error::InternalErr(err.to_string()))?;
+
+    return Ok(format!("{:?}", csv_df).to_string());
+}

backend/windmill-api/src/jobs.rs

@@ -36,7 +36,7 @@ use sqlx::{query_scalar, types::Uuid, FromRow, Postgres, Transaction};
 use tower_http::cors::{Any, CorsLayer};
 use urlencoding::encode;
 use windmill_audit::{audit_log, ActionKind};
-use windmill_common::worker::{CUSTOM_TAGS_PER_WORKSPACE, SERVER_CONFIG};
+use windmill_common::worker::{to_raw_value, CUSTOM_TAGS_PER_WORKSPACE, SERVER_CONFIG};
 use windmill_common::{
     db::UserDB,
     error::{self, to_anyhow, Error},
@@ -49,7 +49,10 @@ use windmill_common::{
     utils::{not_found_if_none, now_from_db, paginate, require_admin, Pagination, StripPath},
 };
 use windmill_common::{get_latest_deployed_hash_for_path, BASE_URL};
-use windmill_queue::{empty_args, job_is_complete, push, PushArgs, PushIsolationLevel};
+use windmill_queue::{
+    add_completed_job_error, get_queued_job, get_result_by_id_from_running_flow, job_is_complete,
+    push, CanceledBy, PushArgs, PushIsolationLevel,
+};
 
 pub fn workspaced_service() -> Router {
     let cors = CorsLayer::new()
@@ -361,7 +364,7 @@ async fn get_job(
 async fn get_job_internal(db: &DB, workspace_id: &str, job_id: Uuid) -> error::Result<Job> {
     let cjob_maybe = sqlx::query_as::<_, CompletedJob>("SELECT 
         id, workspace_id, parent_job, created_by, created_at, duration_ms, success, script_hash, script_path, 
-        CASE WHEN pg_column_size(args) < 2000000 THEN args ELSE '{\"reason\": \"WINDMILL_TOO_BIG\"}'::jsonb END as args, CASE WHEN pg_column_size(result) < 2000000 THEN result ELSE '\"WINDMILL_TOO_BIG\"'::jsonb END as result, logs, deleted, raw_code, canceled, canceled_by, canceled_reason, job_kind, env_id,
+        CASE WHEN args is null or pg_column_size(args) < 2000000 THEN args ELSE '{\"reason\": \"WINDMILL_TOO_BIG\"}'::jsonb END as args, CASE WHEN result is null or pg_column_size(result) < 2000000 THEN result ELSE '\"WINDMILL_TOO_BIG\"'::jsonb END as result, logs, deleted, raw_code, canceled, canceled_by, canceled_reason, job_kind, env_id,
         schedule_path, permissioned_as, flow_status, raw_flow, is_flow_step, language, started_at, is_skipped,
         raw_lock, email, visible_to_owner, mem_peak, tag, priority
         FROM completed_job WHERE id = $1 AND workspace_id = $2")
@@ -375,7 +378,7 @@ async fn get_job_internal(db: &DB, workspace_id: &str, job_id: Uuid) -> error::R
     } else {
         let job_o = sqlx::query_as::<_, QueuedJob>(
             "SELECT  id, workspace_id, parent_job, created_by, created_at, started_at, scheduled_for, running,
-                script_hash, script_path, CASE WHEN pg_column_size(args) < 2000000 THEN args ELSE '{\"reason\": \"WINDMILL_TOO_BIG\"}'::jsonb END as args, logs, raw_code, canceled, canceled_by, canceled_reason, last_ping, 
+                script_hash, script_path, CASE WHEN args is null or pg_column_size(args) < 2000000 THEN args ELSE '{\"reason\": \"WINDMILL_TOO_BIG\"}'::jsonb END as args, logs, raw_code, canceled, canceled_by, canceled_reason, last_ping, 
                 job_kind, env_id, schedule_path, permissioned_as, flow_status, raw_flow, is_flow_step, language,
                  suspend, suspend_until, same_worker, raw_lock, pre_run_error, email, visible_to_owner, mem_peak, 
                 root_job, leaf_jobs, tag, concurrent_limit, concurrency_time_window_s, timeout, flow_step_id, cache_ttl, priority
@@ -639,17 +642,51 @@ async fn list_queue_jobs(
 async fn cancel_all(
     authed: ApiAuthed,
     Extension(db): Extension<DB>,
+    Extension(rsmq): Extension<Option<rsmq_async::MultiplexedRsmq>>,
+
     Path(w_id): Path<String>,
 ) -> error::JsonResult<Vec<Uuid>> {
     require_admin(authed.is_admin, &authed.username)?;
 
-    let uuids = sqlx::query_scalar!(
-        "UPDATE queue SET canceled = true,  canceled_by = $2, scheduled_for = now(), suspend = 0 WHERE workspace_id = $1 AND schedule_path IS NULL RETURNING id",
+    let mut jobs = sqlx::query!(
+        "UPDATE queue SET canceled = true,  canceled_by = $2, scheduled_for = now(), suspend = 0 WHERE workspace_id = $1 AND schedule_path IS NULL RETURNING id, running",
         w_id,
         authed.username
     )
     .fetch_all(&db)
     .await?;
+
+    let username = authed.username;
+    for j in jobs.iter() {
+        if !j.running {
+            let e = serde_json::json!({"message": format!("Job canceled: cancel_all by {username}"), "name": "Canceled", "reason": "cancel_all", "canceler": username});
+            let mut tx = db.begin().await?;
+            let job_running = get_queued_job(j.id, &w_id, &mut tx).await?;
+            tx.commit().await?;
+
+            if let Some(job_running) = job_running {
+                let add_job = add_completed_job_error(
+                    &db,
+                    &job_running,
+                    format!("canceled by {username}: cancel_all"),
+                    job_running.mem_peak.unwrap_or(0),
+                    Some(CanceledBy {
+                        username: Some(username.to_string()),
+                        reason: Some("cancel_all".to_string()),
+                    }),
+                    e,
+                    rsmq.clone(),
+                    "server",
+                )
+                .await;
+                if let Err(e) = add_job {
+                    tracing::error!("Failed to add canceled job: {}", e);
+                }
+            }
+        }
+    }
+    let uuids = jobs.iter_mut().map(|j| j.id).collect::<Vec<_>>();
+
     Ok(Json(uuids))
 }
 
@@ -1527,6 +1564,53 @@ fn decode_payload<D: DeserializeOwned>(t: String) -> anyhow::Result<D> {
     serde_json::from_slice(vec.as_slice()).context("invalid json")
 }
 
+#[derive(Deserialize)]
+pub struct DecodeQuery {
+    pub include_query: Option<String>,
+}
+
+#[derive(Deserialize)]
+pub struct IncludeQuery {
+    pub include_query: Option<String>,
+}
+
+pub struct DecodeQueries(pub HashMap<String, Box<RawValue>>);
+
+#[axum::async_trait]
+impl<S> FromRequest<S, axum::body::Body> for DecodeQueries
+where
+    S: Send + Sync,
+{
+    type Rejection = Response;
+
+    async fn from_request(
+        req: Request<axum::body::Body>,
+        _state: &S,
+    ) -> Result<Self, Self::Rejection> {
+        let query = req.uri().query().unwrap_or("");
+        let include_query = serde_urlencoded::from_str::<IncludeQuery>(query)
+            .map(|x| x.include_query)
+            .ok()
+            .flatten()
+            .unwrap_or_default();
+        let parse_query_args = include_query
+            .split(",")
+            .map(|s| s.to_string())
+            .collect::<Vec<_>>();
+        let mut args = HashMap::new();
+        if !parse_query_args.is_empty() {
+            let queries =
+                serde_urlencoded::from_str::<HashMap<String, String>>(query).unwrap_or_default();
+            parse_query_args.iter().for_each(|h| {
+                if let Some(v) = queries.get(h) {
+                    args.insert(h.to_string(), to_raw_value(v));
+                }
+            });
+        }
+        Ok(DecodeQueries(args))
+    }
+}
+
 pub fn add_raw_string(
     raw_string: Option<String>,
     mut args: serde_json::Map<String, serde_json::Value>,
@@ -1594,14 +1678,16 @@ pub async fn run_flow_by_path(
     let flow_path = flow_path.to_path();
     check_scopes(&authed, || format!("run:flow/{flow_path}"))?;
 
-    let tag = sqlx::query_scalar!(
-        "SELECT tag from flow WHERE path = $1 and workspace_id = $2",
+    let (tag, dedicated_worker) = sqlx::query!(
+        "SELECT tag, dedicated_worker from flow WHERE path = $1 and workspace_id = $2",
         flow_path,
         w_id
     )
     .fetch_optional(&db)
     .await?
-    .flatten();
+    .map(|x| (x.tag, x.dedicated_worker))
+    .unwrap_or_else(|| (None, None));
+
     check_tag_available_for_workspace(&w_id, &tag).await?;
     let scheduled_for = run_query.get_scheduled_for(&db).await?;
     let tx = PushIsolationLevel::Isolated(user_db.clone(), authed.clone().into(), rsmq);
@@ -1609,7 +1695,7 @@ pub async fn run_flow_by_path(
         &db,
         tx,
         &w_id,
-        JobPayload::Flow(flow_path.to_string()),
+        JobPayload::Flow { path: flow_path.to_string(), dedicated_worker },
         args,
         &authed.username,
         &authed.email,
@@ -1763,33 +1849,25 @@ struct Guard {
     done: bool,
     id: Uuid,
     w_id: String,
-    db: UserDB,
-    authed: ApiAuthed,
+    db: DB,
 }
 
 impl Drop for Guard {
     fn drop(&mut self) {
         if !&self.done {
             let id = self.id;
-            let username = self.authed.username.clone();
             let w_id = self.w_id.clone();
             let db = self.db.clone();
-            let authed = self.authed.clone();
 
             tracing::info!("http connection broke, marking job {id} as canceled");
             tokio::spawn(async move {
-                let tx = db.begin(&authed).await.ok();
-                if let Some(mut tx) = tx {
-                    let _ = sqlx::query!(
-                "UPDATE queue SET canceled = true, canceled_reason = 'http connection broke', canceled_by = $1 WHERE id = $2 AND workspace_id = $3",
-                username,
+                let _ = sqlx::query!(
+                "UPDATE queue SET canceled = true, canceled_reason = 'http connection broke', canceled_by = queue.created_by WHERE id = $1 AND workspace_id = $2",
                 id,
                 w_id
             )
-            .execute(&mut *tx)
+            .execute(&db)
             .await;
-                    let _ = tx.commit().await;
-                }
             });
         }
     }
@@ -1801,10 +1879,10 @@ pub struct WindmillStatusCode {
     result: Option<Box<RawValue>>,
 }
 async fn run_wait_result<T>(
-    authed: ApiAuthed,
-    Extension(user_db): Extension<UserDB>,
+    db: &DB,
     uuid: Uuid,
     Path((w_id, _)): Path<(String, T)>,
+    node_id: Option<String>,
 ) -> error::Result<Response> {
     let mut result;
     let timeout = SERVER_CONFIG.read().await.timeout_wait_result.clone();
@@ -1814,27 +1892,29 @@ async fn run_wait_result<T>(
         (timeout * 1000) as u64
     };
 
-    let mut g = Guard {
-        done: false,
-        id: uuid,
-        w_id: w_id.clone(),
-        db: user_db.clone(),
-        authed: authed.clone(),
-    };
+    let mut g = Guard { done: false, id: uuid, w_id: w_id.clone(), db: db.clone() };
 
     let fast_poll_duration = *WAIT_RESULT_FAST_POLL_DURATION_SECS as u64 * 1000;
     let mut accumulated_delay = 0 as u64;
 
     loop {
-        let mut tx = user_db.clone().begin(&authed).await?;
-        result =
-            sqlx::query("SELECT result FROM completed_job WHERE id = $1 AND workspace_id = $2")
-                .bind(uuid)
-                .bind(&w_id)
-                .fetch_optional(&mut *tx)
-                .await?;
-
-        drop(tx);
+        if let Some(node_id) = node_id.as_ref() {
+            result = get_result_by_id_from_running_flow(&db, &w_id, &uuid, node_id, None)
+                .await
+                .ok();
+        } else {
+            let row =
+                sqlx::query("SELECT result FROM completed_job WHERE id = $1 AND workspace_id = $2")
+                    .bind(uuid)
+                    .bind(&w_id)
+                    .fetch_optional(db)
+                    .await?;
+            if let Some(row) = row {
+                result = Some(RawResult::from_row(&row)?.result.to_owned());
+            } else {
+                result = None;
+            }
+        }
 
         if result.is_some() {
             break;
@@ -1853,7 +1933,6 @@ async fn run_wait_result<T>(
     }
     if let Some(result) = result {
         g.done = true;
-        let result = RawResult::from_row(&result)?.result;
 
         let status_code = serde_json::from_str::<WindmillStatusCode>(result.get());
         match status_code {
@@ -1917,6 +1996,7 @@ pub async fn run_wait_result_job_by_path_get(
     Extension(db): Extension<DB>,
     Path((w_id, script_path)): Path<(String, StripPath)>,
     Query(run_query): Query<RunJobQuery>,
+    DecodeQueries(queries): DecodeQueries,
 ) -> error::Result<Response> {
     #[cfg(feature = "enterprise")]
     check_license_key_valid().await?;
@@ -1929,13 +2009,17 @@ pub async fn run_wait_result_job_by_path_get(
         .map(decode_payload)
         .map(|x| x.map_err(|e| Error::InternalErr(e.to_string())));
 
-    let payload_args = if let Some(payload) = payload_r {
+    let mut payload_args = if let Some(payload) = payload_r {
         payload?
     } else {
         HashMap::new()
     };
+    queries.iter().for_each(|(k, v)| {
+        payload_args.insert(k.to_string(), v.clone());
+    });
 
-    let args = PushArgs { extra: payload_args, args: sqlx::types::Json(empty_args()) };
+    let inner_args: HashMap<String, Box<RawValue>> = HashMap::new();
+    let args = PushArgs { extra: payload_args, args: sqlx::types::Json(inner_args) };
 
     check_queue_too_long(&db, QUEUE_LIMIT_WAIT_RESULT.or(run_query.queue_limit)).await?;
     let script_path = script_path.to_path();
@@ -1971,7 +2055,7 @@ pub async fn run_wait_result_job_by_path_get(
     .await?;
     tx.commit().await?;
 
-    run_wait_result(authed, Extension(user_db), uuid, Path((w_id, script_path))).await
+    run_wait_result(&db, uuid, Path((w_id, script_path)), None).await
 }
 
 pub async fn run_wait_result_flow_by_path_get(
@@ -1981,8 +2065,8 @@ pub async fn run_wait_result_flow_by_path_get(
     Extension(user_db): Extension<UserDB>,
     Extension(db): Extension<DB>,
     Path((w_id, flow_path)): Path<(String, StripPath)>,
-
     Query(run_query): Query<RunJobQuery>,
+    DecodeQueries(queries): DecodeQueries,
 ) -> error::Result<Response> {
     #[cfg(feature = "enterprise")]
     check_license_key_valid().await?;
@@ -1996,12 +2080,16 @@ pub async fn run_wait_result_flow_by_path_get(
         .map(decode_payload)
         .map(|x| x.map_err(|e| Error::InternalErr(e.to_string())));
 
-    let payload_args = if let Some(payload) = payload_r {
+    let mut payload_args = if let Some(payload) = payload_r {
         payload?
     } else {
         HashMap::new()
     };
 
+    queries.iter().for_each(|(k, v)| {
+        payload_args.insert(k.to_string(), v.clone());
+    });
+
     let args = PushArgs { extra: payload_args, args: sqlx::types::Json(HashMap::new()) };
 
     run_wait_result_flow_by_path_internal(
@@ -2080,7 +2168,7 @@ async fn run_wait_result_script_by_path_internal(
     .await?;
     tx.commit().await?;
 
-    run_wait_result(authed, Extension(user_db), uuid, Path((w_id, script_path))).await
+    run_wait_result(&db, uuid, Path((w_id, script_path)), None).await
 }
 
 pub async fn run_wait_result_script_by_hash(
@@ -2148,7 +2236,7 @@ pub async fn run_wait_result_script_by_hash(
     .await?;
     tx.commit().await?;
 
-    run_wait_result(authed, Extension(user_db), uuid, Path((w_id, script_hash))).await
+    run_wait_result(&db, uuid, Path((w_id, script_hash)), None).await
 }
 
 pub async fn run_wait_result_flow_by_path(
@@ -2186,14 +2274,16 @@ async fn run_wait_result_flow_by_path_internal(
 
     let scheduled_for = run_query.get_scheduled_for(&db).await?;
 
-    let tag = sqlx::query_scalar!(
-        "SELECT tag from flow WHERE path = $1 and workspace_id = $2",
+    let (tag, dedicated_worker, early_return) = sqlx::query!(
+        "SELECT tag, dedicated_worker, value->>'early_return' as early_return from flow WHERE path = $1 and workspace_id = $2",
         flow_path,
         w_id
     )
     .fetch_optional(&db)
     .await?
-    .flatten();
+    .map(|x| (x.tag, x.dedicated_worker, x.early_return))
+    .unwrap_or_else(|| (None, None, None));
+
     check_tag_available_for_workspace(&w_id, &tag).await?;
     let tx = PushIsolationLevel::Isolated(user_db.clone(), authed.clone().into(), rsmq);
 
@@ -2201,7 +2291,7 @@ async fn run_wait_result_flow_by_path_internal(
         &db,
         tx,
         &w_id,
-        JobPayload::Flow(flow_path.to_string()),
+        JobPayload::Flow { path: flow_path.to_string(), dedicated_worker },
         args,
         &authed.username,
         &authed.email,
@@ -2223,7 +2313,7 @@ async fn run_wait_result_flow_by_path_internal(
     .await?;
     tx.commit().await?;
 
-    run_wait_result(authed, Extension(user_db), uuid, Path((w_id, flow_path))).await
+    run_wait_result(&db, uuid, Path((w_id, flow_path)), early_return).await
 }
 
 async fn run_preview_job(
@@ -2351,7 +2441,7 @@ async fn add_batch_jobs(
                 JobPayload::RawFlow { value: fv.clone(), path: None, restarted_from: None }
             } else {
                 if let Some(path) = batch_info.path.as_ref() {
-                    JobPayload::Flow(path.to_string())
+                    JobPayload::Flow { path: path.to_string(), dedicated_worker: None }
                 } else {
                     Err(anyhow::anyhow!(
                         "Path is required if no value is not provided"
@@ -2788,7 +2878,7 @@ async fn get_completed_job<'a>(
     Path((w_id, id)): Path<(String, Uuid)>,
 ) -> error::Result<Response> {
     let job_o = sqlx::query("SELECT id, workspace_id, parent_job, created_by, created_at, duration_ms, success, script_hash, script_path, 
-    CASE WHEN pg_column_size(args) < 2000000 THEN args ELSE '\"WINDMILL_TOO_BIG\"'::jsonb END as args, CASE WHEN pg_column_size(result) < 2000000 THEN result ELSE '\"WINDMILL_TOO_BIG\"'::jsonb END as result, logs, deleted, raw_code, canceled, canceled_by, canceled_reason, job_kind, env_id,
+    CASE WHEN args is null or pg_column_size(args) < 2000000 THEN args ELSE '\"WINDMILL_TOO_BIG\"'::jsonb END as args, CASE WHEN result is null or pg_column_size(result) < 2000000 THEN result ELSE '\"WINDMILL_TOO_BIG\"'::jsonb END as result, logs, deleted, raw_code, canceled, canceled_by, canceled_reason, job_kind, env_id,
     schedule_path, permissioned_as, flow_status, raw_flow, is_flow_step, language, started_at, is_skipped,
     raw_lock, email, visible_to_owner, mem_peak, tag, priority FROM completed_job WHERE id = $1 AND workspace_id = $2")
         .bind(id)
@@ -2907,7 +2997,7 @@ async fn delete_completed_job<'a>(
 
     require_admin(authed.is_admin, &authed.username)?;
     let job_o = sqlx::query(
-        "UPDATE completed_job SET logs = '', result = null, deleted = true WHERE id = $1 AND workspace_id = $2 \
+        "UPDATE completed_job SET args = null, logs = '', result = null, deleted = true WHERE id = $1 AND workspace_id = $2 \
          RETURNING *",
     )
     .bind(id)

backend/windmill-api/src/lib.rs

@@ -57,6 +57,7 @@ mod granular_acls;
 mod groups;
 mod inputs;
 mod integration;
+pub mod job_helpers;
 pub mod jobs;
 pub mod oauth2;
 mod openai;
@@ -111,7 +112,7 @@ pub async fn run_server(
     rsmq: Option<rsmq_async::MultiplexedRsmq>,
     addr: SocketAddr,
     mut rx: tokio::sync::broadcast::Receiver<()>,
-    port_tx: tokio::sync::oneshot::Sender<u16>,
+    port_tx: tokio::sync::oneshot::Sender<String>,
     server_mode: bool,
 ) -> anyhow::Result<()> {
     if let Some(mut rsmq) = rsmq.clone() {
@@ -179,31 +180,32 @@ pub async fn run_server(
                         // Reordered alphabetically
                         .nest("/acls", granular_acls::workspaced_service())
                         .nest("/apps", apps::workspaced_service())
-                        .nest("/raw_apps", raw_apps::workspaced_service())
                         .nest("/audit", audit::workspaced_service())
                         .nest("/capture", capture::workspaced_service())
+                        .nest(
+                            "/embeddings",
+                            embeddings::workspaced_service(embeddings_db.clone()),
+                        )
+                        .nest("/drafts", drafts::workspaced_service())
                         .nest("/favorites", favorite::workspaced_service())
                         .nest("/flows", flows::workspaced_service())
                         .nest("/folders", folders::workspaced_service())
                         .nest("/groups", groups::workspaced_service())
                         .nest("/inputs", inputs::workspaced_service())
+                        .nest("/job_helpers", job_helpers::workspaced_service())
                         .nest("/jobs", jobs::workspaced_service())
                         .nest("/oauth", oauth2::workspaced_service())
+                        .nest("/openai", openai::workspaced_service())
+                        .nest("/raw_apps", raw_apps::workspaced_service())
                         .nest("/resources", resources::workspaced_service())
                         .nest("/schedules", schedule::workspaced_service())
                         .nest("/scripts", scripts::workspaced_service())
-                        .nest("/drafts", drafts::workspaced_service())
                         .nest(
                             "/users",
                             users::workspaced_service().layer(Extension(argon2.clone())),
                         )
                         .nest("/variables", variables::workspaced_service())
-                        .nest("/workspaces", workspaces::workspaced_service())
-                        .nest("/openai", openai::workspaced_service())
-                        .nest(
-                            "/embeddings",
-                            embeddings::workspaced_service(embeddings_db.clone()),
-                        ),
+                        .nest("/workspaces", workspaces::workspaced_service()),
                 )
                 .nest("/workspaces", workspaces::global_service())
                 .nest(
@@ -245,6 +247,10 @@ pub async fn run_server(
                     "/w/:workspace_id/jobs_u",
                     jobs::global_service().layer(cors.clone()),
                 )
+                .nest(
+                    "/w/:workspace_id/resources_u",
+                    resources::public_service().layer(cors.clone()),
+                )
                 .nest(
                     "/w/:workspace_id/capture_u",
                     capture::global_service().layer(cors),
@@ -278,7 +284,7 @@ pub async fn run_server(
     );
 
     port_tx
-        .send(server.local_addr().port())
+        .send(format!("http://localhost:{}", server.local_addr().port()))
         .expect("Failed to send port");
 
     let server = server.with_graceful_shutdown(async {

backend/windmill-api/src/oauth2.rs

@@ -858,7 +858,10 @@ async fn slack_command(
     if let Some(settings) = settings {
         if let Some(path) = &settings.slack_command_script {
             let (payload, tag) = if let Some(path) = path.strip_prefix("flow/") {
-                (JobPayload::Flow(path.to_string()), None)
+                (
+                    JobPayload::Flow { path: path.to_string(), dedicated_worker: None },
+                    None,
+                )
             } else {
                 let path = path.strip_prefix("script/").unwrap_or_else(|| path);
                 let (

backend/windmill-api/src/openai.rs

@@ -126,8 +126,9 @@ async fn proxy(
         "https://api.openai.com/v1"
     };
 
+    let url = format!("{}/{}", base_url, openai_path);
     let mut request = HTTP_CLIENT
-        .post(base_url.to_string() + "/" + &openai_path)
+        .post(url)
         .header("content-type", "application/json")
         .body(body);
 

backend/windmill-api/src/resources.rs

@@ -12,11 +12,14 @@ use crate::{
     webhook_util::{WebhookMessage, WebhookShared},
 };
 use axum::{
+    body,
     extract::{Extension, Path, Query},
+    response::Response,
     routing::{delete, get, post},
     Json, Router,
 };
-use hyper::StatusCode;
+use bytes::Bytes;
+use hyper::{header, StatusCode};
 use serde::{Deserialize, Serialize};
 use serde_json::{value::RawValue, Value};
 use sql_builder::{bind::Bind, SqlBuilder};
@@ -56,6 +59,10 @@ pub fn workspaced_service() -> Router {
         .route("/type/create", post(create_resource_type))
 }
 
+pub fn public_service() -> Router {
+    Router::new().route("/custom_component/:name", get(custom_component))
+}
+
 #[derive(FromRow, Serialize, Deserialize)]
 pub struct ResourceType {
     pub workspace_id: String,
@@ -240,6 +247,7 @@ async fn list_resources(
 async fn get_resource(
     authed: ApiAuthed,
     Extension(user_db): Extension<UserDB>,
+    Extension(db): Extension<DB>,
     Path((w_id, path)): Path<(String, StripPath)>,
 ) -> JsonResult<ListableResource> {
     let path = path.to_path();
@@ -262,7 +270,9 @@ async fn get_resource(
     .fetch_optional(&mut *tx)
     .await?;
     tx.commit().await?;
-
+    if resource_o.is_none() {
+        explain_resource_perm_error(&path, &w_id, &db).await?;
+    }
     let resource = not_found_if_none(resource_o, "Resource", path)?;
     Ok(Json(resource))
 }
@@ -288,6 +298,7 @@ async fn exists_resource(
 async fn get_resource_value(
     authed: ApiAuthed,
     Extension(user_db): Extension<UserDB>,
+    Extension(db): Extension<DB>,
     Path((w_id, path)): Path<(String, StripPath)>,
 ) -> JsonResult<Option<serde_json::Value>> {
     let path = path.to_path();
@@ -300,12 +311,82 @@ async fn get_resource_value(
     )
     .fetch_optional(&mut *tx)
     .await?;
+
     tx.commit().await?;
+    if value_o.is_none() {
+        explain_resource_perm_error(&path, &w_id, &db).await?;
+    }
 
     let value = not_found_if_none(value_o, "Resource", path)?;
     Ok(Json(value))
 }
 
+async fn explain_resource_perm_error(
+    path: &str,
+    w_id: &str,
+    db: &sqlx::Pool<Postgres>,
+) -> windmill_common::error::Result<()> {
+    let extra_perms = sqlx::query_scalar!(
+        "SELECT extra_perms from resource WHERE path = $1 AND workspace_id = $2",
+        path,
+        w_id
+    )
+    .fetch_optional(db)
+    .await?
+    .ok_or_else(|| Error::NotFound(format!("Resource {} not found", path)))?;
+    if path.starts_with("f/") {
+        let folder = path.split("/").nth(1).ok_or_else(|| {
+            Error::BadRequest(format!(
+                "path {} should have at least 2 components separated by /",
+                path
+            ))
+        })?;
+        let folder_extra_perms = sqlx::query_scalar!(
+            "SELECT extra_perms from folder WHERE name = $1 AND workspace_id = $2",
+            folder,
+            w_id
+        )
+        .fetch_optional(db)
+        .await?;
+        return Err(Error::NotAuthorized(format!(
+            "Resource exists but you don't have access to it:\nresource perms: {}\nfolder perms: {}",
+            serde_json::to_string_pretty(&extra_perms).unwrap_or_default(), serde_json::to_string_pretty(&folder_extra_perms).unwrap_or_default()
+        )));
+    } else {
+        return Err(Error::NotAuthorized(format!(
+            "Resource exists but you don't have access to it:\nresource perms: {}",
+            serde_json::to_string_pretty(&extra_perms).unwrap_or_default()
+        )));
+    }
+}
+
+async fn custom_component(
+    authed: ApiAuthed,
+    Extension(user_db): Extension<UserDB>,
+    Path((w_id, name)): Path<(String, String)>,
+) -> Result<Response> {
+    let mut tx = user_db.begin(&authed).await?;
+
+    let cc_o = sqlx::query_scalar!(
+        "SELECT value->>'js' FROM resource
+        WHERE path = $1 AND workspace_id = $2",
+        format!("f/app_custom/{name}"),
+        &w_id
+    )
+    .fetch_optional(&mut *tx)
+    .await?
+    .flatten();
+
+    tx.commit().await?;
+
+    let cc = not_found_if_none(cc_o, "Custom Component", name)?;
+    let res = Response::builder().header(header::CONTENT_TYPE, "text/javascript");
+
+    Ok(res
+        .body(body::boxed(body::Full::from(Bytes::from(cc))))
+        .unwrap())
+}
+
 #[derive(Deserialize)]
 struct JobInfo {
     job_id: Option<Uuid>,
@@ -313,6 +394,7 @@ struct JobInfo {
 async fn get_resource_value_interpolated(
     authed: ApiAuthed,
     Extension(user_db): Extension<UserDB>,
+    Extension(db): Extension<DB>,
     Tokened { token }: Tokened,
     Path((w_id, path)): Path<(String, StripPath)>,
     Query(job_info): Query<JobInfo>,
@@ -328,11 +410,23 @@ async fn get_resource_value_interpolated(
     .fetch_optional(&mut *tx)
     .await?;
     tx.commit().await?;
+    if value_o.is_none() {
+        explain_resource_perm_error(&path, &w_id, &db).await?;
+    }
 
     let value = not_found_if_none(value_o, "Resource", path)?;
     if let Some(value) = value {
         Ok(Json(Some(
-            transform_json_value(&authed, &user_db, &w_id, value, &job_info.job_id, &token).await?,
+            transform_json_value(
+                &authed,
+                &user_db,
+                &db,
+                &w_id,
+                value,
+                &job_info.job_id,
+                &token,
+            )
+            .await?,
         )))
     } else {
         Ok(Json(None))
@@ -345,6 +439,7 @@ use async_recursion::async_recursion;
 pub async fn transform_json_value<'c>(
     authed: &ApiAuthed,
     user_db: &UserDB,
+    db: &DB,
     workspace: &str,
     v: Value,
     job_id: &Option<Uuid>,
@@ -354,8 +449,8 @@ pub async fn transform_json_value<'c>(
         Value::String(y) if y.starts_with("$var:") => {
             let path = y.strip_prefix("$var:").unwrap();
             let tx: Transaction<'_, Postgres> = user_db.clone().begin(authed).await?;
-            let v =
-                crate::variables::get_value_internal(tx, workspace, path, &authed.username).await?;
+            let v = crate::variables::get_value_internal(tx, db, workspace, path, &authed.username)
+                .await?;
             Ok(Value::String(v))
         }
         Value::String(y) if y.starts_with("$res:") => {
@@ -374,7 +469,7 @@ pub async fn transform_json_value<'c>(
             tx.commit().await?;
             let v = not_found_if_none(v, "Resource", path)?;
             if let Some(v) = v {
-                transform_json_value(authed, user_db, workspace, v, job_id, token).await
+                transform_json_value(authed, user_db, db, workspace, v, job_id, token).await
             } else {
                 Ok(Value::Null)
             }
@@ -432,7 +527,7 @@ pub async fn transform_json_value<'c>(
             for (a, b) in m.clone().into_iter() {
                 m.insert(
                     a.clone(),
-                    transform_json_value(authed, user_db, workspace, b, job_id, token).await?,
+                    transform_json_value(authed, user_db, db, workspace, b, job_id, token).await?,
                 );
             }
             Ok(Value::Object(m))

backend/windmill-api/src/schedule.rs

@@ -8,7 +8,7 @@
 
 use crate::{
     db::{ApiAuthed, DB},
-    settings::set_global_setting_internal,
+    settings::{delete_global_setting, set_global_setting_internal},
     users::maybe_refresh_folders,
     utils::require_super_admin,
 };
@@ -26,7 +26,6 @@ use windmill_audit::{audit_log, ActionKind};
 use windmill_common::{
     db::UserDB,
     error::{Error, JsonResult, Result},
-    jobs::JobKind,
     schedule::Schedule,
     utils::{not_found_if_none, paginate, Pagination, StripPath},
 };
@@ -43,6 +42,7 @@ pub fn workspaced_service() -> Router {
         .route("/delete/*path", delete(delete_schedule))
         .route("/setenabled/*path", post(set_enabled))
         .route("/setdefaulthandler", post(set_default_error_handler))
+    // .route("/catchup/*path", post(do_catchup).get(list_catchup))
 }
 
 pub fn global_service() -> Router {
@@ -70,16 +70,23 @@ pub struct NewSchedule {
 
 #[derive(Serialize, Deserialize)]
 pub struct ErrorOrRecoveryHandler {
-    pub handler_type: String, // 'error' or 'recovery'
+    pub handler_type: HandlerType,
     pub override_existing: bool,
 
-    pub path: String,
+    pub path: Option<String>,
     pub extra_args: Option<serde_json::Value>,
     pub number_of_occurence: Option<i32>,
     pub number_of_occurence_exact: Option<bool>,
     pub workspace_handler_muted: Option<bool>,
 }
 
+#[derive(Serialize, Deserialize)]
+#[serde(rename_all(serialize = "lowercase", deserialize = "lowercase"))]
+pub enum HandlerType {
+    Error,
+    Recovery,
+}
+
 async fn check_path_conflict<'c>(
     tx: &mut Transaction<'c, Postgres>,
     w_id: &str,
@@ -119,16 +126,6 @@ async fn create_schedule(
         ));
     }
 
-    #[cfg(not(feature = "enterprise"))]
-    if ns.on_failure.is_some()
-        && ns.on_failure.as_ref().unwrap()
-            == "script/hub/5792/workspace-or-schedule-error-handler-slack"
-    {
-        return Err(Error::BadRequest(
-            "Slack error handler is only available in enterprise version".to_string(),
-        ));
-    }
-
     #[cfg(not(feature = "enterprise"))]
     if ns.on_failure_times.is_some() && ns.on_failure_times.unwrap() > 1 {
         return Err(Error::BadRequest(
@@ -220,17 +217,7 @@ async fn edit_schedule(
 
     cron::Schedule::from_str(&es.schedule).map_err(|e| Error::BadRequest(e.to_string()))?;
 
-    let is_flow = sqlx::query_scalar!(
-        "SELECT is_flow FROM schedule WHERE path = $1 AND workspace_id = $2",
-        path,
-        w_id
-    )
-    .fetch_optional(&mut tx)
-    .await?;
-
-    let is_flow = not_found_if_none(is_flow, "Schedule", &path)?;
-
-    clear_schedule(tx.transaction_mut(), path, is_flow, &w_id).await?;
+    clear_schedule(tx.transaction_mut(), path, &w_id).await?;
     let schedule = sqlx::query_as!(
         Schedule,
         "UPDATE schedule SET schedule = $1, timezone = $2, args = $3, on_failure = $4, on_failure_times = $5, on_failure_exact = $6, on_failure_extra_args = $7, on_recovery = $8, on_recovery_times = $9, on_recovery_extra_args = $10, ws_error_handler_muted = $11 
@@ -445,7 +432,7 @@ pub async fn set_enabled(
 
     let schedule = not_found_if_none(schedule_o, "Schedule", path)?;
 
-    clear_schedule(tx.transaction_mut(), path, schedule.is_flow, &w_id).await?;
+    clear_schedule(tx.transaction_mut(), path, &w_id).await?;
 
     audit_log(
         &mut tx,
@@ -469,6 +456,54 @@ pub async fn set_enabled(
     ))
 }
 
+// pub async fn do_catchup(
+//     authed: ApiAuthed,
+//     Extension(db): Extension<DB>,
+//     Extension(user_db): Extension<UserDB>,
+//     Extension(rsmq): Extension<Option<rsmq_async::MultiplexedRsmq>>,
+//     Path((w_id, path)): Path<(String, StripPath)>,
+//     Json(payload): Json<SetEnabled>,
+// ) -> Result<String> {
+//     let mut tx: QueueTransaction<'_, rsmq_async::MultiplexedRsmq> =
+//         (rsmq, user_db.begin(&authed).await?).into();
+//     let path = path.to_path();
+//     let schedule_o = sqlx::query_as!(
+//         Schedule,
+//         "UPDATE schedule SET enabled = $1, email = $2 WHERE path = $3 AND workspace_id = $4 RETURNING *",
+//         &payload.enabled,
+//         authed.email,
+//         path,
+//         w_id
+//     )
+//     .fetch_optional(&mut tx)
+//     .await?;
+
+//     let schedule = not_found_if_none(schedule_o, "Schedule", path)?;
+
+//     clear_schedule(tx.transaction_mut(), path, &w_id).await?;
+
+//     audit_log(
+//         &mut tx,
+//         &authed.username,
+//         "schedule.setenabled",
+//         ActionKind::Update,
+//         &w_id,
+//         Some(path),
+//         Some([("enabled", payload.enabled.to_string().as_ref())].into()),
+//     )
+//     .await?;
+
+//     if payload.enabled {
+//         tx = push_scheduled_job(&db, tx, schedule).await?;
+//     }
+//     tx.commit().await?;
+
+//     Ok(format!(
+//         "succesfully updated schedule at path {} to status {}",
+//         path, payload.enabled
+//     ))
+// }
+
 async fn delete_schedule(
     authed: ApiAuthed,
     Extension(user_db): Extension<UserDB>,
@@ -477,6 +512,8 @@ async fn delete_schedule(
     let mut tx = user_db.begin(&authed).await?;
     let path = path.to_path();
 
+    clear_schedule(&mut tx, path, &w_id).await?;
+
     sqlx::query!(
         "DELETE FROM schedule WHERE path = $1 AND workspace_id = $2",
         path,
@@ -508,66 +545,88 @@ async fn set_default_error_handler(
     Json(payload): Json<ErrorOrRecoveryHandler>,
 ) -> Result<()> {
     require_super_admin(&db, &authed.email).await?;
-    let (key, value) = match payload.handler_type.as_str() {
-        "error" => {
+    let (key, value) = match payload.handler_type {
+        HandlerType::Error => {
             let key = format!("default_error_handler_{}", w_id);
-            let value = serde_json::json!({
-                "wsErrorHandlerMuted": payload.workspace_handler_muted,
-                "errorHandlerPath": payload.path,
-                "errorHandlerExtraArgs": payload.extra_args,
-                "failedTimes": payload.number_of_occurence,
-                "failedExact": payload.number_of_occurence_exact,
-            });
-            Ok((key, value))
+            if let Some(payload_path) = payload.path.as_ref() {
+                let value = serde_json::json!({
+                    "wsErrorHandlerMuted": payload.workspace_handler_muted,
+                    "errorHandlerPath": payload_path,
+                    "errorHandlerExtraArgs": payload.extra_args,
+                    "failedTimes": payload.number_of_occurence,
+                    "failedExact": payload.number_of_occurence_exact,
+                });
+                (key, Some(value))
+            } else {
+                (key, None)
+            }
         }
-        "recovery" => {
+        HandlerType::Recovery => {
             let key = format!("default_recovery_handler_{}", w_id);
-            let value = serde_json::json!({
-                "recoveryHandlerPath": payload.path,
-                "recoveryHandlerExtraArgs": payload.extra_args,
-                "recoveredTimes": payload.number_of_occurence,
-            });
-            Ok((key, value))
+            if let Some(payload_path) = payload.path.as_ref() {
+                let value = serde_json::json!({
+                    "recoveryHandlerPath": payload_path,
+                    "recoveryHandlerExtraArgs": payload.extra_args,
+                    "recoveredTimes": payload.number_of_occurence,
+                });
+                (key, Some(value))
+            } else {
+                (key, None)
+            }
         }
-        _ => Err(Error::BadRequest(
-            "handler_type must be either 'error' or 'recovery'".to_string(),
-        )),
-    }?;
+    };
 
-    set_global_setting_internal(&db, key, value).await?;
+    if let Some(value_content) = value {
+        set_global_setting_internal(&db, key, value_content).await?;
+    } else {
+        delete_global_setting(&db, key.as_str()).await?;
+    }
 
     if payload.override_existing {
-        match payload.handler_type.as_str() {
-            "error" => {
-                sqlx::query!(
-                    "UPDATE schedule SET ws_error_handler_muted = $1, on_failure = $2, on_failure_extra_args = $3, on_failure_times = $4, on_failure_exact = $5 WHERE workspace_id = $6",
-                    payload.workspace_handler_muted,
-                    payload.path,
-                    payload.extra_args,
-                    payload.number_of_occurence,
-                    payload.number_of_occurence_exact,
-                    w_id,
-                )
-                .execute(&db)
-                .await?;
-                Ok(())
+        match payload.handler_type {
+            HandlerType::Error => {
+                if payload.path.is_some() {
+                    sqlx::query!(
+                        "UPDATE schedule SET ws_error_handler_muted = $1, on_failure = $2, on_failure_extra_args = $3, on_failure_times = $4, on_failure_exact = $5 WHERE workspace_id = $6",
+                        payload.workspace_handler_muted,
+                        payload.path,
+                        payload.extra_args,
+                        payload.number_of_occurence,
+                        payload.number_of_occurence_exact,
+                        w_id,
+                    )
+                    .execute(&db)
+                    .await?;
+                } else {
+                    sqlx::query!(
+                        "UPDATE schedule SET ws_error_handler_muted = false, on_failure = NULL, on_failure_extra_args = NULL, on_failure_times = NULL, on_failure_exact = NULL WHERE workspace_id = $1",
+                        w_id,
+                    )
+                    .execute(&db)
+                    .await?;
+                }
             }
-            "recovery" => {
-                sqlx::query!(
-                    "UPDATE schedule SET on_recovery = $1, on_recovery_extra_args = $2, on_recovery_times = $3 WHERE workspace_id = $4",
-                    payload.path,
-                    payload.extra_args,
-                    payload.number_of_occurence,
-                    w_id,
-                )
-                .execute(&db)
-                .await?;
-                Ok(())
+            HandlerType::Recovery => {
+                if payload.path.is_some() {
+                    sqlx::query!(
+                        "UPDATE schedule SET on_recovery = $1, on_recovery_extra_args = $2, on_recovery_times = $3 WHERE workspace_id = $4",
+                        payload.path,
+                        payload.extra_args,
+                        payload.number_of_occurence,
+                        w_id,
+                    )
+                    .execute(&db)
+                    .await?;
+                } else {
+                    sqlx::query!(
+                        "UPDATE schedule SET on_recovery = NULL, on_recovery_extra_args = NULL, on_recovery_times = NULL WHERE workspace_id = $1",
+                        w_id,
+                    )
+                    .execute(&db)
+                    .await?;
+                }
             }
-            _ => Err(Error::BadRequest(
-                "handler_type must be either 'error' or 'recovery'".to_string(),
-            )),
-        }?;
+        }
     }
     Ok(())
 }
@@ -616,18 +675,11 @@ pub struct EditSchedule {
 pub async fn clear_schedule<'c>(
     db: &mut Transaction<'c, Postgres>,
     path: &str,
-    is_flow: bool,
     w_id: &str,
 ) -> Result<()> {
-    let job_kind = if is_flow {
-        JobKind::Flow
-    } else {
-        JobKind::Script
-    };
     sqlx::query!(
-        "DELETE FROM queue WHERE schedule_path = $1 AND running = false AND job_kind = $2 AND workspace_id = $3",
+        "DELETE FROM queue WHERE schedule_path = $1 AND running = false AND workspace_id = $2",
         path,
-        job_kind as JobKind,
         w_id
     )
     .execute(&mut **db)
@@ -639,3 +691,9 @@ pub async fn clear_schedule<'c>(
 pub struct SetEnabled {
     pub enabled: bool,
 }
+
+#[derive(Deserialize)]
+pub struct Catchup {
+    pub from: DateTime<Utc>,
+    pub to: Option<DateTime<Utc>>,
+}

backend/windmill-api/src/scripts.rs

@@ -525,7 +525,7 @@ async fn create_script(
         }
 
         for schedule in schedulables {
-            clear_schedule(tx.transaction_mut(), &schedule.path, false, &w_id).await?;
+            clear_schedule(tx.transaction_mut(), &schedule.path, &w_id).await?;
 
             if schedule.enabled {
                 tx = push_scheduled_job(&db, tx, schedule).await?;
@@ -598,11 +598,22 @@ async fn create_script(
             }
             _ => ns.content,
         };
+        let tag = if ns.dedicated_worker.is_some_and(|x| x) {
+            Some(format!("{}:{}", &w_id, &ns.path,))
+        } else {
+            ns.tag
+        };
         let (_, new_tx) = windmill_queue::push(
             &db,
             tx,
             &w_id,
-            JobPayload::Dependencies { hash, dependencies, language: ns.language, path: ns.path },
+            JobPayload::Dependencies {
+                hash,
+                dependencies,
+                language: ns.language,
+                path: ns.path,
+                dedicated_worker: ns.dedicated_worker,
+            },
             PushArgs::empty(),
             &authed.username,
             &authed.email,
@@ -616,7 +627,7 @@ async fn create_script(
             false,
             None,
             true,
-            ns.tag,
+            tag,
             None,
             None,
             None,
@@ -803,6 +814,8 @@ async fn raw_script_by_path(
     }
     let path = path
         .trim_end_matches(".py")
+        .trim_end_matches(".bun.ts")
+        .trim_end_matches(".deno.ts")
         .trim_end_matches(".ts")
         .trim_end_matches(".go")
         .trim_end_matches(".sh");

backend/windmill-api/src/settings.rs

@@ -159,7 +159,9 @@ pub async fn get_global_setting(
     authed: ApiAuthed,
     Path(key): Path<String>,
 ) -> JsonResult<serde_json::Value> {
-    require_super_admin(&db, &authed.email).await?;
+    if !key.starts_with("default_error_handler_") && !key.starts_with("default_recovery_handler_") {
+        require_super_admin(&db, &authed.email).await?;
+    }
     let value = sqlx::query!("SELECT value FROM global_settings WHERE name = $1", key)
         .fetch_optional(&db)
         .await?
@@ -170,7 +172,13 @@ pub async fn get_global_setting(
 
 pub async fn send_stats(Extension(db): Extension<DB>, authed: ApiAuthed) -> Result<String> {
     require_super_admin(&db, &authed.email).await?;
-    windmill_common::stats::send_stats(&"manual".to_string(), &HTTP_CLIENT, &db).await?;
+    windmill_common::stats::send_stats(
+        &"manual".to_string(),
+        &windmill_common::utils::Mode::Server,
+        &HTTP_CLIENT,
+        &db,
+    )
+    .await?;
 
     Ok("Sent stats".to_string())
 }

backend/windmill-api/src/users.rs

@@ -42,6 +42,7 @@ use time::OffsetDateTime;
 use tower_cookies::{Cookie, Cookies};
 use tracing::{Instrument, Span};
 use windmill_audit::{audit_log, ActionKind};
+use windmill_common::oauth2::REQUIRE_PREEXISTING_USER_FOR_OAUTH;
 use windmill_common::users::truncate_token;
 use windmill_common::worker::{CLOUD_HOSTED, SERVER_CONFIG};
 use windmill_common::{
@@ -93,6 +94,7 @@ pub fn global_service() -> Router {
             "/tutorial_progress",
             post(update_tutorial_progress).get(get_tutorial_progress),
         )
+        .route("/leave_instance", post(leave_instance))
     // .route("/list_invite_codes", get(list_invite_codes))
     // .route("/create_invite_code", post(create_invite_code))
     // .route("/signup", post(signup))
@@ -1415,6 +1417,29 @@ async fn add_user_to_workspace<'c>(
     Ok(tx)
 }
 
+async fn leave_instance(
+    Extension(db): Extension<DB>,
+    ApiAuthed { email, username, .. }: ApiAuthed,
+) -> Result<String> {
+    let mut tx = db.begin().await?;
+    sqlx::query!("DELETE FROM password WHERE email = $1", &email)
+        .execute(&mut *tx)
+        .await?;
+
+    audit_log(
+        &mut *tx,
+        &username,
+        "workspaces.leave",
+        ActionKind::Delete,
+        "global",
+        Some(&email),
+        None,
+    )
+    .await?;
+    tx.commit().await?;
+
+    Ok(format!("Left instance",))
+}
 async fn update_workspace_user(
     ApiAuthed { username, is_admin, .. }: ApiAuthed,
     Extension(db): Extension<DB>,
@@ -1577,6 +1602,9 @@ async fn create_user(
         ));
     }
 
+    #[cfg(not(feature = "enterprise"))]
+    _check_nb_of_user(&db).await?;
+
     sqlx::query!(
         "INSERT INTO password(email, verified, password_hash, login_type, super_admin, name, \
          company)
@@ -2289,13 +2317,23 @@ pub struct LoginUserInfo {
 }
 
 async fn _check_nb_of_user(db: &DB) -> Result<()> {
-    let nb_groups =
+    let nb_users_sso =
         sqlx::query_scalar!("SELECT COUNT(*) FROM password WHERE login_type != 'password'",)
             .fetch_one(db)
             .await?;
-    if nb_groups.unwrap_or(0) >= 50 {
+    if nb_users_sso.unwrap_or(0) >= 10 {
         return Err(Error::BadRequest(
-            "You have reached the maximum number of oauth users accounts (50) without an enterprise license"
+            "You have reached the maximum number of oauth users accounts (10) without an enterprise license"
+                .to_string(),
+        ));
+    }
+
+    let nb_users = sqlx::query_scalar!("SELECT COUNT(*) FROM password",)
+        .fetch_one(db)
+        .await?;
+    if nb_users.unwrap_or(0) >= 50 {
+        return Err(Error::BadRequest(
+            "You have reached the maximum number of accounts (50) without an enterprise license"
                 .to_string(),
         ));
     }
@@ -2316,9 +2354,12 @@ pub async fn login_externally(
             .bind(email)
             .fetch_optional(&mut *tx)
             .await?;
+    let require_existing_user =
+        REQUIRE_PREEXISTING_USER_FOR_OAUTH.load(std::sync::atomic::Ordering::Relaxed);
+
     if let Some((email, login_type, super_admin)) = login {
         let login_type = serde_json::json!(login_type);
-        if login_type == client_name {
+        if require_existing_user || login_type == client_name {
             crate::users::create_session_token(&email, super_admin, &mut tx, cookies).await?;
         } else {
             return Err(error::Error::BadRequest(format!(
@@ -2350,6 +2391,13 @@ pub async fn login_externally(
             .await?;
         };
     } else {
+        if require_existing_user {
+            return Err(error::Error::BadRequest(format!(
+                "no user with the email associated to this login exists (windmill is set to only \
+                     allow oauth logins for existing users)"
+            )));
+        }
+
         let mut name = user.clone().and_then(|x| x.name);
         if (name.is_none() || name == Some(String::new())) && user.is_some() {
             name = user.clone().unwrap().displayName;

backend/windmill-api/src/variables.rs

@@ -107,6 +107,7 @@ struct GetVariableQuery {
 async fn get_variable(
     authed: ApiAuthed,
     Extension(user_db): Extension<UserDB>,
+    Extension(db): Extension<DB>,
     Query(q): Query<GetVariableQuery>,
     Path((w_id, path)): Path<(String, StripPath)>,
 ) -> JsonResult<ListableVariable> {
@@ -128,7 +129,12 @@ async fn get_variable(
     .fetch_optional(&mut *tx)
     .await?;
 
-    let variable = not_found_if_none(variable_o, "Variable", &path)?;
+    let variable = if let Some(variable) = variable_o {
+        variable
+    } else {
+        explain_variable_perm_error(&path, &w_id, &db).await?;
+        unreachable!()
+    };
 
     let decrypt_secret = q.decrypt_secret.unwrap_or(true);
 
@@ -172,17 +178,20 @@ async fn get_variable(
 async fn get_value(
     authed: ApiAuthed,
     Extension(user_db): Extension<UserDB>,
+    Extension(db): Extension<DB>,
+
     Path((w_id, path)): Path<(String, StripPath)>,
 ) -> JsonResult<String> {
     let path = path.to_path();
     let tx = user_db.begin(&authed).await?;
-    return get_value_internal(tx, &w_id, &path, &authed.username)
+    return get_value_internal(tx, &db, &w_id, &path, &authed.username)
         .await
         .map(Json);
 }
 
 pub async fn get_value_internal<'c>(
     mut tx: Transaction<'c, Postgres>,
+    db: &DB,
     w_id: &str,
     path: &str,
     username: &str,
@@ -194,7 +203,12 @@ pub async fn get_value_internal<'c>(
     .fetch_optional(&mut *tx)
     .await?;
 
-    let variable = not_found_if_none(variable_o, "Variable", &path)?;
+    let variable = if let Some(variable) = variable_o {
+        variable
+    } else {
+        explain_variable_perm_error(path, w_id, db).await?;
+        unreachable!()
+    };
 
     let r = if variable.is_secret {
         audit_log(
@@ -226,6 +240,45 @@ pub async fn get_value_internal<'c>(
     Ok(r)
 }
 
+async fn explain_variable_perm_error(
+    path: &str,
+    w_id: &str,
+    db: &sqlx::Pool<Postgres>,
+) -> windmill_common::error::Result<()> {
+    let extra_perms = sqlx::query_scalar!(
+        "SELECT extra_perms from variable WHERE path = $1 AND workspace_id = $2",
+        path,
+        w_id
+    )
+    .fetch_optional(db)
+    .await?
+    .ok_or_else(|| Error::NotFound(format!("Variable {} not found", path)))?;
+    if path.starts_with("f/") {
+        let folder = path.split("/").nth(1).ok_or_else(|| {
+            Error::BadRequest(format!(
+                "path {} should have at least 2 components separated by /",
+                path
+            ))
+        })?;
+        let folder_extra_perms = sqlx::query_scalar!(
+            "SELECT extra_perms from folder WHERE name = $1 AND workspace_id = $2",
+            folder,
+            w_id
+        )
+        .fetch_optional(db)
+        .await?;
+        return Err(Error::NotAuthorized(format!(
+            "Variable exists but you don't have access to it:\nvariable perms: {}\nfolder perms: {}",
+            serde_json::to_string_pretty(&extra_perms).unwrap_or_default(), serde_json::to_string_pretty(&folder_extra_perms).unwrap_or_default()
+        )));
+    } else {
+        return Err(Error::NotAuthorized(format!(
+            "Variable exists but you don't have access to it:\nvariable perms: {}",
+            serde_json::to_string_pretty(&extra_perms).unwrap_or_default()
+        )));
+    }
+}
+
 async fn exists_variable(
     Extension(db): Extension<DB>,
     Path((w_id, path)): Path<(String, StripPath)>,

backend/windmill-api/src/workspaces.rs

@@ -35,11 +35,14 @@ use chrono::Utc;
 use magic_crypt::MagicCryptTrait;
 #[cfg(feature = "enterprise")]
 use stripe::CustomerId;
+#[cfg(feature = "enterprise")]
+use chrono::{TimeZone, Datelike};
 use uuid::Uuid;
 use windmill_audit::{audit_log, ActionKind};
 use windmill_common::db::UserDB;
 use windmill_common::schedule::Schedule;
 use windmill_common::users::username_to_permissioned_as;
+use windmill_common::worker::CLOUD_HOSTED;
 use windmill_common::{
     error::{to_anyhow, Error, JsonResult, Result},
     flows::Flow,
@@ -50,6 +53,7 @@ use windmill_common::{
 };
 use windmill_queue::QueueTransaction;
 
+
 use hyper::{header, StatusCode};
 use serde::{Deserialize, Serialize};
 use serde_json::{json, Map};
@@ -74,14 +78,17 @@ pub fn workspaced_service() -> Router {
         .route("/edit_auto_invite", post(edit_auto_invite))
         .route("/edit_deploy_to", post(edit_deploy_to))
         .route("/tarball", get(tarball_workspace))
+        .route("/is_premium", get(is_premium))
         .route("/premium_info", get(premium_info))
         .route("/edit_copilot_config", post(edit_copilot_config))
         .route("/get_copilot_info", get(get_copilot_info) )
-        .route("/edit_error_handler", post(edit_error_handler));
+        .route("/edit_error_handler", post(edit_error_handler))
+        .route("/edit_large_file_storage_config", post(edit_large_file_storage_config))
+        .route("/leave", post(leave_workspace));
 
     #[cfg(feature = "enterprise")]
     { 
-        if std::env::var("STRIPE_KEY").is_err() {
+        if STRIPE_KEY.is_none() {
             return router;
         } else {
             tracing::info!("stripe enabled");
@@ -106,8 +113,15 @@ pub fn global_service() -> Router {
         .route("/allowed_domain_auto_invite", get(is_allowed_auto_domain))
         .route("/unarchive/:workspace", post(unarchive_workspace))
         .route("/delete/:workspace", delete(delete_workspace))
+        .route("/create_workspace_require_superadmin", get(create_workspace_require_superadmin))
 }
 
+#[cfg(feature = "enterprise")]
+lazy_static::lazy_static! {
+    pub static ref STRIPE_KEY: Option<String> = std::env::var("STRIPE_KEY").ok();
+}
+
+
 #[derive(FromRow, Serialize)]
 struct Workspace {
     id: String,
@@ -115,6 +129,7 @@ struct Workspace {
     owner: String,
     deleted: bool,
     premium: bool,
+    is_overquota: bool,
 }
 
 #[derive(FromRow, Serialize, Debug)]
@@ -135,6 +150,7 @@ pub struct WorkspaceSettings {
     pub error_handler: Option<String>,
     pub error_handler_extra_args: Option<serde_json::Value>,
     pub error_handler_muted_on_cancel: Option<bool>,
+    pub large_file_storage: Option<serde_json::Value>, // effectively: DatasetsStorage
 }
 
 #[derive(FromRow, Serialize, Debug)]
@@ -178,6 +194,7 @@ struct EditDeployTo {
 #[derive(Deserialize)]
 struct EditAutoInvite {
     operator: Option<bool>,
+    invite_all: Option<bool>,
 }
 
 #[derive(Deserialize)]
@@ -191,6 +208,11 @@ struct EditCopilotConfig {
     code_completion_enabled: bool,
 }
 
+#[derive(Deserialize)]
+struct EditLargeFileStorageConfig {
+    large_file_storage: Option<LargeFileStorage>,
+}
+
 #[derive(Deserialize)]
 struct CreateWorkspace {
     id: String,
@@ -250,6 +272,18 @@ pub struct EditErrorHandler {
     pub error_handler_muted_on_cancel: Option<bool>,
 }
 
+#[derive(Serialize, Deserialize, Debug)]
+#[serde(tag = "type")]
+pub enum LargeFileStorage {
+    S3Storage(S3Storage),
+    // TODO: Add a filesystem type here in the future if needed
+}
+
+#[derive(Serialize, Deserialize, Debug)]
+pub struct S3Storage {
+    pub s3_resource_path: String,
+}
+
 async fn list_pending_invites(
     authed: ApiAuthed,
     Extension(user_db): Extension<UserDB>,
@@ -268,10 +302,30 @@ async fn list_pending_invites(
     Ok(Json(rows))
 }
 
-#[derive(Serialize, FromRow)]
+async fn is_premium(
+    authed: ApiAuthed,
+    Extension(db): Extension<DB>,
+    Path(w_id): Path<String>,
+) -> JsonResult<bool> {
+    require_admin(authed.is_admin, &authed.username)?;
+    let mut tx = db.begin().await?;
+    let row = sqlx::query_scalar!(
+        "SELECT premium FROM workspace WHERE workspace.id = $1",
+        &w_id
+       
+    )
+    .fetch_one(&mut *tx)
+    .await?;
+    tx.commit().await?;
+    Ok(Json(row))
+}
+
+
+#[derive(Serialize)]
 pub struct PremiumWorkspaceInfo {
     pub premium: bool,
     pub usage: Option<i32>,
+    pub seats: Option<i32>,
 }
 async fn premium_info(
     authed: ApiAuthed,
@@ -280,20 +334,46 @@ async fn premium_info(
 ) -> JsonResult<PremiumWorkspaceInfo> {
     require_admin(authed.is_admin, &authed.username)?;
     let mut tx = db.begin().await?;
-    let row = sqlx::query_as::<_, PremiumWorkspaceInfo>(
-        "SELECT premium, usage.usage FROM workspace LEFT JOIN usage ON usage.id = $1 AND month_ = EXTRACT(YEAR FROM current_date) * 12 + EXTRACT(MONTH FROM current_date) AND usage.is_workspace IS true WHERE workspace.id = $1",
+    let row = sqlx::query!(
+        r#"SELECT premium, usage.usage as "usage?", workspace_settings.customer_id, workspace_settings.plan FROM workspace LEFT JOIN workspace_settings ON workspace_settings.workspace_id = $1 LEFT JOIN usage ON usage.id = $1 AND month_ = EXTRACT(YEAR FROM current_date) * 12 + EXTRACT(MONTH FROM current_date) AND usage.is_workspace IS true WHERE workspace.id = $1"#,
+       &w_id
     )
-    .bind(w_id)
     .fetch_one(&mut *tx)
     .await?;
     tx.commit().await?;
-    Ok(Json(row))
+    let result = PremiumWorkspaceInfo {
+        premium: row.premium,
+        usage: row.usage,
+        seats: None,
+    };
+    #[cfg(feature = "enterprise")]
+    let mut result = result;
+    #[cfg(feature = "enterprise")]
+    if row.premium && row.plan == Some("team".to_string()) {
+        let customer_id = row.customer_id.ok_or(Error::InternalErr(format!("no customer id for workspace {}", w_id)))?;
+        let client = stripe::Client::new(STRIPE_KEY.clone().ok_or(Error::InternalErr(format!("stripe key not set")))?);
+        let customer_id = CustomerId::from_str(&customer_id).map_err(to_anyhow)?;
+        let subscriptions = stripe::Subscription::list(
+            &client,
+            &stripe::ListSubscriptions { customer: Some(customer_id.clone()), limit: Some(1), ..Default::default() },
+        ).await.map_err(to_anyhow)?;
+        if subscriptions.data.len() > 1 {
+            return Err(Error::InternalErr(format!("multiple subscriptions for customer {}, please contact us at ccontact@windmill.dev", customer_id)));
+        }
+        let subscription = subscriptions.data.get(0).ok_or_else(|| Error::InternalErr(format!("no subscription for customer {}", customer_id)))?;
+        result.seats = subscription.items.data.iter().filter_map(|item|{
+            item.price.clone().map(|p| p.metadata.map(|m| m.get("plan").filter(|plan| plan == &"team").map(|_| item.quantity.map(|x| x as i32)))).flatten().flatten().flatten()
+        }).collect::<Vec<_>>().get(0).copied();
+    }
+    
+    Ok(Json(result))
 }
 
 #[cfg(feature = "enterprise")]
 #[derive(Deserialize)]
 struct PlanQuery {
     plan: String,
+    seats: Option<i32>,
 }
 
 #[cfg(feature = "enterprise")]
@@ -301,38 +381,70 @@ async fn stripe_checkout(
     authed: ApiAuthed,
     Path(w_id): Path<String>,
     Query(plan): Query<PlanQuery>,
+    Extension(db): Extension<DB>,
 ) -> Result<Redirect> {
     // #[cfg(feature = "enterprise")]
     {
         require_admin(authed.is_admin, &authed.username)?;
-
-        let client = stripe::Client::new(std::env::var("STRIPE_KEY").expect("STRIPE_KEY"));
+        let client = stripe::Client::new(STRIPE_KEY.clone().ok_or(Error::InternalErr(format!("stripe key not set")))?);
         let base_url = BASE_URL.read().await.clone();
         let success_rd = format!("{}/workspace_settings/checkout?success=true", base_url);
         let failure_rd = format!("{}/workspace_settings/checkout?success=false", base_url);
         let checkout_session = {
-            let mut params = stripe::CreateCheckoutSession::new(&failure_rd, &success_rd);
+            let mut params = stripe::CreateCheckoutSession::new(&success_rd);
             params.mode = Some(stripe::CheckoutSessionMode::Subscription);
+            params.cancel_url = Some(&failure_rd);
             params.line_items = match plan.plan.as_str() {
                 "team" => Some(vec![
                     stripe::CreateCheckoutSessionLineItems {
-                        quantity: Some(1),
+                        quantity: Some(plan.seats.unwrap_or(1) as u64),
                         price: Some("price_1NCNOgGU3NdFi9eLuG4fZuEP".to_string()),
-                        ..Default::default()
-                    },
-                    stripe::CreateCheckoutSessionLineItems {
-                        quantity: None,
-                        price: Some("price_1NCNCpGU3NdFi9eLbiE6Ca42".to_string()),
+                        adjustable_quantity: Some(stripe::CreateCheckoutSessionLineItemsAdjustableQuantity {
+                            enabled: true,
+                            minimum: Some(1),
+                            ..Default::default()
+                        }),
                         ..Default::default()
                     },
                 ]),
                 _ => Err(Error::BadRequest("invalid plan".to_string()))?,
             };
-            params.customer_email = Some(&authed.email);
             params.client_reference_id = Some(&w_id);
+            let customer_id = sqlx::query_scalar!(
+                "SELECT customer_id FROM workspace_settings WHERE workspace_id = $1",
+                &w_id
+            )
+            .fetch_one(&db)
+            .await?;
+
+            match customer_id {
+                Some(customer_id) => {
+                    params.customer = Some(CustomerId::from_str(&customer_id).map_err(to_anyhow)?)
+                }
+                _ => params.customer_email = Some(&authed.email),
+            }
+            
+            params.subscription_data = Some(stripe::CreateCheckoutSessionSubscriptionData {
+                metadata: {
+                    let mut map = std::collections::HashMap::new();
+                    map.insert("workspace_id".to_string(), w_id.clone());
+                    Some(map)
+                },
+                billing_cycle_anchor: Some({
+                    // first of the next month (and possibly next year) at noon UTC
+                    let now = Utc::now();
+                    let date = if now.month() == 12 {
+                        Utc.with_ymd_and_hms(now.year() + 1, 1, 1, 12, 0, 0).single().unwrap()
+                    } else {
+                        Utc.with_ymd_and_hms(now.year(), now.month() + 1, 1, 12, 0, 0).single().unwrap()
+                    };
+                    date.timestamp()
+                }),
+                ..Default::default()
+            });
+
             stripe::CheckoutSession::create(&client, params)
-                .await
-                .unwrap()
+                .await.map_err(to_anyhow)?
         };
         let uri = checkout_session
             .url
@@ -355,7 +467,7 @@ async fn stripe_portal(
     .fetch_one(&db)
     .await?
     .ok_or_else(|| Error::InternalErr(format!("no customer id for workspace {}", w_id)))?;
-    let client = stripe::Client::new(std::env::var("STRIPE_KEY").expect("STRIPE_KEY"));
+    let client = stripe::Client::new(STRIPE_KEY.clone().ok_or(Error::InternalErr(format!("stripe key not set")))?);
     let success_rd = format!("{}/workspace_settings?tab=premium", BASE_URL.read().await.clone());
     let portal_session = {
         let customer_id = CustomerId::from_str(&customer_id).unwrap();
@@ -629,7 +741,25 @@ async fn edit_auto_invite(
     Json(ea): Json<EditAutoInvite>,
 ) -> Result<String> {
     require_admin(is_admin, &username)?;
-    let domain = email.split('@').last().unwrap();
+    
+    // #[cfg(not(feature = "enterprise"))]
+    // {
+    //     return Err(Error::BadRequest(
+    //         "Auto-invite is only available on enterprise".to_string(),
+    //     ));
+    // }
+
+    let domain = if ea.invite_all.is_some_and(|x| x) {
+        if  *CLOUD_HOSTED {
+        return Err(Error::BadRequest(
+            "invite_all is only available locally".to_string(),
+        ));
+        } else {
+            "*"
+        }  
+    } else {
+        email.split('@').last().unwrap()
+    };
 
     let mut tx = db.begin().await?;
 
@@ -653,7 +783,7 @@ async fn edit_auto_invite(
         sqlx::query!(
             "INSERT INTO workspace_invite
         (workspace_id, email, is_admin, operator)
-        SELECT $1::text, email, false, $3 FROM password WHERE email LIKE CONCAT('%', $2::text) AND NOT EXISTS (
+        SELECT $1::text, email, false, $3 FROM password WHERE ($2::text = '*' OR email LIKE CONCAT('%', $2::text)) AND NOT EXISTS (
             SELECT 1 FROM usr WHERE workspace_id = $1::text AND email = password.email
         )
         ON CONFLICT DO NOTHING",
@@ -802,6 +932,52 @@ async fn get_copilot_info(
     }))
 }
 
+async fn edit_large_file_storage_config(
+    authed: ApiAuthed,
+    Extension(db): Extension<DB>,
+    Path(w_id): Path<String>,
+    ApiAuthed { is_admin, username, .. }: ApiAuthed,
+    Json(new_config): Json<EditLargeFileStorageConfig>,
+) -> Result<String> {
+    require_admin(is_admin, &username)?;
+
+    let mut tx = db.begin().await?;
+
+    let args_for_audit = format!("{:?}", new_config.large_file_storage);
+    audit_log(
+        &mut *tx,
+        &authed.username,
+        "workspaces.edit_large_file_storage_config",
+        ActionKind::Update,
+        &w_id,
+        Some(&authed.email),
+        Some([("large_file_storage", args_for_audit.as_str())].into()),
+    )
+    .await?;
+
+    if let Some(lfs_config) = new_config.large_file_storage {
+        let serialized_lfs_config = serde_json::to_value::<LargeFileStorage>(lfs_config)
+            .map_err(|err| Error::InternalErr(err.to_string()))?;
+
+        sqlx::query!(
+            "UPDATE workspace_settings SET large_file_storage = $1 WHERE workspace_id = $2",
+            serialized_lfs_config,
+            &w_id
+        )
+        .execute(&mut *tx)
+        .await?;
+    } else {
+        sqlx::query!(
+            "UPDATE workspace_settings SET large_file_storage = NULL WHERE workspace_id = $1",
+            &w_id,
+        )
+        .execute(&mut *tx)
+        .await?;
+    }
+    tx.commit().await?;
+
+    Ok(format!("Edit copilot config for workspace {}", &w_id))
+}
 
 async fn edit_error_handler(
     authed: ApiAuthed,
@@ -812,14 +988,6 @@ async fn edit_error_handler(
 ) -> Result<String> {
     require_admin(is_admin, &username)?;
 
-    #[cfg(not(feature = "enterprise"))]
-    if ee.error_handler.as_ref().is_some_and(|val|  val == "script/hub/2431/slack/schedule-error-handler-slack")
-    {
-        return Err(Error::BadRequest(
-            "Slack error handler is only available in enterprise version".to_string(),
-        ));
-    }
-
     let mut tx = db.begin().await?;
     
     sqlx::query_as!(
@@ -924,17 +1092,26 @@ async fn check_name_conflict<'c>(tx: &mut Transaction<'c, Postgres>, w_id: &str)
 
 lazy_static::lazy_static! {
 
-    pub static ref CREATE_WORKSPACE_REQUIRE_SUPERADMIN: bool = std::env::var("CREATE_WORKSPACE_REQUIRE_SUPERADMIN").is_ok_and(|x| x.parse::<bool>().unwrap_or(true));
+    pub static ref CREATE_WORKSPACE_REQUIRE_SUPERADMIN: bool = {
+        match std::env::var("CREATE_WORKSPACE_REQUIRE_SUPERADMIN") {
+            Ok(val) => val == "true",
+            Err(_) => true,
+        }
+    };
 
 }
 
+async fn create_workspace_require_superadmin() -> String {
+    format!("{}", *CREATE_WORKSPACE_REQUIRE_SUPERADMIN)
+}
+
 async fn _check_nb_of_workspaces(db: &DB) -> Result<()> {
     let nb_workspaces = sqlx::query_scalar!("SELECT COUNT(*) FROM workspace WHERE id != 'admins' AND deleted = false",)
         .fetch_one(db)
         .await?;
-    if nb_workspaces.unwrap_or(0) >= 3 {
+    if nb_workspaces.unwrap_or(0) >= 2 {
         return Err(Error::BadRequest(
-            "You have reached the maximum number of workspaces (3 outside of default worskapce 'admins') without an enterprise license. Archive/delete another workspace to create a new one"
+            "You have reached the maximum number of workspaces (2 outside of default worskapce 'admins') without an enterprise license. Archive/delete another workspace to create a new one"
                 .to_string(),
         ));
     }
@@ -1036,6 +1213,13 @@ async fn create_workspace(
     .execute(&mut *tx)
     .await?;
 
+    sqlx::query!(
+        "INSERT INTO folder (workspace_id, name, display_name, owners, extra_perms) VALUES ($1, 'app_custom', 'App Custom Components', ARRAY[]::TEXT[], '{\"g/all\": false}') ON CONFLICT DO NOTHING",
+        nw.id,
+    )
+    .execute(&mut *tx)
+    .await?;
+
     sqlx::query!(
         "INSERT INTO folder (workspace_id, name, display_name, owners, extra_perms) VALUES ($1, 'app_groups', 'App Groups', ARRAY[]::TEXT[], '{\"g/all\": false}') ON CONFLICT DO NOTHING",
         nw.id,
@@ -1123,6 +1307,31 @@ async fn archive_workspace(
     Ok(format!("Archived workspace {}", &w_id))
 }
 
+async fn leave_workspace(
+    Extension(db): Extension<DB>,
+    Path(w_id): Path<String>,
+    ApiAuthed { email, username, .. }: ApiAuthed,
+) -> Result<String> {
+    let mut tx = db.begin().await?;
+    sqlx::query!("DELETE FROM usr WHERE workspace_id = $1 AND email = $2", &w_id, &email)
+        .execute(&mut *tx)
+        .await?;
+
+    audit_log(
+        &mut *tx,
+        &username,
+        "workspaces.leave",
+        ActionKind::Delete,
+        &w_id,
+        Some(&email),
+        None,
+    )
+    .await?;
+    tx.commit().await?;
+
+    Ok(format!("Left workspace {}", &w_id))
+}
+
 async fn unarchive_workspace(
     Extension(db): Extension<DB>,
     Path(w_id): Path<String>,
@@ -1166,6 +1375,15 @@ async fn delete_workspace(
     let mut tx = db.begin().await?;
     require_super_admin(&db, &email).await?;
 
+    sqlx::query!("DELETE FROM queue WHERE workspace_id = $1", &w_id)
+        .execute(&mut *tx)
+        .await?;
+    sqlx::query!("DELETE FROM capture WHERE workspace_id = $1", &w_id)
+        .execute(&mut *tx)
+        .await?;
+    sqlx::query!("DELETE FROM draft WHERE workspace_id = $1", &w_id)
+        .execute(&mut *tx)
+        .await?;
     sqlx::query!("DELETE FROM script WHERE workspace_id = $1", &w_id)
         .execute(&mut *tx)
         .await?;
@@ -1251,7 +1469,7 @@ pub async fn invite_user_to_all_auto_invite_worspaces(db: &DB, email: &str) -> R
     let mut tx = db.begin().await?;
     let domain = email.split('@').last().unwrap();
     let workspaces = sqlx::query!(
-        "SELECT workspace_id, auto_invite_operator FROM workspace_settings WHERE auto_invite_domain = $1",
+        "SELECT workspace_id, auto_invite_operator FROM workspace_settings WHERE auto_invite_domain = $1 OR auto_invite_domain = '*'",
         domain
     )
     .fetch_all(&mut *tx)
@@ -1289,7 +1507,8 @@ async fn invite_user(
     sqlx::query!(
         "INSERT INTO workspace_invite
             (workspace_id, email, is_admin, operator)
-            VALUES ($1, $2, $3, $4)",
+            VALUES ($1, $2, $3, $4) ON CONFLICT (workspace_id, email)
+            DO UPDATE SET is_admin = $3, operator = $4",
         &w_id,
         nu.email,
         nu.is_admin,
@@ -1634,6 +1853,7 @@ async fn tarball_workspace(
                 ScriptLang::Mysql => "my.sql",
                 ScriptLang::Bigquery => "bq.sql",
                 ScriptLang::Snowflake => "sf.sql",
+                ScriptLang::Mssql => "ms.sql",
                 ScriptLang::Graphql => "gql",
                 ScriptLang::Nativets => "fetch.ts",
                 ScriptLang::Bun => "bun.ts",

backend/windmill-common/src/flows.rs

@@ -35,7 +35,11 @@ pub struct Flow {
     pub extra_perms: serde_json::Value,
     #[serde(skip_serializing_if = "Option::is_none")]
     pub draft_only: Option<bool>,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub dedicated_worker: Option<bool>,
+    #[serde(skip_serializing_if = "Option::is_none")]
     pub tag: Option<String>,
+    #[serde(skip_serializing_if = "Option::is_none")]
     pub ws_error_handler_muted: Option<bool>,
 }
 
@@ -54,6 +58,7 @@ pub struct ListableFlow {
     pub has_draft: bool,
     #[serde(skip_serializing_if = "Option::is_none")]
     pub draft_only: Option<bool>,
+    #[serde(skip_serializing_if = "Option::is_none")]
     pub ws_error_handler_muted: Option<bool>,
 }
 
@@ -68,6 +73,7 @@ pub struct NewFlow {
     pub draft_only: Option<bool>,
     pub tag: Option<String>,
     pub ws_error_handler_muted: Option<bool>,
+    pub dedicated_worker: Option<bool>,
 }
 
 #[derive(Deserialize, Serialize, Debug, Clone, Default)]
@@ -89,7 +95,7 @@ pub struct FlowValue {
     #[serde(skip_serializing_if = "Option::is_none")]
     pub cache_ttl: Option<u32>,
     #[serde(skip_serializing_if = "Option::is_none")]
-    pub ws_error_handler_muted: Option<bool>,
+    pub early_return: Option<String>,
     #[serde(skip_serializing_if = "Option::is_none")]
     // Priority at the flow level
     pub priority: Option<i16>,
@@ -341,3 +347,24 @@ pub struct ListFlowQuery {
     pub order_desc: Option<bool>,
     pub starred_only: Option<bool>,
 }
+
+pub fn add_virtual_items_if_necessary(modules: &mut Vec<FlowModule>) {
+    if modules.len() > 0
+        && (modules[modules.len() - 1].sleep.is_some()
+            || modules[modules.len() - 1].suspend.is_some())
+    {
+        modules.push(FlowModule {
+            id: format!("{}-v", modules[modules.len() - 1].id),
+            value: FlowModuleValue::Identity,
+            stop_after_if: None,
+            summary: Some("Virtual module needed for suspend/sleep when last module".to_string()),
+            mock: None,
+            retry: None,
+            sleep: None,
+            suspend: None,
+            cache_ttl: None,
+            timeout: None,
+            priority: None,
+        });
+    }
+}

backend/windmill-common/src/global_settings.rs

@@ -12,6 +12,7 @@ pub const DISABLE_STATS_SETTING: &str = "disable_stats";
 pub const EXPOSE_METRICS_SETTING: &str = "expose_metrics";
 pub const EXPOSE_DEBUG_METRICS_SETTING: &str = "expose_debug_metrics";
 pub const KEEP_JOB_DIR_SETTING: &str = "keep_job_dir";
+pub const REQUIRE_PREEXISTING_USER_FOR_OAUTH_SETTING: &str = "require_preexisting_user_for_oauth";
 
 pub const ENV_SETTINGS: [&str; 54] = [
     "DISABLE_NSJAIL",

backend/windmill-common/src/jobs.rs

@@ -283,15 +283,20 @@ pub enum JobPayload {
         hash: ScriptHash,
         dependencies: String,
         language: ScriptLang,
+        dedicated_worker: Option<bool>,
     },
     FlowDependencies {
         path: String,
+        dedicated_worker: Option<bool>,
     },
     AppDependencies {
         path: String,
         version: i64,
     },
-    Flow(String),
+    Flow {
+        path: String,
+        dedicated_worker: Option<bool>,
+    },
     RestartedFlow {
         completed_job_id: Uuid,
         step_id: String,
@@ -401,10 +406,18 @@ pub async fn get_payload_tag_from_prefixed_path(
     let (payload, tag) = if path.starts_with("script/") {
         script_path_to_payload(path.strip_prefix("script/").unwrap(), &db, w_id).await?
     } else if path.starts_with("flow/") {
-        (
-            JobPayload::Flow(path.strip_prefix("flow/").unwrap().to_string()),
-            None,
+        let path = path.strip_prefix("flow/").unwrap().to_string();
+        let r = sqlx::query!(
+            "SELECT tag, dedicated_worker from flow WHERE path = $1 and workspace_id = $2",
+            &path,
+            &w_id,
         )
+        .fetch_optional(db)
+        .await?;
+        let (tag, dedicated_worker) = r
+            .map(|x| (x.tag, x.dedicated_worker))
+            .unwrap_or_else(|| (None, None));
+        (JobPayload::Flow { path, dedicated_worker }, tag)
     } else {
         return Err(Error::BadRequest(format!(
             "path must start with script/ or flow/ (got {})",

backend/windmill-common/src/oauth2.rs

@@ -6,9 +6,20 @@
  * LICENSE-AGPL for a copy of the license.
  */
 
+use std::sync::atomic::AtomicBool;
+
 use hmac::Hmac;
 use sha2::Sha256;
 
 pub type HmacSha256 = Hmac<Sha256>;
 
 pub const WORKSPACE_SLACK_BOT_TOKEN_PATH: &str = "f/slack_bot/bot_token";
+
+lazy_static::lazy_static! {
+
+    pub static ref REQUIRE_PREEXISTING_USER_FOR_OAUTH: AtomicBool = AtomicBool::new(std::env::var("REQUIRE_PREEXISTING_USER_FOR_OAUTH")
+        .ok()
+        .and_then(|x| x.parse::<bool>().ok())
+        .unwrap_or(false));
+
+}

backend/windmill-common/src/scripts.rs

@@ -42,6 +42,7 @@ pub enum ScriptLang {
     Bigquery,
     Snowflake,
     Graphql,
+    Mssql,
 }
 
 impl ScriptLang {
@@ -58,6 +59,7 @@ impl ScriptLang {
             ScriptLang::Mysql => "mysql",
             ScriptLang::Bigquery => "bigquery",
             ScriptLang::Snowflake => "snowflake",
+            ScriptLang::Mssql => "mssql",
             ScriptLang::Graphql => "graphql",
         }
     }
@@ -166,6 +168,7 @@ pub struct Script {
     pub concurrent_limit: Option<i32>,
     #[serde(skip_serializing_if = "Option::is_none")]
     pub concurrency_time_window_s: Option<i32>,
+    #[serde(skip_serializing_if = "Option::is_none")]
     pub dedicated_worker: Option<bool>,
     #[serde(skip_serializing_if = "Option::is_none")]
     pub ws_error_handler_muted: Option<bool>,

backend/windmill-common/src/stats.rs

@@ -4,7 +4,7 @@ use crate::{
     error::{to_anyhow, Result},
     global_settings::DISABLE_STATS_SETTING,
     scripts::ScriptLang,
-    utils::{get_uid, GIT_VERSION},
+    utils::{get_uid, Mode, GIT_VERSION},
     DB,
 };
 use chrono::Utc;
@@ -34,15 +34,22 @@ pub async fn get_disable_stats_setting(db: &DB) -> bool {
     false
 }
 
-pub async fn schedule_stats(db: &DB, instance_name: String, http_client: &reqwest::Client) -> () {
+pub async fn schedule_stats(
+    instance_name: String,
+    mode: Mode,
+    db: &DB,
+    http_client: &reqwest::Client,
+) -> () {
     let http_client = http_client.clone();
     let db = db.clone();
     tokio::spawn(async move {
+        tokio::time::sleep(tokio::time::Duration::from_secs(180)).await;
+
         loop {
             let disabled = get_disable_stats_setting(&db).await;
             if !disabled {
                 tracing::info!("Sending stats");
-                let result = send_stats(&instance_name, &http_client, &db).await;
+                let result = send_stats(&instance_name, &mode, &http_client, &db).await;
                 if result.is_err() {
                     tracing::info!("Error sending stats: {}", result.err().unwrap());
                 } else {
@@ -83,6 +90,7 @@ struct JobsUsage {
 
 pub async fn send_stats(
     instance_name: &String,
+    mode: &Mode,
     http_client: &reqwest::Client,
     db: &DB,
 ) -> Result<()> {
@@ -108,13 +116,32 @@ pub async fn send_stats(
             .collect::<Vec<serde_json::Value>>();
 
     let workers_usage = sqlx::query!(
-        "SELECT COUNT(*) FROM worker_ping WHERE ping_at > NOW() - INTERVAL '5 minutes'"
+        "SELECT COUNT(*) FROM worker_ping WHERE ping_at > NOW() - INTERVAL '2 minutes'"
     )
     .fetch_one(db)
     .await?
     .count
     .unwrap_or(0);
 
+    let users_count = sqlx::query!("SELECT author.count as author_count, operator.count as operator_count FROM (SELECT count(*)::INT FROM usr where usr.operator IS false) as author, (SELECT count(*)::INT FROM usr where usr.operator IS true) as operator")
+        .fetch_one(db)
+        .await?;
+
+    let vcpus = std::process::Command::new("cat")
+        .args(["/sys/fs/cgroup/cpu.max"])
+        .output()
+        .ok()
+        .map(|o| {
+            String::from_utf8_lossy(&o.stdout)
+                .to_string()
+                .split(" ")
+                .map(|s| s.to_string())
+                .collect::<Vec<String>>()
+                .get(0)
+                .map(|s| s.to_string())
+        })
+        .flatten();
+
     let payload = serde_json::json!({
         "uid": uid,
         "version": GIT_VERSION,
@@ -122,6 +149,12 @@ pub async fn send_stats(
         "jobs_usage": jobs_usage,
         "login_type_usage": login_type_usage,
         "workers_usage": workers_usage,
+        "users_usage": {
+            "author_count": users_count.author_count.unwrap_or(0),
+            "operator_count": users_count.operator_count.unwrap_or(0),
+        },
+        "mode": mode,
+        "vcpus": vcpus,
     });
 
     let request = http_client

backend/windmill-common/src/utils.rs

@@ -162,3 +162,12 @@ pub async fn get_uid(db: &DB) -> Result<String> {
 
     Ok(uid)
 }
+
+#[derive(Serialize, Deserialize, PartialEq, Clone)]
+#[serde(rename_all = "lowercase")]
+pub enum Mode {
+    Worker,
+    Agent,
+    Server,
+    Standalone,
+}

backend/windmill-common/src/variables.rs

@@ -75,25 +75,6 @@ pub async fn get_reserved_variables(
     step_id: Option<String>,
 ) -> [ContextualVariable; 15] {
     let state_path = {
-        let flow_path = flow_path
-            .clone()
-            .unwrap_or_else(|| "NO_FLOW_PATH".to_string());
-        let script_path = path.clone().unwrap_or_else(|| "NO_JOB_PATH".to_string());
-        let schedule_path = schedule_path
-            .clone()
-            .map(|x| format!("/{x}"))
-            .unwrap_or_else(String::new);
-
-        let script_path = if script_path.ends_with("/") {
-            "NO_NAME".to_string()
-        } else {
-            script_path
-        };
-
-        format!("{permissioned_as}/{flow_path}/{script_path}{schedule_path}")
-    };
-
-    let state_path_2 = {
         let trigger = if schedule_path.is_some() {
             username.to_string()
         } else {
@@ -200,13 +181,13 @@ pub async fn get_reserved_variables(
         },
         ContextualVariable {
             name: "WM_STATE_PATH".to_string(),
-            value: state_path,
-            description: "State resource path unique to a script and its trigger (legacy, in a migration period against WM_STATE_PATH_NEW)".to_string(),
+            value: state_path.clone(),
+            description: "State resource path unique to a script and its trigger".to_string(),
         },
         ContextualVariable {
             name: "WM_STATE_PATH_NEW".to_string(),
-            value: state_path_2,
-            description: "State resource path unique to a script and its trigger".to_string(),
+            value: state_path,
+            description: "State resource path unique to a script and its trigger (legacy)".to_string(),
         },
         ContextualVariable {
             name: "WM_FLOW_STEP_ID".to_string(),

backend/windmill-common/src/worker.rs

@@ -1,13 +1,12 @@
+use itertools::Itertools;
+use regex::Regex;
+use serde::{Deserialize, Serialize};
+use serde_json::value::RawValue;
 use std::{
     cmp::Reverse,
     collections::{HashMap, HashSet},
     sync::Arc,
 };
-
-use itertools::Itertools;
-use regex::Regex;
-use serde::{Deserialize, Serialize};
-use serde_json::value::RawValue;
 use tokio::sync::RwLock;
 
 use crate::{error, global_settings::CUSTOM_TAGS_SETTING, server::ServerConfig, DB};
@@ -28,6 +27,7 @@ lazy_static::lazy_static! {
         "postgresql".to_string(),
         "bigquery".to_string(),
         "snowflake".to_string(),
+        "mssql".to_string(),
         "graphql".to_string(),
         "dependency".to_string(),
         "flow".to_string(),
@@ -149,7 +149,10 @@ pub async fn update_ping(worker_instance: &str, worker_name: &str, ip: &str, db:
     .expect("insert worker_ping initial value");
 }
 
-pub async fn load_worker_config(db: &DB) -> error::Result<WorkerConfig> {
+pub async fn load_worker_config(
+    db: &DB,
+    killpill_tx: tokio::sync::broadcast::Sender<()>,
+) -> error::Result<WorkerConfig> {
     tracing::info!("Loading config from WORKER_GROUP: {}", *WORKER_GROUP);
     let mut config: WorkerConfigOpt = sqlx::query_scalar!(
         "SELECT config FROM config WHERE name = $1",
@@ -177,16 +180,25 @@ pub async fn load_worker_config(db: &DB) -> error::Result<WorkerConfig> {
             config.dedicated_worker.as_ref().unwrap()
         );
     }
-    let dedicated_worker = config.dedicated_worker.map(|x| {
-        let splitted = x.split(':').to_owned().collect_vec();
-        if splitted.len() != 2 {
-            panic!("DEDICATED_WORKER setting should be in the form of <workspace>:<script_path>")
-        } else {
-            let workspace = splitted[0];
-            let script_path = splitted[1];
-            WorkspacedPath { workspace_id: workspace.to_string(), path: script_path.to_string() }
-        }
-    });
+    let dedicated_worker = config
+        .dedicated_worker
+        .map(|x| {
+            let splitted = x.split(':').to_owned().collect_vec();
+            if splitted.len() != 2 {
+                killpill_tx.send(()).expect("send");
+                return Err(anyhow::anyhow!(
+                    "Invalid dedicated_worker format. Got {x}, expects <workspace_id>:<path>"
+                ));
+            } else {
+                let workspace = splitted[0];
+                let script_path = splitted[1];
+                Ok(WorkspacedPath {
+                    workspace_id: workspace.to_string(),
+                    path: script_path.to_string(),
+                })
+            }
+        })
+        .transpose()?;
     if *WORKER_GROUP == "default" && dedicated_worker.is_none() {
         let mut all_tags = config
             .worker_tags