Compare commits

..

6 Commits

Author SHA1 Message Date
Ádám Kovács
b76426ba83 faet(frontend): Add tour manager store 2023-03-10 20:11:25 +01:00
Ádám Kovács
1cb2a177c6 Merge branch 'main' into tutorials 2023-03-10 19:13:04 +01:00
Ádám Kovács
db87577a1b feat(frontend): Add app editor tour 2023-03-10 19:11:34 +01:00
Ádám Kovács
c219eb0ee9 feat(frontend): Add flow editor tour 2023-03-10 18:36:53 +01:00
Ádám Kovács
bfe5b56c99 feat(frontend): Add script editor tour 2023-03-10 09:38:34 +01:00
Ádám Kovács
4453690521 feat(frontend): Add onboarding tour to home page 2023-03-09 20:11:27 +01:00
344 changed files with 8370 additions and 13977 deletions

View File

@@ -4,7 +4,7 @@ VERSION=$1
echo "Updating versions to: $VERSION"
sed -i -e "/^version =/s/= .*/= \"$VERSION\"/" backend/Cargo.toml
sed -i -e "/^export const VERSION =/s/= .*/= \"v$VERSION\";/" cli/main.ts
sed -i -e "/^const VERSION =/s/= .*/= \"v$VERSION\";/" cli/main.ts
sed -i -e "/version: /s/: .*/: $VERSION/" backend/windmill-api/openapi.yaml
sed -i -e "/version: /s/: .*/: $VERSION/" openflow.openapi.yaml
sed -i -e "/\"version\": /s/: .*,/: \"$VERSION\",/" frontend/package.json

View File

@@ -1,177 +1,6 @@
# Changelog
## [1.83.1](https://github.com/windmill-labs/windmill/compare/v1.83.0...v1.83.1) (2023-03-28)
### Bug Fixes
* **cli:** plain secrets might be undefined ([569a55e](https://github.com/windmill-labs/windmill/commit/569a55e45b34641b0fb4569387166f3aa89ce35f))
## [1.83.0](https://github.com/windmill-labs/windmill/compare/v1.82.0...v1.83.0) (2023-03-28)
### Features
* **backend:** allow relative imports for python ([a5500ea](https://github.com/windmill-labs/windmill/commit/a5500ea40a77b2e0408e2a644190a8f65b18cd1d))
* **backend:** execute /bin/bash instead of /bin/sh for bash scripts ([021fa23](https://github.com/windmill-labs/windmill/commit/021fa23f9ffcd11548977a4589eb9bc2815243cf))
* **backend:** improve relative importsfor deno ([eaac598](https://github.com/windmill-labs/windmill/commit/eaac598af308cedea8f0f8fc7c189a4640b4366b))
* **backend:** increase timeout for premium workspace ([00b70d9](https://github.com/windmill-labs/windmill/commit/00b70d9aaac8ae979782492d7754060a3c2c9567))
* **frontend:** add pagination ([33c07d3](https://github.com/windmill-labs/windmill/commit/33c07d3e63f96673719ecb15e45f4cd9e18be80e))
* **frontend:** Add quick style settings to app editor ([#1308](https://github.com/windmill-labs/windmill/issues/1308)) ([ac24862](https://github.com/windmill-labs/windmill/commit/ac2486219cd91df3a7fe11d37894797a881cac6c))
* **frontend:** add recompute as a primitive ([449d3ae](https://github.com/windmill-labs/windmill/commit/449d3ae5ddeceef3fbcb7a815a4dba16c9639fb3))
* **frontend:** add textareacomponent + fix multiselect style + select multi components ([2b31653](https://github.com/windmill-labs/windmill/commit/2b31653a8aa06807678e8609cfa62cf0f2f55dce))
* **frontend:** multiselect components for apps ([577dec5](https://github.com/windmill-labs/windmill/commit/577dec5c5733cdf88e8586ce6c27159920c69c8a))
* **frontend:** use rich json editor for arrays of objects and for object in ArgInput ([b95afaa](https://github.com/windmill-labs/windmill/commit/b95afaa9bb41b102181657453a564f44f4511983))
### Bug Fixes
* **apps:** improve app table actionButtons behavior under many clicks ([8e3d8ac](https://github.com/windmill-labs/windmill/commit/8e3d8acc80de971ee115d6903d24864d8263f08b))
* **cli:** add --plain-secrets ([98d51e2](https://github.com/windmill-labs/windmill/commit/98d51e219df1680507114f9b57ec0b0a4a234b5c))
* **frontend:** add a modal that is always mounted to make sure compon… ([#1328](https://github.com/windmill-labs/windmill/issues/1328)) ([a527cb8](https://github.com/windmill-labs/windmill/commit/a527cb8222a2ff80dae38ebae7dc5ea0979d74c5))
* **frontend:** Disable app keyboard navigation on focused inputs ([#1326](https://github.com/windmill-labs/windmill/issues/1326)) ([da24e9a](https://github.com/windmill-labs/windmill/commit/da24e9ab0625a7503c498c179022ea4011a03170))
* **frontend:** persist description for schemas ([1a48673](https://github.com/windmill-labs/windmill/commit/1a4867302f72aaae8f422ac8f53812c116cc383d))
* **frontend:** Revert app upload input ([#1330](https://github.com/windmill-labs/windmill/issues/1330)) ([fa457bb](https://github.com/windmill-labs/windmill/commit/fa457bb7099bd31c2315eaf7f7f2c40900b2ae39))
* **frontend:** Small app fixes ([#1331](https://github.com/windmill-labs/windmill/issues/1331)) ([75306c8](https://github.com/windmill-labs/windmill/commit/75306c831616d9a01cc3a4681732aab93153f1a9))
## [1.82.0](https://github.com/windmill-labs/windmill/compare/v1.81.0...v1.82.0) (2023-03-24)
### Features
* **backend:** introduce RESTART_ZOMBIE_JOBS and ZOMBIE_JOB_TIMEOUT ([47a7f71](https://github.com/windmill-labs/windmill/commit/47a7f7163aae3fe807e766c824085b4d1b75c8c8))
### Bug Fixes
* **backend:** do not consider FlowPreview as potential zombie job ([f7c30b5](https://github.com/windmill-labs/windmill/commit/f7c30b5d2f16e15f36208e07126557fd7ed84801))
* **backend:** increase dynamic js timeout + improve client passing ([34e25f0](https://github.com/windmill-labs/windmill/commit/34e25f0f96fe637cc42f4017a064c40def5d67ef))
* **cli:** improve diff speed + fix replacing cli ([b999c98](https://github.com/windmill-labs/windmill/commit/b999c9894b4011b735f37df485fe403c22c00512))
* **frontend:** Fix AppTable error display + clear errors when removing a component + properly detect that latest component run had an error ([#1322](https://github.com/windmill-labs/windmill/issues/1322)) ([c15bc8a](https://github.com/windmill-labs/windmill/commit/c15bc8a7bfb3bef2634e6093088967137cd06239))
* **frontend:** fix refresh with manual dependencies ([#1319](https://github.com/windmill-labs/windmill/issues/1319)) ([a47031a](https://github.com/windmill-labs/windmill/commit/a47031a41e6a3392101e280dcd1aea098f898447))
* **frontend:** fix settings panel ([#1323](https://github.com/windmill-labs/windmill/issues/1323)) ([30b8e47](https://github.com/windmill-labs/windmill/commit/30b8e474df5b71b7e7b36d3fe5974a289cf0dfae))
* **frontend:** Fix transformer ([#1321](https://github.com/windmill-labs/windmill/issues/1321)) ([addabcc](https://github.com/windmill-labs/windmill/commit/addabcceb0c90782ba4a934bb3822f8cc9865069))
* **frontend:** remove unnecessary div ([#1318](https://github.com/windmill-labs/windmill/issues/1318)) ([e193a0b](https://github.com/windmill-labs/windmill/commit/e193a0bcdf6690b007594d2f1325a7ec26603129))
## [1.81.0](https://github.com/windmill-labs/windmill/compare/v1.80.1...v1.81.0) (2023-03-21)
### Features
* **apps:** add action on form/button/formbutton ([2593218](https://github.com/windmill-labs/windmill/commit/2593218cbf07c05521a270797055ddb22dc22b8d))
### Bug Fixes
* **frontend:** Remove action outline on preview mode ([#1313](https://github.com/windmill-labs/windmill/issues/1313)) ([a7c4f1a](https://github.com/windmill-labs/windmill/commit/a7c4f1a12e02e8627a5955b75d572e9cf11d8122))
## [1.80.1](https://github.com/windmill-labs/windmill/compare/v1.80.0...v1.80.1) (2023-03-21)
### Bug Fixes
* **cli:** add support for non metadataed scripts ([42f6d2e](https://github.com/windmill-labs/windmill/commit/42f6d2e0ee6294f8a1d97f5f62f2adb6edfd2fed))
## [1.80.0](https://github.com/windmill-labs/windmill/compare/v1.79.0...v1.80.0) (2023-03-20)
### Features
* **apps:** add transformers for data sources ([0abacac](https://github.com/windmill-labs/windmill/commit/0abacac06c7dd586b48c66ff47b7589fe692205b))
* **frontend:** App set tab ([#1307](https://github.com/windmill-labs/windmill/issues/1307)) ([48413a7](https://github.com/windmill-labs/windmill/commit/48413a78c5e7e0ee8208711f15135d81136b7386))
### Bug Fixes
* **frontend:** add missing optional chaining ([#1306](https://github.com/windmill-labs/windmill/issues/1306)) ([29b1cc6](https://github.com/windmill-labs/windmill/commit/29b1cc6ff0ebc5edcad24a1780113889c507075d))
* **frontend:** App button triggered by ([#1304](https://github.com/windmill-labs/windmill/issues/1304)) ([cf2d031](https://github.com/windmill-labs/windmill/commit/cf2d031e8e89faa2cd7fa58436cbe7cf4d9045f9))
## [1.79.0](https://github.com/windmill-labs/windmill/compare/v1.78.0...v1.79.0) (2023-03-17)
### Features
* **frontend:** add listeners for frontend scripts ([597e38e](https://github.com/windmill-labs/windmill/commit/597e38ef367d38fa97fc443ccb2c721e5964fece))
* **frontend:** add table actions navigation ([#1298](https://github.com/windmill-labs/windmill/issues/1298)) ([c3ba1a6](https://github.com/windmill-labs/windmill/commit/c3ba1a6ab97484a08a5a20187bb858a5af7025cb))
* **frontend:** App component triggers ([#1303](https://github.com/windmill-labs/windmill/issues/1303)) ([078cb1b](https://github.com/windmill-labs/windmill/commit/078cb1bf3e4de08cb018578f04d24392a6462f69))
* **frontend:** Component control ([#1293](https://github.com/windmill-labs/windmill/issues/1293)) ([bd927a2](https://github.com/windmill-labs/windmill/commit/bd927a27ed9581dbf67ea3694f9d989f8d71d2ed))
### Bug Fixes
* **frontend:** App panel styling ([#1284](https://github.com/windmill-labs/windmill/issues/1284)) ([c1dd35c](https://github.com/windmill-labs/windmill/commit/c1dd35c3f0fcbc1be43273f82a873c3c07863417))
* **frontend:** Display app context search on top ([#1300](https://github.com/windmill-labs/windmill/issues/1300)) ([bd3ee81](https://github.com/windmill-labs/windmill/commit/bd3ee81b14846f16ccd16461de99b46fe68be6ba))
* **frontend:** fix horizontal splitpanes ([#1301](https://github.com/windmill-labs/windmill/issues/1301)) ([ea3dab4](https://github.com/windmill-labs/windmill/commit/ea3dab411b3d5dd772e04c8831e789e2470aaf28))
* **frontend:** fix map render ([#1297](https://github.com/windmill-labs/windmill/issues/1297)) ([0092721](https://github.com/windmill-labs/windmill/commit/00927210fd68c31cb793ef4f0efea05711ebcf00))
* **frontend:** Hide archive toggle with empty list ([#1296](https://github.com/windmill-labs/windmill/issues/1296)) ([bac831b](https://github.com/windmill-labs/windmill/commit/bac831b23ce85a683ddbd4537900670a0b7d12a8))
## [1.78.0](https://github.com/windmill-labs/windmill/compare/v1.77.0...v1.78.0) (2023-03-16)
### Features
* **frontend:** app textcomponent editable + tooltip ([11567d6](https://github.com/windmill-labs/windmill/commit/11567d6280ea60f1a8c3c6607c724179775cbbe3))
### Bug Fixes
* **backend:** whitelist for include_header was ignored in some cases ([183a459](https://github.com/windmill-labs/windmill/commit/183a4591df700ab4720de6e92a83631256940089))
* **frontend:** improve rendering performance after component moving ([6f890f2](https://github.com/windmill-labs/windmill/commit/6f890f2120885f90d986fbd655096b45bf9de539))
* **frontend:** remove staticOutputs from apps ([dbdfd62](https://github.com/windmill-labs/windmill/commit/dbdfd626386398180ecba7976714f86365eeccd8))
## [1.77.0](https://github.com/windmill-labs/windmill/compare/v1.76.0...v1.77.0) (2023-03-14)
### Features
* **apps:** state can be used as input in apps ([2f0acb9](https://github.com/windmill-labs/windmill/commit/2f0acb9ffa8dace4a886527dcee49809d019b271))
* **apps:** tabs can be made pages or invisible + better frontend scripts reactivity ([cd645d0](https://github.com/windmill-labs/windmill/commit/cd645d0935f2d06e0ff71f14d2cf63accd378ff3))
* **deno:** add support for custom npm repo ([#1291](https://github.com/windmill-labs/windmill/issues/1291)) ([944795f](https://github.com/windmill-labs/windmill/commit/944795f6eeaa7d01ab1a35a80570a55c363723e6))
* **frontend:** add setTab to frontend scripts ([c2a97c5](https://github.com/windmill-labs/windmill/commit/c2a97c53cfff0fdb35dd8bc249490566eebdc1a9))
* **frontend:** app components output panel ([#1283](https://github.com/windmill-labs/windmill/issues/1283)) ([751edcf](https://github.com/windmill-labs/windmill/commit/751edcf9b8e0976a1d073603c9eff5dc6e714490))
### Bug Fixes
* **backend:** do not cache reference to workspace scripts ([eb73f2a](https://github.com/windmill-labs/windmill/commit/eb73f2a687f6faad301b9038ab8585450bec7481))
* **frontend:** fix app tabs ([#1288](https://github.com/windmill-labs/windmill/issues/1288)) ([c71a577](https://github.com/windmill-labs/windmill/commit/c71a577fead90c9cd01a736b54d859ec4f0b7807))
* **frontend:** fix container deletion ([#1287](https://github.com/windmill-labs/windmill/issues/1287)) ([bc870bd](https://github.com/windmill-labs/windmill/commit/bc870bd03eb76cb8bc0e0c861f6cd8a9c661186b))
* **frontend:** Update setting accordion ([#1285](https://github.com/windmill-labs/windmill/issues/1285)) ([dea12e8](https://github.com/windmill-labs/windmill/commit/dea12e8870ece998bb6607723cbaab9b9a958f22))
## [1.76.0](https://github.com/windmill-labs/windmill/compare/v1.75.0...v1.76.0) (2023-03-13)
### Features
* **frontend:** add frontend (JS) scripts to apps ([f0b1b1f](https://github.com/windmill-labs/windmill/commit/f0b1b1f752731ba434b960a75624118152f53c00))
* **frontend:** Copy, Cut and Paste ([#1279](https://github.com/windmill-labs/windmill/issues/1279)) ([82c139e](https://github.com/windmill-labs/windmill/commit/82c139ed0992be401e250cfb7ecc0fca61b76772))
* **frontend:** disabled for action buttons can now depend on row ([75f87e7](https://github.com/windmill-labs/windmill/commit/75f87e7e1117a9c12afcf626379e94b134a9a493))
* **frontend:** improve drag-n-drop behavior ([cfd489a](https://github.com/windmill-labs/windmill/commit/cfd489a55059e7b6843f99bab261c70b3852e6a2))
### Bug Fixes
* **backend:** improve worker ping api ([c958480](https://github.com/windmill-labs/windmill/commit/c958480ce83844a989f58dd5a70eb288582e2194))
* **frontend:** General fixes and updates ([#1281](https://github.com/windmill-labs/windmill/issues/1281)) ([3e5a179](https://github.com/windmill-labs/windmill/commit/3e5a179eb8cd8001f49c92305141dade1571e20f))
## [1.75.0](https://github.com/windmill-labs/windmill/compare/v1.74.2...v1.75.0) (2023-03-11)
### Features
* add filter jobs by args or result ([3b44f9a](https://github.com/windmill-labs/windmill/commit/3b44f9a72ca0466a44963a4b9657a0ee59b44753))
* **apps:** add resource picker ([8681e83](https://github.com/windmill-labs/windmill/commit/8681e83b574141acbf7e5a389a9e8a4f340336d1))
* **bash:** add default argument handling for bash ([1d5c194](https://github.com/windmill-labs/windmill/commit/1d5c194f09ffba963d52e418c5954843d84ae337))
* **frontend-apps:** add variable picker for static string input on apps ([bc440f8](https://github.com/windmill-labs/windmill/commit/bc440f8d4154ce464c0e027d93b7a0a3b76d782e))
* **frontend:** make runs filters synced with query args ([61a5e1f](https://github.com/windmill-labs/windmill/commit/61a5e1f1accc988628b785b3b9be04c4ea719874))
### Bug Fixes
* **backend:** add killpill for lines reading ([7c825c2](https://github.com/windmill-labs/windmill/commit/7c825c212dd0f1e8be427eabd9a9756303241d1b))
* **cli:** many small fixes ([ce32370](https://github.com/windmill-labs/windmill/commit/ce323709a94d27fb24214719180ea1aafc66d646))
## [1.74.2](https://github.com/windmill-labs/windmill/compare/v1.74.1...v1.74.2) (2023-03-09)

View File

@@ -11,5 +11,5 @@ http://{$BASE_URL} {
https://{$BASE_URL} {
bind {$ADDRESS}
reverse_proxy /ws/* http://localhost:3001
}
}
}

View File

@@ -26,8 +26,7 @@ Open-source developer infrastructure for internal tools. Self-hostable alternati
# Windmill - Turn scripts into workflows and UIs that you can share and run at scale
Windmill is <b>fully open-sourced (AGPLv3)</b> and Windmill Labs offers
dedicated instance and commercial support and licenses.
Windmill is <b>fully open-sourced (AGPLv3)</b> and Windmill Labs offers dedicated instance and commercial support and licenses.
![Windmill Diagram](/imgs/stacks.svg)
@@ -75,7 +74,7 @@ https://user-images.githubusercontent.com/275584/218350457-bc2fdc3b-e667-4da5-a2
4. Build complex UI on top of your scripts and flows.
![Step 5](./imgs/windmill-builder.png)
Scripts and flows can also be triggered by a cron schedule '_/5 _ \* \* \*' or
Scripts and flows can also be triggered by a cron schedule '*/5 * * * *' or
through webhooks.
You can build your entire infra on top of Windmill!
@@ -83,49 +82,46 @@ You can build your entire infra on top of Windmill!
## Show me some actual script code
```typescript
import * as wmill from "https://deno.land/x/windmill@v1.62.0/mod.ts";
import * as wmill from "https://deno.land/x/windmill@v1.62.0/mod.ts"
//import any dependency from npm
import cowsay from "npm:cowsay@1.5.0";
import cowsay from 'npm:cowsay@1.5.0'
export async function main(
a: number,
// unions generate enums
b: "my" | "enum",
// default parameters prefill the field
d = "default arg",
// nested objects work c = { nested: "object" },
// permissioned and typed json
db: wmill.Resource<"postgresql">
) {
const email = Deno.env.get("WM_EMAIL");
// variables are permissioned and by path
let variable = await wmill.getVariable("f/company-folder/my_secret");
const lastTimeRun = await wmill.getState();
// logs are printed and always inspectable
console.log(cowsay.say({ text: "hello " + email + " " + lastTimeRun }));
await wmill.setState(Date.now());
a: number,
// unions generate enums
b: "my" | "enum",
// default parameters prefill the field
d = "default arg",
// nested objects work c = { nested: "object" },
// permissioned and typed json
db: wmill.Resource<"postgresql">) {
// return is serialized as JSON
return { foo: d, variable };
const email = Deno.env.get('WM_EMAIL')
// variables are permissioned and by path
let variable = await wmill.getVariable('f/company-folder/my_secret')
const lastTimeRun = await wmill.getState()
// logs are printed and always inspectable
console.log(cowsay.say({ text: "hello " + email + " " + lastTimeRun }))
await wmill.setState(Date.now())
// return is serialized as JSON
return { foo: d, variable };
}
```
## CLI
We have a powerful CLI to interact with the windmill platform and sync your
scripts from local files, github repos and to run scripts and flows on the
instance from local commands. See
scripts from local files, github repos and to run scripts and flows on the instance from local commands. See
[more details](https://github.com/windmill-labs/windmill/tree/main/cli)
![CLI Screencast](./cli/vhs/output/setup.gif)
### Running scripts locally
You can run your script locally easily, you simply need to pass the right
environment variables for the `wmill` client library to fetch resource and
variables from your instance if necessary. See more:
<https://docs.windmill.dev/docs/advanced/local_development/>
You can run your script locally easily, you simply need to pass the right environment variables for the `wmill` client library to fetch resource and variables from your instance if necessary. See more: <https://docs.windmill.dev/docs/advanced/local_development/>
## Stack
@@ -191,6 +187,7 @@ compiling from source or using without a postgres super user, see
Go to http://localhost et voilà :)
The default super-admin user is: admin@windmill.dev / changeme
From there, you can create other users (do not forget to change the password!)
@@ -202,9 +199,9 @@ We publish helm charts at:
### Postgres without superuser
If you do not want, or cannot (for instance, in AWS Aurora or Cloud sql) use a
postgres superuser, you can run `./init-db-as-superuser.sql` to init the
required users for windmill.
If you do not want, or cannot (for instance, in AWS Aurora or Cloud sql) use a postgres superuser,
you can run `./init-db-as-superuser.sql` to init the required users for windmill.
### Commercial license
@@ -278,8 +275,8 @@ You may also add your own custom OAuth2 IdP and OAuth2 Resource provider:
### Resource types
You will also want to import all the approved resource types from
[WindmillHub](https://hub.windmill.dev). A setup script will prompt you to have
it being synced automatically everyday.
[WindmillHub](https://hub.windmill.dev). A setup script will prompt
you to have it being synced automatically everyday.
## Environment Variables
@@ -288,16 +285,14 @@ it being synced automatically everyday.
| DATABASE_URL | | The Postgres database url. | All |
| DISABLE_NSJAIL | true | Disable Nsjail Sandboxing | Worker |
| SERVER_BIND_ADDR | 0.0.0.0 | IP Address on which to bind listening socket | Server |
| PORT | 8000 | Exposed port | Server |
| PORT | 8000 | Exposed port | Server | |
| NUM_WORKERS | 3 | The number of worker per Worker instance (set to 1 on Eks to have 1 pod = 1 worker, set to 0 for an API only instance) | Worker |
| DISABLE_SERVER | false | Binary would operate as a worker only instance | Worker |
| METRICS_ADDR | None | The socket addr at which to expose Prometheus metrics at the /metrics path. Set to "true" to expose it on port 8001 | All |
| JSON_FMT | false | Output the logs in json format instead of logfmt | All |
| BASE_URL | http://localhost:8000 | The base url that is exposed publicly to access your instance | Server |
| BASE_INTERNAL_URL | http://localhost:8000 | The base url that is reachable by your workers to talk to the Servers. This help avoiding going through the external load balancer for VPC-internal requests. | Worker |
| TIMEOUT | 300 | The maximum time of execution of a script. When reached, the job is failed as having timedout. | Worker |
| ZOMBIE_JOB_TIMEOUT | 30 | The timeout after which a job is considered to be zombie if the worker did not send pings about processing the job (every server check for zombie jobs every 30s) | Server |
| RESTART_ZOMBIE_JOBS | true | If true then a zombie job is restarted (in-place with the same uuid and some logs), if false the zombie job is failed | Server |
| TIMEOUT | 300 | The timeout in seconds for the execution of a script | Worker |
| SLEEP_QUEUE | 50 | The number of ms to sleep in between the last check for new jobs in the DB. It is multiplied by NUM_WORKERS such that in average, for one worker instance, there is one pull every SLEEP_QUEUE ms. | Worker |
| MAX_LOG_SIZE | 500000 | The maximum number of characters a job can emit (log + result) | Worker |
| DISABLE_NUSER | false | If Nsjail is enabled, disable the nsjail's `clone_newuser` setting | Worker |
@@ -306,7 +301,7 @@ it being synced automatically everyday.
| S3_CACHE_BUCKET (EE only) | None | The S3 bucket to sync the cache of the workers to | Worker |
| TAR_CACHE_RATE (EE only) | 100 | The rate at which to tar the cache of the workers. 100 means every 100th job in average (uniformly randomly distributed). | Worker |
| SLACK_SIGNING_SECRET | None | The signing secret of your Slack app. See [Slack documentation](https://api.slack.com/authentication/verifying-requests-from-slack) | Server |
| COOKIE_DOMAIN | None | The domain of the cookie. If not set, the cookie will be set by the browser based on the full origin | Server |
| COOKIE_DOMAIN | None | The domain of the cookie. If not set, the cookie will be set by the browser based on the full origin | Server | |
| DENO_PATH | /usr/bin/deno | The path to the deno binary. | Worker |
| PYTHON_PATH | /usr/local/bin/python3 | The path to the python binary. | Worker |
| GO_PATH | /usr/bin/go | The path to the go binary. | Worker |
@@ -321,7 +316,6 @@ it being synced automatically everyday.
| QUEUE_LIMIT_WAIT_RESULT | None | The number of max jobs in the queue before rejecting immediately the request in 'run_wait_result' endpoint. Takes precedence on the query arg. If none is specified, there are no limit. | Worker |
| DENO_AUTH_TOKENS | None | Custom DENO_AUTH_TOKENS to pass to worker to allow the use of private modules | Worker |
| DENO_FLAGS | None | Override the flags passed to deno (default --allow-all) to tighten permissions. Minimum permissions needed are "--allow-read=args.json --allow-write=result.json" | Worker |
| NPM_CONFIG_REGISTRY | None | Registry to use for NPM dependencies, set if you have a private repository you need to use instead of the default public NPM registry | Worker |
| PIP_LOCAL_DEPENDENCIES | None | Specify dependencies that are installed locally and do not need to be solved nor installed again | |
| ADDITIONAL_PYTHON_PATHS | None | Specify python paths (separated by a :) to be appended to the PYTHONPATH of the python jobs. To be used with PIP_LOCAL_DEPENDENCIES to use python codebases within Windmill | Worker |
| INCLUDE_HEADERS | None | Whitelist of headers that are passed to jobs as args (separated by a comma) | Server |

722
backend/Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,6 @@
[package]
name = "windmill"
version = "1.83.1"
version = "1.74.2"
authors.workspace = true
edition.workspace = true
@@ -19,7 +19,7 @@ members = [
]
[workspace.package]
version = "1.83.1"
version = "1.74.2"
authors = ["Ruben Fiszel <ruben@windmill.dev>"]
edition = "2021"
@@ -122,7 +122,7 @@ itertools = "^0"
regex = "^1"
deno_core = "^0"
async-recursion = "^1"
swc_common = "0.29.39"
swc_common = "^0"
swc_ecma_parser = "0.128.2"
swc_ecma_ast = "0.98.1"
base64 = "0.21.0"
@@ -153,4 +153,3 @@ async-stripe = { version = "0.14", features = [
"checkout",
] }
async_zip = { version = "0.0.11", features = ["full"] }
once_cell = "1.17.1"

View File

@@ -1 +0,0 @@
-- Add down migration script here

File diff suppressed because it is too large Load Diff

View File

@@ -1 +0,0 @@
-- Add down migration script here

View File

@@ -1,11 +0,0 @@
-- Add up migration script here
CREATE POLICY see_extra_perms_user ON app FOR ALL
USING (extra_perms ? CONCAT('u/', current_setting('session.user')))
WITH CHECK ((extra_perms ->> CONCAT('u/', current_setting('session.user')))::boolean);
CREATE POLICY see_extra_perms_groups ON app FOR ALL
USING (extra_perms ?| regexp_split_to_array(current_setting('session.pgroups'), ',')::text[])
WITH CHECK (exists(
SELECT key, value FROM jsonb_each_text(extra_perms)
WHERE SPLIT_PART(key, '/', 1) = 'g' AND key = ANY(regexp_split_to_array(current_setting('session.pgroups'), ',')::text[])
AND value::boolean));

View File

@@ -58,9 +58,6 @@ fn parse_go_typ(typ: &parser_go_ast::Expr) -> (Option<String>, Typ) {
Some((*name).to_string()),
match *name {
"int" => Typ::Int,
"int16" => Typ::Int,
"int32" => Typ::Int,
"int64" => Typ::Int,
"string" => Typ::Str(None),
"bool" => Typ::Bool,
_ => Typ::Unknown,

View File

@@ -181,7 +181,6 @@ static PYTHON_IMPORTS_REPLACEMENT: phf::Map<&'static str, &'static str> = phf_ma
"git" => "GitPython",
"u" => "requests",
"f" => "requests",
"." => "requests",
"shopify" => "ShopifyAPI",
"seleniumwire" => "selenium-wire",
"openbb-terminal" => "openbb[all]",
@@ -225,23 +224,12 @@ pub fn parse_python_imports(code: &str) -> error::Result<Vec<String>> {
StmtKind::Import { names } => Some(
names
.into_iter()
.map(|x| {
let name = x.node.name;
if name.starts_with('.') {
".".to_string()
} else {
name.split('.').next().unwrap_or("").to_string()
}
})
.map(|x| x.node.name.split('.').next().unwrap_or("").to_string())
.map(replace_import)
.collect::<Vec<String>>(),
),
StmtKind::ImportFrom { level, module: Some(mod_), names: _ } => {
let imprt = if level.is_some() && level.unwrap() > 0 {
".".to_string()
} else {
mod_.split('.').next().unwrap_or("").replace("_", "-")
};
StmtKind::ImportFrom { level: _, module: Some(mod_), names: _ } => {
let imprt = mod_.split('.').next().unwrap_or("").replace("_", "-");
Some(vec![replace_import(imprt)])
}
@@ -454,7 +442,6 @@ import os
import wmill
from zanzibar.estonie import talin
import matplotlib.pyplot as plt
from . import tests
def main():
pass
@@ -462,7 +449,7 @@ def main():
";
let r = parse_python_imports(code)?;
// println!("{}", serde_json::to_string(&r)?);
assert_eq!(r, vec!["wmill", "zanzibar", "matplotlib", "requests"]);
assert_eq!(r, vec!["wmill", "zanzibar", "matplotlib"]);
Ok(())
}

View File

@@ -246,7 +246,7 @@ fn tstype_to_typ(ts_type: &TsType) -> (Typ, bool) {
pub fn eval_sync(code: &str) -> Result<serde_json::Value, String> {
let mut context = JsRuntime::new(RuntimeOptions::default());
let code = format!("let x = {}; x", code);
let res = context.execute_script("<anon>", code);
let res = context.execute_script("<anon>", &code);
match res {
Ok(global) => {
let scope = &mut context.handle_scope();

View File

@@ -2501,57 +2501,6 @@
},
"query": "SELECT set_config('session.user', $1, true)"
},
"6d50a8dc9cfc040b6f37b58053daa0709671e008a10ab4189114ceff66efe603": {
"describe": {
"columns": [
{
"name": "worker",
"ordinal": 0,
"type_info": "Varchar"
},
{
"name": "worker_instance",
"ordinal": 1,
"type_info": "Varchar"
},
{
"name": "last_ping",
"ordinal": 2,
"type_info": "Int4"
},
{
"name": "started_at",
"ordinal": 3,
"type_info": "Timestamptz"
},
{
"name": "ip",
"ordinal": 4,
"type_info": "Varchar"
},
{
"name": "jobs_executed",
"ordinal": 5,
"type_info": "Int4"
}
],
"nullable": [
false,
false,
null,
false,
false,
false
],
"parameters": {
"Left": [
"Int8",
"Int8"
]
}
},
"query": "SELECT worker, worker_instance, EXTRACT(EPOCH FROM (now() - ping_at))::integer as last_ping, started_at, ip, jobs_executed FROM worker_ping ORDER BY ping_at desc LIMIT $1 OFFSET $2"
},
"6dfd1ac7168f098a9e7c92d530a470c0a156041d5081053d43d9efb00d81c464": {
"describe": {
"columns": [
@@ -2573,6 +2522,57 @@
},
"query": "SELECT policy from app WHERE path = $1 AND workspace_id = $2"
},
"6fc2cfae9df83eb24ea33e4c9567740100f4dd2285afc3ef474fc70041b0567b": {
"describe": {
"columns": [
{
"name": "worker",
"ordinal": 0,
"type_info": "Varchar"
},
{
"name": "worker_instance",
"ordinal": 1,
"type_info": "Varchar"
},
{
"name": "ping_at",
"ordinal": 2,
"type_info": "Timestamptz"
},
{
"name": "started_at",
"ordinal": 3,
"type_info": "Timestamptz"
},
{
"name": "ip",
"ordinal": 4,
"type_info": "Varchar"
},
{
"name": "jobs_executed",
"ordinal": 5,
"type_info": "Int4"
}
],
"nullable": [
false,
false,
false,
false,
false,
false
],
"parameters": {
"Left": [
"Int8",
"Int8"
]
}
},
"query": "SELECT * FROM worker_ping ORDER BY ping_at desc LIMIT $1 OFFSET $2"
},
"701f215eb14ba67a79afea15d7effc0dd394ba6c4a72c95d4560c5a377015d4e": {
"describe": {
"columns": [],
@@ -2984,6 +2984,20 @@
},
"query": "UPDATE group_ SET summary = $1 WHERE name = $2 AND workspace_id = $3"
},
"83fc9bf32b9b40987feff61fea82e20613e74efb7bf8ba52485566667ccf2284": {
"describe": {
"columns": [],
"nullable": [],
"parameters": {
"Left": [
"Varchar",
"Text",
"Text"
]
}
},
"query": "UPDATE resource SET path = $1 WHERE path = $2 AND workspace_id = $3"
},
"8543f029d9784234e4c6a6dcd7b03e62d544b98be261334ee210594e0bb839f2": {
"describe": {
"columns": [
@@ -3154,21 +3168,6 @@
},
"query": "INSERT INTO usage (id, is_workspace, month_, usage) \n VALUES ($1, false, EXTRACT(YEAR FROM current_date) * 12 + EXTRACT(MONTH FROM current_date), 0) \n ON CONFLICT (id, is_workspace, month_) DO UPDATE SET usage = usage.usage + 1 \n RETURNING usage.usage"
},
"8c0131a9cc61f2daa258d49767242bcaab6bb34a977ff7fb0c18aa9202d11f47": {
"describe": {
"columns": [],
"nullable": [],
"parameters": {
"Left": [
"Varchar",
"Jsonb",
"Text",
"Text"
]
}
},
"query": "UPDATE resource SET path = $1, value = $2 WHERE path = $3 AND workspace_id = $4"
},
"8c11511a74a41a65f448249a00ebe6964a61d00c2f7b4875a55e64741bf1f0ca": {
"describe": {
"columns": [
@@ -4698,27 +4697,6 @@
},
"query": "SELECT null FROM queue WHERE id = $1 FOR UPDATE"
},
"be1de9116b18a40681cb8c5ec7578fd7c5f7ce77f7af63f766c2899d85fe0bef": {
"describe": {
"columns": [
{
"name": "value",
"ordinal": 0,
"type_info": "Jsonb"
}
],
"nullable": [
true
],
"parameters": {
"Left": [
"Text",
"Text"
]
}
},
"query": "SELECT value FROM resource WHERE path = $1 AND workspace_id = $2"
},
"be7a99a5bb6858323ca61dd51077010f51ba58ae76b9a413339255024dcb524d": {
"describe": {
"columns": [],
@@ -5122,6 +5100,55 @@
},
"query": "select hash from script where path = $1 AND workspace_id = $2 AND\n created_at = (SELECT max(created_at) FROM script WHERE path = $1 AND workspace_id = $2) AND\n deleted = false"
},
"cc34f0298cfdaffe892a286e6b4390c287d33298e3342ccd1e10d48a9c3cc5c0": {
"describe": {
"columns": [
{
"name": "id",
"ordinal": 0,
"type_info": "Uuid"
},
{
"name": "workspace_id",
"ordinal": 1,
"type_info": "Varchar"
},
{
"name": "last_ping",
"ordinal": 2,
"type_info": "Timestamptz"
}
],
"nullable": [
false,
false,
false
],
"parameters": {
"Left": [
"Text",
{
"Custom": {
"kind": {
"Enum": [
"script",
"preview",
"flow",
"dependencies",
"flowpreview",
"script_hub",
"identity",
"flowdependencies"
]
},
"name": "job_kind"
}
}
]
}
},
"query": "UPDATE queue SET running = false WHERE last_ping < now() - ($1 || ' seconds')::interval AND running = true AND job_kind != $2 AND same_worker = false RETURNING id, workspace_id, last_ping"
},
"d0308abac80575038203b60bb66d3b39b586939da0421a595e47c7a759616431": {
"describe": {
"columns": [],
@@ -6409,72 +6436,6 @@
},
"query": "SELECT client, refresh_token FROM account WHERE workspace_id = $1 AND id = $2"
},
"fc2ec6fc4e22e46cc35b00503eff1eac35a6fdbc82a634b7eb9da4d7931f3582": {
"describe": {
"columns": [
{
"name": "id",
"ordinal": 0,
"type_info": "Uuid"
},
{
"name": "workspace_id",
"ordinal": 1,
"type_info": "Varchar"
},
{
"name": "last_ping",
"ordinal": 2,
"type_info": "Timestamptz"
}
],
"nullable": [
false,
false,
false
],
"parameters": {
"Left": [
"Text",
{
"Custom": {
"kind": {
"Enum": [
"script",
"preview",
"flow",
"dependencies",
"flowpreview",
"script_hub",
"identity",
"flowdependencies"
]
},
"name": "job_kind"
}
},
{
"Custom": {
"kind": {
"Enum": [
"script",
"preview",
"flow",
"dependencies",
"flowpreview",
"script_hub",
"identity",
"flowdependencies"
]
},
"name": "job_kind"
}
}
]
}
},
"query": "UPDATE queue SET running = false, started_at = null, logs = logs || '\nRestarted job after not receiving job''s ping for too long the ' || now() || '\n\n' WHERE last_ping < now() - ($1 || ' seconds')::interval AND running = true AND job_kind != $2 AND job_kind != $3 AND same_worker = false RETURNING id, workspace_id, last_ping"
},
"fcb6ce4fab662602827281e3e85fffaa48542f8d590100d0e7eead2ad4ad4873": {
"describe": {
"columns": [],

View File

@@ -42,7 +42,7 @@ async fn main() -> anyhow::Result<()> {
let server_bind_address: IpAddr = std::env::var("SERVER_BIND_ADDR")
.ok()
.and_then(|x| x.parse().ok())
.and_then(|x| x.parse().ok() )
.unwrap_or(IpAddr::from(DEFAULT_SERVER_BIND_ADDR));
let port: u16 = std::env::var("PORT")
@@ -91,8 +91,6 @@ Windmill Community Edition {GIT_VERSION}
"BASE_URL",
"BASE_INTERNAL_URL",
"TIMEOUT",
"ZOMBIE_JOB_TIMEOUT",
"RESTART_ZOMBIE_JOBS",
"SLEEP_QUEUE",
"MAX_LOG_SIZE",
"SERVER_BIND_ADDR",
@@ -114,7 +112,6 @@ Windmill Community Edition {GIT_VERSION}
"QUEUE_LIMIT_WAIT_RESULT",
"DENO_AUTH_TOKENS",
"DENO_FLAGS",
"NPM_CONFIG_REGISTRY",
"PIP_LOCAL_DEPENDENCIES",
"ADDITIONAL_PYTHON_PATHS",
"INCLUDE_HEADERS",
@@ -164,8 +161,6 @@ Windmill Community Edition {GIT_VERSION}
};
futures::try_join!(shutdown_signal, server_f, metrics_f, workers_f, monitor_f)?;
} else {
tracing::info!("Nothing to do, exiting.");
}
Ok(())
}

View File

@@ -686,7 +686,7 @@ def main(error, port):
result,
json!({
"recv": 42,
"from failure module": {"error": {"name": "IndexError", "stack": " File \"/tmp/tmp/main/step_0.py\", line 5, in main\n return sock.recv(1)[0]\n", "message": "index out of range"}},
"from failure module": {"error": {"name": "IndexError", "stack": " File \"/tmp/inner.py\", line 5, in main\n return sock.recv(1)[0]\n", "message": "index out of range"}},
})
);
}
@@ -2069,7 +2069,7 @@ async fn test_branchall_skip_failure(db: Pool<Postgres>) {
assert_eq!(
result,
serde_json::json!([{"error": {"name": "Error", "stack": "Error: failure\n at main (file:///tmp/main.ts:1:31)\n at run (file:///tmp/wrapper.ts:9:26)\n at file:///tmp/wrapper.ts:14:1", "message": "failure"}}, [1,3]])
serde_json::json!([{"error": {"name": "Error", "stack": "Error: failure\n at main (file:///tmp/inner.ts:1:31)\n at run (file:///tmp/main.ts:9:26)\n at file:///tmp/main.ts:14:1", "message": "failure"}}, [1,3]])
);
let flow: FlowValue = serde_json::from_value(json!({
@@ -2103,7 +2103,7 @@ async fn test_branchall_skip_failure(db: Pool<Postgres>) {
assert_eq!(
result,
serde_json::json!([ {"error": {"name": "Error", "stack": "Error: failure\n at main (file:///tmp/main.ts:1:31)\n at run (file:///tmp/wrapper.ts:9:26)\n at file:///tmp/wrapper.ts:14:1", "message": "failure"}}, [1, 2]])
serde_json::json!([ {"error": {"name": "Error", "stack": "Error: failure\n at main (file:///tmp/inner.ts:1:31)\n at run (file:///tmp/main.ts:9:26)\n at file:///tmp/main.ts:14:1", "message": "failure"}}, [1, 2]])
);
}

View File

@@ -69,4 +69,3 @@ async-stripe = { workspace = true, optional = true }
lazy_static.workspace = true
prometheus.workspace = true
async_zip.workspace = true
regex.workspace = true

View File

@@ -1,7 +1,7 @@
openapi: "3.0.3"
info:
version: 1.83.1
version: 1.74.2
title: Windmill API
contact:
@@ -204,6 +204,7 @@ paths:
schema:
type: string
/w/{workspace}/users/is_owner/{path}:
get:
summary: is owner of path
@@ -1548,7 +1549,7 @@ paths:
required: true
content:
application/json:
schema:
schema:
type: object
properties:
value: {}
@@ -2102,6 +2103,7 @@ paths:
items:
type: string
/w/{workspace}/scripts/create:
post:
summary: create script
@@ -2833,6 +2835,7 @@ paths:
schema:
$ref: "#/components/schemas/AppWithLastVersion"
/w/{workspace}/apps/secret_of/{path}:
get:
summary: get public secret of app
@@ -3165,14 +3168,12 @@ paths:
- $ref: "#/components/parameters/ScriptExactPath"
- $ref: "#/components/parameters/ScriptStartPath"
- $ref: "#/components/parameters/ScriptExactHash"
- $ref: "#/components/parameters/StartedBefore"
- $ref: "#/components/parameters/StartedAfter"
- $ref: "#/components/parameters/CreatedBefore"
- $ref: "#/components/parameters/CreatedAfter"
- $ref: "#/components/parameters/Success"
- $ref: "#/components/parameters/JobKinds"
- $ref: "#/components/parameters/Suspended"
- $ref: "#/components/parameters/Running"
- $ref: "#/components/parameters/ArgsFilter"
- $ref: "#/components/parameters/ResultFilter"
responses:
"200":
description: All available queued jobs
@@ -3197,12 +3198,10 @@ paths:
- $ref: "#/components/parameters/ScriptExactPath"
- $ref: "#/components/parameters/ScriptStartPath"
- $ref: "#/components/parameters/ScriptExactHash"
- $ref: "#/components/parameters/StartedBefore"
- $ref: "#/components/parameters/StartedAfter"
- $ref: "#/components/parameters/CreatedBefore"
- $ref: "#/components/parameters/CreatedAfter"
- $ref: "#/components/parameters/Success"
- $ref: "#/components/parameters/JobKinds"
- $ref: "#/components/parameters/ArgsFilter"
- $ref: "#/components/parameters/ResultFilter"
- name: is_skipped
description: is the job skipped
in: query
@@ -3236,11 +3235,9 @@ paths:
- $ref: "#/components/parameters/ScriptExactPath"
- $ref: "#/components/parameters/ScriptStartPath"
- $ref: "#/components/parameters/ScriptExactHash"
- $ref: "#/components/parameters/StartedBefore"
- $ref: "#/components/parameters/StartedAfter"
- $ref: "#/components/parameters/CreatedBefore"
- $ref: "#/components/parameters/CreatedAfter"
- $ref: "#/components/parameters/JobKinds"
- $ref: "#/components/parameters/ArgsFilter"
- $ref: "#/components/parameters/ResultFilter"
- name: is_skipped
description: is the job skipped
in: query
@@ -3441,7 +3438,7 @@ paths:
/w/{workspace}/jobs/resume_urls/{id}/{resume_id}:
get:
summary: get resume urls given a job_id, resume_id and a nonce to resume a flow
summary: get resume urls given a job_id, resume_id and a nonce to resume a flow
operationId: getResumeUrls
tags:
- job
@@ -3467,7 +3464,7 @@ paths:
properties:
approvalPage:
type: string
resume:
resume:
type: string
cancel:
type: string
@@ -4112,7 +4109,7 @@ paths:
type: string
owners:
type: array
items:
items:
type: string
extra_perms:
additionalProperties:
@@ -4146,7 +4143,7 @@ paths:
properties:
owners:
type: array
items:
items:
type: string
extra_perms:
additionalProperties:
@@ -4317,8 +4314,7 @@ paths:
required: true
schema:
type: string
enum:
[script, group_, resource, schedule, variable, flow, folder, app]
enum: [script, group_, resource, schedule, variable, flow, folder, app]
responses:
"200":
description: acls
@@ -4343,8 +4339,7 @@ paths:
required: true
schema:
type: string
enum:
[script, group_, resource, schedule, variable, flow, folder, app]
enum: [script, group_, resource, schedule, variable, flow, folder, app]
requestBody:
description: acl to add
required: true
@@ -4380,8 +4375,7 @@ paths:
required: true
schema:
type: string
enum:
[script, group_, resource, schedule, variable, flow, folder, app]
enum: [script, group_, resource, schedule, variable, flow, folder, app]
requestBody:
description: acl to add
required: true
@@ -4401,18 +4395,19 @@ paths:
schema:
type: string
/w/{workspace}/capture_u/{path}:
post:
summary: update flow preview capture
operationId: updateCapture
tags:
- capture
parameters:
- $ref: "#/components/parameters/WorkspaceId"
- $ref: "#/components/parameters/Path"
responses:
"204":
description: flow preview captured
post:
summary: update flow preview capture
operationId: updateCapture
tags:
- capture
parameters:
- $ref: "#/components/parameters/WorkspaceId"
- $ref: "#/components/parameters/Path"
responses:
"204":
description: flow preview captured
/w/{workspace}/capture/{path}:
put:
@@ -4619,15 +4614,15 @@ components:
in: query
schema:
type: string
StartedBefore:
name: started_before
CreatedBefore:
name: created_before
description: filter on created before (inclusive) timestamp
in: query
schema:
type: string
format: date-time
StartedAfter:
name: started_after
CreatedAfter:
name: created_after
description: filter on created after (exclusive) timestamp
in: query
schema:
@@ -4651,19 +4646,6 @@ components:
in: query
schema:
type: boolean
ArgsFilter:
name: args
description: filter on jobs containing those args as a json subset (@> in postgres)
in: query
schema:
type: string
ResultFilter:
name: result
description: filter on jobs containing those result as a json subset (@> in postgres)
in: query
schema:
type: string
After:
name: after
description: filter on created after (exclusive) timestamp
@@ -5568,8 +5550,9 @@ components:
type: string
worker_instance:
type: string
last_ping:
type: number
ping_at:
type: string
format: date-time
started_at:
type: string
format: date-time

View File

@@ -320,10 +320,6 @@ async fn create_app(
app.policy.on_behalf_of = Some(username_to_permissioned_as(&authed.username));
app.policy.on_behalf_of_email = Some(authed.email);
if &app.path == "" {
return Err(Error::BadRequest("App path cannot be empty".to_string()));
}
let id = sqlx::query_scalar!(
"INSERT INTO app
(workspace_id, path, summary, policy, versions)

View File

@@ -16,11 +16,10 @@ use axum::{
routing::{delete, get, post},
Json, Router,
};
use lazy_static::lazy_static;
use regex::Regex;
use itertools::Itertools;
use windmill_audit::{audit_log, ActionKind};
use windmill_common::{
error::{self, to_anyhow, Error, JsonResult, Result},
error::{self, Error, JsonResult, Result},
users::username_to_permissioned_as,
utils::{not_found_if_none, paginate, Pagination},
};
@@ -138,10 +137,6 @@ async fn check_name_conflict<'c>(
return Ok(());
}
lazy_static! {
static ref VALID_FOLDER_NAME: Regex = Regex::new(r#"^[a-zA-Z_0-9]+$"#).unwrap();
}
async fn create_folder(
authed: Authed,
Extension(user_db): Extension<UserDB>,
@@ -151,12 +146,6 @@ async fn create_folder(
) -> Result<String> {
let mut tx = user_db.begin(&authed).await?;
if !VALID_FOLDER_NAME.is_match(&ng.name) {
return Err(windmill_common::error::Error::BadRequest(format!(
"Folder name can only contain alphanumeric characters, underscores"
)));
}
check_name_conflict(&mut tx, &w_id, &ng.name).await?;
let owner = username_to_permissioned_as(&authed.username);
let owners = &ng.owners.unwrap_or(vec![owner.clone()]);
@@ -273,26 +262,13 @@ async fn update_folder(
sqlb.and_where_eq("workspace_id", "?".bind(&w_id));
if let Some(display_name) = ng.display_name {
sqlb.set("display_name", "?".bind(&display_name));
sqlb.set("display_name", display_name);
}
if let Some(owners) = ng.owners {
sqlb.set(
"owners",
"?".bind(&format!(
"{{{}}}",
owners
.iter()
.map(|x| format!("\"{x}\""))
.collect::<Vec<_>>()
.join(","),
)),
);
sqlb.set_str("owners", format!("{{{}}}", owners.into_iter().join(",")));
}
if let Some(extra_perms) = ng.extra_perms {
sqlb.set(
"extra_perms",
"?".bind(&serde_json::to_string(&extra_perms).map_err(to_anyhow)?),
);
sqlb.set_str("extra_perms", extra_perms.to_string());
}
sqlb.returning("*");

View File

@@ -34,7 +34,7 @@ use windmill_queue::{get_queued_job, push, JobKind, JobPayload, QueuedJob, RawCo
use crate::{
db::{UserDB, DB},
users::{require_owner_of_path, Authed, OptAuthed},
users::{require_owner_of_path, Authed},
variables::get_workspace_key,
BASE_URL,
};
@@ -60,6 +60,7 @@ pub fn workspaced_service() -> Router {
.route("/run/preview_flow", post(run_preview_flow_job))
.route("/list", get(list_jobs))
.route("/queue/list", get(list_queue_jobs))
.route("/queue/cancel/:id", post(cancel_job_api))
.route("/completed/list", get(list_completed_jobs))
.route("/completed/get/:id", get(get_completed_job))
.route("/completed/get_result/:id", get(get_completed_job_result))
@@ -97,7 +98,6 @@ pub fn global_service() -> Router {
)
.route("/get/:id", get(get_job))
.route("/getupdate/:id", get(get_job_update))
.route("/queue/cancel/:id", post(cancel_job_api))
}
async fn get_result_by_id(
@@ -109,24 +109,20 @@ async fn get_result_by_id(
}
async fn cancel_job_api(
OptAuthed(opt_authed): OptAuthed,
Extension(db): Extension<DB>,
authed: Authed,
Extension(user_db): Extension<UserDB>,
Path((w_id, id)): Path<(String, Uuid)>,
Json(CancelJob { reason }): Json<CancelJob>,
) -> error::Result<String> {
let tx = db.begin().await?;
let tx = user_db.begin(&authed).await?;
let username = match opt_authed {
Some(authed) => authed.username,
None => "anonymous".to_string(),
};
let (mut tx, job_option) = windmill_queue::cancel_job(&username, reason, id, &w_id, tx).await?;
let (mut tx, job_option) =
windmill_queue::cancel_job(&authed.username, reason, id, &w_id, tx).await?;
if let Some(id) = job_option {
audit_log(
&mut tx,
&username,
&authed.username,
"jobs.cancel",
ActionKind::Delete,
&w_id,
@@ -304,6 +300,9 @@ pub fn add_include_headers(
headers: HeaderMap,
mut args: serde_json::Map<String, serde_json::Value>,
) -> serde_json::Map<String, serde_json::Value> {
if include_header.is_none() {
return args;
}
let whitelist = include_header
.as_ref()
.map(|s| s.split(",").map(|s| s.to_string()).collect::<Vec<_>>())
@@ -322,22 +321,19 @@ pub fn add_include_headers(
});
args
}
#[derive(Deserialize)]
pub struct ListQueueQuery {
pub script_path_start: Option<String>,
pub script_path_exact: Option<String>,
pub script_hash: Option<String>,
pub created_by: Option<String>,
pub started_before: Option<chrono::DateTime<chrono::Utc>>,
pub started_after: Option<chrono::DateTime<chrono::Utc>>,
pub created_before: Option<chrono::DateTime<chrono::Utc>>,
pub created_after: Option<chrono::DateTime<chrono::Utc>>,
pub running: Option<bool>,
pub parent_job: Option<String>,
pub order_desc: Option<bool>,
pub job_kinds: Option<String>,
pub suspended: Option<bool>,
// filter by matching a subset of the args using base64 encoded json subset
pub args: Option<String>,
}
fn list_queue_jobs_query(w_id: &str, lq: &ListQueueQuery, fields: &[&str]) -> SqlBuilder {
@@ -366,11 +362,11 @@ fn list_queue_jobs_query(w_id: &str, lq: &ListQueueQuery, fields: &[&str]) -> Sq
if let Some(pj) = &lq.parent_job {
sqlb.and_where_eq("parent_job", "?".bind(pj));
}
if let Some(dt) = &lq.started_before {
sqlb.and_where_le("started_at", format!("to_timestamp({})", dt.timestamp()));
if let Some(dt) = &lq.created_before {
sqlb.and_where_lt("created_at", format!("to_timestamp({})", dt.timestamp()));
}
if let Some(dt) = &lq.started_after {
sqlb.and_where_ge("started_at", format!("to_timestamp({})", dt.timestamp()));
if let Some(dt) = &lq.created_after {
sqlb.and_where_gt("created_at", format!("to_timestamp({})", dt.timestamp()));
}
if let Some(s) = &lq.suspended {
@@ -380,7 +376,6 @@ fn list_queue_jobs_query(w_id: &str, lq: &ListQueueQuery, fields: &[&str]) -> Sq
sqlb.and_where_eq("suspend", 0);
}
}
if let Some(jk) = &lq.job_kinds {
sqlb.and_where_in(
"job_kind",
@@ -388,10 +383,6 @@ fn list_queue_jobs_query(w_id: &str, lq: &ListQueueQuery, fields: &[&str]) -> Sq
);
}
if let Some(args) = &lq.args {
sqlb.and_where("args @> ?".bind(&args.replace("'", "''")));
}
sqlb
}
@@ -464,14 +455,13 @@ async fn list_jobs(
script_path_exact: lq.script_path_exact,
script_hash: lq.script_hash,
created_by: lq.created_by,
started_before: lq.started_before,
started_after: lq.started_after,
created_before: lq.created_before,
created_after: lq.created_after,
running: None,
parent_job: lq.parent_job,
order_desc: Some(true),
job_kinds: lq.job_kinds,
suspended: lq.suspended,
args: lq.args,
},
&[
"'QueuedJob' as typ",
@@ -1146,14 +1136,14 @@ where
struct InPayload {
payload: Option<String>,
}
}
}
fn decode_payload<D: DeserializeOwned, T: AsRef<[u8]>>(t: T) -> anyhow::Result<D> {
let vec = base64::engine::general_purpose::URL_SAFE
.decode(t)
.context("invalid base64")?;
serde_json::from_slice(vec.as_slice()).context("invalid json")
fn decode_payload<D: DeserializeOwned, T: AsRef<[u8]>>(t: T) -> anyhow::Result<D> {
let vec = base64::engine::general_purpose::URL_SAFE
.decode(t)
.context("invalid base64")?;
serde_json::from_slice(vec.as_slice()).context("invalid json")
}
}
}
pub async fn run_flow_by_path(
authed: Authed,
@@ -1697,11 +1687,11 @@ fn list_completed_jobs_query(
if let Some(pj) = &lq.parent_job {
sqlb.and_where_eq("parent_job", "?".bind(pj));
}
if let Some(dt) = &lq.started_before {
sqlb.and_where_le("started_at", format!("to_timestamp({})", dt.timestamp()));
if let Some(dt) = &lq.created_before {
sqlb.and_where_lt("created_at", format!("to_timestamp({})", dt.timestamp()));
}
if let Some(dt) = &lq.started_after {
sqlb.and_where_ge("started_at", format!("to_timestamp({})", dt.timestamp()));
if let Some(dt) = &lq.created_after {
sqlb.and_where_gt("created_at", format!("to_timestamp({})", dt.timestamp()));
}
if let Some(sk) = &lq.is_skipped {
sqlb.and_where_eq("is_skipped", sk);
@@ -1716,14 +1706,6 @@ fn list_completed_jobs_query(
);
}
if let Some(args) = &lq.args {
sqlb.and_where("args @> ?".bind(&args.replace("'", "''")));
}
if let Some(result) = &lq.result {
sqlb.and_where("result @> ?".bind(&result.replace("'", "''")));
}
sqlb
}
#[derive(Deserialize, Clone)]
@@ -1732,8 +1714,8 @@ pub struct ListCompletedQuery {
pub script_path_exact: Option<String>,
pub script_hash: Option<String>,
pub created_by: Option<String>,
pub started_before: Option<chrono::DateTime<chrono::Utc>>,
pub started_after: Option<chrono::DateTime<chrono::Utc>>,
pub created_before: Option<chrono::DateTime<chrono::Utc>>,
pub created_after: Option<chrono::DateTime<chrono::Utc>>,
pub success: Option<bool>,
pub parent_job: Option<String>,
pub order_desc: Option<bool>,
@@ -1741,10 +1723,6 @@ pub struct ListCompletedQuery {
pub is_skipped: Option<bool>,
pub is_flow_step: Option<bool>,
pub suspended: Option<bool>,
// filter by matching a subset of the args using base64 encoded json subset
pub args: Option<String>,
// filter by matching a subset of the result using base64 encoded json subset
pub result: Option<String>,
}
async fn list_completed_jobs(

View File

@@ -25,9 +25,7 @@ use axum::{
Json, Router,
};
use hyper::{header::LOCATION, StatusCode};
use lazy_static::lazy_static;
use rand::rngs::OsRng;
use regex::Regex;
use retainer::Cache;
use serde::{Deserialize, Serialize};
use sqlx::FromRow;
@@ -37,7 +35,6 @@ use tracing::{Instrument, Span};
use windmill_audit::{audit_log, ActionKind};
use windmill_common::{
error::{self, Error, JsonResult, Result},
users::SUPERADMIN_SECRET_EMAIL,
utils::{not_found_if_none, rd_string, require_admin, Pagination, StripPath},
};
use windmill_queue::CLOUD_HOSTED;
@@ -269,7 +266,7 @@ impl AuthCache {
.unwrap_or(false)
{
Some(Authed {
email: SUPERADMIN_SECRET_EMAIL.to_string(),
email: "superadmin_secret@windmill.dev".to_string(),
username: "superadmin_secret".to_string(),
is_admin: true,
groups: Vec::new(),
@@ -1000,10 +997,6 @@ async fn decline_invite(
}
}
lazy_static! {
pub static ref VALID_USERNAME: Regex = Regex::new(r#"^[a-zA-Z_0-9]+$"#).unwrap();
}
async fn accept_invite(
Authed { email, .. }: Authed,
Extension(db): Extension<DB>,
@@ -1012,13 +1005,6 @@ async fn accept_invite(
if &nu.username == "bot" {
return Err(Error::BadRequest("bot is a reserved username".to_string()));
}
if !VALID_USERNAME.is_match(&nu.username) {
return Err(windmill_common::error::Error::BadRequest(format!(
"Usermame can only contain alphanumeric characters and underscores"
)));
}
let mut tx = db.begin().await?;
let r = sqlx::query!(
@@ -1091,12 +1077,6 @@ async fn add_user_to_workspace<'c>(
)));
}
if !VALID_USERNAME.is_match(username) {
return Err(windmill_common::error::Error::BadRequest(format!(
"Usermame can only contain alphanumeric characters and underscores"
)));
}
let already_exists_email = sqlx::query_scalar!(
"SELECT EXISTS(SELECT 1 FROM usr WHERE workspace_id = $1 AND email = $2)",
&w_id,
@@ -1302,12 +1282,6 @@ async fn create_user(
require_super_admin(&mut tx, &email).await?;
if nu.email == SUPERADMIN_SECRET_EMAIL {
return Err(Error::BadRequest(
"The superadmin email is a reserved email".into(),
));
}
sqlx::query!(
"INSERT INTO password(email, verified, password_hash, login_type, super_admin, name, \
company)

View File

@@ -7,24 +7,16 @@
*/
use sqlx::{Postgres, Transaction};
use windmill_common::{
error::{self, Error},
users::SUPERADMIN_SECRET_EMAIL,
};
use windmill_common::error::{self, Error};
pub async fn require_super_admin<'c>(
db: &mut Transaction<'c, Postgres>,
email: &str,
) -> error::Result<()> {
if email == SUPERADMIN_SECRET_EMAIL {
return Ok(());
}
let is_admin = sqlx::query_scalar!("SELECT super_admin FROM password WHERE email = $1", email)
.fetch_optional(db)
.fetch_one(db)
.await
.map_err(|e| Error::InternalErr(format!("fetching super admin: {e}")))?
.unwrap_or(false);
.map_err(|e| Error::InternalErr(format!("fetching super admin: {e}")))?;
if !is_admin {
Err(Error::NotAuthorized(
"This endpoint require caller to be a super admin".to_owned(),

View File

@@ -26,7 +26,6 @@ use axum::{
Json, Router,
};
use hyper::StatusCode;
use serde_json::Value;
use windmill_audit::{audit_log, ActionKind};
use windmill_common::{
error::{Error, JsonResult, Result},
@@ -379,27 +378,9 @@ async fn update_variable(
if !authed.is_admin {
require_owner_of_path(&w_id, &authed.username, &authed.groups, &path, &db).await?;
}
let mut v = sqlx::query_scalar!(
"SELECT value FROM resource WHERE path = $1 AND workspace_id = $2",
path,
w_id
)
.fetch_optional(&mut tx)
.await?
.flatten();
if let Some(old_v) = v {
v = Some(replace_path(
old_v,
&format!("$var:{path}"),
&format!("$var:{npath}"),
))
}
sqlx::query!(
"UPDATE resource SET path = $1, value = $2 WHERE path = $3 AND workspace_id = $4",
"UPDATE resource SET path = $1 WHERE path = $2 AND workspace_id = $3",
npath,
v,
path,
w_id
)
@@ -438,23 +419,6 @@ async fn update_variable(
Ok(format!("variable {} updated (npath: {:?})", path, npath))
}
fn replace_path(v: serde_json::Value, path: &str, npath: &str) -> Value {
match v {
Value::Object(v) => Value::Object(
v.into_iter()
.map(|(k, v)| (k, replace_path(v, path, npath)))
.collect(),
),
Value::Array(arr) => Value::Array(
arr.into_iter()
.map(|v| replace_path(v, path, npath))
.collect(),
),
Value::String(s) if s == path => Value::String(npath.to_owned()),
_ => v,
}
}
pub async fn build_crypt<'c>(
db: &mut Transaction<'c, Postgres>,
w_id: &str,

View File

@@ -28,7 +28,7 @@ pub fn global_service() -> Router {
struct WorkerPing {
worker: String,
worker_instance: String,
last_ping: Option<i32>,
ping_at: chrono::DateTime<chrono::Utc>,
started_at: chrono::DateTime<chrono::Utc>,
ip: String,
jobs_executed: i32,
@@ -45,7 +45,7 @@ async fn list_worker_pings(
let rows = sqlx::query_as!(
WorkerPing,
"SELECT worker, worker_instance, EXTRACT(EPOCH FROM (now() - ping_at))::integer as last_ping, started_at, ip, jobs_executed FROM worker_ping ORDER BY ping_at desc LIMIT $1 OFFSET $2",
"SELECT * FROM worker_ping ORDER BY ping_at desc LIMIT $1 OFFSET $2",
per_page as i64,
offset as i64
)

View File

@@ -16,9 +16,8 @@ use crate::{
db::{UserDB, DB},
folders::Folder,
resources::{Resource, ResourceType},
users::{Authed, WorkspaceInvite, NEW_USER_WEBHOOK, VALID_USERNAME},
users::{Authed, WorkspaceInvite, NEW_USER_WEBHOOK},
utils::require_super_admin,
variables::build_crypt,
HTTP_CLIENT,
};
#[cfg(feature = "enterprise")]
@@ -31,7 +30,6 @@ use axum::{
routing::{delete, get, post},
Json, Router,
};
use magic_crypt::MagicCryptTrait;
#[cfg(feature = "enterprise")]
use stripe::CustomerId;
use windmill_audit::{audit_log, ActionKind};
@@ -988,11 +986,6 @@ async fn add_user(
require_admin(is_admin, &username)?;
let mut tx = db.begin().await?;
if !VALID_USERNAME.is_match(&nu.username) {
return Err(windmill_common::error::Error::BadRequest(format!(
"Usermame can only contain alphanumeric characters and underscores"
)));
}
sqlx::query!(
"INSERT INTO usr
@@ -1007,16 +1000,6 @@ async fn add_user(
.execute(&mut tx)
.await?;
sqlx::query_as!(
Group,
"INSERT INTO usr_to_group (workspace_id, usr, group_) VALUES ($1, $2, $3) ON CONFLICT DO NOTHING",
&w_id,
nu.username,
"all",
)
.execute(&mut tx)
.await?;
tx.commit().await?;
if let Some(new_user_webhook) = NEW_USER_WEBHOOK.clone() {
@@ -1090,7 +1073,6 @@ struct ScriptMetadata {
schema: Option<Schema>,
is_template: bool,
lock: Vec<String>,
kind: String,
}
enum ArchiveImpl {
@@ -1142,7 +1124,6 @@ impl ArchiveImpl {
#[derive(Deserialize)]
struct ArchiveQueryParams {
archive_type: Option<String>,
plain_secret: Option<bool>,
}
#[inline]
@@ -1188,7 +1169,7 @@ async fn tarball_workspace(
authed: Authed,
Extension(db): Extension<DB>,
Path(w_id): Path<String>,
Query(ArchiveQueryParams { archive_type, plain_secret }): Query<ArchiveQueryParams>,
Query(ArchiveQueryParams { archive_type }): Query<ArchiveQueryParams>,
) -> Result<([(headers::HeaderName, String); 2], impl IntoResponse)> {
require_admin(authed.is_admin, &authed.username)?;
@@ -1254,7 +1235,6 @@ async fn tarball_workspace(
description: script.description,
schema: script.schema,
is_template: script.is_template,
kind: script.kind.to_string(),
lock,
};
let metadata_str = serde_json::to_string_pretty(&metadata).unwrap();
@@ -1325,15 +1305,7 @@ async fn tarball_workspace(
.fetch_all(&db)
.await?;
let mc = build_crypt(&mut db.begin().await?, &w_id).await?;
for mut var in variables {
if plain_secret.unwrap_or(false) && var.value.is_some() {
var.value = Some(
mc.decrypt_base64_to_string(var.value.unwrap())
.map_err(|e| Error::InternalErr(e.to_string()))?,
);
}
for var in variables {
let var_str = &to_string_without_metadata(&var, false).unwrap();
archive
.write_to_archive(&var_str, &format!("{}.variable.json", var.path))

View File

@@ -7,7 +7,7 @@
*/
use std::{
fmt::{self, Display},
fmt::Display,
hash::{Hash, Hasher},
};
@@ -103,18 +103,6 @@ pub enum ScriptKind {
Approval,
}
impl Display for ScriptKind {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.write_str(match self {
ScriptKind::Trigger => "trigger",
ScriptKind::Failure => "failure",
ScriptKind::Script => "script",
ScriptKind::Approval => "approval",
})?;
Ok(())
}
}
#[derive(Serialize)]
#[cfg_attr(feature = "sqlx", derive(sqlx::FromRow))]
pub struct Script {

View File

@@ -6,8 +6,6 @@
* LICENSE-AGPL for a copy of the license.
*/
pub const SUPERADMIN_SECRET_EMAIL: &str = "superadmin_secret@windmill.dev";
pub fn username_to_permissioned_as(user: &str) -> String {
if user.contains('@') {
user.to_string()

View File

@@ -658,7 +658,7 @@ impl QueuedJob {
self.script_path
.as_ref()
.map(String::as_str)
.unwrap_or("tmp/main")
.unwrap_or("NO_FLOW_PATH")
}
}

View File

@@ -48,4 +48,3 @@ deno_core.workspace = true
const_format.workspace = true
git-version.workspace = true
dyn-iter.workspace = true
once_cell.workspace = true

View File

@@ -61,8 +61,8 @@ mount {
mount {
src: "{JOB_DIR}/wrapper.ts"
dst: "/tmp/wrapper.ts"
src: "{JOB_DIR}/inner.ts"
dst: "/tmp/inner.ts"
is_bind: true
}

View File

@@ -66,8 +66,8 @@ mount {
}
mount {
src: "{JOB_DIR}/{MAIN}.py"
dst: "/tmp/{MAIN}.py"
src: "{JOB_DIR}/inner.py"
dst: "/tmp/inner.py"
is_bind: true
}
@@ -79,8 +79,8 @@ mount {
}
mount {
src: "{JOB_DIR}/wrapper.py"
dst: "/tmp/wrapper.py"
src: "{JOB_DIR}/main.py"
dst: "/tmp/main.py"
is_bind: true
}

View File

@@ -6,11 +6,9 @@
* LICENSE-AGPL for a copy of the license.
*/
use std::{cell::RefCell, collections::HashMap, rc::Rc};
use std::collections::HashMap;
use deno_core::{
op, serde_v8, v8, v8::IsolateHandle, Extension, JsRuntime, OpState, RuntimeOptions,
};
use deno_core::{op, serde_v8, v8, v8::IsolateHandle, Extension, JsRuntime, RuntimeOptions};
use itertools::Itertools;
use lazy_static::lazy_static;
use regex::Regex;
@@ -19,7 +17,10 @@ use tokio::{sync::oneshot, time::timeout};
use uuid::Uuid;
use windmill_common::{error::Error, flow_status::JobResult};
use crate::AuthedClient;
pub struct EvalCreds {
pub workspace: String,
pub token: String,
}
#[derive(Debug, Clone)]
pub struct IdContext {
@@ -28,23 +29,22 @@ pub struct IdContext {
pub previous_id: String,
}
pub struct OptAuthedClient(Option<AuthedClient>);
pub async fn eval_timeout(
expr: String,
env: Vec<(String, serde_json::Value)>,
authed_client: Option<&AuthedClient>,
creds: Option<EvalCreds>,
by_id: Option<IdContext>,
base_internal_url: &str,
) -> anyhow::Result<serde_json::Value> {
let expr2 = expr.clone();
let (sender, mut receiver) = oneshot::channel::<IsolateHandle>();
let has_client = authed_client.is_some();
let authed_client = authed_client.cloned();
let base_internal_url: String = base_internal_url.to_string();
timeout(
std::time::Duration::from_millis(10000),
std::time::Duration::from_millis(3000),
tokio::task::spawn_blocking(move || {
let mut ops = vec![];
if authed_client.is_some() {
if creds.is_some() {
ops.extend([
// An op for summing an array of numbers
// The op-layer automatically deserializes inputs
@@ -54,12 +54,12 @@ pub async fn eval_timeout(
])
}
if by_id.is_some() && authed_client.is_some() {
if by_id.is_some() {
ops.push(op_get_result::decl());
ops.push(op_get_id::decl());
}
let ext = Extension::builder("js_eval").ops(ops).build();
let ext = Extension::builder().ops(ops).build();
// Use our snapshot to provision our new runtime
let options = RuntimeOptions {
extensions: vec![ext],
@@ -68,11 +68,6 @@ pub async fn eval_timeout(
};
let mut js_runtime = JsRuntime::new(options);
{
let op_state = js_runtime.op_state();
let mut op_state = op_state.borrow_mut();
op_state.put(OptAuthedClient(authed_client.clone()));
}
sender
.send(js_runtime.v8_isolate().thread_safe_handle())
@@ -91,7 +86,14 @@ pub async fn eval_timeout(
let expr = replace_with_await_result(expr);
let r = runtime.block_on(eval(&mut js_runtime, &expr, env, by_id, has_client))?;
let r = runtime.block_on(eval(
&mut js_runtime,
&expr,
env,
creds,
by_id,
&base_internal_url,
))?;
Ok(r) as anyhow::Result<Value>
}),
@@ -102,7 +104,7 @@ pub async fn eval_timeout(
isolate.terminate_execution();
};
Error::ExecutionErr(format!(
"The expression of evaluation `{expr2}` took too long to execute (>10000ms)"
"The expression of evaluation `{expr2}` took too long to execute (>3000ms)"
))
})??
}
@@ -148,8 +150,9 @@ async fn eval(
context: &mut JsRuntime,
expr: &str,
env: Vec<(String, serde_json::Value)>,
creds: Option<EvalCreds>,
by_id: Option<IdContext>,
has_client: bool,
base_internal_url: &str,
) -> anyhow::Result<serde_json::Value> {
let exprs = expr
.trim()
@@ -166,7 +169,7 @@ async fn eval(
exprs.last().unwrap()
)
};
let (api_code, by_id_code) = if has_client {
let (api_code, by_id_code) = if let Some(EvalCreds { workspace, token }) = creds {
let by_id_code = if let Some(by_id) = by_id {
format!(
r#"
@@ -183,12 +186,12 @@ async function result_by_id(node_id) {{
}}
}} else {{
let flow_job_id = "{}";
return await Deno.core.opAsync("op_get_id", [ flow_job_id, node_id]);
return await Deno.core.opAsync("op_get_id", [workspace, flow_job_id, token, base_url, node_id]);
}}
}}
async function get_result(id) {{
return await Deno.core.opAsync("op_get_result", [id]);
return await Deno.core.opAsync("op_get_result", [workspace, id, token, base_url]);
}}
const results = new Proxy({{}}, {{
get: function(target, name, receiver) {{
@@ -219,13 +222,17 @@ const results = new Proxy({{}}, {{
let api_code = format!(
r#"
let workspace = "{workspace}";
let base_url = "{}";
let token = "{token}";
async function variable(path) {{
return await Deno.core.opAsync("op_variable", [path]);
return await Deno.core.opAsync("op_variable", [workspace, path, token, base_url]);
}}
async function resource(path) {{
return await Deno.core.opAsync("op_resource", [path]);
return await Deno.core.opAsync("op_resource", [workspace, path, token, base_url]);
}}
"#,
base_internal_url,
);
(api_code, by_id_code)
} else {
@@ -252,7 +259,7 @@ async function resource(path) {{
.join(""),
);
tracing::debug!("{}", code);
let global = context.execute_script("<anon>", code)?;
let global = context.execute_script("<anon>", &code)?;
let global = context.resolve_value(global).await?;
let scope = &mut context.handle_scope();
@@ -274,83 +281,59 @@ async function resource(path) {{
// TODO: Can we a) share the api configuration here somehow or b) just implement this natively in deno, via the deno client?
#[op]
async fn op_variable(
op_state: Rc<RefCell<OpState>>,
args: Vec<String>,
) -> Result<String, anyhow::Error> {
let path = &args[0];
let client = op_state.borrow().borrow::<OptAuthedClient>().0.clone();
if let Some(client) = client {
let result = client
.get_client()
.get_variable(&client.workspace, path, None)
.await?;
Ok(result.into_inner().value.unwrap_or_else(|| "".to_owned()))
} else {
anyhow::bail!("No client found in op state");
}
async fn op_variable(args: Vec<String>) -> Result<String, anyhow::Error> {
let workspace = &args[0];
let path = &args[1];
let token = &args[2];
let base_url = &args[3];
let client = windmill_api_client::create_client(base_url, token.clone());
let result = client.get_variable(workspace, path, None).await?;
Ok(result.into_inner().value.unwrap_or_else(|| "".to_owned()))
}
#[op]
async fn op_get_result(
op_state: Rc<RefCell<OpState>>,
args: Vec<String>,
) -> Result<serde_json::Value, anyhow::Error> {
let id = &args[0];
let client = op_state.borrow().borrow::<OptAuthedClient>().0.clone();
if let Some(client) = client {
let result = client
.get_client()
.get_completed_job_result(&client.workspace, &id.parse()?)
.await?
.clone();
Ok(serde_json::json!(result))
} else {
anyhow::bail!("No client found in op state");
}
async fn op_get_result(args: Vec<String>) -> Result<serde_json::Value, anyhow::Error> {
let workspace = &args[0];
let id = &args[1];
let token = &args[2];
let base_url = &args[3];
let client = windmill_api_client::create_client(base_url, token.clone());
let result = client
.get_completed_job_result(workspace, &id.parse()?)
.await?
.clone();
Ok(serde_json::json!(result))
}
#[op]
async fn op_get_id(
op_state: Rc<RefCell<OpState>>,
args: Vec<String>,
) -> Result<Option<serde_json::Value>, anyhow::Error> {
let flow_job_id = &args[0];
let node_id = &args[1];
async fn op_get_id(args: Vec<String>) -> Result<Option<serde_json::Value>, anyhow::Error> {
let workspace = &args[0];
let flow_job_id = &args[1];
let token = &args[2];
let base_url = &args[3];
let node_id = &args[4];
let client = op_state.borrow().borrow::<OptAuthedClient>().0.clone();
if let Some(client) = client {
let result = client
.get_client()
.result_by_id(&client.workspace, flow_job_id, node_id)
.await
.map_or(None, |e| Some(e.into_inner()));
Ok(result)
} else {
anyhow::bail!("No client found in op state");
}
let client = windmill_api_client::create_client(base_url, token.clone());
let result = client
.result_by_id(workspace, flow_job_id, node_id)
.await
.map_or(None, |e| Some(e.into_inner()));
Ok(result)
}
#[op]
async fn op_resource(
op_state: Rc<RefCell<OpState>>,
args: Vec<String>,
) -> Result<serde_json::Value, anyhow::Error> {
let path = &args[0];
let client = op_state.borrow().borrow::<OptAuthedClient>().0.clone();
if let Some(client) = client {
let result = client
.get_client()
.get_resource(&client.workspace, path)
.await?;
Ok(result
.into_inner()
.value
.unwrap_or_else(|| serde_json::json!({})))
} else {
anyhow::bail!("No client found in op state");
}
async fn op_resource(args: Vec<String>) -> Result<serde_json::Value, anyhow::Error> {
let workspace = &args[0];
let path = &args[1];
let token = &args[2];
let base_url = &args[3];
let client = windmill_api_client::create_client(base_url, token.clone());
let result = client.get_resource(workspace, path).await?;
Ok(result
.into_inner()
.value
.unwrap_or_else(|| serde_json::json!({})))
}
#[cfg(test)]
@@ -370,7 +353,7 @@ mod tests {
let code = "value.test + params.test";
let mut runtime = JsRuntime::new(RuntimeOptions::default());
let res = eval(&mut runtime, code, env, None, false).await?;
let res = eval(&mut runtime, code, env, None, None, String::new().as_str()).await?;
assert_eq!(res, json!(4));
Ok(())
}
@@ -383,7 +366,7 @@ mod tests {
multiline template`";
let mut runtime = JsRuntime::new(RuntimeOptions::default());
let res = eval(&mut runtime, code, env, None, false).await?;
let res = eval(&mut runtime, code, env, None, None, String::new().as_str()).await?;
assert_eq!(res, json!("my 5\nmultiline template"));
Ok(())
}
@@ -396,7 +379,7 @@ multiline template`";
];
let code = r#"params.test"#;
let res = eval_timeout(code.to_string(), env, None, None).await?;
let res = eval_timeout(code.to_string(), env, None, None, String::new().as_str()).await?;
assert_eq!(res, json!(2));
Ok(())
}

View File

@@ -9,10 +9,8 @@
use const_format::concatcp;
use itertools::Itertools;
use lazy_static::lazy_static;
use once_cell::sync::OnceCell;
use regex::Regex;
use sqlx::{Pool, Postgres, Transaction};
use windmill_api_client::Client;
use std::{
borrow::Borrow, collections::HashMap, io, os::unix::process::ExitStatusExt, panic,
process::Stdio, time::Duration,
@@ -25,7 +23,7 @@ use windmill_common::{
flows::{FlowModuleValue, FlowValue},
scripts::{ScriptHash, ScriptLang},
utils::rd_string,
variables, BASE_URL, users::SUPERADMIN_SECRET_EMAIL,
variables, BASE_URL,
};
use windmill_queue::{canceled_job_to_result, get_queued_job, pull, JobKind, QueuedJob, CLOUD_HOSTED};
@@ -36,7 +34,7 @@ use tokio::{
io::{AsyncBufReadExt, AsyncReadExt, AsyncWriteExt, BufReader},
process::{Child, Command},
sync::{
mpsc::{self, Sender}, watch, broadcast
mpsc::{self, Sender}, watch, broadcast,
},
time::{interval, sleep, Instant, MissedTickBehavior},
};
@@ -238,10 +236,11 @@ pub async fn create_token_for_owner<'c>(
) -> error::Result<(Transaction<'c, Postgres>, String)> {
// TODO: Bad implementation. We should not have access to this DB here.
let token: String = rd_string(30);
let is_super_admin = sqlx::query_scalar!("SELECT super_admin FROM password WHERE email = $1", email)
let is_super_admin = owner.contains('@')
&& sqlx::query_scalar!("SELECT super_admin FROM password WHERE email = $1", owner)
.fetch_optional(&mut tx)
.await?
.unwrap_or(false) || email == SUPERADMIN_SECRET_EMAIL;
.unwrap_or(false);
sqlx::query_scalar!(
"INSERT INTO token
@@ -265,7 +264,7 @@ const ROOT_CACHE_DIR: &str = "/tmp/windmill/cache/";
const PIP_CACHE_DIR: &str = concatcp!(ROOT_CACHE_DIR, "pip");
const DENO_CACHE_DIR: &str = concatcp!(ROOT_CACHE_DIR, "deno");
const GO_CACHE_DIR: &str = concatcp!(ROOT_CACHE_DIR, "go");
const NUM_SECS_PING: u64 = 5;
const NUM_SECS_ENV_CHECK: u64 = 15;
const NUM_SECS_SYNC: u64 = 60 * 10;
const INCLUDE_DEPS_PY_SH_CONTENT: &str = include_str!("../nsjail/download_deps.py.sh");
@@ -332,7 +331,8 @@ lazy_static::lazy_static! {
.map(|x| format!(";{x}"))
.unwrap_or_else(|| String::new());
static ref NPM_CONFIG_REGISTRY: Option<String> = std::env::var("NPM_CONFIG_REGISTRY").ok();
static ref DENO_FLAGS: Option<Vec<String>> = std::env::var("DENO_FLAGS")
.ok()
@@ -390,19 +390,10 @@ lazy_static::lazy_static! {
.ok()
.and_then(|x| x.parse::<u16>().ok())
.unwrap_or(DEFAULT_TIMEOUT as u16);
static ref TIMEOUT_DURATION: Duration = Duration::from_secs(*TIMEOUT as u64);
static ref ZOMBIE_JOB_TIMEOUT: String = std::env::var("ZOMBIE_JOB_TIMEOUT")
.ok()
.and_then(|x| x.parse::<String>().ok())
.unwrap_or_else(|| "30".to_string());
pub static ref RESTART_ZOMBIE_JOBS: bool = std::env::var("RESTART_ZOMBIE_JOBS")
.ok()
.and_then(|x| x.parse::<bool>().ok())
.unwrap_or(true);
static ref ZOMBIE_JOB_TIMEOUT: String = (*TIMEOUT as u32 * 5).to_string();
static ref SESSION_TOKEN_EXPIRY: i32 = (*TIMEOUT as i32) * 2;
}
@@ -410,21 +401,6 @@ lazy_static::lazy_static! {
//only matter if CLOUD_HOSTED
const MAX_RESULT_SIZE: usize = 1024 * 1024 * 2; // 2MB
#[derive(Clone)]
pub struct AuthedClient {
pub base_internal_url: String,
pub workspace: String,
pub token: String,
pub client: OnceCell<Client>
}
impl AuthedClient {
pub fn get_client(&self) -> &Client {
return self.client.get_or_init(|| {
windmill_api_client::create_client(&self.base_internal_url, self.token.clone())
});
}
}
#[tracing::instrument(level = "trace")]
@@ -577,7 +553,7 @@ pub async fn run_worker(
let do_break = async {
if last_ping.elapsed().as_secs() > NUM_SECS_PING {
if last_ping.elapsed().as_secs() > NUM_SECS_ENV_CHECK {
sqlx::query!(
"UPDATE worker_ping SET ping_at = now(), jobs_executed = $1 WHERE worker = $2",
jobs_executed,
@@ -651,8 +627,7 @@ pub async fn run_worker(
.with_label_values(label_values.as_slice()),
};
let job_root = job.root_job.map(|x| x.to_string()).unwrap_or_else(|| "none".to_string());
tracing::info!(worker = %worker_name, id = %job.id, root_id = %job_root, "fetched job {}, root job: {}", job.id, job_root);
tracing::info!(worker = %worker_name, id = %job.id, "fetched job {}", job.id);
let job_dir = format!("{worker_dir}/{}", job.id);
@@ -695,13 +670,13 @@ pub async fn run_worker(
)
.await.expect("could not create job token");
tx.commit().await.expect("could not commit job token");
let authed_client = AuthedClient { base_internal_url: base_internal_url.to_string(), token: token.clone(), workspace: job.workspace_id.to_string(), client: OnceCell::new() };
let job_client = windmill_api_client::create_client(base_internal_url, token.clone());
let is_flow = job.job_kind == JobKind::Flow || job.job_kind == JobKind::FlowPreview || job.job_kind == JobKind::FlowDependencies;
if let Some(err) = handle_queued_job(
job.clone(),
db,
&authed_client,
&job_client,
token,
&worker_name,
&worker_dir,
@@ -715,7 +690,7 @@ pub async fn run_worker(
{
handle_job_error(
db,
&authed_client,
&job_client,
job,
err,
Some(metrics),
@@ -759,7 +734,7 @@ pub async fn run_worker(
async fn handle_job_error(
db: &Pool<Postgres>,
client: &AuthedClient,
client: &windmill_api_client::Client,
job: QueuedJob,
err: Error,
metrics: Option<Metrics>,
@@ -858,7 +833,7 @@ fn extract_error_value(log_lines: &str) -> serde_json::Value {
async fn handle_queued_job(
job: QueuedJob,
db: &sqlx::Pool<sqlx::Postgres>,
client: &AuthedClient,
client: &windmill_api_client::Client,
token: String,
worker_name: &str,
worker_dir: &str,
@@ -890,9 +865,6 @@ async fn handle_queued_job(
}
_ => {
let mut logs = "".to_string();
if let Some(log_str) = &job.logs {
logs.push_str(&log_str);
}
if job.is_flow_step {
update_flow_status_in_progress(
@@ -1041,14 +1013,14 @@ async fn write_file(dir: &str, path: &str, content: &str) -> error::Result<File>
#[async_recursion]
async fn transform_json_value(
name: &str,
client: &AuthedClient,
client: &windmill_api_client::Client,
workspace: &str,
v: Value,
) -> error::Result<Value> {
match v {
Value::String(y) if y.starts_with("$var:") => {
let path = y.strip_prefix("$var:").unwrap();
let v = client.get_client()
let v = client
.get_variable(workspace, path, Some(true))
.await
.map_err(|_| Error::NotFound(format!("Variable {path} not found for `{name}`")))
@@ -1064,7 +1036,7 @@ async fn transform_json_value(
"Argument `{name}` is an invalid resource path: {path}",
)));
}
let v = client.get_client()
let v = client
.get_resource_value(workspace, path)
.await
.map_err(|_| Error::NotFound(format!("Resource {path} not found for `{name}`")))?
@@ -1088,7 +1060,7 @@ async fn transform_json_value(
async fn handle_code_execution_job(
job: &QueuedJob,
db: &sqlx::Pool<sqlx::Postgres>,
client: &AuthedClient,
client: &windmill_api_client::Client,
token: String,
job_dir: &str,
worker_dir: &str,
@@ -1234,7 +1206,7 @@ async fn handle_go_job(
logs: &mut String,
job: &QueuedJob,
db: &sqlx::Pool<sqlx::Postgres>,
client: &AuthedClient,
client: &windmill_api_client::Client,
token: String,
inner_content: &str,
job_dir: &str,
@@ -1270,8 +1242,7 @@ async fn handle_go_job(
db,
true,
skip_go_mod,
worker_name,
&job.workspace_id,
worker_name
)
.await?;
@@ -1389,7 +1360,7 @@ func Run(req Req) (interface{{}}, error){{
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()?;
handle_child(&job.id, db, logs, build_go, false, worker_name, &job.workspace_id).await?;
handle_child(&job.id, db, logs, build_go, false, worker_name).await?;
Command::new(NSJAIL_PATH.as_str())
.current_dir(job_dir)
@@ -1415,7 +1386,7 @@ func Run(req Req) (interface{{}}, error){{
.stderr(Stdio::piped())
.spawn()?
};
handle_child(&job.id, db, logs, child, !*DISABLE_NSJAIL, worker_name, &job.workspace_id).await?;
handle_child(&job.id, db, logs, child, !*DISABLE_NSJAIL, worker_name).await?;
read_result(job_dir).await
}
@@ -1466,7 +1437,7 @@ async fn handle_bash_job(
.replace("{SHARED_MOUNT}", shared_mount),
)
.await?;
let mut cmd_args = vec!["--config", "run.config.proto", "--", "/bin/bash", "main.sh"];
let mut cmd_args = vec!["--config", "run.config.proto", "--", "/bin/sh", "main.sh"];
cmd_args.extend(args);
Command::new(NSJAIL_PATH.as_str())
.current_dir(job_dir)
@@ -1481,7 +1452,7 @@ async fn handle_bash_job(
} else {
let mut cmd_args = vec!["main.sh"];
cmd_args.extend(&args);
Command::new("/bin/bash")
Command::new("/bin/sh")
.current_dir(job_dir)
.env_clear()
.envs(reserved_variables)
@@ -1493,7 +1464,7 @@ async fn handle_bash_job(
.stderr(Stdio::piped())
.spawn()?
};
handle_child(&job.id, db, logs, child, !*DISABLE_NSJAIL, worker_name, &job.workspace_id).await?;
handle_child(&job.id, db, logs, child, !*DISABLE_NSJAIL, worker_name).await?;
//for now bash jobs have an empty result object
Ok(serde_json::json!(logs
.lines()
@@ -1510,32 +1481,12 @@ fn capitalize(s: &str) -> String {
}
}
fn get_common_deno_proc_envs(token: &str, base_internal_url: &str) -> HashMap<String, String> {
let hostname_base = BASE_URL.split("://").last().unwrap_or("localhost");
let hostname_internal = base_internal_url.split("://").last().unwrap_or("localhost");
let deno_auth_tokens_base = DENO_AUTH_TOKENS.as_str();
let deno_auth_tokens =
format!("{token}@{hostname_base};{token}@{hostname_internal}{deno_auth_tokens_base}",);
let mut deno_envs: HashMap<String, String> = HashMap::from([
(String::from("PATH"), PATH_ENV.clone()),
(String::from("DENO_AUTH_TOKENS"), deno_auth_tokens),
(String::from("BASE_INTERNAL_URL"), base_internal_url.to_string()),
(String::from("NO_COLOR"), String::from("true")),
]);
if let Some(ref s) = *NPM_CONFIG_REGISTRY {
deno_envs.insert(String::from("NPM_CONFIG_REGISTRY"), s.clone());
}
return deno_envs;
}
#[tracing::instrument(level = "trace", skip_all)]
async fn handle_deno_job(
logs: &mut String,
job: &QueuedJob,
db: &sqlx::Pool<sqlx::Postgres>,
client: &AuthedClient,
client: &windmill_api_client::Client,
token: String,
job_dir: &str,
inner_content: &String,
@@ -1556,14 +1507,14 @@ async fn handle_deno_job(
// TODO: Separately cache dependencies here using `deno cache --reload --lock=lock.json src/deps.ts` (https://deno.land/manual@v1.27.0/linking_to_external_code/integrity_checking)
// Then require caching below using --cached-only. This makes it so we require zero network interaction when running the process below
let _ = write_file(job_dir, "main.ts", inner_content).await?;
let _ = write_file(job_dir, "inner.ts", inner_content).await?;
let sig = trace_span!("parse_deno_signature")
.in_scope(|| windmill_parser_ts::parse_deno_signature(inner_content))?;
create_args_and_out_file(client, job, job_dir).await?;
let spread = sig.args.into_iter().map(|x| x.name).join(",");
let wrapper_content: String = format!(
r#"
import {{ main }} from "./main.ts";
import {{ main }} from "./inner.ts";
const args = await Deno.readTextFile("args.json")
.then(JSON.parse)
@@ -1581,35 +1532,25 @@ run().catch(async (e) => {{
}});
"#,
);
write_file(job_dir, "main.ts", &wrapper_content).await?;
let w_id = job.workspace_id.clone();
let script_path_split = job.script_path().split("/");
let script_path_parts_len = script_path_split.clone().count();
let mut relative_mounts = "".to_string();
for c in 0..script_path_parts_len {
relative_mounts += ",\n ";
relative_mounts += &format!("\"./{}\": \"{base_internal_url}/api/w/{w_id}/scripts/raw/p/{}{}\"",
(0..c).map(|_| "../").join(""),
&script_path_split.clone().take(script_path_parts_len - c - 1).join("/"),
if c == script_path_parts_len - 1 { "" } else { "/" },
);
}
write_file(job_dir, "wrapper.ts", &wrapper_content).await?;
let import_map = format!(
r#"{{
"imports": {{
"/": "{base_internal_url}/api/w/{w_id}/scripts/raw/p/",
"./wrapper.ts": "./wrapper.ts",
"./main.ts": "./main.ts"{relative_mounts}
"./": "./"
}}
}}"#,
}}"#
);
write_file(job_dir, "import_map.json", &import_map).await?;
let mut reserved_variables = get_reserved_variables(job, &token, db).await?;
reserved_variables.insert("RUST_LOG".to_string(), "info".to_string());
let common_deno_proc_envs = get_common_deno_proc_envs(&token, base_internal_url);
//do not cache local dependencies
let reload = format!("--reload={base_internal_url}");
let hostname_base = BASE_URL.split("://").last().unwrap_or("localhost");
let hostname_internal = base_internal_url.split("://").last().unwrap_or("localhost");
let deno_auth_tokens_base = DENO_AUTH_TOKENS.as_str();
let deno_auth_tokens =
format!("{token}@{hostname_base};{token}@{hostname_internal}{deno_auth_tokens_base}",);
let child = async {
Ok(if !*DISABLE_NSJAIL {
let _ = write_file(
@@ -1633,7 +1574,6 @@ run().catch(async (e) => {{
}
args.push("--import-map");
args.push("/tmp/import_map.json");
args.push(&reload);
args.push("--unstable");
if let Some(deno_flags) = DENO_FLAGS.as_ref() {
for flag in deno_flags {
@@ -1642,25 +1582,26 @@ run().catch(async (e) => {{
} else {
args.push("-A");
}
args.push("/tmp/wrapper.ts");
args.push("/tmp/main.ts");
Command::new(NSJAIL_PATH.as_str())
.current_dir(job_dir)
.env_clear()
.envs(reserved_variables)
.envs(common_deno_proc_envs)
.env("PATH", PATH_ENV.as_str())
.env("DENO_AUTH_TOKENS", deno_auth_tokens)
.env("BASE_INTERNAL_URL", base_internal_url)
.args(args)
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()?
} else {
let mut args = Vec::new();
let script_path = format!("{job_dir}/wrapper.ts");
let script_path = format!("{job_dir}/main.ts");
let import_map_path = format!("{job_dir}/import_map.json");
args.push("run");
args.push("--import-map");
args.push(&import_map_path);
args.push(&reload);
args.push("--unstable");
if let Some(deno_flags) = DENO_FLAGS.as_ref() {
for flag in deno_flags {
@@ -1674,8 +1615,11 @@ run().catch(async (e) => {{
.current_dir(job_dir)
.env_clear()
.envs(reserved_variables)
.envs(common_deno_proc_envs)
.env("PATH", PATH_ENV.as_str())
.env("DENO_DIR", DENO_CACHE_DIR)
.env("DENO_AUTH_TOKENS", deno_auth_tokens)
.env("BASE_INTERNAL_URL", base_internal_url)
.env("NO_COLOR", "true")
.args(args)
.stdout(Stdio::piped())
.stderr(Stdio::piped())
@@ -1684,13 +1628,13 @@ run().catch(async (e) => {{
}
.instrument(trace_span!("create_deno_jail"))
.await?;
handle_child(&job.id, db, logs, child, !*DISABLE_NSJAIL, worker_name, &job.workspace_id).await?;
handle_child(&job.id, db, logs, child, !*DISABLE_NSJAIL, worker_name).await?;
read_result(job_dir).await
}
#[tracing::instrument(level = "trace", skip_all)]
async fn create_args_and_out_file(
client: &AuthedClient,
client: &windmill_api_client::Client,
job: &QueuedJob,
job_dir: &str,
) -> Result<(), Error> {
@@ -1706,7 +1650,7 @@ async fn create_args_and_out_file(
}
lazy_static! {
static ref RELATIVE_IMPORT_REGEX: Regex = Regex::new(r#"(import|from)\s(((u|f)\.)|\.)"#).unwrap();
static ref RELATIVE_IMPORT_REGEX: Regex = Regex::new(r#"(import|from)\s(u|f)\."#).unwrap();
}
#[tracing::instrument(level = "trace", skip_all)]
@@ -1718,11 +1662,11 @@ async fn handle_python_job(
job: &QueuedJob,
logs: &mut String,
db: &sqlx::Pool<sqlx::Postgres>,
client: &AuthedClient,
client: &windmill_api_client::Client,
token: String,
inner_content: &String,
shared_mount: &str,
base_internal_url: &str
base_internal_url: &str,
) -> error::Result<serde_json::Value> {
create_dependencies_dir(job_dir).await;
@@ -1736,7 +1680,7 @@ async fn handle_python_job(
if requirements.is_empty() {
"".to_string()
} else {
pip_compile(&job.id, &requirements, logs, job_dir, db, worker_name, &job.workspace_id)
pip_compile(&job.id, &requirements, logs, job_dir, db, worker_name)
.await
.map_err(|e| {
Error::ExecutionErr(format!("pip compile failed: {}", e.to_string()))
@@ -1777,14 +1721,9 @@ async fn handle_python_job(
let relative_imports = RELATIVE_IMPORT_REGEX.is_match(&inner_content);
let script_path_splitted = &job.script_path().split("/");
let dirs = script_path_splitted.clone().take(script_path_splitted.clone().count() - 1).join("/").replace("-", "_");
let last = script_path_splitted.clone().last().unwrap().replace("-", "_");
let module_dir = format!("{}/{}", job_dir, dirs );
tokio::fs::create_dir_all(format!("{module_dir}/")).await?;
let _ = write_file(&module_dir, &format!("{last}.py"), inner_content).await?;
let _ = write_file(job_dir, "inner.py", inner_content).await?;
if relative_imports {
let _ = write_file(&job_dir, "loader.py", RELATIVE_PYTHON_LOADER).await?;
let _ = write_file(job_dir, "loader.py", RELATIVE_PYTHON_LOADER).await?;
}
let sig = windmill_parser_py::parse_python_signature(inner_content)?;
@@ -1845,7 +1784,6 @@ async fn handle_python_job(
.join("\n")
};
let module_dir_dot = dirs.replace("/", ".").replace("-", "_");
let wrapper_content: String = format!(
r#"
import json
@@ -1854,8 +1792,8 @@ import json
{import_datetime}
import traceback
import sys
from {module_dir_dot} import {last} as inner_script
inner_script = __import__("inner")
with open("args.json") as f:
kwargs = json.load(f, strict=False)
@@ -1882,7 +1820,7 @@ except Exception as e:
sys.exit(1)
"#,
);
write_file(job_dir, "wrapper.py", &wrapper_content).await?;
write_file(job_dir, "main.py", &wrapper_content).await?;
let mut reserved_variables = get_reserved_variables(job, &token, db).await?;
let additional_python_paths_folders = additional_python_paths.iter().join(":");
@@ -1910,7 +1848,6 @@ mount {{
.replace("{CLONE_NEWUSER}", &(!*DISABLE_NUSER).to_string())
.replace("{SHARED_MOUNT}", shared_mount)
.replace("{SHARED_DEPENDENCIES}", shared_deps.as_str())
.replace("{MAIN}", format!("{dirs}/{last}").as_str())
.replace(
"{ADDITIONAL_PYTHON_PATHS}",
additional_python_paths_folders.as_str(),
@@ -1942,8 +1879,7 @@ mount {{
"--",
PYTHON_PATH.as_str(),
"-u",
"-m",
"wrapper",
"/tmp/main.py",
])
.stdout(Stdio::piped())
.stderr(Stdio::piped())
@@ -1955,13 +1891,13 @@ mount {{
.envs(reserved_variables)
.env("PATH", PATH_ENV.as_str())
.env("BASE_INTERNAL_URL", base_internal_url)
.args(vec!["-u", "-m", "wrapper"])
.args(vec!["-u", "main.py"])
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()?
};
handle_child(&job.id, db, logs, child, !*DISABLE_NSJAIL, worker_name, &job.workspace_id).await?;
handle_child(&job.id, db, logs, child, !*DISABLE_NSJAIL, worker_name).await?;
read_result(job_dir).await
}
@@ -2007,8 +1943,7 @@ async fn handle_dependency_job(
logs,
job_dir,
db,
worker_name,
&job.workspace_id,
worker_name
)
.await;
match content {
@@ -2074,8 +2009,7 @@ async fn handle_flow_dependency_job(
logs,
job_dir,
db,
worker_name,
&job.workspace_id,
worker_name
)
.await;
match new_lock {
@@ -2191,13 +2125,12 @@ async fn capture_dependency_job(
logs: &mut String,
job_dir: &str,
db: &sqlx::Pool<sqlx::Postgres>,
worker_name: &str,
w_id: &str,
worker_name: &str
) -> error::Result<String> {
match job_language {
ScriptLang::Python3 => {
create_dependencies_dir(job_dir).await;
pip_compile(job_id, job_raw_code, logs, job_dir, db, worker_name, w_id).await
pip_compile(job_id, job_raw_code, logs, job_dir, db, worker_name).await
}
ScriptLang::Go => {
install_go_dependencies(
@@ -2208,8 +2141,7 @@ async fn capture_dependency_job(
db,
false,
false,
worker_name,
w_id
worker_name
)
.await
}
@@ -2227,8 +2159,7 @@ async fn pip_compile(
logs: &mut String,
job_dir: &str,
db: &Pool<Postgres>,
worker_name: &str,
w_id: &str,
worker_name: &str
) -> error::Result<String> {
logs.push_str(&format!("\nresolving dependencies..."));
set_logs(logs, job_id, db).await;
@@ -2261,7 +2192,7 @@ async fn pip_compile(
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()?;
handle_child(job_id, db, logs, child, false, worker_name, &w_id)
handle_child(job_id, db, logs, child, false, worker_name)
.await
.map_err(|e| Error::ExecutionErr(format!("Lock file generation failed: {e:?}")))?;
let path_lock = format!("{job_dir}/requirements.txt");
@@ -2284,8 +2215,7 @@ async fn install_go_dependencies(
db: &sqlx::Pool<sqlx::Postgres>,
preview: bool,
skip_go_mod: bool,
worker_name: &str,
w_id: &str
worker_name: &str
) -> error::Result<String> {
if !skip_go_mod {
gen_go_mymod(code, job_dir).await?;
@@ -2296,7 +2226,7 @@ async fn install_go_dependencies(
.stderr(Stdio::piped())
.spawn()?;
handle_child(job_id, db, logs, child, false, worker_name, w_id).await?;
handle_child(job_id, db, logs, child, false, worker_name).await?;
}
let child = Command::new(GO_PATH.as_str())
.current_dir(job_dir)
@@ -2304,7 +2234,7 @@ async fn install_go_dependencies(
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()?;
handle_child(job_id, db, logs, child, false, worker_name, &w_id)
handle_child(job_id, db, logs, child, false, worker_name)
.await
.map_err(|e| Error::ExecutionErr(format!("Lock file generation failed: {e:?}")))?;
@@ -2411,7 +2341,6 @@ async fn handle_child(
mut child: Child,
nsjail: bool,
worker_name: &str,
w_id: &str,
) -> error::Result<()> {
let update_job_interval = Duration::from_millis(500);
let write_logs_delay = Duration::from_millis(500);
@@ -2487,29 +2416,11 @@ async fn handle_child(
let wait_on_child = async {
let db = db.clone();
#[cfg(not(feature = "enterprise"))]
let timeout_duration = *TIMEOUT_DURATION;
#[cfg(feature = "enterprise")]
let premium_workspace = sqlx::query_scalar!("SELECT premium FROM workspace WHERE id = $1", w_id)
.fetch_one(&db)
.await
.map_err(|e| {
tracing::error!(%e, "error getting premium workspace for job {job_id}: {e}");
}).unwrap_or(false);
#[cfg(feature = "enterprise")]
let timeout_duration = if premium_workspace {
*TIMEOUT_DURATION*6 //30mins
} else {
*TIMEOUT_DURATION
};
let kill_reason = tokio::select! {
biased;
result = child.wait() => return result.map(Ok),
Ok(()) = too_many_logs.changed() => KillReason::TooManyLogs,
_ = sleep(timeout_duration) => KillReason::Timeout,
_ = sleep(*TIMEOUT_DURATION) => KillReason::Timeout,
_ = update_job => KillReason::Cancelled,
};
tx.send(()).expect("rx should never be dropped");
@@ -2768,7 +2679,7 @@ pub async fn handle_zombie_jobs_periodically(
handle_zombie_jobs(db, base_internal_url).await;
tokio::select! {
_ = tokio::time::sleep(Duration::from_secs(30)) => (),
_ = tokio::time::sleep(Duration::from_secs(60)) => (),
_ = rx.recv() => {
println!("received killpill for monitor job");
break;
@@ -2778,35 +2689,28 @@ pub async fn handle_zombie_jobs_periodically(
}
async fn handle_zombie_jobs(db: &Pool<Postgres>, base_internal_url: &str) {
if *RESTART_ZOMBIE_JOBS {
let restarted = sqlx::query!(
"UPDATE queue SET running = false, started_at = null, logs = logs || '\nRestarted job after not receiving job''s ping for too long the ' || now() || '\n\n' WHERE last_ping < now() - ($1 || ' seconds')::interval AND running = true AND job_kind != $2 AND job_kind != $3 AND same_worker = false RETURNING id, workspace_id, last_ping",
*ZOMBIE_JOB_TIMEOUT,
JobKind::Flow: JobKind,
JobKind::FlowPreview: JobKind,
)
.fetch_all(db)
.await
.ok()
.unwrap_or_else(|| vec![]);
let restarted = sqlx::query!(
"UPDATE queue SET running = false WHERE last_ping < now() - ($1 || ' seconds')::interval AND running = true AND job_kind != $2 AND same_worker = false RETURNING id, workspace_id, last_ping",
*ZOMBIE_JOB_TIMEOUT,
JobKind::Flow: JobKind,
)
.fetch_all(db)
.await
.ok()
.unwrap_or_else(|| vec![]);
QUEUE_ZOMBIE_RESTART_COUNT.inc_by(restarted.len() as _);
for r in restarted {
tracing::info!(
"restarted zombie job {} {} {}",
r.id,
r.workspace_id,
r.last_ping
);
}
QUEUE_ZOMBIE_RESTART_COUNT.inc_by(restarted.len() as _);
for r in restarted {
tracing::info!(
"restarted zombie job {} {} {}",
r.id,
r.workspace_id,
r.last_ping
);
}
let mut timeout_query = "SELECT * FROM queue WHERE last_ping < now() - ($1 || ' seconds')::interval AND running = true AND job_kind != $2".to_string();
if *RESTART_ZOMBIE_JOBS {
timeout_query.push_str(" AND same_worker = true");
};
let timeouts = sqlx::query_as::<_, QueuedJob>(
&timeout_query
"SELECT * FROM queue WHERE last_ping < now() - ($1 || ' seconds')::interval AND running = true AND job_kind != $2 AND same_worker = true",
)
.bind(ZOMBIE_JOB_TIMEOUT.as_str())
.bind(JobKind::Flow)
@@ -2818,7 +2722,7 @@ async fn handle_zombie_jobs(db: &Pool<Postgres>, base_internal_url: &str) {
QUEUE_ZOMBIE_DELETE_COUNT.inc_by(timeouts.len() as _);
for job in timeouts {
tracing::info!(
"timedout zombie job {} {}",
"timedouts zombie same_worker job {} {}",
job.id,
job.workspace_id,
);
@@ -2838,15 +2742,13 @@ async fn handle_zombie_jobs(db: &Pool<Postgres>, base_internal_url: &str) {
.await
.expect("could not create job token");
tx.commit().await.expect("could not commit job token");
let client = AuthedClient { base_internal_url: base_internal_url.to_string(), token: token.clone(), workspace: job.workspace_id.to_string(), client: OnceCell::new() };
let client = windmill_api_client::create_client(base_internal_url, token.clone());
let last_ping = job.last_ping.clone();
let _ = handle_job_error(
db,
&client,
job,
error::Error::ExecutionErr(format!("Job timed out after no ping from job since {} (ZOMBIE_JOB_TIMEOUT: {})",
last_ping.map(|x| x.to_string()).unwrap_or_else(|| "no ping".to_string()), *ZOMBIE_JOB_TIMEOUT)),
error::Error::ExecutionErr("Same worker job timed out".to_string()),
None,
true,
same_worker_tx_never_used,
@@ -2949,7 +2851,7 @@ async fn handle_python_reqs(
.spawn()?
};
let child = handle_child(&job.id, db, logs, child, false, worker_name, &job.workspace_id).await;
let child = handle_child(&job.id, db, logs, child, false, worker_name).await;
tracing::info!(
worker_name = %worker_name,
job_id = %job.id,

View File

@@ -11,8 +11,8 @@ use std::iter;
use std::time::Duration;
use crate::jobs::{add_completed_job, add_completed_job_error, schedule_again_if_scheduled};
use crate::js_eval::{eval_timeout, IdContext};
use crate::{worker, AuthedClient, KEEP_JOB_DIR};
use crate::js_eval::{eval_timeout, EvalCreds, IdContext};
use crate::{worker, KEEP_JOB_DIR};
use anyhow::Context;
use async_recursion::async_recursion;
use dyn_iter::DynIter;
@@ -40,7 +40,7 @@ use windmill_queue::{
// #[instrument(level = "trace", skip_all)]
pub async fn update_flow_status_after_job_completion(
db: &DB,
client: &AuthedClient,
client: &windmill_api_client::Client,
flow: uuid::Uuid,
job_id_for_status: &Uuid,
w_id: &str,
@@ -120,7 +120,8 @@ pub async fn update_flow_status_after_job_completion(
let stop_early = success
&& if let Some(expr) = r.stop_early_expr.clone() {
compute_bool_from_expr(expr, &r.args, result.clone(), None, Some(client)).await?
compute_bool_from_expr(expr, &r.args, result.clone(), None, None, base_internal_url)
.await?
} else {
false
};
@@ -395,12 +396,6 @@ pub async fn update_flow_status_after_job_completion(
.await?
.ok_or_else(|| Error::InternalErr(format!("requiring flow to be in the queue")))?;
let job_root = flow_job
.root_job
.map(|x| x.to_string())
.unwrap_or_else(|| "none".to_string());
tracing::info!(id = %flow_job.id, root_id = %job_root, "update flow status");
let raw_flow = flow_job.parse_raw_flow();
let module = raw_flow.as_ref().and_then(|module| {
module_index.and_then(|i| module.modules.get(i).or(module.failure_module.as_ref()))
@@ -603,7 +598,8 @@ async fn compute_bool_from_expr(
flow_args: &Option<serde_json::Value>,
result: serde_json::Value,
by_id: Option<IdContext>,
client: Option<&AuthedClient>,
creds: Option<EvalCreds>,
base_internal_url: &str,
) -> error::Result<bool> {
let flow_input = flow_args.clone().unwrap_or_else(|| json!({}));
match eval_timeout(
@@ -614,8 +610,9 @@ async fn compute_bool_from_expr(
("previous_result".to_string(), result),
]
.into(),
client,
creds,
by_id,
base_internal_url,
)
.await?
{
@@ -690,10 +687,12 @@ async fn transform_input(
flow_args: &Option<serde_json::Value>,
last_result: serde_json::Value,
input_transforms: &HashMap<String, InputTransform>,
workspace: &str,
token: &str,
resumes: &[Value],
approvers: Vec<String>,
by_id: &IdContext,
client: &AuthedClient,
base_internal_url: &str,
) -> windmill_common::error::Result<Map<String, serde_json::Value>> {
let mut mapped = serde_json::Map::new();
@@ -732,13 +731,19 @@ async fn transform_input(
context.push(("error".to_string(), error.unwrap().clone()));
}
let v = eval_timeout(expr.to_string(), context, Some(client), Some(by_id.clone()))
.await
.map_err(|e| {
Error::ExecutionErr(format!(
"Error during isolated evaluation of expression `{expr}`:\n{e}"
))
})?;
let v = eval_timeout(
expr.to_string(),
context,
Some(EvalCreds { workspace: workspace.to_string(), token: token.to_string() }),
Some(by_id.clone()),
base_internal_url,
)
.await
.map_err(|e| {
Error::ExecutionErr(format!(
"Error during isolated evaluation of expression `{expr}`:\n{e}"
))
})?;
mapped.insert(key.to_string(), v);
()
}
@@ -752,7 +757,7 @@ async fn transform_input(
pub async fn handle_flow(
flow_job: &QueuedJob,
db: &sqlx::Pool<sqlx::Postgres>,
client: &AuthedClient,
client: &windmill_api_client::Client,
last_result: serde_json::Value,
same_worker_tx: Sender<Uuid>,
worker_dir: &str,
@@ -792,18 +797,12 @@ async fn push_next_flow_job(
mut status: FlowStatus,
flow: FlowValue,
db: &sqlx::Pool<sqlx::Postgres>,
client: &AuthedClient,
client: &windmill_api_client::Client,
mut last_result: serde_json::Value,
same_worker_tx: Sender<Uuid>,
worker_dir: &str,
base_internal_url: &str,
) -> error::Result<()> {
let job_root = flow_job
.root_job
.map(|x| x.to_string())
.unwrap_or_else(|| "none".to_string());
tracing::info!(id = %flow_job.id, root_id = %job_root, "pushing next flow job");
let mut i = usize::try_from(status.step)
.with_context(|| format!("invalid module index {}", status.step))?;
@@ -872,6 +871,7 @@ async fn push_next_flow_job(
.into(),
None,
None,
base_internal_url,
)
.await
.map_err(|e| {
@@ -1114,15 +1114,15 @@ async fn push_next_flow_job(
_ => (),
}
let mut transform_context: Option<IdContext> = None;
let mut transform_context: Option<TransformContext> = None;
let args: windmill_common::error::Result<_> = match &module.value {
FlowModuleValue::Script { input_transforms, .. }
| FlowModuleValue::RawScript { input_transforms, .. }
| FlowModuleValue::Flow { input_transforms, .. } => {
let ctx = get_transform_context(&flow_job, previous_id.clone(), &status).await?;
let ctx = get_transform_context(db, &flow_job, previous_id.clone(), &status).await?;
transform_context = Some(ctx);
let by_id = transform_context.as_ref().unwrap();
let (token, by_id) = transform_context.as_ref().unwrap();
transform_input(
&flow_job.args,
last_result.clone(),
@@ -1131,10 +1131,12 @@ async fn push_next_flow_job(
} else {
&module.input_transforms
},
&flow_job.workspace_id,
&token,
resume_messages.as_slice(),
approvers,
by_id,
client,
base_internal_url,
)
.await
}
@@ -1167,17 +1169,17 @@ async fn push_next_flow_job(
flow_job,
&flow,
transform_context,
db,
tx,
&module,
&status,
&status_module,
last_result.clone(),
previous_id,
client,
base_internal_url,
)
.await?;
tx.commit().await?;
tracing::info!(id = %flow_job.id, root_id = %job_root, "next flow transform computed");
let (job_payloads, next_status) = match next_flow_transform {
NextFlowTransform::Continue(job_payload, next_state) => (job_payload, next_state),
@@ -1374,7 +1376,6 @@ async fn push_next_flow_job(
};
tx.commit().await?;
tracing::info!(id = %flow_job.id, root_id = %job_root, "all next flow jobs pushed");
if continue_on_same_worker {
if !is_one_uuid {
@@ -1506,17 +1507,20 @@ async fn script_path_to_payload<'c>(
Ok(job_payload)
}
type TransformContext = (String, IdContext);
async fn compute_next_flow_transform<'c>(
flow_job: &QueuedJob,
flow: &FlowValue,
by_id: Option<IdContext>,
transform_context: Option<TransformContext>,
db: &DB,
mut tx: sqlx::Transaction<'c, sqlx::Postgres>,
module: &FlowModule,
status: &FlowStatus,
status_module: &FlowStatusModule,
last_result: serde_json::Value,
previous_id: String,
client: &AuthedClient,
base_internal_url: &str,
) -> error::Result<(sqlx::Transaction<'c, sqlx::Postgres>, NextFlowTransform)> {
match &module.value {
FlowModuleValue::Identity => Ok((
@@ -1547,7 +1551,7 @@ async fn compute_next_flow_transform<'c>(
FlowModuleValue::RawScript { path, content, language, lock, .. } => {
let path = path
.clone()
.or_else(|| Some(format!("{}/step-{}", flow_job.script_path(), status.step)));
.or_else(|| Some(format!("{}/{}", flow_job.script_path(), status.step)));
Ok((
tx,
NextFlowTransform::Continue(
@@ -1569,10 +1573,10 @@ async fn compute_next_flow_transform<'c>(
FlowStatusModule::WaitingForPriorSteps { .. }
| FlowStatusModule::WaitingForEvents { .. }
| FlowStatusModule::WaitingForExecutor { .. } => {
let by_id = if let Some(x) = by_id {
let (token, by_id) = if let Some(x) = transform_context {
x
} else {
get_transform_context(&flow_job, previous_id, &status).await?
get_transform_context(db, &flow_job, previous_id, &status).await?
};
let flow_input = flow_job.args.clone().unwrap_or_else(|| json!({}));
/* Iterator is an InputTransform, evaluate it into an array. */
@@ -1585,8 +1589,10 @@ async fn compute_next_flow_transform<'c>(
("previous_result".to_string(), last_result.clone()),
]
},
Some(client),
token,
flow_job.workspace_id.clone(),
Some(by_id),
base_internal_url,
)
.await?
.into_array()
@@ -1703,14 +1709,19 @@ async fn compute_next_flow_transform<'c>(
| FlowStatusModule::WaitingForEvents { .. }
| FlowStatusModule::WaitingForExecutor { .. } => {
let mut branch_chosen = BranchChosen::Default;
let idcontext = get_transform_context(&flow_job, previous_id, &status).await?;
let (token, idcontext) =
get_transform_context(db, &flow_job, previous_id, &status).await?;
for (i, b) in branches.iter().enumerate() {
let pred = compute_bool_from_expr(
b.expr.to_string(),
&flow_job.args,
last_result.clone(),
Some(idcontext.clone()),
Some(client),
Some(EvalCreds {
workspace: flow_job.workspace_id.clone(),
token: token.to_string(),
}),
base_internal_url,
)
.await?;
@@ -1875,31 +1886,59 @@ async fn compute_next_flow_transform<'c>(
}
async fn get_transform_context(
db: &DB,
flow_job: &QueuedJob,
previous_id: String,
status: &FlowStatus,
) -> error::Result<IdContext> {
) -> error::Result<TransformContext> {
let tx = db.begin().await?;
let (tx, new_token) = crate::create_token_for_owner(
tx,
&flow_job.workspace_id,
&flow_job.permissioned_as,
"transform-input",
10,
&flow_job.email,
)
.await?;
//we need to commit asap otherwise the token won't be valid for auth to check outside of this transaction
//which will happen with client http calls
tx.commit().await?;
let steps_results: HashMap<String, JobResult> = status
.modules
.iter()
.filter_map(|x| x.job_result().map(|y| (x.id(), y)))
.collect();
Ok(IdContext { flow_job: flow_job.id, steps_results, previous_id })
Ok((
new_token,
IdContext { flow_job: flow_job.id, steps_results, previous_id },
))
}
async fn evaluate_with<F>(
transform: InputTransform,
vars: F,
client: Option<&AuthedClient>,
token: String,
workspace: String,
by_id: Option<IdContext>,
base_internal_url: &str,
) -> anyhow::Result<serde_json::Value>
where
F: FnOnce() -> Vec<(String, serde_json::Value)>,
{
match transform {
InputTransform::Static { value } => Ok(value),
InputTransform::Javascript { expr } => eval_timeout(expr, vars(), client, by_id).await,
InputTransform::Javascript { expr } => {
eval_timeout(
expr,
vars(),
Some(EvalCreds { workspace, token }),
by_id,
base_internal_url,
)
.await
}
}
}
trait IntoArray: Sized {

View File

@@ -1,22 +1,12 @@
import { requireLogin, resolveWorkspace, validatePath } from "./context.ts";
import { Any, decoverto, model, property } from "./decoverto.ts";
import { Any, model, property } from "./decoverto.ts";
import {
AppService,
AppWithLastVersion,
colors,
Command,
ListableApp,
microdiff,
Policy,
Table,
} from "./deps.ts";
import {
Difference,
GlobalOptions,
PushDiffs,
Resource,
setValueByPath,
} from "./types.ts";
import { Difference, PushDiffs, Resource, setValueByPath } from "./types.ts";
@model()
export class AppFile implements Resource, PushDiffs {
@@ -27,6 +17,7 @@ export class AppFile implements Resource, PushDiffs {
@property(Any)
policy: Policy;
constructor(value: string, summary: string, policy: Policy) {
this.value = value;
this.summary = summary;
@@ -35,20 +26,16 @@ export class AppFile implements Resource, PushDiffs {
async pushDiffs(
workspace: string,
remotePath: string,
diffs: Difference[]
diffs: Difference[],
): Promise<void> {
let app: AppWithLastVersion | undefined = undefined;
try {
app = await AppService.getAppByPath({ workspace, path: remotePath });
} catch (e) {}
if (app) {
if (await AppService.existsApp({ workspace, path: remotePath })) {
console.log(
colors.bold.yellow(
`Applying ${diffs.length} diffs to existing app... ${remotePath}`
)
`Applying ${diffs.length} diffs to existing app...`,
),
);
const changeset: {
path?: string | undefined;
summary?: string | undefined;
value?: any;
policy?: Policy | undefined;
@@ -56,10 +43,14 @@ export class AppFile implements Resource, PushDiffs {
for (const diff of diffs) {
if (
diff.type !== "REMOVE" &&
diff.path[0] !== "value" &&
diff.path[0] !== "policy" &&
(diff.path.length !== 1 ||
!["summary"].includes(diff.path[0] as string))
(
diff.path[0] !== "value" && diff.path[0] !== "policy" && (
diff.path.length !== 1 ||
!["path", "summary"].includes(
diff.path[0] as string,
)
)
)
) {
throw new Error("Invalid app diff with path " + diff.path);
}
@@ -70,21 +61,8 @@ export class AppFile implements Resource, PushDiffs {
}
}
if (
(!changeset?.policy ||
JSON.stringify(changeset?.policy) == JSON.stringify(app.policy)) &&
(!changeset?.value ||
JSON.stringify(changeset?.value) == JSON.stringify(app.value)) &&
(!changeset?.summary || changeset.summary == app.summary)
) {
console.log(
colors.yellow(`No changes to push for app ${remotePath}, skipping`)
);
return;
}
const hasChanges = Object.values(changeset).some(
(v) => v !== null && typeof v !== "undefined"
const hasChanges = Object.values(changeset).some((v) =>
v !== null && typeof v !== "undefined"
);
if (!hasChanges) {
return;
@@ -97,7 +75,6 @@ export class AppFile implements Resource, PushDiffs {
});
} else {
console.log(colors.yellow.bold("Creating new app..."));
await AppService.createApp({
workspace,
requestBody: {
@@ -110,70 +87,19 @@ export class AppFile implements Resource, PushDiffs {
}
}
async push(workspace: string, remotePath: string): Promise<void> {
let existing: AppWithLastVersion | undefined;
try {
existing = await AppService.getAppByPath({
workspace: workspace,
path: remotePath,
});
} catch {
existing = undefined;
}
await this.pushDiffs(
workspace,
remotePath,
microdiff({}, this, { cyclesFix: false })
microdiff(existing ?? {}, this, { cyclesFix: false }),
);
}
}
async function list(opts: GlobalOptions) {
const workspace = await resolveWorkspace(opts);
await requireLogin(opts);
let page = 0;
const perPage = 10;
const total: ListableApp[] = [];
while (true) {
const res = await AppService.listApps({
workspace: workspace.workspaceId,
page,
perPage,
});
page += 1;
total.push(...res);
if (res.length < perPage) {
break;
}
}
new Table()
.header(["path", "summary"])
.padding(2)
.border(true)
.body(total.map((x) => [x.path, x.summary]))
.render();
}
async function push(opts: GlobalOptions, filePath: string) {
const remotePath = filePath.split(".")[0];
if (!validatePath(remotePath)) {
return;
}
const workspace = await resolveWorkspace(opts);
await requireLogin(opts);
await pushApp(filePath, workspace.workspaceId, remotePath);
console.log(colors.bold.underline.green("App pushed"));
}
export async function pushApp(
filePath: string,
workspace: string,
remotePath: string
) {
const data = decoverto
.type(AppFile)
.rawToInstance(await Deno.readTextFile(filePath));
await data.push(workspace, remotePath);
}
const command = new Command()
.description("app related commands")
.action(list as any)
.command("push", "push a local app ")
.arguments("<file_path:file>")
.action(push as any);
export default command;

View File

@@ -18,7 +18,7 @@ export type Context = {
};
async function tryResolveWorkspace(
opts: GlobalOptions
opts: GlobalOptions,
): Promise<
{ isError: false; value: Workspace } | { isError: true; error: string }
> {
@@ -49,20 +49,17 @@ async function tryResolveWorkspace(
}
export async function resolveWorkspace(
opts: GlobalOptions
opts: GlobalOptions,
): Promise<Workspace> {
const res = await tryResolveWorkspace(opts);
if (res.isError) {
console.log(colors.red.bold(res.error));
return Deno.exit(-1);
} else {
return res.value;
}
}
export async function requireLogin(
opts: GlobalOptions
): Promise<GlobalUserInfo> {
export async function requireLogin(opts: GlobalOptions): Promise<GlobalUserInfo> {
const workspace = await resolveWorkspace(opts);
let token = await tryGetLoginInfo(opts);
@@ -76,7 +73,7 @@ export async function requireLogin(
return await UserService.globalWhoami();
} catch {
console.log(
"! Could not reach API given existing credentials. Attempting to reauth..."
"! Could not reach API given existing credentials. Attempting to reauth...",
);
const newToken = await loginInteractive(workspace.remote);
if (!newToken) {
@@ -88,14 +85,14 @@ export async function requireLogin(
setClient(
token,
workspace.remote.substring(0, workspace.remote.length - 1)
workspace.remote.substring(0, workspace.remote.length - 1),
);
return await UserService.globalWhoami();
}
}
export async function tryResolveVersion(
opts: GlobalOptions
opts: GlobalOptions,
): Promise<number | undefined> {
if ((opts as any).__cache_version) {
return (opts as any).__cache_version;
@@ -105,24 +102,41 @@ export async function tryResolveVersion(
if (workspaceRes.isError) return undefined;
const response = await fetch(
new URL(new URL(workspaceRes.value.remote).origin + "/api/version")
new URL(new URL(workspaceRes.value.remote).origin + "/api/version"),
);
const version = await response.text();
try {
return Number.parseInt(
version.split("-", 1)[0].replaceAll(".", "").replace("v", "")
version.split("-", 1)[0].replaceAll(".", "").replace("v", ""),
);
} catch {
return undefined;
}
}
export function validatePath(path: string): boolean {
if (!(path.startsWith("g") || path.startsWith("u") || path.startsWith("f"))) {
export async function validatePath(
opts: GlobalOptions,
path: string,
): Promise<boolean> {
const backendVersion = await tryResolveVersion(opts);
if (path.startsWith("f")) {
if (!backendVersion || backendVersion >= 1550) {
return true;
}
console.log(
`Attempting to use folders, but the current remote does not have support. Remote version is ${backendVersion} but folders are supported from 1560.`,
);
return false;
}
if (
!(path.startsWith("g") ||
path.startsWith("u"))
) {
console.log(
colors.red(
"Given remote path looks invalid. Remote paths are typically of the form <u|g|f>/<username|group|folder>/..."
)
"Given remote path looks invalid. Remote paths are typically of the form <u|g|f>/<username|group|folder>/...",
),
);
return false;
}

View File

@@ -33,7 +33,9 @@ export { passwordGenerator } from "https://deno.land/x/password_generator@latest
export { nanoid } from "https://deno.land/x/nanoid@v3.0.0/mod.ts";
export * as cbor from "https://deno.land/x/cbor@v1.4.1/index.js";
export { default as Murmurhash3 } from "https://deno.land/x/murmurhash@v1.0.0/mod.ts";
export { default as microdiff } from "https://deno.land/x/microdiff@v1.3.1/index.ts";
export {
default as microdiff,
} from "https://deno.land/x/microdiff@v1.3.1/index.ts";
export { default as objectHash } from "https://deno.land/x/object_hash@2.0.3.1/mod.ts";
export { default as gitignore_parser } from "npm:gitignore-parser";
export { default as JSZip } from "npm:jszip@3.7.1";

View File

@@ -20,6 +20,7 @@ import { requireLogin, resolveWorkspace, validatePath } from "./context.ts";
import { resolve, track_job } from "./script.ts";
import { Any, decoverto, model, property } from "./decoverto.ts";
// this is effectively "OpenFlow" but a copy as it is accepted by the CLI
@model()
export class FlowFile implements Resource, PushDiffs {
@@ -39,7 +40,7 @@ export class FlowFile implements Resource, PushDiffs {
async pushDiffs(
workspace: string,
remotePath: string,
diffs: Difference[]
diffs: Difference[],
): Promise<void> {
if (
await FlowService.existsFlowByPath({
@@ -49,8 +50,8 @@ export class FlowFile implements Resource, PushDiffs {
) {
console.log(
colors.bold.yellow(
`Applying ${diffs.length} diffs to existing flow... ${remotePath}`
)
`Applying ${diffs.length} diffs to existing flow... ${remotePath}`,
),
);
// TODO: Make these optional in backend (not path ofc)
@@ -65,11 +66,14 @@ export class FlowFile implements Resource, PushDiffs {
for (const diff of diffs) {
if (
diff.type !== "REMOVE" &&
diff.path[0] !== "value" &&
(diff.path.length !== 1 ||
!["summary", "description", "schema"].includes(
diff.path[0] as string
))
(
diff.path[0] !== "value" && (
diff.path.length !== 1 ||
!["summary", "description", "schema"].includes(
diff.path[0] as string,
)
)
)
) {
throw new Error("Invalid flow diff with path " + diff.path);
}
@@ -79,8 +83,8 @@ export class FlowFile implements Resource, PushDiffs {
setValueByPath(changeset, diff.path, null);
}
}
const hasChanges = Object.values(changeset).some(
(v) => v !== null && typeof v !== "undefined"
const hasChanges = Object.values(changeset).some((v) =>
v !== null && typeof v !== "undefined"
);
if (!hasChanges) {
return;
@@ -89,8 +93,7 @@ export class FlowFile implements Resource, PushDiffs {
const update = {
...changeset,
...base_changeset,
};
}
await FlowService.updateFlow({
workspace: workspace,
path: remotePath,
@@ -111,10 +114,20 @@ export class FlowFile implements Resource, PushDiffs {
}
}
async push(workspace: string, remotePath: string): Promise<void> {
let remote: Flow | undefined;
try {
remote = await FlowService.getFlowByPath({
workspace,
path: remotePath,
});
} catch {
remote = undefined;
}
await this.pushDiffs(
workspace,
remotePath,
microdiff({}, this, { cyclesFix: false })
microdiff(remote ?? {}, this, { cyclesFix: false }),
);
}
}
@@ -122,24 +135,24 @@ export class FlowFile implements Resource, PushDiffs {
type Options = GlobalOptions;
async function push(opts: Options, filePath: string, remotePath: string) {
if (!validatePath(remotePath)) {
if (!await validatePath(opts, remotePath)) {
return;
}
const workspace = await resolveWorkspace(opts);
await requireLogin(opts);
await pushFlow(filePath, workspace.workspaceId, remotePath);
await pushFlow(filePath, workspace.remote, remotePath);
console.log(colors.bold.underline.green("Flow pushed"));
}
export async function pushFlow(
filePath: string,
workspace: string,
remotePath: string
remotePath: string,
) {
const data = decoverto
.type(FlowFile)
.rawToInstance(await Deno.readTextFile(filePath));
const data = decoverto.type(FlowFile).rawToInstance(
await Deno.readTextFile(filePath),
);
await data.push(workspace, remotePath);
}
@@ -168,7 +181,13 @@ async function list(opts: GlobalOptions & { showArchived?: boolean }) {
.header(["path", "summary", "edited by"])
.padding(2)
.border(true)
.body(total.map((x) => [x.path, x.summary, x.edited_by]))
.body(
total.map((x) => [
x.path,
x.summary,
x.edited_by,
]),
)
.render();
}
async function run(
@@ -176,13 +195,14 @@ async function run(
data?: string;
silent: boolean;
},
path: string
path: string,
) {
const workspace = await resolveWorkspace(opts);
await requireLogin(opts);
const input = opts.data ? await resolve(opts.data) : {};
const id = await JobService.runFlowByPath({
workspace: workspace.workspaceId,
path,
@@ -217,7 +237,7 @@ async function run(
if (!opts.silent) {
console.log(colors.green.underline.bold("Flow ran to completion"));
console.log();
console.log()
}
const jobInfo = await JobService.getCompletedJob({
workspace: workspace.workspaceId,
@@ -232,7 +252,7 @@ const command = new Command()
.action(list as any)
.command(
"push",
"push a local flow spec. This overrides any remote versions."
"push a local flow spec. This overrides any remote versions.",
)
.arguments("<file_path:string> <remote_path:string>")
.action(push as any)
@@ -240,11 +260,11 @@ const command = new Command()
.arguments("<path:string>")
.option(
"-d --data <data:string>",
"Inputs specified as a JSON string or a file using @<filename> or stdin using @-."
"Inputs specified as a JSON string or a file using @<filename> or stdin using @-.",
)
.option(
"-s --silent",
"Do not ouput anything other then the final output. Useful for scripting."
"Do not ouput anything other then the final output. Useful for scripting.",
)
.action(run as any);

View File

@@ -1,4 +1,4 @@
import { colors, Command, FolderService, microdiff } from "./deps.ts";
import { colors, Command, Folder, FolderService, microdiff } from "./deps.ts";
import { requireLogin, resolveWorkspace, validatePath } from "./context.ts";
import {
Difference,
@@ -20,16 +20,8 @@ import {
export class FolderFile implements Resource, PushDiffs {
@property(array(() => String))
owners: Array<string> | undefined;
@property(
map(
() => String,
() => Boolean,
{ shape: MapShape.Object }
)
)
@property(map(() => String, () => Boolean, { shape: MapShape.Object }))
extra_perms: Map<string, boolean> | undefined;
@property(() => String)
display_name: string | undefined;
async push(workspace: string, remotePath: string): Promise<void> {
if (remotePath.startsWith("/")) {
@@ -39,17 +31,23 @@ export class FolderFile implements Resource, PushDiffs {
remotePath = remotePath.substring(2);
}
let existing: Folder | undefined;
try {
existing = await FolderService.getFolder({ workspace, name: remotePath });
} catch {
existing = undefined;
}
await this.pushDiffs(
workspace,
remotePath,
microdiff({}, this, { cyclesFix: false })
microdiff(existing ?? {}, this, { cyclesFix: false }),
);
}
async pushDiffs(
workspace: string,
remotePath: string,
diffs: Difference[]
diffs: Difference[],
): Promise<void> {
if (remotePath.startsWith("/")) {
remotePath = remotePath.substring(1);
@@ -61,34 +59,29 @@ export class FolderFile implements Resource, PushDiffs {
// TODO: Support this in backend
let exists: boolean;
try {
exists = !!(await FolderService.getFolder({
workspace,
name: remotePath,
}));
exists = !!await FolderService.getFolder({ workspace, name: remotePath });
} catch {
exists = false;
}
if (exists) {
console.log(
colors.bold.yellow(
`Applying ${diffs.length} diffs to existing folder... ${remotePath}`
)
`Applying ${diffs.length} diffs to existing folder...`,
),
);
const changeset: {
owners?: string[] | undefined;
extra_perms?: any;
display_name?: string | undefined;
} = {};
for (const diff of diffs) {
if (
diff.type !== "REMOVE" &&
(diff.path.length !== 1 ||
!["owners", "extra_perms", "display_name"].includes(
diff.path[0] as string
))
(
diff.path.length !== 1 ||
!["owners", "extra_perms"].includes(diff.path[0] as string)
)
) {
console.log(diff.path);
throw new Error("Invalid folder diff with path " + diff.path);
}
if (diff.type === "CREATE" || diff.type === "CHANGE") {
@@ -98,27 +91,18 @@ export class FolderFile implements Resource, PushDiffs {
}
}
const hasChanges = Object.values(changeset).some(
(v) => v !== null && typeof v !== "undefined"
const hasChanges = Object.values(changeset).some((v) =>
v !== null && typeof v !== "undefined"
);
if (!hasChanges) {
return;
}
try {
await FolderService.updateFolder({
workspace: workspace,
name: remotePath,
requestBody: {
...changeset,
extra_perms: changeset.extra_perms
? Object.fromEntries(this.extra_perms?.entries() ?? [])
: undefined,
},
});
} catch (e) {
console.error(colors.red.bold(e.body));
throw e;
}
await FolderService.updateFolder({
workspace: workspace,
name: remotePath,
requestBody: changeset,
});
} else {
console.log(colors.bold.yellow("Creating new folder: " + remotePath));
await FolderService.createFolder({
@@ -137,7 +121,7 @@ async function push(opts: GlobalOptions, filePath: string, remotePath: string) {
const workspace = await resolveWorkspace(opts);
await requireLogin(opts);
if (!validatePath(remotePath)) {
if (!await validatePath(opts, remotePath)) {
return;
}
@@ -155,11 +139,11 @@ async function push(opts: GlobalOptions, filePath: string, remotePath: string) {
export async function pushFolder(
workspace: string,
filePath: string,
remotePath: string
remotePath: string,
) {
const data = decoverto
.type(FolderFile)
.rawToInstance(await Deno.readTextFile(filePath));
const data = decoverto.type(FolderFile).rawToInstance(
await Deno.readTextFile(filePath),
);
data.push(workspace, remotePath);
}
@@ -167,7 +151,7 @@ const command = new Command()
.description("resource related commands")
.command(
"push",
"push a local folder spec. This overrides any remote versions."
"push a local folder spec. This overrides any remote versions.",
)
.arguments("<file_path:string> <remote_path:string>")
.action(push as any);

View File

@@ -1,11 +1,5 @@
import {
Command,
CompletionsCommand,
DenoLandProvider,
UpgradeCommand,
} from "./deps.ts";
import { Command, CompletionsCommand, DenoLandProvider, UpgradeCommand } from "./deps.ts";
import flow from "./flow.ts";
import app from "./apps.ts";
import script from "./script.ts";
import workspace from "./workspace.ts";
import resource from "./resource.ts";
@@ -19,21 +13,21 @@ import sync from "./sync.ts";
import { tryResolveVersion } from "./context.ts";
import { GlobalOptions } from "./types.ts";
export const VERSION = "v1.83.1";
const VERSION = "v1.74.2";
let command: any = new Command()
.name("wmill")
.description("A simple CLI tool for windmill.")
.action(() => command.showHelp())
.globalOption(
"--workspace <workspace:string>",
"Specify the target workspace. This overrides the default workspace."
"Specify the target workspace. This overrides the default workspace.",
)
.globalOption(
"--token <token:string>",
"Specify an API token. This will override any stored token."
"Specify an API token. This will override any stored token.",
)
.version(VERSION)
.command("app", app)
.command("flow", flow)
.command("script", script)
.command("workspace", workspace)
@@ -65,12 +59,14 @@ let command: any = new Command()
"--unstable",
],
provider: new DenoLandProvider({ name: "wmill" }),
})
}),
)
.command("completions", new CompletionsCommand());
if (Number.parseInt(VERSION.replace("v", "").replace(".", "")) > 1700) {
command = command.command("push", push).command("pull", pull);
command = command
.command("push", push)
.command("pull", pull);
}
try {

View File

@@ -5,27 +5,25 @@ import { Workspace } from "./workspace.ts";
export async function downloadZip(
workspace: Workspace,
plainSecrets: boolean | undefined
): Promise<JSZip | undefined> {
const requestHeaders: HeadersInit = new Headers();
requestHeaders.set("Authorization", "Bearer " + workspace.token);
requestHeaders.set("Content-Type", "application/octet-stream");
const zipResponse = await fetch(
workspace.remote +
"api/w/" +
workspace.workspaceId +
"/workspaces/tarball?archive_type=zip&plain_secret=" +
(plainSecrets ?? false),
workspace.remote + "api/w/" + workspace.workspaceId +
"/workspaces/tarball?archive_type=zip",
{
headers: requestHeaders,
method: "GET",
}
},
);
if (!zipResponse.ok) {
console.log(
colors.red("Failed to request tarball from API " + zipResponse.statusText)
colors.red(
"Failed to request tarball from API " + zipResponse.statusText,
),
);
throw new Error(await zipResponse.text());
}
@@ -35,18 +33,18 @@ export async function downloadZip(
async function stub(
_opts: GlobalOptions & { override: boolean },
_dir: string
_dir: string,
) {
console.log(
colors.red.underline(
'Pull is deprecated. Use "sync pull --raw" instead. See <TODO_LINK_HERE> for more information.'
)
'Pull is deprecated. Use "sync pull --raw" instead. See <TODO_LINK_HERE> for more information.',
),
);
}
const command = new Command()
.description(
"Pull all definitions in the current workspace from the API and write them to disk."
"Pull all definitions in the current workspace from the API and write them to disk.",
)
.arguments("<dir:string>")
.action(stub as any);

View File

@@ -13,6 +13,7 @@ import {
EditResourceType,
microdiff,
ResourceService,
ResourceType,
Table,
} from "./deps.ts";
import { Any, decoverto, model, property } from "./decoverto.ts";
@@ -25,10 +26,19 @@ export class ResourceTypeFile implements ResourceI, PushDiffs {
description?: string;
async push(workspace: string, remotePath: string): Promise<void> {
await this.pushDiffs(
let existing: ResourceType | undefined;
try {
existing = await ResourceService.getResourceType({
workspace,
path: remotePath,
});
} catch {
existing = undefined;
}
this.pushDiffs(
workspace,
remotePath,
microdiff({}, this, { cyclesFix: false }),
microdiff(existing ?? {}, this, { cyclesFix: false }),
);
}
@@ -55,8 +65,8 @@ export class ResourceTypeFile implements ResourceI, PushDiffs {
return;
}
console.log(
colors.yellow.bold(
`Applying ${diffs.length} diffs to existing resource type... ${remotePath}`,
colors.yellow(
`Applying ${diffs.length} diffs to existing resource type...`,
),
);
const changeset: EditResourceType = {};

View File

@@ -34,7 +34,7 @@ export class ResourceFile implements Resource2, PushDiffs {
async pushDiffs(
workspace: string,
remotePath: string,
diffs: Difference[]
diffs: Difference[],
): Promise<void> {
if (
await ResourceService.existsResource({
@@ -43,9 +43,7 @@ export class ResourceFile implements Resource2, PushDiffs {
})
) {
console.log(
colors.yellow.bold(
`Applying ${diffs.length} diffs to existing resource... ${remotePath}`
)
colors.yellow(`Applying ${diffs.length} diffs to existing resource...`),
);
const changeset: EditResource = {
@@ -53,18 +51,22 @@ export class ResourceFile implements Resource2, PushDiffs {
};
for (const diff of diffs) {
if (diff.path[0] === "is_oauth") {
//is_oauth is not updatable
console.log(
colors.yellow(
"! is_oauth has been removed in newer versions. Ignoring.",
),
);
continue;
}
if (
diff.type !== "REMOVE" &&
diff.path[0] !== "value" &&
(diff.path.length !== 1 || diff.path[0] !== "description") &&
diff.path[0] !== "resource_type"
(
diff.path[0] !== "value" && (
diff.path.length !== 1 ||
diff.path[0] !== "description"
)
)
) {
console.log(
colors.red("Invalid variable diff with path " + diff.path)
);
throw new Error("Invalid folder diff with path " + diff.path);
}
if (diff.type === "CREATE" || diff.type === "CHANGE") {
@@ -74,8 +76,8 @@ export class ResourceFile implements Resource2, PushDiffs {
}
}
const hasChanges = Object.values(changeset).some(
(v) => v !== null && typeof v !== "undefined"
const hasChanges = Object.values(changeset).some((v) =>
v !== null && typeof v !== "undefined"
);
if (!hasChanges) {
return;
@@ -90,8 +92,8 @@ export class ResourceFile implements Resource2, PushDiffs {
if (typeof this.is_oauth !== "undefined") {
console.log(
colors.yellow(
"! is_oauth has been removed in newer versions. Ignoring."
)
"! is_oauth has been removed in newer versions. Ignoring.",
),
);
}
@@ -108,10 +110,19 @@ export class ResourceFile implements Resource2, PushDiffs {
}
}
async push(workspace: string, remotePath: string): Promise<void> {
let existing: Resource | undefined;
try {
existing = await ResourceService.getResource({
workspace,
path: remotePath,
});
} catch {
existing = undefined;
}
await this.pushDiffs(
workspace,
remotePath,
microdiff({}, this, { cyclesFix: false })
microdiff(existing ?? {}, this, { cyclesFix: false }),
);
}
}
@@ -119,11 +130,11 @@ export class ResourceFile implements Resource2, PushDiffs {
export async function pushResource(
workspace: string,
filePath: string,
remotePath: string
remotePath: string,
) {
const data = decoverto
.type(ResourceFile)
.rawToInstance(await Deno.readTextFile(filePath));
const data = decoverto.type(ResourceFile).rawToInstance(
await Deno.readTextFile(filePath),
);
await data.push(workspace, remotePath);
}
@@ -132,7 +143,7 @@ async function push(opts: PushOptions, filePath: string, remotePath: string) {
const workspace = await resolveWorkspace(opts);
await requireLogin(opts);
if (!validatePath(remotePath)) {
if (!await validatePath(opts, remotePath)) {
return;
}
@@ -179,7 +190,7 @@ const command = new Command()
.action(list as any)
.command(
"push",
"push a local resource spec. This overrides any remote versions."
"push a local resource spec. This overrides any remote versions.",
)
.arguments("<file_path:string> <remote_path:string>")
.action(push as any);

View File

@@ -32,11 +32,8 @@ export class ScriptFile {
if (data == null) return data;
if (
data === "script" ||
data === "failure" ||
data === "trigger" ||
data === "command" ||
data === "approvial"
data === "script" || data === "failure" || data === "trigger" ||
data === "command" || data === "approvial"
) {
return data;
}
@@ -45,7 +42,6 @@ export class ScriptFile {
},
toPlain: (data) => data,
})
@property(() => String)
kind?: "script" | "failure" | "trigger" | "command" | "approval";
constructor(summary: string, description: string) {
@@ -55,11 +51,14 @@ export class ScriptFile {
}
type PushOptions = GlobalOptions;
async function push(opts: PushOptions, filePath: string) {
async function push(
opts: PushOptions,
filePath: string,
remotePath: string,
contentPath?: string,
) {
const workspace = await resolveWorkspace(opts);
const remotePath = filePath.split(".")[0];
if (!validatePath(remotePath)) {
if (!await validatePath(opts, remotePath)) {
return;
}
@@ -67,136 +66,89 @@ async function push(opts: PushOptions, filePath: string) {
if (!fstat.isFile) {
throw new Error("file path must refer to a file.");
}
let contentPath: string;
let metaPath: string | undefined;
if (filePath.endsWith(".script.json")) {
metaPath = filePath;
if (!contentPath) {
contentPath = await findContentFile(filePath);
} else {
contentPath = filePath;
metaPath = undefined;
const fstat = await Deno.stat(filePath);
if (!fstat.isFile) {
throw new Error("content path must refer to a file.");
}
}
await requireLogin(opts);
await pushScript(metaPath, contentPath, workspace.workspaceId, remotePath);
await pushScript(filePath, contentPath, workspace.workspaceId, remotePath);
console.log(colors.bold.underline.green(`Script ${remotePath} pushed`));
}
export async function handleScriptMetadata(
path: string,
workspace: string,
alreadySynced: string[]
): Promise<boolean> {
export async function handleScriptMetadata(path: string, workspace: string, alreadySynced: string[]): Promise<boolean> {
if (path.endsWith(".script.json")) {
const contentPath = await findContentFile(path);
return handleFile(
contentPath,
await Deno.readTextFile(contentPath),
workspace,
alreadySynced
);
const contentPath = await findContentFile(path)
return handleFile(contentPath, await Deno.readTextFile(contentPath), workspace, alreadySynced)
} else {
return false;
return false
}
}
export async function handleFile(
path: string,
content: string,
workspace: string,
alreadySynced: string[]
): Promise<boolean> {
if (
path.endsWith(".ts") ||
path.endsWith(".py") ||
path.endsWith(".go") ||
path.endsWith(".sh")
) {
export async function handleFile(path: string, content: string, workspace: string, alreadySynced: string[]): Promise<boolean> {
if (path.endsWith(".ts") || path.endsWith(".py") || path.endsWith(".go") || path.endsWith(".sh")) {
if (alreadySynced.includes(path)) {
return true;
return true
}
alreadySynced.push(path);
alreadySynced.push(path)
const remotePath = path.substring(0, path.length - 3);
const metaPath = remotePath + ".script.json";
let typed = undefined;
let typed = undefined
try {
await Deno.stat(metaPath);
typed = JSON.parse(await Deno.readTextFile(metaPath));
await Deno.stat(metaPath)
typed = JSON.parse(await Deno.readTextFile(metaPath))
typed = decoverto.type(ScriptFile).plainToInstance(typed);
} catch {}
} catch { }
const language = inferContentTypeFromFilePath(path);
let remote = undefined;
try {
remote = await ScriptService.getScriptByPath({
const remote = await ScriptService.getScriptByPath({
workspace,
path: remotePath,
});
} catch {}
if (remote) {
if (content === remote.content) {
if (
typed == undefined ||
(typed.description === remote.description &&
typed.summary === remote.summary &&
typed.is_template === remote.is_template &&
typed.kind == remote.kind &&
remote?.lock == typed.lock?.join("\n") &&
JSON.stringify(typed.schema) == JSON.stringify(remote.schema))
) {
console.log(
colors.yellow(
`No change to push for script ${remotePath}, skipping`
)
);
return true;
}
}
await ScriptService.createScript({
workspace,
requestBody: {
content,
description: typed?.description ?? "",
description: typed.description,
language,
path: remotePath,
summary: typed?.summary ?? "",
is_template: typed?.is_template,
kind: typed?.kind,
lock: typed?.lock,
summary: typed.summary,
is_template: typed.is_template,
kind: typed.kind,
lock: typed.lock,
parent_hash: remote.hash,
schema: typed?.schema,
schema: typed.schema,
},
});
console.log(
colors.yellow.bold(`Creating script with a parent ${remotePath}`)
);
} else {
console.log(colors.yellow.bold(`Creating script with a parent ${remotePath}`))
} catch {
// no parent hash
await ScriptService.createScript({
workspace: workspace,
requestBody: {
content,
description: typed?.description ?? "",
description: typed.description,
language,
path: remotePath,
summary: typed?.summary ?? "",
is_template: typed?.is_template,
kind: typed?.kind,
lock: typed?.lock,
summary: typed.summary,
is_template: typed.is_template,
kind: typed.kind,
lock: typed.lock,
parent_hash: undefined,
schema: typed?.schema,
schema: typed.schema,
},
});
console.log(
colors.yellow.bold(`Creating script without parent ${remotePath}`)
);
console.log(colors.yellow.bold(`Creating script without parent ${remotePath}`))
}
return true;
return true
}
return false;
return false
}
export async function findContentFile(filePath: string) {
@@ -215,7 +167,7 @@ export async function findContentFile(filePath: string) {
.then((e) => {
return { path: x, file: e };
});
})
}),
)
)
.filter((x) => x.file)
@@ -223,7 +175,7 @@ export async function findContentFile(filePath: string) {
if (validCandidates.length > 1) {
throw new Error(
"No content path given and more then one candidate found: " +
validCandidates.join(", ")
validCandidates.join(", "),
);
}
if (validCandidates.length < 1) {
@@ -233,7 +185,7 @@ export async function findContentFile(filePath: string) {
}
export function inferContentTypeFromFilePath(
contentPath: string
contentPath: string,
): "python3" | "deno" | "go" | "bash" {
let language = contentPath.substring(contentPath.lastIndexOf("."));
if (language == ".ts") language = "deno";
@@ -241,9 +193,7 @@ export function inferContentTypeFromFilePath(
if (language == ".sh") language = "bash";
if (language == ".go") language = "go";
if (
language != "python3" &&
language != "deno" &&
language != "go" &&
language != "python3" && language != "deno" && language != "go" &&
language != "bash"
) {
throw new Error("Invalid language: " + language);
@@ -252,20 +202,18 @@ export function inferContentTypeFromFilePath(
}
export async function pushScript(
filePath: string | undefined,
filePath: string,
contentPath: string,
workspace: string,
remotePath: string
remotePath: string,
) {
const data = filePath
? decoverto
.type(ScriptFile)
.rawToInstance(await Deno.readTextFile(filePath))
: undefined;
const data = decoverto.type(ScriptFile).rawToInstance(
await Deno.readTextFile(filePath),
);
const content = await Deno.readTextFile(contentPath);
const language = inferContentTypeFromFilePath(contentPath);
let parent_hash = data?.parent_hash;
let parent_hash = data.parent_hash;
if (!parent_hash) {
try {
parent_hash = (
@@ -284,15 +232,15 @@ export async function pushScript(
workspace: workspace,
requestBody: {
path: remotePath,
summary: data?.summary ?? "",
summary: data.summary,
content: content,
description: data?.description ?? "",
description: data.description,
language: language,
is_template: data?.is_template,
kind: data?.kind,
lock: data?.lock,
is_template: data.is_template,
kind: data.kind,
lock: data.lock,
parent_hash: parent_hash,
schema: data?.schema,
schema: data.schema,
},
});
}
@@ -322,7 +270,14 @@ async function list(opts: GlobalOptions & { showArchived?: boolean }) {
.header(["path", "summary", "language", "created by"])
.padding(2)
.border(true)
.body(total.map((x) => [x.path, x.summary, x.language, x.created_by]))
.body(
total.map((x) => [
x.path,
x.summary,
x.language,
x.created_by,
]),
)
.render();
}
@@ -333,15 +288,14 @@ export async function resolve(input: string): Promise<Record<string, any>> {
if (input == "@-") {
input = new TextDecoder().decode(await readAll(Deno.stdin));
}
if (input[0] == "@") {
} if (input[0] == "@") {
input = await Deno.readTextFile(input.substring(1));
}
try {
return JSON.parse(input);
} catch (e) {
console.error("Impossible to parse input as JSON", input);
throw e;
console.error("Impossible to parse input as JSON", input)
throw e
}
}
@@ -350,11 +304,12 @@ async function run(
data?: string;
silent: boolean;
},
path: string
path: string,
) {
const workspace = await resolveWorkspace(opts);
await requireLogin(opts);
const input = opts.data ? await resolve(opts.data) : {};
const id = await JobService.runScriptByPath({
workspace: workspace.workspaceId,
@@ -368,13 +323,12 @@ async function run(
while (true) {
try {
const result =
(
await JobService.getCompletedJob({
workspace: workspace.workspaceId,
id,
})
).result ?? {};
const result = (
await JobService.getCompletedJob({
workspace: workspace.workspaceId,
id,
})
).result ?? {};
console.log(result);
break;
@@ -389,9 +343,9 @@ export async function track_job(workspace: string, id: string) {
const result = await JobService.getCompletedJob({ workspace, id });
console.log(result.logs);
console.log();
console.log()
console.log(colors.bold.underline.green("Job Completed"));
console.log();
console.log()
return;
} catch {
/* ignore */
@@ -442,7 +396,7 @@ export async function track_job(workspace: string, id: string) {
if (running && updates.running === false) {
running = false;
console.log(
colors.yellow("Job suspended. Waiting for it to continue...")
colors.yellow("Job suspended. Waiting for it to continue..."),
);
}
}
@@ -453,13 +407,15 @@ export async function track_job(workspace: string, id: string) {
if ((final_job.logs?.length ?? -1) > logOffset) {
console.log(final_job.logs!.substring(logOffset));
}
console.log("\n");
console.log("\n")
if (final_job.success) {
console.log(colors.bold.underline.green("Job Completed"));
} else {
console.log(colors.bold.underline.red("Job Completed"));
}
console.log();
console.log()
} catch {
console.log("Job appears to have completed, but no data can be retrieved");
}
@@ -484,22 +440,22 @@ const command = new Command()
.action(list as any)
.command(
"push",
"push a local script spec. This overrides any remote versions. Can use a script file (.ts, .js, .py, .sh) or a script spec file (.json). "
"push a local script spec. This overrides any remote versions.",
)
.arguments("<file_path:file>")
.arguments("<file_path:string> <remote_path:string> [content_path:string]")
.action(push as any)
.command("show", "show a scripts content")
.arguments("<path:file>")
.arguments("<path:string>")
.action(show as any)
.command("run", "run a script by path")
.arguments("<path:file>")
.arguments("<path:string>")
.option(
"-d --data <data:file>",
"Inputs specified as a JSON string or a file using @<filename> or stdin using @-."
"-d --data <data:string>",
"Inputs specified as a JSON string or a file using @<filename> or stdin using @-.",
)
.option(
"-s --silent",
"Do not ouput anything other then the final output. Useful for scripting."
"Do not ouput anything other then the final output. Useful for scripting.",
)
.action(run as any);

View File

@@ -13,7 +13,7 @@ import {
ResourceService,
VariableService,
AppService,
FlowService,
FlowService
} from "./deps.ts";
import {
Difference,
@@ -25,14 +25,16 @@ import {
import { downloadZip } from "./pull.ts";
import { FolderFile } from "./folder.ts";
import { ResourceTypeFile } from "./resource-type.ts";
import { handleScriptMetadata, ScriptFile } from "./script.ts";
import {
handleScriptMetadata,
ScriptFile,
} from "./script.ts";
import { ResourceFile } from "./resource.ts";
import { FlowFile } from "./flow.ts";
import { VariableFile } from "./variable.ts";
import { handleFile } from "./script.ts";
import { equal } from "https://deno.land/x/equal@v1.5.0/mod.ts";
import * as Diff from "npm:diff";
import { equal } from "https://deno.land/x/equal/mod.ts";
import { diffCharacters } from "https://deno.land/x/diff/mod.ts";
type DynFSElement = {
isDirectory: boolean;
path: string;
@@ -42,7 +44,10 @@ type DynFSElement = {
};
async function FSFSElement(p: string): Promise<DynFSElement> {
function _internal_element(localP: string, isDir: boolean): DynFSElement {
function _internal_element(
localP: string,
isDir: boolean,
): DynFSElement {
return {
isDirectory: isDir,
path: localP.substring(p.length + 1),
@@ -108,14 +113,16 @@ function ZipFSElement(zip: JSZip): DynFSElement {
async function* readDirRecursiveWithIgnore(
ignore: (path: string, isDirectory: boolean) => boolean,
root: DynFSElement
): AsyncGenerator<{
path: string;
ignored: boolean;
isDirectory: boolean;
getContentBytes(): Promise<Uint8Array>;
getContentText(): Promise<string>;
}> {
root: DynFSElement,
): AsyncGenerator<
{
path: string;
ignored: boolean;
isDirectory: boolean;
getContentBytes(): Promise<Uint8Array>;
getContentText(): Promise<string>;
}
> {
const stack: {
path: string;
isDirectory: boolean;
@@ -123,20 +130,18 @@ async function* readDirRecursiveWithIgnore(
c(): AsyncIterable<DynFSElement>;
getContentBytes(): Promise<Uint8Array>;
getContentText(): Promise<string>;
}[] = [
{
path: root.path,
ignored: ignore(root.path, root.isDirectory),
isDirectory: root.isDirectory,
c: root.getChildren,
getContentBytes(): Promise<Uint8Array> {
throw undefined;
},
getContentText(): Promise<string> {
throw undefined;
},
}[] = [{
path: root.path,
ignored: ignore(root.path, root.isDirectory),
isDirectory: root.isDirectory,
c: root.getChildren,
getContentBytes(): Promise<Uint8Array> {
throw undefined;
},
];
getContentText(): Promise<string> {
throw undefined;
},
}];
while (stack.length > 0) {
const e = stack.pop()!;
@@ -156,17 +161,17 @@ async function* readDirRecursiveWithIgnore(
}
type Added = { name: "added"; path: string; content: string };
type Deleted = { name: "deleted"; path: string };
type Edit = { name: "edited"; path: string; before: string; after: string };
type Deleted = { name: "deleted"; path: string; };
type Edit = { name: "edited"; path: string; before: string; after: string; };
type Change = Added | Deleted | Edit;
async function elementsToMap(
els: DynFSElement,
ignore: (path: string, isDirectory: boolean) => boolean
): Promise<{ [key: string]: string }> {
async function elementsToMap(els: DynFSElement, ignore: (path: string, isDirectory: boolean) => boolean): Promise<{ [key: string]: string }> {
const map: { [key: string]: string } = {};
for await (const entry of readDirRecursiveWithIgnore(ignore, els)) {
for await (const entry of readDirRecursiveWithIgnore(
ignore,
els,
)) {
if (entry.isDirectory || entry.ignored) continue;
const content = await entry.getContentText();
map[entry.path] = content;
@@ -174,26 +179,20 @@ async function elementsToMap(
return map;
}
async function compareDynFSElement(
els1: DynFSElement,
els2: DynFSElement | undefined,
ignore: (path: string, isDirectory: boolean) => boolean
els1: DynFSElement, els2: DynFSElement,
ignore: (path: string, isDirectory: boolean) => boolean,
raw: boolean
): Promise<Change[]> {
const [m1, m2] = els2
? await Promise.all([
elementsToMap(els1, ignore),
elementsToMap(els2, ignore),
])
: [await elementsToMap(els1, ignore), {}];
const [m1, m2] = raw ? [await elementsToMap(els1, ignore), {}] :
await Promise.all([elementsToMap(els1, ignore), elementsToMap(els2, ignore)]);
const changes: Change[] = [];
for (const [k, v] of Object.entries(m1)) {
if (m2[k] === undefined) {
changes.push({ name: "added", path: k, content: v });
} else if (
m2[k] != v &&
(!k.endsWith(".json") || !equal(JSON.parse(v), JSON.parse(m2[k])))
) {
} else if (m2[k] != v && (!k.endsWith(".json") || !equal(JSON.parse(v), JSON.parse(m2[k])))) {
// await Deno.writeTextFile("/tmp/k", m2[k])
// await Deno.writeTextFile("/tmp/v", v)
// console.log(k)
@@ -209,59 +208,54 @@ async function compareDynFSElement(
}
}
return changes;
return changes
}
const isNotWmillFile = (p: string, isDirectory: boolean) => {
if (p.endsWith("/")) {
return false;
return false
}
if (isDirectory) {
return !p.startsWith("u/") && !p.startsWith("f/") && !p.startsWith("g/");
return !p.startsWith("u/") && !p.startsWith("f/") && !p.startsWith("g/")
}
try {
const typ = getTypeStrFromPath(p);
if (typ == "resource-type") {
return p.includes("/");
const typ = getTypeStrFromPath(p)
if (typ == 'resource-type') {
return p.includes('/')
} else {
return !p.startsWith("u/") && !p.startsWith("f/") && !p.startsWith("g/");
return !p.startsWith("u/") && !p.startsWith("f/") && !p.startsWith("g/")
}
} catch {
return true;
return true
}
};
}
const isWhitelisted = (p: string) => {
return p == "./" || p == "" || p == "u" || p == "f" || p == "g";
};
return p == "./" || p == "" || p == "u" || p == "f" || p == "g"
}
async function ignoreF() {
try {
const ignore: {
accepts(file: string): boolean;
denies(file: string): boolean;
} = gitignore_parser.compile(await Deno.readTextFile(".wmillignore"));
} = gitignore_parser.compile(
await Deno.readTextFile(".wmillignore"),
);
return (p: string, isDirectory: boolean) => {
return (
!isWhitelisted(p) &&
(isNotWmillFile(p, isDirectory) || ignore.denies(p))
);
};
return !isWhitelisted(p) && (isNotWmillFile(p, isDirectory) || ignore.denies(p));
}
} catch (e) {
return (p: string, isDirectory: boolean) =>
!isWhitelisted(p) && isNotWmillFile(p, isDirectory);
return (p: string, isDirectory: boolean) => !isWhitelisted(p) && isNotWmillFile(p, isDirectory)
}
}
async function pull(
opts: GlobalOptions & {
raw: boolean;
yes: boolean;
failConflicts: boolean;
plainSecrets?: boolean;
}
opts: GlobalOptions & { raw: boolean; yes: boolean, failConflicts: boolean },
) {
if (!opts.raw) {
await ensureDir(path.join(Deno.cwd(), ".wmill"));
}
@@ -269,98 +263,74 @@ async function pull(
const workspace = await resolveWorkspace(opts);
await requireLogin(opts);
console.log(
colors.gray(
"Computing the files to update locally to match remote (taking .wmillignore into account)"
)
);
const remote = ZipFSElement(
(await downloadZip(workspace, opts.plainSecrets))!
);
const local = opts.raw
? undefined
: await FSFSElement(path.join(Deno.cwd(), opts.raw ? "" : ".wmill"));
const changes = await compareDynFSElement(remote, local, await ignoreF());
console.log(colors.gray("Computing the files to update locally to match remote (taking .wmillignore into account)"));
const remote = ZipFSElement((await downloadZip(workspace))!)
const local = await FSFSElement(path.join(Deno.cwd(), opts.raw ? "" : ".wmill"))
const changes = await compareDynFSElement(remote, local, await ignoreF(), opts.raw)
console.log(`remote -> local: ${changes.length} changes to apply`);
if (changes.length > 0) {
prettyChanges(changes);
prettyChanges(changes)
if (
!opts.yes &&
!opts.raw &&
!(await Confirm.prompt({
message: `Do you want to apply these ${changes.length} changes?`,
default: true,
}))
!opts.yes && !opts.raw && !(await Confirm.prompt({ message: `Do you want to apply these ${changes.length} changes?`, default: true }))
) {
return;
return
}
const conflicts = [];
const conflicts = []
console.log(colors.gray(`Applying changes to files ...`));
for await (const change of changes) {
const target = path.join(Deno.cwd(), change.path);
const stateTarget = path.join(Deno.cwd(), ".wmill", change.path);
const stateTarget = path.join(Deno.cwd(), ".wmill", change.path)
if (change.name === "edited") {
try {
const currentLocal = await Deno.readTextFile(target);
if (currentLocal !== change.before && currentLocal !== change.after) {
console.log(
colors.red(
`Conflict detected on ${change.path}\nBoth local and remote have been modified.`
)
);
const currentLocal = await Deno.readTextFile(target)
if (currentLocal !== change.before) {
console.log(colors.red(`Conflict detected on ${change.path}\nBoth local and remote have been modified.`))
if (opts.failConflicts) {
conflicts.push({
local: currentLocal,
change,
path: change.path,
});
conflicts.push({ local: currentLocal, change, path: change.path })
continue;
} else if (opts.yes) {
console.log(
colors.red(
`Override local version with remote since --yes was passed and no --fail-conflicts.`
)
);
} else {
showConflict(change.path, currentLocal, change.after);
if (
await Confirm.prompt(
"Preserve local (push to change remote and avoid seeing this again)?"
)
) {
console.log(colors.red(`Override local version with remote since --yes was passed and no --fail-conflicts.`))
}
else {
showConflict(change.path, currentLocal, change.after)
if (await Confirm.prompt("Preserve local (push to change remote and avoid seeing this again)?")) {
continue;
}
}
}
} catch {}
} catch { }
if (change.path.endsWith(".json")) {
const diffs = microdiff(
JSON.parse(change.before),
JSON.parse(change.after),
{ cyclesFix: false }
);
const diffs =
microdiff(
JSON.parse(change.before),
JSON.parse(change.after),
{ cyclesFix: false },
)
console.log(
`Editing ${getTypeStrFromPath(change.path)} json ${change.path}`
console.log(`Editing ${getTypeStrFromPath(change.path)} json ${change.path}`)
await applyDiff(
diffs,
target,
);
await applyDiff(diffs, target);
} else {
console.log(`Editing script ${change.path}`);
console.log(`Editing script ${change.path}`)
await Deno.writeTextFile(target, change.after);
}
if (!opts.raw) {
await ensureDir(path.dirname(stateTarget));
await ensureDir(path.dirname(stateTarget))
await Deno.copyFile(target, stateTarget);
}
} else if (change.name === "added") {
await ensureDir(path.dirname(target));
await ensureDir(path.dirname(target))
if (!opts.raw) {
await ensureDir(path.dirname(stateTarget));
console.log(
`Adding ${getTypeStrFromPath(change.path)} ${change.path}`
);
await ensureDir(path.dirname(stateTarget))
console.log(`Adding ${getTypeStrFromPath(change.path)} ${change.path}`)
}
await Deno.writeTextFile(target, change.content);
if (!opts.raw) {
@@ -368,10 +338,8 @@ async function pull(
}
} else if (change.name === "deleted") {
try {
console.log(
`Deleting ${getTypeStrFromPath(change.path)} ${change.path}`
);
await Deno.remove(target);
console.log(`Deleting ${getTypeStrFromPath(change.path)} ${change.path}`)
await Deno.remove(target)
if (!opts.raw) {
await Deno.remove(stateTarget);
}
@@ -384,46 +352,41 @@ async function pull(
}
if (opts.failConflicts) {
if (conflicts.length > 0) {
console.error(colors.red(`Conflicts were found`));
console.log("Conflicts:");
console.error(colors.red(`Conflicts were found`))
console.log("Conflicts:")
for (const conflict of conflicts) {
showConflict(conflict.path, conflict.local, conflict.change.after);
showConflict(conflict.path, conflict.local, conflict.change.after)
}
console.log(
colors.red(`Please resolve theses conflicts manually by either:
console.log(colors.red(`Please resolve theses conflicts manually by either:
- reverting the content back to its remote (\`wmill pull\` and refuse to preserve local when prompted)
- pushing the changes with \`wmill push --skip-pull\` to override wmill with all your local changes
`)
);
Deno.exit(1);
`))
Deno.exit(1)
}
}
console.log(
colors.green.underline(
`Done! All ${changes.length} changes applied locally.`
)
);
console.log(colors.green.underline(`Done! All ${changes.length} changes applied locally.`));
}
function showConflict(path: string, local: string, remote: string) {
console.log(colors.yellow(`- ${path}`));
console.log(colors.yellow(`- ${path}`))
let finalString = "";
for (const part of Diff.diffLines(local, remote)) {
if (part.removed) {
for (const character of diffCharacters(local, remote)) {
if (character.wasRemoved) {
// print red if removed without newline
finalString += `\x1b[31m${part.value}\x1b[0m`;
} else if (part.added) {
finalString += `\x1b[31m${character.character}\x1b[0m`;
} else if (character.wasAdded) {
// print green if added
finalString += `\x1b[32m${part.value}\x1b[0m`;
finalString += `\x1b[32m${character.character}\x1b[0m`;
} else {
// print white if unchanged
finalString += `\x1b[37m${part.value}\x1b[0m`;
finalString += `\x1b[37m${character.character}\x1b[0m`;
}
}
console.log(finalString);
console.log("\x1b[31mlocal\x1b[31m - \x1b[32mremote\x1b[32m");
console.log();
console.log("\x1b[31mlocal\x1b[31m - \x1b[32mremote\x1b[32m")
console.log()
}
async function applyDiff(diffs: Difference[], file: string) {
ensureDir(path.dirname(file));
@@ -452,258 +415,175 @@ async function pull(
}
}
function prettyChanges(changes: Change[]) {
for (const change of changes) {
if (change.name === "added") {
console.log(
colors.green(`+ ${getTypeStrFromPath(change.path)} ` + change.path)
);
console.log(colors.green(`+ ${getTypeStrFromPath(change.path)} ` + change.path));
} else if (change.name === "deleted") {
console.log(
colors.red(`- ${getTypeStrFromPath(change.path)} ` + change.path)
);
console.log(colors.red(`- ${getTypeStrFromPath(change.path)} ` + change.path));
} else if (change.name === "edited") {
console.log(
colors.yellow(`~ ${getTypeStrFromPath(change.path)} ` + change.path)
);
console.log(colors.yellow(`~ ${getTypeStrFromPath(change.path)} ` + change.path));
}
}
}
function prettyDiff(diffs: Difference[]) {
for (const diff of diffs) {
let pathString = "";
for (const pathSegment of diff.path) {
if (typeof pathSegment === "string") {
pathString += ".";
pathString += pathSegment;
} else {
pathString += "[";
pathString += pathSegment;
pathString += "]";
}
}
if (diff.type === "REMOVE" || diff.type === "CHANGE") {
console.log(colors.red("- " + pathString + " = " + diff.oldValue));
}
if (diff.type === "CREATE" || diff.type === "CHANGE") {
console.log(colors.green("+ " + pathString + " = " + diff.value));
}
}
}
// function prettyDiff(diffs: Difference[]) {
// for (const diff of diffs) {
// let pathString = "";
// for (const pathSegment of diff.path) {
// if (typeof pathSegment === "string") {
// pathString += ".";
// pathString += pathSegment;
// } else {
// pathString += "[";
// pathString += pathSegment;
// pathString += "]";
// }
// }
// if (diff.type === "REMOVE" || diff.type === "CHANGE") {
// console.log(colors.red("- " + pathString + " = " + diff.oldValue));
// }
// if (diff.type === "CREATE" || diff.type === "CHANGE") {
// console.log(colors.green("+ " + pathString + " = " + diff.value));
// }
// }
// }
function removeSuffix(str: string, suffix: string) {
return str.slice(0, str.length - suffix.length);
}
async function push(
opts: GlobalOptions & {
raw: boolean;
yes: boolean;
skipPull: boolean;
failConflicts: boolean;
plainSecrets?: boolean;
}
) {
async function push(opts: GlobalOptions & { raw: boolean, yes: boolean, skipPull: boolean, failConflicts: boolean }) {
if (!opts.raw) {
if (!opts.skipPull) {
console.log(
colors.gray("You need to be up-to-date before pushing, pulling first.")
);
await pull(opts);
console.log(colors.green("Pull done, now pushing."));
console.log();
console.log(colors.gray("You need to be up-to-date before pushing, pulling first."))
await pull(opts)
console.log(colors.green("Pull done, now pushing."))
console.log()
}
}
const workspace = await resolveWorkspace(opts);
await requireLogin(opts);
console.log(
colors.gray(
"Computing the files to update on the remote to match local (taking .wmillignore into account)"
)
);
const remote = opts.raw
? undefined
: ZipFSElement((await downloadZip(workspace, opts.plainSecrets))!);
const local = await FSFSElement(path.join(Deno.cwd(), ""));
const changes = await compareDynFSElement(local, remote, await ignoreF());
console.log(colors.gray("Computing the files to update on the remote to match local (taking .wmillignore into account)"));
const remote = ZipFSElement((await downloadZip(workspace))!)
const local = await FSFSElement(path.join(Deno.cwd(), ""))
const changes = await compareDynFSElement(local, remote, await ignoreF(), opts.raw)
console.log(`remote <- local: ${changes.length} changes to apply`);
if (changes.length > 0) {
prettyChanges(changes);
prettyChanges(changes)
if (
!opts.yes &&
!(await Confirm.prompt({
message: `Do you want to apply these ${changes.length} changes?`,
default: true,
}))
!opts.yes && !(await Confirm.prompt({ message: `Do you want to apply these ${changes.length} changes?`, default: true }))
) {
return;
return
}
console.log(colors.gray(`Applying changes to files ...`));
const alreadySynced: string[] = [];
const alreadySynced: string[] = []
for await (const change of changes) {
const stateTarget = path.join(Deno.cwd(), ".wmill", change.path);
let stateExists = true;
try {
await Deno.stat(stateTarget);
} catch {
stateExists = false;
}
const stateTarget = path.join(Deno.cwd(), ".wmill", change.path)
if (change.name === "edited") {
if (
await handleScriptMetadata(
change.path,
workspace.workspaceId,
alreadySynced
)
) {
if (!opts.raw && stateExists) {
if (await handleScriptMetadata(change.path, workspace.workspaceId, alreadySynced)) {
if (!opts.raw) {
await Deno.writeTextFile(stateTarget, change.after);
}
continue;
} else if (
await handleFile(
change.path,
change.after,
workspace.workspaceId,
alreadySynced
)
) {
if (!opts.raw && stateExists) {
continue
} else if (await handleFile(change.path, change.after, workspace.workspaceId, alreadySynced)) {
if (!opts.raw) {
await Deno.writeTextFile(stateTarget, change.after);
}
continue;
continue
}
if (!opts.raw) {
await ensureDir(path.dirname(stateTarget));
console.log(
`Editing ${getTypeStrFromPath(change.path)} ${change.path}`
);
await ensureDir(path.dirname(stateTarget))
console.log(`Editing ${getTypeStrFromPath(change.path)} ${change.path}`)
}
const obj = inferTypeFromPath(change.path, JSON.parse(change.after));
const obj = inferTypeFromPath(change.path, JSON.parse(change.after))
const diff = microdiff(
inferTypeFromPath(change.path, JSON.parse(change.before)),
obj,
{ cyclesFix: false }
);
const diff = microdiff(inferTypeFromPath(change.path, JSON.parse(change.before)), obj, { cyclesFix: false });
await applyDiff(
workspace.workspaceId,
change.path.split(".")[0],
obj,
diff,
opts.plainSecrets
);
if (!opts.raw && stateExists) {
if (!opts.raw) {
await Deno.writeTextFile(stateTarget, change.after);
}
} else if (change.name === "added") {
if (change.path.endsWith(".script.json")) {
continue;
} else if (
await handleFile(
change.path,
change.content,
workspace.workspaceId,
alreadySynced
)
) {
continue;
continue
} else if (await handleFile(change.path, change.content, workspace.workspaceId, alreadySynced)) {
continue
}
if (!opts.raw && stateExists) {
await ensureDir(path.dirname(stateTarget));
console.log(
`Adding ${getTypeStrFromPath(change.path)} ${change.path}`
);
if (!opts.raw) {
await ensureDir(path.dirname(stateTarget))
console.log(`Adding ${getTypeStrFromPath(change.path)} ${change.path}`)
}
const obj = inferTypeFromPath(change.path, JSON.parse(change.content));
const obj = inferTypeFromPath(change.path, JSON.parse(change.content))
const diff = microdiff({}, obj, { cyclesFix: false });
await applyDiff(
workspace.workspaceId,
change.path.split(".")[0],
obj,
diff,
opts.plainSecrets
);
if (!opts.raw && stateExists) {
if (!opts.raw) {
await Deno.writeTextFile(stateTarget, change.content);
}
} else if (change.name === "deleted") {
if (!change.path.includes(".json")) {
continue;
continue
}
console.log(
`Deleting ${getTypeStrFromPath(change.path)} ${change.path}`
);
const typ = getTypeStrFromPath(change.path);
console.log(`Deleting ${getTypeStrFromPath(change.path)} ${change.path}`)
const typ = getTypeStrFromPath(change.path)
const workspaceId = workspace.workspaceId;
switch (typ) {
case "script": {
const script = await ScriptService.getScriptByPath({
workspace: workspaceId,
path: removeSuffix(change.path, ".script.json"),
});
await ScriptService.deleteScriptByHash({
workspace: workspaceId,
hash: script.hash,
});
const script = await ScriptService.getScriptByPath({ workspace: workspaceId, path: removeSuffix(change.path, ".script.json") })
await ScriptService.deleteScriptByHash({ workspace: workspaceId, hash: script.hash })
break;
}
case "folder":
await FolderService.deleteFolder({
workspace: workspaceId,
name: change.path.split("/")[1],
});
await FolderService.deleteFolder({ workspace: workspaceId, name: change.path.split('/')[1] })
break;
case "resource":
await ResourceService.deleteResource({
workspace: workspaceId,
path: removeSuffix(change.path, ".resource.json"),
});
await ResourceService.deleteResource({ workspace: workspaceId, path: removeSuffix(change.path, ".resource.json") })
break;
case "resource-type":
await ResourceService.deleteResourceType({
workspace: workspaceId,
path: removeSuffix(change.path, ".resource-type.json"),
});
break;
await ResourceService.deleteResourceType({ workspace: workspaceId, path: removeSuffix(change.path, ".resource-type.json") })
break
case "flow":
await FlowService.deleteFlowByPath({
workspace: workspaceId,
path: removeSuffix(change.path, ".flow.json"),
});
break;
await FlowService.deleteFlowByPath({ workspace: workspaceId, path: removeSuffix(change.path, ".flow.json") })
break
case "app":
await AppService.deleteApp({
workspace: workspaceId,
path: removeSuffix(change.path, ".app.json"),
});
break;
await AppService.deleteApp({ workspace: workspaceId, path: removeSuffix(change.path, ".app.json") })
break
case "variable":
await VariableService.deleteVariable({
workspace: workspaceId,
path: removeSuffix(change.path, ".variable.json"),
});
break;
await VariableService.deleteVariable({ workspace: workspaceId, path: removeSuffix(change.path, ".variable.json") })
break
default:
break;
}
try {
await Deno.remove(stateTarget);
} catch {}
Deno.remove(stateTarget)
} catch { }
}
}
console.log(
colors.green.underline(
`Done! All ${changes.length} changes pushed to the remote workspace.`
)
);
console.log(colors.green.underline(`Done! All ${changes.length} changes pushed to the remote workspace.`));
}
async function applyDiff(
workspace: string,
remotePath: string,
@@ -715,11 +595,10 @@ async function push(
| ResourceTypeFile
| FolderFile,
diffs: Difference[],
plainSecrets?: boolean
) {
if (file instanceof ScriptFile) {
throw new Error(
"This code path should be unreachable - we should never generate diffs for scripts"
"This code path should be unreachable - we should never generate diffs for scripts",
);
} else if (file instanceof FolderFile) {
const parts = remotePath.split("/");
@@ -730,14 +609,14 @@ async function push(
}
}
if (diffs.length === 0) {
console.log("No diffs to apply to " + remotePath);
console.log("No diffs to apply to " + remotePath)
return;
}
try {
await file.pushDiffs(workspace, remotePath, diffs, plainSecrets);
await file.pushDiffs(workspace, remotePath, diffs);
} catch (e) {
console.error("Failing to apply diffs to " + remotePath);
console.error(JSON.stringify(e));
console.error("Failing to apply diffs to " + remotePath)
console.error(e.body)
}
}
}
@@ -745,28 +624,20 @@ async function push(
const command = new Command()
.command("pull")
.description(
"Pull any remote changes and apply them locally. Use --raw for usage without local state tracking."
)
.option(
"--fail-conflicts",
"Error on conflicts (both remote and local have changes on the same item)"
"Pull any remote changes and apply them locally. Use --raw for usage without local state tracking.",
)
.option("--fail-conflicts", "Error on conflicts (both remote and local have changes on the same item)")
.option("--yes", "Pull without needing confirmation")
.option("--raw", "Pull without using state, just overwrite.")
.option("--plain-secrets", "Pull secrets as plain text")
.action(pull as any)
.command("push")
.description(
"Push any local changes and apply them remotely. Use --raw for usage without local state tracking."
"Push any local changes and apply them remotely. Use --raw for usage without local state tracking.",
)
.option(
"--fail-conflicts",
"Error on conflicts (both remote and local have changes on the same item)"
)
.option("--skip-pull", "Push without pulling first (you have pulled prior)")
.option("--fail-conflicts", "Error on conflicts (both remote and local have changes on the same item)")
.option("--skip-pull", "Push without pulling first")
.option("--yes", "Push without needing confirmation")
.option("--raw", "Push without using state, just overwrite.")
.option("--plain-secrets", "Push secrets as plain text")
.action(push as any);
export default command;

View File

@@ -11,11 +11,7 @@ import { AppFile } from "./apps.ts";
// TODO: Remove this & replace with a "pull" that lets the object either pull the remote version or return undefined.
// Then combine those with diffing, which then gives the new push impl
export interface Resource {
push(
workspace: string,
remotePath: string,
plainSecrets?: boolean
): Promise<void>;
push(workspace: string, remotePath: string): Promise<void>;
}
export interface PushDiffs {
@@ -23,7 +19,6 @@ export interface PushDiffs {
workspace: string,
remotePath: string,
diffs: Difference[],
plainSecrets?: boolean
): Promise<void>;
}
@@ -51,7 +46,7 @@ export type Difference = DifferenceCreate | DifferenceRemove | DifferenceChange;
export function setValueByPath(
obj: any,
path: (string | number)[],
value: any
value: any,
) {
let i;
let lastObj = undefined;
@@ -89,7 +84,7 @@ export type GlobalOptions = {
export function inferTypeFromPath(
p: string,
obj: any
obj: any,
):
| ScriptFile
| VariableFile
@@ -120,7 +115,7 @@ export function inferTypeFromPath(
}
export function getTypeStrFromPath(
p: string
p: string,
):
| "script"
| "variable"
@@ -130,13 +125,8 @@ export function getTypeStrFromPath(
| "folder"
| "app" {
const parsed = path.parse(p);
if (
parsed.ext == ".go" ||
parsed.ext == ".ts" ||
parsed.ext == ".sh" ||
parsed.ext == ".py"
) {
return "script";
if (parsed.ext == ".go" || parsed.ext == ".ts" || parsed.ext == ".sh" || parsed.ext == ".py") {
return 'script'
}
if (parsed.name === "folder.meta") {
@@ -145,12 +135,9 @@ export function getTypeStrFromPath(
const typeEnding = parsed.name.split(".").at(-1);
if (
typeEnding === "script" ||
typeEnding === "variable" ||
typeEnding === "flow" ||
typeEnding === "resource" ||
typeEnding === "resource-type" ||
typeEnding === "app"
typeEnding === "script" || typeEnding === "variable" ||
typeEnding === "flow" || typeEnding === "resource" ||
typeEnding === "resource-type" || typeEnding === "app"
) {
return typeEnding;
} else {

View File

@@ -11,6 +11,7 @@ import {
colors,
Command,
EditVariable,
ListableVariable,
microdiff,
Table,
VariableService,
@@ -35,7 +36,7 @@ async function list(opts: GlobalOptions) {
x.is_secret ? "true" : "false",
x.account ?? "-",
x.value ?? "-",
])
]),
)
.render();
}
@@ -62,31 +63,24 @@ export class VariableFile implements Resource, PushDiffs {
workspace: string,
remotePath: string,
diffs: Difference[],
plainSecrets?: boolean
): Promise<void> {
if (await VariableService.existsVariable({ workspace, path: remotePath })) {
console.log(
colors.bold.yellow(
`Applying ${diffs.length} diffs to existing variable... ${remotePath}`
)
`Applying ${diffs.length} diffs to existing variable...`,
),
);
const changeset: EditVariable = {};
for (const diff of diffs) {
if (
diff.type !== "REMOVE" &&
(diff.path.length !== 1 ||
![
"path",
"value",
"is_secret",
"description",
"account",
"is_oauth",
].includes(diff.path[0] as string))
(
diff.path.length !== 1 ||
!["path", "value", "is_secret", "description"].includes(
diff.path[0] as string,
)
)
) {
console.log(
colors.red("Invalid variable diff with path " + diff.path)
);
throw new Error("Invalid variable diff with path " + diff.path);
}
if (diff.type === "CREATE" || diff.type === "CHANGE") {
@@ -96,24 +90,25 @@ export class VariableFile implements Resource, PushDiffs {
}
}
const hasChanges = Object.values(changeset).some(
(v) => v !== null && typeof v !== "undefined"
const hasChanges = Object.values(changeset).some((v) =>
v !== null && typeof v !== "undefined"
);
if (!hasChanges) {
return;
}
await VariableService.updateVariable({
workspace,
path: remotePath,
alreadyEncrypted: !plainSecrets,
alreadyEncrypted: true,
requestBody: changeset,
});
console.log(changeset);
} else {
console.log(colors.yellow.bold("Creating new variable..."));
await VariableService.createVariable({
workspace,
alreadyEncrypted: !plainSecrets,
alreadyEncrypted: true,
requestBody: {
path: remotePath,
description: this.description,
@@ -125,29 +120,29 @@ export class VariableFile implements Resource, PushDiffs {
});
}
}
async push(
workspace: string,
remotePath: string,
plainSecrets?: boolean
): Promise<void> {
async push(workspace: string, remotePath: string): Promise<void> {
let existing: ListableVariable | undefined;
try {
existing = await VariableService.getVariable({
workspace: workspace,
path: remotePath,
});
} catch {
existing = undefined;
}
await this.pushDiffs(
workspace,
remotePath,
microdiff({}, this, { cyclesFix: false }),
plainSecrets
microdiff(existing ?? {}, this, { cyclesFix: false }),
);
}
}
async function push(
opts: GlobalOptions & { plainSecrets: boolean },
filePath: string,
remotePath: string
) {
async function push(opts: GlobalOptions, filePath: string, remotePath: string) {
const workspace = await resolveWorkspace(opts);
await requireLogin(opts);
if (!validatePath(remotePath)) {
if (!await validatePath(opts, remotePath)) {
return;
}
@@ -158,12 +153,7 @@ async function push(
console.log(colors.bold.yellow("Pushing variable..."));
await pushVariable(
workspace.workspaceId,
filePath,
remotePath,
opts.plainSecrets
);
await pushVariable(workspace.workspaceId, filePath, remotePath);
console.log(colors.bold.underline.green(`Variable ${remotePath} pushed`));
}
@@ -171,12 +161,11 @@ export async function pushVariable(
workspace: string,
filePath: string,
remotePath: string,
plainSecrets: boolean
) {
const data = decoverto
.type(VariableFile)
.rawToInstance(await Deno.readTextFile(filePath));
await data.push(workspace, remotePath, plainSecrets);
const data = decoverto.type(VariableFile).rawToInstance(
await Deno.readTextFile(filePath),
);
await data.push(workspace, remotePath);
}
const command = new Command()
@@ -184,10 +173,9 @@ const command = new Command()
.action(list as any)
.command(
"push",
"Push a local variable spec. This overrides any remote versions."
"Push a local variable spec. This overrides any remote versions.",
)
.arguments("<file_path:string> <remote_path:string>")
.option("--plain-secrets", "Push secrets as plain text")
.action(push as any);
export default command;

View File

@@ -30,7 +30,7 @@ export class Workspace {
remote: string,
workspaceId: string,
name: string,
token: string
token: string,
) {
this.remote = remote;
this.workspaceId = workspaceId;
@@ -40,7 +40,7 @@ export class Workspace {
}
function makeWorkspaceStream(
readable: ReadableStream<Uint8Array>
readable: ReadableStream<Uint8Array>,
): ReadableStream<Workspace> {
return readable
.pipeThrough(new DelimiterStream(new TextEncoder().encode("\n")))
@@ -59,7 +59,7 @@ function makeWorkspaceStream(
/* ignore */
}
},
})
}),
);
}
@@ -86,7 +86,7 @@ export async function allWorkspaces(): Promise<Workspace[]> {
}
async function getActiveWorkspaceName(
opts: GlobalOptions
opts: GlobalOptions,
): Promise<string | undefined> {
if (opts.workspace) {
return opts.workspace;
@@ -99,7 +99,7 @@ async function getActiveWorkspaceName(
}
export async function getActiveWorkspace(
opts: GlobalOptions
opts: GlobalOptions,
): Promise<Workspace | undefined> {
const name = await getActiveWorkspaceName(opts);
if (!name) {
@@ -109,7 +109,7 @@ export async function getActiveWorkspace(
}
export async function getWorkspaceByName(
workspaceName: string
workspaceName: string,
): Promise<Workspace | undefined> {
const workspaceStream = await getWorkspaceStream();
for await (const workspace of workspaceStream) {
@@ -136,40 +136,36 @@ async function list(opts: GlobalOptions) {
} else {
return a;
}
})
}),
)
.render();
console.log("Active: " + colors.green.bold(activeName || "none"));
console.log('Active: ' + colors.green.bold(activeName || 'none'))
}
async function switchC(opts: GlobalOptions, workspaceName: string) {
if (opts.workspace) {
console.log(
colors.red.bold(
"! Workspace needs to be specified as positional argument, not as option."
)
"! Workspace needs to be specified as positional argument, not as option.",
),
);
return;
}
const all = await allWorkspaces();
if (all.findIndex((x) => x.name === workspaceName) === -1) {
console.log(
colors.red.bold(
`! This workspace profile ${workspaceName} does not exist locally.`
)
);
console.log("available workspace profiles:");
console.log(colors.red.bold(`! This workspace profile ${workspaceName} does not exist locally.`));
console.log("available workspace profiles:")
for (const w of all) {
console.log(" - " + w.name);
console.log(' - ' + w.name)
}
return;
}
return await Deno.writeTextFile(
(await getRootStore()) + "/activeWorkspace",
workspaceName
workspaceName,
);
}
@@ -181,13 +177,13 @@ export async function add(
},
workspaceName: string | undefined,
workspaceId: string | undefined,
remote: string | undefined
remote: string | undefined,
) {
if (opts.workspace) {
console.log(
colors.red.bold(
"! Workspace needs to be specified as positional argument, not as option."
)
"! Workspace needs to be specified as positional argument, not as option.",
),
);
return;
}
@@ -197,11 +193,7 @@ export async function add(
}
if (!workspaceId) {
workspaceId = await Input.prompt({
message: "Enter the ID of this workspace",
default: workspaceName,
suggestions: [workspaceName],
});
workspaceId = await Input.prompt({ message: "Enter the ID of this workspace", default: workspaceName, suggestions: [workspaceName] });
}
if (!remote) {
@@ -217,8 +209,8 @@ export async function add(
await Input.prompt({
message: "Enter the Remote URL",
suggestions: ["https://app.windmill.dev/"],
default: "https://app.windmill.dev/",
})
default: "https://app.windmill.dev/"
}),
).toString();
}
}
@@ -231,26 +223,22 @@ export async function add(
setClient(
token,
remote.endsWith("/") ? remote.substring(0, remote.length - 1) : remote
remote.endsWith("/") ? remote.substring(0, remote.length - 1) : remote,
);
let alreadyExists = false;
let alreadyExists = false
try {
alreadyExists = await WorkspaceService.existsWorkspace({
requestBody: { id: workspaceId },
});
})
} catch (e) {
console.log(
colors.red.bold("! Credentials or instance is invalid. Aborting.")
);
throw e;
console.log(colors.red.bold("! Credentials or instance is invalid. Aborting."));
throw e
}
if (opts.create) {
if (!alreadyExists) {
console.log(
colors.yellow(
`Workspace at id ${workspaceId} on ${remote} does not exist. Creating...`
)
);
if (
!alreadyExists
) {
console.log(colors.yellow(`Workspace at id ${workspaceId} on ${remote} does not exist. Creating...`));
await WorkspaceService.createWorkspace({
requestBody: {
id: workspaceId,
@@ -260,87 +248,63 @@ export async function add(
});
}
} else if (!alreadyExists) {
console.log(
colors.red.bold(
`! Workspace at id ${workspaceId} on ${remote} does not exist. Re-run with --create to create it. Aborting.`
)
);
console.log(
"On that instance and with those credentials, the workspaces that you can access are:"
);
const workspaces = await WorkspaceService.listWorkspaces();
console.log(colors.red.bold(`! Workspace at id ${workspaceId} on ${remote} does not exist. Re-run with --create to create it. Aborting.`));
console.log("On that instance and with those credentials, the workspaces that you can access are:")
const workspaces = await WorkspaceService.listWorkspaces()
for (const workspace of workspaces) {
console.log(`- ${workspace.id} (name: ${workspace.name})`);
console.log(`- ${workspace.id} (name: ${workspace.name})`)
}
Deno.exit(1);
}
await addWorkspace(
{
name: workspaceName,
remote: remote,
workspaceId: workspaceId,
token: token,
},
opts
);
await addWorkspace({
name: workspaceName,
remote: remote,
workspaceId: workspaceId,
token: token,
}, opts);
await Deno.writeTextFile(
(await getRootStore()) + "/activeWorkspace",
workspaceName
);
console.log(
colors.green.underline(
`Added workspace ${workspaceName} for ${workspaceId} on ${remote}!`
)
workspaceName,
);
console.log(colors.green.underline(`Added workspace ${workspaceName} for ${workspaceId} on ${remote}!`));
}
export async function addWorkspace(workspace: Workspace, opts: any) {
workspace.remote = new URL(workspace.remote).toString(); // add trailing slash in all cases!
await removeWorkspace(workspace.name, true, opts);
const file = await Deno.open((await getRootStore()) + "remotes.ndjson", {
append: true,
write: true,
read: true,
create: true,
});
await removeWorkspace(workspace.name, true, opts);
await file.write(new TextEncoder().encode(JSON.stringify(workspace) + "\n"));
file.close();
}
export async function removeWorkspace(
name: string,
silent: boolean,
opts: any
) {
export async function removeWorkspace(name: string, silent: boolean, opts: any) {
const orgWorkspaces = await allWorkspaces();
if (orgWorkspaces.findIndex((x) => x.name === name) === -1) {
if (!silent) {
console.log(
colors.red.bold(`! Workspace profile ${name} does not exist locally`)
);
console.log("available workspace profiles:");
await list(opts);
console.log(colors.red.bold(`! Workspace profile ${name} does not exist locally`));
console.log("available workspace profiles:")
await list(opts)
}
return;
}
if (silent) {
console.log(colors.yellow(`Replacing existing workspace ${name}`));
console.log(colors.yellow(`Replacing existing workspace ${name}`))
}
await Deno.writeTextFile(
(await getRootStore()) + "remotes.ndjson",
orgWorkspaces
.filter((x) => x.name !== name)
.map((x) => JSON.stringify(x))
.join("\n") + "\n"
.join("\n"),
);
if (!silent) {
console.log(
colors.green.underline(`Succesfully removed workspace ${name}!`)
);
console.log(colors.green.underline(`Succesfully removed workspace ${name}!`));
}
}
@@ -349,10 +313,10 @@ async function remove(_opts: GlobalOptions, name: string) {
}
async function whoami(_opts: GlobalOptions) {
await requireLogin(_opts);
console.log(await UserService.globalWhoami());
await requireLogin(_opts)
console.log(await UserService.globalWhoami())
const activeName = await getActiveWorkspaceName(_opts);
console.log("Active: " + colors.green.bold(activeName || "none"));
console.log('Active: ' + colors.green.bold(activeName || 'none'))
}
const command = new Command()
@@ -369,14 +333,14 @@ const command = new Command()
.option("-c --create", "Create the workspace if it does not exist")
.option(
"--create-workspace-name <workspace_name:string>",
"Specify the workspace name. Ignored if --create is not specified or the workspace already exists. Will default to the workspace id."
"Specify the workspace name. Ignored if --create is not specified or the workspace already exists. Will default to the workspace id.",
)
.option(
"--create-username <username:string>",
"Specify your own username in the newly created workspace. Ignored if --create is not specified or the workspace already exists.",
{
default: "admin",
}
},
)
.action(add as any)
.command("remove")

View File

@@ -1,59 +1,41 @@
import {
JobService,
ResourceService,
VariableService,
} from "./windmill-api/index.ts";
import { OpenAPI } from "./windmill-api/index.ts";
import { JobService, ResourceService, VariableService } from './windmill-api/index.ts'
import { OpenAPI } from './windmill-api/index.ts'
export {
AdminService,
AuditService,
FlowService,
GranularAclService,
GroupService,
JobService,
ResourceService,
VariableService,
ScriptService,
ScheduleService,
SettingsService,
UserService,
WorkspaceService,
} from "./windmill-api/index.ts";
AdminService, AuditService, FlowService, GranularAclService, GroupService,
JobService, ResourceService, VariableService, ScriptService, ScheduleService, SettingsService,
UserService, WorkspaceService
} from './windmill-api/index.ts'
// @ts-ignore: Otherwise BigInt is not supported for export
BigInt.prototype.toJSON = function () {
return this.toString();
return this.toString();
};
export { pgSql, pgClient } from "./pg.ts";
export { pgSql, pgClient } from './pg.ts'
export type Sql = string;
export type Email = string;
export type Base64 = string;
export type Resource<S extends string> = any;
export type Sql = string
export type Email = string
export type Base64 = string
export type Resource<S extends string> = any
export const SHARED_FOLDER = "/shared";
export const SHARED_FOLDER = '/shared'
export function setClient(token: string, baseUrl: string) {
OpenAPI.WITH_CREDENTIALS = true;
OpenAPI.TOKEN = token;
OpenAPI.BASE = baseUrl + "/api";
OpenAPI.WITH_CREDENTIALS = true
OpenAPI.TOKEN = token
OpenAPI.BASE = baseUrl + '/api'
}
setClient(
Deno.env.get("WM_TOKEN") ?? "no_token",
Deno.env.get("BASE_INTERNAL_URL") ??
Deno.env.get("BASE_URL") ??
"http://localhost:8000"
);
setClient(Deno.env.get("WM_TOKEN") ?? 'no_token', Deno.env.get("BASE_INTERNAL_URL") ?? Deno.env.get("BASE_URL") ?? 'http://localhost:8000')
/**
* Create a client configuration from env variables
* @returns client configuration
*/
export function getWorkspace(): string {
return Deno.env.get("WM_WORKSPACE") ?? "no_workspace";
return Deno.env.get("WM_WORKSPACE") ?? 'no_workspace'
}
/**
@@ -62,30 +44,27 @@ export function getWorkspace(): string {
* @param undefinedIfEmpty if the resource does not exist, return undefined instead of throwing an error
* @returns resource value
*/
export async function getResource(
path?: string,
undefinedIfEmpty?: boolean
): Promise<any> {
const workspace = getWorkspace();
path = path ?? getStatePath();
try {
const resource = await ResourceService.getResource({ workspace, path });
return await _transformLeaf(resource.value);
} catch (e: any) {
if (undefinedIfEmpty && e.status === 404) {
return undefined;
} else {
throw Error(`Resource not found at ${path} or not visible to you`);
export async function getResource(path?: string, undefinedIfEmpty?: boolean): Promise<any> {
const workspace = getWorkspace()
path = path ?? getStatePath()
try {
const resource = await ResourceService.getResource({ workspace, path })
return await _transformLeaf(resource.value)
} catch (e: any) {
if (undefinedIfEmpty && e.status === 404) {
return undefined
} else {
throw Error(`Resource not found at ${path} or not visible to you`)
}
}
}
}
export function getStatePath(): string {
const state_path = Deno.env.get("WM_STATE_PATH");
if (state_path === undefined) {
throw Error("State path not set");
}
return state_path;
const state_path = Deno.env.get("WM_STATE_PATH")
if (state_path === undefined) {
throw Error("State path not set")
}
return state_path
}
/**
@@ -94,29 +73,16 @@ export function getStatePath(): string {
* @param value new value of the resource to set
* @param initializeToTypeIfNotExist if the resource does not exist, initialize it with this type
*/
export async function setResource(
value: any,
path?: string,
initializeToTypeIfNotExist?: string
): Promise<void> {
path = path ?? getStatePath();
const workspace = getWorkspace();
if (await ResourceService.existsResource({ workspace, path })) {
await ResourceService.updateResourceValue({
workspace,
path,
requestBody: { value },
});
} else if (initializeToTypeIfNotExist) {
await ResourceService.createResource({
workspace,
requestBody: { path, value, resource_type: initializeToTypeIfNotExist },
});
} else {
throw Error(
`Resource at path ${path} does not exist and no type was provided to initialize it`
);
}
export async function setResource(value: any, path?: string, initializeToTypeIfNotExist?: string): Promise<void> {
path = path ?? getStatePath()
const workspace = getWorkspace()
if (await ResourceService.existsResource({ workspace, path })) {
await ResourceService.updateResourceValue({ workspace, path, requestBody: { value } })
} else if (initializeToTypeIfNotExist) {
await ResourceService.createResource({ workspace, requestBody: { path, value, resource_type: initializeToTypeIfNotExist } })
} else {
throw Error(`Resource at path ${path} does not exist and no type was provided to initialize it`)
}
}
/**
@@ -125,7 +91,7 @@ export async function setResource(
* @deprecated use setState instead
*/
export async function setInternalState(state: any): Promise<void> {
await setResource(state, undefined, "state");
await setResource(state, undefined, 'state')
}
/**
@@ -133,41 +99,39 @@ export async function setInternalState(state: any): Promise<void> {
* @param state state to set
*/
export async function setState(state: any): Promise<void> {
await setResource(state, undefined, "state");
await setResource(state, undefined, 'state')
}
/**
* Set the shared state
* @param state state to set
*/
export async function setSharedState(
state: any,
path = "state.json"
): Promise<void> {
await Deno.writeTextFile(SHARED_FOLDER + "/" + path, JSON.stringify(state));
export async function setSharedState(state: any, path = 'state.json'): Promise<void> {
await Deno.writeTextFile(SHARED_FOLDER + '/' + path, JSON.stringify(state))
}
/**
* Get the shared state
* @param state state to set
*/
export async function getSharedState(path = "state.json"): Promise<any> {
return JSON.parse(await Deno.readTextFile(SHARED_FOLDER + "/" + path));
export async function getSharedState(path = 'state.json'): Promise<any> {
return JSON.parse(await Deno.readTextFile(SHARED_FOLDER + '/' + path))
}
/**
* Get the internal state
* @deprecated use getState instead
*/
export async function getInternalState(): Promise<any> {
return await getResource(getStatePath(), true);
return await getResource(getStatePath(), true)
}
/**
* Get the state shared across executions
*/
export async function getState(): Promise<any> {
return await getResource(getStatePath(), true);
return await getResource(getStatePath(), true)
}
/**
@@ -176,43 +140,38 @@ export async function getState(): Promise<any> {
* @returns variable value
*/
export async function getVariable(path: string): Promise<string | undefined> {
const workspace = getWorkspace();
try {
const variable = await VariableService.getVariable({ workspace, path });
return variable.value;
} catch (e: any) {
throw Error(`Variable not found at ${path} or not visible to you`);
}
const workspace = getWorkspace()
try {
const variable = await VariableService.getVariable({ workspace, path })
return variable.value
} catch (e: any) {
throw Error(`Variable not found at ${path} or not visible to you`)
}
}
async function transformLeaves(d: {
[key: string]: any;
}): Promise<{ [key: string]: any }> {
for (const k in d) {
d[k] = await _transformLeaf(d[k]);
}
return d;
async function transformLeaves(d: { [key: string]: any }): Promise<{ [key: string]: any }> {
for (const k in d) {
d[k] = await _transformLeaf(d[k])
}
return d
}
const VAR_RESOURCE_PREFIX = "$var:";
const RES_RESOURCE_PREFIX = "$res:";
const VAR_RESOURCE_PREFIX = "$var:"
async function _transformLeaf(v: any): Promise<any> {
if (typeof v === "object") {
return transformLeaves(v);
} else if (typeof v === "string" && v.startsWith(VAR_RESOURCE_PREFIX)) {
const varName = v.substring(VAR_RESOURCE_PREFIX.length);
return await getVariable(varName);
} else if (typeof v === "string" && v.startsWith(RES_RESOURCE_PREFIX)) {
const resName = v.substring(RES_RESOURCE_PREFIX.length);
return await getResource(resName);
} else {
return v;
}
if (typeof v === 'object') {
return transformLeaves(v)
}
else if (typeof v === 'string' && v.startsWith(VAR_RESOURCE_PREFIX)) {
const varName = v.substring(VAR_RESOURCE_PREFIX.length)
return await getVariable(varName)
} else {
return v
}
}
export async function databaseUrlFromResource(path: string): Promise<string> {
const resource = await getResource(path);
return `postgresql://${resource.user}:${resource.password}@${resource.host}:${resource.port}/${resource.dbname}?sslmode=${resource.sslmode}`;
const resource = await getResource(path)
return `postgresql://${resource.user}:${resource.password}@${resource.host}:${resource.port}/${resource.dbname}?sslmode=${resource.sslmode}`
}
/**
@@ -221,84 +180,78 @@ export async function databaseUrlFromResource(path: string): Promise<string> {
* @returns approval page UI URL, resume and cancel API URLs for resumeing the flow
*/
export async function getResumeUrls(approver?: string): Promise<{
approvalPage: string;
resume: string;
cancel: string;
approvalPage: string;
resume: string;
cancel: string;
}> {
const nonce = Math.floor(Math.random() * 4294967295);
const workspace = getWorkspace();
return await JobService.getResumeUrls({
workspace,
resumeId: nonce,
approver,
id: Deno.env.get("WM_JOB_ID") ?? "NO_JOB_ID",
});
const nonce = Math.floor(Math.random() * 4294967295);
const workspace = getWorkspace()
return await JobService.getResumeUrls({ workspace, resumeId: nonce, approver, id: Deno.env.get("WM_JOB_ID") ?? 'NO_JOB_ID' })
}
/**
* @deprecated use getResumeUrls instead
*/
export function getResumeEndpoints(approver?: string): Promise<{
approvalPage: string;
resume: string;
cancel: string;
approvalPage: string;
resume: string;
cancel: string;
}> {
return getResumeUrls(approver);
return getResumeUrls(approver)
}
export function base64ToUint8Array(data: string): Uint8Array {
return Uint8Array.from(atob(data), (c) => c.charCodeAt(0));
return Uint8Array.from(atob(data), c => c.charCodeAt(0))
}
export function uint8ArrayToBase64(arrayBuffer: Uint8Array): string {
let base64 = "";
const encodings =
"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
let base64 = ''
const encodings = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/'
const bytes = new Uint8Array(arrayBuffer);
const byteLength = bytes.byteLength;
const byteRemainder = byteLength % 3;
const mainLength = byteLength - byteRemainder;
const bytes = new Uint8Array(arrayBuffer)
const byteLength = bytes.byteLength
const byteRemainder = byteLength % 3
const mainLength = byteLength - byteRemainder
let a, b, c, d;
let chunk;
let a, b, c, d
let chunk
// Main loop deals with bytes in chunks of 3
for (let i = 0; i < mainLength; i = i + 3) {
// Combine the three bytes into a single integer
chunk = (bytes[i] << 16) | (bytes[i + 1] << 8) | bytes[i + 2];
// Main loop deals with bytes in chunks of 3
for (let i = 0; i < mainLength; i = i + 3) {
// Combine the three bytes into a single integer
chunk = (bytes[i] << 16) | (bytes[i + 1] << 8) | bytes[i + 2]
// Use bitmasks to extract 6-bit segments from the triplet
a = (chunk & 16515072) >> 18; // 16515072 = (2^6 - 1) << 18
b = (chunk & 258048) >> 12; // 258048 = (2^6 - 1) << 12
c = (chunk & 4032) >> 6; // 4032 = (2^6 - 1) << 6
d = chunk & 63; // 63 = 2^6 - 1
// Use bitmasks to extract 6-bit segments from the triplet
a = (chunk & 16515072) >> 18 // 16515072 = (2^6 - 1) << 18
b = (chunk & 258048) >> 12 // 258048 = (2^6 - 1) << 12
c = (chunk & 4032) >> 6 // 4032 = (2^6 - 1) << 6
d = chunk & 63 // 63 = 2^6 - 1
// Convert the raw binary segments to the appropriate ASCII encoding
base64 += encodings[a] + encodings[b] + encodings[c] + encodings[d];
}
// Convert the raw binary segments to the appropriate ASCII encoding
base64 += encodings[a] + encodings[b] + encodings[c] + encodings[d]
}
// Deal with the remaining bytes and padding
if (byteRemainder == 1) {
chunk = bytes[mainLength];
// Deal with the remaining bytes and padding
if (byteRemainder == 1) {
chunk = bytes[mainLength]
a = (chunk & 252) >> 2; // 252 = (2^6 - 1) << 2
a = (chunk & 252) >> 2 // 252 = (2^6 - 1) << 2
// Set the 4 least significant bits to zero
b = (chunk & 3) << 4; // 3 = 2^2 - 1
// Set the 4 least significant bits to zero
b = (chunk & 3) << 4 // 3 = 2^2 - 1
base64 += encodings[a] + encodings[b] + "==";
} else if (byteRemainder == 2) {
chunk = (bytes[mainLength] << 8) | bytes[mainLength + 1];
base64 += encodings[a] + encodings[b] + '=='
} else if (byteRemainder == 2) {
chunk = (bytes[mainLength] << 8) | bytes[mainLength + 1]
a = (chunk & 64512) >> 10; // 64512 = (2^6 - 1) << 10
b = (chunk & 1008) >> 4; // 1008 = (2^6 - 1) << 4
a = (chunk & 64512) >> 10 // 64512 = (2^6 - 1) << 10
b = (chunk & 1008) >> 4 // 1008 = (2^6 - 1) << 4
// Set the 2 least significant bits to zero
c = (chunk & 15) << 2; // 15 = 2^4 - 1
// Set the 2 least significant bits to zero
c = (chunk & 15) << 2 // 15 = 2^4 - 1
base64 += encodings[a] + encodings[b] + encodings[c] + "=";
}
base64 += encodings[a] + encodings[b] + encodings[c] + '='
}
return base64;
return base64
}

View File

@@ -1 +0,0 @@
VITE_DEFAULT_WEBHOOK_TYPE=async

1
frontend/.gitignore vendored
View File

@@ -7,4 +7,3 @@ node_modules
CaddyfileRemoteRuben
tests-out/
storageState.json
.env.production

View File

@@ -17,7 +17,7 @@ In the root folder:
```bash
docker build . -t windmill
docker compose up db windmill_server windmill_worker
docker-compose up db windmill_server windmill_worker
```
### 2. Backend is run by cargo
@@ -145,25 +145,3 @@ Recommended config for VS Code:
```
- turn _format on save_ on
## Building
The project is built with [SvelteKit](https://kit.svelte.dev/) and uses as output static files.
There are others adapters for sveltekit, but we use the static adapter.
To build the frontend as static assets, use:
```
npm run build
```
The output is in the `build` folder.
The default build assume you serve every non static files as the 200.html file which is catchall. If you prefer a normal layout, you can use:
```
NOTCATCHALL=true npm run build
```
which will generate an index.html and allow you to serve the frontend with any static server.
Env variables used for build are set in .env file. See [https://vitejs.dev/guide/env-and-mode.html#env-files](https://vitejs.dev/guide/env-and-mode.html#env-files) for more details.

View File

@@ -1,12 +1,12 @@
{
"name": "windmill",
"version": "1.83.1",
"version": "1.74.2",
"lockfileVersion": 2,
"requires": true,
"packages": {
"": {
"name": "windmill",
"version": "1.83.1",
"version": "1.74.2",
"dependencies": {
"@fortawesome/free-brands-svg-icons": "^6.2.1",
"@fortawesome/free-solid-svg-icons": "^6.2.1",
@@ -23,7 +23,6 @@
"date-fns": "^2.29.3",
"fast-equals": "^4.0.3",
"highlight.js": "^11.7.0",
"lodash": "^4.17.21",
"lucide-svelte": "^0.115.0",
"monaco-editor": "0.33.0",
"monaco-editor-workers": "0.33.0",
@@ -45,11 +44,11 @@
"@tailwindcss/typography": "^0.5.8",
"@types/d3": "^7.4.0",
"@types/d3-zoom": "^3.0.2",
"@types/lodash": "^4.14.191",
"@types/node": "^18.11.18",
"@types/vscode": "~1.74.0",
"@typescript-eslint/eslint-plugin": "^5.49.0",
"@typescript-eslint/parser": "^5.48.0",
"@windmill-labs/svelte-grid": "^5.1.6",
"@zerodevx/svelte-toast": "^0.8.1",
"autoprefixer": "^10.4.13",
"cssnano": "^5.1.14",
@@ -65,8 +64,8 @@
"postcss-load-config": "^4.0.1",
"prettier": "^2.8.3",
"prettier-plugin-svelte": "^2.9.0",
"shepherd.js": "^10.0.1",
"simple-svelte-autocomplete": "^2.5.1",
"style-to-object": "^0.4.1",
"stylelint-config-recommended": "^9.0.0",
"svelte": "^3.55.1",
"svelte-awesome": "^3.0.0",
@@ -78,7 +77,7 @@
"svelte-preprocess": "^5.0.1",
"svelte-range-slider-pips": "^2.1.1",
"svelte-splitpanes": "^0.7.3",
"svelte2tsx": "^0.6.10",
"svelte2tsx": "^0.6.1",
"tailwindcss": "^3.2.2",
"tslib": "^2.4.1",
"typescript": "^4.9.3",
@@ -1288,12 +1287,6 @@
"resolved": "https://registry.npmjs.org/@types/json-schema/-/json-schema-7.0.11.tgz",
"integrity": "sha512-wOuvG1SN4Us4rez+tylwwwCV1psiNVOkJeM3AUWUNWg/jDQY2+HE/444y5gc+jBmRqASOm2Oeh5c1axHobwRKQ=="
},
"node_modules/@types/lodash": {
"version": "4.14.191",
"resolved": "https://registry.npmjs.org/@types/lodash/-/lodash-4.14.191.tgz",
"integrity": "sha512-BdZ5BCCvho3EIXw6wUCXHe7rS53AIDPLE+JzwgT+OsJk53oBfbSmZZ7CX4VaRoN78N+TJpFi9QPlfIVNmJYWxQ==",
"dev": true
},
"node_modules/@types/minimist": {
"version": "1.2.2",
"resolved": "https://registry.npmjs.org/@types/minimist/-/minimist-1.2.2.tgz",
@@ -1609,6 +1602,12 @@
"url": "https://opencollective.com/typescript-eslint"
}
},
"node_modules/@windmill-labs/svelte-grid": {
"version": "5.1.6",
"resolved": "https://registry.npmjs.org/@windmill-labs/svelte-grid/-/svelte-grid-5.1.6.tgz",
"integrity": "sha512-wXHIG7XeMxwxnfaHajhjvz9c0u5uGgiQtTk/QxAVyViGdshsclfIfIWZF113QQBKCKp2jY/zP1cj64BrqioRoQ==",
"dev": true
},
"node_modules/@zeit/schemas": {
"version": "2.21.0",
"resolved": "https://registry.npmjs.org/@zeit/schemas/-/schemas-2.21.0.tgz",
@@ -4080,12 +4079,6 @@
"integrity": "sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew==",
"dev": true
},
"node_modules/inline-style-parser": {
"version": "0.1.1",
"resolved": "https://registry.npmjs.org/inline-style-parser/-/inline-style-parser-0.1.1.tgz",
"integrity": "sha512-7NXolsK4CAS5+xvdj5OMMbI962hU/wvwoxk+LWR9Ek9bVtyuuYScDN6eS0rUm6TxApFpw7CX1o4uJzcd4AyD3Q==",
"dev": true
},
"node_modules/internmap": {
"version": "2.0.3",
"resolved": "https://registry.npmjs.org/internmap/-/internmap-2.0.3.tgz",
@@ -4416,11 +4409,6 @@
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/lodash": {
"version": "4.17.21",
"resolved": "https://registry.npmjs.org/lodash/-/lodash-4.17.21.tgz",
"integrity": "sha512-v2kDEe57lecTulaDIuNTPy3Ry4gLGJ6Z1O3vE1krgXZNrsQ+LFTGHVxVjcXPs17LhbZVGedAJv8XZ1tvj5FvSg=="
},
"node_modules/lodash.castarray": {
"version": "4.4.0",
"resolved": "https://registry.npmjs.org/lodash.castarray/-/lodash.castarray-4.4.0.tgz",
@@ -6666,6 +6654,24 @@
"node": ">=8"
}
},
"node_modules/shepherd.js": {
"version": "10.0.1",
"resolved": "https://registry.npmjs.org/shepherd.js/-/shepherd.js-10.0.1.tgz",
"integrity": "sha512-R32v4b4b0N1gK/vxvRtdhty+SBZlBPcPTSYCwcaAQvFd0n6Xki7cRH6Sx0oD9WB/HCkqCNM1msyIyylXmq635w==",
"dev": true,
"dependencies": {
"@popperjs/core": "^2.11.5",
"deepmerge": "^4.2.2",
"smoothscroll-polyfill": "^0.4.4"
},
"engines": {
"node": "12.* || 14.* || >= 16"
},
"funding": {
"type": "individual",
"url": "https://github.com/sponsors/rwwagner90"
}
},
"node_modules/signal-exit": {
"version": "3.0.7",
"resolved": "https://registry.npmjs.org/signal-exit/-/signal-exit-3.0.7.tgz",
@@ -6752,6 +6758,12 @@
"url": "https://github.com/chalk/slice-ansi?sponsor=1"
}
},
"node_modules/smoothscroll-polyfill": {
"version": "0.4.4",
"resolved": "https://registry.npmjs.org/smoothscroll-polyfill/-/smoothscroll-polyfill-0.4.4.tgz",
"integrity": "sha512-TK5ZA9U5RqCwMpfoMq/l1mrH0JAR7y7KRvOBx0n2869aLxch+gT9GhN3yUfjiw+d/DiF1mKo14+hd62JyMmoBg==",
"dev": true
},
"node_modules/sorcery": {
"version": "0.11.0",
"resolved": "https://registry.npmjs.org/sorcery/-/sorcery-0.11.0.tgz",
@@ -6966,15 +6978,6 @@
"dev": true,
"peer": true
},
"node_modules/style-to-object": {
"version": "0.4.1",
"resolved": "https://registry.npmjs.org/style-to-object/-/style-to-object-0.4.1.tgz",
"integrity": "sha512-HFpbb5gr2ypci7Qw+IOhnP2zOU7e77b+rzM+wTzXzfi1PrtBCX0E7Pk4wL4iTLnhzZ+JgEGAhX81ebTg/aYjQw==",
"dev": true,
"dependencies": {
"inline-style-parser": "0.1.1"
}
},
"node_modules/stylehacks": {
"version": "5.1.1",
"resolved": "https://registry.npmjs.org/stylehacks/-/stylehacks-5.1.1.tgz",
@@ -7373,9 +7376,9 @@
}
},
"node_modules/svelte2tsx": {
"version": "0.6.10",
"resolved": "https://registry.npmjs.org/svelte2tsx/-/svelte2tsx-0.6.10.tgz",
"integrity": "sha512-7CtUexhSHppSpLwVX7yI9OxJp7+esCNTuOF/CaVxWTgWp88QF/2/dCbpND/cTZmBxht5cFFQBwoLXXqFLdYe3Q==",
"version": "0.6.1",
"resolved": "https://registry.npmjs.org/svelte2tsx/-/svelte2tsx-0.6.1.tgz",
"integrity": "sha512-O/1+5UyChfmhp1/GUv8b8iveTrn6eZwHxEXc+rw7LMKRidr9KHk5w/EiliLjDUwHa2VA6CoEty+CQylROVU4Sw==",
"dev": true,
"dependencies": {
"dedent-js": "^1.0.1",
@@ -7383,7 +7386,7 @@
},
"peerDependencies": {
"svelte": "^3.55",
"typescript": "^4.9.4 || ^5.0.0"
"typescript": "^4.9.4"
}
},
"node_modules/svg-tags": {
@@ -9111,12 +9114,6 @@
"resolved": "https://registry.npmjs.org/@types/json-schema/-/json-schema-7.0.11.tgz",
"integrity": "sha512-wOuvG1SN4Us4rez+tylwwwCV1psiNVOkJeM3AUWUNWg/jDQY2+HE/444y5gc+jBmRqASOm2Oeh5c1axHobwRKQ=="
},
"@types/lodash": {
"version": "4.14.191",
"resolved": "https://registry.npmjs.org/@types/lodash/-/lodash-4.14.191.tgz",
"integrity": "sha512-BdZ5BCCvho3EIXw6wUCXHe7rS53AIDPLE+JzwgT+OsJk53oBfbSmZZ7CX4VaRoN78N+TJpFi9QPlfIVNmJYWxQ==",
"dev": true
},
"@types/minimist": {
"version": "1.2.2",
"resolved": "https://registry.npmjs.org/@types/minimist/-/minimist-1.2.2.tgz",
@@ -9312,6 +9309,12 @@
"eslint-visitor-keys": "^3.3.0"
}
},
"@windmill-labs/svelte-grid": {
"version": "5.1.6",
"resolved": "https://registry.npmjs.org/@windmill-labs/svelte-grid/-/svelte-grid-5.1.6.tgz",
"integrity": "sha512-wXHIG7XeMxwxnfaHajhjvz9c0u5uGgiQtTk/QxAVyViGdshsclfIfIWZF113QQBKCKp2jY/zP1cj64BrqioRoQ==",
"dev": true
},
"@zeit/schemas": {
"version": "2.21.0",
"resolved": "https://registry.npmjs.org/@zeit/schemas/-/schemas-2.21.0.tgz",
@@ -11143,12 +11146,6 @@
"integrity": "sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew==",
"dev": true
},
"inline-style-parser": {
"version": "0.1.1",
"resolved": "https://registry.npmjs.org/inline-style-parser/-/inline-style-parser-0.1.1.tgz",
"integrity": "sha512-7NXolsK4CAS5+xvdj5OMMbI962hU/wvwoxk+LWR9Ek9bVtyuuYScDN6eS0rUm6TxApFpw7CX1o4uJzcd4AyD3Q==",
"dev": true
},
"internmap": {
"version": "2.0.3",
"resolved": "https://registry.npmjs.org/internmap/-/internmap-2.0.3.tgz",
@@ -11399,11 +11396,6 @@
"p-locate": "^5.0.0"
}
},
"lodash": {
"version": "4.17.21",
"resolved": "https://registry.npmjs.org/lodash/-/lodash-4.17.21.tgz",
"integrity": "sha512-v2kDEe57lecTulaDIuNTPy3Ry4gLGJ6Z1O3vE1krgXZNrsQ+LFTGHVxVjcXPs17LhbZVGedAJv8XZ1tvj5FvSg=="
},
"lodash.castarray": {
"version": "4.4.0",
"resolved": "https://registry.npmjs.org/lodash.castarray/-/lodash.castarray-4.4.0.tgz",
@@ -13022,6 +13014,17 @@
"integrity": "sha512-7++dFhtcx3353uBaq8DDR4NuxBetBzC7ZQOhmTQInHEd6bSrXdiEyzCvG07Z44UYdLShWUyXt5M/yhz8ekcb1A==",
"dev": true
},
"shepherd.js": {
"version": "10.0.1",
"resolved": "https://registry.npmjs.org/shepherd.js/-/shepherd.js-10.0.1.tgz",
"integrity": "sha512-R32v4b4b0N1gK/vxvRtdhty+SBZlBPcPTSYCwcaAQvFd0n6Xki7cRH6Sx0oD9WB/HCkqCNM1msyIyylXmq635w==",
"dev": true,
"requires": {
"@popperjs/core": "^2.11.5",
"deepmerge": "^4.2.2",
"smoothscroll-polyfill": "^0.4.4"
}
},
"signal-exit": {
"version": "3.0.7",
"resolved": "https://registry.npmjs.org/signal-exit/-/signal-exit-3.0.7.tgz",
@@ -13082,6 +13085,12 @@
"is-fullwidth-code-point": "^3.0.0"
}
},
"smoothscroll-polyfill": {
"version": "0.4.4",
"resolved": "https://registry.npmjs.org/smoothscroll-polyfill/-/smoothscroll-polyfill-0.4.4.tgz",
"integrity": "sha512-TK5ZA9U5RqCwMpfoMq/l1mrH0JAR7y7KRvOBx0n2869aLxch+gT9GhN3yUfjiw+d/DiF1mKo14+hd62JyMmoBg==",
"dev": true
},
"sorcery": {
"version": "0.11.0",
"resolved": "https://registry.npmjs.org/sorcery/-/sorcery-0.11.0.tgz",
@@ -13244,15 +13253,6 @@
"dev": true,
"peer": true
},
"style-to-object": {
"version": "0.4.1",
"resolved": "https://registry.npmjs.org/style-to-object/-/style-to-object-0.4.1.tgz",
"integrity": "sha512-HFpbb5gr2ypci7Qw+IOhnP2zOU7e77b+rzM+wTzXzfi1PrtBCX0E7Pk4wL4iTLnhzZ+JgEGAhX81ebTg/aYjQw==",
"dev": true,
"requires": {
"inline-style-parser": "0.1.1"
}
},
"stylehacks": {
"version": "5.1.1",
"resolved": "https://registry.npmjs.org/stylehacks/-/stylehacks-5.1.1.tgz",
@@ -13533,9 +13533,9 @@
}
},
"svelte2tsx": {
"version": "0.6.10",
"resolved": "https://registry.npmjs.org/svelte2tsx/-/svelte2tsx-0.6.10.tgz",
"integrity": "sha512-7CtUexhSHppSpLwVX7yI9OxJp7+esCNTuOF/CaVxWTgWp88QF/2/dCbpND/cTZmBxht5cFFQBwoLXXqFLdYe3Q==",
"version": "0.6.1",
"resolved": "https://registry.npmjs.org/svelte2tsx/-/svelte2tsx-0.6.1.tgz",
"integrity": "sha512-O/1+5UyChfmhp1/GUv8b8iveTrn6eZwHxEXc+rw7LMKRidr9KHk5w/EiliLjDUwHa2VA6CoEty+CQylROVU4Sw==",
"dev": true,
"requires": {
"dedent-js": "^1.0.1",

View File

@@ -1,6 +1,6 @@
{
"name": "windmill",
"version": "1.83.1",
"version": "1.74.2",
"scripts": {
"dev": "vite dev",
"build": "vite build",
@@ -23,11 +23,11 @@
"@tailwindcss/typography": "^0.5.8",
"@types/d3": "^7.4.0",
"@types/d3-zoom": "^3.0.2",
"@types/lodash": "^4.14.191",
"@types/node": "^18.11.18",
"@types/vscode": "~1.74.0",
"@typescript-eslint/eslint-plugin": "^5.49.0",
"@typescript-eslint/parser": "^5.48.0",
"@windmill-labs/svelte-grid": "^5.1.6",
"@zerodevx/svelte-toast": "^0.8.1",
"autoprefixer": "^10.4.13",
"cssnano": "^5.1.14",
@@ -43,8 +43,8 @@
"postcss-load-config": "^4.0.1",
"prettier": "^2.8.3",
"prettier-plugin-svelte": "^2.9.0",
"shepherd.js": "^10.0.1",
"simple-svelte-autocomplete": "^2.5.1",
"style-to-object": "^0.4.1",
"stylelint-config-recommended": "^9.0.0",
"svelte": "^3.55.1",
"svelte-awesome": "^3.0.0",
@@ -56,7 +56,7 @@
"svelte-preprocess": "^5.0.1",
"svelte-range-slider-pips": "^2.1.1",
"svelte-splitpanes": "^0.7.3",
"svelte2tsx": "^0.6.10",
"svelte2tsx": "^0.6.1",
"tailwindcss": "^3.2.2",
"tslib": "^2.4.1",
"typescript": "^4.9.3",
@@ -80,7 +80,6 @@
"date-fns": "^2.29.3",
"fast-equals": "^4.0.3",
"highlight.js": "^11.7.0",
"lodash": "^4.17.21",
"lucide-svelte": "^0.115.0",
"monaco-editor": "0.33.0",
"monaco-editor-workers": "0.33.0",

View File

@@ -1,2 +1,79 @@
/// <reference types="@sveltejs/kit" />
declare type Item = import('svelte-dnd-action').Item
declare type DndEvent<ItemType = Item> = import('svelte-dnd-action').DndEvent<ItemType>
declare namespace svelte.JSX {
interface HTMLAttributes<T> {
onconsider?: (event: CustomEvent<DndEvent<ItemType>> & { target: EventTarget & T }) => void
onfinalize?: (event: CustomEvent<DndEvent<ItemType>> & { target: EventTarget & T }) => void
}
}
declare module '@windmill-labs/svelte-grid' {
import type { SvelteComponentTyped } from 'svelte'
export interface Size {
w: number
h: number
}
export interface Positon {
x: number
y: number
}
interface ItemLayout extends Size, Positon {
fixed?: boolean
resizable?: boolean
draggable?: boolean
customDragger?: boolean
customResizer?: boolean
min?: Size
max?: Size
}
export type Item<T> = T & { [width: number]: ItemLayout; data: any }
export type FilledItem<T> = T & { [width: number]: Required<ItemLayout>; data: any }
export interface Props<T> {
fillSpace?: boolean
items: FilledItem<T>[]
rowHeight: number
cols: [number, number][]
gap?: [number, number]
fastStart?: boolean
throttleUpdate?: number
throttleResize?: number
onTopId?: string
scroller?: undefined
sensor?: number
parentWidth?: number
}
export interface Slots<T> {
default: { item: ItemLayout; dataItem: Item<T> }
}
export default class Grid<T = {}> extends SvelteComponentTyped<
Props<T>,
{
pointerup: CustomEvent<{ id: string }>
mount: CustomEvent<>
},
Slots<T>
> { }
}
declare module '@windmill-labs/svelte-grid/build/helper/index.mjs' {
import { ItemLayout } from '@windmill-labs/svelte-grid'
const x: {
normalize(items: any[], col: any): unknown[]
adjust(items: any[], col: any): unknown[]
findSpace(item: any, items: any, cols: any): unknown
item<T>(obj: ItemLayout): Required<ItemLayout>
}
export default x
}

View File

@@ -1,4 +1,4 @@
import type { Script } from './gen'
import type { Script } from "./gen"
export type OwnerKind = 'group' | 'user' | 'folder'
@@ -14,7 +14,7 @@ export interface SchemaProperty {
enum?: string[]
contentEncoding?: 'base64' | 'binary'
format?: string
items?: { type?: 'string' | 'number' | 'bytes' | 'object'; contentEncoding?: 'base64' }
items?: { type?: 'string' | 'number' | 'bytes'; contentEncoding?: 'base64' }
properties?: { [name: string]: SchemaProperty }
}
@@ -28,14 +28,11 @@ export type Schema = {
export type Meta = { ownerKind: OwnerKind; owner: string; name: string }
type Enumerate<N extends number, Acc extends number[] = []> = Acc['length'] extends N
? Acc[number]
: Enumerate<N, [...Acc, Acc['length']]>
? Acc[number]
: Enumerate<N, [...Acc, Acc['length']]>
/** An inclusive range of integer numbers */
export type IntRange<F extends number, T extends number> =
| F
| Exclude<Enumerate<T>, Enumerate<F>>
| T
export type IntRange<F extends number, T extends number> = F | Exclude<Enumerate<T>, Enumerate<F>> | T
export function pathToMeta(path: string): Meta {
const splitted = path.split('/')

View File

@@ -81,7 +81,7 @@
$: key =
apiTokenApps[resource_type]?.key ??
(args != undefined
? Object.keys(args).filter((x) => ['token', 'password', 'api_key', 'key'].includes(x))[0]
? Object.keys(args).filter((x) => ['token', 'password', 'api_key'].includes(x))[0]
: undefined)
let scopes: string[] = []
@@ -259,7 +259,6 @@
args['token'] == '' &&
args['password'] == '' &&
args['api_key'] == '' &&
args['key'] == '' &&
key != undefined) ||
(step == 3 && pathError != '') ||
!isValid
@@ -397,7 +396,7 @@
bind:error={pathError}
bind:path
initialPath=""
namePlaceholder={resource_type}
namePlaceholder="{resource_type}"
kind="resource"
/>
@@ -434,7 +433,7 @@
{:else}
<Path
initialPath=""
namePlaceholder={resource_type}
namePlaceholder="{resource_type}"
bind:error={pathError}
bind:path
kind="resource"

View File

@@ -7,7 +7,7 @@
faPlus
} from '@fortawesome/free-solid-svg-icons'
import { setInputCat as computeInputCat } from '$lib/utils'
import { setInputCat as computeInputCat, type InputCat } from '$lib/utils'
import { Badge, Button } from './common'
import { createEventDispatcher } from 'svelte'
import Icon from 'svelte-awesome'
@@ -26,9 +26,6 @@
import type ItemPicker from './ItemPicker.svelte'
import NumberTypeNarrowing from './NumberTypeNarrowing.svelte'
import Range from './Range.svelte'
import JsonEditor from './apps/editor/settingsPanel/inputEditor/JsonEditor.svelte'
import { fade } from 'svelte/transition'
import { X } from 'lucide-svelte'
export let label: string = ''
export let value: any
@@ -47,7 +44,7 @@
export let disabled = false
export let editableSchema = false
export let itemsType:
| { type?: 'string' | 'number' | 'bytes' | 'object'; contentEncoding?: 'base64' }
| { type?: 'string' | 'number' | 'bytes'; contentEncoding?: 'base64' }
| undefined = undefined
export let displayHeader = true
export let properties: { [name: string]: SchemaProperty } | undefined = undefined
@@ -96,7 +93,7 @@
export function evalValueToRaw() {
if (value) {
rawValue = JSON.stringify(value, null, 2)
rawValue = JSON.stringify(value, null, 4)
}
}
@@ -155,21 +152,18 @@
value = enum_?.[0]
} else if (inputCat == 'boolean') {
value = false
} else if (inputCat == 'list') {
value = []
}
}
}
}
$: inputCat = computeInputCat(type, format, itemsType?.type, enum_, contentEncoding)
let redraw = 0
</script>
<div class="flex flex-col w-full min-w-[250px]">
<div>
{#if displayHeader}
<FieldHeader {label} {required} {type} {contentEncoding} {format} />
<FieldHeader {label} {required} {type} {contentEncoding} {format} {itemsType} />
{/if}
{#if editableSchema}
<div class="p-2 my-1 text-xs border-solid border border-gray-400">
@@ -192,7 +186,6 @@
use:autosize
rows="1"
bind:value={description}
on:keydown|stopPropagation
placeholder="Field description"
/>
{#if type == 'string' && format != 'date-time'}
@@ -205,7 +198,6 @@
<select bind:value={itemsType}>
<option value={undefined}>No specific item type</option>
<option value={{ type: 'string' }}> Items are strings</option>
<option value={{ type: 'object' }}> Items are objects (JSON)</option>
<option value={{ type: 'number' }}>Items are numbers</option>
<option value={{ type: 'string', contentEncoding: 'base64' }}
>Items are bytes</option
@@ -240,6 +232,7 @@
<input
{autofocus}
on:focus={(e) => {
window.dispatchEvent(new Event('pointerup'))
dispatch('focus')
}}
{disabled}
@@ -251,12 +244,14 @@
bind:value
min={extra['min']}
max={extra['max']}
on:input={() => dispatch('input', { value, isRaw: true })}
/>
{/if}
{:else if inputCat == 'boolean'}
<Toggle
on:pointerdown={(e) => {
e?.stopPropagation()
window.dispatchEvent(new Event('pointerup'))
}}
{disabled}
class={valid
@@ -268,46 +263,44 @@
<span>&nbsp; Not set</span>
{/if}
{:else if inputCat == 'list'}
<div class="w-full">
<div class="w-full">
{#key redraw}
{#if Array.isArray(value)}
{#each value ?? [] as v, i}
<div class="flex max-w-md mt-1 w-full items-center">
{#if itemsType?.type == 'number'}
<input type="number" bind:value={v} />
{:else if itemsType?.type == 'string' && itemsType?.contentEncoding == 'base64'}
<input
type="file"
class="my-6"
on:change={(x) => fileChanged(x, (val) => (value[i] = val))}
multiple={false}
/>
{:else if itemsType?.type == 'object'}
<JsonEditor code={JSON.stringify(v, null, 2)} bind:value={v} />
{:else}
<input type="text" bind:value={v} />
{/if}
<button
transition:fade|local={{ duration: 100 }}
class="rounded-full p-1 bg-white/60 duration-200 hover:bg-gray-200"
aria-label="Clear"
on:click={() => {
value.splice(i, 1)
redraw += 1
}}
>
<X size={14} />
</button>
</div>
{/each}
{/if}
{/key}
<div>
<div>
{#each value ?? [] as v, i}
<div class="flex flex-row max-w-md mt-1">
{#if itemsType?.type == 'number'}
<input autofocus={autofocus && i == 0} type="number" bind:value={v} />
{:else if itemsType?.type == 'string' && itemsType?.contentEncoding == 'base64'}
<input
autofocus={autofocus && i == 0}
type="file"
class="my-6"
on:change={(x) => fileChanged(x, (val) => (value[i] = val))}
multiple={false}
/>
{:else}
<input autofocus={autofocus && i == 0} type="text" bind:value={v} />
{/if}
<Button
variant="border"
color="red"
size="sm"
btnClasses="mx-6"
on:click={() => {
value = value.filter((el) => el != v)
if (value.length == 0) {
value = undefined
}
}}
>
<Icon data={faMinus} />
</Button>
</div>
{/each}
</div>
<Button
variant="border"
color="dark"
size="xs"
color="blue"
size="sm"
btnClasses="mt-1"
on:click={() => {
if (value == undefined || !Array.isArray(value)) {
@@ -319,6 +312,9 @@
<Icon data={faPlus} class="mr-2" />
Add item
</Button>
<span class="ml-2">
{(value ?? []).length} item{(value ?? []).length > 1 ? 's' : ''}
</span>
</div>
{:else if inputCat == 'resource-object'}
<ObjectResourceInput {format} bind:value />
@@ -331,21 +327,31 @@
bind:args={value}
/>
</div>
{:else if disabled}
<textarea disabled />
{:else}
<JsonEditor
bind:editor
<textarea
bind:this={el}
on:focus={(e) => {
window.dispatchEvent(new Event('pointerup'))
dispatch('focus')
}}
code={rawValue}
bind:value
{autofocus}
{disabled}
use:autosize
style="max-height: {maxHeight}"
on:input={() => {
dispatch('input', { rawValue: value, isRaw: false })
}}
class="col-span-10 {valid
? ''
: 'border border-red-700 border-opacity-30 focus:border-red-700 focus:border-opacity-30 bg-red-100'}"
placeholder={defaultValue ? JSON.stringify(defaultValue, null, 4) : ''}
bind:value={rawValue}
/>
{/if}
{:else if inputCat == 'enum'}
<select
on:focus={(e) => {
window.dispatchEvent(new Event('pointerup'))
dispatch('focus')
}}
{disabled}
@@ -362,11 +368,16 @@
<div class="border my-1 mb-4 w-full border-gray-400">
<SimpleEditor
on:focus={(e) => {
window.dispatchEvent(new Event('pointerup'))
dispatch('focus')
}}
on:blur={() => dispatch('blur')}
bind:this={editor}
lang={inputCat}
bind:code={value}
on:change={async () => {
dispatch('input', { rawValue: value, isRaw: false })
}}
autoHeight
/>
</div>
@@ -396,10 +407,11 @@
rows="1"
bind:this={el}
on:focus={(e) => {
window.dispatchEvent(new Event('pointerup'))
dispatch('focus')
}}
on:blur={() => dispatch('blur')}
use:autosize
on:keydown|stopPropagation
type="text"
{disabled}
class="col-span-10 {valid
@@ -407,6 +419,9 @@
: 'border border-red-700 border-opacity-30 focus:border-red-700 focus:border-opacity-30 bg-red-100'}"
placeholder={defaultValue ?? ''}
bind:value
on:input={() => {
dispatch('input', { rawValue: value, isRaw: false })
}}
/>
{#if itemPicker}
<div class="ml-1 relative">
@@ -435,6 +450,19 @@
{/if}
</div>
{/if}
{#if !required && inputCat != 'resource-object'}
<!-- <Tooltip placement="bottom" content="Reset to default value">
<Button
on:click={() => (value = undefined)}
{disabled}
color="alternative"
size="sm"
class="h-8"
>
<Icon data={faArrowRotateLeft} />
</Button>
</Tooltip> -->
{/if}
<slot name="actions" />
</div>
{#if !compact || (error && error != '')}

View File

@@ -6,7 +6,6 @@
import { Button, Drawer, DrawerContent } from './common'
import autosize from 'svelte-autosize'
import { ClipboardCopy } from 'lucide-svelte'
import Portal from 'svelte-portal'
export let result: any
export let requireHtmlApproval = false
@@ -92,13 +91,28 @@
let jsonViewer: Drawer
</script>
<Drawer bind:this={jsonViewer} size="900px">
<DrawerContent title="Expanded Result" on:close={jsonViewer.closeDrawer}>
<svelte:fragment slot="actions">
<Button
on:click={() => copyToClipboard(JSON.stringify(result, null, 4))}
color="light"
size="xs"
>
<div class="flex gap-2 items-center">Copy to clipboard <ClipboardCopy /> </div>
</Button>
</svelte:fragment>
<Highlight language={json} code={JSON.stringify(result, null, 4).replace(/\\n/g, '\n')} />
</DrawerContent>
</Drawer>
<div class="inline-highlight">
{#if result != undefined}
{#if resultKind && resultKind != 'json'}
<div class="mb-2 text-gray-500 text-sm bg-gray-50/20">
as JSON&nbsp;<input class="windmillapp" type="checkbox" bind:checked={forceJson} /></div
>{/if}{#if typeof result == 'object' && Object.keys(result).length > 0}<div
class="mb-2 min-w-[360px] text-sm text-gray-700 relative"
class="mb-2 min-w-[400px] text-sm text-gray-700 relative"
>The result keys are: <b>{truncate(Object.keys(result).join(', '), 50)}</b>
<div class="text-gray-500 text-sm absolute top-0 right-2">
<button on:click={jsonViewer.openDrawer}>Expand JSON</button>
@@ -234,23 +248,6 @@
<Highlight language={json} code={JSON.stringify(result, null, 4).replace(/\\n/g, '\n')} />
{/if}
{:else}
<div class="text-gray-500 text-sm">No result: {JSON.stringify(result)}</div>
<div class="text-gray-500 text-sm">No result</div>
{/if}
</div>
<Portal>
<Drawer bind:this={jsonViewer} size="900px">
<DrawerContent title="Expanded Result" on:close={jsonViewer.closeDrawer}>
<svelte:fragment slot="actions">
<Button
on:click={() => copyToClipboard(JSON.stringify(result, null, 4))}
color="light"
size="xs"
>
<div class="flex gap-2 items-center">Copy to clipboard <ClipboardCopy /> </div>
</Button>
</svelte:fragment>
<Highlight language={json} code={JSON.stringify(result, null, 4).replace(/\\n/g, '\n')} />
</DrawerContent>
</Drawer>
</Portal>

View File

@@ -2,10 +2,9 @@
import type { DropdownItem } from '$lib/utils'
import { createEventDispatcher } from 'svelte'
import Icon from 'svelte-awesome'
import { MoreVertical } from 'lucide-svelte'
import { MoreHorizontal } from 'lucide-svelte'
import { Button, Menu } from './common'
import { goto } from '$app/navigation'
import { twMerge } from 'tailwind-merge'
type Alignment = 'start' | 'end'
type Side = 'top' | 'bottom'
@@ -14,24 +13,15 @@
export let dropdownItems: DropdownItem[]
export let name: string | undefined = undefined
export let placement: Placement = 'bottom-start'
export let btnClasses = ''
$: buttonClass = twMerge('!border-0 bg-transparent !p-[6px]', btnClasses)
export let btnClasses = '!text-blue-500 bg-transparent'
const dispatch = createEventDispatcher()
</script>
<Menu {placement} let:close>
<Button
nonCaptureEvent
color="dark"
variant="border"
size="xs"
btnClasses={buttonClass}
{...$$restProps}
slot="trigger"
>
<Button nonCaptureEvent color="light" size="xs" {btnClasses} {...$$restProps} slot="trigger">
{#if !$$slots.default}
<MoreVertical size={20} />
<MoreHorizontal size={20} />
{:else}
<slot />
{/if}

View File

@@ -461,6 +461,10 @@
}
})
editor.onDidBlurEditorText(() => {
dispatch('blur')
})
reloadWebsocket()
return () => {

View File

@@ -232,8 +232,14 @@
<Badge color={validCode ? 'green' : 'red'} class="min-w-[60px] mr-3">
{validCode ? 'Valid' : 'Invalid'}
</Badge>
<div class="flex items-center divide-x">
<Popover notClickable placement="bottom" disapperTimoout={0} class="pr-1" disablePopup={!iconOnly}>
<div id="script-tutorial-3" class="flex items-center divide-x">
<Popover
notClickable
placement="bottom"
disapperTimoout={0}
class="pr-1"
disablePopup={!iconOnly}
>
<Button
color="light"
btnClasses="!font-medium !h-full"
@@ -245,11 +251,15 @@
>
+Context Var
</Button>
<svelte:fragment slot="text">
Add context variable
</svelte:fragment>
<svelte:fragment slot="text">Add context variable</svelte:fragment>
</Popover>
<Popover notClickable placement="bottom" disapperTimoout={0} class="px-1" disablePopup={!iconOnly}>
<Popover
notClickable
placement="bottom"
disapperTimoout={0}
class="px-1"
disablePopup={!iconOnly}
>
<Button
color="light"
btnClasses="!font-medium !h-full"
@@ -261,11 +271,15 @@
>
+Variable
</Button>
<svelte:fragment slot="text">
Add variable
</svelte:fragment>
<svelte:fragment slot="text">Add variable</svelte:fragment>
</Popover>
<Popover notClickable placement="bottom" disapperTimoout={0} class="px-1" disablePopup={!iconOnly}>
<Popover
notClickable
placement="bottom"
disapperTimoout={0}
class="px-1"
disablePopup={!iconOnly}
>
<Button
btnClasses="!font-medium !h-full"
size="xs"
@@ -277,11 +291,15 @@
>
+Resource
</Button>
<svelte:fragment slot="text">
Add resource
</svelte:fragment>
<svelte:fragment slot="text">Add resource</svelte:fragment>
</Popover>
<Popover notClickable placement="bottom" disapperTimoout={0} class="px-1" disablePopup={!iconOnly}>
<Popover
notClickable
placement="bottom"
disapperTimoout={0}
class="px-1"
disablePopup={!iconOnly}
>
<Button
btnClasses="!font-medium !h-full"
size="xs"
@@ -293,11 +311,15 @@
>
Reset
</Button>
<svelte:fragment slot="text">
Reset
</svelte:fragment>
<svelte:fragment slot="text">Reset</svelte:fragment>
</Popover>
<Popover notClickable placement="bottom" disapperTimoout={0} class="px-1" disablePopup={!iconOnly}>
<Popover
notClickable
placement="bottom"
disapperTimoout={0}
class="px-1"
disablePopup={!iconOnly}
>
<Button
btnClasses="!font-medium !h-full"
size="xs"
@@ -320,11 +342,15 @@
{/if}
</span>
</Button>
<svelte:fragment slot="text">
Reload assistant
</svelte:fragment>
<svelte:fragment slot="text">Reload assistant</svelte:fragment>
</Popover>
<Popover notClickable placement="bottom" disapperTimoout={0} class="px-1" disablePopup={!iconOnly}>
<Popover
notClickable
placement="bottom"
disapperTimoout={0}
class="px-1"
disablePopup={!iconOnly}
>
<Button
btnClasses="!font-medium"
size="xs"
@@ -342,7 +368,13 @@
</Popover>
</div>
</div>
<Popover notClickable placement="bottom" disapperTimoout={0} class="px-1" disablePopup={!iconOnly}>
<Popover
notClickable
placement="bottom"
disapperTimoout={0}
class="px-1"
disablePopup={!iconOnly}
>
<Button
btnClasses="!font-medium"
size="xs"
@@ -354,9 +386,7 @@
>
Script
</Button>
<svelte:fragment slot="text">
Script
</svelte:fragment>
<svelte:fragment slot="text">Script</svelte:fragment>
</Popover>
</div>

View File

@@ -6,6 +6,9 @@
export let contentEncoding = ''
export let type: string | undefined = undefined
export let required = false
export let itemsType:
| { type?: 'string' | 'number' | 'bytes'; contentEncoding?: 'base64' }
| undefined = undefined
</script>
<div class="inline-flex flex-row items-center truncated">

View File

@@ -20,6 +20,7 @@
import { loadFlowSchedule, type Schedule } from './flows/scheduleUtils'
import type { FlowEditorContext } from './flows/types'
import { cleanInputs } from './flows/utils'
import { Tour } from './tutorial'
export let initialPath: string = ''
export let selectedId: string | undefined
@@ -27,6 +28,7 @@
export let loading = false
export let flowStore: Writable<Flow>
export let flowStateStore: Writable<FlowState>
export let tour = false
async function createSchedule(path: string) {
const { cron, args, enabled } = $scheduleStore
@@ -118,7 +120,6 @@
if (leave) {
goto(`/flows/get/${$flowStore.path}?workspace_id=${$workspaceStore}`)
} else if (initialPath !== $flowStore.path) {
initialPath = $flowStore.path
goto(`/flows/edit/${$flowStore.path}?workspace_id=${$workspaceStore}`)
}
}
@@ -195,14 +196,6 @@
loadHubScripts()
function onKeyDown(event: KeyboardEvent) {
let classes = event.target?.['className']
if (
(typeof classes === 'string' && classes.includes('inputarea')) ||
['INPUT', 'TEXTAREA'].includes(document.activeElement?.tagName!)
) {
return
}
switch (event.key) {
case 'Z':
if (event.ctrlKey) {
@@ -255,6 +248,10 @@
<svelte:window on:keydown={onKeyDown} />
{#if tour}
<Tour tutorial="flow" />
{/if}
{#if !$userStore?.operator}
<ScriptEditorDrawer bind:this={$scriptEditorDrawer} />

View File

@@ -88,7 +88,7 @@
minHeight={400}
modules={flow?.value?.modules}
failureModule={flow?.value?.failure_module}
on:select={(e) => (stepDetail = e.detail)}
on:click={(e) => (stepDetail = e.detail)}
/>
</div>
{#if !noSide}

View File

@@ -22,7 +22,7 @@
const dispatch = createEventDispatcher()
export let jobId: string
export let workspaceId: string | undefined = undefined
export let flowState: FlowState | undefined = undefined
export let flowJobIds:
| {
@@ -102,7 +102,7 @@
localFlowModuleStates[mod.id ?? '']?.scheduled_for == undefined
) {
JobService.getJob({
workspace: workspaceId ?? $workspaceStore ?? '',
workspace: $workspaceStore ?? '',
id: mod.job ?? ''
}).then((job) => {
localFlowModuleStates[mod.id ?? ''] = {
@@ -122,7 +122,7 @@
if (jobId != '00000000-0000-0000-0000-000000000000') {
try {
const newJob = await JobService.getJob({
workspace: workspaceId ?? $workspaceStore ?? '',
workspace: $workspaceStore ?? '',
id: jobId ?? ''
})
if (JSON.stringify(newJob) !== JSON.stringify(job)) {
@@ -159,7 +159,7 @@
})
async function loadOwner(path: string) {
is_owner = await isOwner(path, $userStore!, workspaceId ?? $workspaceStore!)
is_owner = await isOwner(path, $userStore!, $workspaceStore!)
}
let selected: 'graph' | 'sequence' = 'graph'
@@ -207,7 +207,7 @@
variant="border"
on:click={async () =>
await JobService.resumeSuspendedFlowAsOwner({
workspace: workspaceId ?? $workspaceStore ?? '',
workspace: $workspaceStore ?? '',
id: job?.id ?? '',
requestBody: JSON.parse(payload)
})}
@@ -305,7 +305,6 @@
</Button>
<div class="border p-6" class:hidden={forloop_selected != loopJobId}>
<svelte:self
{workspaceId}
bind:suspend_status
bind:retry_status
bind:flowState
@@ -380,7 +379,6 @@
<li class="w-full border border-gray-600 p-6 space-y-2 bg-blue-50/50">
{#if [FlowStatusModule.type.IN_PROGRESS, FlowStatusModule.type.SUCCESS, FlowStatusModule.type.FAILURE].includes(mod.type)}
<svelte:self
{workspaceId}
bind:suspend_status
bind:retry_status
bind:flowState

View File

@@ -64,6 +64,7 @@
type={toAny(v)?.type}
contentEncoding={toAny(v)?.contentEncoding}
format={toAny(v)?.format}
itemsType={toAny(v)?.itemsType}
/><span class="ml-4 mt-2 text-xs"
>{toAny(v)?.default != undefined
? 'default: ' + JSON.stringify(toAny(v)?.default)

View File

@@ -3,8 +3,6 @@
import { workspaceStore } from '$lib/stores'
export let name: string
export let tabular = false
export let order = ['scripts', 'flows', 'apps', 'schedules', 'variables', 'resources']
$: $workspaceStore && loadUsage()
@@ -15,18 +13,10 @@
}
</script>
{#if tabular}
{#each order as key}
<td class="text-center">
{usage[key] ?? ''}
</td>
<div class="flex flex-col text-xs text-gray-600">
{#each Object.entries(usage) as [k, v]}
<div>
{k}: {v}
</div>
{/each}
{:else}
<div class="flex flex-col text-xs text-gray-600">
{#each Object.entries(usage) as [k, v]}
<div>
{k}: {v}
</div>
{/each}
</div>
{/if}
</div>

View File

@@ -234,14 +234,6 @@
name,
requestBody: { username: member_name }
})
await GranularAclService.removeGranularAcls({
workspace: $workspaceStore ?? '',
path: name,
kind: 'group_',
requestBody: {
owner: 'u/' + member_name
}
})
loadGroup()
}}>remove</button
>

View File

@@ -133,6 +133,7 @@
contentEncoding={schema.properties[argName].contentEncoding}
required={schema.required.includes(argName)}
type={schema.properties[argName].type}
itemsType={schema.properties[argName].items}
/>
{#if isStaticTemplate(inputCat)}

View File

@@ -1,318 +0,0 @@
<script lang="ts">
import { faMinus, faPlus } from '@fortawesome/free-solid-svg-icons'
import { setInputCat as computeInputCat } from '$lib/utils'
import { Badge, Button } from './common'
import { createEventDispatcher } from 'svelte'
import Icon from 'svelte-awesome'
import FieldHeader from './FieldHeader.svelte'
import ObjectResourceInput from './ObjectResourceInput.svelte'
import ResourcePicker from './ResourcePicker.svelte'
import type { SchemaProperty } from '$lib/common'
import autosize from 'svelte-autosize'
import Toggle from './Toggle.svelte'
import Range from './Range.svelte'
import LightweightSchemaForm from './LightweightSchemaForm.svelte'
export let label: string = ''
export let value: any
export let defaultValue: any = undefined
export let description: string = ''
export let format: string = ''
export let contentEncoding: 'base64' | 'binary' | undefined = undefined
export let type: string | undefined = undefined
export let required = false
export let pattern: undefined | string = undefined
export let valid = required ? false : true
export let maxRows = 10
export let enum_: string[] | undefined = undefined
export let itemsType:
| { type?: 'string' | 'number' | 'bytes' | 'object'; contentEncoding?: 'base64' }
| undefined = undefined
export let displayHeader = true
export let properties: { [name: string]: SchemaProperty } | undefined = undefined
export let extra: Record<string, any> = {}
const dispatch = createEventDispatcher()
$: maxHeight = maxRows ? `${1 + maxRows * 1.2}em` : `auto`
$: validateInput(pattern, value)
let error: string = ''
let el: HTMLTextAreaElement | undefined = undefined
let rawValue: string | undefined = undefined
$: {
if (rawValue) {
try {
value = JSON.parse(rawValue)
error = ''
} catch (err) {
error = err.toString()
}
}
}
$: {
error = ''
if (inputCat === 'object') {
evalValueToRaw()
validateInput(pattern, value)
}
}
export function evalValueToRaw() {
if (value) {
rawValue = JSON.stringify(value, null, 4)
}
}
function fileChanged(e: any, cb: (v: string | undefined) => void) {
let t = e.target
if (t && 'files' in t && t.files.length > 0) {
let reader = new FileReader()
reader.onload = (e: any) => {
cb(e.target.result.split('base64,')[1])
}
reader.readAsDataURL(t.files[0])
} else {
cb(undefined)
}
}
export function focus() {
el?.focus()
if (el) {
el.style.height = '5px'
el.style.height = el.scrollHeight + 50 + 'px'
}
}
function validateInput(pattern: string | undefined, v: any): void {
if (required && (v == undefined || v == null || v === '')) {
error = 'This field is required'
valid = false
} else {
if (pattern && !testRegex(pattern, v)) {
error = `Should match ${pattern}`
valid = false
} else {
error = ''
valid = true
}
}
}
function testRegex(pattern: string, value: any): boolean {
try {
const regex = new RegExp(pattern)
return regex.test(value)
} catch (err) {
return false
}
}
$: {
if (value == undefined || value == null) {
value = defaultValue
if (defaultValue === undefined || defaultValue === null) {
if (inputCat === 'string') {
value = ''
} else if (inputCat == 'enum') {
value = enum_?.[0]
} else if (inputCat == 'boolean') {
value = false
}
}
}
}
$: inputCat = computeInputCat(type, format, itemsType?.type, enum_, contentEncoding)
</script>
<div class="flex flex-col w-full min-w-[250px]">
<div>
{#if displayHeader}
<FieldHeader {label} {required} {type} {contentEncoding} {format} />
{/if}
{#if description}
<div class="text-sm italic pb-1">
{description}
</div>
{/if}
<div class="flex space-x-1">
{#if inputCat == 'number'}
{#if extra['min'] != undefined && extra['max'] != undefined}
<div class="flex w-full gap-1">
<span>{extra['min']}</span>
<div class="grow">
<Range bind:value min={extra['min']} max={extra['max']} />
</div>
<span>{extra['max']}</span>
<span class="mx-2"><Badge large color="blue">{value}</Badge></span>
</div>
{:else}
<input
on:focus={(e) => {
dispatch('focus')
}}
type="number"
class={valid
? ''
: 'border border-red-700 border-opacity-30 focus:border-red-700 focus:border-opacity-30 bg-red-100'}
placeholder={defaultValue ?? ''}
bind:value
min={extra['min']}
max={extra['max']}
/>
{/if}
{:else if inputCat == 'boolean'}
<Toggle
on:pointerdown={(e) => {
e?.stopPropagation()
}}
class={valid
? ''
: 'border border-red-700 border-opacity-30 focus:border-red-700 focus:border-opacity-30 bg-red-100'}
bind:checked={value}
/>
{#if type == 'boolean' && value == undefined}
<span>&nbsp; Not set</span>
{/if}
{:else if inputCat == 'list'}
<div>
<div>
{#each value ?? [] as v, i}
<div class="flex flex-row max-w-md mt-1">
{#if itemsType?.type == 'number'}
<input type="number" bind:value={v} />
{:else if itemsType?.type == 'string' && itemsType?.contentEncoding == 'base64'}
<input
type="file"
class="my-6"
on:change={(x) => fileChanged(x, (val) => (value[i] = val))}
multiple={false}
/>
{:else}
<input type="text" bind:value={v} />
{/if}
<Button
variant="border"
color="red"
size="sm"
btnClasses="mx-6"
on:click={() => {
value = value.filter((el) => el != v)
if (value.length == 0) {
value = undefined
}
}}
>
<Icon data={faMinus} />
</Button>
</div>
{/each}
</div>
<Button
variant="border"
color="blue"
size="sm"
btnClasses="mt-1"
on:click={() => {
if (value == undefined || !Array.isArray(value)) {
value = []
}
value = value.concat('')
}}
>
<Icon data={faPlus} class="mr-2" />
Add item
</Button>
<span class="ml-2">
{(value ?? []).length} item{(value ?? []).length > 1 ? 's' : ''}
</span>
</div>
{:else if inputCat == 'resource-object'}
<ObjectResourceInput {format} bind:value />
{:else if inputCat == 'object'}
{#if properties && Object.keys(properties).length > 0}
<div class="p-4 pl-8 border rounded w-full">
<LightweightSchemaForm
schema={{ properties, $schema: '', required: [], type: 'object' }}
bind:args={value}
/>
</div>
{:else}
<textarea
bind:this={el}
on:focus={(e) => {
dispatch('focus')
}}
use:autosize
style="max-height: {maxHeight}"
class="col-span-10 {valid
? ''
: 'border border-red-700 border-opacity-30 focus:border-red-700 focus:border-opacity-30 bg-red-100'}"
placeholder={defaultValue ? JSON.stringify(defaultValue, null, 4) : ''}
bind:value={rawValue}
/>
{/if}
{:else if inputCat == 'enum'}
<select
on:focus={(e) => {
dispatch('focus')
}}
class="px-6"
bind:value
>
{#each enum_ ?? [] as e}
<option>{e}</option>
{/each}
</select>
{:else if inputCat == 'date'}
<input class="inline-block" type="datetime-local" bind:value />
{:else if inputCat == 'base64'}
<input
type="file"
class="my-6"
on:change={(x) => fileChanged(x, (val) => (value = val))}
multiple={false}
/>
{:else if inputCat == 'resource-string'}
<ResourcePicker
bind:value
resourceType={format.split('-').length > 1
? format.substring('resource-'.length)
: undefined}
/>
{:else if inputCat == 'string'}
<div class="flex flex-col w-full">
<div class="flex flex-row w-full items- justify-between">
<textarea
rows="1"
bind:this={el}
on:focus={(e) => {
dispatch('focus')
}}
use:autosize
type="text"
class="col-span-10 {valid
? ''
: 'border border-red-700 border-opacity-30 focus:border-red-700 focus:border-opacity-30 bg-red-100'}"
placeholder={defaultValue ?? ''}
bind:value
/>
</div>
</div>
{/if}
<slot name="actions" />
</div>
</div>
</div>

View File

@@ -1,35 +0,0 @@
<script lang="ts">
import type { Schema } from '$lib/common'
import LightweightArgInput from './LightweightArgInput.svelte'
export let schema: Schema
export let args: Record<string, any> | undefined = undefined
$: if (args === undefined) {
args = {}
}
</script>
<div class="w-full">
{#each Object.keys(schema.properties ?? {}) as argName, i (argName)}
<div>
{#if typeof args == 'object' && schema?.properties[argName] && args}
<LightweightArgInput
label={argName}
description={schema.properties[argName].description}
bind:value={args[argName]}
type={schema.properties[argName].type}
required={schema.required.includes(argName)}
pattern={schema.properties[argName].pattern}
defaultValue={schema.properties[argName].default}
enum_={schema.properties[argName].enum}
format={schema.properties[argName].format}
contentEncoding={schema.properties[argName].contentEncoding}
properties={schema.properties[argName].properties}
itemsType={schema.properties[argName].items}
extra={schema.properties[argName]}
/>
{/if}
</div>
{/each}
</div>

View File

@@ -60,7 +60,6 @@
if (testJob && !testJob.canceled && testJob.type == 'CompletedJob' && `result` in testJob) {
if ($flowStateStore[mod.id]?.previewResult) {
$flowStateStore[mod.id].previewResult = testJob.result
$flowStateStore = $flowStateStore
}
}
}

View File

@@ -11,7 +11,7 @@
<span class="flex items-center space-x-2 mb-2">
<h1>{title}</h1>
{#if tooltip}
<Tooltip scale={0.9}>{tooltip}</Tooltip>
<Tooltip>{tooltip}</Tooltip>
{/if}
</span>
{:else}

View File

@@ -23,7 +23,6 @@
import FolderEditor from './FolderEditor.svelte'
import GroupEditor from './GroupEditor.svelte'
import { random_adj } from './random_positive_adjetive'
import Badge from './common/badge/Badge.svelte'
type PathKind = 'resource' | 'script' | 'variable' | 'flow' | 'schedule' | 'app'
let meta: Meta | undefined = undefined
@@ -200,10 +199,6 @@
}
function initPath() {
if (path != undefined && path != '') {
meta = pathToMeta(path)
return
}
if (initialPath == undefined || initialPath == '') {
reset()
} else {
@@ -311,7 +306,7 @@
{#if meta != undefined}
<div class="flex gap-4 shrink">
<label class="block">
<span class="text-gray-700 text-sm whitespace-nowrap">Owner</span>
<span class="text-gray-700 text-sm whitespace-nowrap">&nbsp;</span>
<ToggleButtonGroup
class="mt-0.5"
@@ -414,26 +409,8 @@
{/if}
</div>
<div class="flex flex-col w-full mt-4">
<div class="flex justify-start w-full">
<Badge
color="gray"
class="center-center !bg-gray-300 !text-gray-600 !w-[70px] !h-[24px] rounded-r-none"
>
Full path
</Badge>
<input
type="text"
readonly
value={path}
size={path?.length || 50}
class="font-mono !text-xs max-w-[calc(100%-70px)] !w-auto !h-[24px] !py-0 !border-l-0 !rounded-l-none"
on:focus={({ currentTarget }) => {
currentTarget.select()
}}
/>
<!-- <span class="font-mono text-sm break-all">{path}</span> -->
</div>
<div class="flex-row flex justify-between w-full">
<div><span class="font-mono text-sm break-all">{path}</span></div>
<div class="text-red-600 text-2xs">{error}</div>
</div>
</div>

View File

@@ -88,7 +88,7 @@
}}
items={collection}
class="text-clip grow min-w-0"
placeholder="{resourceType ?? 'any'} resource"
placeholder="{resourceType} resource"
inputStyles={SELECT_INPUT_DEFAULT_STYLE.inputStyles}
containerStyles={SELECT_INPUT_DEFAULT_STYLE.containerStyles}
/>

View File

@@ -45,7 +45,7 @@
label: 'Failed',
data:
failed?.map((job) => ({
x: job.started_at as any,
x: job.created_at as any,
y: job.duration_ms,
id: job.id,
path: job.script_path
@@ -57,7 +57,7 @@
label: 'Successful',
data:
success?.map((job) => ({
x: job.started_at as any,
x: job.created_at as any,
y: job.duration_ms,
id: job.id,
path: job.script_path
@@ -71,10 +71,7 @@
enabled: true,
modifierKey: 'ctrl' as 'ctrl',
onPanComplete: ({ chart }) => {
dispatch('zoom', {
min: addSeconds(new Date(chart.scales.x.min), -1),
max: addSeconds(new Date(chart.scales.x.max), 1)
})
dispatch('zoom', { min: new Date(chart.scales.x.min), max: new Date(chart.scales.x.max) })
}
},
zoom: {
@@ -83,10 +80,7 @@
},
mode: 'x' as 'x',
onZoom: ({ chart }) => {
dispatch('zoom', {
min: addSeconds(new Date(chart.scales.x.min), -1),
max: addSeconds(new Date(chart.scales.x.max), 1)
})
dispatch('zoom', { min: new Date(chart.scales.x.min), max: new Date(chart.scales.x.max) })
}
}
}
@@ -94,22 +88,6 @@
function getPath(x: any): string {
return x.path
}
$: minTime = getMinTime(jobs)
function addSeconds(date: Date, seconds: number): Date {
date.setTime(date.getTime() + seconds * 1000)
return date
}
function getMinTime(jobs: CompletedJob[] | undefined): Date {
return addSeconds(new Date(jobs?.[jobs?.length - 1]?.started_at ?? new Date().toString()), -15)
}
$: maxTime = getMaxTime(jobs)
function getMaxTime(jobs: CompletedJob[] | undefined): Date {
return addSeconds(new Date(jobs?.[0]?.started_at ?? new Date().toString()), 15)
}
</script>
<Scatter
@@ -137,8 +115,7 @@
display: false
},
type: 'time',
min: minTime,
max: maxTime
min: jobs?.[jobs?.length - 1]?.created_at ?? new Date().toString()
},
y: {
grid: {

View File

@@ -12,7 +12,6 @@
import Toggle from './Toggle.svelte'
import Tooltip from './Tooltip.svelte'
import { flip } from 'svelte/animate'
import Portal from 'svelte-portal'
const dispatch = createEventDispatcher()
@@ -179,10 +178,8 @@
/>
<div class="ml-2">
<Tooltip>
Arguments can be edited either using the wizard, or by editing their JSON Schema,
<a href="https://docs.windmill.dev/docs/reference/#script-parameters-to-json-schema"
>see docs</a
>
Arguments can be edited either using the wizard, or by editing their JSON Schema,
<a href="https://docs.windmill.dev/docs/reference/#script-parameters-to-json-schema">see docs</a>
</Tooltip>
</div>
</div>
@@ -284,13 +281,11 @@
</div>
</div>
<Portal>
<SchemaModal
bind:this={schemaModal}
bind:property={modalProperty}
bind:error={argError}
on:save={handleAddOrEditArgument}
bind:editing
bind:oldArgName
/>
</Portal>
<SchemaModal
bind:this={schemaModal}
bind:property={modalProperty}
bind:error={argError}
on:save={handleAddOrEditArgument}
bind:editing
bind:oldArgName
/>

View File

@@ -1,6 +1,6 @@
<script lang="ts">
import type { Schema } from '$lib/common'
import { VariableService } from '$lib/gen'
import { VariableService, type InputTransform } from '$lib/gen'
import { workspaceStore } from '$lib/stores'
import { allTrue } from '$lib/utils'
import { faPlus } from '@fortawesome/free-solid-svg-icons'
@@ -10,7 +10,7 @@
import VariableEditor from './VariableEditor.svelte'
export let schema: Schema
export let args: Record<string, any> = {}
export let args: Record<string, InputTransform | any> = {}
export let disabledArgs: string[] = []
export let disabled = false

View File

@@ -1,23 +1,23 @@
<script lang="ts">
import { Script, ScriptService } from '$lib/gen'
import { goto } from '$app/navigation'
import { page } from '$app/stores'
import { inferArgs } from '$lib/infer'
import { initialCode } from '$lib/script_helpers'
import { initialCode, isInitialCode } from '$lib/script_helpers'
import { userStore, workspaceStore } from '$lib/stores'
import { emptySchema, encodeState, sendUserToast, setQueryWithoutLoad } from '$lib/utils'
import Path from './Path.svelte'
import RadioButton from './RadioButton.svelte'
import ScriptEditor from './ScriptEditor.svelte'
import ScriptSchema from './ScriptSchema.svelte'
import CenteredPage from './CenteredPage.svelte'
import { dirtyStore } from './common/confirmationModal/dirtyStore'
import { Button, ButtonPopup, ButtonPopupItem, Kbd } from './common'
import { faPen, faSave } from '@fortawesome/free-solid-svg-icons'
import { faChevronDown, faChevronUp, faPen, faSave } from '@fortawesome/free-solid-svg-icons'
import Breadcrumb from './common/breadcrumb/Breadcrumb.svelte'
import LanguageIcon from './common/languageIcons/LanguageIcon.svelte'
import type { SupportedLanguage } from '$lib/common'
import Tooltip from './Tooltip.svelte'
import SettingSection from './SettingSection.svelte'
export let script: Script
export let initialPath: string = ''
@@ -31,34 +31,14 @@
['Go', Script.language.GO],
['Bash', Script.language.BASH]
]
const scriptKindOptions: { value: Script.kind; title: string; desc?: string }[] = [
{
value: Script.kind.SCRIPT,
title: 'Action'
},
{
value: Script.kind.TRIGGER,
title: 'Trigger',
desc: 'First module of flows to trigger them based on external changes. These kind of scripts are usually running on a schedule to periodically look for changes.'
},
{
value: Script.kind.APPROVAL,
title: 'Approval',
desc: 'Send notifications externally to ask for approval to continue a flow.'
},
{
value: Script.kind.FAILURE,
title: 'Error Handler',
desc: 'Handle errors in flows after all retry attempts have been exhausted.'
}
]
let viewScriptKind = script.kind !== Script.kind.SCRIPT
let pathError = ''
let summaryC: HTMLInputElement | undefined = undefined
let pathC: Path | undefined = undefined
let loadingSave = false
$: setQueryWithoutLoad($page.url, [{ key: 'state', value: encodeState(script) }])
$: setQueryWithoutLoad($page.url, 'state', encodeState(script))
$: step = Number($page.url.searchParams.get('step')) || 1
if (script.content == '') {
@@ -73,6 +53,7 @@
script.content = initialCode(language, kind, template)
}
let loadingSave = false
async function editScript(leave: boolean): Promise<void> {
loadingSave = true
try {
@@ -230,111 +211,132 @@
<!-- metadata -->
{#if step === 1}
<CenteredPage>
<SettingSection title="Path" element="h2">
<Path
bind:this={pathC}
bind:error={pathError}
bind:path={script.path}
{initialPath}
on:enter={() => changeStep(2)}
namePlaceholder="script"
kind="script"
/>
</SettingSection>
<SettingSection title="Summary" element="h2">
<input
type="text"
bind:this={summaryC}
bind:value={script.summary}
placeholder="Short summary to be displayed when listed"
/>
</SettingSection>
<SettingSection title="Language" element="h2">
{#if lockedLanguage}
<div class="text-sm text-gray-600 italic mb-2">
As a forked script, the language '{script.language}' cannot be modified.
</div>
{/if}
<div class="flex flex-row gap-2 flex-wrap">
{#each langs as [label, lang]}
{@const isPicked = script.language == lang && template == 'script'}
<Button
size="sm"
variant="border"
color={isPicked ? 'blue' : 'dark'}
btnClasses={isPicked ? '!border-2 !bg-blue-50/75' : 'm-[1px]'}
on:click={() => {
script.language = lang
template = 'script'
initContent(lang, script.kind, template)
}}
disabled={lockedLanguage}
>
<LanguageIcon {lang} />
<span class="ml-2 py-2">{label}</span>
</Button>
{/each}
<h2 class="border-b pb-1 mt-8 mb-2">Path</h2>
<Path
bind:this={pathC}
bind:error={pathError}
bind:path={script.path}
{initialPath}
on:enter={() => changeStep(2)}
namePlaceholder="script"
kind="script"
/>
<h2 class="border-b pb-1 mt-12 mb-4">Summary</h2>
<input
type="text"
bind:this={summaryC}
bind:value={script.summary}
placeholder="Short summary to be displayed when listed"
/>
<h2 class="border-b pb-1 mt-12 mb-6">Language</h2>
{#if lockedLanguage}
<div class="text-sm text-gray-600 italic mb-2">
As a forked script, the language '{script.language}' cannot be modified.
</div>
{/if}
<div class="flex flex-row gap-2 flex-wrap">
{#each langs as [label, lang]}
{@const isPicked = script.language == lang && template == 'script'}
<Button
size="sm"
variant="border"
color={template == 'pgsql' ? 'blue' : 'dark'}
btnClasses={template == 'pgsql' ? '!border-2 !bg-blue-50/75' : 'm-[1px]'}
color={isPicked ? 'blue' : 'dark'}
btnClasses={isPicked ? '!border-2 !bg-blue-50/75' : 'm-[1px]'}
on:click={() => {
script.language = lang
template = 'script'
initContent(lang, script.kind, template)
}}
disabled={lockedLanguage}
on:click={() => {
script.language = Script.language.DENO
template = 'pgsql'
initContent(script.language, script.kind, template)
}}
>
<LanguageIcon lang="pgsql" /><span class="ml-2 py-2">PostgreSQL</span>
<LanguageIcon {lang} /><span class="ml-2 py-4">{label}</span>
</Button>
<!-- <Button
size="sm"
variant="border"
color={template == 'mysql' ? 'blue' : 'dark'}
btnClasses={template == 'mysql' ? '!border-2 !bg-blue-50/75' : 'm-[1px]'}
on:click={() => {
script.language = Script.language.DENO
template = 'mysql'
initContent(script.language, script.kind, template)
}}
>
<LanguageIcon lang="mysql" /><span class="ml-2 py-2">MySQL</span>
</Button> -->
</div>
</SettingSection>
<SettingSection
title="Script kind"
element="h3"
tooltip="Tag this script as having a specific purpose inside flows. If it won't be used in flows,
you don't have to worry about this."
accordion
>
<div class="flex flex-wrap gap-2">
{#each scriptKindOptions as { value, title, desc }}
{@const isPicked = script.kind === value}
<Button
size="sm"
variant="border"
color={isPicked ? 'blue' : 'dark'}
btnClasses="font-medium {isPicked ? '!bg-blue-50/75' : ''}"
on:click={() => {
{/each}
<Button
size="sm"
variant="border"
color={template == 'pgsql' ? 'blue' : 'dark'}
btnClasses={template == 'pgsql' ? '!border-2 !bg-blue-50/75' : 'm-[1px]'}
disabled={lockedLanguage}
on:click={() => {
script.language = Script.language.DENO
template = 'pgsql'
initContent(script.language, script.kind, template)
}}
>
<LanguageIcon lang="pgsql" /><span class="ml-2 py-4">PostgreSQL</span>
</Button>
<!-- <Button
size="sm"
variant="border"
color={template == 'mysql' ? 'blue' : 'dark'}
btnClasses={template == 'mysql' ? '!border-2 !bg-blue-50/75' : 'm-[1px]'}
on:click={() => {
script.language = Script.language.DENO
template = 'mysql'
initContent(script.language, script.kind, template)
}}
>
<LanguageIcon lang="mysql" /><span class="ml-2 py-4">MySQL</span>
</Button> -->
</div>
<div class="mt-16 mb-4">
<Button
color="light"
size="sm"
endIcon={{ icon: viewScriptKind ? faChevronUp : faChevronDown }}
on:click={() => (viewScriptKind = !viewScriptKind)}
>
Tag this script as having a specific purpose inside flows
</Button>
</div>
{#if viewScriptKind}
<div class="max-w-lg">
<RadioButton
label="Script Type"
options={[
['Action', Script.kind.SCRIPT],
[
{
title: 'Trigger',
desc: `First module of flows to trigger them based on watching changes external periodically using an internal state`
},
Script.kind.TRIGGER
],
[
{
title: 'Approval',
desc: `Send notification externally to ask for approval to continue a flow`
},
Script.kind.APPROVAL
],
[
{
title: 'Error Handler',
desc: `Handle errors for flows after all retries attempts have been exhausted`
},
Script.kind.FAILURE
]
// ['Command Handler', Script.kind.COMMAND]
]}
on:change={(e) => {
if (isInitialCode(script.content)) {
template = 'script'
script.kind = value
initContent(script.language, value, template)
}}
disabled={lockedLanguage}
>
{title}
{#if desc}
<Tooltip class="mb-0.5 ml-1">
{desc}
</Tooltip>
{/if}
</Button>
{/each}
initContent(script.language, e.detail, template)
}
}}
bind:value={script.kind}
/>
</div>
</SettingSection>
{/if}
<!-- <div class="ml-3">
<Toggle
bind:checked={script.is_template}
options={{ right: 'Save as a workspace template' }}
/>
</div> -->
</CenteredPage>
{:else if step === 2}
<ScriptEditor
@@ -344,6 +346,7 @@
lang={script.language}
{initialArgs}
{kind}
tour
/>
{:else if step === 3}
<CenteredPage>

View File

@@ -17,6 +17,7 @@
import { Button, Kbd } from './common'
import SplitPanesWrapper from './splitPanes/SplitPanesWrapper.svelte'
import WindmillIcon from './icons/WindmillIcon.svelte'
import { Tour } from './tutorial'
// Exported
export let schema: Schema = emptySchema()
@@ -27,6 +28,7 @@
export let initialArgs: Record<string, any> = {}
export let fixedOverflowWidgets = true
export let noSyncFromGithub = false
export let tour = false
let websocketAlive = { pyright: false, black: false, deno: false, go: false }
@@ -114,6 +116,10 @@
<svelte:window on:keydown={onKeyDown} />
{#if tour}
<Tour tutorial="script" />
{/if}
<div class="border-b-2 shadow-sm px-1 pr-4" bind:clientWidth={width}>
<div class="flex justify-between space-x-2">
<EditorBar
@@ -146,6 +152,7 @@
<Splitpanes class="!overflow-visible">
<Pane size={60} minSize={10} class="!overflow-visible">
<div
id="script-tutorial-1"
class="pl-2 h-full !overflow-visible"
on:mouseleave={() => {
inferSchema(code)
@@ -180,7 +187,7 @@
</Pane>
<Pane size={40} minSize={10}>
<div class="flex flex-col h-full">
<div class="px-2 w-full border-b py-1">
<div id="script-tutorial-4" class="px-2 w-full border-b py-1">
{#if testIsLoading}
<Button on:click={testJobLoader?.cancelJob} btnClasses="w-full" color="red" size="xs">
<WindmillIcon
@@ -211,7 +218,7 @@
</div>
<Splitpanes horizontal class="!max-h-[calc(100%-43px)]">
<Pane size={33}>
<div class="px-2">
<div id="script-tutorial-2" class="px-2">
<div class="break-words relative font-sans">
<SchemaForm compact {schema} bind:args bind:isValid />
</div>

View File

@@ -1,47 +0,0 @@
<script lang="ts">
import { ChevronDown } from 'lucide-svelte'
import { slide } from 'svelte/transition'
import type { IntRange } from '../common'
import Tooltip from './Tooltip.svelte'
export let title: string
export let tooltip: string = ''
export let element: `h${IntRange<1, 6>}` = 'h2'
export let accordion = false
let showContent = !accordion
</script>
<div class="border-b [&:has(button:hover)]:border-gray-400 duration-200 pb-1 mt-8 mb-2">
{#if accordion}
<button
class="flex w-full justify-start items-center"
on:click={() => (showContent = !showContent)}
>
<span class="rounded-full hover:bg-gray-100 focus:bg-gray-100 p-1 mr-1">
<ChevronDown size={22} class="rotate-0 duration-300 {showContent ? '!-rotate-180' : ''}" />
</span>
<svelte:element this={element}>
{title}
{#if tooltip}
<Tooltip scale={0.9} class="mb-0.5">
{tooltip}
</Tooltip>
{/if}
</svelte:element>
</button>
{:else}
<svelte:element this={element}>
{title}
{#if tooltip}
<Tooltip scale={0.9} class="mb-0.5">
{tooltip}
</Tooltip>
{/if}
</svelte:element>
{/if}
</div>
{#if showContent}
<div transition:slide|local={{ duration: 300 }}>
<slot />
</div>
{/if}

View File

@@ -99,7 +99,7 @@
}
}
export function format() {
function format() {
if (editor) {
code = getCode()
editor.getAction('editor.action.formatDocument').run()
@@ -176,6 +176,7 @@
editor.onDidBlurEditorText(() => {
code = getCode()
dispatch('blur')
})
if (lang == 'javascript') {

View File

@@ -2,18 +2,13 @@
import { ChevronDown, ChevronUp } from 'lucide-svelte'
import { slide } from 'svelte/transition'
import Button from './common/button/Button.svelte'
import Tooltip from './Tooltip.svelte'
export let text: string
export let tooltip: string | undefined = undefined
export let view = false
</script>
<Button color="light" on:click={() => (view = !view)} variant="border"
>{text}
{#if tooltip}
<Tooltip wrapperClass="mx-1">{tooltip}</Tooltip>
{/if}
{#if !view}<ChevronDown />{:else}<ChevronUp />{/if}</Button
>
{#if view}

View File

@@ -573,13 +573,10 @@
editor.onDidBlurEditorText(() => {
code = getCode()
dispatch('blur')
})
}
export function focus() {
editor?.focus()
}
onMount(() => {
if (browser) {
loadMonaco()

View File

@@ -10,14 +10,13 @@
export let disabled = false
export let textClass = ''
export let textStyle = ''
export let color: 'blue' | 'red' = 'blue'
export let size: 'sm' | 'xs' = 'sm'
const id = (Math.random() + 1).toString(36).substring(10)
const dispatch = createEventDispatcher()
</script>
<span class="{$$props.class} z-auto">
<span class="{$$props.class ?? ''} z-auto">
<label
for={id}
class="inline-flex items-center mt-2 duration-200 {disabled
@@ -27,7 +26,7 @@
{#if Boolean(options?.left)}
<span
class={twMerge(
'mr-2 font-medium duration-200',
'ml-2 font-medium duration-200',
disabled ? 'text-gray-500' : 'text-gray-900',
size === 'xs' ? 'text-xs' : 'text-sm',
textClass
@@ -38,7 +37,7 @@
</span>
{/if}
<!-- svelte-ignore a11y-click-events-have-key-events -->
<div class="relative" on:click|stopPropagation>
<div class="relative" on:pointerdown on:click|stopPropagation>
<input
on:focus
on:click
@@ -56,9 +55,7 @@
class="w-11 h-6 bg-gray-200 rounded-full peer peer-focus:ring-4 peer-focus:ring-blue-300
peer-checked:after:translate-x-full peer-checked:after:border-white after:content-['']
after:absolute after:top-0.5 after:left-[2px] after:bg-white after:border-gray-300
after:border after:rounded-full after:h-5 after:w-5 after:transition-all {color == 'red'
? 'peer-checked:bg-red-600'
: 'peer-checked:bg-blue-600'}"
after:border after:rounded-full after:h-5 after:w-5 after:transition-all peer-checked:bg-blue-600"
/>
</div>
{#if Boolean(options?.right)}

View File

@@ -18,7 +18,5 @@
data={faInfoCircle}
{scale}
/>
<svelte:fragment slot="text">
<slot />
</svelte:fragment>
<span slot="text"> <slot /> </span>
</Popover>

View File

@@ -57,8 +57,7 @@
decryptSecret: false
})
can_write =
getV.workspace_id == $workspaceStore &&
canWrite(edit_path, getV.extra_perms ?? {}, $userStore)
getV.workspace_id == $workspaceStore && canWrite(path, getV.extra_perms ?? {}, $userStore)
variable = {
value: getV.value ?? '',

View File

@@ -9,7 +9,9 @@
...new Set(
workers
.filter((worker) => {
return worker.ip != 'unretrievable IP' && worker.last_ping && worker.last_ping < 300
const date = new Date().getTime() - 300 * 60
const ping_at = new Date(worker.ping_at).getTime()
return worker.ip != 'unretrievable IP' && ping_at > date
})
.map((worker) => worker.ip)
)

View File

@@ -1,149 +1,176 @@
<script lang="ts">
import { Button } from '$lib/components/common'
import { Button, type ButtonType } from '$lib/components/common'
import { getContext } from 'svelte'
import type { AppInput } from '../../inputType'
import type { AppViewerContext, ComponentCustomCSS, RichConfigurations } from '../../types'
import type { Output } from '../../rx'
import type { AppViewerContext, ComponentCustomCSS } from '../../types'
import AlignWrapper from '../helpers/AlignWrapper.svelte'
import InputValue from '../helpers/InputValue.svelte'
import type RunnableComponent from '../helpers/RunnableComponent.svelte'
import RunnableWrapper from '../helpers/RunnableWrapper.svelte'
import { loadIcon } from '../icon'
import { twMerge } from 'tailwind-merge'
import { initConfig, initOutput } from '../../editor/appUtils'
import { components } from '../../editor/component'
import ResolveConfig from '../helpers/ResolveConfig.svelte'
import { goto } from '$app/navigation'
export let id: string
export let componentInput: AppInput | undefined
export let configuration: RichConfigurations
export let configuration: Record<string, AppInput>
export let recomputeIds: string[] | undefined = undefined
export let extraQueryParams: Record<string, any> = {}
export let horizontalAlignment: 'left' | 'center' | 'right' | undefined = undefined
export let verticalAlignment: 'top' | 'center' | 'bottom' | undefined = undefined
export let noWFull = false
export let preclickAction: (() => Promise<void>) | undefined = undefined
export let customCss: ComponentCustomCSS<'buttoncomponent'> | undefined = undefined
export let customCss: ComponentCustomCSS<'button'> | undefined = undefined
export let render: boolean
export let initializing: boolean | undefined = false
export let extraKey: string | undefined = undefined
export let controls: { left: () => boolean; right: () => boolean | string } | undefined =
undefined
export const staticOutputs: string[] = ['loading', 'result']
const { worldStore, app, componentControl, selectedComponent } =
getContext<AppViewerContext>('AppViewerContext')
let resolvedConfig = initConfig(
components['buttoncomponent'].initialData.configuration,
configuration
)
$: initializing = resolvedConfig?.label == undefined
let outputs = initOutput($worldStore, id, {
result: undefined,
loading: false
})
if (controls) {
$componentControl[id] = controls
}
const { runnableComponents, worldStore, app } = getContext<AppViewerContext>('AppViewerContext')
let labelValue: string
let color: ButtonType.Color
let size: ButtonType.Size
let runnableComponent: RunnableComponent
let disabled: boolean | undefined = undefined
let fillContainer: boolean | undefined = undefined
let gotoUrl: string | undefined = undefined
let gotoNewTab: boolean | undefined = undefined
let isLoading: boolean = false
let ownClick: boolean = false
let triggerOnAppLoad = false
let beforeIcon: undefined | string = undefined
let afterIcon: undefined | string = undefined
let beforeIconComponent: any
let afterIconComponent: any
$: resolvedConfig.beforeIcon && handleBeforeIcon()
$: resolvedConfig.afterIcon && handleAfterIcon()
$: beforeIcon && handleBeforeIcon()
$: afterIcon && handleAfterIcon()
async function handleBeforeIcon() {
if (resolvedConfig.beforeIcon) {
beforeIconComponent = await loadIcon(resolvedConfig.beforeIcon)
if (beforeIcon) {
beforeIconComponent = await loadIcon(beforeIcon)
}
}
async function handleAfterIcon() {
if (resolvedConfig.afterIcon) {
afterIconComponent = await loadIcon(resolvedConfig.afterIcon)
if (afterIcon) {
afterIconComponent = await loadIcon(afterIcon)
}
}
$: resolvedConfig?.triggerOnAppLoad && runnableComponent?.runComponent()
$: outputs = $worldStore?.outputsById[id] as {
result: Output<Array<any>>
loading: Output<boolean>
}
$: triggerOnAppLoad && runnableComponent?.runComponent()
$: if (outputs?.loading != undefined) {
outputs.loading.set(false, true)
}
$: outputs?.loading.subscribe({
next: (value) => {
isLoading = value
if (ownClick && !value) {
ownClick = false
}
}
})
$: loading = isLoading && ownClick
let errors: Record<string, string> = {}
$: errorsMessage = Object.values(errors)
.filter((x) => x != '')
.join('\n')
let runnableWrapper: RunnableWrapper
async function handleClick(event: CustomEvent) {
event?.stopPropagation()
event?.preventDefault()
$selectedComponent = [id]
if (preclickAction) {
await preclickAction()
}
ownClick = true
if (!runnableComponent) {
runnableWrapper.onSuccess()
if (gotoUrl) {
if (gotoNewTab) {
window.open(gotoUrl, '_blank')
} else {
goto(gotoUrl)
}
}
} else {
await runnableComponent?.runComponent()
}
if (recomputeIds) {
await Promise.all(recomputeIds.map((id) => $runnableComponents?.[id]?.()))
}
}
let loading = false
</script>
{#each Object.keys(components['buttoncomponent'].initialData.configuration) as key (key)}
<ResolveConfig
{id}
{extraKey}
{key}
bind:resolvedConfig={resolvedConfig[key]}
configuration={configuration[key]}
/>
{/each}
<InputValue {id} input={configuration.label} bind:value={labelValue} />
<InputValue {id} input={configuration.goto} bind:value={gotoUrl} />
<InputValue {id} input={configuration.color} bind:value={color} />
<InputValue {id} input={configuration.size} bind:value={size} />
<InputValue {id} input={configuration.beforeIcon} bind:value={beforeIcon} />
<InputValue {id} input={configuration.afterIcon} bind:value={afterIcon} />
<InputValue {id} input={configuration.triggerOnAppLoad} bind:value={triggerOnAppLoad} />
<!-- gotoNewTab={resolvedConfig.onSuccess.selected == 'goto'} -->
<InputValue
{id}
input={configuration.disabled}
bind:value={disabled}
bind:error={errors.disabled}
/>
<InputValue {id} input={configuration.fillContainer} bind:value={fillContainer} />
<InputValue {id} input={configuration.gotoNewTab} bind:value={gotoNewTab} />
<RunnableWrapper
bind:this={runnableWrapper}
{recomputeIds}
flexWrap
bind:runnableComponent
bind:loading
{componentInput}
doOnSuccess={resolvedConfig.onSuccess}
{id}
{extraQueryParams}
autoRefresh={false}
goto={gotoUrl}
{gotoNewTab}
{render}
{outputs}
{extraKey}
>
<AlignWrapper {noWFull} {horizontalAlignment} {verticalAlignment}>
{#if errorsMessage}
<div class="text-red-500 text-xs">{errorsMessage}</div>
{/if}
<Button
on:pointerdown={(e) => e.stopPropagation()}
btnClasses={twMerge(
$app.css?.['buttoncomponent']?.['button']?.class,
customCss?.button?.class,
resolvedConfig.fillContainer ? 'w-full h-full' : ''
customCss?.button.class,
fillContainer ? 'w-full h-full' : ''
)}
style={[$app.css?.['buttoncomponent']?.['button']?.style, customCss?.button?.style].join(';')}
disabled={resolvedConfig.disabled}
style={[$app.css?.['buttoncomponent']?.['button']?.style, customCss?.button.style].join(';')}
{disabled}
on:pointerdown={(e) => {
e?.stopPropagation()
window.dispatchEvent(new Event('pointerup'))
}}
on:click={handleClick}
size={resolvedConfig.size}
color={resolvedConfig.color}
{size}
{color}
{loading}
>
<span class="truncate inline-flex gap-2 items-center">
{#if resolvedConfig.beforeIcon && beforeIconComponent}
{#if beforeIcon && beforeIconComponent}
<svelte:component this={beforeIconComponent} size={14} />
{/if}
<div>{resolvedConfig.label}</div>
{#if resolvedConfig.afterIcon && afterIconComponent}
<div>{labelValue}</div>
{#if afterIcon && afterIconComponent}
<svelte:component this={afterIconComponent} size={14} />
{/if}
</span>

View File

@@ -3,39 +3,34 @@
import { faUser } from '@fortawesome/free-solid-svg-icons'
import { getContext } from 'svelte'
import { Icon } from 'svelte-awesome'
import { initConfig, initOutput } from '../../editor/appUtils'
import { components } from '../../editor/component'
import type { AppInput } from '../../inputType'
import type { AppViewerContext, ComponentCustomCSS, RichConfigurations } from '../../types'
import type { Output } from '../../rx'
import type { AppViewerContext, ComponentCustomCSS } from '../../types'
import { concatCustomCss } from '../../utils'
import AlignWrapper from '../helpers/AlignWrapper.svelte'
import ResolveConfig from '../helpers/ResolveConfig.svelte'
import InputValue from '../helpers/InputValue.svelte'
import type RunnableComponent from '../helpers/RunnableComponent.svelte'
import RunnableWrapper from '../helpers/RunnableWrapper.svelte'
export let id: string
export let componentInput: AppInput | undefined
export let configuration: RichConfigurations
export let configuration: Record<string, AppInput>
export let recomputeIds: string[] | undefined = undefined
export let extraQueryParams: Record<string, any> = {}
export let horizontalAlignment: 'left' | 'center' | 'right' | undefined = undefined
export let customCss: ComponentCustomCSS<'formcomponent'> | undefined = undefined
export let customCss: ComponentCustomCSS<'container' | 'button'> | undefined = undefined
export let render: boolean
export const staticOutputs: string[] = ['loading', 'result']
const { app, worldStore, stateId } = getContext<AppViewerContext>('AppViewerContext')
const { app, runnableComponents, worldStore, stateId } =
getContext<AppViewerContext>('AppViewerContext')
let outputs = initOutput($worldStore, id, {
result: undefined,
loading: false
})
let resolvedConfig = initConfig(
components['formcomponent'].initialData.configuration,
configuration
)
let labelValue: string = 'Default label'
let color: ButtonType.Color
let size: ButtonType.Size
let runnableComponent: RunnableComponent
let goto: string | undefined = undefined
let isLoading: boolean = false
@@ -43,34 +38,40 @@
$stateId != undefined &&
(componentInput?.type != 'runnable' || Object.keys(componentInput?.fields ?? {}).length == 0)
$: css = concatCustomCss($app.css?.formcomponent, customCss)
$: outputs = $worldStore?.outputsById[id] as {
result: Output<Array<any>>
loading: Output<boolean>
}
let loading = false
$: if (outputs?.loading != undefined) {
outputs.loading.set(false, true)
}
$: outputs?.loading.subscribe({
next: (value) => {
isLoading = value
}
})
$: css = concatCustomCss($app.css?.formcomponent, customCss)
</script>
{#each Object.keys(components['formcomponent'].initialData.configuration) as key (key)}
<ResolveConfig
{id}
{key}
bind:resolvedConfig={resolvedConfig[key]}
configuration={configuration[key]}
/>
{/each}
<InputValue {id} input={configuration.goto} bind:value={goto} />
<InputValue {id} input={configuration.label} bind:value={labelValue} />
<InputValue {id} input={configuration.color} bind:value={color} />
<InputValue {id} input={configuration.size} bind:value={size} />
<RunnableWrapper
{recomputeIds}
{render}
bind:runnableComponent
bind:loading
{componentInput}
{id}
doOnSuccess={resolvedConfig.onSuccess}
{goto}
{extraQueryParams}
autoRefresh={false}
forceSchemaDisplay={true}
runnableClass="!block"
runnableStyle={css?.container?.style}
{outputs}
runnableStyle={css?.container.style}
>
<AlignWrapper {horizontalAlignment}>
<div
@@ -98,14 +99,21 @@
style={css?.button?.style ?? ''}
on:pointerdown={(e) => {
e?.stopPropagation()
window.dispatchEvent(new Event('pointerup'))
}}
on:click={() => {
runnableComponent?.runComponent()
if (recomputeIds) {
recomputeIds.forEach((id) => {
$runnableComponents[id]?.()
})
}
}}
size={resolvedConfig.size}
color={resolvedConfig.color}
{size}
{color}
>
{resolvedConfig.label}
{labelValue}
</Button>
{/if}
</div>

View File

@@ -1,46 +1,44 @@
<script lang="ts">
import { Button } from '$lib/components/common'
import { Button, type ButtonType } from '$lib/components/common'
import { faUser } from '@fortawesome/free-solid-svg-icons'
import { getContext } from 'svelte'
import { Icon } from 'svelte-awesome'
import type { AppInput } from '../../inputType'
import type { AppViewerContext, ComponentCustomCSS, RichConfigurations } from '../../types'
import type { Output } from '../../rx'
import type { AppViewerContext, ComponentCustomCSS } from '../../types'
import AlignWrapper from '../helpers/AlignWrapper.svelte'
import InputValue from '../helpers/InputValue.svelte'
import type RunnableComponent from '../helpers/RunnableComponent.svelte'
import RunnableWrapper from '../helpers/RunnableWrapper.svelte'
import Portal from 'svelte-portal'
import Modal from '$lib/components/common/modal/Modal.svelte'
import { concatCustomCss } from '../../utils'
import { initConfig, initOutput } from '../../editor/appUtils'
import { components } from '../../editor/component'
import ResolveConfig from '../helpers/ResolveConfig.svelte'
import AlwaysMountedModal, {
getModal
} from '$lib/components/common/modal/AlwaysMountedModal.svelte'
export let id: string
export let componentInput: AppInput | undefined
export let configuration: RichConfigurations
export let configuration: Record<string, AppInput>
export let recomputeIds: string[] | undefined = undefined
export let extraQueryParams: Record<string, any> = {}
export let horizontalAlignment: 'left' | 'center' | 'right' | undefined = undefined
export let verticalAlignment: 'top' | 'center' | 'bottom' | undefined = undefined
export let customCss: ComponentCustomCSS<'formbuttoncomponent'> | undefined = undefined
export let customCss: ComponentCustomCSS<'button' | 'popup'> | undefined = undefined
export let render: boolean
const { app, worldStore } = getContext<AppViewerContext>('AppViewerContext')
export const staticOutputs: string[] = ['loading', 'result']
let outputs = initOutput($worldStore, id, {
result: undefined,
loading: false
})
const { app, runnableComponents, worldStore } = getContext<AppViewerContext>('AppViewerContext')
let resolvedConfig = initConfig(
components['formbuttoncomponent'].initialData.configuration,
configuration
)
let labelValue: string = 'Default label'
let color: ButtonType.Color
let size: ButtonType.Size
let runnableComponent: RunnableComponent
let disabled: boolean | undefined = undefined
let isLoading: boolean = false
let ownClick: boolean = false
let errors: Record<string, string> = {}
let open: boolean = false
$: errorsMessage = Object.values(errors)
.filter((x) => x != '')
@@ -49,96 +47,120 @@
$: noInputs =
componentInput?.type != 'runnable' || Object.keys(componentInput?.fields ?? {}).length == 0
$: outputs = $worldStore?.outputsById[id] as {
result: Output<Array<any>>
loading: Output<boolean>
}
$: if (outputs?.loading != undefined) {
outputs.loading.set(false, true)
}
$: outputs?.loading.subscribe({
next: (value) => {
isLoading = value
if (ownClick && !value) {
ownClick = false
}
}
})
$: loading = isLoading && ownClick
$: css = concatCustomCss($app?.css?.formbuttoncomponent, customCss)
let runnableWrapper: RunnableWrapper
let loading = false
</script>
{#each Object.keys(components['formbuttoncomponent'].initialData.configuration) as key (key)}
<ResolveConfig
{id}
{key}
bind:resolvedConfig={resolvedConfig[key]}
configuration={configuration[key]}
/>
{/each}
<InputValue {id} input={configuration.label} bind:value={labelValue} />
<InputValue {id} input={configuration.color} bind:value={color} />
<InputValue {id} input={configuration.size} bind:value={size} />
<InputValue
{id}
input={configuration.disabled}
bind:value={disabled}
bind:error={errors.disabled}
/>
<AlwaysMountedModal
title={resolvedConfig.label ?? ''}
class={css?.popup?.class}
style={css?.popup?.style}
>
<RunnableWrapper
bind:this={runnableWrapper}
{recomputeIds}
{render}
bind:runnableComponent
{componentInput}
{id}
{extraQueryParams}
autoRefresh={false}
forceSchemaDisplay={true}
runnableClass="!block"
{outputs}
doOnSuccess={resolvedConfig.onSuccess}
<Portal>
<Modal
{open}
title={labelValue}
class={css?.popup.class}
style={css?.popup.style}
on:canceled={() => {
open = false
}}
on:confirmed={() => {
open = false
}}
>
<div class="flex flex-col gap-2 px-4 w-full">
<div>
{#if noInputs}
<div class="text-gray-600 italic text-sm my-4">
Run forms are associated with a runnable that has user inputs.
<br />
Once a script or flow is chosen, set some <strong>Runnable Inputs</strong> to
<strong>
User Input
<Icon data={faUser} scale={1.3} class="rounded-sm bg-gray-200 p-1 ml-0.5" />
</strong>
</div>
{/if}
</div>
<div class="flex justify-end">
<Button
{loading}
btnClasses="my-1"
on:pointerdown={(e) => {
e?.stopPropagation()
}}
on:click={async () => {
if (!runnableComponent) {
runnableWrapper.onSuccess()
} else {
<RunnableWrapper
{render}
bind:runnableComponent
{componentInput}
{id}
{extraQueryParams}
autoRefresh={false}
forceSchemaDisplay={true}
runnableClass="!block"
>
<div class="flex flex-col gap-2 px-4 w-full">
<div>
{#if noInputs}
<div class="text-gray-600 italic text-sm my-4">
Run forms are associated with a runnable that has user inputs.
<br />
Once a script or flow is chosen, set some <strong>Runnable Inputs</strong> to
<strong>
User Input
<Icon data={faUser} scale={1.3} class="rounded-sm bg-gray-200 p-1 ml-0.5" />
</strong>
</div>
{/if}
</div>
<div class="flex justify-end">
<Button
{loading}
btnClasses="my-1"
on:pointerdown={(e) => {
e?.stopPropagation()
window.dispatchEvent(new Event('pointerup'))
}}
on:click={async () => {
await runnableComponent?.runComponent()
}
getModal().close()
}}
size="xs"
color="dark"
>
Submit
</Button>
if (recomputeIds) {
recomputeIds.forEach((id) => {
$runnableComponents[id]?.()
})
}
open = false
}}
size="xs"
color="dark"
>
Submit
</Button>
</div>
</div>
</div>
</RunnableWrapper>
</AlwaysMountedModal>
</RunnableWrapper>
</Modal>
</Portal>
<AlignWrapper {horizontalAlignment} {verticalAlignment}>
{#if errorsMessage}
<div class="text-red-500 text-xs">{errorsMessage}</div>
{/if}
<Button
disabled={resolvedConfig.disabled ?? false}
size={resolvedConfig.size ?? 'md'}
color={resolvedConfig.color}
{disabled}
{size}
{color}
btnClasses={css?.button?.class ?? ''}
style={css?.button?.style ?? ''}
on:click={(e) => {
getModal().open()
open = true
}}
>
{resolvedConfig.label}
{labelValue}
</Button>
</AlignWrapper>

View File

@@ -13,26 +13,21 @@
} from 'chart.js'
import type { ChartOptions } from 'chart.js'
import RunnableWrapper from '../helpers/RunnableWrapper.svelte'
import type { AppInput, RichAppInput } from '../../inputType'
import type { AppInput } from '../../inputType'
import InputValue from '../helpers/InputValue.svelte'
import { concatCustomCss } from '../../utils'
import { getContext } from 'svelte'
import type { AppViewerContext, ComponentCustomCSS, RichConfigurations } from '../../types'
import { initOutput } from '../../editor/appUtils'
import type { AppViewerContext, ComponentCustomCSS } from '../../types'
export let id: string
export let componentInput: AppInput | undefined
export let configuration: RichConfigurations
export let configuration: Record<string, AppInput>
export let initializing: boolean | undefined = undefined
export let customCss: ComponentCustomCSS<'barchartcomponent'> | undefined = undefined
export let customCss: ComponentCustomCSS<'container'> | undefined = undefined
export let render: boolean
const { app, worldStore } = getContext<AppViewerContext>('AppViewerContext')
let outputs = initOutput($worldStore, id, {
result: undefined,
loading: false
})
export const staticOutputs: string[] = ['loading', 'result']
const { app } = getContext<AppViewerContext>('AppViewerContext')
ChartJS.register(
Title,
@@ -95,7 +90,7 @@
<InputValue {id} input={configuration.theme} bind:value={theme} />
<InputValue {id} input={configuration.line} bind:value={lineChart} />
<RunnableWrapper {outputs} {render} autoRefresh {componentInput} {id} bind:initializing bind:result>
<RunnableWrapper {render} flexWrap autoRefresh {componentInput} {id} bind:initializing bind:result>
<div class="w-full h-full {css?.container?.class ?? ''}" style={css?.container?.style ?? ''}>
{#if result}
{#if lineChart}

View File

@@ -2,7 +2,6 @@
import DisplayResult from '$lib/components/DisplayResult.svelte'
import { getContext } from 'svelte'
import { twMerge } from 'tailwind-merge'
import { initOutput } from '../../editor/appUtils'
import type { AppInput } from '../../inputType'
import {
IS_APP_PUBLIC_CONTEXT_KEY,
@@ -14,42 +13,33 @@
export let id: string
export let componentInput: AppInput | undefined
export let initializing: boolean | undefined = undefined
export let customCss: ComponentCustomCSS<'displaycomponent'> | undefined = undefined
export let customCss: ComponentCustomCSS<'header' | 'container'> | undefined = undefined
export let render: boolean
const requireHtmlApproval = getContext<boolean | undefined>(IS_APP_PUBLIC_CONTEXT_KEY)
const { app, worldStore } = getContext<AppViewerContext>('AppViewerContext')
const { app } = getContext<AppViewerContext>('AppViewerContext')
let result: any = undefined
const outputs = initOutput($worldStore, id, {
result: undefined,
loading: false
})
export const staticOutputs: string[] = ['result', 'loading']
</script>
<RunnableWrapper {outputs} {render} {componentInput} {id} bind:initializing bind:result>
<div class="flex flex-col w-full h-full">
<div
class={twMerge(
'w-full border-b px-2 text-xs p-1 font-semibold bg-gray-500 text-white rounded-t-sm',
$app.css?.['displaycomponent']?.['header']?.class,
customCss?.header?.class
)}
>
Results
</div>
<div
style={twMerge(
$app.css?.['displaycomponent']?.['container']?.style,
customCss?.container?.style
)}
class={twMerge(
'p-2 grow',
$app.css?.['displaycomponent']?.['container']?.class,
customCss?.container?.class
)}
>
<DisplayResult {result} {requireHtmlApproval} />
</div>
<RunnableWrapper {render} flexWrap {componentInput} {id} bind:initializing bind:result>
<div
class={twMerge(
'w-full border-b px-2 text-xs p-1 font-semibold bg-gray-500 text-white rounded-t-sm',
$app.css?.['displaycomponent']?.['header']?.class,
customCss?.header?.class
)}
>
Results
</div>
<div
class={twMerge(
'p-2',
$app.css?.['displaycomponent']?.['container']?.class,
customCss?.container?.class
)}
>
<DisplayResult {result} {requireHtmlApproval} />
</div>
</RunnableWrapper>

View File

@@ -1,6 +1,5 @@
<script lang="ts">
import { getContext } from 'svelte'
import { initOutput } from '../../editor/appUtils'
import type { AppInput } from '../../inputType'
import type { AppViewerContext, ComponentCustomCSS } from '../../types'
import { concatCustomCss } from '../../utils'
@@ -9,15 +8,11 @@
export let id: string
export let componentInput: AppInput | undefined
export let initializing: boolean | undefined = undefined
export let customCss: ComponentCustomCSS<'htmlcomponent'> | undefined = undefined
export let customCss: ComponentCustomCSS<'container'> | undefined = undefined
export let render: boolean
const { app, worldStore, selectedComponent } = getContext<AppViewerContext>('AppViewerContext')
const outputs = initOutput($worldStore, id, {
result: undefined,
loading: false
})
export const staticOutputs: string[] = ['result', 'loading']
const { app } = getContext<AppViewerContext>('AppViewerContext')
let result: string | undefined = undefined
let h: number | undefined = undefined
@@ -34,14 +29,10 @@
bind:clientHeight={h}
bind:clientWidth={w}
>
<button
class="absolute bottom-0 left-0 text-xs border px-2 py-0.5 bg-white/80"
on:click={() => ($selectedComponent = [id])}>Select</button
>
<RunnableWrapper
{outputs}
{render}
autoRefresh
flexWrap
{componentInput}
{id}
bind:initializing
@@ -58,7 +49,5 @@
: 'No html'}
/>
{/key}
<!-- svelte-ignore a11y-click-events-have-key-events -->
<div on:click|stopPropagation class="absolute top-0 h-full w-full" />
</RunnableWrapper>
</div>

View File

@@ -1,12 +1,7 @@
<script lang="ts">
import { getContext } from 'svelte'
import { initOutput } from '../../editor/appUtils'
import type {
AppViewerContext,
ComponentCustomCSS,
RichConfiguration,
RichConfigurations
} from '../../types'
import type { AppInput } from '../../inputType'
import type { AppViewerContext, ComponentCustomCSS } from '../../types'
import { concatCustomCss } from '../../utils'
import { AlignWrapper, InputValue } from '../helpers'
import { loadIcon } from '../icon'
@@ -14,14 +9,12 @@
export let id: string
export let horizontalAlignment: 'left' | 'center' | 'right' | undefined = 'left'
export let verticalAlignment: 'top' | 'center' | 'bottom' | undefined = undefined
export let configuration: RichConfigurations
export let customCss: ComponentCustomCSS<'iconcomponent'> | undefined = undefined
export let configuration: Record<string, AppInput>
export const staticOutputs: string[] = []
export let customCss: ComponentCustomCSS<'container' | 'icon'> | undefined = undefined
export let render: boolean
const { app, worldStore } = getContext<AppViewerContext>('AppViewerContext')
//used so that we can count number of outputs setup for first refresh
initOutput($worldStore, id, {})
const { app } = getContext<AppViewerContext>('AppViewerContext')
let icon: string | undefined = undefined
let size: number

View File

@@ -1,56 +1,44 @@
<script lang="ts">
import { getContext } from 'svelte'
import { twMerge } from 'tailwind-merge'
import { initConfig, initOutput } from '../../editor/appUtils'
import { components, selectOptions } from '../../editor/component'
import type { AppViewerContext, ComponentCustomCSS, RichConfigurations } from '../../types'
import type { staticValues } from '../../editor/componentsPanel/componentStaticValues'
import type { AppInput } from '../../inputType'
import type { AppViewerContext, ComponentCustomCSS } from '../../types'
import { concatCustomCss } from '../../utils'
import Loader from '../helpers/Loader.svelte'
import ResolveConfig from '../helpers/ResolveConfig.svelte'
import InputValue from '../helpers/InputValue.svelte'
type FitOption = (typeof staticValues)['objectFitOptions'][number]
export let id: string
export let configuration: RichConfigurations
export let customCss: ComponentCustomCSS<'imagecomponent'> | undefined = undefined
export let configuration: Record<string, AppInput>
export const staticOutputs: string[] = ['loading']
export let customCss: ComponentCustomCSS<'image'> | undefined = undefined
export let render: boolean
let resolvedConfig = initConfig(
components['imagecomponent'].initialData.configuration,
configuration
)
const { app, worldStore } = getContext<AppViewerContext>('AppViewerContext')
const fit: Record<string, string> = {
const { app } = getContext<AppViewerContext>('AppViewerContext')
const fit: Record<FitOption, string> = {
cover: 'object-cover',
contain: 'object-contain',
fill: 'object-fill'
}
//used so that we can count number of outputs setup for first refresh
initOutput($worldStore, id, {})
let source: string | undefined = undefined
let imageFit: FitOption | undefined = undefined
let altText: string | undefined = undefined
$: css = concatCustomCss($app.css?.imagecomponent, customCss)
</script>
{#each Object.keys(components['imagecomponent'].initialData.configuration) as key (key)}
<ResolveConfig
{id}
{key}
bind:resolvedConfig={resolvedConfig[key]}
configuration={configuration[key]}
/>
{/each}
<InputValue {id} input={configuration.source} bind:value={source} />
<InputValue {id} input={configuration.imageFit} bind:value={imageFit} />
<InputValue {id} input={configuration.altText} bind:value={altText} />
{#if render}
<Loader loading={resolvedConfig.source == undefined}>
<img
on:pointerdown|preventDefault
src={resolvedConfig.source}
alt={resolvedConfig.altText}
style={css?.image?.style ?? ''}
class={twMerge(
`w-full h-full ${fit[resolvedConfig.imageFit || 'cover']}`,
css?.image?.class ?? ''
)}
/>
</Loader>
<img
on:pointerdown|preventDefault
src={source}
alt={altText}
style={css?.image?.style ?? ''}
class={twMerge(`w-full h-full ${fit[imageFit || 'cover']}`, css?.image?.class ?? '')}
/>
{/if}

View File

@@ -1,12 +1,8 @@
<script lang="ts">
import { getContext } from 'svelte'
import { getContext, onMount } from 'svelte'
import { concatCustomCss } from '../../utils'
import type {
AppViewerContext,
ComponentCustomCSS,
RichConfiguration,
RichConfigurations
} from '../../types'
import type { AppInput } from '../../inputType'
import type { AppViewerContext, ComponentCustomCSS } from '../../types'
import { InputValue } from '../helpers'
import { twMerge } from 'tailwind-merge'
import { Map, View, Feature } from 'ol'
@@ -15,7 +11,8 @@
import { Vector as VectorLayer, Tile as TileLayer } from 'ol/layer'
import { Point } from 'ol/geom'
import { defaults as defaultControls } from 'ol/control'
import { findGridItem, initOutput } from '../../editor/appUtils'
import { findGridItem } from '../../editor/appUtils'
import type { Output } from '../../rx'
interface Marker {
lon: number
@@ -32,19 +29,20 @@
} as const
export let id: string
export let configuration: RichConfigurations
export let customCss: ComponentCustomCSS<'mapcomponent'> | undefined = undefined
export let configuration: Record<string, AppInput>
export const staticOutputs: string[] = ['mapRegion']
export let customCss: ComponentCustomCSS<'map'> | undefined = undefined
export let render: boolean
const { app, worldStore, selectedComponent, connectingInput, focusedGrid, mode } =
getContext<AppViewerContext>('AppViewerContext')
let outputs = initOutput($worldStore, id, {
mapRegion: {
topLeft: { lat: 0, lon: 0 },
bottomRight: { lat: 0, lon: 0 }
}
})
$: outputs = $worldStore?.outputsById[id] as {
mapRegion: Output<{
topLeft: { lat: number; lon: number }
bottomRight: { lat: number; lon: number }
}>
}
let map: Map
let mapElement: HTMLDivElement
@@ -69,7 +67,7 @@
function selectComponent() {
if (!$connectingInput.opened) {
$selectedComponent = [id]
$selectedComponent = id
$focusedGrid = undefined
}
}
@@ -128,7 +126,7 @@
createMarkerLayers()?.forEach((l) => map.addLayer(l))
}
$: if (!map && mapElement) {
onMount(() => {
useGeographic()
map = new Map({
target: mapElement,
@@ -147,7 +145,7 @@
})
})
updateRegionOutput()
}
})
$: css = concatCustomCss($app.css?.mapcomponent, customCss)
@@ -175,7 +173,6 @@
updateRegionOutput()
if (z) {
//@ts-ignore
gridItem.data.configuration.zoom.value = z
}
@@ -185,9 +182,7 @@
}
if (gridItem) {
//@ts-ignore
gridItem.data.configuration.longitude.value = center[0]
//@ts-ignore
gridItem.data.configuration.latitude.value = center[1]
}
}

View File

@@ -3,31 +3,23 @@
import { twMerge } from 'tailwind-merge'
import { getDocument, type PDFDocumentProxy, type PDFPageProxy } from 'pdfjs-dist'
import 'pdfjs-dist/build/pdf.worker.entry'
import type {
AppViewerContext,
ComponentCustomCSS,
RichConfiguration,
RichConfigurations
} from '../../types'
import type { AppInput } from '../../inputType'
import type { AppViewerContext, ComponentCustomCSS } from '../../types'
import { concatCustomCss } from '../../utils'
import InputValue from '../helpers/InputValue.svelte'
import { throttle } from '../../../../utils'
import { Button } from '../../../common'
import { Download, Loader2, MoveHorizontal, ZoomIn, ZoomOut } from 'lucide-svelte'
import { fade } from 'svelte/transition'
import { findGridItem, initOutput } from '../../editor/appUtils'
import { findGridItem } from '../../editor/appUtils'
export let id: string
export let configuration: RichConfigurations
export let customCss: ComponentCustomCSS<'pdfcomponent'> | undefined = undefined
export let configuration: Record<string, AppInput>
export const staticOutputs: string[] = ['loading']
export let customCss: ComponentCustomCSS<'container'> | undefined = undefined
export let render: boolean
const { app, mode, worldStore, selectedComponent } =
getContext<AppViewerContext>('AppViewerContext')
const outputs = initOutput($worldStore, id, {
loading: false
})
const { app, mode, selectedComponent } = getContext<AppViewerContext>('AppViewerContext')
let source: string | ArrayBuffer | undefined = undefined
let wrapper: HTMLDivElement | undefined = undefined
@@ -67,7 +59,6 @@
return
}
try {
outputs.loading.set(true)
await resetDoc()
doc = await getDocument(src).promise
pageNumber = 1
@@ -78,7 +69,6 @@
error = err?.message ?? (typeof err === 'string' ? err : 'Error loading PDF')
console.log(err)
}
outputs.loading.set(false)
}
async function renderPdf(scaleToViewport = true, resizing = false) {
@@ -160,9 +150,7 @@
function syncZoomValue() {
const gridItem = findGridItem($app, id)
//@ts-ignore
if (gridItem && gridItem.data.configuration.zoom.value !== zoom) {
//@ts-ignore
gridItem.data.configuration.zoom.value = zoom
}
@@ -312,7 +300,7 @@
style="padding-top: {controlsHeight ?? 0}px; {css?.container?.style ?? ''}"
/>
{/if}
{#if $mode !== 'preview' && $selectedComponent?.includes(id)}
{#if $mode !== 'preview' && $selectedComponent === id}
<button
class="fixed z-10 bottom-0 left-0 px-2 py-0.5 bg-indigo-500/90
hover:bg-indigo-500 focus:bg-indigo-500 duration-200 text-white text-2xs"

Some files were not shown because too many files have changed in this diff Show More