Compare commits
50 Commits
v1.86.0
...
add-dev-do
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
5131ee8b89 | ||
|
|
a5f6d73f7d | ||
|
|
670c84b901 | ||
|
|
92a293488e | ||
|
|
0cd1e65e46 | ||
|
|
6aa1008933 | ||
|
|
9434bbb18b | ||
|
|
e6632a32c9 | ||
|
|
54c4d03173 | ||
|
|
3932e5dfb9 | ||
|
|
9b3d6a3dd9 | ||
|
|
58d4b556eb | ||
|
|
8552c92837 | ||
|
|
955a213a50 | ||
|
|
e82f5388b6 | ||
|
|
8a44f8e779 | ||
|
|
d45e6c94ab | ||
|
|
60da67a725 | ||
|
|
0718931616 | ||
|
|
41831d58ed | ||
|
|
36816877b4 | ||
|
|
0040e15805 | ||
|
|
72317e9b54 | ||
|
|
4cdad376b3 | ||
|
|
866228b663 | ||
|
|
7eacca4caa | ||
|
|
1526537f56 | ||
|
|
1f705cab2c | ||
|
|
f2d3c8208b | ||
|
|
1b04537c9a | ||
|
|
23e374b10d | ||
|
|
5fc72ea2e6 | ||
|
|
3f5df1ee41 | ||
|
|
3dabac153f | ||
|
|
d0e0e1fdf2 | ||
|
|
8e9c491650 | ||
|
|
df4c6289ac | ||
|
|
72c8d3921d | ||
|
|
e911869d99 | ||
|
|
e1712e63a6 | ||
|
|
2031e1ebd0 | ||
|
|
de8dc1e9cd | ||
|
|
de87d7ac27 | ||
|
|
2b003c684f | ||
|
|
3097510550 | ||
|
|
0c0b2d88cc | ||
|
|
1ffed41cf9 | ||
|
|
dac61d1c98 | ||
|
|
facb67093c | ||
|
|
341a9662b7 |
33
CHANGELOG.md
33
CHANGELOG.md
@@ -1,6 +1,39 @@
|
||||
# Changelog
|
||||
|
||||
|
||||
## [1.87.0](https://github.com/windmill-labs/windmill/compare/v1.86.0...v2.0.0) (2023-04-11)
|
||||
|
||||
|
||||
### ⚠ BREAKING CHANGES
|
||||
|
||||
* **frontend:** Add option to return file names ([#1380](https://github.com/windmill-labs/windmill/issues/1380))
|
||||
|
||||
### Features
|
||||
|
||||
* **backend:** add instance events webhook ([f2d3c82](https://github.com/windmill-labs/windmill/commit/f2d3c8208b6daa49f304f355752145de47138a3c))
|
||||
* **backend:** extend cached resolution for go ([dac61d1](https://github.com/windmill-labs/windmill/commit/dac61d1c982576d7589e16ab01c8cc8bad6e1686))
|
||||
* **backend:** Redis based queue ([#1324](https://github.com/windmill-labs/windmill/issues/1324)) ([d45e6c9](https://github.com/windmill-labs/windmill/commit/d45e6c94abed609357b18d4daa7de6b2ea0ba978))
|
||||
* **frontend:** Add option to return file names ([#1380](https://github.com/windmill-labs/windmill/issues/1380)) ([3dabac1](https://github.com/windmill-labs/windmill/commit/3dabac153f302f48210d15ebaec514e72717300f))
|
||||
* **python:** cache dependency resolution ([facb670](https://github.com/windmill-labs/windmill/commit/facb67093ce7d3b0874d0d559fb272ed822ce360))
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* **backend:** nested deno relative imports ([955a213](https://github.com/windmill-labs/windmill/commit/955a213a504c1f3b8811c930823e87fe7dba101a))
|
||||
* **cli:** overwrite archived scripts ([1f705ca](https://github.com/windmill-labs/windmill/commit/1f705cab2ce8c79829f22fc6af9e06ecba7450b1))
|
||||
* **frontend:** Add missing stopPropagation ([#1394](https://github.com/windmill-labs/windmill/issues/1394)) ([58d4b55](https://github.com/windmill-labs/windmill/commit/58d4b556ebbd76c6f07f1a16d601a9d824b99f7e))
|
||||
* **frontend:** fix app init issue ([d0e0e1f](https://github.com/windmill-labs/windmill/commit/d0e0e1fdf27d9a7fb86c66e43398786b64d8b6b7))
|
||||
* **frontend:** Fix frontend dependencies ([#1379](https://github.com/windmill-labs/windmill/issues/1379)) ([8e9c491](https://github.com/windmill-labs/windmill/commit/8e9c49165060a4a7f831b8be075593f89d867784))
|
||||
* **frontend:** Fix icon picker input ([#1389](https://github.com/windmill-labs/windmill/issues/1389)) ([8a44f8e](https://github.com/windmill-labs/windmill/commit/8a44f8e7796f13698e2a99af9f3772f5e676604b))
|
||||
* **frontend:** Fix mac shortcuts ([#1381](https://github.com/windmill-labs/windmill/issues/1381)) ([41831d5](https://github.com/windmill-labs/windmill/commit/41831d58ed593bb283600b76170f6e76783e0eae))
|
||||
* **frontend:** fix popover configuration to avoid content shift ([#1377](https://github.com/windmill-labs/windmill/issues/1377)) ([2031e1e](https://github.com/windmill-labs/windmill/commit/2031e1ebd0dc020da104ee84a0294c86babcefaf))
|
||||
* **frontend:** remove stopPropagation that was preventing components dnd ([#1378](https://github.com/windmill-labs/windmill/issues/1378)) ([de8dc1e](https://github.com/windmill-labs/windmill/commit/de8dc1e9cd7beea2ce62656e9e7676214f77a110))
|
||||
|
||||
|
||||
### Performance Improvements
|
||||
|
||||
* parallelize more operations for deno jobs ([e911869](https://github.com/windmill-labs/windmill/commit/e911869d990956463834ac9ff35c52ba8236e362))
|
||||
|
||||
## [1.86.0](https://github.com/windmill-labs/windmill/compare/v1.85.0...v1.86.0) (2023-04-08)
|
||||
|
||||
|
||||
|
||||
@@ -85,7 +85,7 @@ COPY .git/ .git/
|
||||
RUN CARGO_NET_GIT_FETCH_WITH_CLI=true cargo build --release --features "$features"
|
||||
|
||||
|
||||
FROM python:3.11.2-slim-buster
|
||||
FROM python:3.11.3-slim-buster
|
||||
ARG TARGETPLATFORM
|
||||
|
||||
ARG APP=/usr/src/app
|
||||
|
||||
10
README.md
10
README.md
@@ -70,10 +70,10 @@ https://user-images.githubusercontent.com/275584/218350457-bc2fdc3b-e667-4da5-a2
|
||||
|
||||
3. Make it flow! You can chain your scripts or scripts made by the community
|
||||
shared on [WindmillHub](https://hub.windmill.dev).
|
||||

|
||||

|
||||
|
||||
4. Build complex UI on top of your scripts and flows.
|
||||

|
||||

|
||||
|
||||
Scripts and flows can also be triggered by a cron schedule '_/5 _ \* \* \*' or
|
||||
through webhooks.
|
||||
@@ -123,7 +123,7 @@ instance from local commands. See
|
||||
### Running scripts locally
|
||||
|
||||
You can run your script locally easily, you simply need to pass the right
|
||||
environment variables for the `wmill` client library to fetch resource and
|
||||
environment variables for the `wmill` client library to fetch resources and
|
||||
variables from your instance if necessary. See more:
|
||||
<https://docs.windmill.dev/docs/advanced/local_development/>
|
||||
|
||||
@@ -315,6 +315,8 @@ it being synced automatically everyday.
|
||||
| DENO_PATH | /usr/bin/deno | The path to the deno binary. | Worker |
|
||||
| PYTHON_PATH | /usr/local/bin/python3 | The path to the python binary. | Worker |
|
||||
| GO_PATH | /usr/bin/go | The path to the go binary. | Worker |
|
||||
| GOPRIVATE | | The GOPRIVATE env variable to use private go modules | Worker |
|
||||
| NETRC | | The netrc content to use a private go registry | Worker |
|
||||
| PIP_INDEX_URL | None | The index url to pass for pip. | Worker |
|
||||
| PIP_EXTRA_INDEX_URL | None | The extra index url to pass to pip. | Worker |
|
||||
| PIP_TRUSTED_HOST | None | The trusted host to pass to pip. | Worker |
|
||||
@@ -332,7 +334,7 @@ it being synced automatically everyday.
|
||||
| INCLUDE_HEADERS | None | Whitelist of headers that are passed to jobs as args (separated by a comma) | Server |
|
||||
| WHITELIST_WORKSPACES | None | Whitelist of workspaces this worker takes job from | Worker |
|
||||
| BLACKLIST_WORKSPACES | None | Blacklist of workspaces this worker takes job from | Worker |
|
||||
| NEW_USER_WEBHOOK | None | Webhook to notify of a new user added, signup/invite. Can hook back to windmill to send emails | Server |
|
||||
| INSTANCE_EVENTS_WEBHOOK | None | Webhook to notify of events such as new user added, signup/invite. Can hook back to windmill to send emails | Server |
|
||||
|
||||
## Run a local dev setup
|
||||
|
||||
|
||||
@@ -18,4 +18,4 @@ rustflags = [
|
||||
rustflags = [
|
||||
"-C", "link-arg=-undefined",
|
||||
"-C", "link-arg=dynamic_lookup",
|
||||
]
|
||||
]
|
||||
400
backend/Cargo.lock
generated
400
backend/Cargo.lock
generated
@@ -178,6 +178,55 @@ dependencies = [
|
||||
"zstd-safe",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "async-executor"
|
||||
version = "1.5.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "6fa3dc5f2a8564f07759c008b9109dc0d39de92a88d5588b8a5036d286383afb"
|
||||
dependencies = [
|
||||
"async-lock",
|
||||
"async-task",
|
||||
"concurrent-queue",
|
||||
"fastrand",
|
||||
"futures-lite",
|
||||
"slab",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "async-global-executor"
|
||||
version = "2.3.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f1b6f5d7df27bd294849f8eec66ecfc63d11814df7a4f5d74168a2394467b776"
|
||||
dependencies = [
|
||||
"async-channel",
|
||||
"async-executor",
|
||||
"async-io",
|
||||
"async-lock",
|
||||
"blocking",
|
||||
"futures-lite",
|
||||
"once_cell",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "async-io"
|
||||
version = "1.13.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "0fc5b45d93ef0529756f812ca52e44c221b35341892d3dcc34132ac02f3dd2af"
|
||||
dependencies = [
|
||||
"async-lock",
|
||||
"autocfg",
|
||||
"cfg-if",
|
||||
"concurrent-queue",
|
||||
"futures-lite",
|
||||
"log",
|
||||
"parking",
|
||||
"polling",
|
||||
"rustix",
|
||||
"slab",
|
||||
"socket2",
|
||||
"waker-fn",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "async-lock"
|
||||
version = "2.7.0"
|
||||
@@ -214,7 +263,33 @@ checksum = "0e97ce7de6cf12de5d7226c73f5ba9811622f4db3a5b91b55c53e987e5f91cba"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 2.0.13",
|
||||
"syn 2.0.14",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "async-std"
|
||||
version = "1.12.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "62565bb4402e926b29953c785397c6dc0391b7b446e45008b0049eb43cec6f5d"
|
||||
dependencies = [
|
||||
"async-channel",
|
||||
"async-global-executor",
|
||||
"async-io",
|
||||
"async-lock",
|
||||
"crossbeam-utils",
|
||||
"futures-channel",
|
||||
"futures-core",
|
||||
"futures-io",
|
||||
"futures-lite",
|
||||
"gloo-timers",
|
||||
"kv-log-macro",
|
||||
"log",
|
||||
"memchr",
|
||||
"once_cell",
|
||||
"pin-project-lite",
|
||||
"pin-utils",
|
||||
"slab",
|
||||
"wasm-bindgen-futures",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -240,6 +315,12 @@ dependencies = [
|
||||
"uuid 0.8.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "async-task"
|
||||
version = "4.4.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ecc7ab41815b3c653ccd2978ec3255c81349336702dfdf62ee6f7069b12a3aae"
|
||||
|
||||
[[package]]
|
||||
name = "async-timer"
|
||||
version = "0.7.4"
|
||||
@@ -259,7 +340,7 @@ checksum = "b9ccdd8f2a161be9bd5c023df56f1b2a0bd1d83872ae53b71a84a12c9bf6e842"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 2.0.13",
|
||||
"syn 2.0.14",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -286,6 +367,12 @@ dependencies = [
|
||||
"num-traits",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "atomic-waker"
|
||||
version = "1.1.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "1181e1e0d1fce796a03db1ae795d67167da795f9cf4a39c37589e85ef57f26d3"
|
||||
|
||||
[[package]]
|
||||
name = "autocfg"
|
||||
version = "1.1.0"
|
||||
@@ -294,9 +381,9 @@ checksum = "d468802bab17cbc0cc575e9b053f41e72aa36bfa6b7f55e3529ffa43161b97fa"
|
||||
|
||||
[[package]]
|
||||
name = "axum"
|
||||
version = "0.6.12"
|
||||
version = "0.6.14"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "349f8ccfd9221ee7d1f3d4b33e1f8319b3a81ed8f61f2ea40b37b859794b4491"
|
||||
checksum = "6e5abc76e6ef57bf438c067d2898481475994102eeb5d662a0b2162d0c2fdcf1"
|
||||
dependencies = [
|
||||
"async-trait",
|
||||
"axum-core",
|
||||
@@ -327,9 +414,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "axum-core"
|
||||
version = "0.3.3"
|
||||
version = "0.3.4"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "b2f958c80c248b34b9a877a643811be8dbca03ca5ba827f2b63baf3a81e5fc4e"
|
||||
checksum = "759fa577a247914fd3f7f76d62972792636412fbfd634cd452f6a385a74d2d2c"
|
||||
dependencies = [
|
||||
"async-trait",
|
||||
"bytes",
|
||||
@@ -360,6 +447,19 @@ version = "1.6.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "8c3c1a368f70d6cf7302d78f8f7093da241fb8e8807c05cc9e51a125895a6d5b"
|
||||
|
||||
[[package]]
|
||||
name = "bb8"
|
||||
version = "0.7.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "2e9f4fa9768efd269499d8fba693260cfc670891cf6de3adc935588447a77cc8"
|
||||
dependencies = [
|
||||
"async-trait",
|
||||
"futures-channel",
|
||||
"futures-util",
|
||||
"parking_lot 0.11.2",
|
||||
"tokio",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "better_scoped_tls"
|
||||
version = "0.1.0"
|
||||
@@ -434,6 +534,21 @@ version = "0.2.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "8d696c370c750c948ada61c69a0ee2cbbb9c50b1019ddb86d9317157a99c2cae"
|
||||
|
||||
[[package]]
|
||||
name = "blocking"
|
||||
version = "1.3.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "77231a1c8f801696fc0123ec6150ce92cffb8e164a02afb9c8ddee0e9b65ad65"
|
||||
dependencies = [
|
||||
"async-channel",
|
||||
"async-lock",
|
||||
"async-task",
|
||||
"atomic-waker",
|
||||
"fastrand",
|
||||
"futures-lite",
|
||||
"log",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "bstr"
|
||||
version = "0.2.17"
|
||||
@@ -601,7 +716,7 @@ dependencies = [
|
||||
"heck",
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 2.0.13",
|
||||
"syn 2.0.14",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -620,6 +735,20 @@ dependencies = [
|
||||
"unicode-width",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "combine"
|
||||
version = "4.6.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "35ed6e9d84f0b51a7f52daf1c7d71dd136fd7a3f41a8462b8cdb8c78d920fad4"
|
||||
dependencies = [
|
||||
"bytes",
|
||||
"futures-core",
|
||||
"memchr",
|
||||
"pin-project-lite",
|
||||
"tokio",
|
||||
"tokio-util",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "concolor-override"
|
||||
version = "1.0.0"
|
||||
@@ -792,6 +921,16 @@ dependencies = [
|
||||
"typenum",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "ctor"
|
||||
version = "0.1.26"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "6d2301688392eb071b0bf1a37be05c469d3cc4dbbd95df672fe28ab021e6a096"
|
||||
dependencies = [
|
||||
"quote",
|
||||
"syn 1.0.109",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "cxx"
|
||||
version = "1.0.94"
|
||||
@@ -816,7 +955,7 @@ dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"scratch",
|
||||
"syn 2.0.13",
|
||||
"syn 2.0.14",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -833,7 +972,7 @@ checksum = "2345488264226bf682893e25de0769f3360aac9957980ec49361b083ddaa5bc5"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 2.0.13",
|
||||
"syn 2.0.14",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -1287,7 +1426,7 @@ checksum = "89ca545a94061b6365f2c7355b4b32bd20df3ff95f02da9329b34ccc3bd6ee72"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 2.0.13",
|
||||
"syn 2.0.14",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -1396,6 +1535,28 @@ dependencies = [
|
||||
"url",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "gloo-timers"
|
||||
version = "0.2.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "9b995a66bb87bebce9a0f4a95aed01daca4872c050bfcb21653361c03bc35e5c"
|
||||
dependencies = [
|
||||
"futures-channel",
|
||||
"futures-core",
|
||||
"js-sys",
|
||||
"wasm-bindgen",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "gosyn"
|
||||
version = "0.2.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "248a4b26b282ec2e90272f7e7bd4764eaa6378dee13497ef09aa0efebd80cc85"
|
||||
dependencies = [
|
||||
"strum",
|
||||
"unic-ucd-category",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "h2"
|
||||
version = "0.3.16"
|
||||
@@ -1784,6 +1945,15 @@ version = "0.2.19"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "a4933f3f57a8e9d9da04db23fb153356ecaf00cbd14aee46279c33dc80925c37"
|
||||
|
||||
[[package]]
|
||||
name = "kv-log-macro"
|
||||
version = "1.0.7"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "0de8b303297635ad57c9f5059fd9cee7a47f8e8daa09df0fcd07dd39fb22977f"
|
||||
dependencies = [
|
||||
"log",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "lalrpop"
|
||||
version = "0.19.9"
|
||||
@@ -1966,6 +2136,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "abb12e687cfb44aa40f41fc3978ef76448f9b6038cad6aef4259d3c095a2382e"
|
||||
dependencies = [
|
||||
"cfg-if",
|
||||
"value-bag",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -2014,6 +2185,12 @@ dependencies = [
|
||||
"regex-automata",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "matches"
|
||||
version = "0.1.10"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "2532096657941c2fea9c289d370a250971c689d4f143798ff67113ec042024a5"
|
||||
|
||||
[[package]]
|
||||
name = "matchit"
|
||||
version = "0.7.0"
|
||||
@@ -2238,9 +2415,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "openssl"
|
||||
version = "0.10.49"
|
||||
version = "0.10.50"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "4d2f106ab837a24e03672c59b1239669a0596406ff657c3c0835b6b7f0f35a33"
|
||||
checksum = "7e30d8bc91859781f0a943411186324d580f2bbeb71b452fe91ae344806af3f1"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"cfg-if",
|
||||
@@ -2259,7 +2436,7 @@ checksum = "a948666b637a0f465e8564c73e89d4dde00d72d4d473cc972f390fc3dcee7d9c"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 2.0.13",
|
||||
"syn 2.0.14",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -2270,9 +2447,9 @@ checksum = "ff011a302c396a5197692431fc1948019154afc178baf7d8e37367442a4601cf"
|
||||
|
||||
[[package]]
|
||||
name = "openssl-sys"
|
||||
version = "0.9.84"
|
||||
version = "0.9.85"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3a20eace9dc2d82904039cb76dcf50fb1a0bba071cfd1629720b5d6f1ddba0fa"
|
||||
checksum = "0d3d193fb1488ad46ffe3aaabc912cc931d02ee8518fe2959aea8ef52718b0c0"
|
||||
dependencies = [
|
||||
"cc",
|
||||
"libc",
|
||||
@@ -2549,6 +2726,22 @@ dependencies = [
|
||||
"syn 1.0.109",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "polling"
|
||||
version = "2.7.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "4be1c66a6add46bff50935c313dae30a5030cf8385c5206e8a95e9e9def974aa"
|
||||
dependencies = [
|
||||
"autocfg",
|
||||
"bitflags",
|
||||
"cfg-if",
|
||||
"concurrent-queue",
|
||||
"libc",
|
||||
"log",
|
||||
"pin-project-lite",
|
||||
"windows-sys 0.48.0",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "ppv-lite86"
|
||||
version = "0.2.17"
|
||||
@@ -2634,7 +2827,7 @@ dependencies = [
|
||||
"schemars",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"syn 2.0.13",
|
||||
"syn 2.0.14",
|
||||
"thiserror",
|
||||
"typify",
|
||||
"unicode-ident",
|
||||
@@ -2654,7 +2847,7 @@ dependencies = [
|
||||
"serde_json",
|
||||
"serde_tokenstream",
|
||||
"serde_yaml",
|
||||
"syn 2.0.13",
|
||||
"syn 2.0.14",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -2695,6 +2888,12 @@ dependencies = [
|
||||
"proc-macro2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "radix_fmt"
|
||||
version = "1.0.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ce082a9940a7ace2ad4a8b7d0b1eac6aa378895f18be598230c5f2284ac05426"
|
||||
|
||||
[[package]]
|
||||
name = "rand"
|
||||
version = "0.7.3"
|
||||
@@ -2766,6 +2965,27 @@ dependencies = [
|
||||
"rand_core 0.5.1",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "redis"
|
||||
version = "0.23.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3ea8c51b5dc1d8e5fd3350ec8167f464ec0995e79f2e90a075b63371500d557f"
|
||||
dependencies = [
|
||||
"async-std",
|
||||
"async-trait",
|
||||
"bytes",
|
||||
"combine",
|
||||
"futures-util",
|
||||
"itoa",
|
||||
"percent-encoding",
|
||||
"pin-project-lite",
|
||||
"ryu",
|
||||
"sha1_smol",
|
||||
"tokio",
|
||||
"tokio-util",
|
||||
"url",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "redox_syscall"
|
||||
version = "0.2.16"
|
||||
@@ -2918,6 +3138,21 @@ dependencies = [
|
||||
"zeroize",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "rsmq_async"
|
||||
version = "5.1.5"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "086675eeb88ca55fa3b25c0075615d31dd9eb0adee219a9dcd999090cb371cab"
|
||||
dependencies = [
|
||||
"async-trait",
|
||||
"bb8",
|
||||
"lazy_static",
|
||||
"radix_fmt",
|
||||
"rand 0.8.5",
|
||||
"redis",
|
||||
"thiserror",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "rust-embed"
|
||||
version = "6.6.1"
|
||||
@@ -3027,7 +3262,7 @@ dependencies = [
|
||||
[[package]]
|
||||
name = "rustpython-ast"
|
||||
version = "0.2.0"
|
||||
source = "git+https://github.com/RustPython/RustPython#dde92d1269b5fbd09cf5dba5e7817f0e2a96c49f"
|
||||
source = "git+https://github.com/RustPython/RustPython#c9546c2419f2e4db7e2987f30f0508dc29b30035"
|
||||
dependencies = [
|
||||
"num-bigint",
|
||||
"rustpython-compiler-core",
|
||||
@@ -3036,7 +3271,7 @@ dependencies = [
|
||||
[[package]]
|
||||
name = "rustpython-compiler-core"
|
||||
version = "0.2.0"
|
||||
source = "git+https://github.com/RustPython/RustPython#dde92d1269b5fbd09cf5dba5e7817f0e2a96c49f"
|
||||
source = "git+https://github.com/RustPython/RustPython#c9546c2419f2e4db7e2987f30f0508dc29b30035"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"bstr",
|
||||
@@ -3049,7 +3284,7 @@ dependencies = [
|
||||
[[package]]
|
||||
name = "rustpython-parser"
|
||||
version = "0.2.0"
|
||||
source = "git+https://github.com/RustPython/RustPython#dde92d1269b5fbd09cf5dba5e7817f0e2a96c49f"
|
||||
source = "git+https://github.com/RustPython/RustPython#c9546c2419f2e4db7e2987f30f0508dc29b30035"
|
||||
dependencies = [
|
||||
"ahash 0.7.6",
|
||||
"anyhow",
|
||||
@@ -3111,7 +3346,7 @@ dependencies = [
|
||||
"schemars_derive",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"uuid 1.3.0",
|
||||
"uuid 1.3.1",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -3221,9 +3456,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "serde"
|
||||
version = "1.0.159"
|
||||
version = "1.0.160"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3c04e8343c3daeec41f58990b9d77068df31209f2af111e059e9fe9646693065"
|
||||
checksum = "bb2f3770c8bce3bcda7e149193a069a0f4365bda1fa5cd88e03bca26afc1216c"
|
||||
dependencies = [
|
||||
"serde_derive",
|
||||
]
|
||||
@@ -3250,13 +3485,13 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "serde_derive"
|
||||
version = "1.0.159"
|
||||
version = "1.0.160"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "4c614d17805b093df4b147b51339e7e44bf05ef59fba1e45d83500bcfb4d8585"
|
||||
checksum = "291a097c63d8497e00160b166a967a4a79c64f3facdd01cbd7502231688d77df"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 2.0.13",
|
||||
"syn 2.0.14",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -3322,7 +3557,7 @@ dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"serde",
|
||||
"syn 2.0.13",
|
||||
"syn 2.0.14",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -3376,6 +3611,12 @@ dependencies = [
|
||||
"digest 0.10.6",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "sha1_smol"
|
||||
version = "1.0.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ae1a47186c03a32177042e55dbc5fd5aee900b8e0069a8d70fba96a9375cd012"
|
||||
|
||||
[[package]]
|
||||
name = "sha2"
|
||||
version = "0.9.9"
|
||||
@@ -3602,7 +3843,7 @@ dependencies = [
|
||||
"thiserror",
|
||||
"tokio-stream",
|
||||
"url",
|
||||
"uuid 1.3.0",
|
||||
"uuid 1.3.1",
|
||||
"webpki-roots",
|
||||
"whoami",
|
||||
]
|
||||
@@ -3720,6 +3961,28 @@ version = "0.10.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "73473c0e59e6d5812c5dfe2a064a6444949f089e20eec9a2e5506596494e4623"
|
||||
|
||||
[[package]]
|
||||
name = "strum"
|
||||
version = "0.24.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "063e6045c0e62079840579a7e47a355ae92f60eb74daaf156fb1e84ba164e63f"
|
||||
dependencies = [
|
||||
"strum_macros",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "strum_macros"
|
||||
version = "0.24.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "1e385be0d24f186b4ce2f9982191e7101bb737312ad61c1f2f984f34bcf85d59"
|
||||
dependencies = [
|
||||
"heck",
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"rustversion",
|
||||
"syn 1.0.109",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "subtle"
|
||||
version = "2.4.1"
|
||||
@@ -3866,9 +4129,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "syn"
|
||||
version = "2.0.13"
|
||||
version = "2.0.14"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "4c9da457c5285ac1f936ebd076af6dac17a61cfe7826f2076b4d015cf47bc8ec"
|
||||
checksum = "fcf316d5356ed6847742d036f8a39c3b8435cac10bd528a4bd461928a6ab34d5"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
@@ -3931,7 +4194,7 @@ checksum = "f9456a42c5b0d803c8cd86e73dd7cc9edd429499f37a3550d286d5e86720569f"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 2.0.13",
|
||||
"syn 2.0.14",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -4045,7 +4308,7 @@ checksum = "61a573bdc87985e9d6ddeed1b3d864e8a302c847e40d647746df2f1de209d1ce"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 2.0.13",
|
||||
"syn 2.0.14",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -4358,7 +4621,7 @@ dependencies = [
|
||||
"regress",
|
||||
"schemars",
|
||||
"serde_json",
|
||||
"syn 2.0.13",
|
||||
"syn 2.0.14",
|
||||
"thiserror",
|
||||
"unicode-ident",
|
||||
]
|
||||
@@ -4374,7 +4637,7 @@ dependencies = [
|
||||
"serde",
|
||||
"serde_json",
|
||||
"serde_tokenstream",
|
||||
"syn 2.0.13",
|
||||
"syn 2.0.14",
|
||||
"typify-impl",
|
||||
]
|
||||
|
||||
@@ -4391,7 +4654,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "13a3aaa69b04e5b66cc27309710a569ea23593612387d67daaf102e73aa974fd"
|
||||
dependencies = [
|
||||
"rand 0.8.5",
|
||||
"uuid 1.3.0",
|
||||
"uuid 1.3.1",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -4426,6 +4689,18 @@ dependencies = [
|
||||
"unic-ucd-version",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "unic-ucd-category"
|
||||
version = "0.9.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "1b8d4591f5fcfe1bd4453baaf803c40e1b1e69ff8455c47620440b46efef91c0"
|
||||
dependencies = [
|
||||
"matches",
|
||||
"unic-char-property",
|
||||
"unic-char-range",
|
||||
"unic-ucd-version",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "unic-ucd-ident"
|
||||
version = "0.9.0"
|
||||
@@ -4567,9 +4842,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "uuid"
|
||||
version = "1.3.0"
|
||||
version = "1.3.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "1674845326ee10d37ca60470760d4288a6f80f304007d92e5c53bab78c9cfd79"
|
||||
checksum = "5b55a3fef2a1e3b3a00ce878640918820d3c51081576ac657d23af9fc7928fdb"
|
||||
dependencies = [
|
||||
"getrandom 0.2.9",
|
||||
"serde",
|
||||
@@ -4593,6 +4868,16 @@ version = "0.1.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "830b7e5d4d90034032940e4ace0d9a9a057e7a45cd94e6c007832e39edb82f6d"
|
||||
|
||||
[[package]]
|
||||
name = "value-bag"
|
||||
version = "1.0.0-alpha.9"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "2209b78d1249f7e6f3293657c9779fe31ced465df091bbd433a1cf88e916ec55"
|
||||
dependencies = [
|
||||
"ctor",
|
||||
"version_check",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "vcpkg"
|
||||
version = "0.2.15"
|
||||
@@ -4811,7 +5096,7 @@ checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
|
||||
|
||||
[[package]]
|
||||
name = "windmill"
|
||||
version = "1.86.0"
|
||||
version = "1.87.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"axum",
|
||||
@@ -4823,12 +5108,15 @@ dependencies = [
|
||||
"rand 0.8.5",
|
||||
"reqwest",
|
||||
"rsa",
|
||||
"rsmq_async",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"sha2 0.10.6",
|
||||
"sqlx",
|
||||
"tokio",
|
||||
"tokio-metrics",
|
||||
"tracing",
|
||||
"url",
|
||||
"windmill-api",
|
||||
"windmill-api-client",
|
||||
"windmill-common",
|
||||
@@ -4838,7 +5126,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-api"
|
||||
version = "1.86.0"
|
||||
version = "1.87.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"argon2",
|
||||
@@ -4865,6 +5153,7 @@ dependencies = [
|
||||
"regex",
|
||||
"reqwest",
|
||||
"retainer",
|
||||
"rsmq_async",
|
||||
"rust-embed",
|
||||
"serde",
|
||||
"serde_json",
|
||||
@@ -4895,7 +5184,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-api-client"
|
||||
version = "1.86.0"
|
||||
version = "1.87.0"
|
||||
dependencies = [
|
||||
"base64 0.21.0",
|
||||
"chrono",
|
||||
@@ -4905,12 +5194,12 @@ dependencies = [
|
||||
"reqwest",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"uuid 1.3.0",
|
||||
"uuid 1.3.1",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windmill-audit"
|
||||
version = "1.86.0"
|
||||
version = "1.87.0"
|
||||
dependencies = [
|
||||
"chrono",
|
||||
"serde",
|
||||
@@ -4923,7 +5212,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-common"
|
||||
version = "1.86.0"
|
||||
version = "1.87.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"axum",
|
||||
@@ -4943,12 +5232,12 @@ dependencies = [
|
||||
"tokio",
|
||||
"tracing",
|
||||
"tracing-subscriber",
|
||||
"uuid 1.3.0",
|
||||
"uuid 1.3.1",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser"
|
||||
version = "1.86.0"
|
||||
version = "1.87.0"
|
||||
dependencies = [
|
||||
"serde",
|
||||
"serde_json",
|
||||
@@ -4956,7 +5245,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-bash"
|
||||
version = "1.86.0"
|
||||
version = "1.87.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"itertools",
|
||||
@@ -4971,9 +5260,10 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-go"
|
||||
version = "1.86.0"
|
||||
version = "1.87.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"gosyn",
|
||||
"itertools",
|
||||
"phf",
|
||||
"unicode-general-category",
|
||||
@@ -4983,7 +5273,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-py"
|
||||
version = "1.86.0"
|
||||
version = "1.87.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"itertools",
|
||||
@@ -4998,7 +5288,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-parser-ts"
|
||||
version = "1.86.0"
|
||||
version = "1.87.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"deno_core",
|
||||
@@ -5012,31 +5302,34 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "windmill-queue"
|
||||
version = "1.86.0"
|
||||
version = "1.87.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"chrono",
|
||||
"chrono-tz",
|
||||
"cron",
|
||||
"futures-core",
|
||||
"hex",
|
||||
"hmac",
|
||||
"lazy_static",
|
||||
"prometheus",
|
||||
"reqwest",
|
||||
"rsmq_async",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"sql-builder",
|
||||
"sqlx",
|
||||
"tokio",
|
||||
"tracing",
|
||||
"ulid",
|
||||
"uuid 1.3.0",
|
||||
"uuid 1.3.1",
|
||||
"windmill-audit",
|
||||
"windmill-common",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windmill-worker"
|
||||
version = "1.86.0"
|
||||
version = "1.87.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-recursion",
|
||||
@@ -5053,12 +5346,13 @@ dependencies = [
|
||||
"prometheus",
|
||||
"rand 0.8.5",
|
||||
"regex",
|
||||
"rsmq_async",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"sqlx",
|
||||
"tokio",
|
||||
"tracing",
|
||||
"uuid 1.3.0",
|
||||
"uuid 1.3.1",
|
||||
"windmill-api-client",
|
||||
"windmill-audit",
|
||||
"windmill-common",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "windmill"
|
||||
version = "1.86.0"
|
||||
version = "1.87.0"
|
||||
authors.workspace = true
|
||||
edition.workspace = true
|
||||
|
||||
@@ -19,7 +19,7 @@ members = [
|
||||
]
|
||||
|
||||
[workspace.package]
|
||||
version = "1.86.0"
|
||||
version = "1.87.0"
|
||||
authors = ["Ruben Fiszel <ruben@windmill.dev>"]
|
||||
edition = "2021"
|
||||
|
||||
@@ -28,11 +28,7 @@ name = "windmill"
|
||||
path = "./src/main.rs"
|
||||
|
||||
[features]
|
||||
enterprise = [
|
||||
"windmill-worker/enterprise",
|
||||
"windmill-queue/enterprise",
|
||||
"windmill-api/enterprise",
|
||||
]
|
||||
enterprise = ["windmill-worker/enterprise", "windmill-queue/enterprise", "windmill-api/enterprise"]
|
||||
|
||||
[dependencies]
|
||||
anyhow.workspace = true
|
||||
@@ -52,12 +48,16 @@ git-version.workspace = true
|
||||
rsa.workspace = true
|
||||
base64.workspace = true
|
||||
sha2.workspace = true
|
||||
rsmq_async.workspace = true
|
||||
url.workspace = true
|
||||
|
||||
|
||||
[dev-dependencies]
|
||||
serde_json.workspace = true
|
||||
reqwest.workspace = true
|
||||
windmill-queue.workspace = true
|
||||
axum.workspace = true
|
||||
serde.workspace = true
|
||||
|
||||
[workspace.dependencies]
|
||||
windmill-api = { path = "./windmill-api" }
|
||||
@@ -143,6 +143,7 @@ sqlx = { version = "^0", features = [
|
||||
dotenv = "^0"
|
||||
ulid = { version = "^1", features = ["uuid"] }
|
||||
futures = "^0"
|
||||
futures-core = "^0"
|
||||
tokio-metrics = "0.1.0"
|
||||
lazy_static = "1.4.0"
|
||||
serde_derive = "1.0.147"
|
||||
@@ -154,4 +155,8 @@ async-stripe = { version = "0.14", features = [
|
||||
"checkout",
|
||||
] }
|
||||
async_zip = { version = "0.0.11", features = ["full"] }
|
||||
once_cell = "1.17.1"
|
||||
once_cell = "1.17.1"
|
||||
rsmq_async = { version = "5.1.5" }
|
||||
gosyn = "0.2.2"
|
||||
|
||||
[patch.crates-io]
|
||||
|
||||
@@ -0,0 +1,2 @@
|
||||
-- Add down migration script here
|
||||
DROP TABLE pip_resolution_cache;
|
||||
@@ -0,0 +1,6 @@
|
||||
-- Add up migration script here
|
||||
CREATE TABLE pip_resolution_cache(
|
||||
hash VARCHAR(255) PRIMARY KEY,
|
||||
expiration TIMESTAMP NOT NULL,
|
||||
lockfile TEXT NOT NULL
|
||||
);
|
||||
2
backend/migrations/20230410122342_saved_inputs.down.sql
Normal file
2
backend/migrations/20230410122342_saved_inputs.down.sql
Normal file
@@ -0,0 +1,2 @@
|
||||
DROP TABLE IF EXISTS input;
|
||||
DROP TYPE RUNNABLE_TYPE;
|
||||
13
backend/migrations/20230410122342_saved_inputs.up.sql
Normal file
13
backend/migrations/20230410122342_saved_inputs.up.sql
Normal file
@@ -0,0 +1,13 @@
|
||||
CREATE TYPE RUNNABLE_TYPE AS ENUM ('ScriptHash', 'ScriptPath', 'FlowPath');
|
||||
|
||||
CREATE TABLE IF NOT EXISTS input (
|
||||
id UUID PRIMARY KEY,
|
||||
workspace_id VARCHAR(50) NOT NULL REFERENCES workspace(id),
|
||||
runnable_id VARCHAR(255) NOT NULL,
|
||||
runnable_type RUNNABLE_TYPE NOT NULL,
|
||||
name TEXT NOT NULL,
|
||||
args JSONB NOT NULL,
|
||||
created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(),
|
||||
created_by VARCHAR(50) NOT NULL,
|
||||
is_public BOOLEAN NOT NULL DEFAULT FALSE
|
||||
);
|
||||
@@ -15,3 +15,4 @@ phf.workspace = true
|
||||
unicode-general-category.workspace = true
|
||||
itertools.workspace = true
|
||||
anyhow.workspace = true
|
||||
gosyn.workspace = true
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,347 +0,0 @@
|
||||
#![allow(clippy::large_enum_variant)] // TODO: we allow large enum variant for now, let's profile properly to see if we want to box.
|
||||
|
||||
use crate::parser_go_token::{Position, Token};
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
// https://pkg.go.dev/go/ast#CommentGroup
|
||||
#[derive(Debug)]
|
||||
pub struct CommentGroup {
|
||||
// List []*Comment // len(List) > 0
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#FieldList
|
||||
#[derive(Debug)]
|
||||
pub struct FieldList<'a> {
|
||||
pub opening: Option<Position<'a>>, // position of opening parenthesis/brace, if any
|
||||
pub list: Vec<Field<'a>>, // field list; or nil
|
||||
pub closing: Option<Position<'a>>, // position of closing parenthesis/brace, if any
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#Field
|
||||
#[derive(Debug)]
|
||||
pub struct Field<'a> {
|
||||
pub doc: Option<CommentGroup>, // associated documentation; or nil
|
||||
pub names: Option<Vec<Ident<'a>>>, // field/method/(type) parameter names, or type "type"; or nil
|
||||
pub type_: Option<Expr<'a>>, // field/method/parameter type, type list type; or nil
|
||||
pub tag: Option<BasicLit<'a>>, // field tag; or nil
|
||||
pub comment: Option<CommentGroup>, // line comments; or nil
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#File
|
||||
#[derive(Debug)]
|
||||
pub struct File<'a> {
|
||||
// package name
|
||||
pub decls: Vec<Decl<'a>>, // top-level declarations; or nil // list of all comments in the source file
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#FuncDecl
|
||||
#[derive(Debug)]
|
||||
pub struct FuncDecl<'a> {
|
||||
pub doc: Option<CommentGroup>, // associated documentation; or nil
|
||||
pub recv: Option<FieldList<'a>>, // receiver (methods); or nil (functions)
|
||||
pub name: Ident<'a>, // function/method name
|
||||
pub type_: FuncType<'a>, // function signature: type and value parameters, results, and position of "func" keyword
|
||||
pub body: Option<BlockStmt<'a>>, // function body; or nil for external (non-Go) function
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#BlockStmt
|
||||
#[derive(Debug)]
|
||||
pub struct BlockStmt<'a> {
|
||||
pub lbrace: Position<'a>, // position of "{"
|
||||
pub list: Vec<Stmt>,
|
||||
pub rbrace: Position<'a>, // position of "}", if any (may be absent due to syntax error)
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#FuncType
|
||||
#[derive(Debug)]
|
||||
pub struct FuncType<'a> {
|
||||
pub func: Option<Position<'a>>, // position of "func" keyword (token.NoPos if there is no "func")
|
||||
pub params: FieldList<'a>, // (incoming) parameters; non-nil
|
||||
pub results: Option<FieldList<'a>>, // (outgoing) results; or nil
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#Ident
|
||||
#[derive(Debug)]
|
||||
pub struct Ident<'a> {
|
||||
pub name_pos: Position<'a>, // identifier position
|
||||
pub name: &'a str, // identifier name
|
||||
pub obj: Option<Box<Object<'a>>>, // denoted object; or nil
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#ValueSpec
|
||||
#[derive(Debug)]
|
||||
pub struct ValueSpec<'a> {
|
||||
pub doc: Option<CommentGroup>, // associated documentation; or nil
|
||||
pub names: Vec<Ident<'a>>, // value names (len(Names) > 0)
|
||||
pub type_: Option<Expr<'a>>, // value type; or nil
|
||||
pub values: Option<Vec<Expr<'a>>>, // initial values; or nil
|
||||
pub comment: Option<CommentGroup>, // line comments; or nil
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#BasicLit
|
||||
#[derive(Debug)]
|
||||
pub struct BasicLit<'a> {
|
||||
pub value_pos: Position<'a>, // literal position
|
||||
pub kind: Token, // token.INT, token.FLOAT, token.IMAG, token.CHAR, or token.STRING
|
||||
pub value: &'a str, // literal string; e.g. 42, 0x7f, 3.14, 1e-9, 2.4i, 'a', '\x7f', "foo" or `\m\n\o`
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#Object
|
||||
#[derive(Debug)]
|
||||
pub struct Object<'a> {
|
||||
pub kind: ObjKind,
|
||||
pub name: &'a str, // declared name
|
||||
pub decl: Option<ObjDecl>, // corresponding Field, XxxSpec, FuncDecl, LabeledStmt, AssignStmt, Scope; or nil
|
||||
pub data: Option<usize>, // object-specific data; or nil
|
||||
pub type_: Option<()>, // placeholder for type information; may be nil
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#Ellipsis
|
||||
#[derive(Debug)]
|
||||
pub struct Ellipsis<'a> {
|
||||
pub ellipsis: Position<'a>, // position of "..."
|
||||
pub elt: Option<Box<Expr<'a>>>, // ellipsis element type (parameter lists only); or nil
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#Ellipsis
|
||||
#[derive(Debug)]
|
||||
pub struct TypeAssertExpr<'a> {
|
||||
pub x: Box<Expr<'a>>, // expression
|
||||
pub lparen: Position<'a>, // position of "("
|
||||
pub type_: Box<Expr<'a>>, // asserted type; nil means type switch X.(type)
|
||||
pub rparen: Position<'a>, // position of ")"
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#SliceExpr
|
||||
#[derive(Debug)]
|
||||
pub struct SliceExpr<'a> {
|
||||
pub x: Box<Expr<'a>>, // expression
|
||||
pub lbrack: Position<'a>, // position of "["
|
||||
pub low: Option<Box<Expr<'a>>>, // begin of slice range; or nil
|
||||
pub high: Option<Box<Expr<'a>>>, // end of slice range; or nil
|
||||
pub max: Option<Box<Expr<'a>>>, // maximum capacity of slice; or nil
|
||||
pub slice3: bool, // true if 3-index slice (2 colons present)
|
||||
pub rbrack: Position<'a>, // position of "]"
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#ObjKind
|
||||
#[derive(Debug)]
|
||||
pub enum ObjKind {}
|
||||
|
||||
#[derive(Debug)]
|
||||
pub enum ObjDecl {}
|
||||
|
||||
// https://pkg.go.dev/go/ast#Decl
|
||||
#[derive(Debug)]
|
||||
pub enum Decl<'a> {
|
||||
FuncDecl(FuncDecl<'a>),
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#Scope
|
||||
#[derive(Debug)]
|
||||
pub struct Scope<'a> {
|
||||
pub outer: Option<Box<Scope<'a>>>,
|
||||
pub objects: BTreeMap<&'a str, Object<'a>>,
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#GenDecl
|
||||
#[derive(Debug)]
|
||||
pub struct GenDecl<'a> {
|
||||
pub doc: Option<CommentGroup>, // associated documentation; or nil
|
||||
pub tok_pos: Position<'a>, // position of Tok
|
||||
pub tok: Token, // IMPORT, CONST, TYPE, or VAR
|
||||
pub lparen: Option<Position<'a>>, // position of '(', if any
|
||||
pub specs: Vec<Spec>,
|
||||
pub rparen: Option<Position<'a>>, // position of ')', if any
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#AssignStmt
|
||||
#[derive(Debug)]
|
||||
pub struct AssignStmt<'a> {
|
||||
pub lhs: Vec<Expr<'a>>,
|
||||
pub tok_pos: Position<'a>, // position of Tok
|
||||
pub tok: Token, // assignment token, DEFINE
|
||||
pub rhs: Vec<Expr<'a>>,
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#BinaryExpr
|
||||
#[derive(Debug)]
|
||||
pub struct BinaryExpr<'a> {
|
||||
pub x: Box<Expr<'a>>, // left operand
|
||||
pub op_pos: Position<'a>, // position of Op
|
||||
pub op: Token, // operator
|
||||
pub y: Box<Expr<'a>>, // right operand
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#ReturnStmt
|
||||
#[derive(Debug)]
|
||||
pub struct ReturnStmt<'a> {
|
||||
pub return_: Position<'a>, // position of "return" keyword
|
||||
pub results: Vec<Expr<'a>>, // result expressions; or nil
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#TypeSpec
|
||||
#[derive(Debug)]
|
||||
pub struct TypeSpec<'a> {
|
||||
pub doc: Option<CommentGroup>, // associated documentation; or nil
|
||||
pub name: Option<Ident<'a>>, // type name
|
||||
pub assign: Option<Position<'a>>, // position of '=', if any
|
||||
pub type_: Expr<'a>, // *Ident, *ParenExpr, *SelectorExpr, *StarExpr, or any of the *XxxTypes
|
||||
pub comment: Option<CommentGroup>, // line comments; or nil
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#StructType
|
||||
#[derive(Debug)]
|
||||
pub struct StructType<'a> {
|
||||
pub struct_: Position<'a>, // position of "struct" keyword
|
||||
pub fields: Option<FieldList<'a>>, // list of field declarations
|
||||
pub incomplete: bool, // true if (source) fields are missing in the Fields list
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#StarExpr
|
||||
#[derive(Debug)]
|
||||
pub struct StarExpr<'a> {
|
||||
pub star: Position<'a>, // position of "*"
|
||||
pub x: Box<Expr<'a>>, // operand
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#InterfaceType
|
||||
#[derive(Debug)]
|
||||
pub struct InterfaceType<'a> {
|
||||
pub interface: Position<'a>, // position of "interface" keyword
|
||||
pub methods: Option<FieldList<'a>>, // list of embedded interfaces, methods, or types
|
||||
pub incomplete: bool, // true if (source) methods or types are missing in the Methods list
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#UnaryExpr
|
||||
#[derive(Debug)]
|
||||
pub struct UnaryExpr<'a> {
|
||||
pub op_pos: Position<'a>, // position of Op
|
||||
pub op: Token, // operator
|
||||
pub x: Box<Expr<'a>>, // operand
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#CallExpr
|
||||
#[derive(Debug)]
|
||||
pub struct CallExpr<'a> {
|
||||
pub fun: Box<Expr<'a>>, // function expression
|
||||
pub lparen: Position<'a>, // position of "("
|
||||
pub args: Option<Vec<Expr<'a>>>, // function arguments; or nil
|
||||
pub ellipsis: Option<Position<'a>>, // position of "..." (token.NoPos if there is no "...")
|
||||
pub rparen: Position<'a>, // position of ")"
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#SelectorExpr
|
||||
#[derive(Debug)]
|
||||
pub struct SelectorExpr<'a> {
|
||||
pub x: Box<Expr<'a>>, // expression
|
||||
pub sel: Ident<'a>, // field selector
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#ParenExpr
|
||||
#[derive(Debug)]
|
||||
pub struct ParenExpr<'a> {
|
||||
pub lparen: Position<'a>, // position of "("
|
||||
pub x: Box<Expr<'a>>, // parenthesized expression
|
||||
pub rparen: Position<'a>, // position of ")"
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#FuncLit
|
||||
#[derive(Debug)]
|
||||
pub struct FuncLit<'a> {
|
||||
pub type_: FuncType<'a>, // function type
|
||||
pub body: BlockStmt<'a>, // function body
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#ChanType
|
||||
#[derive(Debug)]
|
||||
pub struct ChanType<'a> {
|
||||
pub begin: Position<'a>, // position of "chan" keyword or "<-" (whichever comes first)
|
||||
pub arrow: Option<Position<'a>>, // position of "<-" (token.NoPos if there is no "<-")
|
||||
pub dir: u8, // channel direction
|
||||
pub value: Box<Expr<'a>>, // value type
|
||||
}
|
||||
|
||||
// htt/opt/visual-studio-code/resources/app/out/vs/code/electron-sandbox/workbench/workbench.htmlps://pkg.go.dev/go/ast#IndexExpr
|
||||
#[derive(Debug)]
|
||||
pub struct IndexExpr<'a> {
|
||||
pub x: Box<Expr<'a>>, // expression
|
||||
pub lbrack: Position<'a>, // position of "["
|
||||
pub index: Box<Expr<'a>>, // index expression
|
||||
pub rbrack: Position<'a>, // position of "]"
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#MapType
|
||||
#[derive(Debug)]
|
||||
pub struct MapType<'a> {
|
||||
pub map: Position<'a>,
|
||||
pub key: Box<Expr<'a>>,
|
||||
pub value: Box<Expr<'a>>,
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#CompositeLit
|
||||
#[derive(Debug)]
|
||||
pub struct CompositeLit<'a> {
|
||||
pub type_: Box<Expr<'a>>, // literal type; or nil
|
||||
pub lbrace: Position<'a>, // position of "{"
|
||||
pub elts: Option<Vec<Expr<'a>>>, // list of composite elements; or nil
|
||||
pub rbrace: Position<'a>, // position of "}"
|
||||
pub incomplete: bool, // true if (source) expressions are missing in the Elts list
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#KeyValueExpr
|
||||
#[derive(Debug)]
|
||||
pub struct KeyValueExpr<'a> {
|
||||
pub key: Box<Expr<'a>>,
|
||||
pub colon: Position<'a>, // position of ":"
|
||||
pub value: Box<Expr<'a>>,
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#ArrayType
|
||||
#[derive(Debug)]
|
||||
pub struct ArrayType<'a> {
|
||||
pub lbrack: Position<'a>, // position of "["
|
||||
pub len: Option<Box<Expr<'a>>>, // Ellipsis node for [...]T array types, nil for slice types
|
||||
pub elt: Box<Expr<'a>>, // element type
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#ChanDir
|
||||
#[derive(Debug)]
|
||||
pub enum ChanDir {
|
||||
SEND = 1 << 0,
|
||||
RECV = 1 << 1,
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#Spec
|
||||
#[derive(Debug)]
|
||||
pub enum Spec {}
|
||||
|
||||
// https://pkg.go.dev/go/ast#Expr
|
||||
#[derive(Debug)]
|
||||
pub enum Expr<'a> {
|
||||
ArrayType(ArrayType<'a>),
|
||||
BasicLit(BasicLit<'a>),
|
||||
BinaryExpr(BinaryExpr<'a>),
|
||||
CallExpr(CallExpr<'a>),
|
||||
ChanType(ChanType<'a>),
|
||||
CompositeLit(CompositeLit<'a>),
|
||||
Ellipsis(Ellipsis<'a>),
|
||||
FuncLit(FuncLit<'a>),
|
||||
FuncType(FuncType<'a>),
|
||||
Ident(Ident<'a>),
|
||||
IndexExpr(IndexExpr<'a>),
|
||||
InterfaceType(InterfaceType<'a>),
|
||||
KeyValueExpr(KeyValueExpr<'a>),
|
||||
MapType(MapType<'a>),
|
||||
ParenExpr(ParenExpr<'a>),
|
||||
SelectorExpr(SelectorExpr<'a>),
|
||||
SliceExpr(SliceExpr<'a>),
|
||||
StarExpr(StarExpr<'a>),
|
||||
StructType(StructType<'a>),
|
||||
TypeAssertExpr(TypeAssertExpr<'a>),
|
||||
UnaryExpr(UnaryExpr<'a>),
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/go/ast#Stmt
|
||||
#[derive(Debug)]
|
||||
pub enum Stmt {}
|
||||
@@ -1,948 +0,0 @@
|
||||
// https://golang.org/ref/spec#Lexical_elements
|
||||
|
||||
use crate::parser_go_token::{Position, Token};
|
||||
use phf::{phf_map, Map};
|
||||
use std::fmt;
|
||||
use unicode_general_category::{get_general_category, GeneralCategory};
|
||||
|
||||
pub type Step<'a> = (Position<'a>, Token, &'a str);
|
||||
|
||||
#[derive(Debug)]
|
||||
pub enum ScannerError {
|
||||
HexadecimalNotFound,
|
||||
OctalNotFound,
|
||||
UnterminatedComment,
|
||||
UnterminatedEscapedChar,
|
||||
UnterminatedRune,
|
||||
UnterminatedString,
|
||||
InvalidDirective,
|
||||
}
|
||||
|
||||
impl std::error::Error for ScannerError {}
|
||||
|
||||
impl fmt::Display for ScannerError {
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
write!(f, "scanner error: {:?}", self)
|
||||
}
|
||||
}
|
||||
|
||||
pub type Result<T> = std::result::Result<T, ScannerError>;
|
||||
|
||||
#[derive(Debug)]
|
||||
pub struct Scanner<'a> {
|
||||
directory: &'a str,
|
||||
file: &'a str,
|
||||
buffer: &'a str,
|
||||
//
|
||||
chars: std::iter::Peekable<std::str::Chars<'a>>,
|
||||
current_char: Option<char>,
|
||||
current_char_len: usize,
|
||||
//
|
||||
offset: usize,
|
||||
line: usize,
|
||||
column: usize,
|
||||
start_offset: usize,
|
||||
start_line: usize,
|
||||
start_column: usize,
|
||||
//
|
||||
hide_column: bool,
|
||||
insert_semi: bool,
|
||||
pending_line_info: Option<LineInfo<'a>>,
|
||||
}
|
||||
|
||||
type LineInfo<'a> = (Option<&'a str>, usize, Option<usize>, bool);
|
||||
|
||||
impl<'a> Scanner<'a> {
|
||||
pub fn new(filename: &'a str, buffer: &'a str) -> Self {
|
||||
let (directory, file) = filename.rsplit_once('/').unwrap_or(("", filename));
|
||||
let mut s = Scanner {
|
||||
directory,
|
||||
file,
|
||||
buffer,
|
||||
//
|
||||
chars: buffer.chars().peekable(),
|
||||
current_char: None,
|
||||
current_char_len: 0,
|
||||
//
|
||||
offset: 0,
|
||||
line: 1,
|
||||
column: 1,
|
||||
start_offset: 0,
|
||||
start_line: 1,
|
||||
start_column: 1,
|
||||
//
|
||||
hide_column: false,
|
||||
insert_semi: false,
|
||||
pending_line_info: None,
|
||||
};
|
||||
s.next(); // read the first character
|
||||
s
|
||||
}
|
||||
|
||||
#[allow(clippy::cognitive_complexity)] // Allow complex scan function
|
||||
pub fn scan(&mut self) -> Result<Step<'a>> {
|
||||
let insert_semi = self.insert_semi;
|
||||
self.insert_semi = false;
|
||||
|
||||
while let Some(c) = self.current_char {
|
||||
self.reset_start();
|
||||
|
||||
match c {
|
||||
' ' | '\t' | '\r' => {
|
||||
self.next();
|
||||
}
|
||||
|
||||
'\n' => {
|
||||
self.next();
|
||||
if insert_semi {
|
||||
return Ok((self.position(), Token::SEMICOLON, "\n"));
|
||||
}
|
||||
}
|
||||
|
||||
_ => break,
|
||||
}
|
||||
}
|
||||
|
||||
if let Some(c) = self.current_char {
|
||||
match c {
|
||||
'+' => {
|
||||
self.next();
|
||||
match self.current_char {
|
||||
Some('=') => {
|
||||
self.next();
|
||||
return Ok((self.position(), Token::ADD_ASSIGN, ""));
|
||||
}
|
||||
Some('+') => {
|
||||
self.insert_semi = true;
|
||||
self.next();
|
||||
return Ok((self.position(), Token::INC, ""));
|
||||
}
|
||||
_ => return Ok((self.position(), Token::ADD, "")),
|
||||
}
|
||||
}
|
||||
|
||||
'-' => {
|
||||
self.next();
|
||||
match self.current_char {
|
||||
Some('=') => {
|
||||
self.next();
|
||||
return Ok((self.position(), Token::SUB_ASSIGN, ""));
|
||||
}
|
||||
Some('-') => {
|
||||
self.insert_semi = true;
|
||||
self.next();
|
||||
return Ok((self.position(), Token::DEC, ""));
|
||||
}
|
||||
_ => return Ok((self.position(), Token::SUB, "")),
|
||||
}
|
||||
}
|
||||
|
||||
'*' => {
|
||||
self.next();
|
||||
match self.current_char {
|
||||
Some('=') => {
|
||||
self.next();
|
||||
return Ok((self.position(), Token::MUL_ASSIGN, ""));
|
||||
}
|
||||
_ => return Ok((self.position(), Token::MUL, "")),
|
||||
}
|
||||
}
|
||||
|
||||
'/' => match self.peek() {
|
||||
Some('=') => {
|
||||
self.next();
|
||||
self.next();
|
||||
return Ok((self.position(), Token::QUO_ASSIGN, ""));
|
||||
}
|
||||
Some('/') => {
|
||||
if insert_semi {
|
||||
return Ok((self.position(), Token::SEMICOLON, "\n"));
|
||||
}
|
||||
return self.scan_line_comment();
|
||||
}
|
||||
Some('*') => {
|
||||
if insert_semi && self.find_line_end() {
|
||||
return Ok((self.position(), Token::SEMICOLON, "\n"));
|
||||
}
|
||||
return self.scan_general_comment();
|
||||
}
|
||||
_ => {
|
||||
self.next();
|
||||
return Ok((self.position(), Token::QUO, ""));
|
||||
}
|
||||
},
|
||||
|
||||
'%' => {
|
||||
self.next();
|
||||
match self.current_char {
|
||||
Some('=') => {
|
||||
self.next();
|
||||
return Ok((self.position(), Token::REM_ASSIGN, ""));
|
||||
}
|
||||
_ => return Ok((self.position(), Token::REM, "")),
|
||||
}
|
||||
}
|
||||
|
||||
'&' => {
|
||||
self.next();
|
||||
match self.current_char {
|
||||
Some('=') => {
|
||||
self.next();
|
||||
return Ok((self.position(), Token::AND_ASSIGN, ""));
|
||||
}
|
||||
Some('&') => {
|
||||
self.next();
|
||||
return Ok((self.position(), Token::LAND, ""));
|
||||
}
|
||||
Some('^') => {
|
||||
self.next();
|
||||
match self.current_char {
|
||||
Some('=') => {
|
||||
self.next();
|
||||
return Ok((self.position(), Token::AND_NOT_ASSIGN, ""));
|
||||
}
|
||||
_ => return Ok((self.position(), Token::AND_NOT, "")),
|
||||
}
|
||||
}
|
||||
_ => return Ok((self.position(), Token::AND, "")),
|
||||
}
|
||||
}
|
||||
|
||||
'|' => {
|
||||
self.next();
|
||||
match self.current_char {
|
||||
Some('=') => {
|
||||
self.next();
|
||||
return Ok((self.position(), Token::OR_ASSIGN, ""));
|
||||
}
|
||||
Some('|') => {
|
||||
self.next();
|
||||
return Ok((self.position(), Token::LOR, ""));
|
||||
}
|
||||
_ => return Ok((self.position(), Token::OR, "")),
|
||||
}
|
||||
}
|
||||
|
||||
'^' => {
|
||||
self.next();
|
||||
match self.current_char {
|
||||
Some('=') => {
|
||||
self.next();
|
||||
return Ok((self.position(), Token::XOR_ASSIGN, ""));
|
||||
}
|
||||
_ => return Ok((self.position(), Token::XOR, "")),
|
||||
}
|
||||
}
|
||||
|
||||
'<' => {
|
||||
self.next();
|
||||
match self.current_char {
|
||||
Some('<') => {
|
||||
self.next();
|
||||
match self.current_char {
|
||||
Some('=') => {
|
||||
self.next();
|
||||
return Ok((self.position(), Token::SHL_ASSIGN, ""));
|
||||
}
|
||||
_ => return Ok((self.position(), Token::SHL, "")),
|
||||
}
|
||||
}
|
||||
Some('=') => {
|
||||
self.next();
|
||||
return Ok((self.position(), Token::LEQ, ""));
|
||||
}
|
||||
Some('-') => {
|
||||
self.next();
|
||||
return Ok((self.position(), Token::ARROW, ""));
|
||||
}
|
||||
_ => return Ok((self.position(), Token::LSS, "")),
|
||||
}
|
||||
}
|
||||
|
||||
'>' => {
|
||||
self.next();
|
||||
match self.current_char {
|
||||
Some('>') => {
|
||||
self.next();
|
||||
match self.current_char {
|
||||
Some('=') => {
|
||||
self.next();
|
||||
return Ok((self.position(), Token::SHR_ASSIGN, ""));
|
||||
}
|
||||
_ => {
|
||||
return Ok((self.position(), Token::SHR, ""));
|
||||
}
|
||||
}
|
||||
}
|
||||
Some('=') => {
|
||||
self.next();
|
||||
return Ok((self.position(), Token::GEQ, ""));
|
||||
}
|
||||
_ => return Ok((self.position(), Token::GTR, "")),
|
||||
}
|
||||
}
|
||||
|
||||
':' => {
|
||||
self.next();
|
||||
match self.current_char {
|
||||
Some('=') => {
|
||||
self.next();
|
||||
return Ok((self.position(), Token::DEFINE, ""));
|
||||
}
|
||||
_ => return Ok((self.position(), Token::COLON, "")),
|
||||
}
|
||||
}
|
||||
|
||||
'!' => {
|
||||
self.next();
|
||||
match self.current_char {
|
||||
Some('=') => {
|
||||
self.next();
|
||||
return Ok((self.position(), Token::NEQ, ""));
|
||||
}
|
||||
_ => return Ok((self.position(), Token::NOT, "")),
|
||||
}
|
||||
}
|
||||
|
||||
',' => {
|
||||
self.next();
|
||||
return Ok((self.position(), Token::COMMA, ""));
|
||||
}
|
||||
|
||||
'(' => {
|
||||
self.next();
|
||||
return Ok((self.position(), Token::LPAREN, ""));
|
||||
}
|
||||
|
||||
')' => {
|
||||
self.insert_semi = true;
|
||||
self.next();
|
||||
return Ok((self.position(), Token::RPAREN, ""));
|
||||
}
|
||||
|
||||
'[' => {
|
||||
self.next();
|
||||
return Ok((self.position(), Token::LBRACK, ""));
|
||||
}
|
||||
|
||||
']' => {
|
||||
self.insert_semi = true;
|
||||
self.next();
|
||||
return Ok((self.position(), Token::RBRACK, ""));
|
||||
}
|
||||
|
||||
'{' => {
|
||||
self.next();
|
||||
return Ok((self.position(), Token::LBRACE, ""));
|
||||
}
|
||||
|
||||
'}' => {
|
||||
self.insert_semi = true;
|
||||
self.next();
|
||||
return Ok((self.position(), Token::RBRACE, ""));
|
||||
}
|
||||
|
||||
';' => {
|
||||
self.next();
|
||||
return Ok((self.position(), Token::SEMICOLON, ";"));
|
||||
}
|
||||
|
||||
'.' => {
|
||||
self.next();
|
||||
match self.current_char {
|
||||
Some('0'..='9') => return self.scan_int_or_float_or_imag(true),
|
||||
Some('.') => match self.peek() {
|
||||
Some('.') => {
|
||||
self.next();
|
||||
self.next();
|
||||
return Ok((self.position(), Token::ELLIPSIS, ""));
|
||||
}
|
||||
_ => return Ok((self.position(), Token::PERIOD, "")),
|
||||
},
|
||||
_ => return Ok((self.position(), Token::PERIOD, "")),
|
||||
}
|
||||
}
|
||||
|
||||
'=' => {
|
||||
self.next();
|
||||
match self.current_char {
|
||||
Some('=') => {
|
||||
self.next();
|
||||
return Ok((self.position(), Token::EQL, ""));
|
||||
}
|
||||
_ => return Ok((self.position(), Token::ASSIGN, "")),
|
||||
}
|
||||
}
|
||||
|
||||
'0'..='9' => return self.scan_int_or_float_or_imag(false),
|
||||
'\'' => return self.scan_rune(),
|
||||
'"' => return self.scan_interpreted_string(),
|
||||
'`' => return self.scan_raw_string(),
|
||||
_ => return self.scan_pkg_or_keyword_or_ident(),
|
||||
};
|
||||
}
|
||||
|
||||
self.reset_start();
|
||||
if insert_semi {
|
||||
Ok((self.position(), Token::SEMICOLON, "\n"))
|
||||
} else {
|
||||
Ok((self.position(), Token::EOF, ""))
|
||||
}
|
||||
}
|
||||
|
||||
// https://golang.org/ref/spec#Keywords
|
||||
// https://golang.org/ref/spec#Identifiers
|
||||
fn scan_pkg_or_keyword_or_ident(&mut self) -> Result<Step<'a>> {
|
||||
self.next();
|
||||
|
||||
while let Some(c) = self.current_char {
|
||||
if !(is_letter(c) || is_unicode_digit(c)) {
|
||||
break;
|
||||
}
|
||||
self.next()
|
||||
}
|
||||
|
||||
let pos = self.position();
|
||||
let literal = self.literal();
|
||||
|
||||
if literal.len() > 1 {
|
||||
if let Some(&token) = KEYWORDS.get(literal) {
|
||||
self.insert_semi = matches!(
|
||||
token,
|
||||
Token::BREAK | Token::CONTINUE | Token::FALLTHROUGH | Token::RETURN
|
||||
);
|
||||
return Ok((pos, token, literal));
|
||||
}
|
||||
}
|
||||
|
||||
self.insert_semi = true;
|
||||
Ok((pos, Token::IDENT, literal))
|
||||
}
|
||||
|
||||
// https://golang.org/ref/spec#Integer_literals
|
||||
// https://golang.org/ref/spec#Floating-point_literals
|
||||
// https://golang.org/ref/spec#Imaginary_literals
|
||||
fn scan_int_or_float_or_imag(&mut self, preceding_dot: bool) -> Result<Step<'a>> {
|
||||
self.insert_semi = true;
|
||||
|
||||
let mut token = Token::INT;
|
||||
let mut digits = "_0123456789";
|
||||
let mut exp = "eE";
|
||||
|
||||
if !preceding_dot {
|
||||
if matches!(self.current_char, Some('0')) {
|
||||
self.next();
|
||||
match self.current_char {
|
||||
Some('b' | 'B') => {
|
||||
digits = "_01";
|
||||
exp = "";
|
||||
self.next();
|
||||
}
|
||||
Some('o' | 'O') => {
|
||||
digits = "_01234567";
|
||||
exp = "";
|
||||
self.next();
|
||||
}
|
||||
Some('x' | 'X') => {
|
||||
digits = "_0123456789abcdefABCDEF";
|
||||
exp = "pP";
|
||||
self.next();
|
||||
}
|
||||
_ => {}
|
||||
};
|
||||
}
|
||||
|
||||
while let Some(c) = self.current_char {
|
||||
if !digits.contains(c) {
|
||||
break;
|
||||
}
|
||||
self.next();
|
||||
}
|
||||
}
|
||||
|
||||
if preceding_dot || matches!(self.current_char, Some('.')) {
|
||||
token = Token::FLOAT;
|
||||
self.next();
|
||||
while let Some(c) = self.current_char {
|
||||
if !digits.contains(c) {
|
||||
break;
|
||||
}
|
||||
self.next();
|
||||
}
|
||||
}
|
||||
|
||||
if !exp.is_empty() {
|
||||
if let Some(c) = self.current_char {
|
||||
if exp.contains(c) {
|
||||
token = Token::FLOAT;
|
||||
self.next();
|
||||
if matches!(self.current_char, Some('-' | '+')) {
|
||||
self.next();
|
||||
}
|
||||
while let Some(c) = self.current_char {
|
||||
if !matches!(c, '_' | '0'..='9') {
|
||||
break;
|
||||
}
|
||||
self.next();
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if matches!(self.current_char, Some('i')) {
|
||||
token = Token::IMAG;
|
||||
self.next();
|
||||
}
|
||||
|
||||
Ok((self.position(), token, self.literal()))
|
||||
}
|
||||
|
||||
// https://golang.org/ref/spec#Rune_literals
|
||||
fn scan_rune(&mut self) -> Result<Step<'a>> {
|
||||
self.insert_semi = true;
|
||||
self.next();
|
||||
|
||||
match self.current_char {
|
||||
Some('\\') => self.require_escaped_char::<'\''>()?,
|
||||
Some(_) => self.next(),
|
||||
_ => return Err(ScannerError::UnterminatedRune),
|
||||
}
|
||||
|
||||
if matches!(self.current_char, Some('\'')) {
|
||||
self.next();
|
||||
return Ok((self.position(), Token::CHAR, self.literal()));
|
||||
}
|
||||
|
||||
Err(ScannerError::UnterminatedRune)
|
||||
}
|
||||
|
||||
// https://golang.org/ref/spec#String_literals
|
||||
fn scan_interpreted_string(&mut self) -> Result<Step<'a>> {
|
||||
self.insert_semi = true;
|
||||
self.next();
|
||||
|
||||
while let Some(c) = self.current_char {
|
||||
match c {
|
||||
'"' => {
|
||||
self.next();
|
||||
return Ok((self.position(), Token::STRING, self.literal()));
|
||||
}
|
||||
'\\' => self.require_escaped_char::<'"'>()?,
|
||||
_ => self.next(),
|
||||
}
|
||||
}
|
||||
|
||||
Err(ScannerError::UnterminatedString)
|
||||
}
|
||||
|
||||
// https://golang.org/ref/spec#String_literals
|
||||
fn scan_raw_string(&mut self) -> Result<Step<'a>> {
|
||||
self.insert_semi = true;
|
||||
self.next();
|
||||
|
||||
while let Some(c) = self.current_char {
|
||||
match c {
|
||||
'`' => {
|
||||
self.next();
|
||||
return Ok((self.position(), Token::STRING, self.literal()));
|
||||
}
|
||||
_ => self.next(),
|
||||
}
|
||||
}
|
||||
|
||||
Err(ScannerError::UnterminatedString)
|
||||
}
|
||||
|
||||
// https://golang.org/ref/spec#Comments
|
||||
fn scan_general_comment(&mut self) -> Result<Step<'a>> {
|
||||
self.next();
|
||||
self.next();
|
||||
|
||||
while let Some(c) = self.current_char {
|
||||
match c {
|
||||
'*' => {
|
||||
self.next();
|
||||
if matches!(self.current_char, Some('/')) {
|
||||
self.next();
|
||||
|
||||
let pos = self.position();
|
||||
let lit = self.literal();
|
||||
|
||||
// look for compiler directives
|
||||
self.directive(&lit["/*".len()..lit.len() - "*/".len()], true)?;
|
||||
|
||||
return Ok((pos, Token::COMMENT, lit));
|
||||
}
|
||||
}
|
||||
_ => self.next(),
|
||||
}
|
||||
}
|
||||
|
||||
Err(ScannerError::UnterminatedComment)
|
||||
}
|
||||
|
||||
// https://golang.org/ref/spec#Comments
|
||||
fn scan_line_comment(&mut self) -> Result<Step<'a>> {
|
||||
self.next();
|
||||
self.next();
|
||||
|
||||
while let Some(c) = self.current_char {
|
||||
if is_newline(c) {
|
||||
break;
|
||||
}
|
||||
self.next();
|
||||
}
|
||||
|
||||
let pos = self.position();
|
||||
let lit = self.literal();
|
||||
|
||||
// look for compiler directives (at the beginning of line)
|
||||
if self.start_column == 1 {
|
||||
self.directive(lit["//".len()..].trim_end(), false)?;
|
||||
}
|
||||
|
||||
Ok((pos, Token::COMMENT, self.literal()))
|
||||
}
|
||||
|
||||
// https://pkg.go.dev/cmd/compile#hdr-Compiler_Directives
|
||||
fn directive(&mut self, input: &'a str, immediate: bool) -> Result<()> {
|
||||
if let Some(line_directive) = input.strip_prefix("line ") {
|
||||
self.pending_line_info = self.parse_line_directive(line_directive)?;
|
||||
if immediate {
|
||||
self.consume_pending_line_info();
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn parse_line_directive(&mut self, line_directive: &'a str) -> Result<Option<LineInfo<'a>>> {
|
||||
if let Some((file, line)) = line_directive.rsplit_once(':') {
|
||||
let line = line.parse().map_err(|_| ScannerError::InvalidDirective)?;
|
||||
|
||||
if let Some((file, l)) = file.rsplit_once(':') {
|
||||
if let Ok(l) = l.parse() {
|
||||
//line :line:col
|
||||
//line filename:line:col
|
||||
/*line :line:col*/
|
||||
/*line filename:line:col*/
|
||||
let file = if !file.is_empty() { Some(file) } else { None };
|
||||
let col = Some(line);
|
||||
let line = l;
|
||||
let hide_column = false;
|
||||
return Ok(Some((file, line, col, hide_column)));
|
||||
}
|
||||
}
|
||||
|
||||
//line :line
|
||||
//line filename:line
|
||||
/*line :line*/
|
||||
/*line filename:line*/
|
||||
Ok(Some((Some(file), line, None, true)))
|
||||
} else {
|
||||
Ok(None)
|
||||
}
|
||||
}
|
||||
|
||||
const fn find_line_end(&self) -> bool {
|
||||
let buffer = self.buffer.as_bytes();
|
||||
let mut in_comment = true;
|
||||
|
||||
let mut i = self.offset;
|
||||
let max = self.buffer.len();
|
||||
while i < max {
|
||||
let c = buffer[i] as char;
|
||||
|
||||
if i < max - 1 {
|
||||
let n = buffer[i + 1] as char;
|
||||
|
||||
if !in_comment && c == '/' && n == '/' {
|
||||
return true;
|
||||
}
|
||||
|
||||
if c == '/' && n == '*' {
|
||||
i += 2;
|
||||
in_comment = true;
|
||||
continue;
|
||||
}
|
||||
|
||||
if c == '*' && n == '/' {
|
||||
i += 2;
|
||||
in_comment = false;
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
if is_newline(c) {
|
||||
return true;
|
||||
}
|
||||
|
||||
if !in_comment && !matches!(c, ' ' | '\t' | '\r') {
|
||||
return false;
|
||||
}
|
||||
|
||||
i += 1;
|
||||
}
|
||||
|
||||
!in_comment
|
||||
}
|
||||
|
||||
fn consume_pending_line_info(&mut self) {
|
||||
if let Some(line_info) = self.pending_line_info.take() {
|
||||
if let Some(file) = line_info.0 {
|
||||
self.file = file;
|
||||
}
|
||||
|
||||
self.line = line_info.1;
|
||||
|
||||
if let Some(column) = line_info.2 {
|
||||
self.column = column;
|
||||
}
|
||||
|
||||
self.hide_column = line_info.3;
|
||||
}
|
||||
}
|
||||
|
||||
fn peek(&mut self) -> Option<char> {
|
||||
self.chars.peek().copied()
|
||||
}
|
||||
|
||||
fn next(&mut self) {
|
||||
self.offset += self.current_char_len;
|
||||
self.column += self.current_char_len;
|
||||
let last_char = self.current_char;
|
||||
|
||||
self.current_char = self.chars.next();
|
||||
if let Some(c) = self.current_char {
|
||||
self.current_char_len = c.len_utf8();
|
||||
if matches!(last_char, Some('\n')) {
|
||||
self.line += 1;
|
||||
self.column = 1;
|
||||
self.consume_pending_line_info();
|
||||
}
|
||||
} else {
|
||||
self.current_char_len = 0
|
||||
}
|
||||
}
|
||||
|
||||
const fn position(&self) -> Position<'a> {
|
||||
Position {
|
||||
directory: self.directory,
|
||||
file: self.file,
|
||||
offset: self.start_offset,
|
||||
line: self.start_line,
|
||||
column: if self.hide_column {
|
||||
0
|
||||
} else {
|
||||
self.start_column
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
fn reset_start(&mut self) {
|
||||
self.start_offset = self.offset;
|
||||
self.start_line = self.line;
|
||||
self.start_column = self.column;
|
||||
}
|
||||
|
||||
fn literal(&self) -> &'a str {
|
||||
&self.buffer[self.start_offset..self.offset]
|
||||
}
|
||||
|
||||
fn require_escaped_char<const DELIM: char>(&mut self) -> Result<()> {
|
||||
self.next();
|
||||
|
||||
let c = self
|
||||
.current_char
|
||||
.ok_or(ScannerError::UnterminatedEscapedChar)?;
|
||||
|
||||
// TODO: move this to the match when const generics can be referenced in patterns
|
||||
if c == DELIM {
|
||||
self.next();
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
match c {
|
||||
'a' | 'b' | 'f' | 'n' | 'r' | 't' | 'v' | '\\' => self.next(),
|
||||
'x' => {
|
||||
self.next();
|
||||
self.require_hex_digits::<2>()?
|
||||
}
|
||||
'u' => {
|
||||
self.next();
|
||||
self.require_hex_digits::<4>()?;
|
||||
}
|
||||
'U' => {
|
||||
self.next();
|
||||
self.require_hex_digits::<8>()?;
|
||||
}
|
||||
'0'..='7' => self.require_octal_digits::<3>()?,
|
||||
_ => return Err(ScannerError::UnterminatedEscapedChar),
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn require_octal_digits<const COUNT: usize>(&mut self) -> Result<()> {
|
||||
for _ in 0..COUNT {
|
||||
let c = self.current_char.ok_or(ScannerError::OctalNotFound)?;
|
||||
|
||||
if !is_octal_digit(c) {
|
||||
return Err(ScannerError::OctalNotFound);
|
||||
}
|
||||
|
||||
self.next();
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn require_hex_digits<const COUNT: usize>(&mut self) -> Result<()> {
|
||||
for _ in 0..COUNT {
|
||||
let c = self.current_char.ok_or(ScannerError::HexadecimalNotFound)?;
|
||||
|
||||
if !is_hex_digit(c) {
|
||||
return Err(ScannerError::HexadecimalNotFound);
|
||||
}
|
||||
|
||||
self.next();
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
impl<'a> IntoIterator for Scanner<'a> {
|
||||
type Item = Result<Step<'a>>;
|
||||
type IntoIter = IntoIter<'a>;
|
||||
|
||||
fn into_iter(self) -> Self::IntoIter {
|
||||
Self::IntoIter::new(self)
|
||||
}
|
||||
}
|
||||
|
||||
pub struct IntoIter<'a> {
|
||||
scanner: Scanner<'a>,
|
||||
done: bool,
|
||||
}
|
||||
|
||||
impl<'a> IntoIter<'a> {
|
||||
const fn new(scanner: Scanner<'a>) -> Self {
|
||||
Self { scanner, done: false }
|
||||
}
|
||||
}
|
||||
|
||||
impl<'a> Iterator for IntoIter<'a> {
|
||||
type Item = Result<Step<'a>>;
|
||||
|
||||
fn next(&mut self) -> Option<Self::Item> {
|
||||
if self.done {
|
||||
return None;
|
||||
}
|
||||
|
||||
match self.scanner.scan() {
|
||||
Ok((pos, tok, lit)) => {
|
||||
if tok == Token::EOF {
|
||||
self.done = true;
|
||||
}
|
||||
Some(Ok((pos, tok, lit)))
|
||||
}
|
||||
Err(err) => {
|
||||
self.done = true;
|
||||
Some(Err(err))
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// https://golang.org/ref/spec#Letters_and_digits
|
||||
|
||||
fn is_letter(c: char) -> bool {
|
||||
c == '_' || is_unicode_letter(c)
|
||||
}
|
||||
|
||||
//const fn is_decimal_digit(c: char) -> bool {
|
||||
//matches!(c, '0'..='9')
|
||||
//}
|
||||
|
||||
//const fn is_binary_digit(c: char) -> bool {
|
||||
//matches!(c, '0'..='1')
|
||||
//}
|
||||
|
||||
const fn is_octal_digit(c: char) -> bool {
|
||||
matches!(c, '0'..='7')
|
||||
}
|
||||
|
||||
const fn is_hex_digit(c: char) -> bool {
|
||||
matches!(c, '0'..='9' | 'A'..='F' | 'a'..='f')
|
||||
}
|
||||
|
||||
// https://golang.org/ref/spec#Characters
|
||||
|
||||
const fn is_newline(c: char) -> bool {
|
||||
c == '\n'
|
||||
}
|
||||
|
||||
//const fn is_unicode_char(c: char) -> bool {
|
||||
//c != '\n'
|
||||
//}
|
||||
|
||||
fn is_unicode_letter(c: char) -> bool {
|
||||
matches!(
|
||||
get_general_category(c),
|
||||
GeneralCategory::UppercaseLetter
|
||||
| GeneralCategory::LowercaseLetter
|
||||
| GeneralCategory::TitlecaseLetter
|
||||
| GeneralCategory::ModifierLetter
|
||||
| GeneralCategory::OtherLetter
|
||||
)
|
||||
}
|
||||
|
||||
fn is_unicode_digit(c: char) -> bool {
|
||||
get_general_category(c) == GeneralCategory::DecimalNumber
|
||||
}
|
||||
|
||||
// https://golang.org/ref/spec#Keywords
|
||||
|
||||
static KEYWORDS: Map<&'static str, Token> = phf_map! {
|
||||
"break" => Token::BREAK,
|
||||
"case" => Token::CASE,
|
||||
"chan" => Token::CHAN,
|
||||
"const" => Token::CONST,
|
||||
"continue" => Token::CONTINUE,
|
||||
|
||||
"default" => Token::DEFAULT,
|
||||
"defer" => Token::DEFER,
|
||||
"else" => Token::ELSE,
|
||||
"fallthrough" => Token::FALLTHROUGH,
|
||||
"for" => Token::FOR,
|
||||
|
||||
"func" => Token::FUNC,
|
||||
"go" => Token::GO,
|
||||
"goto" => Token::GOTO,
|
||||
"if" => Token::IF,
|
||||
"import" => Token::IMPORT,
|
||||
|
||||
"interface" => Token::INTERFACE,
|
||||
"map" => Token::MAP,
|
||||
"package" => Token::PACKAGE,
|
||||
"range" => Token::RANGE,
|
||||
"return" => Token::RETURN,
|
||||
|
||||
"select" => Token::SELECT,
|
||||
"struct" => Token::STRUCT,
|
||||
"switch" => Token::SWITCH,
|
||||
"type" => Token::TYPE,
|
||||
"var" => Token::VAR,
|
||||
};
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::Scanner;
|
||||
|
||||
#[test] // fuzz
|
||||
fn it_should_return_an_error_on_missing_line_number() {
|
||||
let input = "/*line :*/";
|
||||
let mut out: Vec<_> = Scanner::new(file!(), input).into_iter().collect();
|
||||
assert!(out.pop().unwrap().is_err());
|
||||
}
|
||||
}
|
||||
@@ -1,273 +0,0 @@
|
||||
// https://cs.opensource.google/go/go/+/refs/tags/go1.17.2:src/go/token/token.go
|
||||
|
||||
#![allow(non_camel_case_types)] // For consistency with the Go tokens
|
||||
|
||||
use std::fmt;
|
||||
|
||||
#[derive(Clone, Copy, Debug, Default)]
|
||||
pub struct Position<'a> {
|
||||
pub directory: &'a str,
|
||||
pub file: &'a str,
|
||||
pub offset: usize,
|
||||
pub line: usize,
|
||||
pub column: usize,
|
||||
}
|
||||
|
||||
impl<'a> fmt::Display for Position<'a> {
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
if self.file.is_empty() {
|
||||
write!(f, ":{}:{}", self.line, self.column)
|
||||
} else if self.file.starts_with('/') {
|
||||
write!(f, "{}:{}:{}", self.file, self.line, self.column)
|
||||
} else {
|
||||
write!(
|
||||
f,
|
||||
"{}/{}:{}:{}",
|
||||
self.directory, self.file, self.line, self.column
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, Eq, PartialEq)]
|
||||
pub enum Token {
|
||||
EOF,
|
||||
COMMENT,
|
||||
|
||||
IDENT, // main
|
||||
INT, // 12345
|
||||
FLOAT, // 123.45
|
||||
IMAG, // 123.45i
|
||||
CHAR, // 'a'
|
||||
STRING, // "abc"
|
||||
|
||||
ADD, // +
|
||||
SUB, // -
|
||||
MUL, // *
|
||||
QUO, // /
|
||||
REM, // %
|
||||
|
||||
AND, // &
|
||||
OR, // |
|
||||
XOR, // ^
|
||||
SHL, // <<
|
||||
SHR, // >>
|
||||
AND_NOT, // &^
|
||||
|
||||
ADD_ASSIGN, // +=
|
||||
SUB_ASSIGN, // -=
|
||||
MUL_ASSIGN, // *=
|
||||
QUO_ASSIGN, // /=
|
||||
REM_ASSIGN, // %=
|
||||
|
||||
AND_ASSIGN, // &=
|
||||
OR_ASSIGN, // |=
|
||||
XOR_ASSIGN, // ^=
|
||||
SHL_ASSIGN, // <<=
|
||||
SHR_ASSIGN, // >>=
|
||||
AND_NOT_ASSIGN, // &^=
|
||||
|
||||
LAND, // &&
|
||||
LOR, // ||
|
||||
ARROW, // <-
|
||||
INC, // ++
|
||||
DEC, // --
|
||||
|
||||
EQL, // ==
|
||||
LSS, // <
|
||||
GTR, // >
|
||||
ASSIGN, // =
|
||||
NOT, // !
|
||||
|
||||
NEQ, // !=
|
||||
LEQ, // <=
|
||||
GEQ, // >=
|
||||
DEFINE, // :=
|
||||
ELLIPSIS, // ...
|
||||
|
||||
LPAREN, // (
|
||||
LBRACK, // [
|
||||
LBRACE, // {
|
||||
COMMA, // ,
|
||||
PERIOD, // .
|
||||
|
||||
RPAREN, // )
|
||||
RBRACK, // ]
|
||||
RBRACE, // }
|
||||
SEMICOLON, // ;
|
||||
COLON, // :
|
||||
|
||||
BREAK,
|
||||
CASE,
|
||||
CHAN,
|
||||
CONST,
|
||||
CONTINUE,
|
||||
|
||||
DEFAULT,
|
||||
DEFER,
|
||||
ELSE,
|
||||
FALLTHROUGH,
|
||||
FOR,
|
||||
|
||||
FUNC,
|
||||
GO,
|
||||
GOTO,
|
||||
IF,
|
||||
IMPORT,
|
||||
|
||||
INTERFACE,
|
||||
MAP,
|
||||
PACKAGE,
|
||||
RANGE,
|
||||
RETURN,
|
||||
|
||||
SELECT,
|
||||
STRUCT,
|
||||
SWITCH,
|
||||
TYPE,
|
||||
VAR,
|
||||
}
|
||||
|
||||
impl Token {
|
||||
pub const fn is_assign_op(&self) -> bool {
|
||||
use Token::*;
|
||||
matches!(
|
||||
self,
|
||||
ADD_ASSIGN
|
||||
| SUB_ASSIGN
|
||||
| MUL_ASSIGN
|
||||
| QUO_ASSIGN
|
||||
| REM_ASSIGN
|
||||
| AND_ASSIGN
|
||||
| OR_ASSIGN
|
||||
| XOR_ASSIGN
|
||||
| SHL_ASSIGN
|
||||
| SHR_ASSIGN
|
||||
| AND_NOT_ASSIGN
|
||||
)
|
||||
}
|
||||
|
||||
// https://go.dev/ref/spec#Operator_precedence
|
||||
pub fn precedence(&self) -> u8 {
|
||||
use Token::*;
|
||||
match self {
|
||||
MUL | QUO | REM | SHL | SHR | AND | AND_NOT => 5,
|
||||
ADD | SUB | OR | XOR => 4,
|
||||
EQL | NEQ | LSS | LEQ | GTR | GEQ => 3,
|
||||
LAND => 2,
|
||||
LOR => 1,
|
||||
_ => unreachable!(
|
||||
"precedence() is only supported for binary operators, called with: {:?}",
|
||||
self
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
pub const fn lowest_precedence() -> u8 {
|
||||
0
|
||||
}
|
||||
}
|
||||
|
||||
impl From<&Token> for &'static str {
|
||||
fn from(token: &Token) -> Self {
|
||||
use Token::*;
|
||||
|
||||
match token {
|
||||
EOF => "EOF",
|
||||
COMMENT => "COMMENT",
|
||||
|
||||
IDENT => "IDENT",
|
||||
INT => "INT",
|
||||
FLOAT => "FLOAT",
|
||||
IMAG => "IMAG",
|
||||
CHAR => "CHAR",
|
||||
STRING => "STRING",
|
||||
|
||||
ADD => "+",
|
||||
SUB => "-",
|
||||
MUL => "*",
|
||||
QUO => "/",
|
||||
REM => "%",
|
||||
|
||||
AND => "&",
|
||||
OR => "|",
|
||||
XOR => "^",
|
||||
SHL => "<<",
|
||||
SHR => ">>",
|
||||
AND_NOT => "&^",
|
||||
|
||||
ADD_ASSIGN => "+=",
|
||||
SUB_ASSIGN => "-=",
|
||||
MUL_ASSIGN => "*=",
|
||||
QUO_ASSIGN => "/=",
|
||||
REM_ASSIGN => "%=",
|
||||
|
||||
AND_ASSIGN => "&=",
|
||||
OR_ASSIGN => "|=",
|
||||
XOR_ASSIGN => "^=",
|
||||
SHL_ASSIGN => "<<=",
|
||||
SHR_ASSIGN => ">>=",
|
||||
AND_NOT_ASSIGN => "&^=",
|
||||
|
||||
LAND => "&&",
|
||||
LOR => "||",
|
||||
ARROW => "<-",
|
||||
INC => "++",
|
||||
DEC => "--",
|
||||
|
||||
EQL => "==",
|
||||
LSS => "<",
|
||||
GTR => ">",
|
||||
ASSIGN => "=",
|
||||
NOT => "!",
|
||||
|
||||
NEQ => "!=",
|
||||
LEQ => "<=",
|
||||
GEQ => ">=",
|
||||
DEFINE => ":=",
|
||||
ELLIPSIS => "...",
|
||||
|
||||
LPAREN => "(",
|
||||
LBRACK => "[",
|
||||
LBRACE => "{",
|
||||
COMMA => ",",
|
||||
PERIOD => ".",
|
||||
|
||||
RPAREN => ")",
|
||||
RBRACK => "]",
|
||||
RBRACE => "}",
|
||||
SEMICOLON => ";",
|
||||
COLON => ":",
|
||||
|
||||
BREAK => "break",
|
||||
CASE => "case",
|
||||
CHAN => "chan",
|
||||
CONST => "const",
|
||||
CONTINUE => "continue",
|
||||
|
||||
DEFAULT => "default",
|
||||
DEFER => "defer",
|
||||
ELSE => "else",
|
||||
FALLTHROUGH => "fallthrough",
|
||||
FOR => "for",
|
||||
|
||||
FUNC => "func",
|
||||
GO => "go",
|
||||
GOTO => "goto",
|
||||
IF => "if",
|
||||
IMPORT => "import",
|
||||
|
||||
INTERFACE => "interface",
|
||||
MAP => "map",
|
||||
PACKAGE => "package",
|
||||
RANGE => "range",
|
||||
RETURN => "return",
|
||||
|
||||
SELECT => "select",
|
||||
STRUCT => "struct",
|
||||
SWITCH => "switch",
|
||||
TYPE => "type",
|
||||
VAR => "var",
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -218,7 +218,7 @@ pub fn parse_python_imports(code: &str) -> error::Result<Vec<String>> {
|
||||
let ast = parser::parse_program(code, "main.py").map_err(|e| {
|
||||
error::Error::ExecutionErr(format!("Error parsing code: {}", e.to_string()))
|
||||
})?;
|
||||
let imports = ast
|
||||
let mut imports: Vec<String> = ast
|
||||
.into_iter()
|
||||
.filter_map(|x| match x {
|
||||
Located { node, .. } => match node {
|
||||
@@ -250,7 +250,7 @@ pub fn parse_python_imports(code: &str) -> error::Result<Vec<String>> {
|
||||
.filter(|x| !STDIMPORTS.contains(&x.as_str()))
|
||||
.unique()
|
||||
.collect();
|
||||
|
||||
imports.sort();
|
||||
Ok(imports)
|
||||
}
|
||||
}
|
||||
@@ -460,7 +460,7 @@ def main():
|
||||
";
|
||||
let r = parse_python_imports(code)?;
|
||||
// println!("{}", serde_json::to_string(&r)?);
|
||||
assert_eq!(r, vec!["wmill", "zanzibar", "matplotlib", "requests"]);
|
||||
assert_eq!(r, vec!["matplotlib", "requests", "wmill", "zanzibar"]);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
|
||||
@@ -6,21 +6,22 @@
|
||||
* LICENSE-AGPL for a copy of the license.
|
||||
*/
|
||||
use deno_core::{serde_v8, v8, JsRuntime, RuntimeOptions};
|
||||
use serde_json::Value;
|
||||
use windmill_common::error;
|
||||
use windmill_parser::{json_to_typ, Arg, MainArgSignature, ObjectProperty, Typ};
|
||||
|
||||
use swc_common::{sync::Lrc, FileName, SourceMap, SourceMapper, Spanned};
|
||||
use swc_common::{sync::Lrc, FileName, SourceMap, SourceMapper, Span, Spanned};
|
||||
use swc_ecma_ast::{
|
||||
ArrayLit, AssignPat, BigInt, BindingIdent, Bool, Decl, ExportDecl, Expr, FnDecl, Ident, Lit,
|
||||
ModuleDecl, ModuleItem, Number, ObjectLit, Pat, Str, TsArrayType, TsEntityName, TsKeywordType,
|
||||
TsKeywordTypeKind, TsLit, TsLitType, TsOptionalType, TsPropertySignature, TsType,
|
||||
TsTypeElement, TsTypeLit, TsTypeRef, TsUnionOrIntersectionType, TsUnionType,
|
||||
ModuleDecl, ModuleItem, Number, ObjectLit, Param, Pat, Str, TsArrayType, TsEntityName,
|
||||
TsKeywordType, TsKeywordTypeKind, TsLit, TsLitType, TsOptionalType, TsPropertySignature,
|
||||
TsType, TsTypeElement, TsTypeLit, TsTypeRef, TsUnionOrIntersectionType, TsUnionType,
|
||||
};
|
||||
use swc_ecma_parser::{lexer::Lexer, Parser, StringInput, Syntax, TsConfig};
|
||||
|
||||
pub fn parse_deno_signature(code: &str) -> error::Result<MainArgSignature> {
|
||||
pub fn parse_deno_signature(code: &str, skip_dflt: bool) -> error::Result<MainArgSignature> {
|
||||
let cm: Lrc<SourceMap> = Default::default();
|
||||
let fm = cm.new_source_file(FileName::Custom("test.ts".into()), code.into());
|
||||
let fm = cm.new_source_file(FileName::Custom("main.ts".into()), code.into());
|
||||
let lexer = Lexer::new(
|
||||
// We want to parse ecmascript
|
||||
Syntax::Typescript(TsConfig::default()),
|
||||
@@ -54,65 +55,17 @@ pub fn parse_deno_signature(code: &str) -> error::Result<MainArgSignature> {
|
||||
})) if &sym.to_string() == "main" => Some(function.params),
|
||||
_ => None,
|
||||
});
|
||||
|
||||
if let Some(params) = params {
|
||||
Ok(MainArgSignature {
|
||||
let r = MainArgSignature {
|
||||
star_args: false,
|
||||
star_kwargs: false,
|
||||
args: params
|
||||
.into_iter()
|
||||
.map(|x| match x.pat {
|
||||
Pat::Ident(ident) => {
|
||||
let (name, typ, nullable) = binding_ident_to_arg(&ident);
|
||||
Ok(Arg {
|
||||
otyp: None,
|
||||
name,
|
||||
typ,
|
||||
default: None,
|
||||
has_default: ident.id.optional || nullable,
|
||||
})
|
||||
}
|
||||
Pat::Assign(AssignPat { left, right, .. }) => {
|
||||
let (name, mut typ, _nullable) =
|
||||
left.as_ident().map(binding_ident_to_arg).ok_or_else(|| {
|
||||
error::Error::ExecutionErr(format!(
|
||||
"parameter syntax unsupported: `{}`",
|
||||
cm.span_to_snippet(left.span())
|
||||
.unwrap_or_else(|_| cm.span_to_string(left.span()))
|
||||
))
|
||||
})?;
|
||||
|
||||
let span = match *right {
|
||||
Expr::Lit(Lit::Str(Str { span, .. })) => Some(span),
|
||||
Expr::Lit(Lit::Num(Number { span, .. })) => Some(span),
|
||||
Expr::Lit(Lit::BigInt(BigInt { span, .. })) => Some(span),
|
||||
Expr::Lit(Lit::Bool(Bool { span, .. })) => Some(span),
|
||||
Expr::Object(ObjectLit { span, .. }) => Some(span),
|
||||
Expr::Array(ArrayLit { span, .. }) => Some(span),
|
||||
_ => None,
|
||||
};
|
||||
let expr = span
|
||||
.and_then(|x| cm.span_to_snippet(x).ok())
|
||||
.map(|x| serde_json::from_str(&x).map_err(|_| x));
|
||||
|
||||
let default = match expr.clone() {
|
||||
Some(Ok(x)) => Some(x),
|
||||
Some(Err(x)) => eval_sync(&x).ok(),
|
||||
None => None,
|
||||
};
|
||||
|
||||
if typ == Typ::Unknown && default.is_some() {
|
||||
typ = json_to_typ(default.as_ref().unwrap());
|
||||
}
|
||||
Ok(Arg { otyp: None, name, typ, default, has_default: true })
|
||||
}
|
||||
_ => Err(error::Error::ExecutionErr(format!(
|
||||
"parameter syntax unsupported: `{}`",
|
||||
cm.span_to_snippet(x.span())
|
||||
.unwrap_or_else(|_| cm.span_to_string(x.span()))
|
||||
))),
|
||||
})
|
||||
.map(|x| parse_param(x, &cm, skip_dflt))
|
||||
.collect::<Result<Vec<Arg>, error::Error>>()?,
|
||||
})
|
||||
};
|
||||
Ok(r)
|
||||
} else {
|
||||
Err(error::Error::ExecutionErr(
|
||||
"main function was not findable (expected to find 'export function main(...)'"
|
||||
@@ -121,6 +74,75 @@ pub fn parse_deno_signature(code: &str) -> error::Result<MainArgSignature> {
|
||||
}
|
||||
}
|
||||
|
||||
fn parse_param(x: Param, cm: &Lrc<SourceMap>, skip_dflt: bool) -> error::Result<Arg> {
|
||||
let r = match x.pat {
|
||||
Pat::Ident(ident) => {
|
||||
let (name, typ, nullable) = binding_ident_to_arg(&ident);
|
||||
Ok(Arg {
|
||||
otyp: None,
|
||||
name,
|
||||
typ,
|
||||
default: None,
|
||||
has_default: ident.id.optional || nullable,
|
||||
})
|
||||
}
|
||||
Pat::Assign(AssignPat { left, right, .. }) => {
|
||||
let (name, mut typ, _nullable) =
|
||||
left.as_ident().map(binding_ident_to_arg).ok_or_else(|| {
|
||||
error::Error::ExecutionErr(format!(
|
||||
"parameter syntax unsupported: `{}`",
|
||||
cm.span_to_snippet(left.span())
|
||||
.unwrap_or_else(|_| cm.span_to_string(left.span()))
|
||||
))
|
||||
})?;
|
||||
|
||||
let dflt = if skip_dflt {
|
||||
None
|
||||
} else {
|
||||
match *right {
|
||||
Expr::Lit(Lit::Str(Str { value, .. })) => {
|
||||
Some(Value::String(value.to_string()))
|
||||
}
|
||||
Expr::Lit(Lit::Num(Number { value, .. }))
|
||||
if (value == (value as u64) as f64) =>
|
||||
{
|
||||
Some(serde_json::json!(value as u64))
|
||||
}
|
||||
Expr::Lit(Lit::Num(Number { value, .. })) => Some(serde_json::json!(value)),
|
||||
Expr::Lit(Lit::BigInt(BigInt { value, .. })) => Some(serde_json::json!(value)),
|
||||
Expr::Lit(Lit::Bool(Bool { value, .. })) => Some(Value::Bool(value)),
|
||||
Expr::Object(ObjectLit { span, .. }) => eval_span(span, cm),
|
||||
Expr::Array(ArrayLit { span, .. }) => eval_span(span, cm),
|
||||
_ => None,
|
||||
}
|
||||
};
|
||||
|
||||
if typ == Typ::Unknown && dflt.is_some() {
|
||||
typ = json_to_typ(dflt.as_ref().unwrap());
|
||||
}
|
||||
Ok(Arg { otyp: None, name, typ, default: dflt, has_default: true })
|
||||
}
|
||||
_ => Err(error::Error::ExecutionErr(format!(
|
||||
"parameter syntax unsupported: `{}`",
|
||||
cm.span_to_snippet(x.span())
|
||||
.unwrap_or_else(|_| cm.span_to_string(x.span()))
|
||||
))),
|
||||
};
|
||||
r
|
||||
}
|
||||
|
||||
fn eval_span(span: Span, cm: &Lrc<SourceMap>) -> Option<Value> {
|
||||
let expr = cm
|
||||
.span_to_snippet(span)
|
||||
.ok()
|
||||
.map(|x| serde_json::from_str(&x).map_err(|_| x));
|
||||
|
||||
match expr {
|
||||
Some(Ok(x)) => Some(x),
|
||||
Some(Err(x)) => eval_sync(&x).ok(),
|
||||
None => None,
|
||||
}
|
||||
}
|
||||
fn binding_ident_to_arg(BindingIdent { id, type_ann }: &BindingIdent) -> (String, Typ, bool) {
|
||||
let (typ, nullable) = type_ann
|
||||
.as_ref()
|
||||
@@ -282,7 +304,7 @@ export function main(test1?: string, test2: string = \"burkina\",
|
||||
}
|
||||
";
|
||||
assert_eq!(
|
||||
parse_deno_signature(code)?,
|
||||
parse_deno_signature(code, false)?,
|
||||
MainArgSignature {
|
||||
star_args: false,
|
||||
star_kwargs: false,
|
||||
@@ -394,7 +416,7 @@ export function main(test2 = \"burkina\",
|
||||
}
|
||||
";
|
||||
assert_eq!(
|
||||
parse_deno_signature(code)?,
|
||||
parse_deno_signature(code, false)?,
|
||||
MainArgSignature {
|
||||
star_args: false,
|
||||
star_kwargs: false,
|
||||
|
||||
@@ -1319,6 +1319,24 @@
|
||||
},
|
||||
"query": "SELECT workspace_id, name, display_name, owners, extra_perms FROM folder WHERE name = $1 AND workspace_id = $2"
|
||||
},
|
||||
"399a8337a2488fa2ce3da2ef3281a34f8f96ee0d833c2fe3c22a0aa43e306f09": {
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"name": "hash",
|
||||
"ordinal": 0,
|
||||
"type_info": "Varchar"
|
||||
}
|
||||
],
|
||||
"nullable": [
|
||||
false
|
||||
],
|
||||
"parameters": {
|
||||
"Left": []
|
||||
}
|
||||
},
|
||||
"query": "DELETE FROM pip_resolution_cache WHERE expiration <= now() RETURNING hash"
|
||||
},
|
||||
"39bae7dff750565183ac3893b51a8846d7db2f130a6795b54ef94acc232dec8b": {
|
||||
"describe": {
|
||||
"columns": [
|
||||
@@ -1790,6 +1808,19 @@
|
||||
},
|
||||
"query": "\n UPDATE queue\n SET flow_status = JSONB_SET(flow_status, ARRAY['step'], $1)\n WHERE id = $2\n "
|
||||
},
|
||||
"4fb3881cdbb4b9e93e28f460a9b3715bdc6a52b76c89f3a3913023b13c4e085c": {
|
||||
"describe": {
|
||||
"columns": [],
|
||||
"nullable": [],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Varchar",
|
||||
"Text"
|
||||
]
|
||||
}
|
||||
},
|
||||
"query": "INSERT INTO pip_resolution_cache (hash, lockfile, expiration) VALUES ($1, $2, now() + ('3 days')::interval) ON CONFLICT (hash) DO UPDATE SET lockfile = $2"
|
||||
},
|
||||
"5061c0d054bf4f028e7fe51a8f9389024c6ae4492755cadac0f7167e5300bda0": {
|
||||
"describe": {
|
||||
"columns": [],
|
||||
@@ -3008,6 +3039,26 @@
|
||||
},
|
||||
"query": "SELECT workspace_id, name, display_name, owners, extra_perms FROM folder WHERE workspace_id = $1 ORDER BY name desc LIMIT $2 OFFSET $3"
|
||||
},
|
||||
"866b26e56fd376368c759db6eee13f92373e308f584903121b4546ad51aef86e": {
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"name": "lockfile",
|
||||
"ordinal": 0,
|
||||
"type_info": "Text"
|
||||
}
|
||||
],
|
||||
"nullable": [
|
||||
false
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text"
|
||||
]
|
||||
}
|
||||
},
|
||||
"query": "SELECT lockfile FROM pip_resolution_cache WHERE hash = $1"
|
||||
},
|
||||
"8876fa929ffb175cd976a2bca1195704aa9fe7215013ae29e49ef15cb201ba57": {
|
||||
"describe": {
|
||||
"columns": [],
|
||||
@@ -3906,76 +3957,6 @@
|
||||
},
|
||||
"query": "SELECT content FROM script WHERE path = $1 AND workspace_id = $2 AND\n created_at = (SELECT max(created_at) FROM script WHERE path = $1 AND archived = false AND workspace_id = $2)"
|
||||
},
|
||||
"a1c41bbeb2d64fa1e7dfd2ed053191a1de5d786ae8c22e225e450865ecac94e9": {
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"name": "id",
|
||||
"ordinal": 0,
|
||||
"type_info": "Uuid"
|
||||
}
|
||||
],
|
||||
"nullable": [
|
||||
false
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Varchar",
|
||||
"Uuid",
|
||||
"Bool",
|
||||
"Uuid",
|
||||
"Varchar",
|
||||
"Varchar",
|
||||
"Timestamptz",
|
||||
"Int8",
|
||||
"Varchar",
|
||||
"Text",
|
||||
"Text",
|
||||
"Jsonb",
|
||||
{
|
||||
"Custom": {
|
||||
"kind": {
|
||||
"Enum": [
|
||||
"script",
|
||||
"preview",
|
||||
"flow",
|
||||
"dependencies",
|
||||
"flowpreview",
|
||||
"script_hub",
|
||||
"identity",
|
||||
"flowdependencies"
|
||||
]
|
||||
},
|
||||
"name": "job_kind"
|
||||
}
|
||||
},
|
||||
"Varchar",
|
||||
"Jsonb",
|
||||
"Jsonb",
|
||||
"Bool",
|
||||
{
|
||||
"Custom": {
|
||||
"kind": {
|
||||
"Enum": [
|
||||
"python3",
|
||||
"deno",
|
||||
"go",
|
||||
"bash"
|
||||
]
|
||||
},
|
||||
"name": "script_lang"
|
||||
}
|
||||
},
|
||||
"Bool",
|
||||
"Text",
|
||||
"Varchar",
|
||||
"Bool",
|
||||
"Uuid"
|
||||
]
|
||||
}
|
||||
},
|
||||
"query": "INSERT INTO queue\n (workspace_id, id, running, parent_job, created_by, permissioned_as, scheduled_for, \n script_hash, script_path, raw_code, raw_lock, args, job_kind, schedule_path, raw_flow, flow_status, is_flow_step, language, started_at, same_worker, pre_run_error, email, visible_to_owner, root_job)\n VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, CASE WHEN $3 THEN now() END, $19, $20, $21, $22, $23) RETURNING id"
|
||||
},
|
||||
"a227548b6604c56bfc15eb780bd8ee72a89dc6701a50f5048e928bd87baa7b9a": {
|
||||
"describe": {
|
||||
"columns": [
|
||||
@@ -4985,6 +4966,26 @@
|
||||
},
|
||||
"query": "\n INSERT INTO resume_job\n (id, resume_id, job, flow, value, approver)\n VALUES ($1, $2, $3, $4, $5, $6)\n "
|
||||
},
|
||||
"c4fb68ce79c1cec23cadc515285578ca18398504cf703091d06191ff68d95399": {
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"name": "database_length!",
|
||||
"ordinal": 0,
|
||||
"type_info": "Int8"
|
||||
}
|
||||
],
|
||||
"nullable": [
|
||||
null
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Text"
|
||||
]
|
||||
}
|
||||
},
|
||||
"query": "SELECT coalesce(COUNT(*), 0) as \"database_length!\" FROM queue WHERE workspace_id = $1"
|
||||
},
|
||||
"c59dd666b9a316c027e8c319b80ccbab3a220d93b64357981bd4a03324dad1d0": {
|
||||
"describe": {
|
||||
"columns": [
|
||||
@@ -6071,6 +6072,76 @@
|
||||
},
|
||||
"query": "SELECT EXISTS(SELECT 1 FROM schedule WHERE path = $1 AND workspace_id = $2)"
|
||||
},
|
||||
"f014bbaecaa09840c6b66d8b58b942fcf9827a7dfd9450f225fddd2b404c4f36": {
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"name": "id",
|
||||
"ordinal": 0,
|
||||
"type_info": "Uuid"
|
||||
}
|
||||
],
|
||||
"nullable": [
|
||||
false
|
||||
],
|
||||
"parameters": {
|
||||
"Left": [
|
||||
"Varchar",
|
||||
"Uuid",
|
||||
"Bool",
|
||||
"Uuid",
|
||||
"Varchar",
|
||||
"Varchar",
|
||||
"Timestamptz",
|
||||
"Int8",
|
||||
"Varchar",
|
||||
"Text",
|
||||
"Text",
|
||||
"Jsonb",
|
||||
{
|
||||
"Custom": {
|
||||
"kind": {
|
||||
"Enum": [
|
||||
"script",
|
||||
"preview",
|
||||
"flow",
|
||||
"dependencies",
|
||||
"flowpreview",
|
||||
"script_hub",
|
||||
"identity",
|
||||
"flowdependencies"
|
||||
]
|
||||
},
|
||||
"name": "job_kind"
|
||||
}
|
||||
},
|
||||
"Varchar",
|
||||
"Jsonb",
|
||||
"Jsonb",
|
||||
"Bool",
|
||||
{
|
||||
"Custom": {
|
||||
"kind": {
|
||||
"Enum": [
|
||||
"python3",
|
||||
"deno",
|
||||
"go",
|
||||
"bash"
|
||||
]
|
||||
},
|
||||
"name": "script_lang"
|
||||
}
|
||||
},
|
||||
"Bool",
|
||||
"Text",
|
||||
"Varchar",
|
||||
"Bool",
|
||||
"Uuid"
|
||||
]
|
||||
}
|
||||
},
|
||||
"query": "INSERT INTO queue\n (workspace_id, id, running, parent_job, created_by, permissioned_as, scheduled_for, \n script_hash, script_path, raw_code, raw_lock, args, job_kind, schedule_path, raw_flow, flow_status, is_flow_step, language, started_at, same_worker, pre_run_error, email, visible_to_owner, root_job)\n VALUES ($1, $2, $3, $4, $5, $6, COALESCE($7, now()), $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, CASE WHEN $3 THEN now() END, $19, $20, $21, $22, $23) RETURNING id"
|
||||
},
|
||||
"f056b5f3e66a764748925f1bfd3180923fde8c7fdf69088d0e4a5555cc049545": {
|
||||
"describe": {
|
||||
"columns": [
|
||||
|
||||
@@ -10,7 +10,7 @@ use std::net::{IpAddr, Ipv4Addr, SocketAddr};
|
||||
|
||||
use git_version::git_version;
|
||||
use sqlx::{Pool, Postgres};
|
||||
use windmill_common::utils::rd_string;
|
||||
use windmill_common::{utils::rd_string, METRICS_ADDR};
|
||||
|
||||
const GIT_VERSION: &str = git_version!(args = ["--tag", "--always"], fallback = "unknown-version");
|
||||
const DEFAULT_NUM_WORKERS: usize = 3;
|
||||
@@ -30,15 +30,7 @@ async fn main() -> anyhow::Result<()> {
|
||||
.and_then(|x| x.parse::<i32>().ok())
|
||||
.unwrap_or(DEFAULT_NUM_WORKERS as i32);
|
||||
|
||||
let metrics_addr: Option<SocketAddr> = std::env::var("METRICS_ADDR")
|
||||
.ok()
|
||||
.map(|s| {
|
||||
s.parse::<bool>()
|
||||
.map(|b| b.then(|| SocketAddr::from(([0, 0, 0, 0], 8001))))
|
||||
.or_else(|_| s.parse::<SocketAddr>().map(Some))
|
||||
})
|
||||
.transpose()?
|
||||
.flatten();
|
||||
let metrics_addr: Option<SocketAddr> = *METRICS_ADDR;
|
||||
|
||||
let server_bind_address: IpAddr = std::env::var("SERVER_BIND_ADDR")
|
||||
.ok()
|
||||
@@ -57,8 +49,40 @@ async fn main() -> anyhow::Result<()> {
|
||||
.and_then(|x| x.parse::<bool>().ok())
|
||||
.unwrap_or(false);
|
||||
|
||||
let rsmq_config = std::env::var("REDIS_URL").ok().map(|x| {
|
||||
let url = x.parse::<url::Url>().unwrap();
|
||||
let mut config = rsmq_async::RsmqOptions { ..Default::default() };
|
||||
|
||||
config.host = url.host_str().expect("redis host required").to_owned();
|
||||
config.password = url.password().map(|s| s.to_owned());
|
||||
config.db = url
|
||||
.path_segments()
|
||||
.and_then(|mut segments| segments.next())
|
||||
.and_then(|segment| segment.parse().ok())
|
||||
.unwrap_or(0);
|
||||
config.ns = url
|
||||
.query_pairs()
|
||||
.find(|s| s.0 == "rsmq_namespace")
|
||||
.map(|s| s.1)
|
||||
.unwrap_or(std::borrow::Cow::Borrowed("rsmq"))
|
||||
.into_owned();
|
||||
config.port = url.port().unwrap_or(6379).to_string();
|
||||
|
||||
config
|
||||
});
|
||||
|
||||
let db = windmill_common::connect_db(server_mode).await?;
|
||||
|
||||
let rsmq = if let Some(config) = rsmq_config {
|
||||
let mut rsmq = rsmq_async::MultiplexedRsmq::new(config).await.unwrap();
|
||||
|
||||
let _ = rsmq_async::RsmqConnection::create_queue(&mut rsmq, "main_queue", None, None, None)
|
||||
.await;
|
||||
Some(rsmq)
|
||||
} else {
|
||||
None
|
||||
};
|
||||
|
||||
if server_mode {
|
||||
windmill_api::migrate_db(&db).await?;
|
||||
}
|
||||
@@ -104,6 +128,8 @@ Windmill Community Edition {GIT_VERSION}
|
||||
"PYTHON_PATH",
|
||||
"DENO_PATH",
|
||||
"GO_PATH",
|
||||
"GOPRIVATE",
|
||||
"NETRC",
|
||||
"PIP_INDEX_URL",
|
||||
"PIP_EXTRA_INDEX_URL",
|
||||
"PIP_TRUSTED_HOST",
|
||||
@@ -120,16 +146,17 @@ Windmill Community Edition {GIT_VERSION}
|
||||
"INCLUDE_HEADERS",
|
||||
"WHITELIST_WORKSPACES",
|
||||
"BLACKLIST_WORKSPACES",
|
||||
"NEW_USER_WEBHOOK",
|
||||
"INSTANCE_EVENTS_WEBHOOK",
|
||||
"CLOUD_HOSTED",
|
||||
]);
|
||||
|
||||
if server_mode || num_workers > 0 {
|
||||
let addr = SocketAddr::from((server_bind_address, port));
|
||||
|
||||
let rsmq2 = rsmq.clone();
|
||||
let server_f = async {
|
||||
if server_mode {
|
||||
windmill_api::run_server(db.clone(), addr, rx.resubscribe()).await?;
|
||||
windmill_api::run_server(db.clone(), rsmq2, addr, rx.resubscribe()).await?;
|
||||
}
|
||||
Ok(()) as anyhow::Result<()>
|
||||
};
|
||||
@@ -141,15 +168,17 @@ Windmill Community Edition {GIT_VERSION}
|
||||
rx.resubscribe(),
|
||||
num_workers,
|
||||
base_internal_url.clone(),
|
||||
rsmq.clone(),
|
||||
)
|
||||
.await?;
|
||||
}
|
||||
Ok(()) as anyhow::Result<()>
|
||||
};
|
||||
|
||||
let rsmq2 = rsmq.clone();
|
||||
let monitor_f = async {
|
||||
if server_mode {
|
||||
monitor_db(&db, rx.resubscribe(), &base_internal_url);
|
||||
monitor_db(&db, rx.resubscribe(), &base_internal_url, rsmq2);
|
||||
}
|
||||
Ok(()) as anyhow::Result<()>
|
||||
};
|
||||
@@ -189,10 +218,11 @@ fn display_config(envs: Vec<&str>) {
|
||||
)
|
||||
}
|
||||
|
||||
pub fn monitor_db(
|
||||
pub fn monitor_db<R: rsmq_async::RsmqConnection + Send + Sync + Clone + 'static>(
|
||||
db: &Pool<Postgres>,
|
||||
rx: tokio::sync::broadcast::Receiver<()>,
|
||||
base_internal_url: &str,
|
||||
rsmq: Option<R>,
|
||||
) {
|
||||
let db1 = db.clone();
|
||||
let db2 = db.clone();
|
||||
@@ -200,16 +230,17 @@ pub fn monitor_db(
|
||||
let rx2 = rx.resubscribe();
|
||||
let base_internal_url = base_internal_url.to_string();
|
||||
tokio::spawn(async move {
|
||||
windmill_worker::handle_zombie_jobs_periodically(&db1, rx, &base_internal_url).await
|
||||
windmill_worker::handle_zombie_jobs_periodically(&db1, rx, &base_internal_url, rsmq).await
|
||||
});
|
||||
tokio::spawn(async move { windmill_api::delete_expired_items_perdiodically(&db2, rx2).await });
|
||||
}
|
||||
|
||||
pub async fn run_workers(
|
||||
pub async fn run_workers<R: rsmq_async::RsmqConnection + Send + Sync + Clone + 'static>(
|
||||
db: Pool<Postgres>,
|
||||
rx: tokio::sync::broadcast::Receiver<()>,
|
||||
num_workers: i32,
|
||||
base_internal_url: String,
|
||||
rsmq: Option<R>,
|
||||
) -> anyhow::Result<()> {
|
||||
let license_key = std::env::var("LICENSE_KEY").ok();
|
||||
#[cfg(feature = "enterprise")]
|
||||
@@ -239,6 +270,7 @@ pub async fn run_workers(
|
||||
let ip = ip.clone();
|
||||
let rx = rx.resubscribe();
|
||||
let base_internal_url = base_internal_url.clone();
|
||||
let rsmq2 = rsmq.clone();
|
||||
handles.push(tokio::spawn(monitor.instrument(async move {
|
||||
tracing::info!(worker = %worker_name, "starting worker");
|
||||
windmill_worker::run_worker(
|
||||
@@ -249,6 +281,7 @@ pub async fn run_workers(
|
||||
&ip,
|
||||
rx,
|
||||
&base_internal_url,
|
||||
rsmq2,
|
||||
)
|
||||
.await
|
||||
})));
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
use futures::{stream, Stream};
|
||||
use serde::Deserialize;
|
||||
use serde_json::json;
|
||||
use sqlx::{postgres::PgListener, types::Uuid, Pool, Postgres, Transaction};
|
||||
use windmill_api::jobs::{CompletedJob, Job};
|
||||
@@ -87,7 +88,7 @@ impl ApiServer {
|
||||
let addr = sock.local_addr().unwrap();
|
||||
drop(sock);
|
||||
|
||||
let task = tokio::task::spawn(windmill_api::run_server(db.clone(), addr, rx));
|
||||
let task = tokio::task::spawn(windmill_api::run_server(db.clone(), None, addr, rx));
|
||||
|
||||
return Self { addr, tx, task };
|
||||
}
|
||||
@@ -252,9 +253,7 @@ mod suspend_resume {
|
||||
let second = completed.next().await.unwrap();
|
||||
// print_job(second, &db).await;
|
||||
|
||||
let tx = db.begin().await.unwrap();
|
||||
let (tx, token) = windmill_worker::create_token_for_owner(tx, "test-workspace", "u/test-user", "", 100, "").await.unwrap();
|
||||
tx.commit().await.unwrap();
|
||||
let token = windmill_worker::create_token_for_owner(&db, "test-workspace", "u/test-user", "", 100, "").await.unwrap();
|
||||
let secret = reqwest::get(format!(
|
||||
"http://localhost:{port}/api/w/test-workspace/jobs/job_signature/{second}/0?token={token}&approver=ruben"
|
||||
))
|
||||
@@ -355,9 +354,7 @@ mod suspend_resume {
|
||||
/* ... and send a request resume it. */
|
||||
let second = completed.next().await.unwrap();
|
||||
|
||||
let tx = db.begin().await.unwrap();
|
||||
let (tx, token) = windmill_worker::create_token_for_owner(tx, "test-workspace", "u/test-user", "", 100, "").await.unwrap();
|
||||
tx.commit().await.unwrap();
|
||||
let token = windmill_worker::create_token_for_owner(&db, "test-workspace", "u/test-user", "", 100, "").await.unwrap();
|
||||
let secret = reqwest::get(format!(
|
||||
"http://localhost:{port}/api/w/test-workspace/jobs/job_signature/{second}/0?token={token}"
|
||||
))
|
||||
@@ -659,7 +656,7 @@ def main(error, port):
|
||||
},
|
||||
}))
|
||||
.unwrap();
|
||||
let (attempts, responses) = [
|
||||
let (_attempts, responses) = [
|
||||
/* fail the first step twice */
|
||||
(0x00, None),
|
||||
(0x00, None),
|
||||
@@ -681,14 +678,20 @@ def main(error, port):
|
||||
_ => panic!("expected failure module"),
|
||||
}
|
||||
|
||||
assert_eq!(server.close().await, attempts);
|
||||
println!("result: {:#?}", result);
|
||||
assert_eq!(
|
||||
result,
|
||||
json!({
|
||||
"recv": 42,
|
||||
"from failure module": {"error": {"name": "IndexError", "stack": " File \"/tmp/tmp/main/step_0.py\", line 5, in main\n return sock.recv(1)[0]\n", "message": "index out of range"}},
|
||||
})
|
||||
result
|
||||
.get("from failure module")
|
||||
.unwrap()
|
||||
.get("error")
|
||||
.unwrap()
|
||||
.get("name")
|
||||
.unwrap()
|
||||
.clone(),
|
||||
json!("IndexError")
|
||||
);
|
||||
|
||||
assert_eq!(result.get("recv").unwrap().clone(), json!(42));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -824,9 +827,8 @@ impl RunJob {
|
||||
|
||||
async fn push(self, db: &Pool<Postgres>) -> Uuid {
|
||||
let RunJob { payload, args } = self;
|
||||
let tx = db.begin().await.unwrap();
|
||||
let (uuid, tx) = windmill_queue::push(
|
||||
tx,
|
||||
let (uuid, tx) = windmill_queue::push::<rsmq_async::MultiplexedRsmq>(
|
||||
(None, db.begin().await.unwrap()).into(),
|
||||
"test-workspace",
|
||||
payload,
|
||||
args,
|
||||
@@ -844,8 +846,7 @@ impl RunJob {
|
||||
)
|
||||
.await
|
||||
.expect("push has to succeed");
|
||||
|
||||
tx.commit().await.expect("push has to commit");
|
||||
tx.commit().await.unwrap();
|
||||
|
||||
uuid
|
||||
}
|
||||
@@ -915,7 +916,7 @@ fn spawn_test_worker(
|
||||
let ip: &str = Default::default();
|
||||
let future = async move {
|
||||
let base_internal_url = format!("http://localhost:{}", port);
|
||||
windmill_worker::run_worker(
|
||||
windmill_worker::run_worker::<rsmq_async::MultiplexedRsmq>(
|
||||
&db,
|
||||
worker_instance,
|
||||
worker_name,
|
||||
@@ -923,6 +924,7 @@ fn spawn_test_worker(
|
||||
ip,
|
||||
rx,
|
||||
&base_internal_url,
|
||||
None,
|
||||
)
|
||||
.await
|
||||
};
|
||||
@@ -1518,6 +1520,8 @@ async fn test_go_job(db: Pool<Postgres>) {
|
||||
let port = server.addr.port();
|
||||
|
||||
let content = r#"
|
||||
package inner
|
||||
|
||||
import "fmt"
|
||||
|
||||
func main(derp string) (string, error) {
|
||||
@@ -2024,6 +2028,16 @@ async fn test_branchall_simple(db: Pool<Postgres>) {
|
||||
assert_eq!(result, serde_json::json!([[1, 2], [1, 3]]));
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
struct ErrorResult {
|
||||
error: NamedError,
|
||||
}
|
||||
|
||||
#[derive(Deserialize)]
|
||||
struct NamedError {
|
||||
name: String,
|
||||
}
|
||||
|
||||
#[sqlx::test(fixtures("base"))]
|
||||
async fn test_branchall_skip_failure(db: Pool<Postgres>) {
|
||||
initialize_tracing().await;
|
||||
@@ -2059,8 +2073,11 @@ async fn test_branchall_skip_failure(db: Pool<Postgres>) {
|
||||
.unwrap();
|
||||
|
||||
assert_eq!(
|
||||
result,
|
||||
serde_json::json!([{"error": {"name": "Error", "stack": "Error: failure\n at main (file:///tmp/main.ts:1:31)\n at run (file:///tmp/wrapper.ts:9:26)\n at file:///tmp/wrapper.ts:14:1", "message": "failure"}}, [1,3]])
|
||||
serde_json::from_value::<ErrorResult>(result.get(0).unwrap().clone())
|
||||
.unwrap()
|
||||
.error
|
||||
.name,
|
||||
"Error"
|
||||
);
|
||||
|
||||
let flow: FlowValue = serde_json::from_value(json!({
|
||||
@@ -2093,8 +2110,11 @@ async fn test_branchall_skip_failure(db: Pool<Postgres>) {
|
||||
.unwrap();
|
||||
|
||||
assert_eq!(
|
||||
result,
|
||||
serde_json::json!([ {"error": {"name": "Error", "stack": "Error: failure\n at main (file:///tmp/main.ts:1:31)\n at run (file:///tmp/wrapper.ts:9:26)\n at file:///tmp/wrapper.ts:14:1", "message": "failure"}}, [1, 2]])
|
||||
serde_json::from_value::<ErrorResult>(result.get(0).unwrap().clone())
|
||||
.unwrap()
|
||||
.error
|
||||
.name,
|
||||
"Error"
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
@@ -70,4 +70,5 @@ async-stripe = { workspace = true, optional = true }
|
||||
lazy_static.workspace = true
|
||||
prometheus.workspace = true
|
||||
async_zip.workspace = true
|
||||
rsmq_async.workspace = true
|
||||
regex.workspace = true
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
openapi: "3.0.3"
|
||||
|
||||
info:
|
||||
version: 1.86.0
|
||||
version: 1.87.0
|
||||
title: Windmill API
|
||||
|
||||
contact:
|
||||
@@ -2782,6 +2782,27 @@ paths:
|
||||
schema:
|
||||
type: string
|
||||
|
||||
/w/{workspace}/flows/input_history/p/{path}:
|
||||
get:
|
||||
summary: list inputs for previous completed flow jobs
|
||||
operationId: getFlowInputHistoryByPath
|
||||
tags:
|
||||
- flow
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/WorkspaceId"
|
||||
- $ref: "#/components/parameters/ScriptPath"
|
||||
- $ref: "#/components/parameters/Page"
|
||||
- $ref: "#/components/parameters/PerPage"
|
||||
responses:
|
||||
"200":
|
||||
description: input history for completed jobs with this flow path
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: array
|
||||
items:
|
||||
$ref: "#/components/schemas/Input"
|
||||
|
||||
/w/{workspace}/apps/list:
|
||||
get:
|
||||
summary: list all available apps
|
||||
@@ -3322,23 +3343,6 @@ paths:
|
||||
schema:
|
||||
$ref: "#/components/schemas/Job"
|
||||
|
||||
# /w/{workspace}/jobs/flow/current_state/{id}:
|
||||
# get:
|
||||
# summary: get flow current step state
|
||||
# operationId: getJob
|
||||
# tags:
|
||||
# - job
|
||||
# parameters:
|
||||
# - $ref: "#/components/parameters/WorkspaceId"
|
||||
# - $ref: "#/components/parameters/JobId"
|
||||
# responses:
|
||||
# "200":
|
||||
# description: state details
|
||||
# content:
|
||||
# application/json:
|
||||
# schema:
|
||||
# type: string
|
||||
|
||||
/w/{workspace}/jobs_u/getupdate/{id}:
|
||||
get:
|
||||
summary: get job updates
|
||||
@@ -4558,6 +4562,118 @@ paths:
|
||||
"200":
|
||||
description: unstar item
|
||||
|
||||
/w/{workspace}/inputs/history:
|
||||
get:
|
||||
summary: List Inputs used in previously completed jobs
|
||||
operationId: getInputHistory
|
||||
tags:
|
||||
- input
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/WorkspaceId"
|
||||
- $ref: "#/components/parameters/RunnableId"
|
||||
- $ref: "#/components/parameters/RunnableType"
|
||||
- $ref: "#/components/parameters/Page"
|
||||
- $ref: "#/components/parameters/PerPage"
|
||||
responses:
|
||||
"200":
|
||||
description: Input history for completed jobs
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: array
|
||||
items:
|
||||
$ref: "#/components/schemas/Input"
|
||||
|
||||
/w/{workspace}/inputs/list:
|
||||
get:
|
||||
summary: List saved Inputs for a Runnable
|
||||
operationId: listInputs
|
||||
tags:
|
||||
- input
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/WorkspaceId"
|
||||
- $ref: "#/components/parameters/RunnableId"
|
||||
- $ref: "#/components/parameters/RunnableType"
|
||||
- $ref: "#/components/parameters/Page"
|
||||
- $ref: "#/components/parameters/PerPage"
|
||||
responses:
|
||||
"200":
|
||||
description: Saved Inputs for a Runnable
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: array
|
||||
items:
|
||||
$ref: "#/components/schemas/Input"
|
||||
|
||||
/w/{workspace}/inputs/create:
|
||||
post:
|
||||
summary: Create an Input for future use in a script or flow
|
||||
operationId: createInput
|
||||
tags:
|
||||
- input
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/WorkspaceId"
|
||||
- $ref: "#/components/parameters/RunnableId"
|
||||
- $ref: "#/components/parameters/RunnableType"
|
||||
requestBody:
|
||||
description: Input
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/CreateInput"
|
||||
responses:
|
||||
"201":
|
||||
description: Input created
|
||||
content:
|
||||
text/plain:
|
||||
schema:
|
||||
type: string
|
||||
format: uuid
|
||||
|
||||
/w/{workspace}/inputs/update:
|
||||
post:
|
||||
summary: Update an Input
|
||||
operationId: updateInput
|
||||
tags:
|
||||
- input
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/WorkspaceId"
|
||||
requestBody:
|
||||
description: UpdateInput
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/UpdateInput"
|
||||
responses:
|
||||
"201":
|
||||
description: Input updated
|
||||
content:
|
||||
text/plain:
|
||||
schema:
|
||||
type: string
|
||||
format: uuid
|
||||
|
||||
/w/{workspace}/inputs/delete/{input}:
|
||||
post:
|
||||
summary: Delete a Saved Input
|
||||
operationId: deleteInput
|
||||
tags:
|
||||
- input
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/WorkspaceId"
|
||||
- $ref: "#/components/parameters/InputId"
|
||||
responses:
|
||||
"200":
|
||||
description: Input deleted
|
||||
content:
|
||||
text/plain:
|
||||
schema:
|
||||
type: string
|
||||
format: uuid
|
||||
|
||||
components:
|
||||
securitySchemes:
|
||||
bearerAuth:
|
||||
@@ -4567,6 +4683,7 @@ components:
|
||||
type: apiKey
|
||||
in: cookie
|
||||
name: token
|
||||
|
||||
parameters:
|
||||
WorkspaceId:
|
||||
name: workspace
|
||||
@@ -4685,7 +4802,6 @@ components:
|
||||
in: query
|
||||
schema:
|
||||
type: string
|
||||
|
||||
ScriptStartPath:
|
||||
name: script_path_start
|
||||
description: mask to filter matching starting path
|
||||
@@ -4742,7 +4858,6 @@ components:
|
||||
in: query
|
||||
schema:
|
||||
type: string
|
||||
|
||||
ResultFilter:
|
||||
name: result
|
||||
description: filter on jobs containing those result as a json subset (@> in postgres)
|
||||
@@ -4802,9 +4917,26 @@ components:
|
||||
# type: string
|
||||
# enum: ["preview", "script", "dependencies"]
|
||||
# explode: false
|
||||
RunnableId:
|
||||
name: runnable_id
|
||||
in: query
|
||||
schema:
|
||||
type: string
|
||||
RunnableType:
|
||||
name: runnable_type
|
||||
in: query
|
||||
schema:
|
||||
$ref: "#/components/schemas/RunnableType"
|
||||
InputId:
|
||||
name: input
|
||||
in: path
|
||||
required: true
|
||||
schema:
|
||||
type: string
|
||||
|
||||
schemas:
|
||||
$ref: "../../openflow.openapi.yaml#/components/schemas"
|
||||
|
||||
Script:
|
||||
type: object
|
||||
properties:
|
||||
@@ -4875,6 +5007,60 @@ components:
|
||||
type: object
|
||||
additionalProperties: {}
|
||||
|
||||
Input:
|
||||
type: object
|
||||
properties:
|
||||
id:
|
||||
type: string
|
||||
name:
|
||||
type: string
|
||||
args:
|
||||
type: object
|
||||
created_by:
|
||||
type: string
|
||||
created_at:
|
||||
type: string
|
||||
format: date-time
|
||||
is_public:
|
||||
type: boolean
|
||||
required:
|
||||
- id
|
||||
- name
|
||||
- args
|
||||
- created_by
|
||||
- created_at
|
||||
- is_public
|
||||
|
||||
CreateInput:
|
||||
type: object
|
||||
properties:
|
||||
name:
|
||||
type: string
|
||||
args:
|
||||
type: object
|
||||
required:
|
||||
- name
|
||||
- args
|
||||
- created_by
|
||||
|
||||
UpdateInput:
|
||||
type: object
|
||||
properties:
|
||||
id:
|
||||
type: string
|
||||
name:
|
||||
type: string
|
||||
is_public:
|
||||
type: boolean
|
||||
required:
|
||||
- id
|
||||
- name
|
||||
- is_public
|
||||
|
||||
RunnableType:
|
||||
type: string
|
||||
enum: ["ScriptHash", "ScriptPath", "FlowPath"]
|
||||
|
||||
QueuedJob:
|
||||
type: object
|
||||
properties:
|
||||
@@ -5896,6 +6082,7 @@ components:
|
||||
- extra_perms
|
||||
- edited_at
|
||||
- execution_mode
|
||||
|
||||
AppWithLastVersion:
|
||||
type: object
|
||||
properties:
|
||||
|
||||
@@ -37,7 +37,7 @@ use windmill_common::{
|
||||
http_get_from_hub, list_elems_from_hub, not_found_if_none, paginate, Pagination, StripPath,
|
||||
},
|
||||
};
|
||||
use windmill_queue::{push, JobPayload, RawCode};
|
||||
use windmill_queue::{push, JobPayload, QueueTransaction, RawCode};
|
||||
|
||||
pub fn workspaced_service() -> Router {
|
||||
Router::new()
|
||||
@@ -565,6 +565,7 @@ fn digest(code: &str) -> String {
|
||||
async fn execute_component(
|
||||
OptAuthed(opt_authed): OptAuthed,
|
||||
Extension(db): Extension<DB>,
|
||||
Extension(rsmq): Extension<Option<rsmq_async::MultiplexedRsmq>>,
|
||||
Path((w_id, path)): Path<(String, StripPath)>,
|
||||
Json(payload): Json<ExecuteApp>,
|
||||
) -> Result<String> {
|
||||
@@ -583,7 +584,7 @@ async fn execute_component(
|
||||
};
|
||||
|
||||
let path = path.to_path();
|
||||
let mut tx = db.begin().await?;
|
||||
let mut tx: QueueTransaction<'_, _> = (rsmq, db.begin().await?).into();
|
||||
|
||||
let policy = if let Some(static_fields) = payload.clone().force_viewer_static_fields {
|
||||
let mut hm = HashMap::new();
|
||||
@@ -652,8 +653,12 @@ async fn execute_component(
|
||||
}
|
||||
ExecuteApp { args, raw_code: None, path: Some(path), .. } => {
|
||||
let payload = if path.starts_with("script/") {
|
||||
script_path_to_payload(path.strip_prefix("script/").unwrap(), &mut tx, &w_id)
|
||||
.await?
|
||||
script_path_to_payload(
|
||||
path.strip_prefix("script/").unwrap(),
|
||||
tx.transaction_mut(),
|
||||
&w_id,
|
||||
)
|
||||
.await?
|
||||
} else if path.starts_with("flow/") {
|
||||
JobPayload::Flow(path.strip_prefix("flow/").unwrap().to_string())
|
||||
} else {
|
||||
@@ -686,8 +691,8 @@ async fn execute_component(
|
||||
true,
|
||||
)
|
||||
.await?;
|
||||
|
||||
tx.commit().await?;
|
||||
|
||||
Ok(uuid.to_string())
|
||||
}
|
||||
|
||||
|
||||
@@ -6,14 +6,20 @@
|
||||
* LICENSE-AGPL for a copy of the license.
|
||||
*/
|
||||
|
||||
use hyper::StatusCode;
|
||||
use sql_builder::prelude::*;
|
||||
|
||||
use crate::{
|
||||
db::{UserDB, DB},
|
||||
schedule::clear_schedule,
|
||||
users::{maybe_refresh_folders, require_owner_of_path, Authed},
|
||||
webhook_util::{WebhookMessage, WebhookShared},
|
||||
HTTP_CLIENT,
|
||||
};
|
||||
use axum::{
|
||||
extract::{Extension, Path, Query},
|
||||
routing::{delete, get, post},
|
||||
Json, Router,
|
||||
};
|
||||
use hyper::StatusCode;
|
||||
use sql_builder::prelude::*;
|
||||
use sql_builder::SqlBuilder;
|
||||
use sqlx::{Postgres, Transaction};
|
||||
use windmill_audit::{audit_log, ActionKind};
|
||||
@@ -25,15 +31,7 @@ use windmill_common::{
|
||||
http_get_from_hub, list_elems_from_hub, not_found_if_none, paginate, Pagination, StripPath,
|
||||
},
|
||||
};
|
||||
use windmill_queue::{push, schedule::push_scheduled_job, JobPayload};
|
||||
|
||||
use crate::{
|
||||
db::{UserDB, DB},
|
||||
schedule::clear_schedule,
|
||||
users::{maybe_refresh_folders, require_owner_of_path, Authed},
|
||||
webhook_util::{WebhookMessage, WebhookShared},
|
||||
HTTP_CLIENT,
|
||||
};
|
||||
use windmill_queue::{push, schedule::push_scheduled_job, JobPayload, QueueTransaction};
|
||||
|
||||
pub fn workspaced_service() -> Router {
|
||||
Router::new()
|
||||
@@ -180,6 +178,7 @@ async fn create_flow(
|
||||
authed: Authed,
|
||||
Extension(db): Extension<DB>,
|
||||
Extension(user_db): Extension<UserDB>,
|
||||
Extension(rsmq): Extension<Option<rsmq_async::MultiplexedRsmq>>,
|
||||
Extension(webhook): Extension<WebhookShared>,
|
||||
Path(w_id): Path<String>,
|
||||
Json(nf): Json<NewFlow>,
|
||||
@@ -187,10 +186,10 @@ async fn create_flow(
|
||||
// cron::Schedule::from_str(&ns.schedule).map_err(|e| error::Error::BadRequest(e.to_string()))?;
|
||||
let authed = maybe_refresh_folders(&nf.path, &w_id, authed, &db).await;
|
||||
|
||||
let mut tx = user_db.clone().begin(&authed).await?;
|
||||
let mut tx: QueueTransaction<'_, _> = (rsmq, user_db.clone().begin(&authed).await?).into();
|
||||
|
||||
check_path_conflict(&mut tx, &w_id, &nf.path).await?;
|
||||
check_schedule_conflict(&mut tx, &w_id, &nf.path).await?;
|
||||
check_path_conflict(tx.transaction_mut(), &w_id, &nf.path).await?;
|
||||
check_schedule_conflict(tx.transaction_mut(), &w_id, &nf.path).await?;
|
||||
|
||||
sqlx::query!(
|
||||
"INSERT INTO flow (workspace_id, path, summary, description, value, edited_by, edited_at, \
|
||||
@@ -222,13 +221,11 @@ async fn create_flow(
|
||||
)
|
||||
.await?;
|
||||
|
||||
tx.commit().await?;
|
||||
webhook.send_message(
|
||||
w_id.clone(),
|
||||
WebhookMessage::CreateFlow { workspace: w_id.clone(), path: nf.path.clone() },
|
||||
);
|
||||
|
||||
let tx = user_db.begin(&authed).await?;
|
||||
let (dependency_job_uuid, mut tx) = push(
|
||||
tx,
|
||||
&w_id,
|
||||
@@ -247,6 +244,7 @@ async fn create_flow(
|
||||
true,
|
||||
)
|
||||
.await?;
|
||||
|
||||
sqlx::query!(
|
||||
"UPDATE flow SET dependency_job = $1 WHERE path = $2 AND workspace_id = $3",
|
||||
dependency_job_uuid,
|
||||
@@ -286,6 +284,7 @@ async fn check_schedule_conflict<'c>(
|
||||
async fn update_flow(
|
||||
authed: Authed,
|
||||
Extension(user_db): Extension<UserDB>,
|
||||
Extension(rsmq): Extension<Option<rsmq_async::MultiplexedRsmq>>,
|
||||
Extension(db): Extension<DB>,
|
||||
Extension(webhook): Extension<WebhookShared>,
|
||||
Path((w_id, flow_path)): Path<(String, StripPath)>,
|
||||
@@ -294,9 +293,9 @@ async fn update_flow(
|
||||
let flow_path = flow_path.to_path();
|
||||
let authed = maybe_refresh_folders(&flow_path, &w_id, authed, &db).await;
|
||||
|
||||
let mut tx = user_db.clone().begin(&authed).await?;
|
||||
let mut tx: QueueTransaction<'_, _> = (rsmq, user_db.clone().begin(&authed).await?).into();
|
||||
|
||||
check_schedule_conflict(&mut tx, &w_id, flow_path).await?;
|
||||
check_schedule_conflict(tx.transaction_mut(), &w_id, flow_path).await?;
|
||||
|
||||
let schema = nf.schema.map(|x| x.0);
|
||||
let old_dep_job = sqlx::query_scalar!(
|
||||
@@ -323,13 +322,13 @@ async fn update_flow(
|
||||
.await?;
|
||||
|
||||
if nf.path != flow_path {
|
||||
check_schedule_conflict(&mut tx, &w_id, &nf.path).await?;
|
||||
check_schedule_conflict(tx.transaction_mut(), &w_id, &nf.path).await?;
|
||||
|
||||
if !authed.is_admin {
|
||||
require_owner_of_path(&w_id, &authed.username, &authed.groups, &flow_path, &db).await?;
|
||||
}
|
||||
|
||||
let mut schedulables = sqlx::query_as!(
|
||||
let mut schedulables: Vec<Schedule> = sqlx::query_as!(
|
||||
Schedule,
|
||||
"UPDATE schedule SET script_path = $1 WHERE script_path = $2 AND path != $2 AND workspace_id = $3 AND is_flow IS true RETURNING *",
|
||||
nf.path,
|
||||
@@ -352,32 +351,32 @@ async fn update_flow(
|
||||
schedulables.push(schedule);
|
||||
}
|
||||
|
||||
for schedule in schedulables {
|
||||
clear_schedule(&mut tx, flow_path, true).await?;
|
||||
for schedule in schedulables.into_iter() {
|
||||
// TODO: Why is this in the loop in the first place? Seems like it's just doing nothing after the first iteration? Should this use schedule.path?
|
||||
clear_schedule(tx.transaction_mut(), flow_path, true).await?;
|
||||
|
||||
if schedule.enabled {
|
||||
tx = push_scheduled_job(tx, schedule).await?;
|
||||
}
|
||||
}
|
||||
|
||||
audit_log(
|
||||
&mut tx,
|
||||
&authed.username,
|
||||
"flows.update",
|
||||
ActionKind::Create,
|
||||
&w_id,
|
||||
Some(&nf.path.to_string()),
|
||||
Some(
|
||||
[Some(("flow", nf.path.as_str()))]
|
||||
.into_iter()
|
||||
.flatten()
|
||||
.collect(),
|
||||
),
|
||||
)
|
||||
.await?;
|
||||
}
|
||||
|
||||
audit_log(
|
||||
&mut tx,
|
||||
&authed.username,
|
||||
"flows.update",
|
||||
ActionKind::Create,
|
||||
&w_id,
|
||||
Some(&nf.path.to_string()),
|
||||
Some(
|
||||
[Some(("flow", nf.path.as_str()))]
|
||||
.into_iter()
|
||||
.flatten()
|
||||
.collect(),
|
||||
),
|
||||
)
|
||||
.await?;
|
||||
|
||||
tx.commit().await?;
|
||||
webhook.send_message(
|
||||
w_id.clone(),
|
||||
WebhookMessage::UpdateFlow {
|
||||
@@ -387,7 +386,6 @@ async fn update_flow(
|
||||
},
|
||||
);
|
||||
|
||||
let tx = user_db.begin(&authed).await?;
|
||||
let (dependency_job_uuid, mut tx) = push(
|
||||
tx,
|
||||
&w_id,
|
||||
|
||||
282
backend/windmill-api/src/inputs.rs
Normal file
282
backend/windmill-api/src/inputs.rs
Normal file
@@ -0,0 +1,282 @@
|
||||
/*
|
||||
* Author: Ruben Fiszel
|
||||
* Copyright: Windmill Labs, Inc 2022
|
||||
* This file and its contents are licensed under the AGPLv3 License.
|
||||
* Please see the included NOTICE for copyright information and
|
||||
* LICENSE-AGPL for a copy of the license.
|
||||
*/
|
||||
|
||||
use crate::{db::UserDB, jobs::CompletedJob, users::Authed};
|
||||
use axum::{
|
||||
extract::{Path, Query},
|
||||
routing::{get, post},
|
||||
Extension, Json, Router,
|
||||
};
|
||||
use chrono::{DateTime, Utc};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use serde_json::Value;
|
||||
use sqlx::types::Uuid;
|
||||
use std::{
|
||||
fmt::{Display, Formatter},
|
||||
vec,
|
||||
};
|
||||
use windmill_common::{
|
||||
error::JsonResult,
|
||||
scripts::to_i64,
|
||||
utils::{paginate, Pagination},
|
||||
};
|
||||
use windmill_queue::JobKind;
|
||||
|
||||
pub fn workspaced_service() -> Router {
|
||||
Router::new()
|
||||
.route("/history", get(get_input_history))
|
||||
.route("/list", get(list_saved_inputs))
|
||||
.route("/create", post(create_input))
|
||||
.route("/update", post(update_input))
|
||||
.route("/delete/:id", post(delete_input))
|
||||
}
|
||||
|
||||
#[derive(Debug, sqlx::FromRow, Serialize, Deserialize)]
|
||||
pub struct InputRow {
|
||||
pub id: Uuid,
|
||||
pub workspace_id: String,
|
||||
pub runnable_id: String,
|
||||
pub runnable_type: RunnableType,
|
||||
pub name: String,
|
||||
pub args: Value,
|
||||
pub created_at: DateTime<Utc>,
|
||||
pub created_by: String,
|
||||
pub is_public: bool,
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize, Deserialize, sqlx::Type)]
|
||||
#[sqlx(type_name = "runnable_type")]
|
||||
pub enum RunnableType {
|
||||
ScriptHash,
|
||||
ScriptPath,
|
||||
FlowPath,
|
||||
}
|
||||
|
||||
impl Display for RunnableType {
|
||||
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
|
||||
match self {
|
||||
RunnableType::ScriptHash => write!(f, "ScriptHash"),
|
||||
RunnableType::ScriptPath => write!(f, "ScriptPath"),
|
||||
RunnableType::FlowPath => write!(f, "FlowPath"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl RunnableType {
|
||||
fn job_kind(&self) -> JobKind {
|
||||
match self {
|
||||
RunnableType::ScriptHash => JobKind::Script,
|
||||
RunnableType::ScriptPath => JobKind::Script,
|
||||
RunnableType::FlowPath => JobKind::Flow,
|
||||
}
|
||||
}
|
||||
|
||||
fn column_name(&self) -> &'static str {
|
||||
match self {
|
||||
RunnableType::ScriptHash => "script_hash",
|
||||
RunnableType::ScriptPath => "script_path",
|
||||
RunnableType::FlowPath => "script_path",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize, Deserialize)]
|
||||
pub struct RunnableParams {
|
||||
pub runnable_id: String,
|
||||
pub runnable_type: RunnableType,
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize, Deserialize)]
|
||||
pub struct Input {
|
||||
id: Uuid,
|
||||
name: String,
|
||||
created_at: chrono::DateTime<chrono::Utc>,
|
||||
args: serde_json::Value,
|
||||
created_by: String,
|
||||
is_public: bool,
|
||||
}
|
||||
|
||||
async fn get_input_history(
|
||||
authed: Authed,
|
||||
Extension(user_db): Extension<UserDB>,
|
||||
Path(w_id): Path<String>,
|
||||
Query(pagination): Query<Pagination>,
|
||||
Query(r): Query<RunnableParams>,
|
||||
) -> JsonResult<Vec<Input>> {
|
||||
let (per_page, offset) = paginate(pagination);
|
||||
|
||||
let mut tx = user_db.begin(&authed).await?;
|
||||
|
||||
let sql = &format!(
|
||||
"select distinct on (args) * from completed_job \
|
||||
where {} = $1 and job_kind = $2 and workspace_id = $3 \
|
||||
order by args, started_at desc limit $4 offset $5",
|
||||
r.runnable_type.column_name()
|
||||
);
|
||||
|
||||
let query = sqlx::query_as::<_, CompletedJob>(sql);
|
||||
|
||||
let query = match r.runnable_type {
|
||||
RunnableType::ScriptHash => query.bind(to_i64(&r.runnable_id)?),
|
||||
_ => query.bind(&r.runnable_id),
|
||||
};
|
||||
|
||||
let rows = query
|
||||
.bind(r.runnable_type.job_kind())
|
||||
.bind(&w_id)
|
||||
.bind(per_page as i32)
|
||||
.bind(offset as i32)
|
||||
.fetch_all(&mut tx)
|
||||
.await?;
|
||||
|
||||
tx.commit().await?;
|
||||
|
||||
let mut inputs = vec![];
|
||||
|
||||
for row in rows {
|
||||
inputs.push(Input {
|
||||
id: row.id,
|
||||
name: format!(
|
||||
"{} {}",
|
||||
row.created_at.format("%H:%M %-d/%-m"),
|
||||
row.created_by
|
||||
),
|
||||
created_at: row.created_at,
|
||||
args: row.args.unwrap_or(serde_json::json!({})),
|
||||
created_by: row.created_by,
|
||||
is_public: true,
|
||||
});
|
||||
}
|
||||
|
||||
Ok(Json(inputs))
|
||||
}
|
||||
|
||||
async fn list_saved_inputs(
|
||||
authed: Authed,
|
||||
Extension(user_db): Extension<UserDB>,
|
||||
Path(w_id): Path<String>,
|
||||
Query(pagination): Query<Pagination>,
|
||||
Query(r): Query<RunnableParams>,
|
||||
) -> JsonResult<Vec<Input>> {
|
||||
let (per_page, offset) = paginate(pagination);
|
||||
|
||||
let mut tx = user_db.begin(&authed).await?;
|
||||
|
||||
let rows = sqlx::query_as::<_, InputRow>(
|
||||
"select * from input \
|
||||
where runnable_id = $1 and runnable_type = $2 and workspace_id = $3 \
|
||||
and is_public IS true OR created_by = $4 \
|
||||
order by created_at desc limit $5 offset $6",
|
||||
)
|
||||
.bind(&r.runnable_id)
|
||||
.bind(&r.runnable_type)
|
||||
.bind(&w_id)
|
||||
.bind(&authed.username)
|
||||
.bind(per_page as i32)
|
||||
.bind(offset as i32)
|
||||
.fetch_all(&mut tx)
|
||||
.await?;
|
||||
|
||||
tx.commit().await?;
|
||||
|
||||
let mut inputs: Vec<Input> = Vec::new();
|
||||
|
||||
for row in rows {
|
||||
inputs.push(Input {
|
||||
id: row.id,
|
||||
name: row.name,
|
||||
args: row.args,
|
||||
created_by: row.created_by,
|
||||
created_at: row.created_at,
|
||||
is_public: row.is_public,
|
||||
})
|
||||
}
|
||||
|
||||
Ok(Json(inputs))
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize, Deserialize)]
|
||||
pub struct CreateInput {
|
||||
name: String,
|
||||
args: serde_json::Value,
|
||||
}
|
||||
|
||||
async fn create_input(
|
||||
authed: Authed,
|
||||
Extension(user_db): Extension<UserDB>,
|
||||
Path(w_id): Path<String>,
|
||||
Query(r): Query<RunnableParams>,
|
||||
Json(input): Json<CreateInput>,
|
||||
) -> JsonResult<String> {
|
||||
let mut tx = user_db.begin(&authed).await?;
|
||||
|
||||
let id = Uuid::new_v4();
|
||||
|
||||
sqlx::query(
|
||||
"INSERT INTO input (id, workspace_id, runnable_id, runnable_type, name, args, created_by) VALUES ($1, $2, $3, $4, $5, $6, $7)",
|
||||
)
|
||||
.bind(&id)
|
||||
.bind(&w_id)
|
||||
.bind(&r.runnable_id)
|
||||
.bind(&r.runnable_type)
|
||||
.bind(&input.name)
|
||||
.bind(&input.args)
|
||||
.bind(&authed.username)
|
||||
.execute(&mut tx)
|
||||
.await?;
|
||||
|
||||
tx.commit().await?;
|
||||
|
||||
Ok(Json(id.to_string()))
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize, Deserialize)]
|
||||
pub struct UpdateInput {
|
||||
id: Uuid,
|
||||
name: String,
|
||||
is_public: bool,
|
||||
}
|
||||
|
||||
async fn update_input(
|
||||
authed: Authed,
|
||||
Extension(user_db): Extension<UserDB>,
|
||||
Path(w_id): Path<String>,
|
||||
Json(input): Json<UpdateInput>,
|
||||
) -> JsonResult<String> {
|
||||
let mut tx = user_db.begin(&authed).await?;
|
||||
|
||||
sqlx::query("UPDATE input SET name = $1, is_public = $2 WHERE id = $3 and workspace_id = $4")
|
||||
.bind(&input.name)
|
||||
.bind(&input.is_public)
|
||||
.bind(&input.id)
|
||||
.bind(&w_id)
|
||||
.execute(&mut tx)
|
||||
.await?;
|
||||
|
||||
tx.commit().await?;
|
||||
|
||||
Ok(Json(input.id.to_string()))
|
||||
}
|
||||
|
||||
async fn delete_input(
|
||||
authed: Authed,
|
||||
Extension(user_db): Extension<UserDB>,
|
||||
Path((w_id, i_id)): Path<(String, Uuid)>,
|
||||
) -> JsonResult<String> {
|
||||
let mut tx = user_db.begin(&authed).await?;
|
||||
|
||||
sqlx::query("DELETE FROM input WHERE id = $1 and workspace_id = $2")
|
||||
.bind(&i_id)
|
||||
.bind(&w_id)
|
||||
.execute(&mut tx)
|
||||
.await?;
|
||||
|
||||
tx.commit().await?;
|
||||
|
||||
Ok(Json(i_id.to_string()))
|
||||
}
|
||||
@@ -6,6 +6,12 @@
|
||||
* LICENSE-AGPL for a copy of the license.
|
||||
*/
|
||||
|
||||
use crate::{
|
||||
db::{UserDB, DB},
|
||||
users::{require_owner_of_path, Authed, OptAuthed},
|
||||
variables::get_workspace_key,
|
||||
BASE_URL,
|
||||
};
|
||||
use anyhow::Context;
|
||||
use axum::{
|
||||
extract::{FromRequest, Json, Path, Query},
|
||||
@@ -30,13 +36,8 @@ use windmill_common::{
|
||||
users::username_to_permissioned_as,
|
||||
utils::{not_found_if_none, now_from_db, paginate, require_admin, Pagination, StripPath},
|
||||
};
|
||||
use windmill_queue::{get_queued_job, push, JobKind, JobPayload, QueuedJob, RawCode};
|
||||
|
||||
use crate::{
|
||||
db::{UserDB, DB},
|
||||
users::{require_owner_of_path, Authed, OptAuthed},
|
||||
variables::get_workspace_key,
|
||||
BASE_URL,
|
||||
use windmill_queue::{
|
||||
get_queued_job, push, JobKind, JobPayload, QueueTransaction, QueuedJob, RawCode,
|
||||
};
|
||||
|
||||
pub fn workspaced_service() -> Router {
|
||||
@@ -64,6 +65,7 @@ pub fn workspaced_service() -> Router {
|
||||
.route("/run/preview_flow", post(run_preview_flow_job))
|
||||
.route("/list", get(list_jobs))
|
||||
.route("/queue/list", get(list_queue_jobs))
|
||||
.route("/queue/count", get(count_queue_jobs))
|
||||
.route("/completed/list", get(list_completed_jobs))
|
||||
.route("/completed/get/:id", get(get_completed_job))
|
||||
.route("/completed/get_result/:id", get(get_completed_job_result))
|
||||
@@ -114,6 +116,7 @@ async fn get_result_by_id(
|
||||
}
|
||||
|
||||
async fn cancel_job_api(
|
||||
Extension(rsmq): Extension<Option<rsmq_async::MultiplexedRsmq>>,
|
||||
OptAuthed(opt_authed): OptAuthed,
|
||||
Extension(db): Extension<DB>,
|
||||
Path((w_id, id)): Path<(String, Uuid)>,
|
||||
@@ -127,7 +130,7 @@ async fn cancel_job_api(
|
||||
};
|
||||
|
||||
let (mut tx, job_option) =
|
||||
windmill_queue::cancel_job(&username, reason, id, &w_id, tx, false).await?;
|
||||
windmill_queue::cancel_job(&username, reason, id, &w_id, tx, rsmq, false).await?;
|
||||
|
||||
if let Some(id) = job_option {
|
||||
audit_log(
|
||||
@@ -156,6 +159,7 @@ async fn cancel_job_api(
|
||||
}
|
||||
|
||||
async fn force_cancel(
|
||||
Extension(rsmq): Extension<Option<rsmq_async::MultiplexedRsmq>>,
|
||||
OptAuthed(opt_authed): OptAuthed,
|
||||
Extension(db): Extension<DB>,
|
||||
Path((w_id, id)): Path<(String, Uuid)>,
|
||||
@@ -169,7 +173,7 @@ async fn force_cancel(
|
||||
};
|
||||
|
||||
let (mut tx, job_option) =
|
||||
windmill_queue::cancel_job(&username, reason, id, &w_id, tx, true).await?;
|
||||
windmill_queue::cancel_job(&username, reason, id, &w_id, tx, rsmq, true).await?;
|
||||
|
||||
if let Some(id) = job_option {
|
||||
audit_log(
|
||||
@@ -496,6 +500,26 @@ async fn list_queue_jobs(
|
||||
Ok(Json(jobs))
|
||||
}
|
||||
|
||||
#[derive(Serialize, Debug, FromRow)]
|
||||
struct QueueStats {
|
||||
database_length: i64,
|
||||
}
|
||||
|
||||
async fn count_queue_jobs(
|
||||
Extension(db): Extension<DB>,
|
||||
Path(w_id): Path<String>,
|
||||
) -> error::JsonResult<QueueStats> {
|
||||
Ok(Json(
|
||||
sqlx::query_as!(
|
||||
QueueStats,
|
||||
"SELECT coalesce(COUNT(*), 0) as \"database_length!\" FROM queue WHERE workspace_id = $1",
|
||||
w_id
|
||||
)
|
||||
.fetch_one(&db)
|
||||
.await?,
|
||||
))
|
||||
}
|
||||
|
||||
async fn list_jobs(
|
||||
authed: Authed,
|
||||
Extension(user_db): Extension<UserDB>,
|
||||
@@ -793,6 +817,7 @@ async fn get_suspended_flow_info<'c>(
|
||||
pub async fn cancel_suspended_job(
|
||||
/* unauthed */
|
||||
Extension(db): Extension<DB>,
|
||||
Extension(rsmq): Extension<Option<rsmq_async::MultiplexedRsmq>>,
|
||||
Path((w_id, job, resume_id, secret)): Path<(String, Uuid, u32, String)>,
|
||||
Query(approver): Query<QueryApprover>,
|
||||
) -> error::Result<String> {
|
||||
@@ -815,6 +840,7 @@ pub async fn cancel_suspended_job(
|
||||
parent_flow,
|
||||
&w_id,
|
||||
tx,
|
||||
rsmq,
|
||||
false,
|
||||
)
|
||||
.await?;
|
||||
@@ -1208,14 +1234,15 @@ fn decode_payload<D: DeserializeOwned>(t: String) -> anyhow::Result<D> {
|
||||
pub async fn run_flow_by_path(
|
||||
authed: Authed,
|
||||
Extension(user_db): Extension<UserDB>,
|
||||
Extension(rsmq): Extension<Option<rsmq_async::MultiplexedRsmq>>,
|
||||
Path((w_id, flow_path)): Path<(String, StripPath)>,
|
||||
Query(run_query): Query<RunJobQuery>,
|
||||
headers: HeaderMap,
|
||||
Json(args): Json<Option<serde_json::Map<String, serde_json::Value>>>,
|
||||
) -> error::Result<(StatusCode, String)> {
|
||||
let flow_path = flow_path.to_path();
|
||||
let mut tx = user_db.begin(&authed).await?;
|
||||
let scheduled_for = run_query.get_scheduled_for(&mut tx).await?;
|
||||
let mut tx: QueueTransaction<'_, _> = (rsmq, user_db.begin(&authed).await?).into();
|
||||
let scheduled_for = run_query.get_scheduled_for(tx.transaction_mut()).await?;
|
||||
let args = run_query.add_include_headers(headers, args.unwrap_or_default());
|
||||
|
||||
let (uuid, tx) = push(
|
||||
@@ -1243,15 +1270,16 @@ pub async fn run_flow_by_path(
|
||||
pub async fn run_job_by_path(
|
||||
authed: Authed,
|
||||
Extension(user_db): Extension<UserDB>,
|
||||
Extension(rsmq): Extension<Option<rsmq_async::MultiplexedRsmq>>,
|
||||
Path((w_id, script_path)): Path<(String, StripPath)>,
|
||||
Query(run_query): Query<RunJobQuery>,
|
||||
headers: HeaderMap,
|
||||
Json(args): Json<Option<serde_json::Map<String, serde_json::Value>>>,
|
||||
) -> error::Result<(StatusCode, String)> {
|
||||
let script_path = script_path.to_path();
|
||||
let mut tx = user_db.begin(&authed).await?;
|
||||
let job_payload = script_path_to_payload(script_path, &mut tx, &w_id).await?;
|
||||
let scheduled_for = run_query.get_scheduled_for(&mut tx).await?;
|
||||
let mut tx: QueueTransaction<'_, _> = (rsmq, user_db.begin(&authed).await?).into();
|
||||
let job_payload = script_path_to_payload(script_path, tx.transaction_mut(), &w_id).await?;
|
||||
let scheduled_for = run_query.get_scheduled_for(tx.transaction_mut()).await?;
|
||||
let args = run_query.add_include_headers(headers, args.unwrap_or_default());
|
||||
|
||||
let (uuid, tx) = push(
|
||||
@@ -1393,6 +1421,7 @@ lazy_static::lazy_static! {
|
||||
|
||||
pub async fn run_wait_result_job_by_path_get(
|
||||
authed: Authed,
|
||||
Extension(rsmq): Extension<Option<rsmq_async::MultiplexedRsmq>>,
|
||||
Extension(user_db): Extension<UserDB>,
|
||||
Extension(db): Extension<DB>,
|
||||
Path((w_id, script_path)): Path<(String, StripPath)>,
|
||||
@@ -1411,8 +1440,8 @@ pub async fn run_wait_result_job_by_path_get(
|
||||
|
||||
check_queue_too_long(db, QUEUE_LIMIT_WAIT_RESULT.or(run_query.queue_limit)).await?;
|
||||
let script_path = script_path.to_path();
|
||||
let mut tx = user_db.clone().begin(&authed).await?;
|
||||
let job_payload = script_path_to_payload(script_path, &mut tx, &w_id).await?;
|
||||
let mut tx: QueueTransaction<'_, _> = (rsmq, user_db.clone().begin(&authed).await?).into();
|
||||
let job_payload = script_path_to_payload(script_path, tx.transaction_mut(), &w_id).await?;
|
||||
|
||||
let (uuid, tx) = push(
|
||||
tx,
|
||||
@@ -1447,6 +1476,7 @@ pub async fn run_wait_result_job_by_path_get(
|
||||
pub async fn run_wait_result_job_by_path(
|
||||
authed: Authed,
|
||||
Extension(user_db): Extension<UserDB>,
|
||||
Extension(rsmq): Extension<Option<rsmq_async::MultiplexedRsmq>>,
|
||||
Extension(db): Extension<DB>,
|
||||
Path((w_id, script_path)): Path<(String, StripPath)>,
|
||||
Query(run_query): Query<RunJobQuery>,
|
||||
@@ -1455,8 +1485,8 @@ pub async fn run_wait_result_job_by_path(
|
||||
) -> error::JsonResult<serde_json::Value> {
|
||||
check_queue_too_long(db, QUEUE_LIMIT_WAIT_RESULT.or(run_query.queue_limit)).await?;
|
||||
let script_path = script_path.to_path();
|
||||
let mut tx = user_db.clone().begin(&authed).await?;
|
||||
let job_payload = script_path_to_payload(script_path, &mut tx, &w_id).await?;
|
||||
let mut tx: QueueTransaction<'_, _> = (rsmq, user_db.clone().begin(&authed).await?).into();
|
||||
let job_payload = script_path_to_payload(script_path, tx.transaction_mut(), &w_id).await?;
|
||||
|
||||
let args = run_query.add_include_headers(headers, args.unwrap_or_default());
|
||||
|
||||
@@ -1493,6 +1523,7 @@ pub async fn run_wait_result_job_by_path(
|
||||
pub async fn run_wait_result_job_by_hash(
|
||||
authed: Authed,
|
||||
Extension(user_db): Extension<UserDB>,
|
||||
Extension(rsmq): Extension<Option<rsmq_async::MultiplexedRsmq>>,
|
||||
Extension(db): Extension<DB>,
|
||||
Path((w_id, script_hash)): Path<(String, ScriptHash)>,
|
||||
Query(run_query): Query<RunJobQuery>,
|
||||
@@ -1502,8 +1533,9 @@ pub async fn run_wait_result_job_by_hash(
|
||||
check_queue_too_long(db, run_query.queue_limit).await?;
|
||||
|
||||
let hash = script_hash.0;
|
||||
let mut tx = user_db.clone().begin(&authed).await?;
|
||||
let path = get_path_for_hash(&mut tx, &w_id, hash).await?;
|
||||
let mut tx: QueueTransaction<'_, _> = (rsmq, user_db.clone().begin(&authed).await?).into();
|
||||
let path = get_path_for_hash(tx.transaction_mut(), &w_id, hash).await?;
|
||||
|
||||
let args = run_query.add_include_headers(headers, args.unwrap_or_default());
|
||||
|
||||
let (uuid, tx) = push(
|
||||
@@ -1539,6 +1571,7 @@ pub async fn run_wait_result_job_by_hash(
|
||||
pub async fn run_wait_result_flow_by_path(
|
||||
authed: Authed,
|
||||
Extension(user_db): Extension<UserDB>,
|
||||
Extension(rsmq): Extension<Option<rsmq_async::MultiplexedRsmq>>,
|
||||
Extension(db): Extension<DB>,
|
||||
Path((w_id, flow_path)): Path<(String, StripPath)>,
|
||||
Query(run_query): Query<RunJobQuery>,
|
||||
@@ -1548,8 +1581,8 @@ pub async fn run_wait_result_flow_by_path(
|
||||
check_queue_too_long(db, run_query.queue_limit).await?;
|
||||
|
||||
let flow_path = flow_path.to_path();
|
||||
let mut tx = user_db.clone().begin(&authed).await?;
|
||||
let scheduled_for = run_query.get_scheduled_for(&mut tx).await?;
|
||||
let mut tx: QueueTransaction<'_, _> = (rsmq, user_db.clone().begin(&authed).await?).into();
|
||||
let scheduled_for = run_query.get_scheduled_for(tx.transaction_mut()).await?;
|
||||
let args = run_query.add_include_headers(headers, args.unwrap_or_default());
|
||||
|
||||
let (uuid, tx) = push(
|
||||
@@ -1570,7 +1603,6 @@ pub async fn run_wait_result_flow_by_path(
|
||||
!run_query.invisible_to_owner.unwrap_or(false),
|
||||
)
|
||||
.await?;
|
||||
|
||||
tx.commit().await?;
|
||||
|
||||
run_wait_result(
|
||||
@@ -1601,13 +1633,14 @@ pub async fn script_path_to_payload<'c>(
|
||||
async fn run_preview_job(
|
||||
authed: Authed,
|
||||
Extension(user_db): Extension<UserDB>,
|
||||
Extension(rsmq): Extension<Option<rsmq_async::MultiplexedRsmq>>,
|
||||
Path(w_id): Path<String>,
|
||||
Query(run_query): Query<RunJobQuery>,
|
||||
headers: HeaderMap,
|
||||
Json(preview): Json<Preview>,
|
||||
) -> error::Result<(StatusCode, String)> {
|
||||
let mut tx = user_db.begin(&authed).await?;
|
||||
let scheduled_for = run_query.get_scheduled_for(&mut tx).await?;
|
||||
let mut tx: QueueTransaction<'_, _> = (rsmq, user_db.begin(&authed).await?).into();
|
||||
let scheduled_for = run_query.get_scheduled_for(tx.transaction_mut()).await?;
|
||||
let args = run_query.add_include_headers(headers, preview.args.unwrap_or_default());
|
||||
|
||||
let (uuid, tx) = push(
|
||||
@@ -1634,19 +1667,21 @@ async fn run_preview_job(
|
||||
)
|
||||
.await?;
|
||||
tx.commit().await?;
|
||||
|
||||
Ok((StatusCode::CREATED, uuid.to_string()))
|
||||
}
|
||||
|
||||
async fn run_preview_flow_job(
|
||||
authed: Authed,
|
||||
Extension(user_db): Extension<UserDB>,
|
||||
Extension(rsmq): Extension<Option<rsmq_async::MultiplexedRsmq>>,
|
||||
Path(w_id): Path<String>,
|
||||
Query(run_query): Query<RunJobQuery>,
|
||||
headers: HeaderMap,
|
||||
Json(raw_flow): Json<PreviewFlow>,
|
||||
) -> error::Result<(StatusCode, String)> {
|
||||
let mut tx = user_db.begin(&authed).await?;
|
||||
let scheduled_for = run_query.get_scheduled_for(&mut tx).await?;
|
||||
let mut tx: QueueTransaction<'_, _> = (rsmq, user_db.begin(&authed).await?).into();
|
||||
let scheduled_for = run_query.get_scheduled_for(tx.transaction_mut()).await?;
|
||||
let args = run_query.add_include_headers(headers, raw_flow.args.unwrap_or_default());
|
||||
|
||||
let (uuid, tx) = push(
|
||||
@@ -1668,21 +1703,23 @@ async fn run_preview_flow_job(
|
||||
)
|
||||
.await?;
|
||||
tx.commit().await?;
|
||||
|
||||
Ok((StatusCode::CREATED, uuid.to_string()))
|
||||
}
|
||||
|
||||
pub async fn run_job_by_hash(
|
||||
authed: Authed,
|
||||
Extension(user_db): Extension<UserDB>,
|
||||
Extension(rsmq): Extension<Option<rsmq_async::MultiplexedRsmq>>,
|
||||
Path((w_id, script_hash)): Path<(String, ScriptHash)>,
|
||||
Query(run_query): Query<RunJobQuery>,
|
||||
headers: HeaderMap,
|
||||
Json(args): Json<Option<serde_json::Map<String, serde_json::Value>>>,
|
||||
) -> error::Result<(StatusCode, String)> {
|
||||
let hash = script_hash.0;
|
||||
let mut tx = user_db.begin(&authed).await?;
|
||||
let path = get_path_for_hash(&mut tx, &w_id, hash).await?;
|
||||
let scheduled_for = run_query.get_scheduled_for(&mut tx).await?;
|
||||
let mut tx: QueueTransaction<'_, _> = (rsmq, user_db.begin(&authed).await?).into();
|
||||
let path = get_path_for_hash(tx.transaction_mut(), &w_id, hash).await?;
|
||||
let scheduled_for = run_query.get_scheduled_for(tx.transaction_mut()).await?;
|
||||
let args = run_query.add_include_headers(headers, args.unwrap_or_default());
|
||||
|
||||
let (uuid, tx) = push(
|
||||
@@ -1704,6 +1741,7 @@ pub async fn run_job_by_hash(
|
||||
)
|
||||
.await?;
|
||||
tx.commit().await?;
|
||||
|
||||
Ok((StatusCode::CREATED, uuid.to_string()))
|
||||
}
|
||||
|
||||
|
||||
@@ -7,6 +7,13 @@
|
||||
*/
|
||||
|
||||
use crate::oauth2::AllClients;
|
||||
use crate::{
|
||||
db::UserDB,
|
||||
oauth2::{build_oauth_clients, SlackVerifier},
|
||||
tracing_init::{MyMakeSpan, MyOnResponse},
|
||||
users::{Authed, OptAuthed},
|
||||
webhook_util::WebhookShared,
|
||||
};
|
||||
use argon2::Argon2;
|
||||
use axum::{middleware::from_extractor, routing::get, Extension, Router};
|
||||
use db::DB;
|
||||
@@ -22,14 +29,6 @@ use tower_http::{
|
||||
};
|
||||
use windmill_common::utils::rd_string;
|
||||
|
||||
use crate::{
|
||||
db::UserDB,
|
||||
oauth2::{build_oauth_clients, SlackVerifier},
|
||||
tracing_init::{MyMakeSpan, MyOnResponse},
|
||||
users::{Authed, OptAuthed},
|
||||
webhook_util::WebhookShared,
|
||||
};
|
||||
|
||||
mod apps;
|
||||
mod audit;
|
||||
mod capture;
|
||||
@@ -39,6 +38,7 @@ mod flows;
|
||||
mod folders;
|
||||
mod granular_acls;
|
||||
mod groups;
|
||||
mod inputs;
|
||||
pub mod jobs;
|
||||
mod oauth2;
|
||||
mod resources;
|
||||
@@ -81,6 +81,7 @@ lazy_static::lazy_static! {
|
||||
|
||||
pub async fn run_server(
|
||||
db: DB,
|
||||
rsmq: Option<rsmq_async::MultiplexedRsmq>,
|
||||
addr: SocketAddr,
|
||||
mut rx: tokio::sync::broadcast::Receiver<()>,
|
||||
) -> anyhow::Result<()> {
|
||||
@@ -100,6 +101,7 @@ pub async fn run_server(
|
||||
.on_request(()),
|
||||
)
|
||||
.layer(Extension(db.clone()))
|
||||
.layer(Extension(rsmq))
|
||||
.layer(Extension(user_db))
|
||||
.layer(Extension(auth_cache.clone()))
|
||||
.layer(CookieManagerLayer::new())
|
||||
@@ -117,25 +119,27 @@ pub async fn run_server(
|
||||
.nest(
|
||||
"/w/:workspace_id",
|
||||
Router::new()
|
||||
.nest("/scripts", scripts::workspaced_service())
|
||||
// Reordered alphabetically
|
||||
.nest("/acls", granular_acls::workspaced_service())
|
||||
.nest("/apps", apps::workspaced_service())
|
||||
.nest("/audit", audit::workspaced_service())
|
||||
.nest("/capture", capture::workspaced_service())
|
||||
.nest("/favorites", favorite::workspaced_service())
|
||||
.nest("/flows", flows::workspaced_service())
|
||||
.nest("/folders", folders::workspaced_service())
|
||||
.nest("/groups", groups::workspaced_service())
|
||||
.nest("/inputs", inputs::workspaced_service())
|
||||
.nest("/jobs", jobs::workspaced_service().layer(cors.clone()))
|
||||
.nest("/oauth", oauth2::workspaced_service())
|
||||
.nest("/resources", resources::workspaced_service())
|
||||
.nest("/schedules", schedule::workspaced_service())
|
||||
.nest("/scripts", scripts::workspaced_service())
|
||||
.nest(
|
||||
"/users",
|
||||
users::workspaced_service().layer(Extension(argon2.clone())),
|
||||
)
|
||||
.nest("/variables", variables::workspaced_service())
|
||||
.nest("/oauth", oauth2::workspaced_service())
|
||||
.nest("/resources", resources::workspaced_service())
|
||||
.nest("/schedules", schedule::workspaced_service())
|
||||
.nest("/groups", groups::workspaced_service())
|
||||
.nest("/audit", audit::workspaced_service())
|
||||
.nest("/acls", granular_acls::workspaced_service())
|
||||
.nest("/workspaces", workspaces::workspaced_service())
|
||||
.nest("/apps", apps::workspaced_service())
|
||||
.nest("/flows", flows::workspaced_service())
|
||||
.nest("/capture", capture::workspaced_service())
|
||||
.nest("/favorites", favorite::workspaced_service())
|
||||
.nest("/folders", folders::workspaced_service()),
|
||||
.nest("/workspaces", workspaces::workspaced_service()),
|
||||
)
|
||||
.nest("/workspaces", workspaces::global_service())
|
||||
.nest(
|
||||
|
||||
@@ -32,7 +32,8 @@ use windmill_audit::{audit_log, ActionKind};
|
||||
use windmill_common::users::username_to_permissioned_as;
|
||||
use windmill_common::utils::{not_found_if_none, now_from_db};
|
||||
|
||||
use crate::users::{truncate_token, Authed, NEW_USER_WEBHOOK};
|
||||
use crate::users::{truncate_token, Authed};
|
||||
use crate::webhook_util::{InstanceEvent, WebhookShared};
|
||||
use crate::workspaces::invite_user_to_all_auto_invite_worspaces;
|
||||
use crate::{
|
||||
db::{UserDB, DB},
|
||||
@@ -43,7 +44,7 @@ use crate::{BASE_URL, HTTP_CLIENT, IS_SECURE, OAUTH_CLIENTS, SLACK_SIGNING_SECRE
|
||||
use windmill_common::error::{self, to_anyhow, Error};
|
||||
use windmill_common::oauth2::*;
|
||||
|
||||
use windmill_queue::JobPayload;
|
||||
use windmill_queue::{JobPayload, QueueTransaction};
|
||||
|
||||
use std::{fs, str};
|
||||
|
||||
@@ -728,6 +729,7 @@ where
|
||||
async fn slack_command(
|
||||
SlackSig { sig, ts }: SlackSig,
|
||||
Extension(db): Extension<DB>,
|
||||
Extension(rsmq): Extension<Option<rsmq_async::MultiplexedRsmq>>,
|
||||
body: Bytes,
|
||||
) -> error::Result<String> {
|
||||
let form: SlackCommand = serde_urlencoded::from_bytes(&body)
|
||||
@@ -740,7 +742,7 @@ async fn slack_command(
|
||||
}
|
||||
}
|
||||
|
||||
let mut tx = db.begin().await?;
|
||||
let mut tx: QueueTransaction<'_, _> = (rsmq, db.begin().await?).into();
|
||||
let settings = sqlx::query_as!(
|
||||
WorkspaceSettings,
|
||||
"SELECT * FROM workspace_settings WHERE slack_team_id = $1",
|
||||
@@ -756,7 +758,7 @@ async fn slack_command(
|
||||
} else {
|
||||
let path = path.strip_prefix("script/").unwrap_or_else(|| path);
|
||||
let script_hash = windmill_common::get_latest_hash_for_path(
|
||||
&mut tx,
|
||||
tx.transaction_mut(),
|
||||
&settings.workspace_id,
|
||||
path,
|
||||
)
|
||||
@@ -788,14 +790,15 @@ async fn slack_command(
|
||||
true,
|
||||
)
|
||||
.await?;
|
||||
tx.commit().await?;
|
||||
let url = BASE_URL.to_owned();
|
||||
tx.commit().await?;
|
||||
return Ok(format!(
|
||||
"Job launched. See details at {url}/run/{uuid}?workspace={}",
|
||||
&settings.workspace_id
|
||||
));
|
||||
}
|
||||
}
|
||||
tx.commit().await?;
|
||||
|
||||
return Ok(format!(
|
||||
"workspace not properly configured (did you set the script to trigger in the settings?)"
|
||||
@@ -815,6 +818,7 @@ async fn login_callback(
|
||||
Path(client_name): Path<String>,
|
||||
cookies: Cookies,
|
||||
Extension(db): Extension<DB>,
|
||||
Extension(webhook): Extension<WebhookShared>,
|
||||
Json(callback): Json<OAuthCallback>,
|
||||
) -> error::Result<String> {
|
||||
let client_w_config = &OAUTH_CLIENTS
|
||||
@@ -940,14 +944,7 @@ async fn login_callback(
|
||||
}
|
||||
tx.commit().await?;
|
||||
|
||||
if let Some(new_user_webhook) = NEW_USER_WEBHOOK.clone() {
|
||||
let _ = HTTP_CLIENT
|
||||
.post(&new_user_webhook)
|
||||
.json(&serde_json::json!({"email" : &email, "event": "oauth_signup"}))
|
||||
.send()
|
||||
.await
|
||||
.map_err(|e| tracing::error!("Error sending new user webhook: {}", e.to_string()));
|
||||
}
|
||||
webhook.send_instance_event(InstanceEvent::UserSignupOAuth { email: email.clone() });
|
||||
|
||||
Ok("Successfully logged in".to_string())
|
||||
} else {
|
||||
|
||||
@@ -25,7 +25,7 @@ use windmill_common::{
|
||||
schedule::Schedule,
|
||||
utils::{not_found_if_none, paginate, Pagination, StripPath},
|
||||
};
|
||||
use windmill_queue::{self, schedule::push_scheduled_job, JobKind};
|
||||
use windmill_queue::{self, schedule::push_scheduled_job, JobKind, QueueTransaction};
|
||||
|
||||
pub fn workspaced_service() -> Router {
|
||||
Router::new()
|
||||
@@ -79,15 +79,23 @@ async fn create_schedule(
|
||||
authed: Authed,
|
||||
Extension(db): Extension<DB>,
|
||||
Extension(user_db): Extension<UserDB>,
|
||||
Extension(rsmq): Extension<Option<rsmq_async::MultiplexedRsmq>>,
|
||||
Path(w_id): Path<String>,
|
||||
Json(ns): Json<NewSchedule>,
|
||||
) -> Result<String> {
|
||||
let authed = maybe_refresh_folders(&ns.path, &w_id, authed, &db).await;
|
||||
let mut tx: QueueTransaction<'_, _> = (rsmq, user_db.begin(&authed).await?).into();
|
||||
|
||||
let mut tx = user_db.begin(&authed).await?;
|
||||
cron::Schedule::from_str(&ns.schedule).map_err(|e| Error::BadRequest(e.to_string()))?;
|
||||
check_path_conflict(&mut tx, &w_id, &ns.path).await?;
|
||||
check_flow_conflict(&mut tx, &w_id, &ns.path, ns.is_flow, &ns.script_path).await?;
|
||||
check_path_conflict(tx.transaction_mut(), &w_id, &ns.path).await?;
|
||||
check_flow_conflict(
|
||||
tx.transaction_mut(),
|
||||
&w_id,
|
||||
&ns.path,
|
||||
ns.is_flow,
|
||||
&ns.script_path,
|
||||
)
|
||||
.await?;
|
||||
|
||||
let schedule = sqlx::query_as!(
|
||||
Schedule,
|
||||
@@ -139,14 +147,15 @@ async fn edit_schedule(
|
||||
authed: Authed,
|
||||
Extension(db): Extension<DB>,
|
||||
Extension(user_db): Extension<UserDB>,
|
||||
Extension(rsmq): Extension<Option<rsmq_async::MultiplexedRsmq>>,
|
||||
Path((w_id, path)): Path<(String, StripPath)>,
|
||||
Json(es): Json<EditSchedule>,
|
||||
) -> Result<String> {
|
||||
let path = path.to_path();
|
||||
|
||||
let authed = maybe_refresh_folders(&path, &w_id, authed, &db).await;
|
||||
|
||||
let mut tx = user_db.begin(&authed).await?;
|
||||
let mut tx: QueueTransaction<'_, rsmq_async::MultiplexedRsmq> =
|
||||
(rsmq, user_db.begin(&authed).await?).into();
|
||||
|
||||
cron::Schedule::from_str(&es.schedule).map_err(|e| Error::BadRequest(e.to_string()))?;
|
||||
|
||||
@@ -158,7 +167,7 @@ async fn edit_schedule(
|
||||
.fetch_one(&mut tx)
|
||||
.await?;
|
||||
|
||||
clear_schedule(&mut tx, path, is_flow).await?;
|
||||
clear_schedule(tx.transaction_mut(), path, is_flow).await?;
|
||||
let schedule = sqlx::query_as!(
|
||||
Schedule,
|
||||
"UPDATE schedule SET schedule = $1, timezone = $2, args = $3 WHERE path \
|
||||
@@ -173,10 +182,6 @@ async fn edit_schedule(
|
||||
.await
|
||||
.map_err(|e| Error::InternalErr(format!("updating schedule in {w_id}: {e}")))?;
|
||||
|
||||
if schedule.enabled {
|
||||
tx = push_scheduled_job(tx, schedule).await?;
|
||||
}
|
||||
|
||||
audit_log(
|
||||
&mut tx,
|
||||
&authed.username,
|
||||
@@ -192,6 +197,10 @@ async fn edit_schedule(
|
||||
),
|
||||
)
|
||||
.await?;
|
||||
|
||||
if schedule.enabled {
|
||||
tx = push_scheduled_job(tx, schedule).await?;
|
||||
}
|
||||
tx.commit().await?;
|
||||
|
||||
Ok(path.to_string())
|
||||
@@ -270,10 +279,12 @@ pub async fn preview_schedule(
|
||||
pub async fn set_enabled(
|
||||
authed: Authed,
|
||||
Extension(user_db): Extension<UserDB>,
|
||||
Extension(rsmq): Extension<Option<rsmq_async::MultiplexedRsmq>>,
|
||||
Path((w_id, path)): Path<(String, StripPath)>,
|
||||
Json(payload): Json<SetEnabled>,
|
||||
) -> Result<String> {
|
||||
let mut tx = user_db.begin(&authed).await?;
|
||||
let mut tx: QueueTransaction<'_, rsmq_async::MultiplexedRsmq> =
|
||||
(rsmq, user_db.begin(&authed).await?).into();
|
||||
let path = path.to_path();
|
||||
let schedule_o = sqlx::query_as!(
|
||||
Schedule,
|
||||
@@ -288,11 +299,8 @@ pub async fn set_enabled(
|
||||
|
||||
let schedule = not_found_if_none(schedule_o, "Schedule", path)?;
|
||||
|
||||
clear_schedule(&mut tx, path, schedule.is_flow).await?;
|
||||
clear_schedule(tx.transaction_mut(), path, schedule.is_flow).await?;
|
||||
|
||||
if payload.enabled {
|
||||
tx = push_scheduled_job(tx, schedule).await?;
|
||||
}
|
||||
audit_log(
|
||||
&mut tx,
|
||||
&authed.username,
|
||||
@@ -303,7 +311,12 @@ pub async fn set_enabled(
|
||||
Some([("enabled", payload.enabled.to_string().as_ref())].into()),
|
||||
)
|
||||
.await?;
|
||||
|
||||
if payload.enabled {
|
||||
tx = push_scheduled_job(tx, schedule).await?;
|
||||
}
|
||||
tx.commit().await?;
|
||||
|
||||
Ok(format!(
|
||||
"succesfully updated schedule at path {} to status {}",
|
||||
path, payload.enabled
|
||||
|
||||
@@ -6,10 +6,6 @@
|
||||
* LICENSE-AGPL for a copy of the license.
|
||||
*/
|
||||
|
||||
use sql_builder::prelude::*;
|
||||
use windmill_audit::{audit_log, ActionKind};
|
||||
use windmill_parser::MainArgSignature;
|
||||
|
||||
use crate::{
|
||||
db::{UserDB, DB},
|
||||
schedule::clear_schedule,
|
||||
@@ -25,6 +21,7 @@ use axum::{
|
||||
use hyper::StatusCode;
|
||||
use serde::Serialize;
|
||||
use serde_json::json;
|
||||
use sql_builder::prelude::*;
|
||||
use sql_builder::SqlBuilder;
|
||||
use sqlx::{FromRow, Postgres, Transaction};
|
||||
use std::{
|
||||
@@ -32,6 +29,7 @@ use std::{
|
||||
hash::{Hash, Hasher},
|
||||
sync::Arc,
|
||||
};
|
||||
use windmill_audit::{audit_log, ActionKind};
|
||||
use windmill_common::{
|
||||
error::{Error, JsonResult, Result},
|
||||
schedule::Schedule,
|
||||
@@ -44,7 +42,8 @@ use windmill_common::{
|
||||
list_elems_from_hub, not_found_if_none, paginate, require_admin, Pagination, StripPath,
|
||||
},
|
||||
};
|
||||
use windmill_queue::{self, schedule::push_scheduled_job};
|
||||
use windmill_parser::MainArgSignature;
|
||||
use windmill_queue::{self, schedule::push_scheduled_job, QueueTransaction};
|
||||
|
||||
const MAX_HASH_HISTORY_LENGTH_STORED: usize = 20;
|
||||
|
||||
@@ -85,6 +84,7 @@ pub fn workspaced_service() -> Router {
|
||||
.route("/deployment_status/h/:hash", get(get_deployment_status))
|
||||
.route("/list_paths", get(list_paths))
|
||||
}
|
||||
|
||||
async fn list_scripts(
|
||||
authed: Authed,
|
||||
Extension(user_db): Extension<UserDB>,
|
||||
@@ -193,6 +193,7 @@ fn hash_script(ns: &NewScript) -> i64 {
|
||||
async fn create_script(
|
||||
authed: Authed,
|
||||
Extension(user_db): Extension<UserDB>,
|
||||
Extension(rsmq): Extension<Option<rsmq_async::MultiplexedRsmq>>,
|
||||
Extension(webhook): Extension<WebhookShared>,
|
||||
Extension(db): Extension<DB>,
|
||||
Path(w_id): Path<String>,
|
||||
@@ -200,14 +201,14 @@ async fn create_script(
|
||||
) -> Result<(StatusCode, String)> {
|
||||
let hash = ScriptHash(hash_script(&ns));
|
||||
let authed = maybe_refresh_folders(&ns.path, &w_id, authed, &db).await;
|
||||
let mut tx = user_db.begin(&authed).await?;
|
||||
let mut tx: QueueTransaction<'_, _> = (rsmq, user_db.begin(&authed).await?).into();
|
||||
|
||||
if sqlx::query_scalar!(
|
||||
"SELECT 1 FROM script WHERE hash = $1 AND workspace_id = $2",
|
||||
hash.0,
|
||||
&w_id
|
||||
)
|
||||
.fetch_optional(&mut tx)
|
||||
.fetch_optional(tx.transaction_mut())
|
||||
.await?
|
||||
.is_some()
|
||||
{
|
||||
@@ -268,7 +269,7 @@ async fn create_script(
|
||||
)));
|
||||
};
|
||||
|
||||
let ps = get_script_by_hash_internal(&mut tx, &w_id, p_hash).await?;
|
||||
let ps = get_script_by_hash_internal(tx.transaction_mut(), &w_id, p_hash).await?;
|
||||
|
||||
if ps.path != ns.path {
|
||||
if !authed.is_admin {
|
||||
@@ -363,7 +364,7 @@ async fn create_script(
|
||||
.await?;
|
||||
|
||||
for schedule in schedulables {
|
||||
clear_schedule(&mut tx, &schedule.path, false).await?;
|
||||
clear_schedule(tx.transaction_mut(), &schedule.path, false).await?;
|
||||
|
||||
if schedule.enabled {
|
||||
tx = push_scheduled_job(tx, schedule).await?;
|
||||
@@ -371,36 +372,6 @@ async fn create_script(
|
||||
}
|
||||
}
|
||||
|
||||
let mut tx = if needs_lock_gen {
|
||||
let dependencies = match ns.language {
|
||||
ScriptLang::Python3 => {
|
||||
windmill_parser_py::parse_python_imports(&ns.content)?.join("\n")
|
||||
}
|
||||
_ => ns.content,
|
||||
};
|
||||
let (_, tx) = windmill_queue::push(
|
||||
tx,
|
||||
&w_id,
|
||||
windmill_queue::JobPayload::Dependencies { hash, dependencies, language: ns.language },
|
||||
serde_json::Map::new(),
|
||||
&authed.username,
|
||||
&authed.email,
|
||||
username_to_permissioned_as(&authed.username),
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
false,
|
||||
false,
|
||||
None,
|
||||
true,
|
||||
)
|
||||
.await?;
|
||||
tx
|
||||
} else {
|
||||
tx
|
||||
};
|
||||
|
||||
if p_hashes.is_some() && !p_hashes.unwrap().is_empty() {
|
||||
audit_log(
|
||||
&mut tx,
|
||||
@@ -415,7 +386,7 @@ async fn create_script(
|
||||
webhook.send_message(
|
||||
w_id.clone(),
|
||||
WebhookMessage::UpdateScript {
|
||||
workspace: w_id,
|
||||
workspace: w_id.clone(),
|
||||
path: ns.path.clone(),
|
||||
hash: hash.to_string(),
|
||||
},
|
||||
@@ -440,13 +411,41 @@ async fn create_script(
|
||||
webhook.send_message(
|
||||
w_id.clone(),
|
||||
WebhookMessage::CreateScript {
|
||||
workspace: w_id,
|
||||
workspace: w_id.clone(),
|
||||
path: ns.path.clone(),
|
||||
hash: hash.to_string(),
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
if needs_lock_gen {
|
||||
let dependencies = match ns.language {
|
||||
ScriptLang::Python3 => {
|
||||
windmill_parser_py::parse_python_imports(&ns.content)?.join("\n")
|
||||
}
|
||||
_ => ns.content,
|
||||
};
|
||||
let (_, new_tx) = windmill_queue::push(
|
||||
tx,
|
||||
&w_id,
|
||||
windmill_queue::JobPayload::Dependencies { hash, dependencies, language: ns.language },
|
||||
serde_json::Map::new(),
|
||||
&authed.username,
|
||||
&authed.email,
|
||||
username_to_permissioned_as(&authed.username),
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
false,
|
||||
false,
|
||||
None,
|
||||
true,
|
||||
)
|
||||
.await?;
|
||||
tx = new_tx;
|
||||
}
|
||||
|
||||
tx.commit().await?;
|
||||
|
||||
Ok((StatusCode::CREATED, format!("{}", hash)))
|
||||
@@ -799,7 +798,7 @@ async fn parse_python_code_to_jsonschema(Json(code): Json<String>) -> Json<SigPa
|
||||
}
|
||||
|
||||
async fn parse_deno_code_to_jsonschema(Json(code): Json<String>) -> Json<SigParsing> {
|
||||
result_to_sig_parsing(windmill_parser_ts::parse_deno_signature(&code))
|
||||
result_to_sig_parsing(windmill_parser_ts::parse_deno_signature(&code, false))
|
||||
}
|
||||
async fn parse_go_code_to_jsonschema(Json(code): Json<String>) -> Json<SigParsing> {
|
||||
result_to_sig_parsing(windmill_parser_go::parse_go_sig(&code))
|
||||
|
||||
@@ -12,8 +12,9 @@ use crate::{
|
||||
db::{UserDB, DB},
|
||||
folders::get_folders_for_user,
|
||||
utils::require_super_admin,
|
||||
webhook_util::{InstanceEvent, WebhookShared},
|
||||
workspaces::invite_user_to_all_auto_invite_worspaces,
|
||||
COOKIE_DOMAIN, HTTP_CLIENT, IS_SECURE,
|
||||
COOKIE_DOMAIN, IS_SECURE,
|
||||
};
|
||||
use argon2::{password_hash::SaltString, Argon2, PasswordHash, PasswordHasher, PasswordVerifier};
|
||||
use axum::{
|
||||
@@ -1037,6 +1038,7 @@ lazy_static! {
|
||||
|
||||
async fn accept_invite(
|
||||
Authed { email, .. }: Authed,
|
||||
Extension(webhook): Extension<WebhookShared>,
|
||||
Extension(db): Extension<DB>,
|
||||
Json(nu): Json<AcceptInvite>,
|
||||
) -> Result<(StatusCode, String)> {
|
||||
@@ -1082,6 +1084,11 @@ async fn accept_invite(
|
||||
}
|
||||
|
||||
if is_some {
|
||||
webhook.send_instance_event(InstanceEvent::UserJoinedWorkspace {
|
||||
email: email.clone(),
|
||||
workspace: nu.workspace_id.clone(),
|
||||
username: nu.username.clone(),
|
||||
});
|
||||
Ok((
|
||||
StatusCode::CREATED,
|
||||
format!(
|
||||
@@ -1322,6 +1329,7 @@ lazy_static::lazy_static! {
|
||||
async fn create_user(
|
||||
Authed { email, .. }: Authed,
|
||||
Extension(db): Extension<DB>,
|
||||
Extension(webhook): Extension<WebhookShared>,
|
||||
Extension(argon2): Extension<Arc<Argon2<'_>>>,
|
||||
Json(mut nu): Json<NewUser>,
|
||||
) -> Result<(StatusCode, String)> {
|
||||
@@ -1362,19 +1370,9 @@ async fn create_user(
|
||||
.await?;
|
||||
tx.commit().await?;
|
||||
|
||||
if let Some(new_user_webhook) = NEW_USER_WEBHOOK.clone() {
|
||||
let _ = HTTP_CLIENT
|
||||
.post(&new_user_webhook)
|
||||
.json(
|
||||
&serde_json::json!({"email" : &nu.email, "name": &nu.name, "event": "global_add"}),
|
||||
)
|
||||
.send()
|
||||
.await
|
||||
.map_err(|e| tracing::error!("Error sending new user webhook: {}", e.to_string()));
|
||||
}
|
||||
|
||||
invite_user_to_all_auto_invite_worspaces(&db, &nu.email).await?;
|
||||
|
||||
webhook.send_instance_event(InstanceEvent::UserAdded { email: nu.email.clone() });
|
||||
Ok((StatusCode::CREATED, format!("email {} created", nu.email)))
|
||||
}
|
||||
|
||||
@@ -1894,6 +1892,17 @@ pub async fn delete_expired_items_perdiodically(
|
||||
Err(e) => tracing::error!("Error deleting token: {}", e.to_string()),
|
||||
}
|
||||
|
||||
let pip_resolution_r = sqlx::query_scalar!(
|
||||
"DELETE FROM pip_resolution_cache WHERE expiration <= now() RETURNING hash",
|
||||
)
|
||||
.fetch_all(db)
|
||||
.await;
|
||||
|
||||
match pip_resolution_r {
|
||||
Ok(res) => tracing::debug!("deleted {} pip_resolution: {:?}", res.len(), res),
|
||||
Err(e) => tracing::error!("Error deleting pip_resolution: {}", e.to_string()),
|
||||
}
|
||||
|
||||
let magic_links_deleted_r: std::result::Result<Vec<String>, _> = sqlx::query_scalar(
|
||||
"DELETE FROM magic_link WHERE expiration <= now()
|
||||
RETURNING concat(substring(token for 10), '*****')",
|
||||
|
||||
@@ -2,6 +2,7 @@ use std::time::Duration;
|
||||
|
||||
use serde::Serialize;
|
||||
use tokio::{select, sync::mpsc, time::interval};
|
||||
use windmill_common::METRICS_ENABLED;
|
||||
|
||||
use crate::db::DB;
|
||||
|
||||
@@ -12,6 +13,26 @@ lazy_static::lazy_static! {
|
||||
"Histogram of webhook requests made"
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
pub static ref INSTANCE_EVENTS_WEBHOOK: Option<String> = std::env::var("INSTANCE_EVENTS_WEBHOOK").ok();
|
||||
|
||||
}
|
||||
|
||||
pub enum WebhookPayload {
|
||||
WorkspaceEvent(String, WebhookMessage),
|
||||
InstanceEvent(InstanceEvent),
|
||||
}
|
||||
|
||||
#[derive(Serialize)]
|
||||
#[serde(tag = "type")]
|
||||
pub enum InstanceEvent {
|
||||
UserSignupOAuth { email: String },
|
||||
UserAdded { email: String },
|
||||
// UserDeleted { email: String },
|
||||
// UserDeletedWorkspace { workspace: String, email: String },
|
||||
UserAddedWorkspace { workspace: String, email: String },
|
||||
UserInvitedWorkspace { workspace: String, email: String },
|
||||
UserJoinedWorkspace { workspace: String, email: String, username: String },
|
||||
}
|
||||
|
||||
#[derive(Serialize)]
|
||||
@@ -45,12 +66,12 @@ pub enum WebhookMessage {
|
||||
|
||||
#[derive(Clone)]
|
||||
pub struct WebhookShared {
|
||||
pub channel: mpsc::UnboundedSender<(String, WebhookMessage)>,
|
||||
pub channel: mpsc::UnboundedSender<WebhookPayload>,
|
||||
}
|
||||
|
||||
impl WebhookShared {
|
||||
pub fn new(mut shutdown_rx: tokio::sync::broadcast::Receiver<()>, db: DB) -> Self {
|
||||
let (tx, mut rx) = mpsc::unbounded_channel::<(String, WebhookMessage)>();
|
||||
let (tx, mut rx) = mpsc::unbounded_channel::<WebhookPayload>();
|
||||
let _process = tokio::spawn(async move {
|
||||
let client = reqwest::Client::builder()
|
||||
// TODO: investigate pool timeouts and such if TCP load is high
|
||||
@@ -65,7 +86,7 @@ impl WebhookShared {
|
||||
biased;
|
||||
_ = shutdown_rx.recv() => break,
|
||||
r = rx.recv() => match r {
|
||||
Some((workspace_id, message)) => {
|
||||
Some(WebhookPayload::WorkspaceEvent(workspace_id, message)) => {
|
||||
let url_guard = match cache.get(&workspace_id).await {
|
||||
Some(guard) => {
|
||||
guard
|
||||
@@ -89,12 +110,19 @@ impl WebhookShared {
|
||||
};
|
||||
let webook_opt = url_guard.value();
|
||||
if let Some(url) = webook_opt {
|
||||
let timer = WEBHOOK_REQUEST_COUNT.start_timer();
|
||||
let timer = if *METRICS_ENABLED { Some(WEBHOOK_REQUEST_COUNT.start_timer()) } else { None };
|
||||
let _ = client.post(url).json(&message).send().await;
|
||||
timer.stop_and_record();
|
||||
timer.map(|x| x.stop_and_record());
|
||||
drop(url_guard);
|
||||
}
|
||||
},
|
||||
Some(WebhookPayload::InstanceEvent(event)) => {
|
||||
if *METRICS_ENABLED { Some(WEBHOOK_REQUEST_COUNT.start_timer()) } else { None };
|
||||
let r = client.post(INSTANCE_EVENTS_WEBHOOK.as_ref().unwrap()).json(&event).send().await;
|
||||
if let Err(e) = r {
|
||||
tracing::error!("Error sending instance event: {}", e);
|
||||
}
|
||||
},
|
||||
None => break,
|
||||
},
|
||||
_ = futures::future::poll_fn(|cx| cache_purge_interval.poll_tick(cx)) => {
|
||||
@@ -109,6 +137,16 @@ impl WebhookShared {
|
||||
}
|
||||
|
||||
pub fn send_message(&self, workspace_id: String, message: WebhookMessage) {
|
||||
let _ = self.channel.send((workspace_id.clone(), message));
|
||||
let _ = self.channel.send(WebhookPayload::WorkspaceEvent(
|
||||
workspace_id.clone(),
|
||||
message,
|
||||
));
|
||||
}
|
||||
|
||||
pub fn send_instance_event(&self, event: InstanceEvent) {
|
||||
if INSTANCE_EVENTS_WEBHOOK.is_none() {
|
||||
return;
|
||||
}
|
||||
let _ = self.channel.send(WebhookPayload::InstanceEvent(event));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -16,10 +16,10 @@ use crate::{
|
||||
db::{UserDB, DB},
|
||||
folders::Folder,
|
||||
resources::{Resource, ResourceType},
|
||||
users::{Authed, WorkspaceInvite, NEW_USER_WEBHOOK, VALID_USERNAME},
|
||||
users::{Authed, WorkspaceInvite, VALID_USERNAME},
|
||||
utils::require_super_admin,
|
||||
variables::build_crypt,
|
||||
HTTP_CLIENT,
|
||||
webhook_util::{InstanceEvent, WebhookShared},
|
||||
};
|
||||
#[cfg(feature = "enterprise")]
|
||||
use axum::response::Redirect;
|
||||
@@ -943,6 +943,7 @@ pub async fn invite_user_to_all_auto_invite_worspaces(db: &DB, email: &str) -> R
|
||||
async fn invite_user(
|
||||
Authed { username, is_admin, .. }: Authed,
|
||||
Extension(db): Extension<DB>,
|
||||
Extension(webhook): Extension<WebhookShared>,
|
||||
Path(w_id): Path<String>,
|
||||
Json(mut nu): Json<NewWorkspaceInvite>,
|
||||
) -> Result<(StatusCode, String)> {
|
||||
@@ -966,14 +967,10 @@ async fn invite_user(
|
||||
|
||||
tx.commit().await?;
|
||||
|
||||
if let Some(new_user_webhook) = NEW_USER_WEBHOOK.clone() {
|
||||
let _ = &HTTP_CLIENT
|
||||
.post(&new_user_webhook)
|
||||
.json(&serde_json::json!({"email" : &nu.email, "event": "workspace_invite"}))
|
||||
.send()
|
||||
.await
|
||||
.map_err(|e| tracing::error!("Error sending new user webhook: {}", e.to_string()));
|
||||
}
|
||||
webhook.send_instance_event(InstanceEvent::UserInvitedWorkspace {
|
||||
email: nu.email.clone(),
|
||||
workspace: w_id,
|
||||
});
|
||||
|
||||
Ok((
|
||||
StatusCode::CREATED,
|
||||
@@ -984,6 +981,7 @@ async fn invite_user(
|
||||
async fn add_user(
|
||||
Authed { username, is_admin, .. }: Authed,
|
||||
Extension(db): Extension<DB>,
|
||||
Extension(webhook): Extension<WebhookShared>,
|
||||
Path(w_id): Path<String>,
|
||||
Json(mut nu): Json<NewWorkspaceUser>,
|
||||
) -> Result<(StatusCode, String)> {
|
||||
@@ -1022,14 +1020,10 @@ async fn add_user(
|
||||
|
||||
tx.commit().await?;
|
||||
|
||||
if let Some(new_user_webhook) = NEW_USER_WEBHOOK.clone() {
|
||||
let _ = HTTP_CLIENT
|
||||
.post(&new_user_webhook)
|
||||
.json(&serde_json::json!({"email" : &nu.email, "event": "workspace_add"}))
|
||||
.send()
|
||||
.await
|
||||
.map_err(|e| tracing::error!("Error sending new user webhook: {}", e.to_string()));
|
||||
}
|
||||
webhook.send_instance_event(InstanceEvent::UserAddedWorkspace {
|
||||
workspace: w_id.clone(),
|
||||
email: nu.email.clone(),
|
||||
});
|
||||
|
||||
Ok((
|
||||
StatusCode::CREATED,
|
||||
|
||||
@@ -41,8 +41,8 @@ pub struct AuditLog {
|
||||
}
|
||||
|
||||
#[tracing::instrument(level = "trace", skip_all)]
|
||||
pub async fn audit_log<'c>(
|
||||
db: &mut Transaction<'c, Postgres>,
|
||||
pub async fn audit_log<'c, E: sqlx::Executor<'c, Database = Postgres>>(
|
||||
db: E,
|
||||
username: &str,
|
||||
operation: &str,
|
||||
action_kind: ActionKind,
|
||||
|
||||
@@ -30,6 +30,17 @@ pub const DEFAULT_MAX_CONNECTIONS_SERVER: u32 = 50;
|
||||
pub const DEFAULT_MAX_CONNECTIONS_WORKER: u32 = 3;
|
||||
|
||||
lazy_static::lazy_static! {
|
||||
pub static ref METRICS_ADDR: Option<SocketAddr> = std::env::var("METRICS_ADDR")
|
||||
.ok()
|
||||
.map(|s| {
|
||||
s.parse::<bool>()
|
||||
.map(|b| b.then(|| SocketAddr::from(([0, 0, 0, 0], 8001))))
|
||||
.or_else(|_| s.parse::<SocketAddr>().map(Some))
|
||||
})
|
||||
.transpose().ok()
|
||||
.flatten()
|
||||
.flatten();
|
||||
pub static ref METRICS_ENABLED: bool = METRICS_ADDR.is_some();
|
||||
pub static ref BASE_URL: String = std::env::var("BASE_URL").unwrap_or_else(|_| "http://localhost".to_string());
|
||||
pub static ref IS_READY: Arc<std::sync::atomic::AtomicBool> = Arc::new(std::sync::atomic::AtomicBool::new(false));
|
||||
}
|
||||
|
||||
@@ -17,7 +17,7 @@ use serde_json::to_string_pretty;
|
||||
|
||||
use crate::utils::StripPath;
|
||||
|
||||
#[derive(Serialize, Deserialize, Debug, PartialEq, Clone, Hash)]
|
||||
#[derive(Serialize, Deserialize, Debug, PartialEq, Clone, Hash, Eq)]
|
||||
#[cfg_attr(feature = "sqlx", derive(sqlx::Type))]
|
||||
#[cfg_attr(
|
||||
feature = "sqlx",
|
||||
|
||||
@@ -6,10 +6,10 @@
|
||||
* LICENSE-AGPL for a copy of the license.
|
||||
*/
|
||||
|
||||
use rand::{distributions::Alphanumeric, thread_rng, Rng};
|
||||
use serde::Deserialize;
|
||||
|
||||
use crate::error::{Error, Result};
|
||||
use rand::{distributions::Alphanumeric, thread_rng, Rng};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use sha2::{Digest, Sha256};
|
||||
|
||||
pub const MAX_PER_PAGE: usize = 10000;
|
||||
pub const DEFAULT_PER_PAGE: usize = 1000;
|
||||
@@ -19,7 +19,8 @@ pub struct Pagination {
|
||||
pub page: Option<usize>,
|
||||
pub per_page: Option<usize>,
|
||||
}
|
||||
#[derive(Deserialize)]
|
||||
|
||||
#[derive(Debug, Serialize, Deserialize)]
|
||||
pub struct StripPath(pub String);
|
||||
|
||||
impl StripPath {
|
||||
@@ -51,8 +52,8 @@ pub fn paginate(pagination: Pagination) -> (usize, usize) {
|
||||
}
|
||||
|
||||
#[cfg(feature = "sqlx")]
|
||||
pub async fn now_from_db<'c>(
|
||||
db: &mut sqlx::Transaction<'c, sqlx::Postgres>,
|
||||
pub async fn now_from_db<'c, E: sqlx::PgExecutor<'c>>(
|
||||
db: E,
|
||||
) -> Result<chrono::DateTime<chrono::Utc>> {
|
||||
Ok(sqlx::query_scalar!("SELECT now()")
|
||||
.fetch_one(db)
|
||||
@@ -118,3 +119,9 @@ pub fn rd_string(len: usize) -> String {
|
||||
.map(char::from)
|
||||
.collect()
|
||||
}
|
||||
|
||||
pub fn calculate_hash(s: &str) -> String {
|
||||
let mut hasher = Sha256::new();
|
||||
hasher.update(s);
|
||||
format!("{:x}", hasher.finalize())
|
||||
}
|
||||
|
||||
@@ -31,3 +31,6 @@ reqwest.workspace = true
|
||||
lazy_static.workspace = true
|
||||
prometheus.workspace = true
|
||||
cron.workspace = true
|
||||
rsmq_async.workspace = true
|
||||
tokio.workspace = true
|
||||
futures-core.workspace = true
|
||||
|
||||
@@ -22,8 +22,11 @@ use windmill_common::{
|
||||
flows::{FlowModule, FlowModuleValue, FlowValue},
|
||||
scripts::{get_full_hub_script_by_path, HubScript, ScriptHash, ScriptLang},
|
||||
utils::StripPath,
|
||||
METRICS_ENABLED,
|
||||
};
|
||||
|
||||
use crate::QueueTransaction;
|
||||
|
||||
lazy_static::lazy_static! {
|
||||
pub static ref HTTP_CLIENT: Client = reqwest::ClientBuilder::new()
|
||||
.user_agent("windmill/beta")
|
||||
@@ -51,13 +54,15 @@ lazy_static::lazy_static! {
|
||||
|
||||
const MAX_FREE_EXECS: i32 = 1000;
|
||||
const MAX_FREE_CONCURRENT_RUNS: i32 = 15;
|
||||
const RSMQ_MAIN_QUEUE: &'static str = "main_queue";
|
||||
|
||||
pub async fn cancel_job<'c>(
|
||||
pub async fn cancel_job<'c, R: rsmq_async::RsmqConnection + Clone>(
|
||||
username: &str,
|
||||
reason: Option<String>,
|
||||
id: Uuid,
|
||||
w_id: &str,
|
||||
mut tx: Transaction<'c, Postgres>,
|
||||
rsmq: Option<R>,
|
||||
force_rerun: bool,
|
||||
) -> error::Result<(Transaction<'c, Postgres>, Option<Uuid>)> {
|
||||
let job_option = sqlx::query_scalar!(
|
||||
@@ -71,6 +76,12 @@ pub async fn cancel_job<'c>(
|
||||
)
|
||||
.fetch_optional(&mut tx)
|
||||
.await?;
|
||||
if let Some(mut rsmq) = rsmq {
|
||||
rsmq.change_message_visibility(RSMQ_MAIN_QUEUE, &id.to_string(), 0)
|
||||
.await
|
||||
.map_err(|e| anyhow::anyhow!(e))?;
|
||||
}
|
||||
|
||||
let mut jobs = job_option.map(|j| vec![j]).unwrap_or_default();
|
||||
while !jobs.is_empty() {
|
||||
let p_job = jobs.pop();
|
||||
@@ -90,10 +101,11 @@ pub async fn cancel_job<'c>(
|
||||
Ok((tx, job_option))
|
||||
}
|
||||
|
||||
pub async fn pull(
|
||||
pub async fn pull<R: rsmq_async::RsmqConnection + Clone>(
|
||||
db: &Pool<Postgres>,
|
||||
whitelist_workspaces: Option<Vec<String>>,
|
||||
blacklist_workspaces: Option<Vec<String>>,
|
||||
rsmq: Option<R>,
|
||||
) -> windmill_common::error::Result<Option<QueuedJob>> {
|
||||
let mut workspaces_filter = String::new();
|
||||
if let Some(whitelist) = whitelist_workspaces {
|
||||
@@ -105,6 +117,9 @@ pub async fn pull(
|
||||
.collect::<Vec<String>>()
|
||||
.join(",")
|
||||
));
|
||||
if let Some(_rsmq) = rsmq {
|
||||
todo!("REDIS: Implement workspace filters for redis");
|
||||
}
|
||||
}
|
||||
if let Some(blacklist) = blacklist_workspaces {
|
||||
workspaces_filter.push_str(&format!(
|
||||
@@ -115,16 +130,50 @@ pub async fn pull(
|
||||
.collect::<Vec<String>>()
|
||||
.join(",")
|
||||
));
|
||||
if let Some(_rsmq) = rsmq {
|
||||
todo!("REDIS: Implement workspace filters for redis");
|
||||
}
|
||||
}
|
||||
/* Jobs can be started if they:
|
||||
* - haven't been started before,
|
||||
* running = false
|
||||
* - are flows with a step that needed resume,
|
||||
* suspend_until is non-null
|
||||
* and suspend = 0 when the resume messages are received
|
||||
* or suspend_until <= now() if it has timed out */
|
||||
let job: Option<QueuedJob> = sqlx::query_as::<_, QueuedJob>(&format!(
|
||||
"UPDATE queue
|
||||
|
||||
let job: Option<QueuedJob> = if let Some(mut rsmq) = rsmq {
|
||||
// TODO: REDIS: Race conditions / replace last_ping
|
||||
let msg = rsmq
|
||||
.pop_message::<Vec<u8>>(RSMQ_MAIN_QUEUE)
|
||||
.await
|
||||
.map_err(|e| anyhow::anyhow!(e))?;
|
||||
|
||||
if let Some(msg) = msg {
|
||||
let uuid = Uuid::from_bytes_le(
|
||||
msg.message
|
||||
.try_into()
|
||||
.map_err(|_| anyhow::anyhow!("Failed to parsed Redis message"))?,
|
||||
);
|
||||
|
||||
sqlx::query_as::<_, QueuedJob>(
|
||||
"UPDATE queue
|
||||
SET running = true
|
||||
, started_at = coalesce(started_at, now())
|
||||
, last_ping = now()
|
||||
, suspend_until = null
|
||||
WHERE id = $1
|
||||
RETURNING *",
|
||||
)
|
||||
.bind(uuid)
|
||||
.fetch_optional(db)
|
||||
.await?
|
||||
} else {
|
||||
None
|
||||
}
|
||||
} else {
|
||||
/* Jobs can be started if they:
|
||||
* - haven't been started before,
|
||||
* running = false
|
||||
* - are flows with a step that needed resume,
|
||||
* suspend_until is non-null
|
||||
* and suspend = 0 when the resume messages are received
|
||||
* or suspend_until <= now() if it has timed out */
|
||||
sqlx::query_as::<_, QueuedJob>(&format!(
|
||||
"UPDATE queue
|
||||
SET running = true
|
||||
, started_at = coalesce(started_at, now())
|
||||
, last_ping = now()
|
||||
@@ -142,11 +191,12 @@ pub async fn pull(
|
||||
LIMIT 1
|
||||
)
|
||||
RETURNING *"
|
||||
))
|
||||
.fetch_optional(db)
|
||||
.await?;
|
||||
))
|
||||
.fetch_optional(db)
|
||||
.await?
|
||||
};
|
||||
|
||||
if job.is_some() {
|
||||
if job.is_some() && *METRICS_ENABLED {
|
||||
QUEUE_PULL_COUNT.inc();
|
||||
}
|
||||
|
||||
@@ -206,24 +256,26 @@ pub async fn get_result_by_id(
|
||||
}
|
||||
|
||||
#[instrument(level = "trace", skip_all)]
|
||||
pub async fn delete_job(
|
||||
db: &Pool<Postgres>,
|
||||
pub async fn delete_job<'c, R: rsmq_async::RsmqConnection + Clone + Send>(
|
||||
mut tx: QueueTransaction<'c, R>,
|
||||
w_id: &str,
|
||||
job_id: Uuid,
|
||||
) -> windmill_common::error::Result<()> {
|
||||
QUEUE_DELETE_COUNT.inc();
|
||||
) -> windmill_common::error::Result<QueueTransaction<'c, R>> {
|
||||
if *METRICS_ENABLED {
|
||||
QUEUE_DELETE_COUNT.inc();
|
||||
}
|
||||
let job_removed = sqlx::query_scalar!(
|
||||
"DELETE FROM queue WHERE workspace_id = $1 AND id = $2 RETURNING 1",
|
||||
w_id,
|
||||
job_id
|
||||
)
|
||||
.fetch_one(db)
|
||||
.fetch_one(&mut tx)
|
||||
.await
|
||||
.map_err(|e| Error::InternalErr(format!("Error during deletion of job {job_id}: {e}")))?
|
||||
.unwrap_or(0)
|
||||
== 1;
|
||||
tracing::debug!("Job {job_id} deleted: {job_removed}");
|
||||
Ok(())
|
||||
Ok(tx)
|
||||
}
|
||||
|
||||
pub async fn get_queued_job<'c>(
|
||||
@@ -242,9 +294,9 @@ pub async fn get_queued_job<'c>(
|
||||
Ok(r)
|
||||
}
|
||||
|
||||
#[instrument(level = "trace", skip_all)]
|
||||
pub async fn push<'c>(
|
||||
mut tx: Transaction<'c, Postgres>,
|
||||
// #[instrument(level = "trace", skip_all)]
|
||||
pub async fn push<'c, R: rsmq_async::RsmqConnection + Send + 'c>(
|
||||
mut tx: QueueTransaction<'c, R>,
|
||||
workspace_id: &str,
|
||||
job_payload: JobPayload,
|
||||
args: serde_json::Map<String, serde_json::Value>,
|
||||
@@ -259,8 +311,7 @@ pub async fn push<'c>(
|
||||
mut same_worker: bool,
|
||||
pre_run_error: Option<&windmill_common::error::Error>,
|
||||
visible_to_owner: bool,
|
||||
) -> Result<(Uuid, Transaction<'c, Postgres>), Error> {
|
||||
let scheduled_for = scheduled_for_o.unwrap_or_else(chrono::Utc::now);
|
||||
) -> Result<(Uuid, QueueTransaction<'c, R>), Error> {
|
||||
let args_json = serde_json::Value::Object(args);
|
||||
let job_id: Uuid = Ulid::new().into();
|
||||
|
||||
@@ -512,7 +563,7 @@ pub async fn push<'c>(
|
||||
(workspace_id, id, running, parent_job, created_by, permissioned_as, scheduled_for,
|
||||
script_hash, script_path, raw_code, raw_lock, args, job_kind, schedule_path, raw_flow, \
|
||||
flow_status, is_flow_step, language, started_at, same_worker, pre_run_error, email, visible_to_owner, root_job)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, CASE WHEN $3 THEN now() END, $19, $20, $21, $22, $23) \
|
||||
VALUES ($1, $2, $3, $4, $5, $6, COALESCE($7, now()), $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, CASE WHEN $3 THEN now() END, $19, $20, $21, $22, $23) \
|
||||
RETURNING id",
|
||||
workspace_id,
|
||||
job_id,
|
||||
@@ -520,7 +571,7 @@ pub async fn push<'c>(
|
||||
parent_job,
|
||||
user,
|
||||
permissioned_as,
|
||||
scheduled_for,
|
||||
scheduled_for_o,
|
||||
script_hash,
|
||||
script_path.clone(),
|
||||
raw_code,
|
||||
@@ -542,7 +593,9 @@ pub async fn push<'c>(
|
||||
.await
|
||||
.map_err(|e| Error::InternalErr(format!("Could not insert into queue {job_id}: {e}")))?;
|
||||
// TODO: technically the job isn't queued yet, as the transaction can be rolled back. Should be solved when moving these metrics to the queue abstraction.
|
||||
QUEUE_PUSH_COUNT.inc();
|
||||
if *METRICS_ENABLED {
|
||||
QUEUE_PUSH_COUNT.inc();
|
||||
}
|
||||
|
||||
{
|
||||
let uuid_string = job_id.to_string();
|
||||
@@ -577,6 +630,10 @@ pub async fn push<'c>(
|
||||
.instrument(tracing::info_span!("job_run", email = &email))
|
||||
.await?;
|
||||
}
|
||||
if let Some(ref mut rsmq) = tx.rsmq {
|
||||
rsmq.send_message(job_id.to_bytes_le().to_vec(), scheduled_for_o);
|
||||
}
|
||||
|
||||
Ok((uuid, tx))
|
||||
}
|
||||
|
||||
|
||||
@@ -7,6 +7,8 @@
|
||||
*/
|
||||
|
||||
mod jobs;
|
||||
mod queue_transaction;
|
||||
pub mod schedule;
|
||||
|
||||
pub use jobs::*;
|
||||
pub use queue_transaction::*;
|
||||
|
||||
161
backend/windmill-queue/src/queue_transaction.rs
Normal file
161
backend/windmill-queue/src/queue_transaction.rs
Normal file
@@ -0,0 +1,161 @@
|
||||
use std::fmt::Debug;
|
||||
|
||||
use futures_core::{future::BoxFuture, stream::BoxStream};
|
||||
use rsmq_async::{RedisBytes, RsmqConnection};
|
||||
use sqlx::{Postgres, Transaction};
|
||||
|
||||
pub enum RedisOp {
|
||||
SendMessage(RedisBytes, Option<chrono::DateTime<chrono::Utc>>),
|
||||
DeleteMessage(String),
|
||||
}
|
||||
|
||||
impl RedisOp {
|
||||
pub async fn apply<R: RsmqConnection>(self, rsmq: &mut R) -> Result<(), rsmq_async::RsmqError> {
|
||||
match self {
|
||||
RedisOp::SendMessage(bytes, time) => {
|
||||
rsmq.send_message(
|
||||
"main_queue",
|
||||
bytes,
|
||||
time.map(|t| (t - chrono::Utc::now()).num_seconds())
|
||||
.and_then(|e| e.try_into().ok()),
|
||||
)
|
||||
.await?;
|
||||
}
|
||||
RedisOp::DeleteMessage(id) => {
|
||||
rsmq.delete_message("main_queue", &id).await?;
|
||||
}
|
||||
};
|
||||
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
pub struct RedisTransaction<R: RsmqConnection> {
|
||||
rsmq: R,
|
||||
queued_ops: Vec<RedisOp>,
|
||||
}
|
||||
|
||||
impl<R: RsmqConnection> From<R> for RedisTransaction<R> {
|
||||
fn from(value: R) -> Self {
|
||||
Self { rsmq: value, queued_ops: Vec::new() }
|
||||
}
|
||||
}
|
||||
|
||||
impl<R: RsmqConnection> RedisTransaction<R> {
|
||||
pub async fn commit(self) -> Result<(), rsmq_async::RsmqError> {
|
||||
let mut rsmq = self.rsmq;
|
||||
for op in self.queued_ops {
|
||||
op.apply(&mut rsmq).await?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn send_message<E: Into<RedisBytes>>(
|
||||
&mut self,
|
||||
bytes: E,
|
||||
delay_until: Option<chrono::DateTime<chrono::Utc>>,
|
||||
) {
|
||||
self.queued_ops
|
||||
.push(RedisOp::SendMessage(bytes.into(), delay_until))
|
||||
}
|
||||
|
||||
pub fn delete_message(&mut self, id: String) {
|
||||
self.queued_ops.push(RedisOp::DeleteMessage(id))
|
||||
}
|
||||
}
|
||||
|
||||
pub struct QueueTransaction<'c, R: RsmqConnection> {
|
||||
pub rsmq: Option<RedisTransaction<R>>,
|
||||
transaction: Transaction<'c, Postgres>,
|
||||
}
|
||||
|
||||
impl<'c, R: RsmqConnection> From<(Option<R>, Transaction<'c, Postgres>)>
|
||||
for QueueTransaction<'c, R>
|
||||
{
|
||||
fn from(value: (Option<R>, Transaction<'c, Postgres>)) -> Self {
|
||||
Self { rsmq: value.0.map(|e| e.into()), transaction: value.1 }
|
||||
}
|
||||
}
|
||||
|
||||
impl<'c, R: RsmqConnection> Debug for QueueTransaction<'c, R> {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.debug_struct("QueueTransaction")
|
||||
.field("rsmq", &self.rsmq.as_ref().map(|_| ())) // do not require R: Debug
|
||||
.field("transaction", &self.transaction)
|
||||
.finish()
|
||||
}
|
||||
}
|
||||
|
||||
impl<'c, R: RsmqConnection> QueueTransaction<'c, R> {
|
||||
pub async fn commit(self) -> Result<(), windmill_common::error::Error> {
|
||||
self.transaction.commit().await?;
|
||||
if let Some(rsmq) = self.rsmq {
|
||||
rsmq.commit().await.map_err(|e| anyhow::anyhow!(e))?;
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn transaction_mut<'a>(&'a mut self) -> &'a mut Transaction<'c, Postgres> {
|
||||
&mut self.transaction
|
||||
}
|
||||
}
|
||||
|
||||
impl<'c, 'b, R: RsmqConnection + Send> sqlx::Executor<'b> for &'b mut QueueTransaction<'c, R> {
|
||||
type Database = Postgres;
|
||||
|
||||
fn fetch_many<'e, 'q: 'e, E: 'q>(
|
||||
self,
|
||||
query: E,
|
||||
) -> BoxStream<
|
||||
'e,
|
||||
Result<
|
||||
sqlx::Either<
|
||||
<Self::Database as sqlx::Database>::QueryResult,
|
||||
<Self::Database as sqlx::Database>::Row,
|
||||
>,
|
||||
sqlx::Error,
|
||||
>,
|
||||
>
|
||||
where
|
||||
'b: 'e,
|
||||
E: sqlx::Execute<'q, Self::Database>,
|
||||
{
|
||||
self.transaction.fetch_many(query)
|
||||
}
|
||||
|
||||
fn fetch_optional<'e, 'q: 'e, E: 'q>(
|
||||
self,
|
||||
query: E,
|
||||
) -> BoxFuture<'e, Result<Option<<Self::Database as sqlx::Database>::Row>, sqlx::Error>>
|
||||
where
|
||||
'b: 'e,
|
||||
E: sqlx::Execute<'q, Self::Database>,
|
||||
{
|
||||
self.transaction.fetch_optional(query)
|
||||
}
|
||||
|
||||
fn prepare_with<'e, 'q: 'e>(
|
||||
self,
|
||||
sql: &'q str,
|
||||
parameters: &'e [<Self::Database as sqlx::Database>::TypeInfo],
|
||||
) -> BoxFuture<
|
||||
'e,
|
||||
Result<<Self::Database as sqlx::database::HasStatement<'q>>::Statement, sqlx::Error>,
|
||||
>
|
||||
where
|
||||
'b: 'e,
|
||||
{
|
||||
self.transaction.prepare_with(sql, parameters)
|
||||
}
|
||||
|
||||
fn describe<'e, 'q: 'e>(
|
||||
self,
|
||||
sql: &'q str,
|
||||
) -> BoxFuture<'e, Result<sqlx::Describe<Self::Database>, sqlx::Error>>
|
||||
where
|
||||
'b: 'e,
|
||||
{
|
||||
self.transaction.describe(sql)
|
||||
}
|
||||
}
|
||||
@@ -15,11 +15,12 @@ use windmill_common::{
|
||||
users::username_to_permissioned_as,
|
||||
utils::{now_from_db, StripPath},
|
||||
};
|
||||
use crate::{QueueTransaction};
|
||||
|
||||
pub async fn push_scheduled_job<'c>(
|
||||
mut tx: Transaction<'c, Postgres>,
|
||||
pub async fn push_scheduled_job<'c, R: rsmq_async::RsmqConnection + Send + 'c>(
|
||||
mut tx: QueueTransaction<'c, R>,
|
||||
schedule: Schedule,
|
||||
) -> Result<Transaction<'c, Postgres>> {
|
||||
) -> Result<QueueTransaction<'c, R>> {
|
||||
let sched = cron::Schedule::from_str(&schedule.schedule)
|
||||
.map_err(|e| error::Error::BadRequest(e.to_string()))?;
|
||||
|
||||
@@ -70,7 +71,7 @@ pub async fn push_scheduled_job<'c>(
|
||||
} else {
|
||||
JobPayload::ScriptHash {
|
||||
hash: windmill_common::get_latest_hash_for_path(
|
||||
&mut tx,
|
||||
tx.transaction_mut(),
|
||||
&schedule.workspace_id,
|
||||
&schedule.script_path,
|
||||
)
|
||||
@@ -79,7 +80,15 @@ pub async fn push_scheduled_job<'c>(
|
||||
}
|
||||
};
|
||||
|
||||
let (_, mut tx) = push(
|
||||
sqlx::query!(
|
||||
"UPDATE schedule SET error = NULL WHERE workspace_id = $1 AND path = $2",
|
||||
&schedule.workspace_id,
|
||||
&schedule.path
|
||||
)
|
||||
.execute(&mut tx)
|
||||
.await?;
|
||||
|
||||
let (_, tx) = push(
|
||||
tx,
|
||||
&schedule.workspace_id,
|
||||
payload,
|
||||
@@ -97,14 +106,7 @@ pub async fn push_scheduled_job<'c>(
|
||||
true,
|
||||
)
|
||||
.await?;
|
||||
sqlx::query!(
|
||||
"UPDATE schedule SET error = NULL WHERE workspace_id = $1 AND path = $2",
|
||||
&schedule.workspace_id,
|
||||
&schedule.path
|
||||
)
|
||||
.execute(&mut tx)
|
||||
.await?;
|
||||
Ok(tx)
|
||||
Ok(tx) // TODO: Bubble up pushed UUID from here
|
||||
}
|
||||
|
||||
pub async fn get_schedule_opt<'c>(
|
||||
|
||||
@@ -48,4 +48,5 @@ deno_core.workspace = true
|
||||
const_format.workspace = true
|
||||
git-version.workspace = true
|
||||
dyn-iter.workspace = true
|
||||
once_cell.workspace = true
|
||||
once_cell.workspace = true
|
||||
rsmq_async.workspace = true
|
||||
|
||||
@@ -6,23 +6,30 @@
|
||||
* LICENSE-AGPL for a copy of the license.
|
||||
*/
|
||||
|
||||
use sqlx::{Pool, Postgres, Transaction};
|
||||
use sqlx::{Pool, Postgres};
|
||||
use tracing::instrument;
|
||||
use uuid::Uuid;
|
||||
use windmill_common::{error::Error, flow_status::FlowStatusModule, schedule::Schedule};
|
||||
use windmill_queue::{delete_job, schedule::get_schedule_opt, JobKind, QueuedJob, CLOUD_HOSTED};
|
||||
use windmill_common::{
|
||||
error::Error, flow_status::FlowStatusModule, schedule::Schedule, METRICS_ENABLED,
|
||||
};
|
||||
use windmill_queue::{
|
||||
delete_job, schedule::get_schedule_opt, JobKind, QueueTransaction, QueuedJob, CLOUD_HOSTED,
|
||||
};
|
||||
|
||||
#[instrument(level = "trace", skip_all)]
|
||||
pub async fn add_completed_job_error(
|
||||
pub async fn add_completed_job_error<R: rsmq_async::RsmqConnection + Clone + Send>(
|
||||
db: &Pool<Postgres>,
|
||||
queued_job: &QueuedJob,
|
||||
logs: String,
|
||||
e: serde_json::Value,
|
||||
metrics: Option<crate::worker::Metrics>,
|
||||
rsmq: Option<R>,
|
||||
) -> Result<serde_json::Value, Error> {
|
||||
metrics.map(|m| m.worker_execution_failed.inc());
|
||||
if *METRICS_ENABLED {
|
||||
metrics.map(|m| m.worker_execution_failed.inc());
|
||||
}
|
||||
let result = serde_json::json!({ "error": e });
|
||||
let _ = add_completed_job(db, &queued_job, false, false, result.clone(), logs).await?;
|
||||
let _ = add_completed_job(db, &queued_job, false, false, result.clone(), logs, rsmq).await?;
|
||||
Ok(result)
|
||||
}
|
||||
|
||||
@@ -45,13 +52,14 @@ fn flatten_jobs(modules: Vec<FlowStatusModule>) -> Vec<Uuid> {
|
||||
}
|
||||
|
||||
#[instrument(level = "trace", skip_all)]
|
||||
pub async fn add_completed_job(
|
||||
pub async fn add_completed_job<R: rsmq_async::RsmqConnection + Clone + Send>(
|
||||
db: &Pool<Postgres>,
|
||||
queued_job: &QueuedJob,
|
||||
success: bool,
|
||||
skipped: bool,
|
||||
result: serde_json::Value,
|
||||
logs: String,
|
||||
rsmq: Option<R>,
|
||||
) -> Result<Uuid, Error> {
|
||||
let duration =
|
||||
if queued_job.job_kind == JobKind::Flow || queued_job.job_kind == JobKind::FlowPreview {
|
||||
@@ -83,7 +91,7 @@ pub async fn add_completed_job(
|
||||
.ok()
|
||||
.flatten()
|
||||
.flatten();
|
||||
let mut tx = db.begin().await?;
|
||||
let mut tx: QueueTransaction<'_, R> = (rsmq, db.begin().await?).into();
|
||||
let job_id = queued_job.id.clone();
|
||||
sqlx::query!(
|
||||
"INSERT INTO completed_job AS cj
|
||||
@@ -153,7 +161,7 @@ pub async fn add_completed_job(
|
||||
.execute(&mut tx)
|
||||
.await
|
||||
.map_err(|e| Error::InternalErr(format!("Could not add completed job {job_id}: {e}")))?;
|
||||
let _ = delete_job(db, &queued_job.workspace_id, job_id).await?;
|
||||
tx = delete_job(tx, &queued_job.workspace_id, job_id).await?;
|
||||
if !queued_job.is_flow_step
|
||||
&& queued_job.job_kind != JobKind::Flow
|
||||
&& queued_job.job_kind != JobKind::FlowPreview
|
||||
@@ -208,21 +216,24 @@ pub async fn add_completed_job(
|
||||
}
|
||||
|
||||
#[instrument(level = "trace", skip_all)]
|
||||
pub async fn schedule_again_if_scheduled<'c>(
|
||||
mut tx: Transaction<'c, Postgres>,
|
||||
pub async fn schedule_again_if_scheduled<'c, R: rsmq_async::RsmqConnection + Clone + Send + 'c>(
|
||||
mut tx: QueueTransaction<'c, R>,
|
||||
db: &Pool<Postgres>,
|
||||
schedule_path: &str,
|
||||
script_path: &str,
|
||||
w_id: &str,
|
||||
) -> windmill_common::error::Result<Transaction<'c, Postgres>> {
|
||||
let schedule = get_schedule_opt(&mut tx, w_id, schedule_path)
|
||||
.await?
|
||||
.ok_or_else(|| {
|
||||
Error::InternalErr(format!(
|
||||
"Could not find schedule {:?} for workspace {}",
|
||||
schedule_path, w_id
|
||||
))
|
||||
})?;
|
||||
) -> windmill_common::error::Result<QueueTransaction<'c, R>> {
|
||||
let schedule = get_schedule_opt(tx.transaction_mut(), w_id, schedule_path).await?;
|
||||
|
||||
if schedule.is_none() {
|
||||
tracing::error!(
|
||||
"Schedule {schedule_path} in {w_id} not found. Impossible to schedule again"
|
||||
);
|
||||
return Ok(tx);
|
||||
}
|
||||
|
||||
let schedule = schedule.unwrap();
|
||||
|
||||
if schedule.enabled && script_path == schedule.script_path {
|
||||
let res = windmill_queue::schedule::push_scheduled_job(
|
||||
tx,
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -31,11 +31,13 @@ use windmill_common::{
|
||||
type DB = sqlx::Pool<sqlx::Postgres>;
|
||||
|
||||
use windmill_queue::{
|
||||
canceled_job_to_result, get_queued_job, push, JobPayload, QueuedJob, RawCode,
|
||||
canceled_job_to_result, get_queued_job, push, JobPayload, QueueTransaction, QueuedJob, RawCode,
|
||||
};
|
||||
|
||||
// #[instrument(level = "trace", skip_all)]
|
||||
pub async fn update_flow_status_after_job_completion(
|
||||
pub async fn update_flow_status_after_job_completion<
|
||||
R: rsmq_async::RsmqConnection + Send + Sync + Clone,
|
||||
>(
|
||||
db: &DB,
|
||||
client: &AuthedClient,
|
||||
flow: uuid::Uuid,
|
||||
@@ -49,6 +51,7 @@ pub async fn update_flow_status_after_job_completion(
|
||||
worker_dir: &str,
|
||||
stop_early_override: Option<bool>,
|
||||
base_internal_url: &str,
|
||||
rsmq: Option<R>,
|
||||
) -> error::Result<()> {
|
||||
// this is manual tailrecursion because async_recursion blows up the stack
|
||||
let mut depth = 0;
|
||||
@@ -67,6 +70,7 @@ pub async fn update_flow_status_after_job_completion(
|
||||
stop_early_override,
|
||||
base_internal_url,
|
||||
depth,
|
||||
rsmq.clone(),
|
||||
)
|
||||
.await?;
|
||||
while let Some(nrec) = rec {
|
||||
@@ -86,6 +90,7 @@ pub async fn update_flow_status_after_job_completion(
|
||||
nrec.stop_early_override,
|
||||
base_internal_url,
|
||||
depth,
|
||||
rsmq.clone(),
|
||||
)
|
||||
.await?;
|
||||
}
|
||||
@@ -99,7 +104,9 @@ pub struct RecUpdateFlowStatusAfterJobCompletion {
|
||||
stop_early_override: Option<bool>,
|
||||
}
|
||||
// #[instrument(level = "trace", skip_all)]
|
||||
pub async fn update_flow_status_after_job_completion_internal(
|
||||
pub async fn update_flow_status_after_job_completion_internal<
|
||||
R: rsmq_async::RsmqConnection + Send + Sync + Clone,
|
||||
>(
|
||||
db: &DB,
|
||||
client: &AuthedClient,
|
||||
flow: uuid::Uuid,
|
||||
@@ -114,11 +121,12 @@ pub async fn update_flow_status_after_job_completion_internal(
|
||||
stop_early_override: Option<bool>,
|
||||
base_internal_url: &str,
|
||||
depth: u8,
|
||||
rsmq: Option<R>,
|
||||
) -> error::Result<Option<RecUpdateFlowStatusAfterJobCompletion>> {
|
||||
let (should_continue_flow, flow_job, stop_early, skip_if_stop_early, nresult) = {
|
||||
tracing::debug!("UPDATE FLOW STATUS: {flow:?} {success} {result:?} {w_id} {depth}");
|
||||
|
||||
let mut tx = db.begin().await?;
|
||||
let mut tx: QueueTransaction<'_, _> = (rsmq.clone(), db.begin().await?).into();
|
||||
|
||||
let old_status_json = sqlx::query_scalar!(
|
||||
"SELECT flow_status FROM queue WHERE id = $1 AND workspace_id = $2",
|
||||
@@ -154,7 +162,7 @@ pub async fn update_flow_status_after_job_completion_internal(
|
||||
module_status,
|
||||
FlowStatusModule::InProgress { iterator: Some(_), .. }
|
||||
) {
|
||||
compute_skip_loop_failures(flow, old_status.step, &mut tx)
|
||||
compute_skip_loop_failures(flow, old_status.step, tx.transaction_mut())
|
||||
.await?
|
||||
.unwrap_or(false)
|
||||
} else {
|
||||
@@ -197,9 +205,11 @@ pub async fn update_flow_status_after_job_completion_internal(
|
||||
FlowStatusModule::InProgress {
|
||||
branchall: Some(BranchAllStatus { branch, .. }),
|
||||
..
|
||||
} => compute_skip_branchall_failure(flow, old_status.step, *branch, &mut tx)
|
||||
.await?
|
||||
.unwrap_or(false),
|
||||
} => {
|
||||
compute_skip_branchall_failure(flow, old_status.step, *branch, tx.transaction_mut())
|
||||
.await?
|
||||
.unwrap_or(false)
|
||||
}
|
||||
_ => false,
|
||||
};
|
||||
|
||||
@@ -461,7 +471,7 @@ pub async fn update_flow_status_after_job_completion_internal(
|
||||
.context("remove flow status retry")?;
|
||||
}
|
||||
|
||||
let flow_job = get_queued_job(flow, w_id, &mut tx)
|
||||
let flow_job = get_queued_job(flow, w_id, tx.transaction_mut())
|
||||
.await?
|
||||
.ok_or_else(|| Error::InternalErr(format!("requiring flow to be in the queue")))?;
|
||||
|
||||
@@ -503,7 +513,9 @@ pub async fn update_flow_status_after_job_completion_internal(
|
||||
{
|
||||
true
|
||||
}
|
||||
false if has_failure_module(flow, &mut tx).await? && !is_failure_step => true,
|
||||
false if has_failure_module(flow, tx.transaction_mut()).await? && !is_failure_step => {
|
||||
true
|
||||
}
|
||||
false => false,
|
||||
};
|
||||
|
||||
@@ -548,6 +560,7 @@ pub async fn update_flow_status_after_job_completion_internal(
|
||||
logs,
|
||||
canceled_job_to_result(&flow_job),
|
||||
metrics.clone(),
|
||||
rsmq.clone(),
|
||||
)
|
||||
.await?;
|
||||
} else {
|
||||
@@ -558,6 +571,7 @@ pub async fn update_flow_status_after_job_completion_internal(
|
||||
stop_early && skip_if_stop_early,
|
||||
nresult.clone(),
|
||||
logs,
|
||||
rsmq.clone(),
|
||||
)
|
||||
.await?;
|
||||
}
|
||||
@@ -571,6 +585,7 @@ pub async fn update_flow_status_after_job_completion_internal(
|
||||
same_worker_tx.clone(),
|
||||
worker_dir,
|
||||
base_internal_url,
|
||||
rsmq.clone(),
|
||||
)
|
||||
.await
|
||||
{
|
||||
@@ -581,6 +596,7 @@ pub async fn update_flow_status_after_job_completion_internal(
|
||||
"Unexpected error during flow chaining:\n".to_string(),
|
||||
json!({"message": err.to_string(), "name": "InternalError"}),
|
||||
metrics.clone(),
|
||||
rsmq.clone(),
|
||||
)
|
||||
.await;
|
||||
true
|
||||
@@ -831,7 +847,7 @@ async fn transform_input(
|
||||
}
|
||||
|
||||
#[instrument(level = "trace", skip_all)]
|
||||
pub async fn handle_flow(
|
||||
pub async fn handle_flow<R: rsmq_async::RsmqConnection + Send + Sync + Clone>(
|
||||
flow_job: &QueuedJob,
|
||||
db: &sqlx::Pool<sqlx::Postgres>,
|
||||
client: &AuthedClient,
|
||||
@@ -839,6 +855,7 @@ pub async fn handle_flow(
|
||||
same_worker_tx: Sender<Uuid>,
|
||||
worker_dir: &str,
|
||||
base_internal_url: &str,
|
||||
rsmq: Option<R>,
|
||||
) -> anyhow::Result<()> {
|
||||
let value = flow_job
|
||||
.raw_flow
|
||||
@@ -862,6 +879,7 @@ pub async fn handle_flow(
|
||||
same_worker_tx,
|
||||
worker_dir,
|
||||
base_internal_url,
|
||||
rsmq,
|
||||
)
|
||||
.await?;
|
||||
Ok(())
|
||||
@@ -869,7 +887,7 @@ pub async fn handle_flow(
|
||||
|
||||
#[async_recursion]
|
||||
// #[instrument(level = "trace", skip_all)]
|
||||
async fn push_next_flow_job(
|
||||
async fn push_next_flow_job<R: rsmq_async::RsmqConnection + Send + Sync + Clone>(
|
||||
flow_job: &QueuedJob,
|
||||
mut status: FlowStatus,
|
||||
flow: FlowValue,
|
||||
@@ -879,6 +897,7 @@ async fn push_next_flow_job(
|
||||
same_worker_tx: Sender<Uuid>,
|
||||
worker_dir: &str,
|
||||
base_internal_url: &str,
|
||||
rsmq: Option<R>,
|
||||
) -> error::Result<()> {
|
||||
let job_root = flow_job
|
||||
.root_job
|
||||
@@ -915,6 +934,7 @@ async fn push_next_flow_job(
|
||||
worker_dir,
|
||||
None,
|
||||
base_internal_url,
|
||||
rsmq,
|
||||
)
|
||||
.await;
|
||||
}
|
||||
@@ -1085,7 +1105,7 @@ async fn push_next_flow_job(
|
||||
let logs = "Timed out waiting to be resumed".to_string();
|
||||
let result = json!({ "error": {"message": logs, "name": "SuspendedTimeout"}});
|
||||
let _uuid =
|
||||
add_completed_job(db, &flow_job, success, skipped, result, logs).await?;
|
||||
add_completed_job(db, &flow_job, success, skipped, result, logs, rsmq).await?;
|
||||
|
||||
return Ok(());
|
||||
}
|
||||
@@ -1286,7 +1306,7 @@ async fn push_next_flow_job(
|
||||
flow.same_worker && module.suspend.is_none() && module.sleep.is_none();
|
||||
|
||||
/* Finally, push the job into the queue */
|
||||
let mut tx = db.begin().await?;
|
||||
let mut tx = (rsmq.clone(), db.begin().await?).into();
|
||||
let mut uuids = vec![];
|
||||
|
||||
let len = match &job_payloads {
|
||||
@@ -1452,7 +1472,7 @@ async fn push_next_flow_job(
|
||||
json!(i),
|
||||
flow_job.id
|
||||
)
|
||||
.execute(&mut tx)
|
||||
.execute(db)
|
||||
.await?;
|
||||
} else {
|
||||
sqlx::query!(
|
||||
@@ -1468,7 +1488,7 @@ async fn push_next_flow_job(
|
||||
json!(i),
|
||||
flow_job.id
|
||||
)
|
||||
.execute(&mut tx)
|
||||
.execute(db)
|
||||
.await?;
|
||||
};
|
||||
|
||||
|
||||
@@ -1,53 +0,0 @@
|
||||
{
|
||||
"type": "RANDOM",
|
||||
"actions": [
|
||||
{
|
||||
"weight": 1,
|
||||
"action": {
|
||||
"type": "RANDOM",
|
||||
"actions": [
|
||||
{
|
||||
"weight": 1,
|
||||
"action": {
|
||||
"type": "PREVIEW_SCRIPT",
|
||||
"workspace": "demo",
|
||||
"language": "deno",
|
||||
"args": {},
|
||||
"content": "export async function main() { return \"Hello World\"; }"
|
||||
}
|
||||
},
|
||||
{
|
||||
"weight": 1,
|
||||
"action": {
|
||||
"type": "PREVIEW_SCRIPT",
|
||||
"workspace": "demo",
|
||||
"language": "python3",
|
||||
"args": {},
|
||||
"content": "def main(): return \"Hello World\";"
|
||||
}
|
||||
},
|
||||
{
|
||||
"weight": 1,
|
||||
"action": {
|
||||
"type": "PREVIEW_SCRIPT",
|
||||
"workspace": "demo",
|
||||
"language": "go",
|
||||
"args": {},
|
||||
"content": "func main() { return \"Hello World\" }"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"weight": 0.5,
|
||||
"action": {
|
||||
"type": "PREVIEW_SCRIPT",
|
||||
"workspace": "demo",
|
||||
"language": "deno",
|
||||
"args": {},
|
||||
"content": "import { delay } from \"https://deno.land/std@0.131.0/async/delay.ts\"; export async function main() { await delay(1000); return \"Hello World\"; }"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
7
benchmarks/identity.json
Normal file
7
benchmarks/identity.json
Normal file
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"type": "PREVIEW_SCRIPT",
|
||||
"workspace": "demo",
|
||||
"language": "deno",
|
||||
"args": {},
|
||||
"content": "export async function main() { return \"Hello World\"; }"
|
||||
}
|
||||
@@ -27,60 +27,60 @@ await new Command()
|
||||
"The number of workers to run at once.",
|
||||
{
|
||||
default: 1,
|
||||
}
|
||||
},
|
||||
)
|
||||
.option(
|
||||
"-s --seconds <seconds:number>",
|
||||
"How long to run the benchmark for (in seconds).",
|
||||
{
|
||||
default: 30,
|
||||
}
|
||||
},
|
||||
)
|
||||
.option("--max <max:number>", "Maximum number of operations performed.")
|
||||
.option("-e --email <email:string>", "The email to use to login.")
|
||||
.option("-p --password <password:string>", "The password to use to login.")
|
||||
.env(
|
||||
"WM_TOKEN=<token:string>",
|
||||
"The token to use when talking to the API server. Preferred over manual login."
|
||||
"The token to use when talking to the API server. Preferred over manual login.",
|
||||
)
|
||||
.option(
|
||||
"-t --token <token:string>",
|
||||
"The token to use when talking to the API server. Preferred over manual login."
|
||||
"The token to use when talking to the API server. Preferred over manual login.",
|
||||
)
|
||||
.env(
|
||||
"WM_WORKSPACE=<workspace:string>",
|
||||
"The workspace to spawn scripts from."
|
||||
"The workspace to spawn scripts from.",
|
||||
)
|
||||
.option(
|
||||
"-w --workspace <workspace:string>",
|
||||
"The workspace to spawn scripts from.",
|
||||
{ default: "starter" }
|
||||
{ default: "starter" },
|
||||
)
|
||||
.option("-m --metrics <metrics:string>", "The url to scrape metrics from.", {
|
||||
default: "http://localhost:8001/metrics",
|
||||
})
|
||||
.option(
|
||||
"--export-json <export_json:string>",
|
||||
"If set, exports will be into a JSON file."
|
||||
"If set, exports will be into a JSON file.",
|
||||
)
|
||||
.option(
|
||||
"--export-csv <export_csv:string>",
|
||||
"If set, exports will be into a csv file."
|
||||
"If set, exports will be into a csv file.",
|
||||
)
|
||||
.option(
|
||||
"--export-histograms [histograms...:string]",
|
||||
"Mark metrics (without label) that are reported as histograms to export."
|
||||
"Mark metrics (without label) that are reported as histograms to export.",
|
||||
)
|
||||
.option(
|
||||
"--export-simple [simple...:string]",
|
||||
"Mark metrics (without label) that are reported as simple values."
|
||||
"Mark metrics (without label) that are reported as simple values.",
|
||||
)
|
||||
.option(
|
||||
"--maximum-throughput <maximum_throughput:number>",
|
||||
"Maximum number of jobs/flows to start in one second.",
|
||||
{
|
||||
default: Infinity,
|
||||
}
|
||||
},
|
||||
)
|
||||
.option("--use-flows", "Run flows instead of jobs.")
|
||||
.option("--custom <custom_path:string>", "Use custom actions during bench")
|
||||
@@ -89,11 +89,11 @@ await new Command()
|
||||
"The maximum time in ms to wait for jobs to complete.",
|
||||
{
|
||||
default: 90000,
|
||||
}
|
||||
},
|
||||
)
|
||||
.option(
|
||||
"--continous",
|
||||
"Run the benchmark forever. This effectively disables metric collection & exports. No zombie jobs will be tracked."
|
||||
"Run the benchmark forever. This effectively disables metric collection & exports. No zombie jobs will be tracked.",
|
||||
)
|
||||
.option(
|
||||
"--histogram-buckets [buckets...:string]",
|
||||
@@ -114,7 +114,7 @@ await new Command()
|
||||
"0.01",
|
||||
"0.005",
|
||||
],
|
||||
}
|
||||
},
|
||||
)
|
||||
.action(
|
||||
async ({
|
||||
@@ -162,7 +162,7 @@ await new Command()
|
||||
new URL("./scraper.ts", import.meta.url).href,
|
||||
{
|
||||
type: "module",
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
metrics_worker.postMessage({
|
||||
@@ -192,8 +192,8 @@ await new Command()
|
||||
zombieTimeout,
|
||||
},
|
||||
null,
|
||||
4
|
||||
)
|
||||
4,
|
||||
),
|
||||
);
|
||||
|
||||
const config = {
|
||||
@@ -245,19 +245,22 @@ await new Command()
|
||||
const updateState = setInterval(async () => {
|
||||
const elapsed = start ? Math.ceil((Date.now() - start) / 1000) : 0;
|
||||
const sum = jobsSent.reduce((a, b) => a + b, 0);
|
||||
const queue_length = (
|
||||
await windmill.JobService.listQueue({
|
||||
workspace: config.workspace_id,
|
||||
})
|
||||
).length;
|
||||
const queue_length = (await (await fetch(
|
||||
host + "/api/w/" + config.workspace_id + "/jobs/queue/count",
|
||||
{ headers: { ["Authorization"]: "Bearer " + config.token } },
|
||||
)).json()).database_length;
|
||||
await Deno.stdout.write(
|
||||
enc(
|
||||
`elapsed: ${elapsed}/${seconds} | jobs sent: ${JSON.stringify(
|
||||
jobsSent
|
||||
)} (sum: ${sum} thr: ${(sum / elapsed).toFixed(
|
||||
2
|
||||
)}) | queue: ${queue_length} \r`
|
||||
)
|
||||
`elapsed: ${elapsed}/${seconds} | jobs sent: ${
|
||||
JSON.stringify(
|
||||
jobsSent,
|
||||
)
|
||||
} (sum: ${sum} thr: ${
|
||||
(sum / elapsed).toFixed(
|
||||
2,
|
||||
)
|
||||
}) | queue: ${queue_length} \r`,
|
||||
),
|
||||
);
|
||||
}, 100);
|
||||
|
||||
@@ -284,7 +287,7 @@ await new Command()
|
||||
|
||||
const sum = jobsSent.reduce((a, b) => a + b, 0);
|
||||
await Deno.stdout.write(
|
||||
enc(" ".padStart(30) + `\rduration: ${seconds} | jobs sent: ${sum}\n`)
|
||||
enc(" ".padStart(30) + `\rduration: ${seconds} | jobs sent: ${sum}\n`),
|
||||
);
|
||||
|
||||
const shutdown_start = Date.now();
|
||||
@@ -302,7 +305,7 @@ await new Command()
|
||||
};
|
||||
worker.addEventListener("message", l);
|
||||
worker.postMessage(
|
||||
Number.isSafeInteger(zombieTimeout) ? zombieTimeout : 90000
|
||||
Number.isSafeInteger(zombieTimeout) ? zombieTimeout : 90000,
|
||||
);
|
||||
});
|
||||
|
||||
@@ -318,11 +321,10 @@ await new Command()
|
||||
console.log("incorrect results: ", incorrect_results);
|
||||
console.log(
|
||||
"queue length:",
|
||||
(
|
||||
await windmill.JobService.listQueue({
|
||||
workspace: config.workspace_id,
|
||||
})
|
||||
).length
|
||||
(await (await fetch(
|
||||
host + "/api/w/" + config.workspace_id + "/jobs/queue/count",
|
||||
{ headers: { ["Authorization"]: "Bearer " + config.token } },
|
||||
)).json()).database_length,
|
||||
);
|
||||
|
||||
metrics_worker!.postMessage("stop");
|
||||
@@ -346,7 +348,7 @@ await new Command()
|
||||
const value = values[i]!;
|
||||
const mean = value.reduce((acc, e) => acc + e, 0) / values.length;
|
||||
const stdev = Math.sqrt(
|
||||
value.reduce((acc, e) => acc + (e - mean) ** 2) / values.length
|
||||
value.reduce((acc, e) => acc + (e - mean) ** 2) / values.length,
|
||||
);
|
||||
obj[name] = { mean, stdev };
|
||||
}
|
||||
@@ -374,6 +376,6 @@ await new Command()
|
||||
f.close();
|
||||
}
|
||||
console.log("done");
|
||||
}
|
||||
},
|
||||
)
|
||||
.parse();
|
||||
|
||||
@@ -13,6 +13,8 @@ const promise = new Promise<{
|
||||
continous: boolean;
|
||||
max_per_worker: number;
|
||||
custom: Action | undefined;
|
||||
server: string;
|
||||
token: string;
|
||||
}>((resolve, _reject) => {
|
||||
self.onmessage = (evt) => {
|
||||
const sharedConfig = evt.data;
|
||||
@@ -24,6 +26,8 @@ const promise = new Promise<{
|
||||
continous: sharedConfig.continous,
|
||||
max_per_worker: sharedConfig.max_per_worker,
|
||||
custom: sharedConfig.custom,
|
||||
server: sharedConfig.server,
|
||||
token: sharedConfig.token,
|
||||
};
|
||||
self.name = "Worker " + sharedConfig.i;
|
||||
resolve(config);
|
||||
@@ -46,19 +50,20 @@ const updateStatusInterval = setInterval(() => {
|
||||
}, 100);
|
||||
|
||||
while (cont) {
|
||||
const queue_length = (
|
||||
await windmill.JobService.listQueue({ workspace: config.workspace_id })
|
||||
).length;
|
||||
if (queue_length > 500) {
|
||||
const queue_length = (await (await fetch(
|
||||
config.server + "/api/w/" + config.workspace_id + "/jobs/queue/count",
|
||||
{ headers: { ["Authorization"]: "Bearer " + config.token } },
|
||||
)).json()).database_length;
|
||||
if (queue_length > 2500) {
|
||||
console.log(
|
||||
`queue length: ${queue_length} > 500. waiting... `
|
||||
`queue length: ${queue_length} > 2500. waiting... `,
|
||||
);
|
||||
await sleep(0.5);
|
||||
continue;
|
||||
}
|
||||
if (
|
||||
(total_spawned * 1000) / (Date.now() - start_time) >
|
||||
config.per_worker_throughput
|
||||
config.per_worker_throughput
|
||||
) {
|
||||
console.log("at maximum throughput. waiting...");
|
||||
await sleep(0.1);
|
||||
@@ -138,13 +143,13 @@ while (outstanding.length > 0 && Date.now() < end_time) {
|
||||
await Deno.stdout.write(
|
||||
enc(
|
||||
`uuid: ${uuid}, queue length: ${
|
||||
(
|
||||
await windmill.JobService.listQueue({
|
||||
workspace: config.workspace_id,
|
||||
})
|
||||
).length
|
||||
} \r`
|
||||
)
|
||||
(await (await fetch(
|
||||
config.server + "/api/w/" + config.workspace_id +
|
||||
"/jobs/queue/count",
|
||||
{ headers: { ["Authorization"]: "Bearer " + config.token } },
|
||||
)).json()).database_length
|
||||
} \r`,
|
||||
),
|
||||
);
|
||||
} else if (!config.useFlows) {
|
||||
r = r as api.CompletedJob;
|
||||
@@ -156,7 +161,7 @@ while (outstanding.length > 0 && Date.now() < end_time) {
|
||||
" != " +
|
||||
uuid +
|
||||
"job: \n" +
|
||||
JSON.stringify(r, null, 2)
|
||||
JSON.stringify(r, null, 2),
|
||||
);
|
||||
incorrect_results++;
|
||||
}
|
||||
|
||||
@@ -19,7 +19,7 @@ import sync from "./sync.ts";
|
||||
import { tryResolveVersion } from "./context.ts";
|
||||
import { GlobalOptions } from "./types.ts";
|
||||
|
||||
export const VERSION = "v1.86.0";
|
||||
export const VERSION = "v1.87.0";
|
||||
|
||||
let command: any = new Command()
|
||||
.name("wmill")
|
||||
|
||||
@@ -142,6 +142,7 @@ export async function handleFile(
|
||||
typed.summary === remote.summary &&
|
||||
typed.is_template === remote.is_template &&
|
||||
typed.kind == remote.kind &&
|
||||
!remote.archived &&
|
||||
remote?.lock == typed.lock?.join("\n") &&
|
||||
JSON.stringify(typed.schema) == JSON.stringify(remote.schema))
|
||||
) {
|
||||
|
||||
86
docker-compose-dev.yml
Normal file
86
docker-compose-dev.yml
Normal file
@@ -0,0 +1,86 @@
|
||||
version: "3.7"
|
||||
|
||||
services:
|
||||
db:
|
||||
image: postgres:14
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- db_data:/var/lib/postgresql/data
|
||||
expose:
|
||||
- 5432
|
||||
ports:
|
||||
- 5432:5432
|
||||
environment:
|
||||
POSTGRES_PASSWORD: ${DB_PASSWORD}
|
||||
POSTGRES_DB: windmill
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U postgres"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
|
||||
windmill_server:
|
||||
image: ghcr.io/windmill-labs/windmill:main
|
||||
deploy:
|
||||
replicas: 1
|
||||
restart: unless-stopped
|
||||
expose:
|
||||
- 8000
|
||||
ports:
|
||||
- 8000:8000
|
||||
environment:
|
||||
- DATABASE_URL=postgres://postgres:${DB_PASSWORD}@db/windmill?sslmode=disable
|
||||
- BASE_URL=http://${WM_BASE_URL}
|
||||
- RUST_LOG=info
|
||||
## You can set the number of workers to > 0 and not need any separate worker service
|
||||
- NUM_WORKERS=0
|
||||
- DISABLE_SERVER=false
|
||||
- METRICS_ADDR=false
|
||||
depends_on:
|
||||
db:
|
||||
condition: service_healthy
|
||||
# volumes:
|
||||
# - ./oauth.json/:/usr/src/app/oauth.json
|
||||
|
||||
windmill_worker:
|
||||
image: ghcr.io/windmill-labs/windmill:main
|
||||
deploy:
|
||||
replicas: 3
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
- DATABASE_URL=postgres://postgres:${DB_PASSWORD}@db/windmill?sslmode=disable
|
||||
- BASE_URL=http://${WM_BASE_URL}
|
||||
- BASE_INTERNAL_URL=http://windmill_server:8000
|
||||
- RUST_LOG=info
|
||||
- NUM_WORKERS=1
|
||||
- DISABLE_SERVER=true
|
||||
- KEEP_JOB_DIR=false
|
||||
- DENO_PATH=/usr/bin/deno
|
||||
- PYTHON_PATH=/usr/local/bin/python3
|
||||
- METRICS_ADDR=false
|
||||
depends_on:
|
||||
db:
|
||||
condition: service_healthy
|
||||
# to mount the worker folder to debug,, KEEP_JOB_DIR=true and mount /tmp/windmill
|
||||
volumes:
|
||||
- worker_dependency_cache:/tmp/windmill/cache
|
||||
|
||||
lsp:
|
||||
image: ghcr.io/windmill-labs/windmill-lsp:latest
|
||||
restart: unless-stopped
|
||||
expose:
|
||||
- 3001
|
||||
caddy:
|
||||
image: caddy:2.5.2-alpine
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- ./Caddyfile:/etc/caddy/Caddyfile
|
||||
ports:
|
||||
- 80:80
|
||||
- 443:443
|
||||
environment:
|
||||
- BASE_URL=${WM_BASE_URL}
|
||||
|
||||
volumes:
|
||||
db_data: null
|
||||
worker_dependency_cache: null
|
||||
@@ -4,9 +4,9 @@
|
||||
|
||||
:8080 {
|
||||
bind {$ADDRESS}
|
||||
reverse_proxy /api/* http://windmill:8000
|
||||
reverse_proxy /api/* http://localhost:8000
|
||||
reverse_proxy /* http://localhost:3000
|
||||
reverse_proxy /ws/* http://lsp:3001 {
|
||||
lb_policy header "Authorization"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -17,7 +17,7 @@ In the root folder:
|
||||
|
||||
```bash
|
||||
docker build . -t windmill
|
||||
docker compose up db windmill_server windmill_worker
|
||||
docker compose -f docker-compose-dev.yml up db windmill_server windmill_worker
|
||||
```
|
||||
|
||||
### 2. Backend is run by cargo
|
||||
@@ -28,20 +28,23 @@ docker compose up db windmill_server windmill_worker
|
||||
- Install llvm
|
||||
|
||||
**On OSX:**
|
||||
|
||||
```bash
|
||||
brew install llvm caddy gsed
|
||||
|
||||
|
||||
# make LLVM tools available on PATH
|
||||
echo 'export PATH="/opt/homebrew/opt/llvm/bin:$PATH"' >> ~/.zshrc
|
||||
|
||||
|
||||
# now, restart your shell. You should now have the `lld` binary on your PATH.
|
||||
```
|
||||
|
||||
- To test that you have Rust and Cargo installed run `cargo --version`
|
||||
|
||||
- In your terminal, go to the backend directory and run `cargo build`
|
||||
- Run `cargo run`
|
||||
|
||||
**Known issue on M1 Mac while running `cargo build`**
|
||||
|
||||
- You may encounter `linking with cc failed` build time error.
|
||||
- To solve this run:
|
||||
```bash
|
||||
@@ -49,7 +52,6 @@ docker compose up db windmill_server windmill_worker
|
||||
source ~/.zshrc
|
||||
```
|
||||
|
||||
|
||||
**Do a Frontend Build**
|
||||
|
||||
In order to run the backend, you need to have a frontend build inside `frontend/build/`.
|
||||
@@ -70,11 +72,14 @@ npm run build
|
||||
```
|
||||
|
||||
**Known issue while running `npm run build`**
|
||||
|
||||
- You may encounter `FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory` error.
|
||||
- To solve this run:
|
||||
|
||||
```bash
|
||||
export NODE_OPTIONS=--max_old_space_size=8096
|
||||
```
|
||||
|
||||
- run `npm run build` again
|
||||
|
||||
In the root folder:
|
||||
@@ -86,7 +91,7 @@ docker-compose up db
|
||||
In the backend folder:
|
||||
|
||||
```bash
|
||||
DATABASE_URL=postgres://postgres:changeme@127.0.0.1:5433/windmill?sslmode=disable cargo run
|
||||
DATABASE_URL=postgres://postgres:changeme@127.0.0.1:5432/windmill?sslmode=disable cargo run
|
||||
```
|
||||
|
||||
You can now access [http://127.0.0.1:8000](http://127.0.0.1:8000).
|
||||
@@ -146,13 +151,13 @@ Recommended config for VS Code:
|
||||
|
||||
- turn _format on save_ on
|
||||
|
||||
|
||||
## Building
|
||||
|
||||
The project is built with [SvelteKit](https://kit.svelte.dev/) and uses as output static files.
|
||||
There are others adapters for sveltekit, but we use the static adapter.
|
||||
|
||||
To build the frontend as static assets, use:
|
||||
|
||||
```
|
||||
npm run build
|
||||
```
|
||||
@@ -160,10 +165,11 @@ npm run build
|
||||
The output is in the `build` folder.
|
||||
|
||||
The default build assume you serve every non static files as the 200.html file which is catchall. If you prefer a normal layout, you can use:
|
||||
|
||||
```
|
||||
NOTCATCHALL=true npm run build
|
||||
```
|
||||
|
||||
which will generate an index.html and allow you to serve the frontend with any static server.
|
||||
|
||||
Env variables used for build are set in .env file. See [https://vitejs.dev/guide/env-and-mode.html#env-files](https://vitejs.dev/guide/env-and-mode.html#env-files) for more details.
|
||||
|
||||
|
||||
93
frontend/package-lock.json
generated
93
frontend/package-lock.json
generated
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "windmill",
|
||||
"version": "1.86.0",
|
||||
"version": "1.87.0",
|
||||
"lockfileVersion": 2,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "windmill",
|
||||
"version": "1.86.0",
|
||||
"version": "1.87.0",
|
||||
"dependencies": {
|
||||
"@fortawesome/free-brands-svg-icons": "^6.2.1",
|
||||
"@fortawesome/free-solid-svg-icons": "^6.2.1",
|
||||
@@ -29,7 +29,7 @@
|
||||
"svelte-autosize": "^1.0.1",
|
||||
"svelte-chartjs": "^3.1.0",
|
||||
"svelte-portal": "^2.2.0",
|
||||
"svelte-select": "^5.3.1",
|
||||
"svelte-select": "^5.6.0",
|
||||
"svelte-timezone-picker": "^2.0.3",
|
||||
"tailwind-merge": "^1.9.1",
|
||||
"vscode-languageclient": "^8.1.0",
|
||||
@@ -47,21 +47,21 @@
|
||||
"@types/d3-zoom": "^3.0.2",
|
||||
"@types/lodash": "^4.14.191",
|
||||
"@types/node": "^18.11.18",
|
||||
"@types/vscode": "~1.76.0",
|
||||
"@types/vscode": "~1.77.0",
|
||||
"@typescript-eslint/eslint-plugin": "^5.49.0",
|
||||
"@typescript-eslint/parser": "^5.48.0",
|
||||
"@zerodevx/svelte-toast": "^0.8.1",
|
||||
"autoprefixer": "^10.4.13",
|
||||
"cssnano": "^6.0.0",
|
||||
"d3-dag": "^0.11.5",
|
||||
"eslint": "^8.28.0",
|
||||
"eslint": "^8.38.0",
|
||||
"eslint-config-prettier": "^8.6.0",
|
||||
"eslint-plugin-svelte3": "^4.0.0",
|
||||
"monaco-editor-workers": "0.36.0",
|
||||
"ol": "^7.2.2",
|
||||
"openapi-typescript-codegen": "^0.23.0",
|
||||
"path-browserify": "^1.0.1",
|
||||
"pdfjs-dist": "^3.4.120",
|
||||
"pdfjs-dist": "^3.5.141",
|
||||
"postcss": "^8.4.18",
|
||||
"postcss-load-config": "^4.0.1",
|
||||
"prettier": "^2.8.3",
|
||||
@@ -80,7 +80,7 @@
|
||||
"svelte-preprocess": "^5.0.1",
|
||||
"svelte-range-slider-pips": "^2.1.1",
|
||||
"svelte-splitpanes": "^0.7.3",
|
||||
"svelte2tsx": "^0.6.10",
|
||||
"svelte2tsx": "^0.6.11",
|
||||
"tailwindcss": "^3.2.2",
|
||||
"tslib": "^2.4.1",
|
||||
"typescript": "^4.9.3",
|
||||
@@ -636,9 +636,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@eslint/js": {
|
||||
"version": "8.37.0",
|
||||
"resolved": "https://registry.npmjs.org/@eslint/js/-/js-8.37.0.tgz",
|
||||
"integrity": "sha512-x5vzdtOOGgFVDCUs81QRB2+liax8rFg3+7hqM+QhBG0/G3F1ZsoYl97UrqgHgQ9KKT7G6c4V+aTUCgu/n22v1A==",
|
||||
"version": "8.38.0",
|
||||
"resolved": "https://registry.npmjs.org/@eslint/js/-/js-8.38.0.tgz",
|
||||
"integrity": "sha512-IoD2MfUnOV58ghIHCiil01PcohxjbYR/qCxsoC+xNgUwh1EY8jOOrYmu3d3a71+tJJ23uscEV4X2HJWMsPJu4g==",
|
||||
"dev": true,
|
||||
"engines": {
|
||||
"node": "^12.22.0 || ^14.17.0 || >=16.0.0"
|
||||
@@ -1378,9 +1378,9 @@
|
||||
"dev": true
|
||||
},
|
||||
"node_modules/@types/vscode": {
|
||||
"version": "1.76.0",
|
||||
"resolved": "https://registry.npmjs.org/@types/vscode/-/vscode-1.76.0.tgz",
|
||||
"integrity": "sha512-CQcY3+Fe5hNewHnOEAVYj4dd1do/QHliXaknAEYSXx2KEHUzFibDZSKptCon+HPgK55xx20pR+PBJjf0MomnBA==",
|
||||
"version": "1.77.0",
|
||||
"resolved": "https://registry.npmjs.org/@types/vscode/-/vscode-1.77.0.tgz",
|
||||
"integrity": "sha512-MWFN5R7a33n8eJZJmdVlifjig3LWUNRrPeO1xemIcZ0ae0TEQuRc7G2xV0LUX78RZFECY1plYBn+dP/Acc3L0Q==",
|
||||
"dev": true
|
||||
},
|
||||
"node_modules/@typescript-eslint/eslint-plugin": {
|
||||
@@ -2823,15 +2823,15 @@
|
||||
}
|
||||
},
|
||||
"node_modules/eslint": {
|
||||
"version": "8.37.0",
|
||||
"resolved": "https://registry.npmjs.org/eslint/-/eslint-8.37.0.tgz",
|
||||
"integrity": "sha512-NU3Ps9nI05GUoVMxcZx1J8CNR6xOvUT4jAUMH5+z8lpp3aEdPVCImKw6PWG4PY+Vfkpr+jvMpxs/qoE7wq0sPw==",
|
||||
"version": "8.38.0",
|
||||
"resolved": "https://registry.npmjs.org/eslint/-/eslint-8.38.0.tgz",
|
||||
"integrity": "sha512-pIdsD2jwlUGf/U38Jv97t8lq6HpaU/G9NKbYmpWpZGw3LdTNhZLbJePqxOXGB5+JEKfOPU/XLxYxFh03nr1KTg==",
|
||||
"dev": true,
|
||||
"dependencies": {
|
||||
"@eslint-community/eslint-utils": "^4.2.0",
|
||||
"@eslint-community/regexpp": "^4.4.0",
|
||||
"@eslint/eslintrc": "^2.0.2",
|
||||
"@eslint/js": "8.37.0",
|
||||
"@eslint/js": "8.38.0",
|
||||
"@humanwhocodes/config-array": "^0.11.8",
|
||||
"@humanwhocodes/module-importer": "^1.0.1",
|
||||
"@nodelib/fs.walk": "^1.2.8",
|
||||
@@ -4735,14 +4735,17 @@
|
||||
}
|
||||
},
|
||||
"node_modules/pdfjs-dist": {
|
||||
"version": "3.4.120",
|
||||
"resolved": "https://registry.npmjs.org/pdfjs-dist/-/pdfjs-dist-3.4.120.tgz",
|
||||
"integrity": "sha512-B1hw9ilLG4m/jNeFA0C2A0PZydjxslP8ylU+I4XM7Bzh/xWETo9EiBV848lh0O0hLut7T6lK1V7cpAXv5BhxWw==",
|
||||
"version": "3.5.141",
|
||||
"resolved": "https://registry.npmjs.org/pdfjs-dist/-/pdfjs-dist-3.5.141.tgz",
|
||||
"integrity": "sha512-lYIvyi5grtYOIatsfCifIKwxHeAJ8eHyP22DTdvY4pm0yWVSFQnMafpgCPSw8gaNRDDdcHnBVOkqMsyK8SRxZg==",
|
||||
"dev": true,
|
||||
"dependencies": {
|
||||
"path2d-polyfill": "^2.0.1",
|
||||
"web-streams-polyfill": "^3.2.1"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=16"
|
||||
},
|
||||
"optionalDependencies": {
|
||||
"canvas": "^2.11.0"
|
||||
}
|
||||
@@ -6559,9 +6562,9 @@
|
||||
"dev": true
|
||||
},
|
||||
"node_modules/svelte-select": {
|
||||
"version": "5.5.2",
|
||||
"resolved": "https://registry.npmjs.org/svelte-select/-/svelte-select-5.5.2.tgz",
|
||||
"integrity": "sha512-uvTOJyrrD5OnSCAv9A+gLtIeeYHL5PUw3YpnuBkmfp7a3IHUyjgNNM4GIf462Lh9QEEEM3nD0R0E+G/Jp1iq1w==",
|
||||
"version": "5.6.0",
|
||||
"resolved": "https://registry.npmjs.org/svelte-select/-/svelte-select-5.6.0.tgz",
|
||||
"integrity": "sha512-+EDXupp3Lr/hKlXNra1vR1VEGxEEsFxYPs5yDDqQE+xagyAOd8P54IJtU84Z5QN3HYc8l2Hp+tNCCYZoTWWxyQ==",
|
||||
"dependencies": {
|
||||
"@floating-ui/dom": "^1.2.1",
|
||||
"svelte-floating-ui": "1.2.8"
|
||||
@@ -6595,9 +6598,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/svelte2tsx": {
|
||||
"version": "0.6.10",
|
||||
"resolved": "https://registry.npmjs.org/svelte2tsx/-/svelte2tsx-0.6.10.tgz",
|
||||
"integrity": "sha512-7CtUexhSHppSpLwVX7yI9OxJp7+esCNTuOF/CaVxWTgWp88QF/2/dCbpND/cTZmBxht5cFFQBwoLXXqFLdYe3Q==",
|
||||
"version": "0.6.11",
|
||||
"resolved": "https://registry.npmjs.org/svelte2tsx/-/svelte2tsx-0.6.11.tgz",
|
||||
"integrity": "sha512-rRW/3V/6mcejYWmSqcHpmILOSPsOhLgkbKbrTOz82s2n8TywmIsqj2jYPsiL6HeGoUM/atiTD0YKguW4b7ECog==",
|
||||
"dev": true,
|
||||
"dependencies": {
|
||||
"dedent-js": "^1.0.1",
|
||||
@@ -7672,9 +7675,9 @@
|
||||
}
|
||||
},
|
||||
"@eslint/js": {
|
||||
"version": "8.37.0",
|
||||
"resolved": "https://registry.npmjs.org/@eslint/js/-/js-8.37.0.tgz",
|
||||
"integrity": "sha512-x5vzdtOOGgFVDCUs81QRB2+liax8rFg3+7hqM+QhBG0/G3F1ZsoYl97UrqgHgQ9KKT7G6c4V+aTUCgu/n22v1A==",
|
||||
"version": "8.38.0",
|
||||
"resolved": "https://registry.npmjs.org/@eslint/js/-/js-8.38.0.tgz",
|
||||
"integrity": "sha512-IoD2MfUnOV58ghIHCiil01PcohxjbYR/qCxsoC+xNgUwh1EY8jOOrYmu3d3a71+tJJ23uscEV4X2HJWMsPJu4g==",
|
||||
"dev": true
|
||||
},
|
||||
"@floating-ui/core": {
|
||||
@@ -8296,9 +8299,9 @@
|
||||
"dev": true
|
||||
},
|
||||
"@types/vscode": {
|
||||
"version": "1.76.0",
|
||||
"resolved": "https://registry.npmjs.org/@types/vscode/-/vscode-1.76.0.tgz",
|
||||
"integrity": "sha512-CQcY3+Fe5hNewHnOEAVYj4dd1do/QHliXaknAEYSXx2KEHUzFibDZSKptCon+HPgK55xx20pR+PBJjf0MomnBA==",
|
||||
"version": "1.77.0",
|
||||
"resolved": "https://registry.npmjs.org/@types/vscode/-/vscode-1.77.0.tgz",
|
||||
"integrity": "sha512-MWFN5R7a33n8eJZJmdVlifjig3LWUNRrPeO1xemIcZ0ae0TEQuRc7G2xV0LUX78RZFECY1plYBn+dP/Acc3L0Q==",
|
||||
"dev": true
|
||||
},
|
||||
"@typescript-eslint/eslint-plugin": {
|
||||
@@ -9310,15 +9313,15 @@
|
||||
"dev": true
|
||||
},
|
||||
"eslint": {
|
||||
"version": "8.37.0",
|
||||
"resolved": "https://registry.npmjs.org/eslint/-/eslint-8.37.0.tgz",
|
||||
"integrity": "sha512-NU3Ps9nI05GUoVMxcZx1J8CNR6xOvUT4jAUMH5+z8lpp3aEdPVCImKw6PWG4PY+Vfkpr+jvMpxs/qoE7wq0sPw==",
|
||||
"version": "8.38.0",
|
||||
"resolved": "https://registry.npmjs.org/eslint/-/eslint-8.38.0.tgz",
|
||||
"integrity": "sha512-pIdsD2jwlUGf/U38Jv97t8lq6HpaU/G9NKbYmpWpZGw3LdTNhZLbJePqxOXGB5+JEKfOPU/XLxYxFh03nr1KTg==",
|
||||
"dev": true,
|
||||
"requires": {
|
||||
"@eslint-community/eslint-utils": "^4.2.0",
|
||||
"@eslint-community/regexpp": "^4.4.0",
|
||||
"@eslint/eslintrc": "^2.0.2",
|
||||
"@eslint/js": "8.37.0",
|
||||
"@eslint/js": "8.38.0",
|
||||
"@humanwhocodes/config-array": "^0.11.8",
|
||||
"@humanwhocodes/module-importer": "^1.0.1",
|
||||
"@nodelib/fs.walk": "^1.2.8",
|
||||
@@ -10772,9 +10775,9 @@
|
||||
}
|
||||
},
|
||||
"pdfjs-dist": {
|
||||
"version": "3.4.120",
|
||||
"resolved": "https://registry.npmjs.org/pdfjs-dist/-/pdfjs-dist-3.4.120.tgz",
|
||||
"integrity": "sha512-B1hw9ilLG4m/jNeFA0C2A0PZydjxslP8ylU+I4XM7Bzh/xWETo9EiBV848lh0O0hLut7T6lK1V7cpAXv5BhxWw==",
|
||||
"version": "3.5.141",
|
||||
"resolved": "https://registry.npmjs.org/pdfjs-dist/-/pdfjs-dist-3.5.141.tgz",
|
||||
"integrity": "sha512-lYIvyi5grtYOIatsfCifIKwxHeAJ8eHyP22DTdvY4pm0yWVSFQnMafpgCPSw8gaNRDDdcHnBVOkqMsyK8SRxZg==",
|
||||
"dev": true,
|
||||
"requires": {
|
||||
"canvas": "^2.11.0",
|
||||
@@ -12017,9 +12020,9 @@
|
||||
"dev": true
|
||||
},
|
||||
"svelte-select": {
|
||||
"version": "5.5.2",
|
||||
"resolved": "https://registry.npmjs.org/svelte-select/-/svelte-select-5.5.2.tgz",
|
||||
"integrity": "sha512-uvTOJyrrD5OnSCAv9A+gLtIeeYHL5PUw3YpnuBkmfp7a3IHUyjgNNM4GIf462Lh9QEEEM3nD0R0E+G/Jp1iq1w==",
|
||||
"version": "5.6.0",
|
||||
"resolved": "https://registry.npmjs.org/svelte-select/-/svelte-select-5.6.0.tgz",
|
||||
"integrity": "sha512-+EDXupp3Lr/hKlXNra1vR1VEGxEEsFxYPs5yDDqQE+xagyAOd8P54IJtU84Z5QN3HYc8l2Hp+tNCCYZoTWWxyQ==",
|
||||
"requires": {
|
||||
"@floating-ui/dom": "^1.2.1",
|
||||
"svelte-floating-ui": "1.2.8"
|
||||
@@ -12043,9 +12046,9 @@
|
||||
}
|
||||
},
|
||||
"svelte2tsx": {
|
||||
"version": "0.6.10",
|
||||
"resolved": "https://registry.npmjs.org/svelte2tsx/-/svelte2tsx-0.6.10.tgz",
|
||||
"integrity": "sha512-7CtUexhSHppSpLwVX7yI9OxJp7+esCNTuOF/CaVxWTgWp88QF/2/dCbpND/cTZmBxht5cFFQBwoLXXqFLdYe3Q==",
|
||||
"version": "0.6.11",
|
||||
"resolved": "https://registry.npmjs.org/svelte2tsx/-/svelte2tsx-0.6.11.tgz",
|
||||
"integrity": "sha512-rRW/3V/6mcejYWmSqcHpmILOSPsOhLgkbKbrTOz82s2n8TywmIsqj2jYPsiL6HeGoUM/atiTD0YKguW4b7ECog==",
|
||||
"dev": true,
|
||||
"requires": {
|
||||
"dedent-js": "^1.0.1",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "windmill",
|
||||
"version": "1.86.0",
|
||||
"version": "1.87.0",
|
||||
"scripts": {
|
||||
"dev": "vite dev",
|
||||
"build": "vite build",
|
||||
@@ -26,21 +26,21 @@
|
||||
"@types/d3-zoom": "^3.0.2",
|
||||
"@types/lodash": "^4.14.191",
|
||||
"@types/node": "^18.11.18",
|
||||
"@types/vscode": "~1.76.0",
|
||||
"@types/vscode": "~1.77.0",
|
||||
"@typescript-eslint/eslint-plugin": "^5.49.0",
|
||||
"@typescript-eslint/parser": "^5.48.0",
|
||||
"@zerodevx/svelte-toast": "^0.8.1",
|
||||
"autoprefixer": "^10.4.13",
|
||||
"cssnano": "^6.0.0",
|
||||
"d3-dag": "^0.11.5",
|
||||
"eslint": "^8.28.0",
|
||||
"eslint": "^8.38.0",
|
||||
"eslint-config-prettier": "^8.6.0",
|
||||
"eslint-plugin-svelte3": "^4.0.0",
|
||||
"monaco-editor-workers": "0.36.0",
|
||||
"ol": "^7.2.2",
|
||||
"openapi-typescript-codegen": "^0.23.0",
|
||||
"path-browserify": "^1.0.1",
|
||||
"pdfjs-dist": "^3.4.120",
|
||||
"pdfjs-dist": "^3.5.141",
|
||||
"postcss": "^8.4.18",
|
||||
"postcss-load-config": "^4.0.1",
|
||||
"prettier": "^2.8.3",
|
||||
@@ -59,7 +59,7 @@
|
||||
"svelte-preprocess": "^5.0.1",
|
||||
"svelte-range-slider-pips": "^2.1.1",
|
||||
"svelte-splitpanes": "^0.7.3",
|
||||
"svelte2tsx": "^0.6.10",
|
||||
"svelte2tsx": "^0.6.11",
|
||||
"tailwindcss": "^3.2.2",
|
||||
"tslib": "^2.4.1",
|
||||
"typescript": "^4.9.3",
|
||||
@@ -89,7 +89,7 @@
|
||||
"svelte-autosize": "^1.0.1",
|
||||
"svelte-chartjs": "^3.1.0",
|
||||
"svelte-portal": "^2.2.0",
|
||||
"svelte-select": "^5.3.1",
|
||||
"svelte-select": "^5.6.0",
|
||||
"svelte-timezone-picker": "^2.0.3",
|
||||
"tailwind-merge": "^1.9.1",
|
||||
"vscode-languageclient": "^8.1.0",
|
||||
|
||||
@@ -101,8 +101,8 @@
|
||||
>{/if}{#if typeof result == 'object' && Object.keys(result).length > 0}<div
|
||||
class="mb-2 w-full text-sm text-gray-700 relative"
|
||||
>The result keys are: <b>{truncate(Object.keys(result).join(', '), 50)}</b>
|
||||
<div class="text-gray-500 text-sm absolute top-6 right-0">
|
||||
<button on:click={jsonViewer.openDrawer}>Expand JSON</button>
|
||||
<div class="text-gray-500 text-xs absolute top-5 right-0">
|
||||
<button on:click={jsonViewer.openDrawer}>Expand</button>
|
||||
</div></div
|
||||
>{/if}{#if !forceJson && resultKind == 'table-col'}<div
|
||||
class="grid grid-flow-col-dense border border-gray-200 rounded-md"
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
<script lang="ts">
|
||||
import { ResourceService, VariableService } from '$lib/gen'
|
||||
import { getScriptByPath, sendUserToast } from '$lib/utils'
|
||||
import { getModifierKey, getScriptByPath, sendUserToast } from '$lib/utils'
|
||||
|
||||
import {
|
||||
faBroom,
|
||||
@@ -365,10 +365,12 @@
|
||||
{iconOnly}
|
||||
startIcon={{ icon: faBroom }}
|
||||
>
|
||||
Format (Ctrl+S)
|
||||
Format ({getModifierKey()}+S)
|
||||
</Button>
|
||||
<svelte:fragment slot="text">
|
||||
Format <Kbd class="!text-gray-800">Ctrl</Kbd> + <Kbd class="!text-gray-800">S</Kbd>
|
||||
Format <Kbd class="!text-gray-800">{getModifierKey()}</Kbd> + <Kbd class="!text-gray-800">
|
||||
S
|
||||
</Kbd>
|
||||
</svelte:fragment>
|
||||
</Popover>
|
||||
</div>
|
||||
|
||||
@@ -217,20 +217,20 @@
|
||||
|
||||
switch (event.key) {
|
||||
case 'Z':
|
||||
if (event.ctrlKey) {
|
||||
if (event.ctrlKey || event.metaKey) {
|
||||
$flowStore = redo(history)
|
||||
event.preventDefault()
|
||||
}
|
||||
break
|
||||
case 'z':
|
||||
if (event.ctrlKey) {
|
||||
if (event.ctrlKey || event.metaKey) {
|
||||
$flowStore = undo(history, $flowStore)
|
||||
$selectedIdStore = 'Input'
|
||||
event.preventDefault()
|
||||
}
|
||||
break
|
||||
case 's':
|
||||
if (event.ctrlKey) {
|
||||
if (event.ctrlKey || event.metaKey) {
|
||||
saveFlow(false)
|
||||
event.preventDefault()
|
||||
}
|
||||
|
||||
@@ -13,6 +13,7 @@
|
||||
import FlowProgressBar from './flows/FlowProgressBar.svelte'
|
||||
import CapturePayload from './flows/content/CapturePayload.svelte'
|
||||
import { Loader2 } from 'lucide-svelte'
|
||||
import { getModifierKey } from '$lib/utils'
|
||||
|
||||
let capturePayload: CapturePayload
|
||||
export let previewMode: 'upTo' | 'whole'
|
||||
@@ -75,7 +76,7 @@
|
||||
if (open) {
|
||||
switch (event.key) {
|
||||
case 'Enter':
|
||||
if (event.ctrlKey) {
|
||||
if (event.ctrlKey || event.metaKey) {
|
||||
event.preventDefault()
|
||||
runPreview($previewArgs)
|
||||
}
|
||||
@@ -134,7 +135,7 @@
|
||||
btnClasses="w-full max-w-lg"
|
||||
on:click={() => runPreview($previewArgs)}
|
||||
>
|
||||
Test flow <Kbd class="ml-2">Ctrl+Enter</Kbd>
|
||||
Test flow <Kbd class="ml-2">{getModifierKey()}+Enter</Kbd>
|
||||
</Button>
|
||||
{/if}
|
||||
<Button
|
||||
|
||||
@@ -308,6 +308,9 @@
|
||||
: 'border border-red-700 border-opacity-30 focus:border-red-700 focus:border-opacity-30 bg-red-100'}"
|
||||
placeholder={defaultValue ?? ''}
|
||||
bind:value
|
||||
on:pointerdown|stopPropagation={(e) => {
|
||||
dispatch('inputClicked', e)
|
||||
}}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -28,6 +28,7 @@
|
||||
properties={schema.properties[argName].properties}
|
||||
itemsType={schema.properties[argName].items}
|
||||
extra={schema.properties[argName]}
|
||||
on:inputClicked
|
||||
/>
|
||||
{/if}
|
||||
</div>
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
import type { Schema } from '$lib/common'
|
||||
import { ScriptService, type FlowModule, type InputTransform, type Job } from '$lib/gen'
|
||||
import { workspaceStore } from '$lib/stores'
|
||||
import { getScriptByPath } from '$lib/utils'
|
||||
import { getModifierKey, getScriptByPath } from '$lib/utils'
|
||||
import { Loader2 } from 'lucide-svelte'
|
||||
import { getContext } from 'svelte'
|
||||
import { Pane, Splitpanes } from 'svelte-splitpanes'
|
||||
@@ -86,7 +86,7 @@
|
||||
runnable={{ summary: mod.summary ?? '', schema, description: '' }}
|
||||
runAction={(_, args) => runTest(args)}
|
||||
schedulable={false}
|
||||
buttonText="Test (Ctrl+Enter)"
|
||||
buttonText={`Test (${getModifierKey()}+Enter)`}
|
||||
detailed={false}
|
||||
topButton
|
||||
bind:args={stepArgs}
|
||||
|
||||
@@ -317,16 +317,28 @@
|
||||
></span
|
||||
>
|
||||
|
||||
<div class="flex flex-row gap-1 w-full">
|
||||
<div class="flex flex-row items-center gap-1 w-full">
|
||||
<select class="grow w-full" {disabled} bind:value={meta.owner}>
|
||||
{#each folders as { name, write }}
|
||||
<option disabled={!write}>{name}{write ? '' : ' (read-only)'}</option>
|
||||
{/each}
|
||||
</select>
|
||||
<Button variant="border" size="xs" on:click={viewFolder.openDrawer}>
|
||||
<Button
|
||||
title="View folder"
|
||||
btnClasses="!p-1.5"
|
||||
variant="border"
|
||||
size="xs"
|
||||
on:click={viewFolder.openDrawer}
|
||||
>
|
||||
<Icon scale={0.8} data={faEye} /></Button
|
||||
>
|
||||
<Button variant="border" size="xs" on:click={newFolder.openDrawer}>
|
||||
<Button
|
||||
title="New folder"
|
||||
btnClasses="!p-1.5"
|
||||
variant="border"
|
||||
size="xs"
|
||||
on:click={newFolder.openDrawer}
|
||||
>
|
||||
<Icon scale={0.8} data={faPlus} /></Button
|
||||
></div
|
||||
>
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
<script lang="ts">
|
||||
import { createPopperActions } from 'svelte-popperjs'
|
||||
import { createPopperActions, type PopperOptions } from 'svelte-popperjs'
|
||||
import type { PopoverPlacement } from './Popover.model'
|
||||
|
||||
export let placement: PopoverPlacement = 'auto'
|
||||
@@ -9,28 +9,16 @@
|
||||
export let disapperTimoout = 100
|
||||
|
||||
const [popperRef, popperContent] = createPopperActions({ placement })
|
||||
const betterPreventOverflow = (options) => ({
|
||||
name: 'preventOverflow',
|
||||
options,
|
||||
effect: ({ state }) => {
|
||||
const { padding = 0 } = options
|
||||
|
||||
state.elements.popper.style.maxWidth = `calc(100vw - ${padding * 2}px)`
|
||||
}
|
||||
})
|
||||
const extraOpts = {
|
||||
const popperOptions: PopperOptions<{}> = {
|
||||
placement: 'bottom-end',
|
||||
strategy: 'fixed',
|
||||
modifiers: [
|
||||
betterPreventOverflow({ padding: 10 }),
|
||||
{
|
||||
name: 'offset',
|
||||
options: {
|
||||
offset: [8, 8]
|
||||
}
|
||||
},
|
||||
{ name: 'offset', options: { offset: [8, 8] } },
|
||||
{
|
||||
name: 'arrow',
|
||||
options: {
|
||||
padding: 10 // 5px from the edges of the popper
|
||||
padding: 10
|
||||
}
|
||||
}
|
||||
]
|
||||
@@ -59,10 +47,10 @@
|
||||
{/if}
|
||||
{#if showTooltip && !disablePopup}
|
||||
<div
|
||||
use:popperContent={extraOpts}
|
||||
use:popperContent={popperOptions}
|
||||
on:mouseenter={open}
|
||||
on:mouseleave={close}
|
||||
class="z-50 py-2 px-3 rounded-md text-sm font-normal !text-gray-300 bg-gray-800
|
||||
class="z-50 py-2 px-3 rounded-md text-sm font-normal !text-gray-300 bg-gray-800
|
||||
whitespace-normal text-left {popupClass}"
|
||||
>
|
||||
<div class="max-w-sm">
|
||||
|
||||
326
frontend/src/lib/components/SavedInputs.svelte
Normal file
326
frontend/src/lib/components/SavedInputs.svelte
Normal file
@@ -0,0 +1,326 @@
|
||||
<script lang="ts">
|
||||
import { Button } from '$lib/components/common'
|
||||
import { InputService, type Input, RunnableType, type CreateInput } from '$lib/gen/index.js'
|
||||
import { userStore, workspaceStore } from '$lib/stores.js'
|
||||
import { classNames, displayDate, displayDaysAgo, sendUserToast } from '$lib/utils.js'
|
||||
import { faSave } from '@fortawesome/free-solid-svg-icons'
|
||||
import { createEventDispatcher } from 'svelte'
|
||||
import { Pane, Splitpanes } from 'svelte-splitpanes'
|
||||
import ObjectViewer from './propertyPicker/ObjectViewer.svelte'
|
||||
import { ArrowLeftIcon, Edit, X } from 'lucide-svelte'
|
||||
import Toggle from './Toggle.svelte'
|
||||
import Tooltip from './Tooltip.svelte'
|
||||
|
||||
export let scriptHash: string | null = null
|
||||
export let scriptPath: string | null = null
|
||||
export let flowPath: string | null = null
|
||||
|
||||
let runnableId: string | undefined = scriptHash || scriptPath || flowPath || undefined
|
||||
let runnableType: RunnableType | undefined = scriptHash
|
||||
? RunnableType.SCRIPT_HASH
|
||||
: scriptPath
|
||||
? RunnableType.SCRIPT_PATH
|
||||
: flowPath
|
||||
? RunnableType.FLOW_PATH
|
||||
: undefined
|
||||
|
||||
// Are the current Inputs valid and able to be saved?
|
||||
export let isValid: boolean
|
||||
export let args: object
|
||||
|
||||
let previousInputs: Input[] = []
|
||||
interface EditableInput extends Input {
|
||||
isEditing?: boolean
|
||||
isSaving?: boolean
|
||||
}
|
||||
let savedInputs: EditableInput[] = []
|
||||
|
||||
let selectedInput: Input | null
|
||||
|
||||
async function loadInputHistory() {
|
||||
previousInputs = await InputService.getInputHistory({
|
||||
workspace: $workspaceStore!,
|
||||
runnableId,
|
||||
runnableType,
|
||||
perPage: 10
|
||||
})
|
||||
}
|
||||
|
||||
async function loadSavedInputs() {
|
||||
savedInputs = await InputService.listInputs({
|
||||
workspace: $workspaceStore!,
|
||||
runnableId,
|
||||
runnableType,
|
||||
perPage: 10
|
||||
})
|
||||
}
|
||||
|
||||
let savingInputs = false
|
||||
|
||||
async function saveInput(args: object) {
|
||||
savingInputs = true
|
||||
|
||||
const requestBody: CreateInput = {
|
||||
name: 'Saved ' + displayDate(new Date()),
|
||||
args
|
||||
}
|
||||
|
||||
try {
|
||||
let id = await InputService.createInput({
|
||||
workspace: $workspaceStore!,
|
||||
runnableId,
|
||||
runnableType,
|
||||
requestBody
|
||||
})
|
||||
|
||||
const input = {
|
||||
id,
|
||||
created_by: '',
|
||||
created_at: new Date().toISOString(),
|
||||
is_public: false,
|
||||
...requestBody
|
||||
}
|
||||
savedInputs = [input, ...savedInputs]
|
||||
} catch (err) {
|
||||
console.error(err)
|
||||
sendUserToast(`Failed to save Input: ${err}`, true)
|
||||
}
|
||||
|
||||
savingInputs = false
|
||||
}
|
||||
|
||||
async function updateInput(input: EditableInput) {
|
||||
input.isSaving = true
|
||||
|
||||
try {
|
||||
await InputService.updateInput({
|
||||
workspace: $workspaceStore!,
|
||||
requestBody: {
|
||||
id: input.id,
|
||||
name: input.name,
|
||||
is_public: input.is_public
|
||||
}
|
||||
})
|
||||
} catch (err) {
|
||||
console.error(err)
|
||||
sendUserToast(`Failed to update Input: ${err}`, true)
|
||||
}
|
||||
|
||||
input.isSaving = false
|
||||
}
|
||||
|
||||
async function deleteInput(input: Input) {
|
||||
try {
|
||||
await InputService.deleteInput({
|
||||
workspace: $workspaceStore!,
|
||||
input: input.id
|
||||
})
|
||||
savedInputs = savedInputs.filter((i) => i.id !== input.id)
|
||||
if (selectedInput === input) {
|
||||
selectedInput = null
|
||||
}
|
||||
} catch (err) {
|
||||
console.error(err)
|
||||
sendUserToast(`Failed to delete Input: ${err}`, true)
|
||||
}
|
||||
}
|
||||
|
||||
$: {
|
||||
if ($workspaceStore && (scriptHash || scriptPath || flowPath)) {
|
||||
loadInputHistory()
|
||||
loadSavedInputs()
|
||||
}
|
||||
}
|
||||
|
||||
const dispatch = createEventDispatcher()
|
||||
|
||||
const selectArgs = (selected_args: object) => {
|
||||
dispatch('selected_args', selected_args)
|
||||
}
|
||||
</script>
|
||||
|
||||
<div class="min-w-[300px] h-full">
|
||||
<Splitpanes horizontal={true}>
|
||||
<Pane>
|
||||
<div class="w-full flex flex-col gap-4 p-4">
|
||||
<div class="w-full flex justify-between items-center gap-4 flex-wrap">
|
||||
<span class="text-sm font-extrabold flex-shrink-0"
|
||||
>Saved Inputs <Tooltip
|
||||
>Shared tooltips are available to anyone with access to the script</Tooltip
|
||||
></span
|
||||
>
|
||||
<Button
|
||||
on:click={() => saveInput(args)}
|
||||
disabled={!isValid}
|
||||
loading={savingInputs}
|
||||
startIcon={{ icon: faSave }}
|
||||
color="blue"
|
||||
size="xs"
|
||||
>
|
||||
<span>Save Current Input</span>
|
||||
</Button>
|
||||
</div>
|
||||
|
||||
<div class="w-full flex flex-col gap-2 h-full overflow-y-auto p">
|
||||
{#if savedInputs.length > 0}
|
||||
{#each savedInputs as i}
|
||||
<button
|
||||
class={classNames(
|
||||
`w-full flex items-center group justify-between gap-4 py-2 px-4 text-left border rounded-md hover:bg-gray-100 transition-all`,
|
||||
selectedInput === i ? 'border-blue-500 bg-blue-50' : ''
|
||||
)}
|
||||
on:click={() => {
|
||||
if (!i.isEditing) {
|
||||
if (selectedInput === i) {
|
||||
selectedInput = null
|
||||
} else {
|
||||
selectedInput = i
|
||||
}
|
||||
}
|
||||
}}
|
||||
>
|
||||
<div class="w-full h-full items-center justify-between flex gap-1 min-w-0">
|
||||
{#if i.isEditing}
|
||||
<form
|
||||
on:submit={() => {
|
||||
updateInput(i)
|
||||
i.isEditing = false
|
||||
i.isSaving = false
|
||||
}}
|
||||
class="w-full"
|
||||
>
|
||||
<input type="text" bind:value={i.name} class="text-gray-700" />
|
||||
</form>
|
||||
{:else}
|
||||
<small
|
||||
class="whitespace-nowrap overflow-hidden text-ellipsis flex-shrink text-left"
|
||||
>
|
||||
{i.name}
|
||||
</small>
|
||||
{/if}
|
||||
{#if i.created_by == $userStore?.username || $userStore?.is_admin || $userStore?.is_super_admin}
|
||||
<div class="items-center flex gap-2">
|
||||
{#if !i.isEditing}
|
||||
<div class="group-hover:block hidden -my-2">
|
||||
<Toggle
|
||||
size="xs"
|
||||
options={{ right: 'shared' }}
|
||||
bind:checked={i.is_public}
|
||||
on:change={() => {
|
||||
updateInput(i)
|
||||
}}
|
||||
/>
|
||||
</div>
|
||||
{/if}
|
||||
|
||||
<Button
|
||||
loading={i.isSaving}
|
||||
color="gray"
|
||||
size="xs"
|
||||
variant="border"
|
||||
spacingSize="xs2"
|
||||
btnClasses={'group-hover:block hidden'}
|
||||
on:click={(e) => {
|
||||
e.stopPropagation()
|
||||
i.isEditing = !i.isEditing
|
||||
if (!i.isEditing) {
|
||||
updateInput(i)
|
||||
i.isSaving = false
|
||||
}
|
||||
}}
|
||||
>
|
||||
<Edit class="w-4 h-4" />
|
||||
</Button>
|
||||
<Button
|
||||
color="red"
|
||||
size="xs"
|
||||
spacingSize="xs2"
|
||||
variant="border"
|
||||
btnClasses={i.isEditing ? 'block' : 'group-hover:block hidden'}
|
||||
on:click={() => deleteInput(i)}
|
||||
>
|
||||
<X class="w-4 h-4" />
|
||||
</Button>
|
||||
</div>
|
||||
{:else}
|
||||
<span class="text-xs text-gray-600">By {i.created_by}</span>
|
||||
{/if}
|
||||
</div>
|
||||
</button>
|
||||
{/each}
|
||||
{:else}
|
||||
<div class="text-center text-gray-500">No saved Inputs</div>
|
||||
{/if}
|
||||
</div>
|
||||
</div>
|
||||
</Pane>
|
||||
|
||||
<Pane>
|
||||
<div class="w-full flex flex-col gap-4 p-4">
|
||||
<span class="text-sm font-extrabold">Previous Inputs</span>
|
||||
|
||||
<div class="w-full flex flex-col gap-1 p-0 h-full overflow-y-auto">
|
||||
{#if previousInputs.length > 0}
|
||||
{#each previousInputs as i}
|
||||
<button
|
||||
class={classNames(
|
||||
`w-full flex items-center justify-between gap-4 py-2 px-4 text-left border rounded-sm hover:bg-gray-100 transition-a`,
|
||||
selectedInput === i ? 'border-blue-500 bg-blue-50' : ''
|
||||
)}
|
||||
on:click={() => {
|
||||
if (selectedInput === i) {
|
||||
selectedInput = null
|
||||
} else {
|
||||
selectedInput = i
|
||||
}
|
||||
}}
|
||||
>
|
||||
<div class="w-full h-full items-center flex gap-4 min-w-0">
|
||||
<small
|
||||
class="whitespace-nowrap overflow-hidden text-ellipsis flex-shrink text-left"
|
||||
>
|
||||
{displayDaysAgo(i.created_at)} by {i.created_by}
|
||||
</small>
|
||||
</div>
|
||||
</button>
|
||||
{/each}
|
||||
{:else}
|
||||
<div class="text-center text-gray-500">No previous Inputs</div>
|
||||
{/if}
|
||||
</div>
|
||||
</div>
|
||||
</Pane>
|
||||
|
||||
<Pane class="flex flex-col justify-between">
|
||||
<div class="w-full flex flex-col gap-4 p-4 h-full">
|
||||
<span class="text-sm font-extrabold">Preview</span>
|
||||
|
||||
<div class="w-full h-full overflow-auto">
|
||||
{#if Object.keys(selectedInput?.args || {}).length > 0}
|
||||
<div class="border h-full p-2">
|
||||
<ObjectViewer json={selectedInput?.args} />
|
||||
</div>
|
||||
{:else}
|
||||
<div class="text-center text-gray-500">
|
||||
Select an Input to preview scripts arguments
|
||||
</div>
|
||||
{/if}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="w-full flex flex-col p-4">
|
||||
<Button
|
||||
color="blue"
|
||||
btnClasses="w-full"
|
||||
size="sm"
|
||||
spacingSize="xl"
|
||||
on:click={() => selectArgs(selectedInput?.args)}
|
||||
disabled={Object.keys(selectedInput?.args || {}).length === 0}
|
||||
>
|
||||
<ArrowLeftIcon class="w-4 h-4 mr-2" />
|
||||
Use Input
|
||||
</Button>
|
||||
</div>
|
||||
</Pane>
|
||||
</Splitpanes>
|
||||
</div>
|
||||
@@ -22,18 +22,17 @@
|
||||
<div
|
||||
class="pointer-events-auto w-full max-w-sm overflow-hidden bg-white shadow-lg ring-1 ring-black ring-opacity-5 border"
|
||||
>
|
||||
<div class="p-4">
|
||||
<div class="p-2 min-h-[60px]">
|
||||
<div class="flex items-start">
|
||||
<div class="flex-shrink-0">
|
||||
<div class="flex-shrink-0 mt-0.5">
|
||||
{#if error}
|
||||
<XCircleIcon class="h-6 w-6 text-red-400" />
|
||||
<XCircleIcon class="h-4 w-4 text-red-400" />
|
||||
{:else}
|
||||
<CheckCircle2 class="h-6 w-6 text-green-400" />
|
||||
<CheckCircle2 class="h-4 w-4 text-green-400" />
|
||||
{/if}
|
||||
</div>
|
||||
<div class="ml-3 w-0 flex-1 pt-0.5">
|
||||
<p class="text-sm font-medium text-gray-900">{error ? 'Error' : 'Success'}</p>
|
||||
<p class="mt-1 text-sm text-gray-500">{message}</p>
|
||||
<div class="ml-3 w-0 flex-1">
|
||||
<p class="text-sm text-gray-500">{message}</p>
|
||||
</div>
|
||||
<div class="ml-4 flex flex-shrink-0">
|
||||
<button
|
||||
|
||||
@@ -53,9 +53,9 @@
|
||||
}}
|
||||
/>
|
||||
<div
|
||||
class="w-11 h-6 bg-gray-200 rounded-full peer peer-focus:ring-4 peer-focus:ring-blue-300
|
||||
peer-checked:after:translate-x-full peer-checked:after:border-white after:content-['']
|
||||
after:absolute after:top-0.5 after:left-[2px] after:bg-white after:border-gray-300
|
||||
class="w-11 h-6 bg-gray-200 rounded-full peer peer-focus:ring-4 peer-focus:ring-blue-300
|
||||
peer-checked:after:translate-x-full peer-checked:after:border-white after:content-['']
|
||||
after:absolute after:top-0.5 after:left-[2px] after:bg-white after:border-gray-300
|
||||
after:border after:rounded-full after:h-5 after:w-5 after:transition-all {color == 'red'
|
||||
? 'peer-checked:bg-red-600'
|
||||
: 'peer-checked:bg-blue-600'}"
|
||||
|
||||
@@ -67,8 +67,6 @@
|
||||
}
|
||||
}
|
||||
|
||||
$: resolvedConfig?.triggerOnAppLoad && runnableComponent?.runComponent()
|
||||
|
||||
let errors: Record<string, string> = {}
|
||||
$: errorsMessage = Object.values(errors)
|
||||
.filter((x) => x != '')
|
||||
@@ -119,6 +117,7 @@
|
||||
{render}
|
||||
{outputs}
|
||||
{extraKey}
|
||||
refreshOnStart={resolvedConfig.triggerOnAppLoad}
|
||||
>
|
||||
<AlignWrapper {noWFull} {horizontalAlignment} {verticalAlignment}>
|
||||
{#if errorsMessage}
|
||||
|
||||
@@ -59,7 +59,7 @@
|
||||
title="sandbox"
|
||||
srcdoc={result
|
||||
? '<scr' + `ipt type="application/javascript" src="/tailwind.js"></script>` + result
|
||||
: 'No html'}
|
||||
: ''}
|
||||
/>
|
||||
{/key}
|
||||
<!-- svelte-ignore a11y-click-events-have-key-events -->
|
||||
|
||||
@@ -5,6 +5,7 @@
|
||||
import { concatCustomCss } from '../../utils'
|
||||
import { AlignWrapper, InputValue } from '../helpers'
|
||||
import { loadIcon } from '../icon'
|
||||
import InitializeComponent from '../helpers/InitializeComponent.svelte'
|
||||
|
||||
export let id: string
|
||||
export let horizontalAlignment: 'left' | 'center' | 'right' | undefined = 'left'
|
||||
@@ -38,6 +39,8 @@
|
||||
<InputValue {id} input={configuration.color} bind:value={color} />
|
||||
<InputValue {id} input={configuration.strokeWidth} bind:value={strokeWidth} />
|
||||
|
||||
<InitializeComponent {id} />
|
||||
|
||||
<AlignWrapper
|
||||
{render}
|
||||
{horizontalAlignment}
|
||||
|
||||
@@ -7,6 +7,7 @@
|
||||
import { concatCustomCss } from '../../utils'
|
||||
import Loader from '../helpers/Loader.svelte'
|
||||
import ResolveConfig from '../helpers/ResolveConfig.svelte'
|
||||
import InitializeComponent from '../helpers/InitializeComponent.svelte'
|
||||
|
||||
export let id: string
|
||||
export let configuration: RichConfigurations
|
||||
@@ -31,6 +32,8 @@
|
||||
$: css = concatCustomCss($app.css?.imagecomponent, customCss)
|
||||
</script>
|
||||
|
||||
<InitializeComponent {id} />
|
||||
|
||||
{#each Object.keys(components['imagecomponent'].initialData.configuration) as key (key)}
|
||||
<ResolveConfig
|
||||
{id}
|
||||
|
||||
@@ -11,6 +11,7 @@
|
||||
import { Point } from 'ol/geom'
|
||||
import { defaults as defaultControls } from 'ol/control'
|
||||
import { findGridItem, initOutput } from '../../editor/appUtils'
|
||||
import InitializeComponent from '../helpers/InitializeComponent.svelte'
|
||||
|
||||
interface Marker {
|
||||
lon: number
|
||||
@@ -193,6 +194,8 @@
|
||||
<InputValue {id} input={configuration.zoom} bind:value={zoom} />
|
||||
<InputValue {id} input={configuration.markers} bind:value={markers} />
|
||||
|
||||
<InitializeComponent {id} />
|
||||
|
||||
{#if render}
|
||||
<div class="relative h-full w-full">
|
||||
<div
|
||||
|
||||
@@ -11,6 +11,7 @@
|
||||
import type { AppViewerContext, ComponentCustomCSS, RichConfigurations } from '../../types'
|
||||
import { concatCustomCss } from '../../utils'
|
||||
import InputValue from '../helpers/InputValue.svelte'
|
||||
import InitializeComponent from '../helpers/InitializeComponent.svelte'
|
||||
|
||||
export let id: string
|
||||
export let configuration: RichConfigurations
|
||||
@@ -192,6 +193,8 @@
|
||||
<InputValue {id} input={configuration.source} bind:value={source} />
|
||||
<InputValue {id} input={configuration.zoom} bind:value={zoom} />
|
||||
|
||||
<InitializeComponent {id} />
|
||||
|
||||
{#if render}
|
||||
<div class="relative flex flex-col w-full h-full bg-gray-100">
|
||||
{#if source && zoom}
|
||||
|
||||
@@ -226,7 +226,7 @@
|
||||
)}
|
||||
>
|
||||
{String(result)}
|
||||
{#if resolvedConfig.tooltip != ''}
|
||||
{#if resolvedConfig.tooltip && resolvedConfig.tooltip != ''}
|
||||
<Tooltip>{resolvedConfig.tooltip}</Tooltip>
|
||||
{/if}
|
||||
{#if resolvedConfig.copyButton && result}
|
||||
|
||||
@@ -25,6 +25,7 @@
|
||||
{placeholder}
|
||||
on:pointerdown|stopPropagation
|
||||
on:keyup={debounce}
|
||||
on:keydown|stopPropagation
|
||||
class={parentClass}
|
||||
{...$$restProps}
|
||||
/>
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
<script lang="ts">
|
||||
import { getContext } from 'svelte'
|
||||
import { getContext, onMount } from 'svelte'
|
||||
import { initOutput } from '../../editor/appUtils'
|
||||
import type {
|
||||
ConnectedAppInput,
|
||||
@@ -16,13 +16,20 @@
|
||||
export let fields: Record<string, StaticAppInput | ConnectedAppInput | RowAppInput | UserAppInput>
|
||||
export let recomputeOnInputChanged: boolean
|
||||
export let recomputableByRefreshButton: boolean
|
||||
export let noBackendValue: any = undefined
|
||||
|
||||
let result: any = undefined
|
||||
const { worldStore, staticExporter, noBackend } = getContext<AppViewerContext>('AppViewerContext')
|
||||
|
||||
const { worldStore } = getContext<AppViewerContext>('AppViewerContext')
|
||||
let result: any = noBackend ? noBackendValue : undefined
|
||||
|
||||
onMount(() => {
|
||||
$staticExporter[id] = () => {
|
||||
return result
|
||||
}
|
||||
})
|
||||
|
||||
let outputs = initOutput($worldStore, id, {
|
||||
result: undefined,
|
||||
result: result,
|
||||
loading: false
|
||||
})
|
||||
</script>
|
||||
|
||||
@@ -0,0 +1,17 @@
|
||||
<script lang="ts">
|
||||
import { getContext, onDestroy, onMount } from 'svelte'
|
||||
import type { AppViewerContext } from '../../types'
|
||||
|
||||
export let id: string
|
||||
const { initialized } = getContext<AppViewerContext>('AppViewerContext')
|
||||
|
||||
onMount(() => {
|
||||
if (!$initialized.initializedComponents.includes(id)) {
|
||||
$initialized.initializedComponents = [...$initialized.initializedComponents, id]
|
||||
}
|
||||
})
|
||||
|
||||
onDestroy(() => {
|
||||
$initialized.initializedComponents = $initialized.initializedComponents.filter((c) => c !== id)
|
||||
})
|
||||
</script>
|
||||
@@ -41,8 +41,15 @@
|
||||
$: stateId = $worldStore?.stateId
|
||||
|
||||
let timeout: NodeJS.Timeout | undefined = undefined
|
||||
|
||||
let firstDebounce = true
|
||||
const debounce_ms = 50
|
||||
function debounce(cb: () => Promise<void>) {
|
||||
if (firstDebounce) {
|
||||
firstDebounce = false
|
||||
cb()
|
||||
return
|
||||
}
|
||||
if (timeout) {
|
||||
clearTimeout(timeout)
|
||||
}
|
||||
|
||||
@@ -4,6 +4,7 @@
|
||||
import type { Output } from '../../rx'
|
||||
import type { AppViewerContext } from '../../types'
|
||||
import InputValue from './InputValue.svelte'
|
||||
import InitializeComponent from './InitializeComponent.svelte'
|
||||
|
||||
export let componentInput: AppInput
|
||||
export let id: string
|
||||
@@ -25,6 +26,8 @@
|
||||
$: result && outputs && setOutput(result)
|
||||
</script>
|
||||
|
||||
<InitializeComponent {id} />
|
||||
|
||||
{#if componentInput.type !== 'runnable'}
|
||||
<InputValue {id} input={componentInput} bind:value={result} />
|
||||
{/if}
|
||||
|
||||
@@ -8,13 +8,14 @@
|
||||
import { classNames, defaultIfEmptyString, emptySchema, sendUserToast } from '$lib/utils'
|
||||
import { deepEqual } from 'fast-equals'
|
||||
import { Bug } from 'lucide-svelte'
|
||||
import { createEventDispatcher, getContext } from 'svelte'
|
||||
import { createEventDispatcher, getContext, onDestroy } from 'svelte'
|
||||
import type { AppInputs, Runnable } from '../../inputType'
|
||||
import type { Output } from '../../rx'
|
||||
import type { AppViewerContext, InlineScript } from '../../types'
|
||||
import { computeGlobalContext, eval_like } from './eval'
|
||||
import InputValue from './InputValue.svelte'
|
||||
import RefreshButton from './RefreshButton.svelte'
|
||||
import { selectId } from '../../editor/appUtils'
|
||||
|
||||
// Component props
|
||||
export let id: string
|
||||
@@ -31,9 +32,10 @@
|
||||
export let render: boolean
|
||||
export let outputs: { result: Output<any>; loading: Output<boolean> }
|
||||
export let extraKey = ''
|
||||
export let recomputeOnInputChanged: boolean = false
|
||||
export let recomputeOnInputChanged: boolean = true
|
||||
export let loading = false
|
||||
export let recomputableByRefreshButton: boolean = true
|
||||
export let refreshOnStart: boolean = false
|
||||
|
||||
const {
|
||||
worldStore,
|
||||
@@ -47,17 +49,31 @@
|
||||
mode,
|
||||
stateId,
|
||||
state,
|
||||
componentControl
|
||||
componentControl,
|
||||
initialized,
|
||||
selectedComponent,
|
||||
app,
|
||||
connectingInput
|
||||
} = getContext<AppViewerContext>('AppViewerContext')
|
||||
|
||||
const dispatch = createEventDispatcher()
|
||||
|
||||
$runnableComponents[id] = {
|
||||
autoRefresh: autoRefresh && recomputableByRefreshButton,
|
||||
refreshOnStart,
|
||||
cb: async (inlineScript?: InlineScript) => {
|
||||
await executeComponent(true, inlineScript)
|
||||
}
|
||||
}
|
||||
|
||||
if (!$initialized.initializedComponents.includes(id)) {
|
||||
$initialized.initializedComponents = [...$initialized.initializedComponents, id]
|
||||
}
|
||||
|
||||
onDestroy(() => {
|
||||
$initialized.initializedComponents = $initialized.initializedComponents.filter((c) => c !== id)
|
||||
})
|
||||
|
||||
$runnableComponents = $runnableComponents
|
||||
|
||||
let args: Record<string, any> | undefined = undefined
|
||||
@@ -101,7 +117,7 @@
|
||||
function refreshIfAutoRefresh(_src: string) {
|
||||
const refreshEnabled =
|
||||
autoRefresh && ((recomputeOnInputChanged ?? true) || refreshOn?.length > 0)
|
||||
if (refreshEnabled && $worldStore.initialized) {
|
||||
if (refreshEnabled && $initialized.initialized) {
|
||||
setDebouncedExecute()
|
||||
}
|
||||
}
|
||||
@@ -155,6 +171,8 @@
|
||||
}
|
||||
|
||||
async function executeComponent(noToast = false, inlineScriptOverride?: InlineScript) {
|
||||
console.debug('execute', id)
|
||||
|
||||
if (runnable?.type === 'runnableByName' && runnable.inlineScript?.language === 'frontend') {
|
||||
loading = true
|
||||
try {
|
||||
@@ -172,7 +190,7 @@
|
||||
|
||||
$state = $state
|
||||
} catch (e) {
|
||||
sendUserToast('Error running frontend script: ' + e.message, true)
|
||||
sendUserToast(`Error running frontend script ${id}: ` + e.message, true)
|
||||
|
||||
// Manually add a fake job to the job list to show the error
|
||||
|
||||
@@ -188,8 +206,7 @@
|
||||
}
|
||||
loading = false
|
||||
return
|
||||
}
|
||||
if (noBackend) {
|
||||
} else if (noBackend) {
|
||||
if (!noToast) {
|
||||
sendUserToast('This app is not connected to a windmill backend, it is a static preview')
|
||||
}
|
||||
@@ -317,6 +334,11 @@
|
||||
$errorByComponent = $errorByComponent
|
||||
}
|
||||
}
|
||||
|
||||
function handleInputClick(e: CustomEvent) {
|
||||
const event = e as unknown as PointerEvent
|
||||
!$connectingInput.opened && selectId(event, id, selectedComponent, $app)
|
||||
}
|
||||
</script>
|
||||
|
||||
{#each Object.entries(fields ?? {}) as [key, v] (key)}
|
||||
@@ -363,7 +385,11 @@
|
||||
<div class="h-full flex relative flex-row flex-wrap {wrapperClass}" style={wrapperStyle}>
|
||||
{#if (autoRefresh || forceSchemaDisplay) && schemaStripped && Object.keys(schemaStripped?.properties ?? {}).length > 0}
|
||||
<div class="px-2 h-fit min-h-0">
|
||||
<LightweightSchemaForm schema={schemaStripped} bind:args />
|
||||
<LightweightSchemaForm
|
||||
schema={schemaStripped}
|
||||
bind:args
|
||||
on:inputClicked={handleInputClick}
|
||||
/>
|
||||
</div>
|
||||
{/if}
|
||||
|
||||
|
||||
@@ -8,6 +8,7 @@
|
||||
import RunnableComponent from './RunnableComponent.svelte'
|
||||
import { goto } from '$app/navigation'
|
||||
import { sendUserToast } from '$lib/utils'
|
||||
import InitializeComponent from './InitializeComponent.svelte'
|
||||
|
||||
export let componentInput: AppInput | undefined
|
||||
export let id: string
|
||||
@@ -39,6 +40,7 @@
|
||||
export let recomputeIds: string[] = []
|
||||
export let outputs: { result: Output<any>; loading: Output<boolean> }
|
||||
export let extraKey: string | undefined = undefined
|
||||
export let refreshOnStart: boolean = false
|
||||
|
||||
const { staticExporter, noBackend, componentControl, runnableComponents } =
|
||||
getContext<AppViewerContext>('AppViewerContext')
|
||||
@@ -51,7 +53,9 @@
|
||||
result = componentInput?.['value']
|
||||
}
|
||||
onMount(() => {
|
||||
$staticExporter[id] = () => result
|
||||
$staticExporter[id] = () => {
|
||||
return result
|
||||
}
|
||||
})
|
||||
|
||||
function isRunnableDefined(componentInput) {
|
||||
@@ -96,9 +100,11 @@
|
||||
</script>
|
||||
|
||||
{#if componentInput === undefined}
|
||||
<InitializeComponent {id} />
|
||||
<slot />
|
||||
{:else if componentInput.type === 'runnable' && isRunnableDefined(componentInput)}
|
||||
<RunnableComponent
|
||||
{refreshOnStart}
|
||||
{extraKey}
|
||||
bind:loading
|
||||
bind:this={runnableComponent}
|
||||
|
||||
@@ -6,6 +6,7 @@
|
||||
import { concatCustomCss } from '../../utils'
|
||||
import AlignWrapper from '../helpers/AlignWrapper.svelte'
|
||||
import InputValue from '../helpers/InputValue.svelte'
|
||||
import InitializeComponent from '../helpers/InitializeComponent.svelte'
|
||||
|
||||
export let id: string
|
||||
export let configuration: RichConfigurations
|
||||
@@ -34,6 +35,8 @@
|
||||
<InputValue {id} input={configuration.label} bind:value={labelValue} />
|
||||
<InputValue {id} input={configuration.defaultValue} bind:value={defaultValue} />
|
||||
|
||||
<InitializeComponent {id} />
|
||||
|
||||
<AlignWrapper {render} {horizontalAlignment} {verticalAlignment}>
|
||||
<Toggle
|
||||
checked={defaultValue}
|
||||
|
||||
@@ -6,6 +6,7 @@
|
||||
import { concatCustomCss } from '../../utils'
|
||||
import AlignWrapper from '../helpers/AlignWrapper.svelte'
|
||||
import InputValue from '../helpers/InputValue.svelte'
|
||||
import InitializeComponent from '../helpers/InitializeComponent.svelte'
|
||||
|
||||
export let id: string
|
||||
export let configuration: RichConfigurations
|
||||
@@ -41,6 +42,8 @@
|
||||
<InputValue {id} input={configuration.maxDate} bind:value={maxValue} />
|
||||
<InputValue {id} input={configuration.defaultValue} bind:value={defaultValue} />
|
||||
|
||||
<InitializeComponent {id} />
|
||||
|
||||
<AlignWrapper {render} {verticalAlignment}>
|
||||
{#if inputType === 'date'}
|
||||
<input
|
||||
|
||||
@@ -6,6 +6,7 @@
|
||||
import type { AppViewerContext, ComponentCustomCSS, RichConfigurations } from '../../types'
|
||||
import { concatCustomCss } from '../../utils'
|
||||
import InputValue from '../helpers/InputValue.svelte'
|
||||
import InitializeComponent from '../helpers/InitializeComponent.svelte'
|
||||
|
||||
export let id: string
|
||||
export let configuration: RichConfigurations
|
||||
@@ -19,11 +20,11 @@
|
||||
let text: string | undefined = undefined
|
||||
|
||||
let outputs = initOutput($worldStore, id, {
|
||||
result: [] as string[] | undefined
|
||||
result: [] as { name: string; data: string }[] | undefined
|
||||
})
|
||||
|
||||
// Receives Base64 encoded strings from the input component
|
||||
async function handleChange(files: string[] | undefined) {
|
||||
async function handleChange(files: { name: string; data: string }[] | undefined) {
|
||||
outputs?.result.set(files)
|
||||
}
|
||||
|
||||
@@ -34,12 +35,15 @@
|
||||
<InputValue {id} input={configuration.allowMultiple} bind:value={allowMultiple} />
|
||||
<InputValue {id} input={configuration.text} bind:value={text} />
|
||||
|
||||
<InitializeComponent {id} />
|
||||
|
||||
{#if render}
|
||||
<div class="w-full h-full p-1">
|
||||
<FileInput
|
||||
accept={acceptedFileTypes?.length ? acceptedFileTypes?.join(', ') : undefined}
|
||||
multiple={allowMultiple}
|
||||
convertTo="base64"
|
||||
returnFileNames
|
||||
on:change={({ detail }) => {
|
||||
handleChange(detail)
|
||||
}}
|
||||
|
||||
@@ -7,6 +7,7 @@
|
||||
import { concatCustomCss } from '../../utils'
|
||||
import AlignWrapper from '../helpers/AlignWrapper.svelte'
|
||||
import InputValue from '../helpers/InputValue.svelte'
|
||||
import InitializeComponent from '../helpers/InitializeComponent.svelte'
|
||||
|
||||
export let id: string
|
||||
export let configuration: RichConfigurations
|
||||
@@ -55,6 +56,8 @@
|
||||
<InputValue {id} input={configuration.items} bind:value={labels} />
|
||||
<InputValue {id} input={configuration.placeholder} bind:value={placeholder} />
|
||||
|
||||
<InitializeComponent {id} />
|
||||
|
||||
<AlignWrapper {render} {horizontalAlignment} {verticalAlignment}>
|
||||
<div
|
||||
class="app-select w-full"
|
||||
|
||||
@@ -6,6 +6,7 @@
|
||||
import { concatCustomCss } from '../../utils'
|
||||
import AlignWrapper from '../helpers/AlignWrapper.svelte'
|
||||
import InputValue from '../helpers/InputValue.svelte'
|
||||
import InitializeComponent from '../helpers/InitializeComponent.svelte'
|
||||
|
||||
export let id: string
|
||||
export let configuration: RichConfigurations
|
||||
@@ -44,6 +45,8 @@
|
||||
<InputValue {id} input={configuration.placeholder} bind:value={placeholder} />
|
||||
<InputValue {id} input={configuration.defaultValue} bind:value={defaultValue} />
|
||||
|
||||
<InitializeComponent {id} />
|
||||
|
||||
<AlignWrapper {render} {verticalAlignment}>
|
||||
<input
|
||||
on:pointerdown|stopPropagation={() => ($selectedComponent = [id])}
|
||||
|
||||
@@ -7,6 +7,7 @@
|
||||
import { concatCustomCss } from '../../utils'
|
||||
import AlignWrapper from '../helpers/AlignWrapper.svelte'
|
||||
import InputValue from '../helpers/InputValue.svelte'
|
||||
import InitializeComponent from '../helpers/InitializeComponent.svelte'
|
||||
|
||||
export let id: string
|
||||
export let configuration: RichConfigurations
|
||||
@@ -54,6 +55,8 @@
|
||||
bind:value={values[1]}
|
||||
/>
|
||||
|
||||
<InitializeComponent {id} />
|
||||
|
||||
<AlignWrapper {render} {verticalAlignment}>
|
||||
<div class="flex flex-col w-full">
|
||||
<div class="flex items-center w-full gap-1 px-1">
|
||||
|
||||
@@ -8,6 +8,7 @@
|
||||
import InputValue from '../helpers/InputValue.svelte'
|
||||
import { SELECT_INPUT_DEFAULT_STYLE } from '../../../../defaults'
|
||||
import { initOutput } from '../../editor/appUtils'
|
||||
import InitializeComponent from '../helpers/InitializeComponent.svelte'
|
||||
|
||||
export let id: string
|
||||
export let configuration: RichConfigurations
|
||||
@@ -103,6 +104,8 @@
|
||||
<InputValue {id} input={configuration.defaultValue} bind:value={defaultValue} />
|
||||
<InputValue {id} input={configuration.create} bind:value={create} />
|
||||
|
||||
<InitializeComponent {id} />
|
||||
|
||||
<AlignWrapper {render} {horizontalAlignment} {verticalAlignment}>
|
||||
<div
|
||||
class="app-select w-full"
|
||||
|
||||
@@ -7,6 +7,7 @@
|
||||
import { twMerge } from 'tailwind-merge'
|
||||
import { concatCustomCss } from '../../utils'
|
||||
import { initOutput } from '../../editor/appUtils'
|
||||
import InitializeComponent from '../helpers/InitializeComponent.svelte'
|
||||
|
||||
export let id: string
|
||||
export let configuration: RichConfigurations
|
||||
@@ -59,6 +60,7 @@
|
||||
input={configuration.defaultValue}
|
||||
bind:value={values[0]}
|
||||
/>
|
||||
<InitializeComponent {id} />
|
||||
|
||||
<AlignWrapper {render} {verticalAlignment}>
|
||||
<div class="flex items-center w-full gap-1 px-1">
|
||||
|
||||
@@ -6,6 +6,7 @@
|
||||
import { concatCustomCss } from '../../utils'
|
||||
import AlignWrapper from '../helpers/AlignWrapper.svelte'
|
||||
import InputValue from '../helpers/InputValue.svelte'
|
||||
import InitializeComponent from '../helpers/InitializeComponent.svelte'
|
||||
|
||||
export let id: string
|
||||
export let configuration: RichConfigurations
|
||||
@@ -44,6 +45,8 @@
|
||||
<InputValue {id} input={configuration.placeholder} bind:value={placeholder} />
|
||||
<InputValue {id} input={configuration.defaultValue} bind:value={defaultValue} />
|
||||
|
||||
<InitializeComponent {id} />
|
||||
|
||||
{#if inputType === 'textarea'}
|
||||
<textarea
|
||||
class={twMerge(
|
||||
|
||||
@@ -7,6 +7,7 @@
|
||||
import AlignWrapper from '../../helpers/AlignWrapper.svelte'
|
||||
import InputValue from '../../helpers/InputValue.svelte'
|
||||
import CurrencyInput from './CurrencyInput.svelte'
|
||||
import InitializeComponent from '../../helpers/InitializeComponent.svelte'
|
||||
|
||||
export let id: string
|
||||
export let configuration: RichConfigurations
|
||||
@@ -48,6 +49,8 @@
|
||||
<InputValue {id} input={configuration.currency} bind:value={currency} />
|
||||
<InputValue {id} input={configuration.locale} bind:value={locale} />
|
||||
|
||||
<InitializeComponent {id} />
|
||||
|
||||
<AlignWrapper {render} {verticalAlignment}>
|
||||
{#key isNegativeAllowed}
|
||||
{#key locale}
|
||||
|
||||
@@ -4,6 +4,7 @@
|
||||
import SubGridEditor from '../../editor/SubGridEditor.svelte'
|
||||
import type { AppViewerContext, ComponentCustomCSS } from '../../types'
|
||||
import { concatCustomCss } from '../../utils'
|
||||
import InitializeComponent from '../helpers/InitializeComponent.svelte'
|
||||
|
||||
export let id: string
|
||||
export let componentContainerHeight: number
|
||||
@@ -26,6 +27,8 @@
|
||||
$: css = concatCustomCss($app.css?.containercomponent, customCss)
|
||||
</script>
|
||||
|
||||
<InitializeComponent {id} />
|
||||
|
||||
<div class="w-full h-full">
|
||||
{#if $app.subgrids?.[`${id}-0`]}
|
||||
<SubGridEditor
|
||||
|
||||
@@ -12,6 +12,7 @@
|
||||
import { concatCustomCss } from '../../utils'
|
||||
import AlignWrapper from '../helpers/AlignWrapper.svelte'
|
||||
import InputValue from '../helpers/InputValue.svelte'
|
||||
import InitializeComponent from '../helpers/InitializeComponent.svelte'
|
||||
|
||||
export let id: string
|
||||
export let configuration: RichConfigurations
|
||||
@@ -36,6 +37,8 @@
|
||||
<InputValue {id} input={configuration.size} bind:value={size} />
|
||||
<InputValue {id} input={configuration.color} bind:value={color} />
|
||||
|
||||
<InitializeComponent {id} />
|
||||
|
||||
<AlignWrapper
|
||||
{horizontalAlignment}
|
||||
{verticalAlignment}
|
||||
|
||||
@@ -9,6 +9,7 @@
|
||||
import { twMerge } from 'tailwind-merge'
|
||||
import { AlignWrapper } from '../helpers'
|
||||
import { initOutput } from '../../editor/appUtils'
|
||||
import InitializeComponent from '../helpers/InitializeComponent.svelte'
|
||||
|
||||
export let customCss: ComponentCustomCSS<'drawercomponent'> | undefined = undefined
|
||||
export let id: string
|
||||
@@ -70,6 +71,8 @@
|
||||
<InputValue {id} input={configuration.size} bind:value={size} />
|
||||
<InputValue {id} input={configuration.fillContainer} bind:value={fillContainer} />
|
||||
|
||||
<InitializeComponent {id} />
|
||||
|
||||
<Portal target="#app-editor-top-level-drawer">
|
||||
<Drawer let:open bind:this={appDrawer} size="800px" alwaysOpen positionClass="!absolute">
|
||||
<DrawerContent
|
||||
|
||||
@@ -6,6 +6,7 @@
|
||||
import { Pane, Splitpanes } from 'svelte-splitpanes'
|
||||
import { deepEqual } from 'fast-equals'
|
||||
import { initOutput } from '../../editor/appUtils'
|
||||
import InitializeComponent from '../helpers/InitializeComponent.svelte'
|
||||
|
||||
export let id: string
|
||||
export let componentContainerHeight: number
|
||||
@@ -65,6 +66,8 @@
|
||||
}
|
||||
</script>
|
||||
|
||||
<InitializeComponent {id} />
|
||||
|
||||
<div class="h-full w-full border" on:pointerdown={onFocus}>
|
||||
<Splitpanes {horizontal}>
|
||||
{#each sumedup as paneSize, index (index)}
|
||||
|
||||
@@ -7,6 +7,7 @@
|
||||
import type { AppViewerContext, ComponentCustomCSS, RichConfigurations } from '../../types'
|
||||
import { concatCustomCss } from '../../utils'
|
||||
import InputValue from '../helpers/InputValue.svelte'
|
||||
import InitializeComponent from '../helpers/InitializeComponent.svelte'
|
||||
|
||||
export let id: string
|
||||
export let configuration: RichConfigurations
|
||||
@@ -78,6 +79,8 @@
|
||||
|
||||
<InputValue {id} input={configuration.tabsKind} bind:value={resolvedConfig.tabsKind} />
|
||||
|
||||
<InitializeComponent {id} />
|
||||
|
||||
<div class={resolvedConfig.tabsKind == 'sidebar' ? 'flex gap-4 w-full' : 'w-full'}>
|
||||
{#if !resolvedConfig.tabsKind || resolvedConfig.tabsKind == 'tabs' || (resolvedConfig.tabsKind == 'invisibleOnView' && $mode == 'dnd')}
|
||||
<div bind:clientHeight={tabHeight}>
|
||||
|
||||
@@ -80,6 +80,7 @@
|
||||
worldStore,
|
||||
app: appStore,
|
||||
summary: summaryStore,
|
||||
initialized: writable({ initialized: false, initializedComponents: [] }),
|
||||
selectedComponent,
|
||||
mode,
|
||||
connectingInput,
|
||||
|
||||
@@ -25,7 +25,6 @@
|
||||
faClipboard,
|
||||
faExternalLink,
|
||||
faFileExport,
|
||||
faGlobe,
|
||||
faSave
|
||||
} from '@fortawesome/free-solid-svg-icons'
|
||||
import {
|
||||
@@ -42,7 +41,7 @@
|
||||
import { getContext } from 'svelte'
|
||||
import { Icon } from 'svelte-awesome'
|
||||
import { Pane, Splitpanes } from 'svelte-splitpanes'
|
||||
import { appToHubUrl, classNames, copyToClipboard, sendUserToast } from '../../../utils'
|
||||
import { classNames, copyToClipboard, sendUserToast } from '../../../utils'
|
||||
import type {
|
||||
AppInput,
|
||||
ConnectedAppInput,
|
||||
@@ -257,20 +256,20 @@
|
||||
|
||||
switch (event.key) {
|
||||
case 'Z':
|
||||
if (event.ctrlKey) {
|
||||
if (event.ctrlKey || event.metaKey) {
|
||||
$app = redo(history)
|
||||
event.preventDefault()
|
||||
}
|
||||
break
|
||||
case 'z':
|
||||
if (event.ctrlKey) {
|
||||
if (event.ctrlKey || event.metaKey) {
|
||||
$app = undo(history, $app)
|
||||
|
||||
event.preventDefault()
|
||||
}
|
||||
break
|
||||
case 's':
|
||||
if (event.ctrlKey) {
|
||||
if (event.ctrlKey || event.metaKey) {
|
||||
save()
|
||||
event.preventDefault()
|
||||
}
|
||||
@@ -607,14 +606,14 @@
|
||||
appExport.open($app)
|
||||
}
|
||||
},
|
||||
{
|
||||
displayName: 'Publish to Hub',
|
||||
icon: faGlobe,
|
||||
action: () => {
|
||||
const url = appToHubUrl(toStatic($app, $staticExporter, $summary))
|
||||
window.open(url.toString(), '_blank')
|
||||
}
|
||||
},
|
||||
// {
|
||||
// displayName: 'Publish to Hub',
|
||||
// icon: faGlobe,
|
||||
// action: () => {
|
||||
// const url = appToHubUrl(toStatic($app, $staticExporter, $summary))
|
||||
// window.open(url.toString(), '_blank')
|
||||
// }
|
||||
// },
|
||||
{
|
||||
displayName: 'Hub compatible JSON',
|
||||
icon: faFileExport,
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user