diff --git a/README.md b/README.md index d3e87a1af2..12cdac1699 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,11 @@

windmill.dev

-

- . -

-Open-source developer infrastructure for internal tools. Self-hostable alternative to Airplane, Pipedream, Superblocks and a simplified Temporal with autogenerated UIsm and custom UIs to trigger workflows and scripts as internal apps. +Open-source developer infrastructure for internal tools (APIs, background jobs, workflows and UIs). Self-hostable alternative to Airplane, Pipedream, Superblocks and a simplified Temporal with autogenerated UIsm and custom UIs to trigger workflows and scripts as internal apps.

-Scripts are turned into UIs and no-code modules, no-code modules can be composed into very rich flows, and script and flows can be triggered from internal UIs made with a low-code builder. The script languages supported are: Python, TypeScript, Go, Bash, SQL. Scripts can be generated by an AI assistant powered by OpenAI. +Scripts are turned into UIs and no-code modules, no-code modules can be composed into rich flows, and script and flows can be triggered from internal UIs made with a low-code builder. Supported script languages supported are: Python, TypeScript, Go, Bash, SQL.

@@ -36,7 +33,7 @@ Scripts are turned into UIs and no-code modules, no-code modules can be composed Try it - Docs - Discord - Hub - Contributor's guide

-# Windmill - Turn scripts into workflows and UIs that you can share and run at scale +# Windmill - Developer platform for APIs, background jobs, workflows and UIs Windmill is fully open-sourced (AGPLv3) and Windmill Labs offers dedicated instance and commercial support and licenses. @@ -45,12 +42,13 @@ dedicated instance and commercial support and licenses. https://github.com/windmill-labs/windmill/assets/122811744/0b132cd1-ee67-4505-822f-0c7ee7104252 -- [Windmill - Turn scripts into workflows and UIs that you can share and run at scale](#windmill---turn-scripts-into-workflows-and-uis-that-you-can-share-and-run-at-scale) +- [Windmill - Developer platform for APIs, background jobs, workflows and UIs](#windmill---developer-platform-for-apis-background-jobs-workflows-and-uis) - [Main Concepts](#main-concepts) - [Show me some actual script code](#show-me-some-actual-script-code) - [CLI](#cli) - [Running scripts locally](#running-scripts-locally) - [Stack](#stack) + - [Fastest Self-Hostable Workflow Engine](#fastest-self-hostable-workflow-engine) - [Security](#security) - [Sandboxing](#sandboxing) - [Secrets, credentials and sensitive values](#secrets-credentials-and-sensitive-values) @@ -58,13 +56,11 @@ https://github.com/windmill-labs/windmill/assets/122811744/0b132cd1-ee67-4505-82 - [Architecture](#architecture) - [How to self-host](#how-to-self-host) - [Docker compose](#docker-compose) - - [Kubernetes (k8s) and Helm charts](#kubernetes-k8s-and-helm-charts) - - [Postgres without superuser](#postgres-without-superuser) + - [Kubernetes (k8s) and Helm chartss](#kubernetes-k8s-and-helm-chartss) + - [Run from binaries](#run-from-binaries) + - [OAuth, SSO \& SMTP](#oauth-sso--smtp) - [Commercial license](#commercial-license) - - [OAuth for self-hosting](#oauth-for-self-hosting) - - [smtp for self-hosting](#smtp-for-self-hosting) - - [Resource types](#resource-types) - - [Manually fetch latest Windmill binary](#manually-fetch-latest-windmill-binary) + - [Integrations](#integrations) - [Environment Variables](#environment-variables) - [Run a local dev setup](#run-a-local-dev-setup) - [only Frontend](#only-frontend) @@ -75,22 +71,28 @@ https://github.com/windmill-labs/windmill/assets/122811744/0b132cd1-ee67-4505-82 ## Main Concepts 1. Define a minimal and generic script in Python, TypeScript, Go or Bash that - solves a specific task. Here sending a POST request. The code can be defined - in the provided Web IDE or synchronized with your own github repo: + solves a specific task. The code can be defined + in the [provided Web IDE](https://www.windmill.dev/docs/code_editor) or [synchronized with your own GitHub repo](https://www.windmill.dev/docs/advanced/cli/sync) (e.g. through [VS Code](https://www.windmill.dev/docs/cli_local_dev/vscode-extension) extension): + ![Step 1](./imgs/windmill-editor.png) -2. Your scripts parameters are automatically parsed and generate a frontend. - ![Step 2](./imgs/windmill-run.png) ![Step 3](./imgs/windmill-result.png) +2. Your scripts parameters are automatically parsed and [generate a frontend](https://www.windmill.dev/docs/core_concepts/auto_generated_uis). -3. Make it flow! You can chain your scripts or scripts made by the community + ![Step 2](./imgs/windmill-run.png) + + ![Step 3](./imgs/windmill-result.png) + +3. Make it [flow](https://www.windmill.dev/docs/flows/flow_editor)! You can chain your scripts or scripts made by the community shared on [WindmillHub](https://hub.windmill.dev). + ![Step 3](./imgs/windmill-flow.png) -4. Build complex UIs on top of your scripts and flows. +4. Build [complex UIs](https://www.windmill.dev/docs/apps/app_editor) on top of your scripts and flows. + ![Step 4](./imgs/windmill-builder.png) -Scripts and flows can also be triggered by a cron schedule '_/5 _ \* \* \*' or -through webhooks. +Scripts and flows can also be triggered by a [cron schedule](https://www.windmill.dev/docs/core_concepts/scheduling) (e.g. '_/5 _ \* \* \*') or +through [webhooks](https://www.windmill.dev/docs/core_concepts/webhooks). You can build your entire infra on top of Windmill! @@ -146,7 +148,9 @@ instance from local commands. See You can run your script locally easily, you simply need to pass the right environment variables for the `wmill` client library to fetch resources and variables from your instance if necessary. See more: -. +. + +To develop & test locally scripts & flows, we recommend using the Windmill VS Code extension: . ## Stack @@ -167,6 +171,14 @@ variables from your instance if necessary. See more: - python runtime is python3 - golang runtime is 1.19.1 +## Fastest Self-Hostable Workflow Engine + +We have compared Windmill to other self-hostable workflow engines (Airflow, Prefect & Temporal) and Windmill is the most performant solution for both benchmarks: one flow composed of 40 lightweight tasks & one flow composed of 10 long-running tasks. + +All methodology & results on our [Benchmarks](https://www.windmill.dev/docs/misc/benchmarks/competitors#airflow-setup) page. + +![Fastest workflow engine](./imgs/fastest.png) + ## Security ### Sandboxing @@ -204,10 +216,14 @@ back to the database is ~50ms. A typical lightweight deno job will take around We only provide docker-compose setup here. For more advanced setups, like compiling from source or using without a postgres super user, see -[documentation](https://www.windmill.dev/docs/advanced/self_host). +[Self-Host documentation](https://www.windmill.dev/docs/advanced/self_host). ### Docker compose +Windmill can be deployed using 3 files: ([docker-compose.yml](./docker-compose.yml), [Caddyfile](./Caddyfile) and a [.env](./.env)) in a single command. + +Make sure Docker is started, and run: + ``` curl https://raw.githubusercontent.com/windmill-labs/windmill/main/docker-compose.yml -o docker-compose.yml curl https://raw.githubusercontent.com/windmill-labs/windmill/main/Caddyfile -o Caddyfile @@ -222,7 +238,9 @@ The default super-admin user is: admin@windmill.dev / changeme. From there, you can follow the setup app and create other users. -### Kubernetes (k8s) and Helm charts +More details in [Self-Host Documention](https://www.windmill.dev/docs/advanced/self_host#docker). + +### Kubernetes (k8s) and Helm chartss We publish helm charts at: . @@ -247,14 +265,16 @@ Workspace SSO, Microsoft/Azure and Okta) directly from the UI in the superadmin settings. Do note that there is a limit of 50 SSO users on the community edition. +[See documentation](https://www.windmill.dev/docs/misc/setup_oauth). + ### Commercial license -To self-host Windmill, you must respect the terms of the AGPLv3 license which +To self-host Windmill, you must respect the terms of the [AGPLv3 license](https://www.gnu.org/licenses/agpl-3.0.en.html) which you do not need to worry about for personal uses. For business uses, you should -be fine if you do not re-expose it in any way Windmill to your users and are +be fine if you do not re-expose Windmill in any way to your users and are comfortable with AGPLv3. -To re-expose any Windmill parts to your users as a feature of your product, or +To [re-expose any Windmill parts to your users](https://www.windmill.dev/docs/misc/white_labelling) as a feature of your product, or to build a feature on top of Windmill, to comply with AGPLv3 your product must be AGPLv3 or you must get a commercial license. Contact us at if you have any doubts. @@ -264,9 +284,11 @@ your current infrastructure to Windmill, support with tight SLA, and our global cache sync for high-performance/no dependency cache miss of cluster from 10+ nodes to 200+ nodes. -### Resource types +### Integrations -You will also want to import all the approved resource types from +In Windmill, integrations are referred to as [resources and resource types](https://www.windmill.dev/docs/core_concepts/resources_and_types). Each Resource has a Resource Type that defines the schema that the resource needs to implement. + +On self-hosted instances, you might want to import all the approved resource types from [WindmillHub](https://hub.windmill.dev). A setup script will prompt you to have it being synced automatically everyday. @@ -394,4 +416,4 @@ running options. ## Copyright -Windmill Labs, Inc 2023 +Windmill Labs, Inc 2023 \ No newline at end of file diff --git a/imgs/diagram.svg b/imgs/diagram.svg index 86a026a323..f807fd8887 100644 --- a/imgs/diagram.svg +++ b/imgs/diagram.svg @@ -1,3 +1,775 @@ - - -
Custom scripts/flows
Custom scripts/f...

Workers

Workers
Schedule
Schedule
Generated UI
Generated UI
HTTP Post
HTTP Post
Windmill ( On prem or hosted)
Windmill ( On prem or hosted)
Windmill Hub
Windmill Hub
K/V Store
K/V Store
OAuth
OAuth
API Token

API Token
Sandbox
Sandbox
Sandbox
Sandbox
Sandbox
Sandbox
Sandbox
Sandbox
Database
Database
Custom frontend
Custom frontend
Webhook
Webhook
VPC (Optional)
VPC (Optional)
Internet
Internet
Slack command
Slack command
Text is not SVG - cannot display
\ No newline at end of file + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/imgs/fastest.png b/imgs/fastest.png new file mode 100644 index 0000000000..b43a26ba9f Binary files /dev/null and b/imgs/fastest.png differ diff --git a/imgs/windmill-banner.png b/imgs/windmill-banner.png index 653ebd2f30..76d3467c2a 100644 Binary files a/imgs/windmill-banner.png and b/imgs/windmill-banner.png differ diff --git a/imgs/windmill-editor.png b/imgs/windmill-editor.png index 72ccc966cd..16b0dcfdc9 100644 Binary files a/imgs/windmill-editor.png and b/imgs/windmill-editor.png differ diff --git a/imgs/windmill-result.png b/imgs/windmill-result.png index 74ba44040c..fd34ad63a8 100644 Binary files a/imgs/windmill-result.png and b/imgs/windmill-result.png differ diff --git a/imgs/windmill-run.png b/imgs/windmill-run.png index 0f0038f985..d2e760f912 100644 Binary files a/imgs/windmill-run.png and b/imgs/windmill-run.png differ