20 Commits

Author SHA1 Message Date
Alexander Petric
9f10b44c18 update cloudformation template to use latest cli/images + fix cl… (#8417)
* fix: update cloudformation template to use latest cli/images + fix cleanup script

* fix: narrow SG cleanup to k8s-created groups + add CLI install error handling

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 20:12:04 +00:00
Ruben Fiszel
f02ef6d03c refactor: switch operator from CRD to ConfigMap (#7972)
* refactor: switch operator from CRD to ConfigMap

Replace the WindmillInstance CRD with a plain ConfigMap for the K8s
operator. This simplifies deployment (no CRD to install/manage, no
ClusterRole for custom API groups) while keeping the same config schema.

- Replace crd_ee.rs with configmap_ee.rs (parses data.spec YAML key)
- Rewrite reconciler_ee.rs: ConfigMap watcher + Event recorder instead
  of CRD Controller + status subresource
- Add license_key preservation: if absent/empty in ConfigMap but present
  in DB, the DB value is kept
- Remove print_crd_yaml() and "operator crd" subcommand
- Drop schemars, chrono, instance_config_schema dependencies
- Delete manifests/crd.yaml
- Update K8s example and README for ConfigMap approach
- RBAC now only needs a namespace-scoped Role (not ClusterRole)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add superadmin YAML export endpoint and remove cache_clear from operator config

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 00:06:56 +00:00
Ruben Fiszel
82e5f6de48 feat: add Kubernetes operator and instance settings YAML editor (#7836)
* Add windmill-operator crate for Kubernetes CRD-based instance config

Introduces a new `windmill-operator` crate that enables declarative
management of Windmill instance configuration via a Kubernetes
`WindmillInstance` CRD. The operator watches CRD resources and performs
full declarative sync of global_settings and worker configs to the
database, supporting GitOps workflows for instance-level configuration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add tests for windmill-operator CRD and db_sync

- 9 unit tests for CRD serialization, deserialization, metadata, and
  status field behavior
- 15 integration tests for db_sync using #[sqlx::test] with full
  declarative sync coverage: upsert, delete, protected keys,
  idempotency, worker config prefix handling, and end-to-end sync

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Replace untyped BTreeMap CRD fields with typed structs for schema validation

GlobalSettings, SmtpSettings, IndexerSettings, and WorkerGroupConfig now
have explicit typed fields with serde(flatten) catch-all for forward
compatibility. The generated CRD YAML includes a full OpenAPI v3 schema
that Kubernetes validates on kubectl apply.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Type opaque serde_json::Value CRD fields with real structs

Replace most remaining serde_json::Value fields in WindmillInstance CRD
with properly typed structs derived from the codebase:
- oauths: BTreeMap<String, OAuthClient>
- otel: OtelSettings
- otel_tracing_proxy: OtelTracingProxySettings with ScriptLang enum
- critical_error_channels: Vec<CriticalErrorChannel> (untagged enum)
- critical_alerts_on_db_oversize: DbOversizeAlert
- ducklake_settings: DucklakeSettings with nested catalog/storage types
- custom_instance_pg_databases: CustomInstancePgDatabases
- autoscaling (worker config): AutoscalingConfig with integration struct
- custom_tags, default_tags_workspaces: Vec<String>
- default_tags_per_workspace: bool

Still opaque (serde_json::Value): object_store_cache_config (kube-core
can't generate schemas for internally-tagged enums), secret_backend
(EE-private), slack, teams (no clear struct definitions).

Regenerated CRD YAML with full OpenAPI schema (352→703 lines).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Unify instance config types and add bulk GET/PUT API

Move all typed settings (GlobalSettings, WorkerGroupConfig, etc.) from
windmill-operator/crd.rs into windmill-common/instance_config.rs so both
the API server and operator share a single source of truth. Add diff/apply
logic (Merge mode for UI, Replace mode for operator) and InstanceConfig::from_db().

Add GET/PUT /settings/instance_config endpoints so the frontend loads all
settings in 1 call instead of 42, and saves with a single bulk PUT. The
backend handles the diff internally, running pre-write hooks for changed keys.

Refactor windmill-operator/db_sync.rs to use the shared diff+apply functions
and slim crd.rs down to the CRD wrapper with re-exports.

Includes 32 unit tests and 30 integration tests covering serialization,
diff logic, DB roundtrips, protected settings, and edge cases.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add Form/YAML toggle to instance settings UI

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: show Form/YAML toggle regardless of hideTabs prop

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: replace toggle button group with simple YAML toggle

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: mask sensitive fields in YAML view with show/hide toggle

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: hide internal settings and mask sensitive fields in YAML view

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: hide jwt_secret and min_keep_alive_version from API and config exports

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* all

* feat: add secretKeyRef support for sensitive fields in operator CRD

Allow sensitive fields (license_key, hub_api_secret, scim_token,
smtp_password, OAuthClient.secret, custom PG user_pwd) to reference
Kubernetes Secrets via the standard secretKeyRef pattern instead of
inlining values as plaintext YAML. The reconciler resolves all refs
by reading K8s Secrets before syncing to the database.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

* all

* all

* fix: merge main and update dev environment docs

Resolve merge conflicts from origin/main, fix duplicate
UV_INDEX_STRATEGY_SETTING import, and add Playwright MCP
testing instructions to CLAUDE.md.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

* fix: init tracing for CLI subcommands and deduplicate setting side-effects

Initialize tracing subscriber before early-return CLI paths (sync-config,
operator) so tracing calls are not silently dropped. Refactor
set_global_setting_internal to call run_setting_pre_write_hook instead of
duplicating the side-effect logic.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add `wmill instance get-config` CLI command

Dumps the current instance config (global settings + worker configs) as
YAML. Supports --output-file to write to a file instead of stdout.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 22:42:01 +00:00
Alexander Petric
2620300ce9 AWS cloudformation quicklaunch (#5650) 2025-04-21 16:26:28 +02:00
Alexander Petric
3bfaf73a83 otel with grafana loki tempo (#5011)
* otel with grafana loki tempo

* add logging
2025-01-04 03:28:55 +01:00
Alexander Petric
0a7268714c give option to run container as non-root windmill user (#4959)
* main docker file

* fix docker image build test tag

* don't remove tag

* make root user default

* chown tmp folder

* create search and logs folder in order to inherite windmill user permissions

* Dockerfile

* lsp non root

* improving lsp image to get rid of critical vulnerabilities

* formatting

* support /root/.cache mount

* make the cache mount backwards compatible
2025-01-03 16:51:31 +01:00
Alexander Petric
736e241e71 adding otel example + small typos in instance settings (#5009) 2025-01-03 16:26:13 +01:00
hkader-tl
5172c13ab8 fix: ECS terraform db url + ami issues (#4924)
Modifications to improve the terraform experience for bringing up a
windmill cluster. Due to the default RDS settings for the version in the
terraform, we do not want to disable ssl in connecting. Additionally,
change the TF to automatically pull the latest 2023 AMI. Given this is
meant to be a quick start example, this seems better than hard-coding
since eventually the hard-coded ami doesn't exist.
2024-12-14 13:21:14 +01:00
David Zhang
85e50e3a60 Update AMI used in aws-ecs-terraform example (#3448)
Previous AMI no longer exists, updated to latest Amazon Linux 2023
2024-03-21 10:45:19 +01:00
Guillaume Bouvignies
2edba18038 feat: Tag override for scheduled scripts (#2998)
* feat: Tag override for scheduled scripts

* Refactor WorkerTagPicker to component

* Fix test
2024-01-13 00:06:54 +01:00
Guillaume Bouvignies
a44d42fa77 docs: Update ECS terraform networking (#2984) 2024-01-10 17:41:36 +01:00
Guillaume Bouvignies
64315cef08 docs: ECS log format JSON (#2981) 2024-01-10 13:50:46 +01:00
Guillaume Bouvignies
224f18915d docs: AWS ECS terraform deploy (#2980) 2024-01-10 08:17:38 +01:00
Ruben Fiszel
329cf836bd docs: Example on how to monitor Windmill (#2938)
* docs: Example on how to track arbitrary job metrics

* Add how to monitor Windmill servers and workers

* Update README.md

---------

Co-authored-by: gbouv <guillaume@windmill.dev>
2023-12-30 03:30:48 +01:00
Guillaume Bouvignies
ba036e0576 docs: How to query gRPC service using Bun (#2863)
* docs: How to query gRPC service using Bun

* move readme to blogpost
2023-12-20 09:14:01 +01:00
Guillaume Bouvignies
60c2d71611 docs: Processing Kafka messages with Windmill Perpetual scripts (#2859)
* docs: Processing Kafka messages with Windmill Perpetual scripts

* Add while loop and update readme

* move README to blog post
2023-12-19 12:55:38 +01:00
Bret Mogilefsky
5de6973bba Fix typo (#2765) 2023-12-04 07:36:59 +01:00
Guillaume Bouvignies
bb3960c509 fix: minor fixes to private NPM and python registries to get everything working (#2728) 2023-11-29 16:30:45 +01:00
Guillaume Bouvignies
2081e7a8ff chore: Add examples on how to deploy private registries (#2719)
* chore: Add examples on how to deploy private registries

* chore: example for private pypiserver with https

* Allow DENO_CERT certificate with native workers

* Add BUN_TLS_REJECT_UNAUTHORIZED
2023-11-29 11:38:23 +01:00
Guillaume Bouvignies
83eaa51fd8 fix: Deno can talk to private NPM registries behind HTTPS (#2713) 2023-11-27 21:46:19 +01:00