Commit Graph

5945 Commits

Author SHA1 Message Date
Ruben Fiszel
350064c992 nit allow_cache 2025-09-02 17:00:58 +00:00
Ruben Fiszel
c2e403a52a nit allow_cache 2025-09-02 16:29:36 +00:00
Ruben Fiszel
0a9e0d922e nit 2025-09-02 16:28:57 +00:00
Ruben Fiszel
593da3213f add variable caching api 2025-09-02 14:34:46 +00:00
dieriba
f3fd1e90b0 feat: refactor trigger crud (#6472)
* base

* base crate

* websocket updated

* crud refactored

* fix and nits

* fix compiler warning, nits and update repo ref

* fix oss

* fix compilation

* update ref

* fix

* update feature

* listener base

* add listener

* refactor logic done and implemented for postgres

* fix capture

* websocket

* implem for all triggers

* update sqlx,repo ref and fix

* fix oss

* unify struct fix websocket

* nits and fix oss runtime axum error

* perf cache query

* update .sqlx

* update .sqlx

* fix

* fix unused

* fix

* fix

* nits and fix http handler update endpoint

* update .sqlx

* update repo ref

* nits

* fix

* update ref

* fix

* update .sqlx
2025-09-02 11:28:06 +00:00
Ruben Fiszel
c08282a36f improve the notify mechanism for resource change 2025-09-02 11:08:05 +00:00
hugocasa
fbd942f179 fix(backend): retrieve root_job for MiniPulledJob + fix root job for flow jobs (#6490)
* feat(backend): retrieve root_job for MiniPulledJob + fix root job for flow jobs

* nit

* nits

* set root_job only if additional information meaning if not equal to innermost or parent

* nit

* feat: add root job env var

* nits

* fix build

* nit

* fix build

* nit

* sqlx
2025-09-02 09:44:09 +00:00
centdix
7df5f5453f feat(aichat): allow custom system prompt for each mode (#6500)
* add ui to add custom system prompt

* implement backend changes

* add custom prompt in system prompt

* add in openapi

* add max length

* add backend validation
2025-09-02 09:27:37 +00:00
claude[bot]
db36a323a7 feat(perf): add 60-second local cache for variables and resources(#6511)
* feat: add 60-second cache for variables and resources with PostgreSQL invalidation

- Add new var_resource_cache module with 60-second TTL
- Implement PostgreSQL NOTIFY/LISTEN for immediate cache invalidation
- Cache get_variable() for non-secret, non-encrypted reads
- Cache get_resource_value() for all read operations
- Add database triggers on variable/resource table changes
- Initialize cache system in main.rs after database connection
- Add Clone derive to ListableVariable for cache compatibility

Performance benefits:
- Avoids database queries on cache hits
- Immediate invalidation ensures data consistency
- Selective caching respects security constraints

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: move cache to API layer with allow_cache query param

- Move variable/resource cache from windmill-common to windmill-api
- Add allow_cache query parameter to variables and resources endpoints
- Follow raw script cache pattern with timestamp + value structure
- Create proper database migration for notification triggers
- Include encrypted values in cache when appropriate
- Only activate caching when explicitly requested

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

* refactor: move cache to API layer with allow_cache query param

Move PostgreSQL LISTEN logic to main.rs following established pattern
- Remove custom listener initialization from cache module
- Add cache invalidation channels to centralized notification handler
- Simplified cache module to only handle cache operations
- Follow raw script cache pattern for notification handling

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
2025-09-02 07:48:03 +00:00
centdix
e197456b06 fix(mcp): fix path transformation (#6508)
* fix path transformation

* fix mcp url copy

* safer code
2025-09-02 06:50:14 +00:00
Ruben Fiszel
0368c5ca83 remove pin ambiguity on cache 2025-09-01 14:59:25 +00:00
Ruben Fiszel
16912b484d feat: cache relative imports (#6504)
* all

* all

* update
2025-09-01 14:55:08 +00:00
Ruben Fiszel
ac04779df1 chore(main): release 1.538.0 (#6495)
* chore(main): release 1.538.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-09-01 12:42:19 +00:00
hugocasa
c01ed2d62a fix(backend): http triggers early return (#6501) 2025-09-01 12:33:24 +00:00
centdix
8b3ae1984c feat(aichat): cache prompts when using anthropic models (#6489)
* working draft

* add caching

* add anthropic.ts

* cleaning

* better typing

* nit

* use anthropic sdk

* draft

* draft parser

* cleaning

* update openai sdk + use stream method

* fix anthropic base url

* cleaning

* better error handling + cleaning

* rollback to old parsing

* nit

* set strict to false for mistral
2025-09-01 08:55:15 +00:00
Ruben Fiszel
5510b1dd95 typo env nit 2025-08-31 12:23:37 +00:00
claude[bot]
8f46066a64 feat(perf): add 300-second local cache for variable crypt retrieval (#6483)
* feat: add 60-second cache for workspace key retrieval

This implements a cache with 60-second staleness for the get_workspace_key
function to reduce database queries for workspace encryption keys. The cache
follows the same pattern as the existing CUSTOM_ENVS_CACHE but with a
shorter expiration time.

Requested by @rubenfiszel

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* Extend cache staleness from 60 to 300 seconds

* feat: add cache invalidation notifications for workspace keys

Add PostgreSQL LISTEN/NOTIFY mechanism to invalidate workspace key cache
across all servers and workers when workspace keys change.

- Add database migration with trigger function for workspace_key changes
- Add notification handler in main.rs to remove from WORKSPACE_KEY_CACHE
- Follow same pattern as existing workspace environment cache invalidation
- Ensures distributed cache consistency for workspace encryption keys

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

* finish

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
2025-08-31 12:19:59 +00:00
Ruben Fiszel
9041bb9c7a add TAGS_ARE_SENSITIVE env variable 2025-08-30 10:26:24 +00:00
Ruben Fiszel
89e20f5197 chore(main): release 1.537.1 (#6491)
* chore(main): release 1.537.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-08-29 09:43:00 +00:00
Ruben Fiszel
641d5651c5 fix: fix error handling of pre-processor steps 2025-08-29 09:38:25 +00:00
Ruben Fiszel
7a1c28f6d7 chore(main): release 1.537.0 (#6486)
* chore(main): release 1.537.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-08-28 19:29:04 +00:00
hugocasa
2ce11cea3b prevent past due team plans to overuse (#6487)
* prevent past due team plans to overuse

* nit

* update ee ref
2025-08-28 16:56:57 +00:00
pyranota
006f32602c fix: fix relative imports cache invalidation (#6468)
* fix: new script on deps job for secondary scripts

Signed-off-by: pyranota <pyra@duck.com>

* make it work (dirty)

Signed-off-by: pyranota <pyra@duck.com>

* reduce db calls

* remove `triggered_by_relative_import`

Signed-off-by: pyranota <pyra@duck.com>

* add comment to common_dependency_path

Signed-off-by: pyranota <pyra@duck.com>

* add fallback to old behavior

Signed-off-by: pyranota <pyra@duck.com>

* remove TODOs

Signed-off-by: pyranota <pyra@duck.com>

* pass deployed hash to git sync handler function

Signed-off-by: pyranota <pyra@duck.com>

* fix ci

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
2025-08-27 17:44:35 +00:00
Ruben Fiszel
4ad0d255f3 feat: autovacuum or high intensity tables 2025-08-27 17:29:06 +00:00
Ruben Fiszel
86f41ffcde minor nits fix 2025-08-27 15:08:58 +00:00
Ruben Fiszel
eceab931af chore(main): release 1.536.0 (#6471)
* chore(main): release 1.536.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-08-27 13:43:03 +00:00
Ruben Fiszel
41a8727252 fix: do not require locked for scheduled jobs 2025-08-27 13:31:23 +00:00
Ruben Fiszel
a245f70179 not_found_if_none displays location 2025-08-27 13:08:52 +00:00
Alexander Petric
ceb9150f43 feat: email triggers extra args in 'to' header (#6476)
* feat: email triggers extra args in 'to' header

* ee-repo + script helpers
2025-08-27 11:12:05 +00:00
pyranota
5808840b78 fix(go): could not read Username for 'xyz': terminal prompts disabled (#6478)
* fix(go): could not read Username for 'xyz': terminal prompts disabled

Signed-off-by: pyranota <pyra@duck.com>

* remove unused import

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
2025-08-27 11:11:54 +00:00
pyranota
475f405d06 fix(go): exec: "git": executable file not found (#6475) 2025-08-26 16:07:15 +00:00
Ruben Fiszel
0ceb72f012 chore(main): release 1.535.0 (#6460)
* chore(main): release 1.535.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-08-25 14:43:29 +00:00
Ruben Fiszel
a691ae2883 chore(main): release 1.534.1 (#6458)
* chore(main): release 1.534.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-08-25 13:37:53 +00:00
Diego Imbert
16d233bf46 fix: add alias to subquery for older postgres versions (#6455)
* add alias to subquery for older postgres versions

* Update SQLx metadata

---------

Co-authored-by: GitHub Action <action@github.com>
2025-08-25 13:19:10 +00:00
Ruben Fiszel
082312000f chore(main): release 1.534.0 (#6452)
* chore(main): release 1.534.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-08-25 12:46:19 +00:00
hugocasa
8d31c2ab0d feat(backend): support unencrypted connection to mssql (#6453) 2025-08-25 12:39:37 +00:00
Ruben Fiszel
97ed4a539b nits cleanup + faster script index #6450 2025-08-23 22:42:39 +00:00
Ruben Fiszel
8964896c13 chore(main): release 1.533.1 (#6449)
* chore(main): release 1.533.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-08-23 08:02:32 +00:00
Ruben Fiszel
99666426ec nits 2025-08-23 07:54:59 +00:00
Ruben Fiszel
5b338bb749 chore(main): release 1.533.0 (#6448)
* chore(main): release 1.533.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-08-23 02:53:29 +01:00
hugocasa
c13747cda9 fix(frontend): ai agent flow status + UI nits (#6447)
* fix(frontend): ai agent flow status

* nit: prevent undefined node issue

* feat: UI nits + flow status select iter fix

* nit ai agent color in picker
2025-08-22 18:51:13 +01:00
Ruben Fiszel
2046b64ec8 chore(main): release 1.532.0 (#6439)
* chore(main): release 1.532.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-08-22 11:05:42 +01:00
hugocasa
7da79a8bc5 feat: json schema resource (#6433)
* feat: json schema resource

* feat: cache

* fix cleanup

* fix: use format instead of custom property
2025-08-22 10:39:13 +01:00
Ruben Fiszel
ee5e39a3d5 chore(main): release 1.531.0 (#6429)
* chore(main): release 1.531.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-08-22 08:01:58 +00:00
pyranota
d152e8e58f add go1_22_compat annotation (#6432)
* bump go to 1.25

Signed-off-by: pyranota <pyra@duck.com>

* Update Dockerfile

* add comms

---------

Signed-off-by: pyranota <pyra@duck.com>
2025-08-21 15:14:29 +00:00
hugocasa
958e8af782 feat: ai agent steps (#6393)
* feat: ai agent steps base

* better backend and graph

* feat: anthropic, log viewer

* nit

* fix(frontend): hide tool nodes from timeline

* move ai agent actions from flow status to flow status module

* nits and workspace/hub scripts support

* tmp ref

* fix merge

* feat: display agent tools status in the graph

* fix reactivity

* fix flow status

* nit
2025-08-20 22:40:57 +00:00
Ruben Fiszel
58bc913dfc chore(main): release 1.530.0 (#6426)
* chore(main): release 1.530.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-08-20 18:20:55 +00:00
centdix
ae49737676 feat(mcp): add script preview testing tool (#6417)
* add endpoint to test script

* add same for flow

* better tool spec + remove flow preview from tools

* fix

* fix wrong required fields

* feat(mcp): add warning for missing required fields in schema properties

- Add stderr warning when x-mcp-required-fields contains fields not found in body schema properties
- Prevents silent misconfigurations in MCP tool generation
- Helps debug schema validation issues

Co-authored-by: centdix <centdix@users.noreply.github.com>

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: centdix <centdix@users.noreply.github.com>
2025-08-20 14:50:46 +02:00
Ruben Fiszel
26a47ee699 chore(main): release 1.529.0 (#6423)
* chore(main): release 1.529.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-08-20 09:32:32 +00:00
claude[bot]
49ed757424 feat: add prometheus metric queue_running_count (#6413)
* feat: add prometheus metric queue_running_count

Adds a new Prometheus metric queue_running_count that tracks the number
of currently running jobs per tag, similar to the existing queue_count
metric but filtered for running=true instead of running=false.

Changes:
- Added get_queue_running_counts() function in windmill-common/src/queue.rs
- Added QUEUE_RUNNING_COUNT Prometheus metric in monitor.rs
- Added /workers/queue_running_counts API endpoint
- Updated OpenAPI specification
- Added SQLx query cache entry

Requested by @rubenfiszel

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* Remove database insertion for queue_running_count metrics

Keep Prometheus metrics and API endpoint functionality while removing
the database INSERT statements as requested.

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

* sqlx

* improve logic

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-08-19 22:12:36 +00:00