* feat: add 60-second cache for workspace key retrieval
This implements a cache with 60-second staleness for the get_workspace_key
function to reduce database queries for workspace encryption keys. The cache
follows the same pattern as the existing CUSTOM_ENVS_CACHE but with a
shorter expiration time.
Requested by @rubenfiszel
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* Extend cache staleness from 60 to 300 seconds
* feat: add cache invalidation notifications for workspace keys
Add PostgreSQL LISTEN/NOTIFY mechanism to invalidate workspace key cache
across all servers and workers when workspace keys change.
- Add database migration with trigger function for workspace_key changes
- Add notification handler in main.rs to remove from WORKSPACE_KEY_CACHE
- Follow same pattern as existing workspace environment cache invalidation
- Ensures distributed cache consistency for workspace encryption keys
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
* finish
---------
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
* feat: ai agent steps base
* better backend and graph
* feat: anthropic, log viewer
* nit
* fix(frontend): hide tool nodes from timeline
* move ai agent actions from flow status to flow status module
* nits and workspace/hub scripts support
* tmp ref
* fix merge
* feat: display agent tools status in the graph
* fix reactivity
* fix flow status
* nit
* feat: add prometheus metric queue_running_count
Adds a new Prometheus metric queue_running_count that tracks the number
of currently running jobs per tag, similar to the existing queue_count
metric but filtered for running=true instead of running=false.
Changes:
- Added get_queue_running_counts() function in windmill-common/src/queue.rs
- Added QUEUE_RUNNING_COUNT Prometheus metric in monitor.rs
- Added /workers/queue_running_counts API endpoint
- Updated OpenAPI specification
- Added SQLx query cache entry
Requested by @rubenfiszel
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* Remove database insertion for queue_running_count metrics
Keep Prometheus metrics and API endpoint functionality while removing
the database INSERT statements as requested.
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
* sqlx
* improve logic
---------
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
* upgrade duckdb
* basic ducklake works
* ducklake works with custom db catalogs
* fix: pwsh skip already installed modules outside of cache (#6037)
* improve query performance of user stats
* separate ducklake_catalog db
* ducklake settings
* DucklakeSettings frontend
* Ducklake ws settings saved in backend
* fetch ducklake catalog resource
* Ducklake works with configured s3 storage
* Ducklake as asset
* ducklake asset icon
* Fix duckdb array and object args not working properly (#6254)
* Fix bug with comments in duckdb
* Avoid multiple queries when doing ATTACH ducklake
* trunc sig no longer needed now that comments are trimmed
* cache DuckdbConnectionSettingsResponse
* duplicated code
* transform_attach_ducklake contributes to duckdb_connection_settings_cache
* eliminate the need for used_storages
* nit
* cleaner management of the bigquery credentials file
* DBManagerDrawer refactor to prepare for Ducklake
* get ducklake schema
* implement delete for ducklake
* load column metadata for ducklake
* Select query works for ducklake, basic db explorer works !
* duckdb count query
* Support all db ops for ducklake
* clean migrations
* SQL repl for Ducklake
* fix broken database studio
* nit
* assert function
* Ducklake in Editor Bar
* default ducklake syntax + allow extra args
* DucklakeCatalogWizard UI
* nit + remove extra $
* modal when databases do not exist
* cannot be windmill
* Ducklake works safely with instance database
* Avoid sending instance db credentials on network
* resource leak security
* remove fetch_attach_db_conn_str
* prevent instance pg password leak
* hide asset usage count when not available
* case unsensitivity duckdb
* warnings
* disable instance catalog
* use shorthand syntax when inserting with EditorBar
* Instance ducklake catalog is now safe to use
* use safer argon2 pwd
* update package json parsers
* update package json
* better msgs
* tooltips
* disable explore button until saved
* nit
* fix warnings
* better ducklake_user password management
* nit
* Sanitize passwords from errors in ducklake
* DisplayResult broken in job result
* remove superadmin requirement to check databases_exist
* duckdb_connection_settings_v2_inner
* Ducklake works on agent worker (finally)
* ci
* #[allow(dead_code)]
* fix openapi missing response
* Separate +Database button for DuckDB in EditorBar
* Fix dropdown in ducklake settings
* Attempt to fix migration race condition in CI
* update sqlx failing for some offline queries
* avoid temp password for ducklake_user
* nits
* ducklake settings nits
* update duckdb default script
* fix sql repl resetting text on refresh
* avoid pgcrypto extension
---------
Co-authored-by: HugoCasa <hugo@casademont.ch>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
* Moved logic to FlowAssetsProvider
* Remove assetsMap in flow
* do not parse everything on mount + only check for missing assets fields
* add assets field in backend
* remove fallbackAccessTypes
* better structure and less queries / parsing
* Fix assets not showing when pulling raw_flow from jobs
* flow assets ctx for job run
* Fix transitive assets fetching
* Fix input args asset node
* enablePathScriptAndFlowAssets flag
* edit btn for variable
* untrack refresh
* move parseInputArgsAssets
* Assets tab in runs
* Update FlowStatusViewerInner to svelte 5 + fix asset sync bug
* avoid toast error on bad resource
* fetch res metadata for input arg asset
* Job assets viewer in run page
* r/w selector
* remove indigo badge
* store alt_access_type state in ScriptEditor
* Don't parse assets in flow script editor
* Add alt_access_type in backend
* show Read as selected by default to avoid giving the feeling of having made a decision
* keep alt_access_type when reparsing in flow raw scripts
* Remove variable asset kind, and save assets for scripts
* remove all backend asset parsing
* R/W/RW selector button nits
* fix insert into assets not saving alt access type
* support named arguments in python asset parser
* improve asset usage drawer R/W indicator
* update legacy $res: syntax
* reactivity issue
* remove last variable asset stuff
* sqlx prepare
* tooltip explainer
* deprecated variable asset nit
* assets migration
* parse assets (duckdb)
* iterate on assets
* S3 object Preview
* remove pagination
* filterText
* better occurence list
* tweak
* assets in JobPreview
* clone impl
* AssetsDetectedBadge
* improve DbManagerButton + asset dropdown button
* edit resource btn
* warning when incorrect resource
* +Resource in DuckDB
* +S3 Object editor bar
* nit fix rename
* flow asset badge
* More Generic OnChange
* Highlight assets used in modules
* Show occurence count in flow
* Better UX, avoid moving parts
* nit
* Asset nodes
* move to dedicated Asset ctx
* fix layoutNodes not handling first assetsMap
* explore asset btn in flow asset node
* correct offset
* single computeAssetNodes function
* Fix y positioning of nodes with assets
* resource editor
* write mode node (ui)
* accessType in ctx + fix insert button positioning
* right positioning when mixing read and write nodes
* right positioning when mixing R and W assets
* Better layout fix algorithm
* listAssetsByUsage and asset nodes on transitive usages
* refactor + remove linkAssets
* Refactor to allow for custom R/W modes
* AssetsDropdownButton in flow script editor
* R/W/RW selection and changes node pos in flow
* layoutNodes doesnt need recompute now
* fix wrong assumption that nodes recompute when assets change
* r/w/rw multi toggle
* MultiToggle cool animation + clearable
* rename + 1px nit
* remove mini toggle button group, use ToggleButtonGroup
* Combinator parser that detects R / W asset context
* nit fix missing flex-1
* missing order by
* better ui indication for access type
* special x offset case when only one asset node for clarity
* parse getResource in TS with swc ecma parser
* support load and write s3 detection in TS
* Python asset parser
* support wmill api calls without special $res: or s3:// syntax
* detect out of context asset uris python
* do not use access type override when not ambiguous in flow graph
* parse_assets match case in rust
* AsRef<str> refactor
* From impl
* Save flow assets
* Save script asset usages + fixes + save fallback access types
* asset sub icon
* max total asset node width to avoid overlap
* small refactor
* don't parse comments in duckdb assets
* fix assets clearing on parse error
* fix script asset save in wrong place
* load initial asset fallback access types
* support variables
* ui fixes
* Support S3Object as URI in TS client
* support new syntax in python client
* Support +S3Object in EditorBar for TS and python
* Reduce resource requests in assets page
* import windmill client when necessary
* update s3Types.d.ts
* nit fix
* Show input resources and s3 objects as assets
* improve asset icons
* DarkModeObserver refactor
* asset page tabs
* Moved resource variables and s3object pages to assets tabs
* fetch resource usages
* Get variables usages
* move assets usage dropdown to component
* Revert "move assets usage dropdown to component"
This reverts commit 622ea4ab12.
* Revert "Get variables usages"
This reverts commit b11ced4e29.
* Revert "fetch resource usages"
This reverts commit aa5187ad4b.
* Revert "Moved resource variables and s3object pages to assets tabs"
This reverts commit 4430487be4.
* Revert "asset page tabs"
This reverts commit dacc2f0da5.
* move assets usage dropdown to component
* asset icon in asset pages
* tooltip
* details
* Storage selector in S3 File Picker
* make edge less opaque
* Refactor computeAssetNodes to separate in and out nodes
* AssetsOverflowedNode
* nits
* fix assets not being parsed in flows sometimes
* show asset kind and resource_type
* ui nits
* support res:// in duckdb
* add banner for old deployments
* Fix permissionning
* fix broken disable /enable all
* assets page view permission for operators
* Disable ExploreAssetButton for operators
* asset kind as subtitle
* do not spam getResource in assets page. prob. revert fail
* update assets page on workspace change
* reload storage names on ws change
* delete assets on archive / deletion
* sqlx prepare
* missing update when updating user
* add indexes on asset
* better message
* missing loadInit: false
* dead code
* use transaction
* typo
* update package.json
* update package.json
---------
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
* app compiles with every ee substituted
* Replace all oss files content
* Revert "Replace all oss files content"
This reverts commit ea4017d59f.
* delete all ee
* hide all _ee files under private flag
* hide every oss stuff when private flag set
* pub use *
* gitignore and substitute script
* pub mod for ee needed for ee repo
* small mistakes
* remove oidc_oss impl
* ee ref (temp)
* ee ref
* fix --all-features selecting private in OSS CI
* ee repo ref
* allow unused
* backend
* iterate
* all
* all
* all
* iterate
* revert
* all
* add tracing to get of authed client
* all
* all
* lal
* all
* update
* fix
* push
* all
* all
* revert
* frontend
* fix checks
* avoid deadlock
* safer
* fix
* fix
* make resolver
* more updates
* fix build
* fix raw_dependencies job type
* compat with http agent workers
* refactor
* rename
* more refactor
* cleanup
* more tests
* fix s3
* small fixes
* more fixing
* fix endpoint
* nit: update comment
* update ee ref
* update ee ref
* update ee ref
* implement safer `list_available_python_versions`
* add tracing to get of authed client
* internal: Trigger claude when commenting with /aider (#5783)
* add claude instructions files
* call claude too when using aider
* fix
* add draft for linear claude integration
* fix build
* update ee ref
* ignore versions <=3.9
* fix windows build
* correct versions filter
* fix windows build (this time for real)
* inject error to debug CI
* update CI
* undo debug of CI
* fix tests
* remove outdated comment
* update ee repo ref
* Update ee-repo-ref.txt
* Update backend/parsers/windmill-parser-py-imports/src/lib.rs
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
* Update InstanceSetting.svelte
---------
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: centdix <40307056+centdix@users.noreply.github.com>
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
