Compare commits

..

1 Commits

Author SHA1 Message Date
centdix
78bc6b498c feat: add workspace script search tools to script mode
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-10 12:31:38 +00:00
4 changed files with 147 additions and 332 deletions

157
PLAN.md
View File

@@ -1,157 +0,0 @@
# Plan 4: Flow Mode — Add Flow Search Tool with Auto-Search Behavior
## Problem
Flow mode can search for **scripts** (via `search_scripts`) but cannot search for **flows**. When building a new flow, users can't discover existing flows they could reference as sub-flows. The AI should proactively search for existing flows after any user request, just like it already searches for scripts.
## Goal
Add a `search_flows` tool to flow mode and instruct the AI to proactively search for existing flows and scripts after any request, enabling discovery and reuse of flow building blocks.
## Current Architecture
- **File**: `frontend/src/lib/components/copilot/chat/flow/core.ts`
- **Existing tools**: `search_scripts`, `search_hub_scripts`, `resource_type`, `get_instructions_for_code_generation`, `get_db_schema`, `set_flow_json`, `inspect_inline_script`, `set_module_code`, `test_run_flow`, `test_run_step`, `get_lint_errors`
- **`search_scripts`** uses `WorkspaceScriptsSearch` (uFuzzy over `ScriptService.listScripts()`)
- **No flow search** — the AI cannot find existing flows to reuse as sub-flow modules
## Implementation
### 1. Add `search_flows` tool
```typescript
const searchFlowsSchema = z.object({
query: z
.string()
.describe('The query to search for, e.g. "process invoices", "send notification", etc.')
})
const searchFlowsToolDef = createToolDef(
searchFlowsSchema,
'search_flows',
'Search for flows in the workspace. Returns array of {path, summary} objects. Use this to find existing flows that can be referenced as sub-flow modules.'
)
```
### 2. Add `WorkspaceFlowsSearch` class (or reuse shared class)
Either reuse the shared `WorkspaceRunnablesSearch` from Plan 1, or add a dedicated flow search class:
```typescript
class WorkspaceFlowsSearch {
private uf: uFuzzy
private workspace: string | undefined = undefined
private flows: Flow[] | undefined = undefined
constructor() { this.uf = new uFuzzy() }
private async init(workspace: string) {
this.flows = await FlowService.listFlows({ workspace })
this.workspace = workspace
}
async search(query: string, workspace: string) {
if (this.flows === undefined || this.workspace !== workspace) {
await this.init(workspace)
}
const flows = this.flows
if (!flows) throw new Error('Failed to load flows')
const results = this.uf.search(
flows.map((f) => (emptyString(f.summary) ? f.path : f.summary + ' (' + f.path + ')')),
query.trim()
)
return results[2]?.map((id) => ({
path: flows[id].path,
summary: flows[id].summary
})) ?? []
}
}
```
### 3. Add `get_flow_details` tool (to inspect found flows)
```typescript
const getFlowDetailsSchema = z.object({
path: z.string().describe('The path of the flow to inspect')
})
const getFlowDetailsToolDef = createToolDef(
getFlowDetailsSchema,
'get_flow_details',
'Get details of a workspace flow including its modules structure, input schema, and module summaries. Use after search_flows to understand a flow before referencing it.'
)
```
Implementation calls `FlowService.getFlowByPath()` and returns:
- Flow summary, description
- Input schema
- Module structure (ids, summaries, types, paths for script references)
### 4. Update system prompt — Proactive Search Instruction
This is the key difference from script mode. The system prompt should instruct the AI to **always search for existing flows** when creating or modifying a flow:
Update the "Creating Flows" section in `prepareFlowSystemMessage()`:
```
### Creating Flows
1. **Search for existing flows and scripts first** (unless user explicitly asks to write from scratch):
- First: `search_flows` to find existing flows that could be reused as sub-flow modules
- Then: `search_scripts` to find workspace scripts
- Then: `search_hub_scripts` (only consider highly relevant results)
- Only create raw scripts if no suitable script or flow is found
2. **When referencing an existing flow as a sub-module:**
- Use `get_flow_details` to understand the flow's input schema
- Use `type: "flow"` with `path` in the module value
- Define `input_transforms` matching the referenced flow's input schema
```
### 5. Register tools
Add to the `flowTools` array:
```typescript
export const flowTools: Tool<FlowAIChatHelpers>[] = [
// ... existing tools
{
def: searchFlowsToolDef,
fn: async ({ args, workspace, toolId, toolCallbacks }) => {
toolCallbacks.setToolStatus(toolId, {
content: 'Searching for workspace flows related to "' + args.query + '"...'
})
const parsedArgs = searchFlowsSchema.parse(args)
const flowResults = await workspaceFlowsSearch.search(parsedArgs.query, workspace)
toolCallbacks.setToolStatus(toolId, {
content: 'Found ' + flowResults.length + ' flows in the workspace related to "' + args.query + '"'
})
return JSON.stringify(flowResults)
}
},
{
def: getFlowDetailsToolDef,
fn: async ({ args, workspace, toolId, toolCallbacks }) => {
// ... implementation
}
}
]
```
## Files to Modify
| File | Change |
|------|--------|
| `frontend/src/lib/components/copilot/chat/flow/core.ts` | Add `search_flows` and `get_flow_details` tools, update system prompt, add search class, register tools |
| `frontend/src/lib/components/copilot/chat/shared.ts` | Ensure shared flow search utility if extracted |
## Testing
1. Open a flow in flow mode
2. Ask "Create a flow that sends a notification after processing an invoice"
3. AI should proactively search for existing flows related to "notification" and "invoice"
4. AI should search for existing scripts too
5. If matches are found, AI should propose reusing them
6. Ask "I have a flow for Stripe webhook handling, can I add it as a sub-flow here?"
7. AI should find the flow, inspect its inputs, and add it as a `type: "flow"` module

View File

@@ -1,11 +1,8 @@
import {
ScriptService,
FlowService,
type Flow,
type FlowModule,
type InputTransform,
type RawScript,
type Script,
JobService
} from '$lib/gen'
import type {
@@ -13,8 +10,6 @@ import type {
ChatCompletionUserMessageParam
} from 'openai/resources/chat/completions.mjs'
import { z } from 'zod'
import uFuzzy from '@leeoniya/ufuzzy'
import { emptyString } from '$lib/utils'
import {
createDbSchemaTool,
getFormattedResourceTypes,
@@ -33,7 +28,8 @@ import {
findModuleById,
SPECIAL_MODULE_IDS,
formatScriptLintResult,
type ScriptLintResult
type ScriptLintResult,
WorkspaceScriptsSearch
} from '../shared'
import type { ContextElement } from '../context'
import type { ExtendedOpenFlow } from '$lib/components/flows/types'
@@ -336,109 +332,6 @@ const setFlowJsonToolDef = createToolDef(
{ strict: false }
)
class WorkspaceScriptsSearch {
private uf: uFuzzy
private workspace: string | undefined = undefined
private scripts: Script[] | undefined = undefined
constructor() {
this.uf = new uFuzzy()
}
private async init(workspace: string) {
this.scripts = await ScriptService.listScripts({
workspace
})
this.workspace = workspace
}
async search(query: string, workspace: string) {
if (this.scripts === undefined || this.workspace !== workspace) {
await this.init(workspace)
}
const scripts = this.scripts
if (!scripts) {
throw new Error('Failed to load scripts')
}
const results = this.uf.search(
scripts.map((s) => (emptyString(s.summary) ? s.path : s.summary + ' (' + s.path + ')')),
query.trim()
)
const scriptResults =
results[2]?.map((id) => ({
path: scripts[id].path,
summary: scripts[id].summary
})) ?? []
return scriptResults
}
}
class WorkspaceFlowsSearch {
private uf: uFuzzy
private workspace: string | undefined = undefined
private flows: Flow[] | undefined = undefined
constructor() {
this.uf = new uFuzzy()
}
private async init(workspace: string) {
this.flows = await FlowService.listFlows({ workspace })
this.workspace = workspace
}
async search(query: string, workspace: string) {
if (this.flows === undefined || this.workspace !== workspace) {
await this.init(workspace)
}
const flows = this.flows
if (!flows) {
throw new Error('Failed to load flows')
}
const results = this.uf.search(
flows.map((f) => (emptyString(f.summary) ? f.path : f.summary + ' (' + f.path + ')')),
query.trim()
)
return (
results[2]?.map((id) => ({
path: flows[id].path,
summary: flows[id].summary
})) ?? []
)
}
}
const searchFlowsSchema = z.object({
query: z
.string()
.describe('The query to search for, e.g. "process invoices", "send notification", etc.')
})
const searchFlowsToolDef = createToolDef(
searchFlowsSchema,
'search_flows',
'Search for flows in the workspace. Returns array of {path, summary} objects. Use this to find existing flows that can be referenced as sub-flow modules.'
)
const getFlowDetailsSchema = z.object({
path: z.string().describe('The path of the flow to inspect')
})
const getFlowDetailsToolDef = createToolDef(
getFlowDetailsSchema,
'get_flow_details',
'Get details of a workspace flow including its modules structure, input schema, and module summaries. Use after search_flows to understand a flow before referencing it.'
)
const workspaceFlowsSearch = new WorkspaceFlowsSearch()
// Will be overridden by setSchema
const testRunFlowSchema = z.object({
args: z
@@ -525,54 +418,6 @@ export const flowTools: Tool<FlowAIChatHelpers>[] = [
return JSON.stringify(scriptResults)
}
},
{
def: searchFlowsToolDef,
fn: async ({ args, workspace, toolId, toolCallbacks }) => {
toolCallbacks.setToolStatus(toolId, {
content: 'Searching for workspace flows related to "' + args.query + '"...'
})
const parsedArgs = searchFlowsSchema.parse(args)
const flowResults = await workspaceFlowsSearch.search(parsedArgs.query, workspace)
toolCallbacks.setToolStatus(toolId, {
content:
'Found ' + flowResults.length + ' flows in the workspace related to "' + args.query + '"'
})
return JSON.stringify(flowResults)
}
},
{
def: getFlowDetailsToolDef,
fn: async ({ args, workspace, toolId, toolCallbacks }) => {
const parsedArgs = getFlowDetailsSchema.parse(args)
toolCallbacks.setToolStatus(toolId, {
content: 'Getting details for flow "' + parsedArgs.path + '"...'
})
const flow = await FlowService.getFlowByPath({
workspace,
path: parsedArgs.path
})
const modules = flow.value?.modules ?? []
const moduleSummaries = modules.map((m: FlowModule) => ({
id: m.id,
summary: m.summary,
type: m.value.type,
...(m.value.type === 'script' ? { path: m.value.path } : {}),
...(m.value.type === 'flow' ? { path: m.value.path } : {}),
...(m.value.type === 'rawscript' ? { language: m.value.language } : {})
}))
const result = {
path: flow.path,
summary: flow.summary,
description: flow.description,
schema: flow.schema,
modules: moduleSummaries
}
toolCallbacks.setToolStatus(toolId, {
content: 'Retrieved details for flow "' + parsedArgs.path + '"'
})
return JSON.stringify(result)
}
},
{
def: resourceTypeToolDef,
fn: async ({ args, toolId, workspace, toolCallbacks }) => {
@@ -960,8 +805,6 @@ export function prepareFlowSystemMessage(customPrompt?: string): ChatCompletionS
- **View existing inline script code** → \`inspect_inline_script\`
- **Change module code only** → \`set_module_code\`
- **Get language-specific coding instructions** → \`get_instructions_for_code_generation\` (call BEFORE writing code)
- **Find workspace flows** → \`search_flows\` (find existing flows to reuse as sub-flow modules)
- **Get flow details** → \`get_flow_details\` (inspect a flow's input schema and modules before referencing it)
- **Find workspace scripts** → \`search_scripts\`
- **Find Windmill Hub scripts** → \`search_hub_scripts\`
@@ -1168,25 +1011,18 @@ Example: Before writing TypeScript/Bun code, call \`get_instructions_for_code_ge
### Creating Flows
1. **Search for existing flows and scripts first** (unless user explicitly asks to write from scratch):
- First: \`search_flows\` to find existing flows that could be reused as sub-flow modules
- Then: \`search_scripts\` to find workspace scripts
1. **Search for existing scripts first** (unless user explicitly asks to write from scratch):
- First: \`search_scripts\` to find workspace scripts
- Then: \`search_hub_scripts\` (only consider highly relevant results)
- Only create raw scripts if no suitable flow or script is found
- Only create raw scripts if no suitable script is found
2. **When referencing an existing flow as a sub-module:**
- Use \`get_flow_details\` to understand the flow's input schema
- Use \`type: "flow"\` with \`path\` in the module value
- Define \`input_transforms\` matching the referenced flow's input schema
3. **Build the complete flow using \`set_flow_json\`:**
- If using existing flow: use \`type: "flow"\` with \`path\`
2. **Build the complete flow using \`set_flow_json\`:**
- If using existing script: use \`type: "script"\` with \`path\`
- If creating rawscript: use \`type: "rawscript"\` with \`language\` and \`content\`
- **First call \`get_instructions_for_code_generation\` to get the correct code format**
- Always define \`input_transforms\` to connect parameters to flow inputs or previous step results
4. **After making code changes, ALWAYS use \`get_lint_errors\` to check for issues.** Fix any errors before proceeding with testing.
3. **After making code changes, ALWAYS use \`get_lint_errors\` to check for issues.** Fix any errors before proceeding with testing.
### AI Agent Modules

View File

@@ -1,4 +1,4 @@
import { ResourceService, JobService } from '$lib/gen/services.gen'
import { ResourceService, JobService, ScriptService } from '$lib/gen/services.gen'
import type { AIProvider, AIProviderModel, ResourceType, ScriptLang } from '$lib/gen/types.gen'
import { capitalize, isObject, toCamel } from '$lib/utils'
import { get } from 'svelte/store'
@@ -17,7 +17,8 @@ import {
buildTestRunArgs,
buildContextString,
type ScriptLintResult,
formatScriptLintResult
formatScriptLintResult,
WorkspaceScriptsSearch
} from '../shared'
import { setupTypeAcquisition, type DepsToGet } from '$lib/ata'
import { getModelContextWindow } from '../../lib'
@@ -177,7 +178,8 @@ function buildChatSystemPrompt(currentModel: AIProviderModel) {
- The user can ask you questions about a list of \`DATABASES\` that are available in the user's workspace. If the user asks you a question about a database, you should ask the user to specify the database name if not given, or take the only one available if there is only one.
- You can also receive a \`DIFF\` of the changes that have been made to the code. You should use this diff to give better answers.
- Before giving your answer, check again that you carefully followed these instructions.
- When asked to create a script that communicates with an external service, you can use the \`search_hub_scripts\` tool to search for relevant scripts in the hub. Make sure the language is the same as what the user is coding in. If you do not find any relevant scripts, you can use the \`search_npm_packages\` tool to search for relevant packages and their documentation. Always give a link to the documentation in your answer if possible.
- If the user mentions a specific script or wants to find existing scripts in the workspace, use the \`search_workspace_scripts\` tool. You can then use \`get_script_details\` to read the script's code and understand its inputs.
- When asked to create a script that communicates with an external service, first check if a workspace script already exists with \`search_workspace_scripts\`, then use \`search_hub_scripts\` to search for relevant scripts in the hub. Make sure the language is the same as what the user is coding in. If you do not find any relevant scripts, you can use the \`search_npm_packages\` tool to search for relevant packages and their documentation. Always give a link to the documentation in your answer if possible.
- After applying code changes with the \`${editToolName}\` tool, ALWAYS use the \`get_lint_errors\` tool to check for lint errors. If there are errors, fix them before proceeding. Then use the \`test_run_script\` tool to test the code, and iterate on the code until it works as expected (MAX 3 times). If the user cancels the test run, do not try again and wait for the next user instruction.
Important:
@@ -311,6 +313,8 @@ export function prepareScriptTools(
context: ContextElement[]
): Tool<ScriptChatHelpers>[] {
const tools: Tool<ScriptChatHelpers>[] = []
tools.push(searchWorkspaceScriptsTool)
tools.push(getScriptDetailsTool)
if (['python3', 'php', 'bun', 'deno', 'nativets', 'bunnative'].includes(language)) {
tools.push(resourceTypeTool)
}
@@ -570,6 +574,89 @@ export const searchNpmPackagesTool: Tool<ScriptChatHelpers> = {
}
}
const SEARCH_WORKSPACE_SCRIPTS_TOOL: ChatCompletionFunctionTool = {
type: 'function',
function: {
name: 'search_workspace_scripts',
description:
'Search for scripts in the workspace by query. Use this when the user mentions a specific script, wants to find existing scripts, or wants to reuse code from another script.',
parameters: {
type: 'object',
properties: {
query: {
type: 'string',
description: 'The search query (e.g. script name, functionality like "send email")'
}
},
required: ['query']
}
}
}
const GET_SCRIPT_DETAILS_TOOL: ChatCompletionFunctionTool = {
type: 'function',
function: {
name: 'get_script_details',
description:
'Get the full details of a workspace script including its code, inputs schema, and description. Use after search_workspace_scripts to inspect a specific script.',
parameters: {
type: 'object',
properties: {
path: {
type: 'string',
description: 'The path of the script (e.g. "f/marketing/send_email")'
}
},
required: ['path']
}
}
}
const workspaceScriptsSearch = new WorkspaceScriptsSearch()
export const searchWorkspaceScriptsTool: Tool<ScriptChatHelpers> = {
def: SEARCH_WORKSPACE_SCRIPTS_TOOL,
fn: async ({ args, workspace, toolId, toolCallbacks }) => {
toolCallbacks.setToolStatus(toolId, {
content: 'Searching for workspace scripts related to "' + args.query + '"...'
})
const scriptResults = await workspaceScriptsSearch.search(args.query, workspace)
toolCallbacks.setToolStatus(toolId, {
content:
'Found ' +
scriptResults.length +
' scripts in the workspace related to "' +
args.query +
'"'
})
return JSON.stringify(scriptResults)
}
}
export const getScriptDetailsTool: Tool<ScriptChatHelpers> = {
def: GET_SCRIPT_DETAILS_TOOL,
fn: async ({ args, workspace, toolId, toolCallbacks }) => {
toolCallbacks.setToolStatus(toolId, {
content: 'Getting details for script "' + args.path + '"...'
})
const script = await ScriptService.getScriptByPath({
workspace,
path: args.path
})
toolCallbacks.setToolStatus(toolId, {
content: 'Retrieved details for "' + args.path + '"'
})
return JSON.stringify({
path: script.path,
summary: script.summary,
description: script.description,
language: script.language,
schema: script.schema,
content: script.content
})
}
}
export async function fetchNpmPackageTypes(
packageName: string,
version: string = 'latest'

View File

@@ -21,7 +21,15 @@ import { workspaceStore } from '$lib/stores'
import type { ExtendedOpenFlow } from '$lib/components/flows/types'
import type { FunctionParameters } from 'openai/resources/shared.mjs'
import { z } from 'zod'
import { ScriptService, JobService, type CompletedJob, type FlowModule } from '$lib/gen'
import {
ScriptService,
JobService,
type CompletedJob,
type FlowModule,
type Script
} from '$lib/gen'
import uFuzzy from '@leeoniya/ufuzzy'
import { emptyString } from '$lib/utils'
import { scriptLangToEditorLang } from '$lib/scripts'
import { getCurrentModel } from '$lib/aiStore'
import { type editor as meditor } from 'monaco-editor'
@@ -526,6 +534,47 @@ export function createToolDef(
}
}
export class WorkspaceScriptsSearch {
private uf: uFuzzy
private workspace: string | undefined = undefined
private scripts: Script[] | undefined = undefined
constructor() {
this.uf = new uFuzzy()
}
private async init(workspace: string) {
this.scripts = await ScriptService.listScripts({
workspace
})
this.workspace = workspace
}
async search(query: string, workspace: string) {
if (this.scripts === undefined || this.workspace !== workspace) {
await this.init(workspace)
}
const scripts = this.scripts
if (!scripts) {
throw new Error('Failed to load scripts')
}
const results = this.uf.search(
scripts.map((s) => (emptyString(s.summary) ? s.path : s.summary + ' (' + s.path + ')')),
query.trim()
)
const scriptResults =
results[2]?.map((id) => ({
path: scripts[id].path,
summary: scripts[id].summary
})) ?? []
return scriptResults
}
}
const searchHubScriptsSchema = z.object({
query: z
.string()