Files
windmill/integration_tests/ai_agent_tests/providers.py
claude[bot] 02bd9277fa feat(ai): add websearch tool for AI agents (#7399)
* refactor(ai): use responses API for OpenAI/Azure, create 'other' provider for completion endpoint

- Created new 'other.rs' provider using the OpenAI-compatible completion endpoint
- Refactored 'openai.rs' to use the Responses API for both text and image output
- Updated query_builder to route OpenAI/AzureOpenAI to OpenAIQueryBuilder
- All other providers (Mistral, DeepSeek, Groq, etc.) now use OtherQueryBuilder
- Updated OpenRouter to delegate to OtherQueryBuilder instead of OpenAIQueryBuilder

This prepares the codebase for adding websearch tool support using the Responses API.

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* feat(ai): add websearch as AI agent tool type

- Added WebsearchToolValue to ToolValue enum in flows.rs
- Updated all pattern matches to handle websearch tool type
- Added has_websearch parameter to run_agent function
- Websearch tools don't require additional configuration

This prepares the backend for provider-specific websearch implementations.

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* feat(frontend): add websearch as tool option in AI agent UI

- Added WebsearchTool type and utility functions to agentToolUtils.ts
- Added "Web Search" option in tool insertion menu (InsertModuleInner)
- Updated NewAIToolNode to handle pickWebsearchTool event
- Updated AIToolNode to render websearch tools properly
- Updated FlowModuleSchemaMap to create websearch tools

Users can now add websearch tools to AI agents through the UI.

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* feat(ai): implement websearch for OpenAI provider

- Added has_websearch parameter to BuildRequestArgs
- OpenAI provider now adds web_search tool to requests when enabled
- Uses OpenAI Responses API web_search tool type
- Websearch tool is added before other custom tools in the request

Implements websearch functionality for OpenAI and Azure OpenAI providers.

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* feat(ai): implement websearch for Anthropic provider

- Created anthropic.rs provider with native Anthropic API format
- Added web_search tool to Anthropic requests when enabled
- Anthropic uses /messages endpoint with x-api-key authentication
- Updated query_builder to route Anthropic to new provider

Implements websearch functionality for Anthropic Claude models.

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* feat(ai): implement websearch for Gemini/GoogleAI provider

- GoogleAI now uses completion endpoint (other.rs) for text instead of responses API
- Added Google Search grounding when websearch is enabled
- Uses google_search_retrieval tool in request when has_websearch is true
- Updated parse methods to use OtherQueryBuilder for completion endpoint

Implements websearch functionality for Google Gemini models.

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* fix frontend

* fix anthropic and openai

* better for gemini

* structured output

* cleaning

* fix validate tool

* fixes

* cleaning

* cleaning

* fix for openai

* no responses api for azure

* fixes

* fix

* add tests for ai agent

* avoid panic

* better tests

* test user images

* fix tool choice

* always use streaming backend side

* big cleaning

* show annotations plus agent action for open ai websearch use

* show annotations plus agent action for anthropic websearch use

* show annotations plus agent action for google websearch use

* nit forntend

* rm

* fix

* add test for image ouptut

* fix for azure

* add in openflow

* fix

* fix

* nit tests

* fixes

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: centdix <farhadg110@gmail.com>
2025-12-22 11:41:28 +00:00

110 lines
2.7 KiB
Python

"""
AI Provider configurations for testing.
Each provider requires a resource to be set up in the integration-tests workspace.
Resources are created by the setup_providers fixture from environment variables.
Resources created:
- u/admin/openai (openai type)
- u/admin/anthropic (anthropic type)
- u/admin/googleai (googleai type)
- u/admin/openrouter (openrouter type)
- u/admin/bedrock (aws_bedrock type)
"""
from typing import Any
def make_provider_input_transform(kind: str, model: str, resource_path: str) -> dict[str, Any]:
"""
Create an input transform for the provider field in an AI agent module.
The provider input_transform uses a static value with the ProviderWithResource
object containing kind, model, and resource reference.
"""
return {
"type": "static",
"value": {
"kind": kind,
"model": model,
"resource": f"$res:{resource_path}",
},
}
# Provider configurations
OPENAI = {
"name": "openai",
"input_transform": make_provider_input_transform(
kind="openai",
model="gpt-4o-mini",
resource_path="u/admin/openai",
),
}
AZURE_OPENAI = {
"name": "azure_openai",
"input_transform": make_provider_input_transform(
kind="azure_openai",
model="gpt-4o",
resource_path="u/admin/azure_openai",
),
}
ANTHROPIC = {
"name": "anthropic",
"input_transform": make_provider_input_transform(
kind="anthropic",
model="claude-haiku-4-5-20251001",
resource_path="u/admin/anthropic",
),
}
GOOGLE_AI = {
"name": "google_ai",
"input_transform": make_provider_input_transform(
kind="googleai",
model="gemini-2.5-flash",
resource_path="u/admin/googleai",
),
}
BEDROCK = {
"name": "bedrock",
"input_transform": make_provider_input_transform(
kind="aws_bedrock",
model="global.anthropic.claude-haiku-4-5-20251001-v1:0",
resource_path="u/admin/bedrock",
),
}
OPENROUTER = {
"name": "openrouter",
"input_transform": make_provider_input_transform(
kind="openrouter",
model="anthropic/claude-haiku-4.5",
resource_path="u/admin/openrouter",
),
}
# All providers for parametrized tests
ALL_PROVIDERS = [
OPENAI,
AZURE_OPENAI,
ANTHROPIC,
GOOGLE_AI,
BEDROCK,
OPENROUTER,
]
# Vision-capable providers for user_images tests
VISION_PROVIDERS = [
OPENAI, # gpt-4o-mini supports vision
ANTHROPIC, # claude-3 supports vision
GOOGLE_AI, # gemini supports vision
]
def get_provider_ids(providers: list[dict[str, Any]]) -> list[str]:
"""Get provider names for pytest parametrize IDs."""
return [p["name"] for p in providers]