Add web search and fetch tools

docs/content/docs/configuration/features/index.mdx

@@ -9,15 +9,16 @@ The `[features]` section enables and configures optional gateway capabilities. A
 
 ## Feature Overview
 
-| Feature                                                           | Section                       | Purpose                                           |
-| ----------------------------------------------------------------- | ----------------------------- | ------------------------------------------------- |
-| [File Search](/docs/configuration/features/file-search)           | `[features.file_search]`      | RAG file_search tool for Responses API            |
-| [File Processing](/docs/configuration/features/file-processing)   | `[features.file_processing]`  | Document chunking, OCR, virus scanning            |
-| [Response Caching](/docs/configuration/features/response-caching) | `[features.response_caching]` | Exact and semantic response caching               |
-| [Guardrails](/docs/configuration/features/guardrails)             | `[features.guardrails]`       | Content filtering, PII detection, safety          |
-| [Image Fetching](/docs/configuration/features/image-fetching)     | `[features.image_fetching]`   | URL-to-base64 conversion for non-OpenAI providers |
-| [WebSocket](/docs/configuration/features/websocket)               | `[features.websocket]`        | Real-time event subscriptions                     |
-| Model Catalog                                                     | `[features.model_catalog]`    | Enrich models with capabilities and pricing       |
+| Feature                                                           | Section                                          | Purpose                                           |
+| ----------------------------------------------------------------- | ------------------------------------------------ | ------------------------------------------------- |
+| [File Search](/docs/configuration/features/file-search)           | `[features.file_search]`                         | RAG file_search tool for Responses API            |
+| [File Processing](/docs/configuration/features/file-processing)   | `[features.file_processing]`                     | Document chunking, OCR, virus scanning            |
+| [Response Caching](/docs/configuration/features/response-caching) | `[features.response_caching]`                    | Exact and semantic response caching               |
+| [Guardrails](/docs/configuration/features/guardrails)             | `[features.guardrails]`                          | Content filtering, PII detection, safety          |
+| [Image Fetching](/docs/configuration/features/image-fetching)     | `[features.image_fetching]`                      | URL-to-base64 conversion for non-OpenAI providers |
+| [WebSocket](/docs/configuration/features/websocket)               | `[features.websocket]`                           | Real-time event subscriptions                     |
+| [Web Tools](/docs/configuration/features/web-tools)               | `[features.web_search]` / `[features.web_fetch]` | Web search and URL fetching for chat UI           |
+| Model Catalog                                                     | `[features.model_catalog]`                       | Enrich models with capabilities and pricing       |
 
 ## Minimal Configuration
 
@@ -108,6 +109,17 @@ require_auth = true
 enabled = true
 sync_interval_secs = 1800
 api_url = "https://models.dev/api.json"
+
+# Web Search
+[features.web_search]
+provider = "tavily"
+api_key = "${TAVILY_API_KEY}"
+max_results = 10
+
+# Web Fetch
+[features.web_fetch]
+max_response_bytes = 1048576
+timeout_secs = 30
 ```
 
 ## Model Catalog
@@ -169,6 +181,7 @@ Some features have dependencies on other configuration:
 | Semantic Caching     | Vector backend (pgvector or Qdrant)                       |
 | Guardrails (Bedrock) | AWS credentials                                           |
 | WebSocket auth       | Authentication configuration                              |
+| Web Search           | Search provider API key (Tavily or Exa)                   |
 
 <Callout type="info">
   For conceptual documentation on how each feature works, see the [Features Guide](/docs/features).

docs/content/docs/configuration/features/meta.json

@@ -7,6 +7,7 @@
     "response-caching",
     "guardrails",
     "image-fetching",
+    "web-tools",
     "websocket"
   ]
 }

docs/content/docs/configuration/features/web-tools.mdx

@@ -0,0 +1,163 @@
+---
+title: Web Tools
+description: Configure server-side web search and URL fetching tools
+---
+
+import { Callout } from "fumadocs-ui/components/callout";
+
+The `[features.web_search]` and `[features.web_fetch]` sections configure server-side web tools used by the chat UI. Web search requires an external provider API key; web fetch works out of the box.
+
+## Web Search
+
+### Configuration Reference
+
+```toml
+[features.web_search]
+provider = "tavily"
+api_key = "${TAVILY_API_KEY}"
+max_results = 10
+timeout_secs = 30
+cost_microcents_per_request = 10000
+```
+
+| Key                           | Type    | Default | Description                                              |
+| ----------------------------- | ------- | ------- | -------------------------------------------------------- |
+| `provider`                    | string  | —       | Search provider: `"tavily"` or `"exa"` (required)        |
+| `api_key`                     | string  | —       | Provider API key, supports `${ENV_VAR}` (required)       |
+| `max_results`                 | integer | `10`    | Maximum results per search (also caps per-request limit) |
+| `timeout_secs`                | integer | `30`    | Request timeout in seconds                               |
+| `cost_microcents_per_request` | integer | `10000` | Cost per request in microcents ($0.01 default)           |
+
+<Callout type="info">
+  Omit the `[features.web_search]` section entirely to disable web search. There is no separate
+  `enabled` flag — the presence of the section enables the feature.
+</Callout>
+
+### Provider Setup
+
+#### Tavily
+
+1. Sign up at [tavily.com](https://tavily.com) and obtain an API key
+2. Set the environment variable or inline the key:
+
+```toml
+[features.web_search]
+provider = "tavily"
+api_key = "${TAVILY_API_KEY}"
+```
+
+#### Exa
+
+1. Sign up at [exa.ai](https://exa.ai) and obtain an API key
+2. Configure with the `exa` provider:
+
+```toml
+[features.web_search]
+provider = "exa"
+api_key = "${EXA_API_KEY}"
+```
+
+Exa returns full-text content for each result rather than snippets.
+
+### Cost Tracking
+
+Each search request is logged with the configured `cost_microcents_per_request`. The default of `10000` represents $0.01 per search. Adjust to match your provider's actual pricing.
+
+## Web Fetch
+
+### Configuration Reference
+
+```toml
+[features.web_fetch]
+enabled = true
+max_response_bytes = 1048576
+timeout_secs = 30
+allowed_content_types = ["text/html", "text/plain", "application/json", "application/xml", "text/xml", "text/csv", "text/markdown"]
+cost_microcents_per_request = 0
+```
+
+| Key                           | Type     | Default          | Description                                      |
+| ----------------------------- | -------- | ---------------- | ------------------------------------------------ |
+| `enabled`                     | boolean  | `true`           | Enable/disable the web fetch tool                |
+| `max_response_bytes`          | integer  | `1048576` (1 MB) | Maximum response body size in bytes              |
+| `timeout_secs`                | integer  | `30`             | Request timeout in seconds                       |
+| `allowed_content_types`       | string[] | See above        | Content types to accept (prefix match)           |
+| `cost_microcents_per_request` | integer  | `0`              | Cost per request in microcents (free by default) |
+
+<Callout type="info">
+  Omit the `[features.web_fetch]` section entirely to disable web fetch. When present, set `enabled
+  = false` to temporarily disable without removing the configuration.
+</Callout>
+
+### Security
+
+Web fetch includes multiple layers of SSRF protection:
+
+| Protection        | Description                                                    |
+| ----------------- | -------------------------------------------------------------- |
+| URL validation    | Blocks private/loopback IPs by default                         |
+| DNS pinning       | Resolves DNS once and pins the connection to prevent rebinding |
+| Redirect blocking | Rejects HTTP redirects to prevent SSRF via redirect chains     |
+| Content filtering | Only fetches allowed content types                             |
+| Size limits       | Truncates responses at `max_response_bytes`                    |
+
+To allow fetching from private/loopback addresses (development only):
+
+```toml
+[server]
+allow_loopback_urls = true
+allow_private_urls = true
+```
+
+### Content Type Filtering
+
+The `allowed_content_types` list uses prefix matching. For example, `"text/html"` matches `"text/html; charset=utf-8"`. An empty list allows all content types (not recommended).
+
+## Complete Examples
+
+### Development
+
+```toml
+[features.web_search]
+provider = "tavily"
+api_key = "${TAVILY_API_KEY}"
+max_results = 5
+timeout_secs = 10
+
+[features.web_fetch]
+max_response_bytes = 524288
+timeout_secs = 15
+```
+
+### Production
+
+```toml
+[features.web_search]
+provider = "tavily"
+api_key = "${TAVILY_API_KEY}"
+max_results = 10
+timeout_secs = 30
+cost_microcents_per_request = 10000
+
+[features.web_fetch]
+max_response_bytes = 1048576
+timeout_secs = 30
+cost_microcents_per_request = 0
+allowed_content_types = ["text/html", "text/plain", "application/json"]
+```
+
+### Web Fetch Only
+
+```toml
+# No [features.web_search] — web search disabled
+
+[features.web_fetch]
+max_response_bytes = 2097152
+timeout_secs = 60
+```
+
+## Next Steps
+
+- [Web Tools Feature Guide](/docs/features/web-tools) — How web tools work in the chat UI
+- [Frontend Tools](/docs/features/frontend-tools) — Client-side tools (Python, JS, SQL, charts)
+- [Authorization](/docs/features/authorization) — RBAC policies for tool access

docs/content/docs/features/frontend-tools.mdx

@@ -448,16 +448,18 @@ Tools are enabled per-conversation via the toolbar in the chat interface.
 
 ### Tool Dependencies
 
-| Tool        | Requires                     |
-| ----------- | ---------------------------- |
-| Python      | None                         |
-| JavaScript  | None                         |
-| SQL         | Uploaded data files          |
-| Charts      | None (data embedded in spec) |
-| HTML        | None                         |
-| Wikipedia   | Internet connection          |
-| Wikidata    | Internet connection          |
-| File Search | Attached vector store        |
+| Tool        | Requires                                                                        |
+| ----------- | ------------------------------------------------------------------------------- |
+| Python      | None                                                                            |
+| JavaScript  | None                                                                            |
+| SQL         | Uploaded data files                                                             |
+| Charts      | None (data embedded in spec)                                                    |
+| HTML        | None                                                                            |
+| Wikipedia   | Internet connection                                                             |
+| Wikidata    | Internet connection                                                             |
+| File Search | Attached vector store                                                           |
+| Web Search  | [Backend configuration](/docs/configuration/features/web-tools) (Tavily or Exa) |
+| Web Fetch   | [Backend configuration](/docs/configuration/features/web-tools)                 |
 
 ## Execution Flow
 

docs/content/docs/features/index.mdx

@@ -240,6 +240,22 @@ Tool results are displayed inline as interactive artifacts and sent back to the
   <Card title="Frontend Tools" href="/docs/features/frontend-tools" />
 </Cards>
 
+## Web Tools
+
+Server-side web search and URL fetching, proxied through the gateway with SSRF protection and usage tracking.
+
+| Tool       | Provider      | Capabilities                            |
+| ---------- | ------------- | --------------------------------------- |
+| Web Search | Tavily or Exa | Search the web, returns ranked results  |
+| Web Fetch  | Direct HTTP   | Fetch URLs, HTML stripped to plain text |
+
+Web search results appear as inline citations. Both tools require backend configuration.
+
+<Cards>
+  <Card title="Web Tools" href="/docs/features/web-tools" />
+  <Card title="Web Tools Configuration" href="/docs/configuration/features/web-tools" />
+</Cards>
+
 ## MCP Integration
 
 Connect to external tool servers using the Model Context Protocol (MCP).

docs/content/docs/features/meta.json

@@ -16,6 +16,7 @@
     "knowledge-bases",
     "chat-modes",
     "frontend-tools",
+    "web-tools",
     "guardrails",
     "mcp",
     "mcp-agents",