Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 20 additions & 16 deletions fern/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@ tabs:
display-name: Home
icon: home
deployment:
display-name: Deployment & Administration
display-name: Administration & Governance
slug: admin
icon: server
development:
display-name: Development and Integration
Expand Down Expand Up @@ -58,36 +59,39 @@ navigation:
path: ./docs/pages/deployment/overview.mdx
- section: Installation
contents:
- section: On-Premises
contents:
- page: Kubernetes
path: ./docs/pages/deployment/kubernetes.mdx
- page: Zero Dependency Binary
path: ./docs/pages/deployment/binary.mdx
- section: Cloud
contents:
- page: AWS
path: ./docs/pages/deployment/aws.mdx
- page: Azure
path: ./docs/pages/deployment/azure.mdx
- page: GCP
path: ./docs/pages/deployment/gcp.mdx
- page: Azure
path: ./docs/pages/deployment/azure.mdx
- page: AWS
path: ./docs/pages/deployment/aws.mdx
- section: Air Gapped
contents:
- page: Air Gapped
path: ./docs/pages/deployment/air-gapped.mdx
- section: On-Premises
contents:
- page: Kubernetes
path: ./docs/pages/deployment/kubernetes.mdx
- page: Zero Dependency Binary
path: ./docs/pages/deployment/binary.mdx
- section: Administration
contents:
- page: Admin Panel
- page: Admin Console
path: ./docs/pages/deployment/admin-panel.mdx
- page: Clusters
- page: Create an AI System
path: ./docs/pages/deployment/custom_system.mdx
- page: AI System Management
path: ./docs/pages/deployment/cluster-management.mdx
- page: Models

- page: Model Management
path: ./docs/pages/deployment/model-management.mdx
- page: API Keys
path: ./docs/pages/deployment/api-keys.mdx
- page: Security & Monitoring
path: ./docs/pages/deployment/security-monitoring.mdx
- page: Monitoring
path: ./docs/pages/deployment/monitoring.mdx
- tab: development
layout:
- section: Getting Started
Expand Down
4 changes: 2 additions & 2 deletions fern/docs/pages/agent-forge/building-agents.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -198,7 +198,7 @@ When explaining code vulnerabilities, provide:
- ✅ When using MCP tool calls

<Callout intent="info">
**MCP requires "Non-streaming (with MCP capabilities)" mode**: If you plan to connect your agent to external tools via [MCP Integration](/agent-forge/using-agent-forge/mcp), make sure you choose the right mode. MCP tool calls execute in real-time and stream results back as they arrive.
**MCP requires "Non-streaming (with MCP capabilities)" mode**: If you plan to connect your agent to external tools via [MCP Integration](/agent-forge/using-agent-forge/mcp-integration), make sure you choose the right mode. MCP tool calls execute in real-time and stream results back as they arrive.
</Callout>

## Generation Settings (Advanced Settings)
Expand Down Expand Up @@ -448,6 +448,6 @@ After creation, thoroughly test your agent:

Now that you can build agents:

1. **[Configure MCP](/agent-forge/using-agent-forge/mcp)** - Connect your agent to external tools and live data sources
1. **[Configure MCP](/agent-forge/using-agent-forge/mcp-integration)** - Connect your agent to external tools and live data sources
2. **[Create Knowledge Bases](/agent-forge/using-agent-forge/knowledge-base)** - Add organizational knowledge to your agents
3. **[Share with Team](/agent-forge/using-agent-forge/chatting-with-agents)** - Enable organization sharing and collaborate
2 changes: 1 addition & 1 deletion fern/docs/pages/agent-forge/mcp.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Agent Forge supports MCP at two levels:

| Level | Configured By | Available To |
|-------|--------------|--------------|
| **Organization-level** | Admin via Unified CX (coming soon!) | All agents in the organization |
| **Organization-level** | Admin via Admin Console (coming soon!) | All agents in the organization |
| **Agent-level** | Agent Builder and above | That specific agent only |

This layered approach lets administrators centrally manage shared infrastructure while allowing individual agent builders to extend with specialized tools.
Expand Down
2 changes: 1 addition & 1 deletion fern/docs/pages/agent-forge/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -125,5 +125,5 @@ Ready to get started? Continue with:
- [Getting Started](/agent-forge/getting-started/getting-started) - Account setup and first login
- [Chatting with Agents](/agent-forge/using-agent-forge/chatting-with-agents) - Learn the chat interface
- [Building Agents](/agent-forge/using-agent-forge/building-agents) - Create your first agent
- [MCP Integration](/agent-forge/using-agent-forge/mcp) - Connect agents to external tools and data sources
- [MCP Integration](/agent-forge/using-agent-forge/mcp-integration) - Connect agents to external tools and data sources
- [Knowledge Base](/agent-forge/using-agent-forge/knowledge-base) - Add RAG-based knowledge base to agents
187 changes: 99 additions & 88 deletions fern/docs/pages/deployment/admin-panel.mdx
Original file line number Diff line number Diff line change
@@ -1,120 +1,131 @@
# Admin Panel
---
title: Admin Console
subtitle: Central control plane for managing your sovereign AI systems
description: Overview of the Admin Console — manage systems, models, API keys, governance policies, and monitor activity across your infrastructure
---

The Prediction Guard Admin Console is your central control plane for managing your sovereign AI systems. From here, you can create and manage multiple systems, deploy any open model, manage API keys, configure MCP servers, apply governance policies, and monitor all activity across your infrastructure.

## Accessing the Admin Console

Once deployed, access the Admin Console at your deployment's URL (e.g. [admin.predictionguard.com](https://admin.predictionguard.com)) and log in with your admin credentials.

## Navigation

The Admin Console sidebar is organized into three groups:

**Systems**
- **Manage** — Create, view, and manage all your AI systems

**Security**
- **Analyze** — Analyze AI interactions for security and compliance signals
- **Monitor** — Real-time monitoring of system usage and performance
- **Govern** — Configure and apply AI governance policies
- **Audit** — Review audit logs and compliance reports

**Settings**
- **Users** — Manage user accounts and access
- **Organizations** — Manage organizational settings and structure

## Systems: Manage

The Systems page is your starting point — a unified view of all AI systems in your Prediction Guard deployment.

![Systems Page](./ux-screenshots/01-systems.png)

Each system card shows:
- **Status**: Health state (Healthy, Never Connected, Degraded)
- **API Keys**: Number of active API keys
- **Models**: Number of deployed models
- **MCP Servers**: Number of connected MCP servers
- **Location**: Deployment environment (e.g. `kubernetes`, `staging`)
- **Last Update**: Time of last heartbeat from the system

Click **Manage** on any system card to open its management dashboard, where you can configure API keys, models, MCP servers, and advanced settings. Click **Create System** to add a new system.

## Security: Analyze

The Analyze section gives you visibility into the safety and composition of all AI models across your systems. It has two tabs: **Scans** and **BOMs**.

### Scans

![Analyze - Scans](./ux-screenshots/analyze-page-scans.png)

The Prediction Guard admin panel is your central command center for managing your entire Prediction Guard platform. From here, you can create and manage multiple clusters, deploy any open model, configure security settings, and monitor all activity across your infrastructure.
The Scans tab shows safety and security scores for every AI model in your deployment. At a glance you can see:

## Accessing the Admin Panel
- **Models Scanned**: Total number of models that have been analyzed
- **Avg. General Safety Score**: Average safety score across all scanned models (0–100)
- **Avg. Prompt Injection Refusal Rate**: How reliably models resist prompt injection attempts on average

Once your Prediction Guard instance is deployed, you can access the admin panel at:
The model table breaks this down per model, showing **Provider**, **Type**, **General Safety Score**, **Prompt Injection Refusal Rate**, and **Last Scan** date. Use this to compare models, identify weaker performers, and make informed decisions about which models to deploy in sensitive environments.

```
https://admin.predictionguard.com
```
### BOMs (Bill of Materials)

The admin panel is centrally managed by Prediction Guard, providing you with access to manage your clusters and deployments.
![Analyze - BOMs](./ux-screenshots/analyze-page-bom.png)

### Initial Setup
The BOMs tab provides a Bill of Materials for each AI system — a full inventory of everything running in that system:

1. **Login** with your admin credentials
2. **Configure basic settings** (organization name, timezone, etc.)
3. **Set up your first API key** for testing
4. **Deploy your first model** from the model library
- **Private Models**: Models you have deployed from your own repositories
- **Managed Models**: Models managed and maintained by Prediction Guard
- **External Models**: Third-party models connected to your system
- **MCP Servers**: Connected Model Context Protocol servers

## Key Features
See [Model Management](/admin/administration/model-management) for a full guide to deploying all three model types.

### Dashboard
![Admin Panel Dashboard](./admin-panel-dashboard.png)
Each system has an **Export BOM** button to download a full inventory report — useful for compliance audits, vendor assessments, and internal governance reviews.

The dashboard provides a comprehensive overview of your Prediction Guard platform:
- **Multi-cluster overview** with health status across all clusters
- **Real-time usage statistics** and performance metrics
- **Security alerts** and system notifications
- **Quick actions** for common administrative tasks
- **Resource utilization** across your infrastructure
## Security: Monitor

### Model Management
- **Browse available models** from Hugging Face
- **Deploy custom models** from your own repositories
- **Configure model settings** (temperature, max tokens, etc.)
- **Monitor model performance** and usage
The Monitor section provides real-time observability into your AI systems — tracking request volumes, latency, model performance, and resource utilization. Use this to detect anomalies, track usage trends, and ensure your systems are operating within expected parameters.

### API Key Management
- **Create and manage** API keys
- **Set permissions** and rate limits
- **Track usage** per API key
- **Revoke access** when needed
## Security: Govern

### System Configuration
- **Security settings** and policies
- **Resource allocation** and limits
- **Backup and recovery** options
- **Update management** and versioning
The Govern section is where you configure and apply AI governance policies system-wide. Policies set here are enforced across all agents and models within your systems without requiring per-agent configuration.

## Platform Workflow
### Governance Baselines

### After Deployment
Once you've deployed your Prediction Guard cluster, the admin panel becomes your central management hub:
![Govern - Standards](./ux-screenshots/govern-page-standards.png)

1. **Cluster Management**: Monitor and manage your deployed clusters
2. **Model Deployment**: Deploy models from Hugging Face or upload custom models
3. **API Key Management**: Create and manage API keys for application access
4. **Security Configuration**: Set up security policies and monitoring
5. **Monitoring**: Track usage, performance, and security across your platform
Prediction Guard ships with four pre-built governance baselines you can apply with a single click:

### Integration with Deployment Process
The admin panel integrates seamlessly with your deployment:
| Baseline | Description |
|----------|-------------|
| **NIST AI RMF** | The NIST AI Risk Management Framework. Sets recommended thresholds for PII protection, prompt injection detection, toxicity filtering, and factuality checks aligned with NIST's trustworthy AI principles. |
| **NIST 600-1** | The Generative AI Profile of the AI RMF, focused on risks specific to large language models. Tunes factuality, toxicity, and PII policies to stricter thresholds recommended for generative AI deployments. |
| **OWASP** | Based on the OWASP Top 10 for LLM Applications. Directly addresses prompt injection, sensitive data exposure, and toxic or harmful outputs. |
| **OMB M-26-04** | The Office of Management and Budget Memorandum M-26-04, which sets federal requirements for responsible AI use. Enforces PII protections, prompt injection defenses, and factuality/toxicity policies at federally recommended thresholds. |

- **Cluster Status**: View health and status of all deployed clusters
- **Resource Monitoring**: Track CPU, GPU, and memory usage across clusters
- **Model Management**: Deploy and configure models on your clusters
- **API Access**: Create keys for applications to access your deployed models
- **Security Management**: Configure and monitor security across your platform
Click **Apply Configuration** on any baseline to apply it as your system-wide governance policy.

## Quick Start Guide
### Custom Governance Configuration

### 1. Deploy Your First Model
![Govern - Custom Configuration](./ux-screenshots/govern-page-custom-config.png)

1. Navigate to **Models** → **Browse Library**
2. Search for a model (e.g., "llama-2-7b-chat")
3. Click **Deploy** and configure settings
4. Wait for deployment to complete
5. Test the model via API
Below the baselines, the **Governance Configuration** section lets you fine-tune individual policies. Each policy can be independently enabled or disabled, and configured with specific actions:

### 2. Create an API Key
| Policy | Purpose | Available Actions |
|--------|---------|-------------------|
| **PII Policy** | Prevent unauthorized disclosure, storage, or processing of PII within your AI systems | Block, Log Events |
| **Prompt Injection Policy** | Prevent jailbreaking or manipulation of AI instructions to bypass safety filters or access restricted data | Block, Log Events |
| **Toxicity Policy** | Ensure AI outputs remain professional, inclusive, and free from harmful or discriminatory content | Block, Log Events |
| **Factuality Policy** | Mitigate hallucinations and ensure AI-generated information is verifiable | Block |

1. Go to **API Keys** → **Create New**
2. Set a name and description
3. Configure permissions and rate limits
4. Copy the generated key securely
5. Test with a simple API call
<Callout intent="info">
Applying a governance baseline will pre-configure these toggles to the recommended settings for that standard. You can then adjust individual policies from the custom configuration below.
</Callout>

### 3. Configure Security
## Security: Audit

1. Review **Security** → **Policies**
2. Enable input/output filtering as needed
3. Set up PII detection rules
4. Configure injection prevention
5. Test security features
The Audit section provides a tamper-evident log of all significant actions and interactions across your Admin Console — including system changes, model deployments, API key activity, and user actions. Use this for compliance reporting, incident investigation, and access reviews.

## Best Practices
## Settings: Users

### Security
- **Use strong passwords** for admin accounts
- **Enable two-factor authentication** if available
- **Regularly rotate API keys**
- **Monitor access logs** for suspicious activity
Manage user accounts that have access to the Admin Console. From here you can invite new administrators, update roles, and revoke access.

### Model Management
- **Start with smaller models** for testing
- **Monitor resource usage** during deployment
- **Keep models updated** for security patches
- **Document model configurations** for team members
## Settings: Organizations

### Monitoring
- **Set up alerts** for system health
- **Monitor API usage** and performance
- **Track model inference** metrics
- **Review logs** regularly for issues
Configure organizational settings including your organization's name, structure, and any organization-wide defaults that apply across all systems.

---

**Complete documentation coming soon** - Detailed guides for each admin panel feature are being developed.
**Need help?** Contact our support team or join our Discord community for assistance.
Loading
Loading