AI Chat
Proxima Console includes an AI-powered chat assistant that helps infrastructure engineers investigate issues, query metrics, analyze alerts, compare hosts, and execute runbooks — all through natural language.
AI Chat is reached through the full-page /chat view. It is gated behind the per-client ai_chat feature flag (off by default) plus the chat:read permission. The floating chat panel is not yet enabled.
How It Works
User sends message → Backend classifies complexity → Routes to the model
configured for that tier
(via LiteLLM)
→ LLM responds with text or tool calls
→ Backend executes tools directly against stores (RBAC-scoped)
→ Streams response back to user via SSE
The chat assistant has access to the same data you see in the Console UI — hosts, metrics, alerts, compliance scores, changes, runbooks — but queries it through natural language instead of clicking through pages.
Key Features
- Real-time streaming: Responses stream token-by-token as the LLM generates them
- Tool execution: The AI calls infrastructure tools (17 executed directly today; more are defined but not yet wired for in-chat execution) to fetch real data
- RBAC-scoped: All tool calls respect your client permissions — you only see data you're authorized to access
- Context-aware: The chat view receives page context so it knows what you're looking at
- Multi-model routing: Simple questions use cheap models, complex investigations use powerful ones
- BYOK (Bring Your Own Key) (planned): per-client LLM keys are not yet available — chat currently uses the platform key
- Encrypted conversations: Messages encrypted with per-client keys (AES-256-GCM + Vault Transit)
- Write tool safety: Destructive operations require explicit confirmation
Architecture
The AI chat does not use the MCP server for tool execution. Instead, the backend executes tools directly against its own stores with the user's RBAC context. This eliminates network round-trips and ensures proper data scoping.
The MCP server remains available for external clients (Claude Desktop, Cursor, Claude Code) that connect via the MCP protocol.
┌─────────────────────────────────────────────────┐
│ Frontend (full-page /chat view) │
│ POST /api/v1/chat (Streamable HTTP / SSE) │
└───────────────────────┬─────────────────────────┘
│
┌───────────────────────▼─────────────────────────┐
│ Backend ChatHandler │
│ ├── Classify model tier (via LLM) │
│ ├── Pre-fetch page context │
│ ├── Call LiteLLM Gateway → LLM Provider │
│ ├── Execute tool calls directly against stores │
│ │ (HostStore, AlertStore, RunbookStore, etc.) │
│ ├── RBAC scoping on every tool call │
│ ├── Encrypt + store messages (Vault Transit) │
│ └── Stream tokens + progress events via SSE │
└─────────────────────────────────────────────────┘
Available Tools
The AI assistant can call these tools during a conversation:
| Tool | Type | Description |
|---|---|---|
query_hosts | Read | Search and list infrastructure hosts |
get_host_details | Read | Detailed info about a specific host |
compare_hosts | Read | Side-by-side comparison of two hosts |
get_active_alerts | Read | List firing/acknowledged alerts |
get_alert_context | Read | Full alert context with correlation data |
acknowledge_alert | Write | Acknowledge an alert group |
list_runbooks | Read | List available operational runbooks |
preview_runbook | Read | Preview runbook execution plan |
run_runbook | Write | Execute a runbook on a host |
get_runbook_status | Read | Check execution progress and results |
get_compliance_score | Read | Framework compliance scores |
get_compliance_failures | Read | Failing compliance controls |
get_compliance_evaluations | Read | Detailed per-control evaluations |
get_top_failures | Read | Top failing controls across hosts |
list_frameworks | Read | Available compliance frameworks |
get_findings | Read | Security scanner findings (CVEs) |
get_changes | Read | Recent infrastructure change events |
Write tools require additional permissions (alerts:write, runbooks:execute) and destructive runbook execution requires confirmation.
Model Tiers
The backend automatically routes messages to the appropriate model tier based on complexity:
| Tier | Model | When Used |
|---|---|---|
| Standard | a cheap/standard model | Simple lookups, basic questions, formatting |
| Complex | a mid-range model | Investigation, comparison, troubleshooting |
| Expert | a frontier/reasoning model | Root cause analysis, post-mortems, fleet-wide audits |
The concrete model behind each tier is set in infra/litellm/config.yaml (the current
deployment maps all three tiers to DeepSeek models) and can be overridden per deployment via
PROXIMA_CHAT_MODEL_{STANDARD,COMPLEX,EXPERT}. Swapping models needs no code change.
Classification uses a lightweight LLM call (~$0.00005 per classification) with keyword fallback when the LLM is unavailable. Users can override with a "Deep Analysis" toggle.
Permissions
| Permission | Grants |
|---|---|
chat:read | View conversation history |
chat:write | Send messages, create conversations |
Tool-level permissions are checked separately — having chat:write does not bypass alerts:write or runbooks:execute requirements.