AI Chat
Proxima Console includes an AI-powered chat assistant that helps infrastructure engineers investigate issues, query metrics, analyze alerts, compare hosts, and execute runbooks — all through natural language.
AI Chat is reached through the full-page /chat view. It is gated behind the per-client ai_chat feature flag (off by default) plus the chat:read permission. The floating chat panel is not yet enabled.
How It Works
User sends message → Backend classifies complexity → Routes to the model
configured for that tier
(via LiteLLM)
→ LLM responds with text or tool calls
→ Backend executes tools directly against stores (RBAC-scoped)
→ Streams response back to user via SSE
The chat assistant has access to the same data you see in the Console UI — hosts, metrics, alerts, compliance scores, changes, runbooks — but queries it through natural language instead of clicking through pages.
Key Features
- Real-time streaming: Responses stream token-by-token as the LLM generates them
- Tool execution: The AI calls infrastructure tools to fetch real data. The full tool catalog is passed to the model unfiltered, and every tool in it executes — there is no subset that is defined-but-unwired.
- RBAC-scoped: All tool calls respect your client permissions — you only see data you're authorized to access
- Context-aware: The chat view receives page context so it knows what you're looking at
- Multi-model routing: Simple questions use cheap models, complex investigations use powerful ones
- BYOK (Bring Your Own Key) (planned): per-client LLM keys are not yet available — chat currently uses the platform key
- Encrypted conversations: Messages encrypted with per-client keys (AES-256-GCM + Vault Transit)
- Write tool safety: Destructive operations require explicit confirmation
Architecture
The AI chat does not use the MCP server for tool execution. Instead, the backend executes tools directly against its own stores with the user's RBAC context. This eliminates network round-trips and ensures proper data scoping.
The MCP server remains available for external clients (Claude Desktop, Cursor, Claude Code) that connect via the MCP protocol.
┌─────────────────────────────────────────────────┐
│ Frontend (full-page /chat view) │
│ POST /api/v1/chat (Streamable HTTP / SSE) │
└───────────────────────┬─────────────────────────┘
│
┌───────────────────────▼─────────────────────────┐
│ Backend ChatHandler │
│ ├── Classify model tier (via LLM) │
│ ├── Pre-fetch page context │
│ ├── Call LiteLLM Gateway → LLM Provider │
│ ├── Execute tool calls directly against stores │
│ │ (HostStore, AlertStore, RunbookStore, etc.) │
│ ├── RBAC scoping on every tool call │
│ ├── Encrypt + store messages (Vault Transit) │
│ └── Stream tokens + progress events via SSE │
└─────────────────────────────────────────────────┘
Available Tools
The AI assistant can call these tools during a conversation:
| Tool | Type | Description |
|---|---|---|
query_hosts | Read | Search and list infrastructure hosts |
get_host_details | Read | Detailed info about a specific host |
compare_hosts | Read | Side-by-side comparison of two hosts |
get_active_alerts | Read | List firing/acknowledged alerts |
get_alert_context | Read | Full alert context with correlation data |
acknowledge_alert | Write | Acknowledge an alert group |
list_runbooks | Read | List available operational runbooks |
preview_runbook | Read | Preview runbook execution plan |
run_runbook | Write | Execute a runbook on a host |
get_runbook_status | Read | Check execution progress and results |
get_compliance_score | Read | Framework compliance scores |
get_compliance_failures | Read | Failing compliance controls |
get_compliance_evaluations | Read | Detailed per-control evaluations |
get_top_failures | Read | Top failing controls across hosts |
list_frameworks | Read | Available compliance frameworks |
get_findings | Read | Security scanner findings (CVEs) |
get_changes | Read | Recent infrastructure change events |
Write tools require additional permissions (alerts:write, runbooks:execute) and destructive runbook execution requires confirmation.
Model Tiers
The backend automatically routes messages to the appropriate model tier based on complexity:
| Tier | Model | When Used |
|---|---|---|
| Standard | a cheap/standard model | Simple lookups, basic questions, formatting |
| Complex | a mid-range model | Investigation, comparison, troubleshooting |
| Expert | a frontier/reasoning model | Root cause analysis, post-mortems, fleet-wide audits |
The concrete model behind each tier comes from the backend's own defaults, which it requests against the central LiteLLM gateway:
| Tier | Default model |
|---|---|
| Standard | deepseek-v4-pro |
| Complex | gemini-2.5-pro |
| Expert | claude-sonnet-4-6 |
| Translate (NL → PQL) | claude-haiku-4-5-20251001 |
Override any of them with PROXIMA_CHAT_MODEL_{STANDARD,COMPLEX,EXPERT,TRANSLATE}. Swapping models
needs no code change.
infra/litellm/config.yaml is the local docker-compose gateway; the proxima/* aliases it
defines are never requested by the backend unless you point a PROXIMA_CHAT_MODEL_* variable at
one.
Classification uses a lightweight LLM call (~$0.00005 per classification) with keyword fallback when the LLM is unavailable. Users can override with a "Deep Analysis" toggle.
Permissions
| Permission | Grants |
|---|---|
chat:read | View conversation history |
chat:write | Send messages, create conversations |
Tool-level permissions are checked separately — having chat:write does not bypass alerts:write or runbooks:execute requirements.