Monitoring Overview

Proxima Console collects system and integration metrics from every managed host, stores them in VictoriaMetrics, and visualizes them through interactive dashboards at the host, environment, and client level.

How It Works

1. Collection — The agent runs collectors every 60 seconds (configurable). System metrics use gopsutil, plugin metrics query external services (e.g., PostgreSQL, Nginx).
2. Transport — All metrics are batched into a single NATS message and published to proxima.{client}.{env}.{host}.metrics on the TELEMETRY JetStream stream.
3. Processing — The backend MetricsWorker parses the payload, resolves the tenant (client's vm_account_id), attaches system labels (host_id, environment_id, labels_key), and imports to VictoriaMetrics via HTTP.
4. Storage — VictoriaMetrics Cluster stores metrics with 120-day retention and native multi-tenancy (each client is physically isolated by account ID).
5. Query — The frontend fetches metrics through the REST API. The backend translates requests into MetricsQL queries against VictoriaMetrics.

Agent Self-Monitoring

The agent monitors its own resource usage and exports a set of self-monitoring metrics alongside system metrics.

These cover:

• Process metrics — agent CPU usage, memory (RSS/VMS), file descriptors, context switches, disk I/O
• Go runtime — heap/stack allocation, GC pause time, goroutine count
• NATS transport — messages published, bytes sent, RTT, publish errors, outbox depth

All self-monitoring metrics carry an agent_version label, enabling cross-version regression detection. A pre-built Grafana dashboard is available at infra/grafana/dashboards/agent-resources.json.

Resource limits can be configured via environment variables:

Variable	Default	Description
PROXIMA_AGENT_MEMORY_LIMIT_MB	— (disabled)	Go soft memory limit via debug.SetMemoryLimit
PROXIMA_AGENT_COLLECTOR_TIMEOUT	30s	Per-collector context deadline; timed-out collectors degrade gracefully

See Metrics Collected for the full metric list.

Metric Types

Type	Description	Example
Gauge	Point-in-time value	cpu_usage_ratio, memory_used_bytes
Counter	Monotonically increasing total	disk_read_bytes_total, network_transmit_packets_total

Counter metrics are queried using MetricsQL rate() to compute per-second rates at query time — raw counters are never stored as rates.

Multi-Tenancy

Each client in Proxima Console has a unique vm_account_id (auto-incrementing integer). This ID is used in VictoriaMetrics URL paths:

• Write: POST /insert/{accountID}/prometheus/api/v1/import
• Read: GET /select/{accountID}/prometheus/api/v1/query_range

This provides physical data isolation between clients — no query can accidentally access another client's metrics.

Aggregation Levels

Metrics can be queried at three levels:

Level	Scope	Aggregation	Use Case
Host	Single host	None (raw series)	Debugging a specific server
Environment	All hosts in an environment	AVG across hosts	Environment health overview
Client	All hosts across all environments	AVG across hosts	Client-level executive dashboard

Adaptive Step

The backend automatically selects the query step (bucket size) based on the requested time range:

Time Range	Step	Data Points (approx.)
Up to 6 hours	1 minute	360
Up to 24 hours	5 minutes	288
Up to 7 days	15 minutes	672
Over 7 days	1 hour	varies

This balances chart resolution with query performance and data transfer.