Healthcheck & Compliance

Proxima Console includes a healthcheck & compliance system that runs curated security and operational checks on managed hosts, producing compliance scores for hosts, environments, and clients.

Overview

The system consists of:

• Agent-side scanners that execute checks on each host
• Backend result processing that normalizes results, maps to curated rules, and calculates scores
• REST API for managing rules, viewing results, and triggering scans

Architecture

Agent Scanners

Lynis Scanner

Runs the Lynis security auditing tool and parses its report.dat output. Requires Lynis to be pre-installed on the host.

• Runs lynis audit system --no-colors --quick
• Parses a generated report file (written to a temp path such as /tmp/lynis-proxima-<ns>.dat via --report-file) for tests_executed=, suggestion[]=, and warning[]= lines
• Maps 18 Lynis test IDs to curated Proxima rule slugs
• Status codes: 1 = pass, 0 = fail, -1 = skip
• Auto-detected via exec.LookPath("lynis")

OpenSCAP Scanner

Runs OpenSCAP formal compliance profile evaluations using SCAP Security Guide (SSG) datastream files.

• Binary: oscap, Content: SSG datastream files (e.g., /usr/share/xml/scap/ssg/content/ssg-*.ds.xml)
• Default profile: xccdf_org.ssgproject.content_profile_cis_level1_server
• Default timeout: 15 minutes
• Produces both summary Results (pass/fail per rule) and detailed Findings with rule metadata
• Auto-detected via exec.LookPath("oscap")
• Installed via system package manager (libopenscap8/openscap-scanner + scap-security-guide)

Trivy Scanner

Runs Trivy OS package CVE vulnerability scanning on the host filesystem.

• Binary: trivy, Scan type: rootfs / with --scanners vuln
• Default timeout: 10 minutes
• Summary rules: trivy-no-critical-cves, trivy-no-high-cves
• Each CVE is stored as a detailed Finding with metadata (CVE ID, package name, installed version, fixed version, severity, CVSS score)
• Auto-detected via exec.LookPath("trivy")
• Installed as a GitHub release binary download to /usr/local/bin/trivy

Custom Check Runner

Executes bash commands defined in the healthcheck rules. Each rule with a check_command field is run as a shell command.

• Exit code 0 = pass, non-zero = fail
• Per-check timeout (default 30 seconds, configurable per rule)
• Combined stdout + stderr captured (truncated to 4KB)
• Context cancellation support for graceful shutdown

Runner Orchestrator

The Runner coordinates all registered scanners:

• Thread-safe (sync.RWMutex) for hot-reload via config callback
• Runs all available scanners, collects results
• Generates a UUID run_id per execution
• Records timing (started_at, completed_at, duration_ms)
• Scanner type: "lynis", "openscap", "trivy", "custom", or "mixed" when multiple run
• Publishes results to NATS (proxima.{client}.{env}.{host_id}.healthcheck)

Scanner Availability Reporting

The agent reports which healthcheck scanners are available in every heartbeat payload. The backend stores this in the hosts.scanner_status JSONB column and exposes it in host API responses.

{
  "scanner_status": [
    {"name": "lynis", "available": true},
    {"name": "openscap", "available": true},
    {"name": "trivy", "available": true},
    {"name": "custom", "available": true}
  ]
}

This allows the frontend to inform operators which scanners are installed on each host. Scanner availability is detected via exec.LookPath for each binary (lynis, oscap, trivy).

Curated Rule Set

42 rules across 5 categories are seeded in the database:

Category	Rules	Examples
Security	14	SSH root login disabled, firewall enabled, unattended updates, TLS cert validity, auditd running, OpenSCAP CIS compliance, Trivy CVE scanning
Networking	8	NTP synchronized, DNS resolution, no unnecessary open ports, IP forwarding disabled
System	8	Kernel updated, /tmp mounted noexec, log rotation, disk usage, filesystem permissions
Docker	7	No root containers, resource limits, healthchecks defined, no privileged mode
Backup & DR	5	Backup service running, WAL archiving, retention policy, recent database backups

Rule Properties

Each rule has:

• slug -- unique identifier (e.g., ssh-root-login-disabled)
• severity -- critical, high, medium, or low
• source -- lynis, custom, openscap, or trivy
• check_command -- bash command for the custom scanner (optional)
• lynis_test_id -- mapped Lynis test ID (optional)
• enabled -- can be toggled on/off via API
• remediation -- description of how to fix failures

Scoring

Host Score

score = (passed / (passed + failed)) * 100

Skipped and error results are excluded from the score calculation. A host with no results has a null score.

Environment Score

Average of the latest host scores across all hosts in the environment.

Client Score

Average of the latest host scores across all hosts in all environments belonging to the client.

API Endpoints

Categories & Rules

Method	Path	Description
GET	/api/v1/healthcheck/categories	List all categories
GET	/api/v1/healthcheck/rules	List rules (filterable by category, severity, source, enabled)
GET	/api/v1/healthcheck/rules/{ruleID}	Get a single rule
PUT	/api/v1/healthcheck/rules/{ruleID}	Update rule severity and/or enabled status

Runs & Results

Method	Path	Description
GET	/api/v1/hosts/{hostID}/healthcheck/runs	List paginated runs for a host
GET	/api/v1/hosts/{hostID}/healthcheck/runs/latest	Get latest run with results
GET	/api/v1/hosts/{hostID}/healthcheck/runs/{runID}	Get specific run with results

Findings

Method	Path	Description
GET	/api/v1/hosts/{id}/findings	Paginated scanner findings (filter by scanner, severity, type)
GET	/api/v1/hosts/{id}/healthcheck/runs/{runID}/findings	Findings for a specific healthcheck run

Findings are detailed records produced by OpenSCAP and Trivy scanners. Each finding contains scanner-specific metadata (e.g., CVE ID, CVSS score, package name for Trivy; rule ID, profile, result status for OpenSCAP). Findings are stored in the scanner_findings table and linked to healthcheck runs.

Scores

Method	Path	Description
GET	/api/v1/hosts/{hostID}/healthcheck/score	Host compliance score
GET	/api/v1/environments/{envID}/healthcheck/score	Environment aggregate score
GET	/api/v1/clients/{id}/healthcheck/score	Client aggregate score

On-Demand Trigger

Method	Path	Description
POST	/api/v1/hosts/{hostID}/healthcheck/trigger	Trigger a healthcheck scan on a host

Configuration

Healthcheck scanning is configured via the Config Sync system using config type healthcheck.

Example environment-level config:

{
  "config_data": {
    "interval_seconds": 3600,
    "rules": [
      {"slug": "ssh-root-login-disabled", "check_command": "grep -q '^PermitRootLogin no' /etc/ssh/sshd_config"}
    ]
  }
}

Compliance Evaluation

Healthcheck results are the primary evidence source for the Compliance Evaluation system. After each healthcheck run completes, the backend publishes a proxima.compliance.evaluate trigger message on NATS. The compliance worker then evaluates all enabled compliance controls against the run's results, producing per-control verdicts and framework-level compliance scores.

See Compliance Evaluation for details on frameworks, controls, scoring, and the API.

Key Files

File	Description
agent/internal/healthcheck/runner.go	Scanner orchestrator
agent/internal/healthcheck/lynis.go	Lynis scanner implementation
agent/internal/healthcheck/oscap.go	OpenSCAP scanner implementation
agent/internal/healthcheck/trivy.go	Trivy scanner implementation
agent/internal/healthcheck/custom.go	Custom check runner
agent/internal/healthcheck/result.go	Types: Result, Payload, Finding, Scanner interface
backend/internal/worker/healthcheck.go	NATS consumer that processes results and findings
backend/internal/store/healthcheck_rule_store.go	Rule CRUD store
backend/internal/store/healthcheck_run_store.go	Run/result/score store
backend/internal/store/scanner_finding_store.go	FindingsProvider interface + findings store
backend/internal/api/healthcheck_handler.go	REST API handler
backend/internal/api/finding_handler.go	Findings API handler
backend/migrations/000023_healthcheck.up.sql	Schema + seed data