Testing

Proxima Console follows a test-informed development approach. Tests are written alongside or before the code they verify, depending on the component.

When to Write Tests

Write Tests First

These areas benefit from writing tests before implementation:

Business logic and domain rules
Input validation
Utility and helper functions
Data transformations

Write Tests Alongside

These areas are tested as they are built:

Database store methods
NATS integration and message handling
API handlers
Agent collectors

Always Needs Tests

Exported functions that can fail
API endpoints
NATS message handlers
Store methods (database queries)

Does Not Need Tests

main.go and CLI wiring in cmd/
Pure struct definitions with no methods
Generated code (e.g., Swagger docs)

Test Style

All Go tests use table-driven tests with testify/assert and testify/require:

func TestParseSlug(t *testing.T) {
    tests := []struct {
        name    string
        input   string
        want    string
        wantErr bool
    }{
        {"valid slug", "my-client", "my-client", false},
        {"empty string", "", "", true},
        {"with spaces", "my client", "", true},
    }

    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            got, err := ParseSlug(tt.input)
            if tt.wantErr {
                require.Error(t, err)
                return
            }
            require.NoError(t, err)
            assert.Equal(t, tt.want, got)
        })
    }
}

Test Types

Type	Tag/Tool	When It Runs
Unit	(none)	Always in CI
Integration	`//go:build integration`	Needs Docker (PostgreSQL, NATS)
API	(none)	Uses `httptest`, always in CI
Contract	(none)	Validates responses against OpenAPI spec, always in CI
E2E (Backend)	BATS (`scripts/e2e/tests/`)	~1,000 tests, parallel execution with real infrastructure
E2E (Browser)	Playwright (`frontend/e2e/`)	Chromium tests against running backend

Unit Tests

Standard Go tests with no external dependencies. Run on every push.

Integration Tests

Require running infrastructure (PostgreSQL, NATS). Guarded with a build tag:

//go:build integration

package store_test

Run with make test-integration after starting Docker services and setting up the test database (make test-db-setup).

API Tests

Use httptest to test HTTP handlers without starting a real server. Handler dependencies are mocked using interface mocks (function-field structs).

E2E Tests (Backend — BATS)

The backend E2E tests use BATS (Bash Automated Testing System) for structured, parallel test execution. The test suite lives in scripts/e2e/ and currently spans ~1,000 tests across 37 files.

Architecture:

Seed-then-test pattern: All data seeding (NATS publishes, VictoriaMetrics waits) runs once in setup_suite.bash before any test file. Test files are pure assertions — no NATS publishing, no VM waits.
Parallel execution: Up to 5 test files run simultaneously (--jobs 5), while tests within each file run sequentially (--no-parallelize-within-files). Execution time: ~70 seconds (vs ~270s with the previous monolithic script).
Shared libraries: scripts/e2e/lib/ contains common.bash (env loading, state helpers), helpers.bash (DB queries, HTTP helpers, wait utilities), and auth.bash (TOTP generation, admin login).
State sharing: File-based state via /tmp/e2e-bats-state/ — state_set(key, value) / state_get(key) functions enable cross-file data sharing.
BATS vendored: bats-core, bats-assert, and bats-support as git submodules in scripts/e2e/bats/.

Test files: The suite has grown to 37 files. The full list lives in scripts/e2e/tests/; the major areas are:

File	What it covers
`01-health.bats`	Health and readiness endpoints
`02-nats-core.bats`	Auth, NATS registration, inventory, heartbeat, integrations
`03-integration-api.bats`	Integration API endpoints
`04-metrics.bats`	Metrics query, derivative, aggregate, batch
`05-processes.bats`	Process query and drill-down
`06-containers.bats`	Container drill-down and latest metrics
`07-tags-search.bats`	Entity tags, PQL search, saved searches
`08-admin.bats`	Admin API: roles, users, teams, service accounts, audit
`09-auth-flows.bats`	Auth: invite, password, MFA, sessions, OIDC
`10-agent.bats`	Full agent integration
`11-inventory-expansion.bats`	Inventory expansion (security, cron, GPU, hardware)
`12-changes.bats`	Change detection (watched files, versions, events)
`13-nats-jwt.bats`, `14-nats-jwt-lifecycle.bats`	NATS JWT issuance and lifecycle
`15-logs.bats`	Log ingestion and querying
`16-agent-configs.bats`, `22-agent-config-hotreload.bats`	Agent remote config and hot reload
`17-healthcheck.bats`	Healthcheck runs and results
`18-credentials.bats`, `19-config-sync-credentials.bats`	Credentials and config-sync credentials
`20-compliance.bats`, `24-compliance-management.bats`, `23-attestations-exceptions.bats`	Compliance scoring, management, attestations/exceptions
`21-dashboard-api.bats`	Dashboard API
`25-alerts.bats`	Alerting
`26-passkeys.bats`	Passkeys / WebAuthn
`26-runbooks.bats`	Runbooks
`27-assets.bats`	Assets / CMDB
`28-pql-multi-entity.bats`	PQL multi-entity search
`29-k8s-clusters.bats`	Kubernetes clusters
`30-ai-chat.bats`	AI chat (MCP)
`31-terminal.bats`	Interactive terminal
`32-service-desk.bats`	Service desk
`33-databases.bats`	Database inventory
`34-host-lifecycle.bats`	Host lifecycle
`35-misc-endpoints.bats`	Miscellaneous endpoints
`36-fleet.bats`	Agent fleet self-update

Running on the remote E2E host:

make test-e2e                      # Build, upload, and run the full suite on the remote E2E host
make test-e2e-file FILE=04-metrics # Run a single test file on the remote host
make test-e2e-ci                   # Run locally with JUnit output for CI

make test-e2e builds the agent, backend, and frontend, uploads them, and runs the suite over SSH on the remote E2E host (requires E2E_HOST and an SSH key). To run the BATS suite locally against a local proxima_test, use make test-e2e-ci, which invokes the vendored bats binary directly.

Adding new tests: Data seeding goes in scripts/e2e/setup_suite.bash, assertions go in the appropriate .bats test file. Use state_set/state_get for cross-file state sharing.

E2E Tests (Browser — Playwright)

The frontend/e2e/ directory contains Playwright browser tests that exercise the full frontend application against a running backend. These tests use Chromium to simulate real user interactions across authentication, navigation, admin pages, and profile management.

Architecture:

~290 tests across 21 spec files in frontend/e2e/, including auth.spec.ts, admin.spec.ts, clients.spec.ts, hosts.spec.ts, navigation.spec.ts, profile.spec.ts, ai-chat.spec.ts, alerts.spec.ts, changes.spec.ts, compliance.spec.ts, credentials.spec.ts, runbooks.spec.ts, service-desk.spec.ts, passkeys.spec.ts, install-wizard.spec.ts, search.spec.ts, saved-searches.spec.ts, and more
Frontend built with --mode test (relative /api/v1 URLs) and served via vite preview on port 4173 — the preview server proxies API calls to the backend, avoiding CORS issues
Backend runs on port 18080 against the proxima_test database
Admin MFA state is reset before each run so global-setup can enrol a fresh TOTP secret
Global setup authenticates the admin user via API, sets up MFA (TOTP), and saves browser storage state so all tests start pre-authenticated
Global teardown cleans up test data created during tests (users, roles, teams, service accounts with pw-e2e prefix)

Running locally:

The easiest way is to use the automated script, which handles all setup, execution, and cleanup:

./scripts/playwright-test.sh

This script:

Loads .env.test for test configuration
Runs preflight checks (PostgreSQL, Valkey, NATS)
Sets up the test database and resets admin MFA state
Builds the backend and starts it on port 18080
Builds the frontend with --mode test (relative API URLs, proxied by vite preview)
Runs Playwright tests
Cleans up (stops test backend, restarts production service)

For interactive debugging, use cd frontend && npm run test:e2e:ui (requires the backend to already be running on port 18080).

CI integration: On main, the .test:e2e:remote job in .gitlab-ci.yml runs the full E2E suite (BATS + Playwright) on a dedicated remote host. The job uses a lightweight alpine:3.21 image that adds an SSH client, decodes the deploy key, and invokes scripts/e2e-ci-trigger.sh, which uploads the pre-built backend, agent, and frontend (--mode test) artifacts and runs the suite over SSH. It produces JUnit XML (reports/*.xml) and the Playwright HTML report (playwright-report/) as artifacts.

Mocking

Handler tests use interface mocks built with function-field structs:

type mockStore struct {
    GetHostFn func(ctx context.Context, id uuid.UUID) (*domain.Host, error)
}

func (m *mockStore) GetHost(ctx context.Context, id uuid.UUID) (*domain.Host, error) {
    return m.GetHostFn(ctx, id)
}

Worker tests use go-sqlmock for database interaction testing.

Agent and backend transport tests use an embedded NATS test server via nats-server/v2/test.

Edge Case Checklist

Every test suite should cover:

Happy path (valid input, expected output)
Empty, nil, or zero values
Not found (resource does not exist)
Duplicate (conflict on unique constraints)
Invalid input (malformed, wrong type, out of range)
Database or NATS errors (connection failures, timeouts)
Large input (boundary values, max lengths)
Boundary values (off-by-one, pagination edges)

Test Database

Integration and E2E tests use a separate proxima_test database so they never touch development data.

make docker-up         # Start the local dev stack (PostgreSQL, NATS, Valkey, VictoriaMetrics, VictoriaLogs, Tempo, Grafana, Vault, …)
make migrate-up        # Apply migrations to dev database
make test-db-setup     # Create proxima_test, run migrations, seed

make test-db-setup is idempotent — safe to re-run. Use make test-db-reset to drop and recreate from scratch.

Running Tests

make test              # Run all unit and API tests
make test-race         # Run tests with the Go race detector
make test-db-setup     # Create/migrate/seed proxima_test (first time)
make test-integration  # Run integration tests (against proxima_test)
make test-e2e          # Build, upload, and run the BATS + Playwright E2E suite on the remote E2E host
make test-e2e-file FILE=04-metrics  # Run a single BATS e2e test file on the remote host
make test-e2e-ci       # Run the BATS suite locally with JUnit output for CI (against proxima_test, port 18080)
./scripts/playwright-test.sh    # Automated Playwright browser tests (builds backend+frontend, runs tests, cleans up)
cd frontend && npm run test:e2e:ui  # Playwright in interactive UI mode (backend must be running on :18080)
make test-coverage     # Generate a coverage report
make test-db-reset     # Drop and recreate proxima_test

Current Coverage

Component	Coverage	Tests
Agent	~84%	all packages
Backend	~80%	with integration tests
Frontend (unit)	~95%	~350 test files
Backend (E2E)	—	~1,000 BATS tests (37 files)
Frontend (E2E)	—	~290 Playwright browser tests (21 spec files)

The minimum coverage threshold for merging to main is 70%.

When to Write Tests​

Write Tests First​

Write Tests Alongside​

Always Needs Tests​

Does Not Need Tests​

Test Style​

Test Types​

Unit Tests​

Integration Tests​

API Tests​

E2E Tests (Backend — BATS)​

E2E Tests (Browser — Playwright)​

Mocking​

Edge Case Checklist​

Test Database​

Running Tests​

Current Coverage​