> ## Documentation Index
> Fetch the complete documentation index at: https://cona.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Observability

> Three Axiom datasets, OpenTelemetry metrics, Sentry, PostHog — and which apps are actually covered

# Observability

<Warning>
  **Coverage is not uniform across the four apps.** Do not assume a signal exists just because it
  exists somewhere. The matrix below is the important part of this page.
</Warning>

## Coverage matrix

| App                      | Axiom                 | OpenTelemetry          | Sentry                 | PostHog |
| ------------------------ | --------------------- | ---------------------- | ---------------------- | ------- |
| `@cona/webapp`           | ✅ 3 datasets          | ✅                      | ✅ server, client, edge | ✅       |
| `@cona/portal`           | ✅                     | ✅                      | ❌                      | ❌       |
| `@cona/temporal-workers` | ✅ 2 datasets          | ✅ metrics → Axiom OTLP | ❌                      | ❌       |
| `@cona/console`          | ⚠️ `@axiomhq/js` only | ❌                      | ❌                      | ❌       |

Console has the widest blast radius — it can delete an organisation and put the whole
platform into maintenance — and the least instrumentation.

## Pipelines

```mermaid theme={null}
flowchart LR
    subgraph Sources["Emitters"]
        WebServer["webapp server<br/>lib/axiom/server.ts"]
        WebClient["webapp browser<br/>lib/axiom/client.ts"]
        Proxy["proxy.ts<br/>request + security events"]
        Workers["temporal-workers<br/>pino + @axiomhq/pino"]
        Portal["portal<br/>app/lib/logger.ts"]
    end

    Shared["@cona/observability<br/>logger · sampling · semantics<br/>serialization · runtime-context<br/>transports · boundaries · validation"]

    Ingest["api/axiom/ingest<br/>browser log forwarding"]

    subgraph Axiom["Axiom datasets"]
        DsMain["NEXT_PUBLIC_AXIOM_DATASET<br/>application logs"]
        DsSecurity["NEXT_PUBLIC_AXIOM_SECURITY_DATASET<br/>protectAll = true"]
        DsActivity["NEXT_PUBLIC_AXIOM_ACTIVITY_DATASET<br/>Temporal activity logs"]
        Metrics["OTLP metrics<br/>api.axiom.co/v1/metrics"]
    end

    Sentry["Sentry<br/>webapp only"]
    PostHog["PostHog<br/>webapp only"]

    WebServer --> Shared
    Workers --> Shared
    Portal --> Shared
    Shared --> DsMain
    Proxy --> DsSecurity
    Workers --> DsActivity
    Workers --> Metrics
    WebClient --> Ingest --> DsMain
    WebServer --> Sentry
    WebClient --> Sentry
    WebClient --> PostHog

    classDef boundary fill:#f8fafc,stroke:#94a3b8,color:#334155

    classDef app fill:#dbe2fb,stroke:#3B56C5,color:#111827
    classDef pkg fill:#e8eafd,stroke:#4967E6,color:#111827
    classDef observability fill:#f0e8fd,stroke:#7c3bc5,color:#111827

    class WebServer,WebClient,Proxy,Workers,Portal,Ingest app
    class Shared pkg
    class DsMain,DsSecurity,DsActivity,Metrics,Sentry,PostHog observability
    class Sources,Axiom boundary
```

## The three datasets

| Dataset                              | Written by                      | Contents                                                      |
| ------------------------------------ | ------------------------------- | ------------------------------------------------------------- |
| `NEXT_PUBLIC_AXIOM_DATASET`          | all apps                        | application logs                                              |
| `NEXT_PUBLIC_AXIOM_SECURITY_DATASET` | `proxy.ts` via `securityLogger` | auth failures, maintenance blocks, `/admin` and `/dev` access |
| `NEXT_PUBLIC_AXIOM_ACTIVITY_DATASET` | workers                         | Temporal activity logs                                        |

The security logger is created with `protectAll = true`
(`apps/webapp/lib/axiom/server.ts:110`) — a different redaction posture from the general
logger.

Worker dataset names are set in the Fly config: `cona_temporal-workers` and
`cona_temporal-activity`, with `_staging` suffixes in staging.

## Security events

`proxy.ts` emits to the security dataset at four points:

| Event              | Trigger                        |       Line |
| ------------------ | ------------------------------ | ---------: |
| `maintenance_mode` | request blocked by maintenance |   `:63-67` |
| `auth_required`    | no valid session               | `:223-228` |
| `admin_access`     | `/admin` request               | `:251-259` |
| `dev_access`       | `/dev` request                 | `:251-259` |

Logs are flushed through `flushAllLoggers(event)` in a `finally` block so a thrown handler
cannot lose them.

## `@cona/observability`

The shared core, with a colocated test for every module:

| Module               | Role                                                               |
| -------------------- | ------------------------------------------------------------------ |
| `logger.ts`          | the logging interface                                              |
| `sampling.ts`        | deterministic sampling                                             |
| `semantics.ts`       | field naming conventions                                           |
| `serialization.ts`   | safe value serialisation                                           |
| `runtime-context.ts` | request/workflow context propagation                               |
| `transports.ts`      | `createPinoTransport` (workers), `createWebappTransport` (Next.js) |
| `boundaries.ts`      | boundary logging                                                   |
| `validation.ts`      | shape validation                                                   |
| `timing.ts`          | duration helpers                                                   |

### Sampling

Deterministic, controlled by two environment variables (`sampling.ts:3-4`):

* `CONA_LOG_SAMPLING_ENABLED`
* `CONA_LOG_SAMPLING_RULES` — a map of rule name to rate

Being deterministic rather than random means the same request either always samples in or
always samples out, so a traced request stays complete.

## OpenTelemetry

`@cona/opentelemetry` wraps setup and holds the **only** `@prisma/*` import in the
repository — `@prisma/instrumentation` at `src/dependencies.ts:10`.

Workers export metrics straight to Axiom OTLP:

| Setting         | Value                             |
| --------------- | --------------------------------- |
| Endpoint        | `https://api.axiom.co/v1/metrics` |
| Protocol        | `http/protobuf`                   |
| Timeout         | 3,000 ms                          |
| Export interval | 60,000 ms                         |

`worker.ts:377-378` notes OTLP export is best-effort: the telemetry package absorbs
exporter failures so observability can never block shutdown.

Worker-side OTel wiring lives in `telemetry.ts`, `temporal-opentelemetry.ts`, and
`temporal-metrics.ts`.

## Notes

**Staging runs a failure probe.** `OBSERVABILITY_PROBE_ENABLED = 1` in staging, `0` in
production, driving `observabilityFailureProbeWorkflow`.

**Browser logs take a detour.** The client cannot write to Axiom directly, so
`api/axiom/ingest` forwards them. Portal has its own copy of this route.

**Runtime context is split by environment.** `server-runtime-context.ts` and
`browser-runtime-context.ts` in webapp, `worker-runtime-context.ts` in workers, all
feeding `@cona/observability`'s `runtime-context.ts`.

**Error classification is deliberate.** `lib/axiom/request-error-classification.ts`
decides which request failures are worth reporting — expected 4xx responses do not become
alerts.