Redis Caching System

Overview

CONA uses Redis as an in-memory caching layer to significantly improve application performance by reducing database query latency. Redis stores frequently accessed data in memory, providing sub-millisecond access times for cached data.

Why Redis?

Performance Benefits

Query Latency Reduction: Reduces database query time from ~100ms to ~5ms
Workflow Throughput: Improves Temporal workflow performance by reducing I/O bottlenecks
User Experience: Faster page loads and real-time data updates
Cost Efficiency: Reduces database load, allowing for smaller database instances

Use Cases in CONA

Chart of Accounts Caching: Frequently accessed account data and lists
Organization Settings: User preferences and configuration data
Posting Matrix Rules: Complex accounting rules that are expensive to compute
Integration Data: Cached results from external API calls
Session Data: User authentication and session information

Architecture

┌─────────────────┐    Cache Miss     ┌─────────────────┐
│   WebApp        │ ────────────────► │   Supabase      │
│   (Next.js)     │                   │   PostgreSQL    │
└─────────────────┘                   └─────────────────┘
         │                                     │
         │ Cache Hit                           │
         ▼                                     │
┌─────────────────┐    Store Result           │
│     Redis       │ ◄─────────────────────────┘
│   (In-Memory)   │
└─────────────────┘
         ▲
         │ Redis Functions
         ▼
┌─────────────────┐
│   @cona/core    │
│   (Business     │
│    Logic)       │
└─────────────────┘

This documentation provides comprehensive coverage of Redis usage in CONA, including:

Why Redis is used - Performance benefits and use cases
Architecture overview - How Redis fits into the system
Configuration details - Production and development setup
Implementation patterns - Code examples and best practices
Cache version management - Version-prefixed keys; generations expire via TTL (no full-cache flush)
Monitoring integration - How to connect with Redis Insights
Troubleshooting guide - Common issues and solutions
Security considerations - Access control and data protection
Cost optimization - Current costs and optimization strategies

Redis Configuration

Production Environment

Provider: Redis Cloud Configuration:

Database: 13661105
Subscription: 2958056
Region: Frankfurt (fra)
Memory: 250MB (Dataset: 125MB, Total: 250MB)
High Availability: Single zone
Connections: 30 concurrent connections
Persistence: RDB snapshots every 6 hours
Network Cap: 100GB/month

Connection Details:

REDIS_HOST=redis-13661105.c1.eu-west-1-1.ec2.cloud.redislabs.com
REDIS_PORT=16462
REDIS_PASSWORD=<secure-password>
REDIS_TLS=true
REDIS_ENV=production

Connection string - rediss://:{REDIS_PASSWORD}@{REDIS_HOST}:{REDIS_PORT} GUI to connect - Redis Insight

Development Environment

Local Redis Instance: Local instances is running in docker container. To spin up the instances run pnpm run redis:start:dev

REDIS_HOST=localhost
REDIS_PORT=6380
REDIS_PASSWORD=redisLoca1passw0rd
REDIS_TLS=false
REDIS_ENV=development

Environment Variables

Required Variables:

REDIS_HOST - Redis server hostname
REDIS_PORT - Redis server port
REDIS_PASSWORD - Redis authentication password
REDIS_TLS - Set to "true" for TLS connections (production)
REDIS_ENV - Environment identifier for cache key prefixing (defaults to no prefix in production)

Optional Variables:

REDIS_ENABLED - Set to "false" to disable Redis (all cache operations will be no-ops)
REDIS_CACHE_VERSION - Runtime override for the shared cache generation. Normally leave it unset and let DEFAULT_CACHE_VERSION govern (see below)
- Falls back to DEFAULT_CACHE_VERSION ("1") when unset or blank
- If set at all, it must be identical across webapp, temporal-workers, and portal — never derived from a per-service deploy identity
- Every cache key is stored as v{version}:{key}. Changing the version rolls to a new generation; the old one is left to expire via TTL (no FLUSHDB)
- Does not use VERCEL_GIT_COMMIT_SHA (that previously caused cross-service FLUSHDB wars — see CONA-1017 / CONA-1090)

Cache Version Management

Every cache entry is stored under a version-prefixed key — v{version}:{key} — where the version comes from the shared REDIS_CACHE_VERSION (default "1"). Bumping the version rolls all cache reads and writes onto a fresh keyspace; the previous generation is simply never read again and expires via its own TTLs. There is no FLUSHDB and no connect-time version check — independently deployed services (webapp on Vercel, temporal-workers on Fly.io, portal) never touch each other’s data, and ordinary deploys never invalidate the cache. This replaced an earlier design where a version mismatch triggered a global FLUSHDB on connect. Because the version was derived per-service (VERCEL_GIT_COMMIT_SHA on webapp vs "1" on workers), the shared database was flushed back and forth on every deploy and restart — the CONA-1090 “FLUSHDB war”.

How It Works

Key prefixing: getFromCache / setInCache / deleteFromCache / deleteFromCacheByPattern resolve every logical key through toStorageKey(), which prepends v{version}: (and the REDIS_ENV namespace in non-production).
Version bump: changing the version changes the prefix, so subsequent reads miss the old generation and re-populate under the new one.
Old generation expiry: orphaned v{old}:… keys are never read again and expire naturally via their TTLs — no active deletion, no blocking flush.
Non-cache keys are never versioned: counters, locks, rate limits and JTIs (via the atomic.ts helpers) use raw keys and are unaffected by a version bump.

Version Sources (Priority Order)

Runtime override (REDIS_CACHE_VERSION): wins when set. Use it only to roll the generation without a deploy, and set the same string on every service — or on none.
DEFAULT_CACHE_VERSION (packages/core/src/redis/index.ts, currently "1"): the normal mechanism. It ships with the code, so every service that does not pin the env var moves to the new generation on the same deploy.

VERCEL_GIT_COMMIT_SHA is intentionally ignored — per-deploy SHAs differ between services and previously caused the webapp and workers to flush each other’s cache.

Careful: the env var wins over the constant. If a service pins REDIS_CACHE_VERSION, bumping DEFAULT_CACHE_VERSION will not move that service, and you get services split across generations — the class of bug CONA-1090 fixed. Keep it unset everywhere unless you are deliberately using it.

Configuration Examples

Production / Development (default):

# Leave REDIS_CACHE_VERSION unset everywhere. Keys are stored under the generation named
# by DEFAULT_CACHE_VERSION ("v1:"). Ordinary deploys keep the cache; nothing is flushed.

Roll to a new generation (when a cached payload’s shape changes):

// Preferred: packages/core/src/redis/index.ts — ships with the code, so webapp,
// temporal-workers and portal all move together on the same deploy.
const DEFAULT_CACHE_VERSION = "2";

# Alternative, only when you must roll without deploying: set the SAME value in
# webapp, temporal-workers and portal. New reads/writes use "v2:"; "v1:" expires via TTL.
REDIS_CACHE_VERSION=2

When the Cache Rolls

DEFAULT_CACHE_VERSION bump: the deploy carrying it moves every service that does not pin the env var onto a new generation; the old one expires via TTL.
Runtime override change: setting or changing REDIS_CACHE_VERSION (same value on all services) does the same without a deploy.
Ordinary deploys: nothing happens — the cache persists across deploys by design.
Shape change without a bump: old-shaped entries stay reachable under the current prefix until their TTLs expire, so bump the version whenever a cached payload’s shape changes incompatibly.

Cost of a bump

Because the old generation is not deleted, both generations coexist in memory until the old keys’ TTLs expire (most CONA caches use TTLs ≤ 1 hour; the longest are 24 h / 7 d). On the 250 MB production instance this transient overlap is acceptable given how rarely the version changes. Normally you just wait it out.

Do not reclaim that memory with FLUSHDB. This database is shared, and the flush is not limited to the old generation: it wipes every service’s current cache, and also the deliberately unversioned atomic.ts keys — portal PDF JTI replay protection, Xentral rate limiter windows and cooldowns, and locks. That reintroduces exactly the cross-service disruption CONA-1090 removed, and it resets replay protection.If you must reclaim it early, delete only the retired generation by its prefix:

# Dry run first — inspect what would go
redis-cli --scan --pattern 'v1:*' | head

redis-cli --scan --pattern 'v1:*' | xargs -r -n 500 redis-cli del

The prefix scopes this to one generation of one environment: other environments carry their own {REDIS_ENV}: prefix, and unversioned atomic keys never match.

Note: the previous design emitted a redis.cache_flush Axiom event on each connect-time flush. That flush no longer exists, so no new redis.cache_flush events are emitted; historical events remain queryable.

Guards against a split generation

Two checks run at startup so a repeat of CONA-1090 is visible immediately instead of after weeks. A deploy-derived version is rejected. If REDIS_CACHE_VERSION looks like a commit SHA, or matches a variable the platform sets per deploy (VERCEL_GIT_COMMIT_SHA, VERCEL_DEPLOYMENT_ID, FLY_MACHINE_ID, GITHUB_SHA, …), it is ignored and the service uses DEFAULT_CACHE_VERSION instead. A misconfigured service rejoins the shared generation rather than quietly getting a private cache. It logs redis.cache_version_rejected. A version made only of digits — 20260727 — is not rejected. Only the presence of a letter a–f distinguishes a SHA from a deliberate date-style version. Every process publishes the generation it resolved. All records live in one hash, cache_generation:{REDIS_ENV}, with a field per process ({service}:{instance}) holding {generation}|{writtenAt}. Each process rewrites its field every 10 minutes and compares the hash. Agreement logs redis.cache_generation; disagreement logs redis.cache_generation_mismatch at error level naming each instance and its generation. Alert on the mismatch event. All of these — along with redis.env_missing, redis.cache_version_rejected and redis.cache_generation_failed — go through the logger installed by configureRedisObservability, not the module-level consoleLogger. Only the configured one reaches the service’s Axiom dataset; using the console logger would leave the events in stdout and the alert would never fire. One hash instead of one key per process specifically to avoid SCAN: matching cache_generation:* still walks the whole keyspace, which on production is roughly 1,800 round trips per report at COUNT 100, on every process start and every refresh. That is the cost this ticket exists to reduce, so the check must not reintroduce it. HGETALL is one command. Redis cannot expire individual hash fields portably, so freshness is carried in the value: a process that has missed three refreshes is treated as gone and its field is deleted on the next read. That keeps the hash small as instances are replaced, and stops a stopped process from faking a disagreement. That cleanup deletes a field only if it still holds the value the reporter classified as stale, via a small Lua compare-and-delete. HGETALL and HDEL are separate commands, so a process that had been silent long enough to look dead — a frozen serverless instance, or a worker whose event loop stalled — can refresh in between and have its brand-new value deleted from a stale snapshot. It would then be absent from comparisons until its next refresh, and a rollout could report agreement while that process was on a different generation. The record is keyed by process, not by service, and refreshed rather than written once. Both matter for it to work at all:

RedisService has only three values and the portal reports itself as webapp, so a service-only key let the portal overwrite the webapp’s record — and let a starting instance overwrite a draining one during the rollout the check is meant to watch.
A worker that stays up for days would otherwise lose its record and vanish from the comparison. A webapp deployed later would then find only itself and report agreement — missing the long-running-worker-versus-new-webapp case, which is the shape CONA-1090 actually took.

Cache stamps — invalidating without deleting

Deleting keys only clears the caller’s own generation, so during a rollout an older process can keep serving data it should have dropped. It also costs a full keyspace scan per pattern. Cache stamps fix both. A stamp is a token that changes when source data changes. Put it in the key and a mutation stops deleting anything — every reader builds a new key, misses, and refetches:

posting_matrix_rules:{org}:{dimension}:{event}:{stamp}

It goes on the end: keyPrefix observability reads the segment before the first colon, and the pattern deletes still running in phase 2 are anchored at the front, so appending keeps both working. Old and new builds compute the same key, because the stamp is shared data rather than per-process config. They cannot disagree. Stamp keys carry the environment but not the generation. cacheStampKey() produces cache_stamp:{REDIS_ENV}:{scope}. The split is deliberate:

No v{version}: prefix, so every build reads the same stamp. That is what closes the rollout gap; writing a stamp through setInCache would give it the generation prefix and bring the problem straight back.
Namespaced by environment, because environments share one Redis instance. Without it, a staging save would invalidate production’s caches for any organization id present in both databases.

import {
  bumpCacheStamp,
  getCacheStamp,
  REDIS_STAMP_SCOPES,
  withCacheStamp,
} from "@cona/core/redis";

// on mutation — inside the shared invalidation helper, not at the call site
await bumpCacheStamp(REDIS_STAMP_SCOPES.postingMatrix(organizationId));

// on read — once, BEFORE the query, and reused for the write
const stamp = await getCacheStamp(REDIS_STAMP_SCOPES.postingMatrix(organizationId));
const cacheKey =
  stamp === null
    ? null
    : withCacheStamp(REDIS_DYNAMIC_KEYS.glDimensionsList(organizationId), stamp);

if (cacheKey) {
  const cached = await getFromCache<T>(cacheKey);
  // …
  await setInCache(cacheKey, value, 3600);
}

Read the stamp before the query, not after. If a save lands while the query is in flight, the write goes to the pre-save key and no reader ever asks for it — the stale rows are stranded. Re-reading the stamp at write time does the opposite: rows fetched before the save get written under the stamp readers are now using, and that is stale data served as fresh. A stamp only has to be a value that has never been used before. It is not a counter and is not ordered — an earlier draft compared timestamps, which made correctness depend on three separately hosted services agreeing about the clock. Each stamp is a short time prefix plus 48 random bits, so two saves in the same millisecond produce different stamps, a host whose clock lags produces a usable one, and a stamp recreated after eviction cannot repeat a value that a surviving cache entry is still stored under. Both helpers return null when Redis is down — callers should bypass the cache rather than invent a stamp, because two processes inventing different ones is the split this removes.

Rollout (CONA-1096)

Changing the key format is itself a rollout, so it goes in phases. Skipping straight to phase 2 would recreate the very stale window this removes.

Phase	Change	Status
1	Mutations bump the stamp. Nothing reads it. Existing deletes stay.	done
2	Readers put the stamp in the key. Deletes stay.	done
3	Drop the pattern deletes.	todo

Phase 1 must be deployed everywhere before phase 2 ships. That is what makes phase 2 safe: an older process still deletes its own keys, a newer one invalidates via the stamp, and because phase 1 taught the older process to bump that stamp, both are correct at the same time. Phase 3 waits until nothing reads unstamped keys. Phase 2 covers every reader of the postingMatrix scope — fetchPostingMatrixRules, fetchGlDimensions, getGlDimensions, getGlDimensionsWithoutAccounts. The other posting-matrix caches (posting_matrices_grouped, posting_matrix:{org}:{id}) are UI-facing, are not in the scope, and keep their existing deletes. Two things worth knowing while phase 2 is live:

Orphaned entries are left to expire. An invalidated key is not deleted, just never addressed again. It occupies memory for up to its TTL (1 h for these), so expect a modest bump in key count that flattens out.
Each read costs one extra round trip for the stamp. createAccountingImpact already memoizes rule fetches per document, so this is a handful of extra EVALs per document, not per rule evaluation.
A failed stamp bump still invalidates. The list deletes are exact unstamped keys. If bumpCacheStamp returns null while those deletes succeed, the helpers retire the stamp (clearCacheStamp) and clean up the stamped list keys that were stored under it. Retiring is required: deleting the list entry alone leaves the stamp current, so an in-flight miss holding it can rewrite that same key after the delete and serve the pre-save snapshot for the remaining TTL.

Implementation

Import Redis Functions

// Import Redis functions from @cona/core
import {
  startRedis,
  stopRedis,
  getFromCache,
  setInCache,
  deleteFromCache,
  deleteFromCacheByPattern,
  deleteAllChartOfAccountsCache,
  deleteAllPostingMatrixCache,
  deletAllGlDimensionsCache,
  deleteAllOrganizationCache,
  REDIS_DYNAMIC_KEYS,
  // Observability
  configureRedisObservability,
  flushRedisRollup, // flush aggregated rollup data on demand (e.g. before shutdown)
  stopRedisRollup, // stop periodic rollup timer + final flush (called automatically by stopRedis)
} from "@cona/core/redis";

Redis Client Initialization

Redis is automatically initialized on application startup via the startRedis() function. This function:

Connection Management: Handles Redis connection with automatic retry logic
Error Handling: Gracefully handles connection failures and returns status
Version Checking: Automatically checks and updates cache version on connection
Reconnection: Implements exponential backoff retry strategy (4 attempts max)
Return Value: Returns a RedisStartResult object with connection status

// Automatically called on app startup (e.g., in instrumentation.ts)
import { startRedis, type RedisStartResult } from "@cona/core/redis";

const result: RedisStartResult = await startRedis();

// Result structure:
// {
//   connected: boolean,
//   reason: "already-started" | "disabled" | "connected" | "error",
//   message?: string  // Error message if reason is "error"
// }

// Handle result
if (result.connected) {
  if (result.reason === "connected") {
    console.log("✅ Redis connection established successfully");
  } else if (result.reason === "already-started") {
    console.log("ℹ️ Redis already initialized");
  }
} else {
  if (result.reason === "disabled") {
    console.log("ℹ️ Redis is disabled (REDIS_ENABLED=false)");
  } else if (result.reason === "error") {
    console.error("⚠️ Redis connection failed:", result.message);
    // App continues to work without Redis - all cache operations return null
  }
}

Connection Retry Strategy:

Initial delay: 4 seconds
Max attempts: 4
Backoff: Geometric progression (4s, 8s, 16s, 32s)
Total timeout: ~65 seconds (sum of all delays + 5s buffer)

Core Cache Functions

All cache functions include:

Graceful Degradation: Return null or no-op if Redis is unavailable
Environment Prefixing: Automatically prefix keys with REDIS_ENV in non-production
Error Handling: Log warnings but don’t throw errors (allows app to continue)

// Get data from cache
export const getFromCache = async <T>(key: string): Promise<T | null> => {
  // Returns null if Redis disabled, unavailable, or on error
  // Automatically handles environment prefixing
};

// Store data in cache
export const setInCache = async <T>(key: string, value: T, ttl: number): Promise<void> => {
  // No-op if Redis disabled or unavailable
  // Automatically handles environment prefixing
};

// Delete specific key
export const deleteFromCache = async (key: string): Promise<void> => {
  // Throws error on failure (caller can handle)
  // Automatically handles environment prefixing
};

// Delete by pattern (wildcard support)
export const deleteFromCacheByPattern = async (pattern: string): Promise<void> => {
  // Uses SCAN for efficient pattern matching
  // Throws error on failure (caller can handle)
  // Automatically handles environment prefixing
};

Physical Key Prefixing: toStorageKey() prepends the cache generation v{version}: everywhere, plus the REDIS_ENV namespace outside production. Callers always pass the logical key:

// Logical key passed by the caller: "organization:details:org-123"
//
// Production:  "v1:organization:details:org-123"
// Development: "development:v1:organization:details:org-123"
// Staging:     "staging:v1:organization:details:org-123"

If REDIS_ENV is unset the namespace falls back to unknown: — deliberately not production, so an unconfigured service gets its own isolated keyspace rather than writing into production’s. It also shares nothing with correctly-configured services, so startRedis() logs a redis.env_missing error when it is absent. Every deployed service (webapp, temporal-workers, portal) must set it.

Cache Key Conventions

// packages/core/src/redis/constants.ts
export const REDIS_DYNAMIC_KEYS = {
  organizationDetails: (organizationId: string) => `organization:details:${organizationId}`,
  chartOfAccountsListOrderedAsc: (organizationId: string, onlyReconcileAccounts: boolean) =>
    `chart_of_accounts:list:ordered:asc:${organizationId}:${onlyReconcileAccounts}`,
  chartOfAccountsSearch: (organizationId: string) => `chart_of_accounts:search:${organizationId}`,
  postingMatricesGrouped: (organizationId: string) => `posting_matrix:grouped:${organizationId}`,
  glDimensionsList: (organizationId: string) => `gl_dimensions:list:${organizationId}`,
  glDimensionsListNotAccount: (organizationId: string) =>
    `gl_dimensions:list:not_account:${organizationId}`,
};

Key Structure: {domain}:{type}:{organizationId}:{identifier}

Bulk Cache Invalidation Functions

CONA provides specialized functions for invalidating related caches:

// Invalidate all chart of accounts caches
await deleteAllChartOfAccountsCache(organizationId);

// Invalidate all posting matrix caches
await deleteAllPostingMatrixCache(organizationId);

// Invalidate all GL dimensions caches
await deletAllGlDimensionsCache(organizationId);

// Invalidate ALL organization caches (use with caution)
await deleteAllOrganizationCache(organizationId);

Usage Patterns

1. Cache-Aside Pattern

// Get data with caching
export async function getChartOfAccountById(organizationId: string, id: string) {
  const cacheKey = REDIS_DYNAMIC_KEYS.chartOfAccountsById(organizationId, id);

  // Try cache first
  const cached = await getFromCache<ChartOfAccount>(cacheKey);
  if (cached) {
    return cached;
  }

  // Cache miss - fetch from database
  const account = await prisma.chartOfAccount.findUnique({
    where: { id, organizationId },
  });

  if (account) {
    // Store in cache for 1 hour
    await setInCache(cacheKey, account, 3600);
  }

  return account;
}

2. Write-Through Pattern

// Update data with cache invalidation
export async function updateChartOfAccount(organizationId: string, id: string, data: UpdateData) {
  // Update database
  const updated = await prisma.chartOfAccount.update({
    where: { id, organizationId },
    data,
  });

  // Invalidate related caches
  await deleteFromCache(REDIS_DYNAMIC_KEYS.chartOfAccountsListOrderedAsc(organizationId, true));
  await deleteFromCache(REDIS_DYNAMIC_KEYS.chartOfAccountsListOrderedAsc(organizationId, false));
  await deleteFromCache(REDIS_DYNAMIC_KEYS.chartOfAccountsSearch(organizationId));
  await deleteFromCache(REDIS_DYNAMIC_KEYS.chartOfAccountsById(organizationId, id));

  return updated;
}

3. Cache Invalidation Strategies

// Invalidate all chart of accounts for an organization
await deleteFromCacheByPattern(`chart_of_accounts:*:${organizationId}:*`);

// Invalidate specific account
await deleteFromCache(REDIS_DYNAMIC_KEYS.chartOfAccountsById(organizationId, id));

// Use bulk invalidation function (recommended)
await deleteAllChartOfAccountsCache(organizationId);

Monitoring & Debugging

Redis Insights Integration

Production Redis Cloud Dashboard: https://cloud.redis.io/#/databases/13661105/subscription/2958056/view-bdb/configuration Key Metrics to Monitor:

Memory Usage: Track current / 250MB — check Redis Cloud dashboard
Hit Rate: Target >90% cache hit rate
Connection Count: Monitor concurrent connections
Key Count: Track number of cached keys
Operations per Second: Monitor Redis performance
Network Usage: Monitor against the 100 GB/month cap — use the redis.usage_rollup Axiom queries above for per-service and per-key-family breakdowns
Cache Version: The active generation is the v{version}: prefix on cache keys (REDIS_CACHE_VERSION, default "1")
Cache Flushes: The app no longer flushes on connect; redis.cache_flush events exist only as historical records of the retired design

Redis CLI Access

# Connect to production Redis
redis-cli -h redis-13661105.c1.eu-west-1-1.ec2.cloud.redislabs.com -p 16462 -a <password> --tls

# List all keys
KEYS *

# Get key value — note the v{version}: generation prefix on the physical key
# (and a "{REDIS_ENV}:" prefix in front of it outside production)
GET "v1:organization:details:org-123"

# Inspect the active cache generation
KEYS "v1:*"

# Monitor commands in real-time
MONITOR

# Get memory usage
INFO memory

# Get key statistics
INFO keyspace

Application Logging

The Redis implementation includes comprehensive logging:

Connection Events: Logs when Redis connects, reconnects, or fails (redis.connection → Axiom)
Cache Generation: Cache keys are prefixed with the shared v{version}: generation; there is no connect-time version check or flush to log
Aggregate Usage: Emits redis.usage_rollup events periodically (see Axiom Observability section)
Error Handling: Logs warnings when cache operations fail (allows graceful degradation)

Check application logs for messages like:

"Redis connection established successfully"

In Axiom, prefer querying the structured events (e.g. redis.usage_rollup) over free-text log lines — they are always present regardless of log level filtering.

Axiom Observability

Every Redis operation emits structured telemetry to Axiom. This data is available now in the live datasets and can be queried immediately to investigate performance, cache efficiency, and network usage — without any code changes. Four event types are written under the fields.event key:

Event	Level	When emitted
`redis.call`	`debug` (errors → `error`)	Every GET, SET, DEL, SCAN, etc.
`redis.connection`	`info` (errors → `error`)	Every connect / ready / reconnect / error lifecycle transition
`redis.usage_rollup`	`info`	Periodic aggregate (configurable interval, default off)
`redis.cache_flush`	`info`	Retired (CONA-1090) — was emitted on the old connect-time `FLUSHDB`; historical events only

redis.call events are emitted at debug level. In long-running Temporal workers, the pino logger typically runs at info, so per-call debug events are dropped there. redis.usage_rollup fills that gap — it aggregates the same data and emits at info so it always reaches Axiom.

Setup

Call configureRedisObservability() once at process startup, after startRedis(), to wire the correct service identity and logger into all subsequent events. Without this call the events fall back to consoleLogger and never reach Axiom.

import { configureRedisObservability } from "@cona/core/redis";
import { webappLogger } from "@/app/lib/logger"; // or workerRedisLogger in workers

// In instrumentation.ts (webapp)
configureRedisObservability({
  service: "webapp",
  logger: webappLogger,
  // Flush aggregated rollup every 60 s on warm Vercel instances.
  rollupIntervalMs: 60_000,
});

// In worker.ts (Temporal)
configureRedisObservability({
  service: "worker",
  logger: workerRedisLogger,
  // 30 s rollup is the primary visibility mechanism for workers — per-call debug
  // events are filtered out by pino's LOG_LEVEL=info setting.
  rollupIntervalMs: 30_000,
});

Options:

Option	Type	Default	Description
`service`	string	required	`"webapp"` \| `"temporal"` \| `"worker"`
`logger`	Logger	consoleLogger	Logger that routes events to Axiom. Without this, events only go to console.
`rollupIntervalMs`	number	`0` (off)	Interval in ms between `redis.usage_rollup` flushes. Set > 0 to enable periodic aggregation.

The state is stored on globalThis via Symbol.for("cona.redis.observability") so it is shared across all Next.js bundles in the same Node process — the instrumentation file and route handlers always see the same configuration. Shutting down (Temporal workers only): Call flushRedisRollup() before the final pino drain so the last partial window is shipped to Axiom before process.exit() kills the transport:

import { flushRedisRollup, stopRedis } from "@cona/core/redis";

// In worker.ts finally block, before the last logAndFlush:
flushRedisRollup();
await logAndFlush(logger, "info", "All workers stopped, exiting");
stopRedis(); // also calls stopRedisRollup() to clear the interval

`redis.call` Event Schema

Emitted by observeRedisCall() which wraps every Redis command. Each field maps directly to an Axiom column under fields.*.

Field	Type	Description
`event`	`"redis.call"`	Constant discriminator for filtering
`service`	`string`	Originating service: `webapp`, `temporal`, `worker`
`environment`	`string`	Value of `REDIS_ENV` or `NODE_ENV`
`caller`	`string`	Human-readable name of the calling function, e.g. `"getFromCache"`
`operation`	`string`	Lowercase Redis command, e.g. `"get"`, `"set"`, `"scan_del"`
`command`	`string`	Uppercase version of `operation`, e.g. `"GET"`, `"SET"`
`keyPrefix`	`string`	Static key family — first segment before the first colon, e.g. `"chart_of_accounts"`, `"rate_limit"`
`keyCount`	`number`	Number of keys involved in the call
`requestBytes`	`number`	UTF-8 byte size of keys + serialised values sent to Redis
`responseBytes`	`number`	UTF-8 byte size of the raw response received from Redis
`totalBytes`	`number`	`requestBytes + responseBytes`
`totalKb`	`number`	`totalBytes / 1024`, rounded to 2 decimal places
`durationMs`	`number`	Round-trip time in milliseconds
`result`	`string`	Outcome class: `"hit"`, `"miss"`, `"set"`, `"del"`, `"scan_del"`, `"error"`
`cacheHit`	`boolean?`	Only present on GET operations — `true` = cache hit, `false` = miss
`ttlSeconds`	`number?`	Only present on write operations — TTL passed to `SET EX`
`retryCount`	`number?`	Only present when > 0 — number of retries attempted
`pipelineLength`	`number?`	Only present for pipeline calls — number of commands in the batch
`error`	`string?`	Only present when `result = "error"` — error message without raw keys or values

Key prefix extraction strips everything after the first colon so dynamic segments (org IDs, user tokens, etc.) are never exposed in logs:

"chart_of_accounts:list:ordered:asc:org123:true"  →  "chart_of_accounts"
"rate_limit:stripe_app:acc_123:42"                →  "rate_limit"
"recon:quarantine:org123:acc1:gid7"               →  "recon"

`redis.connection` Event Schema

Emitted by attachRedisConnectionLifecycleListeners() on every ioredis lifecycle transition.

Field	Type	Description
`event`	`"redis.connection"`	Constant discriminator for filtering
`service`	`string`	Originating service
`environment`	`string`	Value of `REDIS_ENV` or `NODE_ENV`
`status`	`string`	`"connect"`, `"ready"`, `"reconnecting"`, or `"error"`
`errorName`	`string?`	Only present on `status = "error"`
`errorMessage`	`string?`	Only present on `status = "error"`

`redis.usage_rollup` Event Schema

Emitted at info level by the periodic rollup timer and by flushRedisRollup() at shutdown. Each event covers one (service, caller, operation, keyPrefix) bucket for a single time window.

Field	Type	Description
`event`	`"redis.usage_rollup"`	Constant discriminator for filtering
`service`	`string`	Originating service: `webapp`, `temporal`, `worker`
`environment`	`string`	Value of `REDIS_ENV` or `NODE_ENV`
`windowStartMs`	`number`	Unix ms when this bucket was opened (after last flush)
`windowEndMs`	`number`	Unix ms when this flush was triggered
`windowDurationMs`	`number`	`windowEndMs - windowStartMs`
`caller`	`string`	Calling function name, e.g. `"getFromCache"`
`operation`	`string`	Redis operation, e.g. `"get"`, `"set"`, `"scan_del"`
`command`	`string`	Uppercase Redis command, e.g. `"GET"`, `"SET"`
`keyPrefix`	`string`	Key family (first segment before colon)
`calls`	`number`	Total number of Redis calls in this window
`errors`	`number`	Number of calls that resulted in `"error"`
`requestBytes`	`number`	Sum of request bytes across all calls in the window
`responseBytes`	`number`	Sum of response bytes across all calls in the window
`totalBytes`	`number`	`requestBytes + responseBytes`
`totalKb`	`number`	`totalBytes / 1024`, rounded to 2 decimal places

`redis.cache_flush` Event Schema (retired)

No longer emitted. The connect-time FLUSHDB was removed in CONA-1090 — the cache version now lives in the key prefix (v{version}:). This schema is retained only to interpret historical events already in Axiom.

Field	Type	Description
`event`	`"redis.cache_flush"`	Constant discriminator for filtering
`service`	`string`	Originating service
`environment`	`string`	Value of `REDIS_ENV` or `NODE_ENV`
`reason`	`"version_mismatch"`	Why the flush was triggered
`previousVersion`	`string`	The version that was stored in Redis before the flush
`newVersion`	`string`	The version written after the flush
`keyCount`	`number`	Number of keys that were wiped (from `DBSIZE` pre-flush)

Privacy Guarantees

The observability layer is designed so that raw Redis keys, values, tokens, session IDs, and any PII are never written to Axiom. Only the following are logged:

Byte counts — requestBytes, responseBytes, totalBytes
Normalised key prefix — the static first segment of the key, not the full key
Error messages — stripped of key names and raw values

Example Axiom Queries

Use the cona_webapp dataset for webapp events. For Temporal worker events use cona_temporal-workers (note: redis.call debug events are dropped at the worker’s pino level — use redis.usage_rollup events from that dataset instead). Cache hit rate over time

fields.event == "redis.call" and fields.operation == "get"
| summarize
    hits = countif(fields.cacheHit == true),
    misses = countif(fields.cacheHit == false)
  by bin(_time, 5m)
| extend hitRate = hits / (hits + misses) * 100

Slowest Redis operations (p95 duration)

fields.event == "redis.call"
| summarize p95Duration = percentile(fields.durationMs, 95) by fields.keyPrefix, fields.operation
| sort by p95Duration desc

Network bytes sent to Redis per key family

fields.event == "redis.call"
| summarize totalKb = sum(fields.totalKb) by fields.keyPrefix
| sort by totalKb desc

Recent connection lifecycle events (detect reconnect storms)

fields.event == "redis.connection"
| project _time, fields.service, fields.status, fields.errorMessage
| sort by _time desc

All Redis errors in the last hour

fields.event == "redis.call" and fields.result == "error"
| project _time, fields.service, fields.caller, fields.keyPrefix, fields.error
| sort by _time desc

Volume of SET operations by TTL bucket

fields.event == "redis.call" and fields.operation == "set"
| extend ttlBucket = case(
    fields.ttlSeconds <= 300, "≤5 min",
    fields.ttlSeconds <= 1800, "≤30 min",
    fields.ttlSeconds <= 3600, "≤1 hr",
    ">1 hr"
  )
| summarize count() by ttlBucket

Aggregate network usage per service (rollup — works for both webapp and workers)

['cona_webapp']
| extend eventName = ensure_field('fields.event', typeof(string))
| where eventName == "redis.usage_rollup"
| summarize totalKb = sum(ensure_field('fields.totalKb', typeof(real))) by fields.service
| sort by totalKb desc

Top bandwidth consumers by key family (rollup)

['cona_webapp']
| extend eventName = ensure_field('fields.event', typeof(string))
| where eventName == "redis.usage_rollup"
| summarize
    calls = sum(ensure_field('fields.calls', typeof(int))),
    totalKb = sum(ensure_field('fields.totalKb', typeof(real)))
  by fields.keyPrefix, fields.operation
| sort by totalKb desc

Rollup over the Temporal workers dataset

['cona_temporal-workers']
| extend eventName = ensure_field('fields.event', typeof(string))
| where eventName == "redis.usage_rollup"
| summarize totalKb = sum(ensure_field('fields.totalKb', typeof(real))) by fields.keyPrefix
| sort by totalKb desc

Cache flush history (when and how many keys were wiped)

['cona_webapp']
| extend eventName = ensure_field('fields.event', typeof(string))
| where eventName == "redis.cache_flush"
| project
    _time,
    fields.service,
    fields.previousVersion,
    fields.newVersion,
    fields.keyCount
| sort by _time desc

Best Practices

1. TTL (Time To Live) Strategy

// Different TTLs for different data types
const TTL = {
  SHORT: 300, // 5 minutes - frequently changing data
  MODERATE: 1800, // 30 minutes - moderately unstable data
  MEDIUM: 3600, // 1 hour - moderately stable data
  LONG: 86400, // 24 hours - rarely changing data
  VERY_LONG: 604800, // 7 days - static reference data
};

// Usage
await setInCache(key, data, TTL.MEDIUM);

2. Cache Key Design

Hierarchical: Use colons to separate levels (domain:type:org:id)
Consistent: Follow the same pattern across all cache keys
Descriptive: Make keys self-documenting
Environment-aware: Automatically prefixed with REDIS_ENV in non-production

3. Error Handling

All cache functions are designed to fail gracefully:

// getFromCache - returns null on error (allows fallback to DB)
const cached = await getFromCache<Data>(key);
if (!cached) {
  // Fallback to database
}

// setInCache - no-op on error (allows app to continue)
await setInCache(key, data, ttl); // Won't throw

// deleteFromCache - throws on error (caller can handle)
try {
  await deleteFromCache(key);
} catch (error) {
  // Handle deletion failure
}

4. Memory Management

Set appropriate TTLs to prevent memory bloat
Use bulk invalidation functions for related caches
Monitor memory usage regularly in Redis Insights
Cache version management rolls to a new key generation when REDIS_CACHE_VERSION is bumped; the old generation expires via TTL (no flush)

5. Cache Version Management

Shared across services: Default "1", or the same REDIS_CACHE_VERSION on webapp, workers, and portal
Roll a generation: Bump REDIS_CACHE_VERSION to the same new value in every service when cached payload shapes change; reads/writes move to the new v{version}: prefix and the old generation expires via TTL (no flush)
No per-deploy auto-flush: Ordinary deploys leave Redis intact — do not rely on VERCEL_GIT_COMMIT_SHA
Development: Uses default version or set manually for testing
Monitor Axiom: Watch redis.usage_rollup for hit ratios and network usage after a version bump

Troubleshooting

Common Issues

Connection Timeouts
- Check Redis Cloud status
- Verify network connectivity
- Check connection pool settings
- Review retry logs (max 4 attempts with exponential backoff)
Memory Issues
- Monitor memory usage in Redis Insights
- Check for memory leaks in key patterns
- Adjust TTL values
- Use bulk invalidation functions
Performance Issues
- Check cache hit rates
- Monitor Redis CPU usage
- Verify key patterns are efficient
- After a version bump, expect a brief cold-cache window while the new generation populates
Data Inconsistency
- Ensure proper cache invalidation
- Check for race conditions
- Verify write-through patterns
- Confirm REDIS_CACHE_VERSION matches across all services (a mismatch routes services to different generations)
Stale Cache After Deploy / Restart
- Ordinary deploys keep the cache — bump the shared REDIS_CACHE_VERSION on all services to roll to a fresh generation when a cached payload’s shape changed
- Verify REDIS_CACHE_VERSION is identical across webapp, temporal-workers, and portal (or unset everywhere so all use "1") — a mismatch sends services to different generations
- The old generation expires via TTL — wait it out. Never FLUSHDB this shared database; if you must reclaim memory early, delete the retired generation by prefix (see Cost of a bump)

Debug Commands

# Check Redis status
redis-cli ping

# Get detailed info
redis-cli info

# Check specific key — note the physical key carries the generation prefix
# (and a "{REDIS_ENV}:" prefix in front of it outside production)
redis-cli get "v1:key-name"

# Check key TTL
redis-cli ttl "v1:key-name"

# List what is in the active generation
redis-cli --scan --pattern 'v1:*' | head

# Monitor all commands
redis-cli monitor

FLUSHDB / FLUSHALL are deliberately absent. The database is shared by webapp, temporal-workers and portal, so a flush wipes every service’s cache plus the unversioned atomic keys (JTI replay protection, rate limiter windows, locks). To clear a retired generation, delete it by prefix — see Cost of a bump.

Security

Access Control

Password Protection: All Redis instances use strong passwords
TLS Encryption: Production Redis uses TLS for secure connections
Network Isolation: Redis Cloud provides network-level security
Environment Separation: Development and production use separate instances
Key Prefixing: Non-production environments use REDIS_ENV prefix to prevent cross-contamination

Data Protection

No Sensitive Data: Never cache passwords, tokens, or PII
TTL Enforcement: All cached data has appropriate expiration
Encryption: Sensitive cached data should be encrypted before storage
Version Management: Bumping the shared version rolls cached data to a new generation; stale entries expire via TTL

Cost Optimization

Current Costs

Redis Cloud: ~$10/month for 250MB memory (estimated based on usage)
Performance Gain: Reduces database load, allowing smaller DB instances
ROI: Significant performance improvement for minimal cost

Optimization Strategies

Right-size Memory: Monitor usage and adjust memory allocation
Efficient TTLs: Set appropriate expiration times
Key Compression: Use shorter, efficient key names
Pattern Cleanup: Regular cleanup of unused keys
Version Management: Shared REDIS_CACHE_VERSION bumps clear stale data when payload shapes change (ordinary deploys do not)

Future Enhancements

Planned Features

Cache Warming: Pre-populate cache with frequently accessed data
Distributed Caching: Support for multiple Redis instances
Cache Analytics: Detailed hit/miss ratio tracking
Automatic Scaling: Dynamic memory allocation based on usage
Cache Versioning: Enhanced version management with metadata tracking

Integrations

Temporal Workflows: Cache workflow state and results
API Rate Limiting: Use Redis for rate limiting external APIs
Session Management: Store user sessions in Redis
Real-time Features: Use Redis for pub/sub messaging

​Redis Caching System

​Overview

​Why Redis?

​Performance Benefits

​Use Cases in CONA

​Architecture

​Redis Configuration

​Production Environment

​Development Environment

​Environment Variables

​Cache Version Management

​How It Works

​Version Sources (Priority Order)

​Configuration Examples

​When the Cache Rolls

​Cost of a bump

​Guards against a split generation

​Cache stamps — invalidating without deleting

​Rollout (CONA-1096)

​Implementation

​Import Redis Functions

​Redis Client Initialization

​Core Cache Functions

​Cache Key Conventions

​Bulk Cache Invalidation Functions

​Usage Patterns

​1. Cache-Aside Pattern

​2. Write-Through Pattern

​3. Cache Invalidation Strategies

​Monitoring & Debugging

​Redis Insights Integration

​Redis CLI Access

​Application Logging

​Axiom Observability

​Setup

​redis.call Event Schema

​redis.connection Event Schema

​redis.usage_rollup Event Schema

​redis.cache_flush Event Schema (retired)

​Privacy Guarantees

​Example Axiom Queries

​Best Practices

​1. TTL (Time To Live) Strategy

​2. Cache Key Design

​3. Error Handling

​4. Memory Management

​5. Cache Version Management

​Troubleshooting

​Common Issues

​Debug Commands

​Security

​Access Control

​Data Protection

​Cost Optimization

​Current Costs

​Optimization Strategies

​Future Enhancements

​Planned Features

​Integrations

Redis Caching System

Overview

Why Redis?

Performance Benefits

Use Cases in CONA

Architecture

Redis Configuration

Production Environment

Development Environment

Environment Variables

Cache Version Management

How It Works

Version Sources (Priority Order)

Configuration Examples

When the Cache Rolls

Cost of a bump

Guards against a split generation

Cache stamps — invalidating without deleting

Rollout (CONA-1096)

Implementation

Import Redis Functions

Redis Client Initialization

Core Cache Functions

Cache Key Conventions

Bulk Cache Invalidation Functions

Usage Patterns

1. Cache-Aside Pattern

2. Write-Through Pattern

3. Cache Invalidation Strategies

Monitoring & Debugging

Redis Insights Integration

Redis CLI Access

Application Logging

Axiom Observability

Setup

`redis.call` Event Schema

`redis.connection` Event Schema

`redis.usage_rollup` Event Schema

`redis.cache_flush` Event Schema (retired)

Privacy Guarantees

Example Axiom Queries

Best Practices

1. TTL (Time To Live) Strategy

2. Cache Key Design

3. Error Handling

4. Memory Management

5. Cache Version Management

Troubleshooting

Common Issues

Debug Commands

Security

Access Control

Data Protection

Cost Optimization

Current Costs

Optimization Strategies

Future Enhancements

Planned Features

Integrations