The Two Hard Things in Computer Science
"There are only two hard things in Computer Science: cache invalidation and naming things." — Phil Karlton
Caching is deceptively simple in concept — store frequently accessed data closer to where it is needed. In practice, it introduces subtle bugs, stale data, consistency issues, and failure modes that can be harder to debug than the performance problems they solve.
This article covers caching patterns, invalidation strategies, distributed caching challenges, and a detailed comparison of Redis and Memcached. By the end, you will understand not just how to cache, but when and where caching helps — and when it makes things worse.
Why Cache?
Without cache:
┌────────┐ ┌────────┐ ┌────────┐
│ Client │────▶│ App │────▶│ DB │ 50-200ms per query
└────────┘ └────────┘ └────────┘
With cache:
┌────────┐ ┌────────┐ ┌────────┐
│ Client │────▶│ App │────▶│ Cache │ 1-5ms (cache hit)
└────────┘ └────────┘ └───┬────┘
│ cache miss
┌───▼────┐
│ DB │ 50-200ms (only on miss)
└────────┘
Caching reduces latency, decreases database load, and improves throughput. A well-tuned cache with a 95% hit rate means only 5% of requests hit your database. That is a 20x reduction in database load.
Cache Layers in a Typical System
┌───────────────────────────────────────────────────────┐
│ Client │
│ ┌──────────────────────────────────────────────────┐ │
│ │ Browser Cache (HTTP cache headers) │ │
│ └──────────────────────────────────────────────────┘ │
└───────────────────────┬───────────────────────────────┘
│
┌───────────────────────▼───────────────────────────────┐
│ CDN Cache │
│ Static assets, API responses (Cloudflare, CloudFront) │
└───────────────────────┬───────────────────────────────┘
│
┌───────────────────────▼───────────────────────────────┐
│ Application Server │
│ ┌──────────────────────────────────────────────────┐ │
│ │ In-Memory Cache (process-level, e.g. LRU map) │ │
│ └──────────────────────────────────────────────────┘ │
│ ┌──────────────────────────────────────────────────┐ │
│ │ Distributed Cache (Redis / Memcached) │ │
│ └──────────────────────────────────────────────────┘ │
└───────────────────────┬───────────────────────────────┘
│
┌───────────────────────▼───────────────────────────────┐
│ Database │
│ ┌──────────────────────────────────────────────────┐ │
│ │ Query Cache (MySQL) / Shared Buffers (Postgres) │ │
│ └──────────────────────────────────────────────────┘ │
└───────────────────────────────────────────────────────┘
Caching Patterns
Cache-Aside (Lazy Loading)
The application manages the cache explicitly. On a read, check the cache first. On a miss, load from the database and populate the cache. The cache never talks to the database directly.
Cache Hit:
Client ──▶ App ──▶ Cache ──▶ (data found) ──▶ Return to Client
Cache Miss:
Client ──▶ App ──▶ Cache ──▶ (miss)
│
▼
App ──▶ DB ──▶ (data loaded)
│
▼
App ──▶ Cache (store for next time)
│
▼
Return to Client
import Redis from 'ioredis';
const redis = new Redis();
const CACHE_TTL = 3600; // 1 hour
async function getUserProfile(userId: string) {
const cacheKey = `user:${userId}:profile`;
// Step 1: Check cache
const cached = await redis.get(cacheKey);
if (cached) {
return JSON.parse(cached); // Cache hit
}
// Step 2: Cache miss — load from database
const user = await db.users.findById(userId);
if (!user) return null;
// Step 3: Populate cache for next time
await redis.setex(cacheKey, CACHE_TTL, JSON.stringify(user));
return user;
}
// On update: invalidate cache
async function updateUserProfile(userId: string, data: UserUpdate) {
await db.users.update(userId, data);
await redis.del(`user:${userId}:profile`); // Invalidate
}
Pros: Only caches data that is actually requested. Simple to implement. Cache failures are non-fatal (app falls back to DB).
Cons: First request is always slow (cache miss). Stale data possible if cache is not invalidated on writes.
Read-Through
Similar to cache-aside, but the cache itself is responsible for loading data from the database on a miss. The application only talks to the cache.
// Read-through cache abstraction
class ReadThroughCache {
constructor(
private redis: Redis,
private loader: (key: string) => Promise<unknown>,
private ttl: number = 3600,
) {}
async get(key: string) {
const cached = await this.redis.get(key);
if (cached) return JSON.parse(cached);
// Cache loads from source automatically
const data = await this.loader(key);
if (data) {
await this.redis.setex(key, this.ttl, JSON.stringify(data));
}
return data;
}
}
// Usage
const userCache = new ReadThroughCache(
redis,
(key) => {
const userId = key.replace('user:', '');
return db.users.findById(userId);
},
3600,
);
const user = await userCache.get(`user:${userId}`);
Write-Through
Every write goes to the cache AND the database synchronously. The cache is always up-to-date.
Write-Through:
Client ──▶ App ──▶ Cache ──▶ DB
│
▼
(both updated synchronously)
│
▼
Return to Client
async function updateProduct(productId: string, data: ProductUpdate) {
const cacheKey = `product:${productId}`;
// Write to database first
const updated = await db.products.update(productId, data);
// Then update cache (synchronously)
await redis.setex(cacheKey, CACHE_TTL, JSON.stringify(updated));
return updated;
}
Pros: Cache is always consistent with DB. Reads are always fast after the first write.
Cons: Higher write latency (must write to both). Caches data that may never be read.
Write-Behind (Write-Back)
Writes go to the cache immediately and are asynchronously flushed to the database in batches. The client gets a fast response, and the database is updated eventually.
Write-Behind:
Client ──▶ App ──▶ Cache ──▶ Return to Client (fast)
│
▼ (async, batched)
DB
// Write-behind with batched database writes
class WriteBehindCache {
private writeBuffer: Map<string, unknown> = new Map();
private flushInterval: NodeJS.Timeout;
constructor(private redis: Redis, private db: Database) {
// Flush every 5 seconds
this.flushInterval = setInterval(() => this.flush(), 5000);
}
async write(key: string, value: unknown) {
// Write to cache immediately
await this.redis.setex(key, 3600, JSON.stringify(value));
// Buffer for async DB write
this.writeBuffer.set(key, value);
}
private async flush() {
if (this.writeBuffer.size === 0) return;
const batch = new Map(this.writeBuffer);
this.writeBuffer.clear();
try {
// Batch write to database
await this.db.batchUpsert(Array.from(batch.entries()));
} catch (error) {
// On failure, put items back in buffer for retry
for (const [key, value] of batch) {
this.writeBuffer.set(key, value);
}
logger.error('Write-behind flush failed', { error, batchSize: batch.size });
}
}
destroy() {
clearInterval(this.flushInterval);
this.flush(); // Final flush
}
}
Pros: Very fast writes. Batching reduces database load. Good for high-throughput writes (analytics, counters, activity logs).
Cons: Data loss risk if cache node crashes before flush. Eventually consistent. More complex error handling.
Cache Invalidation Strategies
Time-to-Live (TTL)
The simplest strategy. Cached data expires after a fixed time period.
// Simple TTL
await redis.setex('user:123', 3600, JSON.stringify(user)); // Expires in 1 hour
// Different TTLs for different data types
const TTL = {
userProfile: 3600, // 1 hour — changes infrequently
productListing: 300, // 5 minutes — moderate change rate
stockPrice: 5, // 5 seconds — changes rapidly
featureFlags: 60, // 1 minute — needs reasonably current data
};
await redis.setex(`product:${id}`, TTL.productListing, JSON.stringify(product));
Event-Based Invalidation
Invalidate cache entries when the underlying data changes.
// Using application events for cache invalidation
class CacheInvalidator {
constructor(private redis: Redis) {}
// Called after any user update
async onUserUpdated(userId: string) {
const keys = [
`user:${userId}:profile`,
`user:${userId}:settings`,
`user:${userId}:permissions`,
];
await redis.del(...keys);
}
// Called after product update — invalidate product and related caches
async onProductUpdated(productId: string, categoryId: string) {
await redis.del(`product:${productId}`);
await redis.del(`category:${categoryId}:products`);
await redis.del('homepage:featured-products');
// Invalidate using pattern (expensive, use sparingly)
const keys = await redis.keys(`search:*:${categoryId}:*`);
if (keys.length > 0) {
await redis.del(...keys);
}
}
}
Versioned Keys
Instead of deleting cache entries, change the key so old entries are naturally orphaned and eventually evicted.
// Version-based invalidation
async function getProductCatalog(categoryId: string) {
// Version increments when catalog changes
const version = await redis.get(`catalog:${categoryId}:version`) || '1';
const cacheKey = `catalog:${categoryId}:v${version}`;
const cached = await redis.get(cacheKey);
if (cached) return JSON.parse(cached);
const products = await db.products.findByCategory(categoryId);
await redis.setex(cacheKey, 86400, JSON.stringify(products));
return products;
}
// Invalidate by incrementing version
async function onCatalogUpdated(categoryId: string) {
await redis.incr(`catalog:${categoryId}:version`);
// Old versioned keys will expire naturally via TTL
}
LRU Eviction
When cache memory is full, the Least Recently Used (LRU) policy evicts entries that have not been accessed recently.
Redis maxmemory-policy options:
noeviction — Return error when memory limit reached
allkeys-lru — Evict least recently used keys (most common)
volatile-lru — Evict LRU keys that have TTL set
allkeys-lfu — Evict least frequently used keys
volatile-lfu — Evict LFU keys that have TTL set
allkeys-random — Evict random keys
volatile-random — Evict random keys that have TTL set
volatile-ttl — Evict keys with shortest TTL first
# redis.conf
maxmemory 2gb
maxmemory-policy allkeys-lru
Cache Stampede Prevention
A cache stampede (also called thundering herd) happens when a popular cache key expires and hundreds of concurrent requests simultaneously try to rebuild it, overwhelming the database.
Normal operation:
1000 requests/sec ──▶ Cache (hit) ──▶ Response
Cache key expires:
1000 requests/sec ──▶ Cache (all miss simultaneously)
│
▼
1000 simultaneous DB queries
│
▼
Database crashes
Solution 1: Locking (Mutex)
Only one request rebuilds the cache. Others wait or return stale data.
async function getWithLock(key: string, fetchFn: () => Promise<unknown>) {
// Try cache first
const cached = await redis.get(key);
if (cached) return JSON.parse(cached);
const lockKey = `lock:${key}`;
const lockAcquired = await redis.set(lockKey, '1', 'NX', 'EX', 10);
if (lockAcquired) {
try {
// This request rebuilds the cache
const data = await fetchFn();
await redis.setex(key, 3600, JSON.stringify(data));
return data;
} finally {
await redis.del(lockKey);
}
}
// Lock not acquired — wait and retry
await sleep(100);
const retryResult = await redis.get(key);
if (retryResult) return JSON.parse(retryResult);
// Fallback: hit DB directly (rare case)
return fetchFn();
}
Solution 2: Early Expiration (Probabilistic)
Refresh the cache before it actually expires. Each request has a small chance of triggering a refresh when the TTL is low.
async function getWithEarlyRefresh(
key: string,
fetchFn: () => Promise<unknown>,
ttl: number = 3600,
) {
const raw = await redis.get(key);
if (!raw) {
// True cache miss
const data = await fetchFn();
await redis.setex(key, ttl, JSON.stringify(data));
return data;
}
const remainingTTL = await redis.ttl(key);
const shouldRefresh = remainingTTL < ttl * 0.1; // Last 10% of TTL
if (shouldRefresh && Math.random() < 0.1) {
// 10% chance to refresh in background (non-blocking)
fetchFn().then(async (data) => {
await redis.setex(key, ttl, JSON.stringify(data));
}).catch((err) => logger.error('Background refresh failed', { key, err }));
}
return JSON.parse(raw);
}
Solution 3: Stale-While-Revalidate
Always return cached data (even if stale) while refreshing in the background.
async function getStaleWhileRevalidate(
key: string,
fetchFn: () => Promise<unknown>,
) {
const dataKey = `data:${key}`;
const staleKey = `stale:${key}`;
// Check fresh cache
const fresh = await redis.get(dataKey);
if (fresh) return JSON.parse(fresh);
// Check stale cache (longer TTL)
const stale = await redis.get(staleKey);
// Trigger background refresh
fetchFn().then(async (data) => {
const serialized = JSON.stringify(data);
await redis.setex(dataKey, 300, serialized); // Fresh: 5 min
await redis.setex(staleKey, 86400, serialized); // Stale: 24 hours
}).catch((err) => logger.error('Revalidation failed', { key, err }));
// Return stale data immediately (or fetch synchronously if no stale data)
if (stale) return JSON.parse(stale);
// No stale data either — must wait for fetch
const data = await fetchFn();
const serialized = JSON.stringify(data);
await redis.setex(dataKey, 300, serialized);
await redis.setex(staleKey, 86400, serialized);
return data;
}
Distributed Caching Challenges
Cache Consistency Across Nodes
When running multiple application servers, each with local in-memory caches, you get inconsistency.
Server A updates user, invalidates its local cache.
Server B still has the old cached data.
Server A: user = {name: "Alice Smith"} (updated)
Server B: user = {name: "Alice Jones"} (stale)
Solution: Use a centralized distributed cache (Redis) instead of local caches for mutable data. Or use pub/sub to broadcast invalidation.
// Redis pub/sub for cache invalidation across servers
const subscriber = new Redis();
const publisher = new Redis();
// Each server subscribes to invalidation channel
subscriber.subscribe('cache:invalidate');
subscriber.on('message', (channel, message) => {
const { key } = JSON.parse(message);
localCache.delete(key); // Clear local cache
});
// When data changes, publish invalidation
async function invalidateCache(key: string) {
await redis.del(key); // Clear distributed cache
publisher.publish('cache:invalidate', JSON.stringify({ key })); // Clear all local caches
}
Hot Key Problem
A single cache key receiving disproportionate traffic can overwhelm one Redis node.
// Solution: replicate hot keys across multiple Redis slots
async function getHotKey(baseKey: string) {
// Distribute reads across replicas
const replica = Math.floor(Math.random() * 3);
const key = `${baseKey}:replica:${replica}`;
const cached = await redis.get(key);
if (cached) return JSON.parse(cached);
const data = await fetchFromDB(baseKey);
// Write to all replicas
const pipeline = redis.pipeline();
for (let i = 0; i < 3; i++) {
pipeline.setex(`${baseKey}:replica:${i}`, 300, JSON.stringify(data));
}
await pipeline.exec();
return data;
}
Redis vs Memcached
| Feature | Redis | Memcached |
|---|---|---|
| Data structures | Strings, hashes, lists, sets, sorted sets, streams, bitmaps | Strings only |
| Persistence | RDB snapshots + AOF log | None (pure cache) |
| Replication | Built-in master-replica | None (use consistent hashing) |
| Clustering | Redis Cluster (automatic sharding) | Client-side sharding |
| Pub/Sub | Yes | No |
| Lua scripting | Yes | No |
| Max value size | 512 MB | 1 MB |
| Threading | Single-threaded (I/O threads since 6.0) | Multi-threaded |
| Memory efficiency | Higher overhead per key | More memory efficient for simple strings |
| Use case | Feature-rich caching, queues, sessions, leaderboards | Simple high-throughput string caching |
When to Choose Redis
Use Redis when you need more than simple key-value caching.
// Leaderboard using Redis sorted set
await redis.zadd('leaderboard:daily', score, `user:${userId}`);
const top10 = await redis.zrevrange('leaderboard:daily', 0, 9, 'WITHSCORES');
// Rate limiting using Redis
async function rateLimit(userId: string, limit: number, windowSec: number) {
const key = `ratelimit:${userId}`;
const current = await redis.incr(key);
if (current === 1) {
await redis.expire(key, windowSec);
}
return current <= limit;
}
// Session storage using Redis hash
await redis.hset(`session:${sessionId}`, {
userId: '789',
role: 'admin',
loginAt: Date.now().toString(),
});
await redis.expire(`session:${sessionId}`, 86400);
// Distributed lock using Redis
const acquired = await redis.set(
`lock:${resource}`,
lockId,
'NX',
'EX',
30,
);
When to Choose Memcached
Use Memcached when you have a simple caching workload at very high scale and need maximum memory efficiency.
import Memcached from 'memcached';
const memcached = new Memcached('cache-server:11211');
// Simple get/set — that is all Memcached does
memcached.set('user:123', JSON.stringify(user), 3600, (err) => {
if (err) logger.error('Cache set failed', { err });
});
memcached.get('user:123', (err, data) => {
if (err) return fallbackToDb();
if (data) return JSON.parse(data);
return fallbackToDb();
});
Monitoring Cache Performance
The most important metric is cache hit rate. Below 80%, your cache is not providing much value. Above 95% is excellent.
// Instrument cache operations
class InstrumentedCache {
private hits = 0;
private misses = 0;
constructor(private redis: Redis) {}
async get(key: string): Promise<unknown | null> {
const start = Date.now();
const result = await this.redis.get(key);
const duration = Date.now() - start;
if (result) {
this.hits++;
metrics.histogram('cache.get.duration', duration, { result: 'hit' });
} else {
this.misses++;
metrics.histogram('cache.get.duration', duration, { result: 'miss' });
}
metrics.gauge('cache.hit_rate', this.hitRate);
return result ? JSON.parse(result) : null;
}
get hitRate(): number {
const total = this.hits + this.misses;
return total === 0 ? 0 : this.hits / total;
}
}
# Redis CLI monitoring commands
redis-cli INFO stats
# keyspace_hits:1234567
# keyspace_misses:12345
# Hit rate: 1234567 / (1234567 + 12345) = 99.0%
redis-cli INFO memory
# used_memory_human:1.5G
# maxmemory_human:2G
# mem_fragmentation_ratio:1.03
redis-cli --latency
# min: 0, max: 1, avg: 0.23 (milliseconds)
Key Metrics to Monitor
Cache hit rate — Target: > 90%
Cache miss rate — Inverse of hit rate
Eviction rate — Keys evicted due to memory pressure
Memory usage — Current vs max memory
Latency (p50, p99) — Response time distribution
Connection count — Active client connections
Key count — Total keys stored
TTL distribution — How keys are distributed by expiration
Key Takeaways
- Cache-aside is the most common and safest pattern. Start here unless you have specific requirements for write-through or write-behind.
- TTL is your safety net. Always set a TTL, even if you have event-based invalidation. It prevents permanent stale data if an invalidation event is lost.
- Cache stampedes will happen in production. Implement at least one prevention strategy (locking, early refresh, or stale-while-revalidate).
- Redis is the default choice for most applications. Choose Memcached only when you need maximum memory efficiency for simple string caching at massive scale.
- Monitor cache hit rate relentlessly. A cache with a 60% hit rate is just adding complexity without enough benefit.
- Cache invalidation is the hard part. Design your invalidation strategy before your caching strategy.
- Not everything should be cached. Cache data that is read frequently, changes infrequently, and is expensive to compute or fetch.