Hot keys, stampedes, and three fixes that don't transfer.

00Three problems, not one

01Redis atomicity, precisely

Redis can use extra threads for networking and parsing, but the useful model is that dataset command execution on one node is serialised. That is why a single Redis command is inherently atomic from the application's point of view, and why Redis gets real data parallelism from multiple nodes rather than multiple threads.

SET lock:key token-A NX PX 5000   → SUCCESS
SET lock:key token-B NX PX 5000   → FAIL      (NX: only if absent)
SET lock:key token-C NX PX 5000   → FAIL

existence check + set + TTL are ONE command.
EXISTS followed by SET is two, with a race between them.

When one logical operation needs several commands to behave as one unit, that is what Lua is for: while a script runs on a node, no other client command executes in the middle of it.

Where requests wait while Redis is busy

client
  → OS TCP receive buffer      raw bytes can wait here
  → Redis client query buffer  bytes Redis has read can wait here
  → event-loop readiness       which sockets need attention
  → command execution
  → output buffer → client

observed latency ≈ wait/scheduling + execution + network

There is no single global FIFO queue. This is also why a long Lua script makes cheap GETs look slow — the GET is still cheap, it just waited behind the script.

02Cache stampede

Two levels, because either one alone leaves the other multiplier intact. Per-pod coalescing turns 10,000 concurrent local requests into one leader and 9,999 waiters on the same promise. The distributed lock then turns ~100 pod leaders into one rebuild.

03Locks, TTLs and why the token matters

The lock key is not the cached data. feed:user:123 is the value; lock:feed:user:123 only says who is currently allowed to rebuild it. The winner should release the lock as soon as it has populated the cache — the TTL is the crash-safety fallback, not the normal path.

-- never do this
DEL lock:key

-- because if your lease expired, you may be deleting
-- the NEW owner's lock

-- do this, atomically, in Lua
if redis.call("GET", KEYS[1]) == ARGV[1] then
  return redis.call("DEL", KEYS[1])
end
return 0

GET then DEL from application code is two commands with a race between them. Inside one small Lua script it is one atomic region.

04Dragonfly's different model

Dragonfly partitions its keyspace into shards with a dedicated thread owning each, so different keys really do execute in parallel. The cost is the complexity that multi-key commands now need coordination across shard-threads — which is exactly the complexity Redis avoids by serialising.

05Read-hot keys

One million GETs/sec against a key that already exists. There is no rebuild to deduplicate here, so coalescing is the wrong tool — and putting SET NX PX in front of every read would turn a hot data key into a hot lock key, which is strictly worse because a lock is a write.

06Write-hot keys

You cannot collapse a million likes into one increment. That is the whole distinction, and it is the first thing to say when someone describes a “hot key” without specifying which kind.

07LLD — likes for a viral post

Why Kafka, and not just in-memory batching

Pod-local batching is fast and loses data: a crash before flush drops those likes with no record they existed. Kafka makes the events durable first, and batching becomes an optimisation on top of durability rather than a substitute for it.

The idempotency problem, and the only clean fix

unsafe:
  mark IN_PROGRESS → INCR → crash
  on retry, IN_PROGRESS cannot tell you whether INCR happened

safe — one Lua script, one atomic region:
  if offset already applied:  return DUPLICATE
  INCR video:123:likes:7
  record offset as applied
  return APPLIED

then commit the Kafka offset

Aggregation, and why SET beats INCRBY

absolute (retry-safe)          delta (not retry-safe)
  sum buckets = 100500           delta = +500
  SET total 100500               INCRBY total 500
  retry → SET total 100500       crash, retry → INCRBY total 500
  still 100500                   now 101000  ✗

One more edge: two aggregators can read different snapshots, and an older one must not overwrite a newer total. Either keep one owner per video, or attach a monotonically increasing version and only write if yours is newer.

08Question drill

09Cheat sheet