Scaling and Partitioning

One event is overwhelming the system

Return to the ticket marketplace from the previous chapter. The architecture has a purchase boundary that owns holds and confirmed orders, a derived path for browsing, and an outbox for work that may follow the sale. At noon, tickets for a major final go on sale. Event pages receive hundreds of thousands of reads, buyers contend for the same few sections, and the confirmation transaction begins timing out.

“Add more servers” is not yet a design. Which work is saturated?

The browse API may be short of CPU or connections. The event-detail cache may be suffering a stampede. The inventory store may be waiting on locks for the same seats. Payment may be exhausting an outside quota. The outbox may be healthy while downstream email workers fall behind. Each produces slow requests, but each calls for a different response.

The difficult case is inventory. A seat has one owner because two owners could sell it twice. Scaling must relieve concentrated load without dissolving that decision. This is the governing problem of partitioning: scale the bottleneck; partition the ownership.

Find the pressure before choosing machinery

Start from evidence already available in the design. Overall request rate says little when one event dominates the traffic. Break it down by path and by ownership key.

For browsing, inspect request rate, cache hit ratio, origin load, and latency by event. For reservation, inspect transaction latency, lock wait, conflict rate, timeouts, and writes by event and section. For payment, inspect provider latency, connection use, quota rejection, and the number of attempts whose outcome is unknown. For deferred work, inspect queue age rather than blaming the purchase response for a slow email consumer.

Suppose the evidence says:

• event-page reads are overwhelming the cache origin after a popular listing expires;
• most reservation contention belongs to one event, especially its lower bowl;
• database CPU is moderate, but lock wait and transaction latency are rising;
• payment and the outbox remain within their operating ranges.

That evidence rules out several attractive but irrelevant moves. More stateless purchase instances would send more concurrent work toward the same locks. A read replica would help browse traffic, but not the writer that decides who owns a seat. A queue in front of reservation might smooth arrival, but it would also change the interactive contract and still serialize the hot inventory somewhere.

The first relief is therefore two different moves. Shield the derived browse path with request coalescing, longer-lived stale-but-safe event data, and prewarming before the sale. Isolate the hot event’s canonical inventory so it cannot consume the shared store’s connection and lock budget. The first move copies or caches reads. The second changes placement of owned writes.

This is the restraint an interview answer needs. A mechanism earns its place by relieving a named pressure, not by appearing on a scalable-systems diagram.

Replication copies truth; sharding divides ownership

Replication and sharding are often drawn alike—several database cylinders—but they solve different problems.

A replica holds another copy of the same data. It can absorb reads, shorten a regional read path, or provide a failover candidate. If all inventory writes still pass through one primary or consensus group, replicas have not increased the write capacity of the ownership decision. They have also created a freshness question. A stale seat map is acceptable only if reservation returns to canonical inventory and refuses an already-owned seat.

A shard owns a subset of the data. Different shards can accept writes for different events, tenants, accounts, documents, or key ranges. This increases write and storage capacity when work is distributed among those owners. In exchange, routing, cross-shard operations, rebalancing, and recovery become part of the application design.

For the marketplace, an initial partition by event_id is attractive. Seat availability, holds, and most purchases are event-local. Moving one event to dedicated capacity is understandable to operators, and an event is a natural unit for quotas and archival. But the key has an obvious failure: the final is one event, so all of its writes still go to one shard.

Hashing event_id does not repair that problem. A perfectly even hash distribution across millions of ordinary events still assigns the one hot event to one owner. Hashing distributes many keys; it cannot distribute work inside a single key.

A shard key is a product contract

Choosing a shard key means choosing which operations are cheap, which facts remain together, and which failures are isolated. Test a candidate key against five questions:

• Does it have enough distinct values to use the available partitions?
• Does product behavior distribute traffic across those values, including at peak rather than only on average?
• Do the common reads and writes remain local to one owner?
• Can a large tenant, event, or document be isolated without changing the public API?
• Can ownership move later without losing or duplicating writes?

For ordinary events, event_id may pass. For a hot assigned-seating event, the system can add a second level such as event_id + section_id. Each seat belongs to exactly one section, so a hold and sale for that seat remain local. Several sections can accept reservations concurrently, and a particularly hot section can receive dedicated capacity.

The split creates a cost. An order containing seats from several sections now crosses ownership boundaries. The product can avoid that transaction by restricting a hold to one section, coordinate a small multi-shard operation, or introduce an event-level order owner that confirms only after acquiring section-local holds. The right choice depends on whether cross-section baskets are important enough to repay the coordination cost. The shard key has exposed a product decision that hash(id) would have hidden.

General admission has a different shape. There may be no independent seat objects, only a bounded inventory count. Splitting that counter into pools can increase concurrency, but the pools must not collectively issue more tickets than capacity. Capacity can be allocated to section or bucket owners and rebalanced deliberately; an unconstrained set of cached counters cannot defend the invariant.

The same reasoning transfers. Tenant plus series hash may suit metric ingestion because queries are tenant-local and one large tenant needs spread. Account ID may suit a ledger until transfers between accounts dominate. Time buckets help retention but make the newest bucket hot. Range keys preserve locality but invite skew; hashes improve distribution but scatter range scans. There is no good key independent of the operations it must serve.

Hot objects need an exception path

Many partitioning failures come from designing for a smooth distribution and operating in a lumpy world. Celebrity accounts, live matches, tenant imports, shared documents, flash sales, and dashboards refreshed at 9 a.m. all turn one logical object into a disproportionate share of the system.

Name whether the hotspot is read or write pressure. A hot read can often be copied: CDN and cache shielding, request coalescing, precomputation, or stale-while-revalidate can protect the owner. A hot write cannot be copied so casually because multiple writers must still agree on the result. The choices are to subdivide ownership along a real product boundary, buffer work when the contract permits delay, give the object dedicated capacity, narrow the critical section, or apply admission control.

For the final, the design may combine these moves:

browse:
  edge/CDN -> event cache -> derived availability
  stale display is allowed; reserve always rechecks canonical ownership

reserve:
  placement(event_id, section_id) -> section inventory owner
  conditional hold with buyer, seat, version, and expiry

checkout:
  event order owner -> verify section-local holds -> payment path
                    -> confirm or return an explicit pending state

This shape does not pretend that caching solves contention. It moves abundant reads away from the scarce decision, divides writes only where a seat already has a natural owner, and keeps an event-level place to assemble the purchase. Admission control can cap concurrent reservation attempts before lock queues grow without bound. A virtual waiting room may be justified at this scale, but its fairness, token expiry, bypass protection, and user-visible delay are now part of the product, not invisible infrastructure.

Fan-out moves pressure downstream

One incoming action can create much more than one unit of work. Publishing a post may update millions of inboxes. Changing a permission may invalidate many cached views. Confirming an event sale may update a seat map, issue tickets, notify buyer and seller, feed analytics, and refresh recommendations.

A queue can protect the request path from this multiplier when those effects may lag. It cannot make the multiplier disappear. Partition consumers by a key that preserves required ordering, monitor the age of the oldest work, make effects idempotent, and decide what happens when one hot key monopolizes a partition. Adding consumers achieves nothing if every message for the final must remain on one queue partition and one consumer cannot keep up.

Some products need a hybrid. A social feed can precompute inboxes for ordinary authors but merge posts from enormous accounts at read time. The exception path prevents one celebrity post from becoming millions of synchronous or immediate writes. For ticketing, public availability can be a periodically refreshed derived view while the much smaller stream of successful holds and sales updates canonical inventory. In both cases, the design states which view may lag and where truth is rechecked.

Routing is part of correctness

Once ownership is divided, every request needs the same answer to “where does this key live now?” That answer may come from an explicit placement directory: an event and section map to a shard, with a version attached so routers can detect stale placement. This model suits the marketplace because operators may want to isolate a named event, keep an event in a required region, or move a large customer deliberately.

Consistent hashing is useful for a different placement problem. Keys map into a token space, and adding or removing a node moves only nearby token ranges rather than remapping every key. Virtual nodes or weighted tokens help spread ranges and account for unequal capacity. It works well when there are many independent keys, membership changes, and key-value access matters more than human-controlled locality.

The ring is not the operating system. The design still needs an authority for membership, replication rules, failure detection, overload signals, and a way to transfer and verify data. Nor does consistent hashing cure a hot logical key; all requests for that key still find the same token owner unless the object itself is partitioned.

In an interview, choose between these models rather than naming both by habit. Explicit tenant, event, range, or regional placement favors control and locality. Consistent hashing favors elastic distribution of many independent keys. Some systems combine them—for example, a placement directory assigns a tenant to a pool whose nodes use token ranges—but each layer must have a clear reason.

Rebalancing is the moment the design becomes real

A partitioned system must be able to change its mind. The final was assigned to shared capacity yesterday; today it needs four section owners. A tenant upgrades to dedicated capacity. A shard fills faster than expected. A region must be evacuated. “Add another shard” leaves out the dangerous part.

Trace one move through time. For a section moving from shard A to shard B:

1. The placement authority creates a migration version while A remains the owner of writes.
2. A snapshot or bounded backfill copies seats, holds, versions, and expiry state to B. The copy is throttled so migration does not become the outage.
3. Changes after the snapshot are forwarded or replayed from a durable log.
4. Counts, checksums, versions, and application invariants are compared. Test reads may go to B, but user decisions still follow the declared owner.
5. Routing changes at a clear cutover point. Stale routers are rejected or redirected rather than allowed to write to A indefinitely.
6. A remains available for rollback or repair until B has served correctly through the verification window.

Dual writes are sometimes proposed as a shortcut. They create an ambiguous state whenever one write succeeds and the other fails. If they are used, one owner must remain authoritative and the second path needs replay and reconciliation. A migration log or write forwarding often makes that authority easier to see.

Observe the move with per-partition growth, copy lag, forwarded-write lag, routing-version errors, hold conflicts, transaction latency, and user-visible failures. Rebalancing is not housekeeping after the architecture; it is how the ownership model survives growth.

Earn the second region

Multi-region deployment may be required for latency, data residency, market isolation, or survival of a regional failure. It also multiplies the routing and ownership questions. Putting replicas everywhere does not decide which region may accept a write during a partition.

Ticket browsing is easy to place near readers because it is derived and may lag. Scarce inventory is harder. A defensible first design gives each event a home region that owns holds and sales. Buyers elsewhere pay the write round-trip, while nearby caches serve event details. If the home region fails, the system can fail closed for new sales or promote a sufficiently current replica through a controlled procedure. The latter needs an explicit recovery-point expectation and fencing so the old owner cannot resume writes after promotion.

Active-active writes by partition are possible when different regions own different keys—for example, tenants with fixed home regions. They become much harder when one operation spans regions or when two regions may sell the same scarce item. Conflict resolution after two successful sales is not a useful inventory policy.

State the failure behavior, not just the happy placement:

The event's home region owns reservation writes. Global edges serve cached
event data and route reserve requests home. If that region is unavailable, we
stop new reservations until a fenced promotion completes; we do not accept
conflicting sales in two regions. Search and analytics may continue from
derived copies and show their freshness.

That answer may sound less “global” than active-active, but it preserves the promise that made inventory canonical in the first place.

Narrate the decision, then invite pressure

A scaling answer becomes easy to follow when its sentences preserve causality:

The pressure is lock contention on inventory for one event, not database CPU
across the catalog. I would shield browse reads with a cache and derived view,
then place the hot event on dedicated capacity and partition its assigned seats
by section. A seat still has one owner. The new risks are cross-section orders,
stale routing, and moving a section under load, so I would keep an event-level
order owner, version the placement map, and migrate with one authoritative
writer plus replay. I would not add multi-region inventory writers: the event
has one home region, and conflicting sales are worse than temporary write
unavailability.

An interviewer can now change one assumption. If browse latency is the only problem, remove the write partitioning. If the event uses general admission, replace seat-local ownership with bounded capacity pools. If regulations pin customers to regions, make the placement directory regional. If buyers often combine sections, revisit the key rather than hiding cross-shard coordination.

When a discussion becomes a list of technologies, recover with three questions:

• What resource or ownership boundary is actually under pressure?
• Which operation becomes cheaper after this move, and which becomes harder?
• How will routing and data change when today’s placement stops fitting?

Practice the change of mind

Take one design you already know and give it a specific skew: one tenant produces 60 percent of writes, one document receives 50,000 concurrent edits, or one event sells out in three minutes. Name the first saturated resource and one measurement that would distinguish it from a nearby bottleneck.

Then propose a shard key and try to break it. Ask which common operation crosses shards, whether one logical key remains hot, how a new owner receives live writes during migration, and what a stale router does. Finally remove one assumption—perhaps write volume fits one primary after all—and say which machinery you would now decline.

For a harder variation, move the product into two regions. Decide what each region may write while the link is down. If the answer is “both,” identify the conflict and show why the merge preserves the product invariant. If it cannot, choose an owner or accept bounded unavailability.

The goal is not to produce the most distributed architecture. It is to make placement follow evidence and to keep the system’s meaning visible while ownership moves.

Field reference

PRESSURE
  Name the saturated resource, skew, and proof signal.

RELIEF
  Replicate or cache reads; queue delay-tolerant work; shard owned writes;
  isolate a justified boundary; place regionally for a named requirement.

OWNERSHIP
  Choose the key from product operations. Test cardinality, distribution,
  locality, isolation, and mobility. Give hot objects an exception path.

ROUTING
  Declare the placement authority, route version, stale-router behavior,
  and whether explicit placement or consistent hashing fits the workload.

CHANGE
  Keep one authoritative writer during backfill and replay. Verify, cut over,
  observe, and retain rollback or repair until the new owner is proven.

BOUNDARIES
  State cross-shard work, fan-out, freshness, regional write ownership,
  failure behavior, and cost.

Scaling is complete when the bottleneck has evidence, the relief matches it, and every new partition still has a defensible owner and a way to move. The next question is what happens when requests repeat, replicas lag, or a workflow crosses those owners. Consistency, Idempotency, and Transactions takes up that correctness boundary. Use Back-of-the-Envelope Estimation to quantify the pressure and Distributed Systems Fundamentals for the failure and consistency vocabulary beneath the design.