Senior Engineering Interview Handbook / Chapter 94
Real-Time and Location Systems
A sustained real-time system-design case that develops freshness contracts, location validation, geospatial dispatch, expiring offers, current state, event history, authorized fan-out, repair, and adaptations for tracking, multiplayer, collaborative presence, and bidding.
Page tools
The newest message contains the older fact
18:00:00 driver location sequence=918 near the airport
18:00:04 rider request dispatch offers that driver the trip
18:00:07 driver location sequence=917 across the junction
18:00:12 offer deadline
The last message is newest by arrival and oldest by sequence. It waited in a mobile network queue until after dispatch had used sequence 918.
Should that coordinate move the driver backward across the map? Does it alter the pending offer? If the driver accepts at the deadline, whose clock decides? And while the phone loses connectivity again, what should the rider see?
“Real time” does not answer any of those questions. A transport can deliver messages quickly and still produce a dishonest product. The design needs to say what each participant may believe, how old or inaccurate that belief may be, who can create the authoritative fact, and what happens when freshness fails. That is the freshness contract.
Put the states on different rungs
At the first rung, an eventual update may be enough. A partner’s delivery event can arrive late if the system retains history, converges, and tells the user what it actually knows. Bounded delay fits a courier position or ETA that should be recent within a stated window. A live stream fits cursors or a moving map while viewers are connected. An authoritative tick or deadline is appropriate when the server must order contested actions such as game commands or bids.
These are product promises, not a ranking of technologies. One ride product can use a live stream for the map, bounded freshness for driver coordinates, an expiring server-owned offer, and durable trip history. A bidding product can stream the visible high bid while deciding acceptance at an authoritative deadline. Raising every state to the highest rung would add coordination and cost without making every user experience better.
The useful clarifying questions therefore concern consequences before mechanisms:
- Which state must be fresh: position, availability, ETA, command, bid, score, cursor, selection, or durable status?
- Who emits the input, and who is allowed to turn it into truth?
- What can go wrong if a view is one second, ten seconds, or one minute old?
- How many producers and authorized watchers share a room, trip, auction, or geographic cell?
- What must a disconnected client recover: the latest snapshot, missed events, a durable outcome, or merely current presence?
- Which facts are sensitive, abuse-prone, safety-related, or financially consequential?
Those answers determine whether polling is enough, whether a connection should remain open, where ordering is required, and how much history the system must retain.
Fix the scale and the promises
For the ride case, assume one metropolitan region with 120,000 drivers online at peak and 30,000 active trips. If each online driver sends a point every four seconds, ingestion receives roughly 30,000 location updates per second before retries. If each active trip has two live viewers, one forwarded update per four seconds produces roughly 15,000 downstream deliveries per second. Bursts, support viewers, reconnects, and dense events will push those numbers higher. They are interview assumptions, not claims about a particular company.
The first version supports four user actions: drivers publish location and availability; riders request trips; dispatch offers one trip to candidate drivers under an explicit expiry policy; and authorized participants watch an accepted trip. Payment, route optimization, and pooling can wait unless the interviewer makes one of them central.
The non-functional promises are unequal by design. A rider request is durable and idempotent. An offer has a short server-owned lifetime. Assignment is an authoritative state transition. A driver location is a recent, validated claim, not exact truth. The rider map is smooth enough to be useful and visibly stale when its evidence ages. Location history is retained only for the product, safety, support, fraud, and policy needs established in the prompt.
A small API sketch makes the ownership visible:
POST /drivers/{driver_id}/locations
{session_id, sequence, device_time, latitude, longitude, accuracy_m}
POST /trip-requests
Idempotency-Key: tripreq_72
{rider_id, pickup, destination, product}
POST /offers/{offer_id}/accept
Idempotency-Key: accept_72
GET /trips/{trip_id}/snapshot
-> {version, trip_state, last_location, location_age_ms, confidence}
SUBSCRIBE /trips/{trip_id}/updates?after_version=418
The connection endpoint is deliberately last. It delivers a contract defined elsewhere; it does not own trip truth.
Treat a coordinate as a claim
The driver phone knows what its sensors reported. It does not know whether the marketplace should trust the report, whether the driver remains available, or whether a trip has been assigned. Location ingestion authenticates the session, checks authorization and rate limits, compares sequence and time evidence, and applies plausible-motion and accuracy rules before updating serving state.
Device time is useful evidence but a poor deadline authority. Phones drift, users change clocks, offline clients upload batches, and networks reorder messages. A monotonic sequence within one session can expose duplicates and older samples; server receive time establishes what the backend observed. When a phone restarts and loses its sequence, the new session needs an explicit boundary rather than pretending its counter continues the old one.
The current driver record might contain this compact state:
driver_state
driver_id, session_id, sequence
point, accuracy, device_time, received_at
confidence, availability, active_offer_id, active_trip_id
version
The latest validated point feeds a geospatial serving index. Grid cells, geohashes, S2 cells, or another spatial structure can all support nearby search; the important design questions are ownership, borders, and density. Search must inspect neighboring cells, then filter candidates by actual distance or ETA, availability, vehicle eligibility, and marketplace policy. A stadium emptying after an event may turn one cell into a hot partition, so dense regions may need smaller cells, subdivided ownership, or regional workers that spread one area’s load.
The index remains a fallible projection. It can propose a driver whose location is fresh enough for discovery. It cannot commit the driver to a trip.
Let the expiring offer decide the assignment
At 18:00:04, the rider request creates tripreq_72. A uniqueness guard binds
that idempotency key to the request details, so a browser retry returns the same
attempt rather than creating another race for drivers. Dispatch queries the
nearby-driver index, rechecks the chosen driver’s authoritative availability,
and creates offer off_31 with a server deadline of 18:00:12.
At 18:00:07, delayed location sequence 917 arrives. Ingestion records enough
evidence to diagnose the delay but refuses to replace sequence 918 in current
state. This avoids moving the visible driver backward and avoids corrupting the
next candidate search. It does not cancel off_31: offer state belongs to the
dispatch workflow, not to the location index.
At 18:00:10, the driver accepts. The assignment transition checks all of the facts that must agree at commitment: the offer is still pending under server time, it belongs to this driver and request, the driver has no accepted trip, and the request is still unmatched. One guarded transaction or one owner for that partition changes the offer to accepted, marks the driver unavailable, and creates the trip. A duplicate acceptance returns the same trip. A competing offer, cancellation, or expiry can win instead, but only one transition may.
The durable records separate temporary proposals from business truth:
trip_request: requested -> matching -> assigned | canceled | no_match
offer: pending -> accepted | declined | expired | withdrawn
trip: assigned -> arriving -> in_progress -> completed | canceled
The location update rate is high; the number of trip transitions is much lower. They do not need identical storage. Current location serves dispatch and maps. Trip and offer events explain who decided what, in which order, and why. A durable history supports reconnect, support, fraud review, analytics, and repair without forcing every live read to scan an event log.
Give the rider an honest map
After assignment, a fan-out gateway admits only the rider, driver, and authorized support tools to the trip room. It can coalesce points, attach a version, and send deltas while the connection is healthy. On reconnect, the client presents its last version. The gateway may replay a short retained gap; when that gap is unavailable or unsafe to apply, it sends a fresh snapshot and continues after the snapshot version.
Interpolation can make motion look smooth, but it must not manufacture confidence. When the latest validated point ages beyond the product’s threshold, the UI should fade or stop the pin, show when it last updated, and recompute the ETA only when the evidence justifies it. “Driver location is updating slowly” is a better product state than a precise-looking fiction.
This separation also changes failure handling. If fan-out fails, dispatch can continue while the map falls back to snapshots or polling. If the geospatial index is rebuilding, active trips can still read their last current point while new matching narrows or pauses in the affected area. If the trip owner is unavailable, the system must not accept two drivers merely to preserve a fast response. Degradation follows the promise at risk.
A transport is one consequence of the contract
Polling is often sufficient for infrequent status and simple clients, provided the service randomizes intervals and handles conditional responses. Long polling or server-sent events suits mostly server-to-client updates. A WebSocket or another persistent bidirectional connection suits frequent commands and updates, but it brings connection placement, authorization refresh, backpressure, regional affinity, and reconnect state. Push notifications wake an inactive user; their timing cannot decide correctness. An authoritative room or match server is justified when contested actions need one ordered owner.
The interview answer should choose among these after it knows update frequency, direction, client constraints, and failure behavior. “Use WebSockets” is no more a real-time architecture than “use HTTP” is a commerce architecture.
Press on the boundaries
A useful follow-up changes a condition and forces the design to reveal its policy. If a downtown cell becomes hot, coalesce map updates, subdivide spatial ownership, limit low-priority watchers, and apply backpressure before gateway memory grows without bound. If a region fails, preserve authoritative trip ownership during failover; a stale worker or recovered owner needs a newer epoch or fencing token before it can write again. If offline phones upload old batches, retain them only where history has a stated use and keep them out of current state.
Location and presence are sensitive in aggregate even when each message is small. Authorize every subscription, minimize precise coordinates, separate coarse display from exact internal state where possible, expire sharing when its purpose ends, and audit access to safety-sensitive tools. Defend ingestion against replay, spoofed movement, fake availability, and impossible rates. A plausibility check lowers confidence; it does not prove that a coordinate is honest.
Measure the promise directly. Update age at read time exposes staleness that uptime hides. Ingestion lag and rejected sequences separate producer, network, and backend problems. Offer expiry and acceptance conflicts expose marketplace pressure. Reconnect failures, snapshot age, fan-out latency, hot-cell load, ETA error, and unauthorized subscription attempts connect the system to visible user and operator outcomes.
The same measurements guide cost. Adapt location frequency to motion, battery, and active watchers. Coalesce cursor and map deltas. Avoid route or ETA work when the underlying evidence has not changed enough. Tier history according to its actual replay, support, fraud, and audit use. High-frequency data kept “just in case” is both a cost and a privacy liability.
Transfer the contract, not the diagram
For live location or delivery tracking, the decisive question is confidence. Partner events and GPS points may arrive late, so retain event history, prevent unexplained state regression, and show age or uncertainty. A delivery status can often tolerate bounded delay; a safety-sensitive dispatch decision may not.
For multiplayer state, local prediction can make movement responsive while an authoritative room server owns contested outcomes. Clients send sequenced input commands; the server advances ticks and broadcasts snapshots or deltas. The client reconciles prediction with server state. Damage, score, inventory, and win conditions cannot become true merely because one client rendered them.
For collaborative presence, cursors and selections are ephemeral. Room state should expire by heartbeat or TTL because clean disconnects are not guaranteed. Throttling, coalescing, reconnect snapshots, and privacy controls matter more than durable history for every cursor. Document operations need a separate ordering and conflict contract; presence must not be allowed to dictate it.
For real-time bidding, the visible high bid may lag while acceptance remains authoritative. The server checks eligibility and increment rules, binds retries to an idempotent bid id, applies one receipt-time and tie policy at the deadline, and records accepted and rejected bids for audit. A notification arriving before another bidder’s screen update does not decide the winner.
Each adaptation changes the owner and consequence of truth. The common architecture is not a box diagram. It is the discipline of separating claims, current views, authoritative transitions, routing, durable evidence, and repair.
Rehearse the decision, then critique it
Choose an auction prompt and spend eight minutes writing freshness contracts for auction status, visible high bid, submitted bid, deadline, and settlement. Name the owner, acceptable staleness, failure display, and audit need for each. Only then draw ingestion, authoritative acceptance, fan-out, and history.
Now change one fact: a bid reaches the server before the deadline, but the bidder’s response and every public update arrive after it. Explain the outcome without using notification time or a client clock as authority. Then change the prompt to collaborative presence and identify which parts of the auction design would be wasteful or wrong.
When reviewing a mock, ask:
- Did the answer split state by consequence instead of calling the product “real time” as a whole?
- Did it name which inputs are claims and which component owns commitment?
- Did scale estimates lead to a partitioning, routing, or backpressure choice?
- Did reconnect and late data have visible user behavior as well as backend recovery?
- Could another engineer identify the exact ordering and idempotency boundary?
- Did privacy, abuse, observability, and cost attach to the chosen design rather than appear as a closing recital?
An answer that cannot resolve the late bid is not ready, however polished its diagram. An answer that resolves it but cannot tell the bidder what happened is still missing part of the system.
Field reference
On the interview scratchpad, keep only the sequence that changes the design:
- Split the state and name the consequence of staleness.
- Name the owner of truth; treat other inputs as claims or commands.
- Set the scale, freshness window, ordering boundary, and expiry policy.
- Separate current serving state from durable evidence.
- Route only to authorized regions, rooms, watchers, auctions, or matches.
- Define late data, duplicate, reconnect, snapshot, replay, and stale-display behavior.
- Measure update age and user-visible error; control fan-out, frequency, and retention from those signals.
The shortest memory hook is this: tracking needs bounded delay plus confidence; dispatch needs fresh claims plus authoritative assignment; presence needs expiry; multiplayer needs predicted display plus server truth; bidding needs a live view plus an authoritative deadline.
Data and workflow platforms use the same habits under different names. There, the crucial questions become producer contracts, delivery semantics, consumer lag, replay, and backpressure.