Appendix D - System Design Formula Sheet

Start with the decision that could change

A formula is useful only when its result can alter the architecture. Before calculating, write down the decision at risk: whether to move work off the request path, partition a store, retain less data, protect a hot key, add backpressure, or accept a lower availability target.

Then name the workload path and horizon. “Requests” is too vague; “thumbnail reads during the busiest sustained hour” can be estimated. Round assumptions to one useful digit, keep units attached, and stop when the risky boundary is visible.

Fast conversions

These decimal approximations are usually sufficient for spoken arithmetic. Use binary units only when the distinction affects a real limit.

Follow the work

User count does not create load by itself. Convert behavior into the operation that consumes a constrained resource, then include the work caused by that operation.

average operation rate ~= operations in horizon / seconds in horizon
peak rate ~= average rate * stated peak multiplier
concurrency ~= arrival rate * average time in system
downstream rate ~= input rate * fan-out * work per recipient
payload bandwidth ~= operations/second * bytes/operation

The concurrency relation is Little’s Law used as an approximation: rate and time must describe the same stable population. It is useful for connections, sessions, in-flight requests, and workers. It is not a license to average over a burst or an overloaded system.

Keep average, sustained peak, short burst, and skew separate. A fleet may handle the daily total while one tenant, document, partition, or scheduled event breaks the design.

Account for every durable copy

Start with logical data. Add each physical or derived copy explicitly so that its retention, repair, and deletion obligations remain visible.

logical bytes ~= record count * average encoded record size
daily ingest ~= writes/second * bytes/write * seconds/day
retained logical bytes ~= daily ingest * retention days
replica bytes ~= retained logical bytes * replication factor
total stored bytes ~= replicas + indexes + backups + derived copies + overhead
write bandwidth ~= writes/second * bytes/write * physical write amplification

A replication factor of three means roughly three physical bytes per logical byte before overhead; the extra copies provide redundancy, not additional logical capacity. Erasure coding has a different overhead and repair profile, so do not force it into the replication formula. Compression ratios, index size, and storage-engine amplification need measurements or stated assumptions.

For bandwidth, distinguish application payload from protocol overhead and internal movement. Cross-zone replication, cross-region copies, cache fills, and client egress can cost more than traffic entering the service.

Size queues and caches from behavior

A queue buys time only when the backlog is bounded and recovery has spare capacity.

backlog growth = max(0, arrival rate - service rate) * impairment duration
drain time ~= backlog / (recovery service rate - continuing arrival rate)

If recovery service does not exceed continuing arrival, the queue never drains. Convert the backlog to bytes, include replication and retention, and decide what happens when the bound is reached: reject, shed, coalesce, spill, or degrade.

For a cache, a global hit rate can conceal hot-key and object-size skew. A first pass is still useful:

origin read rate ~= total read rate * (1 - hit rate)
hot-set bytes ~= hot object count * average object size * overhead * copies

The result becomes architectural only after naming freshness, invalidation, miss cost, eviction behavior, and protection against a stampede.

Put availability and latency on a user path

Availability belongs to a user-visible function over a time window, not to a diagram as a whole.

Trace the critical dependencies for that function. A serial dependency can limit the path even when every service advertises an impressive target; retries, replicas, and failover help only when their failures are sufficiently independent. Shared regions, credentials, control planes, data corruption, and bad deployments often defeat that assumption. Graceful degradation can remove a dependency from a reduced user path, but it does not make the full function available.

For latency, p50 is the median, while p95 and p99 expose progressively slower tails. Use an end-to-end percentile for the user promise, then inspect traces to find which stage contributes delay. Do not add component p99 values and call the sum the path p99: the slow requests may not coincide. Average latency also cannot describe a tail objective.

Cost is usage multiplied by a priced unit

Do not invent provider prices. Identify the units whose price you would look up or benchmark, and separate recurring resource cost from engineering and operational cost.

monthly compute ~= instance-hours * price/instance-hour
monthly storage ~= average stored GB * price/GB-month
monthly egress ~= transferred GB * price/GB
request cost ~= billable operations * price/operation
third-party cost ~= billable events * price/event

Add headroom, replicas, standby capacity, backups, observability volume, and inter-region traffic to the appropriate quantity. The largest line item is not automatically the best optimization: first ask whether changing it would weaken latency, durability, recovery, or team operability.

One estimate, carried to a decision

Suppose an event service receives 20,000 events per second. An encoded event averages 1 KB, raw events are retained for 30 days, and the serving store keeps three copies.

daily ingest ~= 20,000/sec * 1 KB * 100,000 sec ~= 2 TB/day
retained logical bytes ~= 2 TB/day * 30 days ~= 60 TB
replica bytes ~= 60 TB * 3 ~= 180 TB

The 180 TB excludes indexes, backups, compaction headroom, and derived analytics data. That omission is useful because it tells you what to measure next. It also makes retention and data layout architectural questions rather than housekeeping.

Now test sensitivity. If the encoded event is 5 KB rather than 1 KB, replica storage approaches 900 TB before overhead. That change may justify a smaller schema, tiered retention, compaction, or keeping only derived aggregates in the serving store. The arithmetic has done its job because a plausible uncertainty can overturn the storage plan.

Rehearse this way: take one completed design, name the decision that an estimate could reverse, and follow one operation through fan-out, payload, retention, copies, failure, and cost. Multiply the most uncertain assumption by ten. If the only conclusion is “use more machines,” locate the partition, coordination, or policy boundary that those machines still cannot divide.

When a required input is unknown, state the threshold it would decide. A benchmark, billing page, measured compression ratio, or traffic distribution can supply the number later. False precision cannot.

Related links

• Back-of-the-Envelope Estimation
• High-Level Architecture and Data Flow
• Scaling and Partitioning
• Reliability and Failure Engineering
• Cost and Architectural Economics