Appendix D — Orders of Magnitude

“A memory access is 100 nanoseconds” is not a performance fact. It is an incomplete record. Which memory level? Sequential or dependent? Warm or cold? On which processor, under which power and contention state? Was the number a single-operation latency, inverse throughput from a batch, or an end-to-end observation that included a runtime? A design review that starts from the naked number can be wrong by three orders of magnitude before it draws its first arrow.

Use this appendix when you need to reject an impossible budget, form a first capacity hypothesis, choose what to measure, or audit a claim that one layer “cannot matter.” The ranges here are calibration aids. They are explicitly dated, scoped, and replaceable. They are not promises from hardware, runtimes, filesystems, or cloud networks.

After working through this reference, you should be able to:

• attach a boundary, unit of work, state, distribution, date, and provenance to a magnitude;
• distinguish operation latency from inverse throughput and bandwidth;
• propagate ranges through a critical path without hiding amplification;
• recognize when locality, queueing, cold state, or failure invalidates a remembered number; and
• name the production observation or representative experiment that must replace an estimate.

Appendix A defines the book’s units, Appendix B supplies its distribution language, and Appendix C connects service demand to utilization and capacity. This appendix supplies provisional inputs for those models.

A number becomes usable only with its record

Write a magnitude as a small evidence record:

The statistic changes the meaning. If a loop completes two million dependent iterations in 3.2 ms, dividing yields roughly 1.6 ns per iteration. That is inverse throughput for that compiled loop. It does not establish that an arbitrary CPU instruction has 1.6 ns latency. A bandwidth result similarly says how quickly a particular stream moved under a particular access pattern; it does not give the delay of one dependent access.

Use three evidence labels consistently:

• Observed means the described boundary was measured.
• Modeled means stated inputs and equations produced the value.
• Assumed means the value is provisional and exists to test sensitivity.

An observed microbenchmark is not automatically stronger than a modeled production path. It may measure the wrong boundary with great precision. Evidence quality depends on fitness for the decision.

Dated calibration ladder: one local host

The following measurements were generated on 2026-07-12 UTC by the companion fixture. The host used a 12th-generation Intel Core i7-1265U with 12 logical CPUs, Linux 7.0.0-27-generic x86-64, Node 24.16.0, V8 13.6, OpenSSL 3.5.6, and zlib 1.3.1. The fixture ran as one foreground process without CPU pinning, frequency control, cache flushing, NUMA control, thermal control, or host isolation.

The table reports nearest-rank p10–p90 across repeated batch measurements after warm-up. The range describes run-to-run variation in this short experiment, not a confidence interval and not the tail of production requests.

One outlier reinforces the boundary lesson: the loopback fixture’s maximum was several milliseconds even though p90 remained below 50 us. A runtime pause or host scheduling event can dominate a path composed of “microsecond operations.” The right conclusion is not to erase the outlier or promote it to a service p99. It is to decide whether the production objective is sensitive to similar pauses and measure that population directly.

The raw samples, environment manifest, method, and verifier live in examples/performance-engineering-system-design-handbook/appendices/orders-of-magnitude/. Re-run them after a runtime, kernel, host, filesystem, or machine change. Preserve the old result when comparing editions; do not silently overwrite a decision’s evidence.

CPU and runtime: cycles are not requests

CPU-scale work spans at least three useful boundaries:

1. instruction or dependency latency, usually expressed in cycles under a named microarchitecture;
2. inverse throughput of a repeated kernel, where multiple operations may overlap; and
3. CPU service demand per useful completion, including language, allocation, branch, cache, syscall, and framework work.

Only the third feeds a service capacity model directly. If a request consumes 1.8 ms of CPU service demand on the current workload, one fully available CPU-equivalent has a modeled ceiling near 1 / 0.0018 = 556 request/s before headroom. Replacing that measurement with “an add is about a nanosecond” loses the executed operation count, cache behavior, branches, runtime overhead, and useful-completion boundary.

The fixture’s 1.60 ns loop result is useful as a smoke alarm. If a design assigns 200 ns to execute a million iterations, it is impossible by roughly four orders of magnitude. It is not useful for claiming the million-iteration production function will finish in 1.60 ms. The loop may be optimized differently; its data may miss cache; other work may run; and the production unit may allocate or block.

Runtime operations need the same care. The observed 34.3 ns Promise-resume inverse throughput covers an already-resolved Promise in one tight async function. A timer, a worker handoff, an epoll wake-up, or a task waiting behind runnable work has another path. Treat “async overhead” as a path decomposition, not one constant.

Memory: access pattern determines the magnitude

Memory discussions often collapse register access, cache hits, dependent DRAM misses, streaming bandwidth, page faults, and remote NUMA access into “memory.” These mechanisms differ in both unit and scale.

Use latency for a dependent access chain whose next address cannot be known early. Use bandwidth for independent streaming work. State working-set size, stride, read/write mix, locality, page state, allocation policy, and concurrency. A warm sequential scan can report tens of GiB/s while a pointer-heavy structure makes only one dependent cache miss at a time. Neither number contradicts the other.

The local 64 MiB scan touched one byte per 64-byte line and reported about 17.5 GiB/s. That is a language-loop result with uncertain cache residency. It can support a rough lower-bound question: reading 512 MiB through the same boundary cannot take less than about 0.5 GiB / 17.5 GiB/s = 28.6 ms. It cannot justify a 28.6 ms prediction for a production scan. Decompression, parsing, page faults, memory allocation, NUMA placement, other readers, and device I/O can all add work.

Failure file — the warm-cache storage claim. A team repeatedly reads the same 8 MiB file and observes several GiB/s. It multiplies that bandwidth by a 2 TiB recovery scan and predicts minutes. The repeated file was served from the page cache. Recovery reads a much larger working set while foreground traffic competes for the device. The changed variables are cache state, working-set size, and concurrency; the remembered bandwidth is not a storage-media measurement.

Storage: name the durability and cache boundary

A storage operation might mean a page-cache read, a system call, a filesystem journal commit, a device cache flush, a replicated database commit, or object retrieval over a network. “Disk latency” names none of them.

For reads, record request size, sequentiality, queue depth, cache state, direct versus buffered I/O, filesystem, device, and tail statistic. For writes, additionally record whether the call returned after copying to memory, after an operating-system durability operation, after device persistence, or after replica acknowledgment. Correctness defines the boundary.

The local fixture observed about 1.41 ms to write and fsync 8 MiB through its host temporary filesystem. That surprising combination of high throughput and low apparent latency is a reason to narrow the claim, not celebrate a universal device result. The measurement includes the operating-system call path but cannot independently prove controller power-loss behavior, physical media state, or a distributed durability guarantee. It is valid only as an observation of that configured stack.

For a rough service-demand model, suppose a storage worker performs a measured 4 KiB durable operation with p50 service time 0.42 ms at queue depth one. The inverse is about 2,381 operations/s, but that is not a safe arrival rate: variance and queueing make latency rise before the theoretical ceiling, and the p50 is not mean service demand. Measure throughput and latency together across queue depth, include correctness checks, and reserve failure and recovery bandwidth.

Communication: topology changes the unit by orders of magnitude

Communication should be described as one-way delay or round-trip time (RTT), payload size, protocol, connection state, topology, statistic, and measurement interval. Do not compare a one-way estimate with an RTT observation or a warm connection with connection establishment.

This ladder deliberately mixes a local reproducible result with dated primary provider measurements to show scale, not interchangeability:

Cross-zone is intentionally not assigned one current folklore constant. The dated 0.61–1.13 ms observations and Azure’s less-than-2 ms design perimeter establish a millisecond order, not a promise. The topology label does not specify buildings, route, packet size, protocol, loss, congestion, placement, or provider. The correct maintained value is the pair-specific telemetry for the decision window. Google Cloud’s dashboard reports median TCP-derived RTT by zone pair; AWS Infrastructure Performance provides intra-zone, inter-zone, and inter-region aggregate p50 and an API. Capture the selected pair, time range, statistic, and health state in the design record.

Network time also amplifies. A request that performs four serial cross-region rounds does not have “a 66 ms network cost”; its propagation floor is roughly 4 × 66 ms = 264 ms before service and queueing. A fan-out to 100 replicas may issue rounds in parallel, but completion follows an order statistic and straggler policy rather than the mean of one call. Chapter 5 develops that tail effect.

Serialization, compression, and cryptography move work across resources

These operations are often justified with one local speed while their end-to-end consequence crosses CPU, memory, and network boundaries.

The local JSON result was about 253 us p50 for a stringify-plus-parse round trip of a 48.7 KiB object. At 4,000 such round trips per second, the modeled CPU demand is approximately:

4,000 round trips/s × 0.000253 CPU-s/round trip = 1.012 CPU-s/s

That is roughly one CPU-equivalent if the measured elapsed time approximates CPU demand under the target concurrency. It does not include network framing, copies outside the runtime, validation, or business logic. It also says nothing about a different object shape.

The zlib fixture took about 3.21 ms p50 for a 1 MiB deflate-plus-inflate round trip on synthetic input. Compression can still lower end-to-end latency if saved transfer time exceeds added CPU and queueing. For a one-way 1 MiB payload over a 100 Mbit/s effective link, raw transfer alone is about 80 ms. If compression reduces it to 0.25 MiB, modeled transfer becomes about 20 ms; several milliseconds of codec work may be worthwhile. On a 25 Gbit/s link, the raw transfer floor is about 0.32 ms, so the same codec can make the path slower. The changed variable is effective link time, not whether compression is “fast.”

The fixture measured roughly 2.15 GiB/s for SHA-256 and 1.41 GiB/s for AES-256-GCM encryption on 1 MiB buffers. Those bandwidths do not predict 1 KiB request latency: fixed setup, allocation, framing, authentication-tag handling, connection protocol, and queueing become material. They also do not justify unsafe key or nonce practices. Cryptographic invariants remain part of correctness; a performance design may change batching or offload, not silently weaken security.

Turn the ladder into a budget, then try to break it

Consider a Mercury API path with these assumed operations:

• parse 50 KiB JSON: 0.13–0.20 ms, using half of the measured local round trip as a provisional range;
• perform six in-memory lookups: 0.02–0.20 ms total, deliberately wide because the fixture does not measure the target structure;
• call two same-zone services in parallel: 0.08–1.5 ms network RTT each, an assumption awaiting pair telemetry;
• execute each dependency: 2–8 ms;
• compress a 1 MiB response: 1.5–4 ms, a provisional share of the local codec round trip;
• transmit 0.25 MiB at 100–500 Mbit/s effective goodput: 4–20 ms.

The critical path is not the sum of every line. The two calls overlap, so use the slower branch:

modeled path = parse + lookups + max(call A, call B) + compression + transfer
low estimate = 0.13 + 0.02 + (0.08 + 2) + 1.5 + 4 = 7.73 ms
high estimate = 0.20 + 0.20 + (1.5 + 8) + 4 + 20 = 33.90 ms

The 7.7–33.9 ms range is a hypothesis, not a latency objective. It omits admission, queueing, TLS record work, scheduling pauses, retries, and response-tail dependence. Its value is diagnostic: transfer and dependency service dominate the high estimate, so measuring a nanosecond-scale CPU primitive first would be poor evidence. Obtain a trace that separates queue, service, codec, and transfer; measure the actual network pairs; and replay the production payload distribution.

Now apply failure. If a dependency slowdown triggers one retry after 10 ms, the path may add another round and consume duplicate downstream work. If compression workers saturate, their 3 ms service becomes tens of milliseconds of wait. If effective goodput falls to 20 Mbit/s, 0.25 MiB needs about 100 ms to transmit. The magnitude table did not become false; the operating state changed.

Failure patterns in magnitude reasoning

Applied exercises with answer guidance

Foundation — range propagation. A service serially performs three cross-region rounds. Pair telemetry shows p50 RTTs of 18, 19, and 18 ms for the relevant window. The dependency service budget is 6–12 ms per round. A defensible median-scale estimate is 55 ms of network plus 18–36 ms of service, or 73–91 ms, before local work and queueing. Adding the three observed p50s does not predict end-to-end p50 when the calls and samples are dependent; it is a planning estimate. The next evidence is a trace over the target request population.

Field — codec trade-off. A 1 MiB payload compresses to 0.30 MiB. Compression and decompression together take 3–7 ms under representative concurrency. At 50 Mbit/s effective goodput, transfer falls from about 160 ms to 48 ms, saving 112 ms before codec cost; compression is favored. At 10 Gbit/s, transfer falls from about 0.80 ms to 0.24 ms, saving only 0.56 ms; this codec path loses. An alternative answer is valid if it includes CPU queueing, one-way versus round-trip placement, payload distribution, and cost. Measure codec service demand and effective goodput together.

Principal — red-team the calibration. A review uses the local 1.41 ms 8 MiB fsync observation to approve a replicated log with a 5 ms p99 commit objective. Reject the inference. The fixture used a local temporary filesystem, uncontrolled device/cache state, p10–p90 batch summaries, and no replica acknowledgment. Require a commit-boundary definition, target storage topology, power-loss semantics, representative record size, queue depth, failure state, p99 distribution, correctness test, and replication trace. A conservative in-memory acknowledgment is not a valid substitute if the invariant requires durable replicated commit.

Field card: maintain the magnitude, not the folklore

Before carrying a number into a design:

• What exact operation and useful-work denominator does it describe?
• Is it latency, inverse throughput, bandwidth, capacity, or end-to-end duration?
• Which boundary, payload, concurrency, access pattern, and topology were measured?
• Is the system warm, cold, contended, degraded, migrating, or recovering?
• Which quantile, sample population, run count, and time window produced the range?
• Is the value observed, modeled, simulated, or assumed?
• Are environment, versions, date, raw evidence, and reproduction path recorded?
• Which correctness and durability semantics are inside the boundary?
• How do serial rounds, fan-out, retries, copies, visits, and queueing amplify it?
• Which changed variable would invalidate transfer?
• What sensitivity result would reverse the architecture choice?
• Which representative measurement must replace the estimate, and who owns it?

Use a dated range to disqualify impossible designs and prioritize evidence. Do not use it to certify a tail objective. The decision rule is: retain a magnitude while the decision is insensitive across its credible range; measure the target boundary when the range crosses a decision threshold, when amplification is nonlinear, or when correctness depends on the timing boundary.

Carry every retained range into the design review as an assumption with an owner, sensitivity, and replacement test. The review should approve only the smallest commitment that remains defensible across that uncertainty.

Sources and evidence scope

• Google Cloud VPC network performance documents its same-zone c2-standard-4 TCP request/response probe boundary and the typical p50/p99 figures cited here, updated 2026-07-22. It does not promise the latency of another instance type, placement, protocol, or application.
• Google Cloud Performance Dashboard metrics explains that zone-pair latency is median TCP-derived RTT sampled from traffic and may include stack or application effects. Use the actual pair and project population rather than treating the aggregate as an SLO.
• AWS Infrastructure Performance documents intra-zone, inter-zone, and inter-region p50 telemetry, its API, and the historical eu-west-1 to us-east-1 sample. The sample proves method and historical scale, not current route performance.
• Azure network round-trip statistics reports monthly p50 from continuous one-minute backbone probes; this appendix uses the dataset dated 2026-07-02 only to illustrate nearby and distant regional scale.
• Azure availability zones describes a less-than-2 ms inter-zone latency perimeter. This is a provider design objective, not evidence for an application’s latency distribution.
• ThousandEyes’ 2018 Public Cloud Performance Benchmark is the original measurement report behind the dated 0.61–1.13 ms AWS inter-AZ illustration. Its age is the point: retain the order for calibration, but replace the value with current pair telemetry before a decision.
• OpenSSL speed documentation shows why buffer size, elapsed versus CPU time, parallelism, algorithm interface, and machine-readable output belong in crypto measurements. The local fixture uses Node’s crypto boundary instead of claiming equivalence to openssl speed.
• zlib technical details describes format behavior, not performance constants. The fixture therefore measures one implementation, level, and corpus and preserves its transfer limit.
• The CPU, memory, local storage, loopback, serialization, compression, cryptography, and runtime ranges are observations from examples/performance-engineering-system-design-handbook/appendices/orders-of-magnitude/results.json. Its verifier checks schema, category coverage, positive samples, quantile arithmetic, and scope fields; it cannot establish production representativeness.