Appendix N — Further Reading Map

You finish a design drill with one consequential blank: nobody can explain whether the queueing result assumes an open arrival process, whether regional promotion preserves one authority, or whether an inference benchmark represents the production request mix. What should you read next?

Do not begin with the subject’s largest book or the newest systems paper. Name the missing claim first. Then choose a short path containing three different forms of evidence:

1. a model source that defines the relationship or abstraction;
2. a mechanism source that explains what a real system can do; and
3. a method or application source that shows how to test, operate, or bound it.

The result of reading is not “I know this topic.” It is a revised model, a more precise claim, and an experiment or design question that could prove the claim wrong. No source here establishes the performance of your system. Each supports a particular kind of reasoning inside a declared transfer limit.

The machine-readable companion at examples/performance-engineering-system-design-handbook/appendices/further-reading-map/sources.json records all 33 sources, annotations, limits, and five reading paths. verify-map.mjs checks that the catalog covers every required concept and source form. The tables below are the reader-facing route map, not a ranked bibliography.

Read by gap, not prestige

Use four fields before following a link:

Source kind is a clue, not a quality ranking. A seminal paper often gives a clean original mechanism but freezes one system in time. A standard defines an interface or method but does not report your implementation’s behavior. A textbook integrates concepts but can smooth away version-specific detail. Official documentation names current behavior but may say little about comparison or failure history. A high-quality engineering report exposes decisions and operational lessons, yet its fleet, risk tolerance, and organizational controls are not yours.

Read an original paper to understand what problem a mechanism was designed to solve. Read a current specification or official manual before asserting what a deployed version guarantees. Read an independent or cross-cutting model to identify assumptions the implementation source does not emphasize. Finally, measure the selected boundary.

Computer architecture and operating systems

Start here when application behavior cannot be explained from source code or host averages. The recurring question is where work becomes instructions, cache and translation activity, scheduling delay, kernel work, device interaction, or waiting—and which observation can distinguish those paths.

A useful order for a host-tail gap

If fleet CPU is 45% while response p99 and runnable delay rise, do not jump straight to a microarchitecture manual. First use Computer Systems to draw the user-code-to-resource path. Use PSI to decide whether tasks lose time to CPU, memory, or I/O pressure at the relevant cgroup and interval. Read the perf security guide before requesting samples. Only then use the vendor manual to interpret counters for the exact processor.

The reading deliverable is a causal diagram and an evidence request: affected population, clock, suspected wait or service stage, counter or trace, capture duration, overhead bound, access owner, and result that would reject the hypothesis. “The CPU is slow” is not a deliverable.

Queueing, measurement, and experimental design

Use this region when a graph looks persuasive but its population, arrival semantics, stability, uncertainty, or causal comparison is unclear. The central discipline is to keep a mathematical identity, an approximation, a metric definition, and an experimental result in their proper categories.

Pair sources by role. Little’s Law can expose a conservation error; Kingman’s approximation can explain why variability deserves a sensitivity analysis; neither validates a benchmark. NIST can guide an experimental comparison; it cannot repair a load generator whose behavior changed under delay. The HdrHistogram discussion exposes that mechanism. RFC 6815 then asks whether the resulting method is safe and meaningful at the chosen boundary.

The expected trace is an experiment card: claim, unit of work, offered and admitted process, inclusion and loss rules, baseline and treatment, controlled factors, randomized or blocked factors, warm-up and steady state, repetition, uncertainty, raw artifact, stop rule, and transfer limit.

Databases, storage, networking, and distributed systems

Enter here with a named state or protocol obligation. “Learn replication” is too broad. Ask whether you need to explain leader authority, acknowledgement durability, read visibility, partition placement, transport completion, device queues, or recovery.

A useful order for regional authority

Suppose a review promises “automatic failover with no lost writes.” Read Raft to separate elected log authority from client-visible business completion. Read Spanner to see how one implemented system connects replication with time and transaction semantics. Use Designing Data-Intensive Applications to compare alternative consistency and replication structures. Then consult the actual database and transport specifications.

The output is not “use consensus.” It is a state and failure ledger: unique effect, client identity, authority, acknowledgement point, durable media, replica set, quorum, fencing, ambiguous outcomes, read rule, RPO, RTO, degraded capacity, repair demand, and evidence for each failure class. Network reachability and consensus safety do not by themselves deliver exactly one external business effect.

Reliability, capacity, and performance engineering

Use this region when the mechanism is understood but the service needs an objective, headroom policy, overload response, retry budget, tail strategy, or recovery plan. These sources are strongest when translated into an explicit control loop rather than copied as configuration advice.

Read the SLO chapter before selecting a capacity number: capacity exists only relative to a valid outcome and objective. Read overload and cascading-failure material together so a local limiter is analyzed with caller behavior and recovery. Use The Tail at Scale only after defining fan-out and completion semantics. A hedge that wins latency by doubling non-idempotent work is not a performance success.

The reading trace is a control contract: signal and population, protected goodput, admission boundary, priority and fairness unit, queue bound, retry owner and budget, degraded mode, failure and recovery reserve, action threshold, reaction time, stop condition, and test plan.

Streaming, search, analytics, and ML systems

These systems share large state, variable work, asynchronous execution, and expensive correctness checks, but they do not share one performance model. Route the gap to clocks and state for streaming, index and scoring behavior for search, stage and scan behavior for analytics, or model quality and serving scenario for ML.

A useful order for an inference-capacity claim

If a vendor result is being converted into production requests per second, begin with MLPerf’s exact scenario, load generator, quality check, division, system description, and compliance artifacts. Use The Tail at Scale to model what happens when the endpoint fans out or batches under an online deadline. Use the NIST handbook to design matched trials and uncertainty reporting. Use the overload sources to define admission and queue bounds.

Produce a transfer ledger with at least: model and precision; quality target; input and output distributions; prompt or tensor sizes; batch policy; arrival process and concurrency; latency events and loss rule; warm state; accelerator, host, network, and power boundary; software versions; goodput definition; failure and recovery state; and cost denominator. If a field differs, classify its likely direction of bias instead of applying a generic discount.

Cross-cutting map by source job

The five concept regions answer where the gap sits. This table answers which kind of source should speak first.

Seminal does not mean current. Current does not mean independent. Official does not mean complete. Peer-reviewed does not mean transferable. Operationally successful does not mean causally isolated. The map works because those weaknesses are paired, made explicit, and tested.

Validate a source before it enters a decision record

For a consequential claim, record this seven-line source ledger:

1. Claim: the exact sentence the source is being asked to support.
2. Authority: author or owning body, publication venue, and why it can speak to this claim.
3. Version: edition, standard number, software release, hardware generation, and publication or retrieval date where behavior changes.
4. Evidence: proof, specification, experiment, implementation description, or operational observation—and which it is not.
5. Boundary: workload, topology, scale, state, failure model, and organization represented.
6. Transfer limit: the first changed condition likely to reverse or weaken the result.
7. Local test: the measurement, trace, review, or failure exercise required before acting.

Reject a source when its accessible artifact cannot support the quoted claim, its version is unknowable and behavior is version-sensitive, its experiment changes several factors without preserving raw evidence, or its authority is merely repetition. Keep a superseded source when it teaches an originating model, but label it historical and pair it with the current specification.

Practice: turn reading into engineering work

Drill 1: the queue that “proves” more servers are needed

A dashboard shows queue age and p99 rising at 75% average CPU. Choose four sources from the map. For each, name its job. Your output must distinguish an identity, an approximation, an OS signal, and an experiment. Finish with two hypotheses that predict different per-resource or per-shard evidence.

Self-check: Little’s Law should not be presented as a CPU diagnosis; Kingman should not become a fitted production result without assumption checks; PSI should not become root cause; the experiment must preserve arrival semantics and the latency population.

Drill 2: “no-loss” regional failover

Choose a consensus source, a database source, a reliability source, and the current specification for the actual product. Write one supported claim and one unsupported claim for each. Then draw acknowledgement, authority transfer, ambiguous client outcome, degraded admission, and repair.

Self-check: the answer separates consensus safety from the business effect, RPO from replica lag, and traffic restoration from full state recovery. It names the evidence needed for fencing and duplicate/loss invariants.

Drill 3: benchmark to forecast

A published ML inference score is 9,000 queries/s. Product forecasts 8,100 production requests/s by applying a 10% discount. Build the transfer ledger from the inference path above. Classify every mismatch as likely optimistic, pessimistic, or unknown; do not combine the classifications into one invented correction factor.

Self-check: model quality, input/output mix, arrival process, batching, deadline, loss, warm state, power, failure capacity, and cost appear. The next action is a representative bounded experiment, not a more confident discount.

Field card: close one knowledge gap

• Name the missing claim, decision, and consequence of being wrong.
• Choose a model, mechanism, and method or application source.
• Prefer the originating paper for intent and the current specification for behavior.
• Annotate what each source adds, the claim it supports, and its transfer limit.
• Produce an engineering trace: equation, state machine, causal path, source ledger, experiment, or control contract.
• Verify version-sensitive claims against current official material.
• Measure the actual workload, boundary, state, failure mode, and objective.
• Record what evidence would reverse the decision.

The decision rule is: add a source to a design argument only when its authority and evidence match one precise claim, its transfer limit is explicit, and local evidence can still falsify the resulting decision; otherwise use it to generate a question, not to close one.

Appendix M reveals where the reasoning breaks. This map turns that break into a bounded study path. The next step is not to finish the bibliography. It is to return to the model, architecture, experiment, or incident with a sharper claim and a better chance of discovering that it is wrong.