Appendix J — Architecture Diagram Notation

Two diagrams show the same checkout service after a zone loss. In the first, three database cylinders sit side by side under a region label; arrows connect them, and the reviewer approves sending writes to the least loaded cylinder. In the second, one cylinder is marked authoritative, one is a lagging replica, and the third is a derived index under repair. The checkout critical path ends at the authority; replication acknowledgement is explicitly outside the commit rule; an 8 TiB recovery transfer competes with foreground traffic across the remaining zone boundary. The second diagram rejects the failover.

The architecture did not change. The visible contract did. A performance diagram is useful only when a reader can recover what is inside what, who owns truth, which work waits for which result, where work can queue, which path determines the objective, and what changes during failure and recovery. Unlabeled boxes and arrows encourage the reviewer to supply those answers from imagination.

Use this appendix when creating a design-review map, a latency or capacity argument, an incident diagram, or a migration plan. After applying it, you should be able to:

• distinguish deployment containment from trust, tenancy, and failure domains;
• encode synchronous calls, asynchronous messages, replication, and bulk transfer without implying false completion semantics;
• mark authority and the lifecycle of replicated, cached, derived, and ephemeral state;
• expose queues, traffic controls, schedulers, path importance, and quantitative budgets; and
• reject a diagram whose meaning depends on color, layout, or undocumented local convention.

The companion registry at examples/performance-engineering-system-design-handbook/appendices/architecture-diagram-notation/notation.json defines the grammar as data. mercury-recovery.json applies it to a bounded recovery scenario, and verify-notation.mjs checks coverage and structural obligations. The fixture is not a renderer. A team may implement these marks in any drawing system if the semantics survive export, print, and review.

Start with a diagram header, not a box

A diagram is a scoped claim. Put its scope where a title block would go:

The same topology needs different diagrams for normal serving and regional recovery because edges, available capacity, state freshness, and critical paths change. Do not place several modes on one page unless the purpose is an aligned comparison. A label such as “recovery after zone-b loss; valid checkout; one durable order within 250 ms” prevents a reader from applying normal-state arrows to the recovery decision.

Containment is a strict grammar

Use enclosure for contains, not “is near.” Nest only relationships that matter to the argument, and label every enclosure.

Containment does not prove isolation. Two processes in different host frames may share a power, network, storage, quota, scheduler, control-plane, or operator failure. Conversely, a tenant boundary can cross several hosts. Draw deployment containment once, then overlay cross-cutting boundaries with explicit labels instead of forcing every concern into the nesting hierarchy.

A box may carry small badges: ×12 for cardinality, 4 cores / 8 GiB for relevant supply, or cold for state. These are annotations, not new shapes. If a drawing exceeds roughly fifteen primary elements, split it into an overview and a detailed path view with stable IDs. Density is not rigor.

Four edges, four completion contracts

Every arrow has direction, a flow kind, and a label. The line answers how work moves; the label answers what completion means.

An asynchronous arrow must not mean “eventually works.” Label the handoff: accepted durably, best-effort notification, or queued; outcome observed elsewhere. A replication edge does not identify the authoritative copy by itself. If a write is acknowledged after one local commit but before remote replication, state that beside the edge. If the policy changes in degraded mode, produce a second panel.

Arrow direction describes transfer, not causality in every sense. A request arrow and response can be one bidirectional labeled call when response bytes do not affect the argument. Draw both directions when asymmetric payload, congestion, streaming, acknowledgement, or cancellation matters. For fan-out, label cardinality (1→24) and completion (first 8, all, quorum 2/3).

State symbols answer “what may be lost or stale?”

Use a cylinder for persisted state and a dotted box for ephemeral state, then place a letter and full legend label inside. Shape alone says too little.

“Database” is an implementation category, not an authority declaration. A cache can be persisted yet discardable. A derived index can be durable yet non-authoritative. A replica promoted during failure may become authoritative; show that as a state transition or separate mode, not by leaving both cylinders marked A.

State symbols should carry the performance fact that changes the decision: lag p99 4.2 s, TTL 30 s, 8 TiB rebuild, or 2,400 tokens in memory. If the number is a model, prefix it M: or state the evidence type in the caption.

Waiting and control deserve first-class marks

Hidden waiting produces hidden latency and hidden recovery time. Place control marks on the path they govern:

• A queue is Q over a short stack. Annotate capacity and either age or wait objective: Q ≤ 4,000; oldest 38 ms. If the queue is unbounded or the bound is unknown, use a warning marker.
• A rate limiter is a gate marked R. Label scope, useful-work unit, interval, burst, and rejection/delay policy: tenant checkout 1,200/s, burst 200, reject 429.
• A load balancer is a diamond marked LB. Label the candidate domain and selection/health rule when skew or failover depends on it.
• A scheduler is a clock marked S. Label priority, fairness, deadline, or work-conserving policy; “scheduler” without policy does not explain who waits.

Place the queue before or after the limiter according to the real admission order. That detail decides whether rejected work consumes memory and wait time. Show per-process and shared queues separately. A queue drawn inside a process frame must not stand in for an external broker.

Add a clock badge to any deadline, timeout, lease, or freshness constraint. Name which clock and which event starts it. 250 ms end-to-end deadline and 130 ms storage budget are different contracts; a timeout configured independently at each hop is not a budget.

Path emphasis carries operational meaning

Use line weight and pattern, plus a text label, so meaning survives grayscale and color-vision variation:

Critical does not mean “important.” A fraud check may be mandatory and therefore critical even when it consumes little latency. A background replication path can become critical to the recovery-time objective. In a recovery diagram, show foreground and recovery paths together where they share CPU, storage, or network; otherwise the drawing hides the exact contention the capacity plan must cover.

Quantities use a label grammar

Write quantity, unit, population, and scope next to the edge or node:

• latency budget: ≤55 ms edge→API, valid checkouts, p99 target;
• throughput: 900 valid checkouts/s or 1,080 attempts/s—never bare 900 RPS when retries matter;
• data volume: 8 TiB usable and, when relevant, 12 TiB transferred after encoding;
• transfer rate: 290 MiB/s net repair, with the reserved and foreground deductions available in the caption; and
• fan-out: 1→24 shards; wait for first 8 valid results.

Use base-10 units for network rates and binary units for memory/storage unless the diagram declares another house policy. Do not place a percentile beside a path without the population and interval when ambiguity changes the claim. Budgets may sum only when they represent compatible serial components; parallel branches and queue distributions require a timeline or explicit completion model.

Overlay trust, tenancy, and failure without conflating them

Cross-cutting boundaries use labeled bands or brackets with distinct patterns:

• Trust boundary: where identity, authorization, confidentiality, integrity, or data handling changes. Mark the protocol and protection relevant to the claim.
• Tenancy boundary: where admission, scheduling, state, keys, quotas, or noisy-neighbor isolation changes. Label the tenant unit and shared resource.
• Failure boundary: the set expected to fail together for the scenario. Label the cause class or independence assumption and the evidence behind it.

These boundaries often overlap but are not interchangeable. A region can be one failure domain for control-plane credentials and several domains for host power. A shared cache may cross tenant boundaries without crossing a network trust boundary. Dashed zone containment must never be treated as proof of independent failure unless the architecture and operations establish it.

Read the Mercury recovery fixture as a ledger

The fixture’s header declares recovery after zone-b loss and defines success as one durable checkout commit within 250 ms. The critical path has three explicit budgets: 35 ms from user to edge, 55 ms from edge to API, and 130 ms from API to the authoritative order store. Their 220 ms total leaves 30 ms for work not expanded in this overview; it is a budget allocation, not an observed percentile.

The optional cache edge carries 20 ms but is marked O; Mercury may omit enrichment while preserving checkout success. A tenant limiter precedes a bounded queue, and the scheduler prioritizes checkout over enrichment. Those positions make an overload decision visible.

The authoritative A store replicates to R outside the checkout acknowledgement rule. During recovery, an 8 TiB bulk transfer runs at a modeled 290 MiB/s net rate toward derived index D. It crosses a failure-domain boundary and shares infrastructure with foreground work. The diagram therefore supports two distinct questions: can checkout meet its 250 ms objective, and can repair meet its recovery objective without destroying checkout goodput? No single “healthy” arrow answers both.

The companion verifier confirms the vocabulary, not the truth of Mercury’s assumptions. Reviewers must still validate containment, independence, budgets, safe transfer rate, and completion semantics against implementation and evidence.

Failure patterns in diagram reviews

Applied exercises

Critique a failover map. Draw three stores, two zones, and a load balancer. Mark one authority, one asynchronous replica, and one derived index. Add the exact acknowledgement rule, lag objective, promotion condition, and the failure boundary. Explain why proximity and arrows alone could have authorized an unsafe write.

Expose overload transfer. Diagram an edge limiter, API queue, worker scheduler, downstream connection pool, and optional enrichment path. Annotate offered attempts, admitted originals, valid goodput, queue bound/age, and deadlines. Move the limiter after the queue and describe which resource and user outcome change.

Separate foreground from recovery. Starting with the Mercury fixture, add a second bulk repair and a region egress cap. Show the path that becomes critical for the recovery objective, the path that remains critical for checkout, and the stop rule needed when repair harms valid goodput.

Diagram release card

• Does the header name scenario, boundary, unit, success, evidence status, and version?
• Is every containment boundary labeled, with no independence inferred from layout?
• Does every edge declare flow kind, direction, and completion semantics?
• Is exactly one authority visible for each state domain in each mode?
• Are replicated, cached, derived, and ephemeral copies distinguishable without color?
• Are queues, limiters, balancers, schedulers, deadlines, and unknown bounds visible where material?
• Are critical, optional, and recovery paths defined against named objectives?
• Do latency, throughput, volume, fan-out, and rate labels include units and scope?
• Are trust, tenancy, and failure boundaries separately labeled?
• Can a grayscale printout and a machine-readable ledger support the same decision?

The decision rule is: use a diagram in a performance decision only when a reviewer can recover scenario, containment, completion, state authority, waiting, path objective, quantitative scope, and cross-cutting boundaries without relying on color, proximity, or private explanation; otherwise treat it as an orientation sketch, not evidence.

Primary references and transfer notes

• C4 model, diagrams provides a hierarchy for software-system, container, component, and deployment views and emphasizes explicit diagram scope and legends. This appendix uses its clarity principles, not its element semantics as a performance proof.
• W3C Trace Context Recommendation standardizes trace-context propagation across distributed boundaries. A propagated context identifies causal correlation; it does not prove synchronous completion, state authority, or full sampling.
• OpenTelemetry tracing specification defines traces, spans, links, and propagation concepts useful when mapping asynchronous causality. Backend rendering conventions are not architecture notation.
• OpenTelemetry semantic conventions supplies stable telemetry names that can connect diagram IDs to evidence. Conventions do not supply workload-specific budgets or failure independence.

Appendix K turns the paths and boundaries defined here into evidence-selection workflows. Use it when a diagram identifies the unknown mechanism but not yet the least risky way to observe it.