API, Schema, and Data-Contract Design for Performance

GET /search?q=orbit&limit=all&include=owner,facets,history

This line looks like a retrieval request. It is also permission to scan an unnamed population, materialize an unnamed number of matches, hydrate three expansions, and keep the connection open for an unnamed time. If the search response contains 40 documents and the client retrieves each owner’s profile separately, the visible request authorizes 41 calls. If history expands without a byte or item limit, the same route can change from a screen read into a bulk export without changing its method or path.

Atlas Search replaces that ambiguity with a work contract:

{
  "query": "orbit",
  "filter": {"tenant_id": "tn_7", "updated_after": "2026-07-01T00:00:00Z"},
  "projection": ["document_id", "title", "owner_summary", "updated_at"],
  "page": {"size": 40, "after": "opaque:8fQ..."},
  "budget": {"service_work_ms": 50, "max_candidates": 10000},
  "consistency": {"minimum_index_position": "atlas:42:9182"}
}

The response can be smaller than 40 items. It cannot silently exceed the documented item, candidate, projection, and service-work ceilings. The cursor binds the continuation to the tenant, normalized filter, sort, projection, authorization context, and index generation. It is not authorization by itself. A client that asks for an unsupported projection or a query whose lower-bound cost already exceeds the budget gets a machine-actionable rejection before expensive work begins.

That change is not cosmetic API style. It limits the amount and shape of work a caller can create, states how a multi-request walk continues, and gives the service room to evolve its index and execution plan without teaching clients its internal offsets. An interface is performance-safe only when it bounds work, exposes completion semantics, supports evolution, and prevents clients from accidentally multiplying cost.

An interface commits both sides to a unit of work

An API is a public scheduling surface. Request fields decide which CPU, memory, storage, network, coordination, and downstream work becomes eligible. Response fields decide payload size, hydration, freshness, compatibility, and whether the client must make more calls. Error fields decide whether clients retry, wait, abandon, reconcile, or amplify an overload.

Start with a user journey, not a table schema. For each operation, record:

• the useful outcome, such as one search page rendered or one reservation durably identified;
• the authoritative state and consistency position required for that outcome;
• the maximum items, bytes, candidates, dependencies, and side effects;
• the end-to-end deadline and the service’s share of it;
• whether the operation is read-only, idempotent, conditionally repeatable, or effectful;
• which tenant, principal, traffic class, and priority own the work; and
• how a caller learns complete, partial, rejected, pending, unknown, or terminal outcome.

Granularity follows from that unit. A tiny endpoint is not automatically efficient. If a screen always needs order, line items, delivery promise, and a compact customer summary, four serial calls expose network latency and version skew without creating useful autonomy. One endpoint that returns every historical customer fact may overfetch, couple unrelated data owners, and make each response unbounded. The useful boundary is the smallest contract that completes a recurring outcome while retaining explicit cost and ownership limits.

Measure granularity in journeys as well as endpoints. Collect calls per logical operation, serial depth, fan-out, payload bytes used versus returned, cacheability, duplicate work, and failures after partial progress. A route that is fast alone can be expensive in its client call graph.

Choose interface form from interaction and completion semantics

Resource, operation, event, streaming, and batch interfaces solve different timing and ownership problems. A system can use all five without inconsistency.

Resource orientation is valuable when standard method semantics, caching, conditional requests, and stable identity fit the domain. Operation orientation is clearer for reserveCapacity, rebuildIndex, or quoteShipment, where a generic row update would hide invariants and workflow state. Events decouple the producer’s critical path only if publication has a durable handoff and consumers own retries. Streaming reduces time-to-first-item and buffering only when backpressure reaches the producer and cancellation releases work. Batches amortize parsing, authentication, network, and commit overhead but need item-count, byte, time, and atomicity boundaries.

Do not use transport as the taxonomy. HTTP can carry operations, streams, and events; RPC can expose resources; a broker can transport commands or facts. The contract is defined by state, sequencing, and completion—not by the library name.

Round trips expose an aggregation decision

The fixture models a 40-result Atlas page. The search call takes 24 ms. Fetching one 5 KiB owner object per result creates 40 additional calls. With eight concurrent profile connections and 9 ms per modeled wave, the client takes five waves:

[ L_{chatty}=24\ \text{ms}+\left\lceil\frac{40}{8}\right\rceil(9\ \text{ms})=69\ \text{ms} ]

This is a lower-bound teaching path: it omits queueing, connection setup, retries, and a slow outlier. A server-side owner-summary batch takes a modeled 14 ms after search, producing two calls and 38 ms. It also returns 921 bytes of projected owner data per result rather than 5,120 bytes:

[ 40(5{,}120)=204{,}800\ \text{bytes},\qquad 40(921)=36{,}840\ \text{bytes} ]

The projection reduces this modeled payload by about 82%. It is not evidence that aggregation always wins. The aggregate path may concentrate work, reduce cache reuse, or cross ownership boundaries. Its contract should cap result count, projection, batch key count, downstream deadline, and stale/missing behavior.

chatty client path                    bounded aggregate path
------------------                    ----------------------
search -> 40 document summaries       search -> 40 document summaries
  |                                    |
  +-> owner 1                          +-> owners.batch(40 IDs, 14 ms budget)
  +-> owner 2 ... owner 40             |
  five connection waves               +-> compact owner summaries or markers
  41 calls total                       2 calls total

An API gateway or backend-for-frontend can aggregate across owners, but it must not become an invisible distributed transaction. Return a missing-owner marker or a defined partial result when owner enrichment is optional. Fail the whole operation only when the user outcome requires every owner and the remaining deadline supports it. Record the dependency count and outcome in the response work receipt or server telemetry.

Pagination must bound a stable walk, not merely split bytes

Offset pagination is simple and sometimes adequate for small, stable sets. At deep offsets it may force the store to find and discard earlier rows; concurrent inserts and deletes can also duplicate or skip items unless a snapshot or stable ordering rule exists. Cursor pagination carries a continuation position, but it is only correct when its ordering and query context are explicit.

A bounded page contract should state:

default page size:       40 items
maximum page size:       100 items
server may return fewer: yes; short page does not alone mean end
end condition:           next_cursor absent
ordering:                updated_at DESC, document_id ASC
cursor binds:            tenant, filter hash, sort, projection, auth class,
                         index generation, continuation position, expiry
cursor authority:        none; authorize every request independently
total count:             omitted by default; estimate labeled with scope/as-of
response byte ceiling:   256 KiB before transport encoding

The AIP-158 pagination guidance is one documented pattern: a maximum page size, an opaque continuation token, otherwise matching query arguments, and an empty token at the end. Its naming is not mandatory outside that ecosystem, but its boundary lessons transfer. Base64 encoding a visible offset does not make a token opaque. Sign or encrypt cursor state as threat analysis requires, expire retained snapshots deliberately, and return a stable reason when the original walk can no longer continue.

Filtering and projection are part of the bound. Permit a supported predicate grammar, indexed sort combinations, maximum expression depth, maximum list cardinality, and a declared behavior for unknown fields. A generic query language without execution budgets can expose joins, regexes, deep nesting, or fan-out that the service cannot admit safely. Projection saves bytes and hydration only when authorization is evaluated before or during field access; it must not become a route around field-level policy.

Cost is a first-class request dimension

Atlas uses a modeled work ledger:

[ C=7+8{,}000(0.002)+200(0.05)+40(0.08)=36.2\ \text{ms} ]

The terms are fixed planning/parse work, candidate scan, rerank, and hydration. The 36.2 ms estimate fits a 50 ms service-work budget. It is not elapsed latency and does not include queue wait or external calls. Production admission should use a cheap conservative estimator before execution, then compare estimate with actual scanned candidates, CPU time, bytes read, dependency calls, and deadline outcome.

Reject, narrow, defer, or require an asynchronous job when the lower bound exceeds budget. Do not silently truncate in a way that looks complete. Cost-aware APIs may expose estimated_cost_class, scanned_candidates, result_completeness, and a stable budget_exceeded reason, but they should not reveal sensitive tenancy or index details.

Overfetch, underfetch, and N+1 are contract defects with different repairs

Overfetch returns fields or nested collections the journey does not use. It costs serialization, transfer, allocation, parsing, cache space, and sometimes authorization or storage reads. Projection, compact summaries, conditional expansions, and separate bulk export routes reduce it.

Underfetch omits data that clients always need, forcing serial calls and inconsistent snapshots. Composite views, batch loaders, server-side joins within one state owner, or derived read models can repair it. A derived view must carry freshness and source-version evidence; otherwise aggregation merely hides staleness.

N+1 appears when one list call triggers one call or query per element. It can exist in a client, resolver, ORM, template renderer, or service aggregator. Detect it with dependency calls per logical operation and with trace fan-out versus result count. Repair options include:

• fetch required data in the original owner query when the state boundary permits it;
• batch keys with a maximum cardinality and deduplicate them;
• precompute a bounded summary view from durable events;
• cache immutable or versioned summaries with explicit freshness; or
• change the screen so optional details load on demand.

The wrong repair is one unlimited include=* knob. It moves N+1 behind the server while preserving unbounded cost.

Write contracts need identity, time, cancellation, and outcome retrieval

Chapter 25 separated logical operation identity from attempts; Chapter 28 located the reservation invariant. The API must expose both decisions. A Ledgerline reservation write can use:

POST /v1/tenants/tn_7/reservations
Idempotency-Key: rsrv_01J...
If-Match: "inventory-epoch-42"
X-Request-Deadline: 2026-07-13T13:00:00.240Z
Content-Type: application/json

{
  "event_id": "evt_92",
  "quantity": 2,
  "request_hash": "sha256:canonical-body-and-operation-scope"
}

The server authenticates the tenant rather than trusting a body field. It scopes the key to tenant and operation kind, compares a canonical request hash, and retains a durable record through the documented retry horizon. A repeated matching request replays the same terminal result. The same key with a different hash returns a conflict. A concurrent matching attempt can return pending with a status URI instead of executing twice.

The deadline is an end-to-end budget, not a promise that the effect has not happened after the client stops waiting. The gRPC deadline guidance documents propagation with elapsed time deducted, while its cancellation guidance makes stopping application-spawned work cooperative. The principle transfers: propagate remaining budget; stop speculative or optional work; do not claim that cancellation rolls back a durable reservation or an external payment.

The response must distinguish:

HTTP defines method properties such as safety and idempotency in RFC 9110, but application idempotency still needs domain scope, identity, retention, and replay behavior. A POST can be made repeatable by such a contract; a nominally idempotent method can still trigger expensive repeated work if its implementation ignores attempt cost.

Errors are control messages, not prose wrappers

A caller needs a stable category, retry classification, authoritative state pointer, and overload advice. RFC 9457 defines a reusable problem-details format for HTTP APIs; RFC 6585 defines 429, and RFC 9110 defines Retry-After. A scoped overload response might be:

HTTP/1.1 429 Too Many Requests
Content-Type: application/problem+json
Retry-After: 2

{
  "type": "https://api.example.test/problems/tenant-concurrency",
  "title": "Tenant concurrency budget exhausted",
  "status": 429,
  "code": "tenant_concurrency_exhausted",
  "retryable": true,
  "retry_after_ms": 1800,
  "scope": "tenant:tn_7:reservation-write",
  "idempotency_key": "rsrv_01J...",
  "status_uri": "/v1/operation-results/rsrv_01J..."
}

The exact delay policy is service-specific. Clients should apply jitter and their remaining deadline; they must not retry a nonrepeatable effect merely because the transport returned 503. Distinguish authentication, authorization, validation, conflict, precondition, admission, dependency, deadline, cancellation, and unknown-outcome failures. Human messages can change; machine codes and field semantics need compatibility governance.

Tenant and priority context must survive every hop without becoming user authority

Work should be attributable to an authenticated tenant, logical operation, cost class, and traffic class from ingress through queues, services, storage, and asynchronous workers. That context drives admission, fairness, quotas, chargeback, traces, and incident analysis.

Do not trust arbitrary priority=critical from a client. The ingress maps authenticated identity and operation to an allowed class, signs or conveys that context over an authenticated internal channel, and downstream components enforce ceilings. When a foreground operation emits background work, define whether priority is inherited, reduced, or independently admitted. An asynchronous retry should not retain emergency priority forever.

Keep observability identifiers separate from authorization. A trace ID correlates work; it does not grant tenant access. A cursor resumes a walk; it does not grant access. An idempotency key retrieves an outcome only after the same principal and scope are authorized.

Compatibility is a matrix across writers, readers, and meaning

“The new schema parses the old bytes” is one cell in a larger compatibility argument. During a rolling deployment or delayed client upgrade, old and new writers interact with old and new readers, stored data, caches, events, and replay jobs.

Protocol Buffers’ proto3 language guide documents fixed field-number identity, defaults, explicit presence, unknown fields, and wire-safe versus unsafe changes. Its best practices emphasize that clients and servers do not update simultaneously. Those rules are format-specific, but the mixed-version discipline is universal.

Unknown-field preservation matters in proxies and read-modify-write paths. A new writer can add field 17; an old binary may parse and reserialize it safely in the binary format, but a conversion through JSON or manual field-by-field copy can drop it. Test the actual translators and language bindings. Define whether unknown enum values are retained, rejected, surfaced as UNRECOGNIZED, or mapped to a safe behavior.

Defaults are behavior. An absent max_results must not mean “unlimited.” An absent priority must not mean “highest.” For updates, distinguish “field absent, leave unchanged” from “field present with zero/false, clear it.” Presence bits, field masks, patch documents, or explicit operation fields can represent that distinction.

Schema evolution includes the work of changing stored reality

A compatible reader does not make a backfill free. Adding a derived search field to 2.4 billion records at 180 logical bytes per record touches 432 GB of logical data. At a modeled 18,000 records/s, one pass takes:

[ \frac{2.4\times10^9}{18{,}000}\approx133{,}333\ \text{s}\approx37.04\ \text{hours} ]

With a teaching I/O amplification factor of three for read, rewrite, index, or replication work, the fixture accounts for 1.296 TB of traffic. These are modeled decimal bytes, not observed storage behavior. Real duration depends on distribution, item size, compaction, replication, throttling, retries, cache effects, and foreground contention.

Use expand–migrate–contract:

1. Expand: deploy readers that accept old and new forms; add new storage/index structures without making them mandatory.
2. Produce: write the new field or event while preserving the old contract; record which version produced it.
3. Migrate: backfill in bounded chunks with checkpoint, ownership, throttle, retry identity, correctness comparison, and pause/rollback rules.
4. Observe: measure remaining old-form reads/writes, mismatch rate, lag, foreground SLO impact, and recovery demand.
5. Contract: remove the old path only after retained data, delayed clients, caches, replays, and disaster recovery no longer require it.

Dual writes are not automatically atomic. Prefer one authoritative write plus a durable change record or transactional outbox when the storage boundary supports it. If two destinations can diverge, make reconciliation a named process with an owner and evidence.

Bulk and long-running work need resource identities of their own

When a request cannot fit a normal deadline or budget, return a durable job identity instead of stretching a synchronous timeout. AIP-151 documents one long-running-operation pattern with result and metadata types; the transferable lesson is stable status, terminal outcome, progress semantics, error shape, and retention.

{
  "job_id": "bulk_01J...",
  "request_hash": "sha256:...",
  "state": "running",
  "accepted_items": 2000000,
  "completed_items": 735000,
  "failed_items": 120,
  "checkpoint": "chunk:1470",
  "result_manifest_uri": null,
  "cancel_semantics": "stop-unscheduled-chunks; completed effects remain",
  "expires_at": "2026-08-12T13:00:00Z"
}

At 500 items per chunk, the fixture creates 4,000 independently checkpointed chunks. Chunk size is a control, not a guarantee: cap serialized bytes and estimated work too. Define whether a bulk request is atomic, per-item independent, or grouped by invariant boundary. For partial failure, return a result manifest keyed by stable item identity, not a giant in-memory error array. A cancellation stops future work and lets in-flight chunks reach a safe boundary; it does not erase committed effects.

The job itself needs admission, tenant fairness, priority decay, deadline or maximum age, progress freshness, retry policy, poison-item handling, reconciliation, and cleanup. Background work still competes for CPU, I/O, replication, and cache.

Partial results must say what is absent and how old the evidence is

A successful transport response can contain incomplete knowledge. Search may time out on two shards, an enrichment dependency may be unavailable, a streaming consumer may receive a prefix, or a model may produce an estimate.

Return enough metadata to prevent false completeness:

{
  "items": ["..."],
  "complete": false,
  "coverage": {"successful_shards": 22, "required_shards": 24},
  "freshness": {"as_of": "2026-07-13T12:59:58.120Z", "index_position": "atlas:42:9182"},
  "confidence": {"kind": "estimated_total", "lower": 9100, "upper": 9800},
  "omissions": [{"scope": "shard-17", "reason": "deadline"}],
  "next_cursor": "opaque:..."
}

Confidence belongs only where there is a defined estimation method and interval meaning. Do not attach a decorative percentage. Freshness needs a clock or logical position, population, and source. A partial page needs a stable rule for whether the cursor retries omitted partitions, advances past them, or invalidates the walk. If correctness requires all shards, return failure rather than laundering incompleteness through 200 OK.

API performance review card

Use this compact artifact before implementation and again with traces under load:

operation / journey: ___________________________________________
useful outcome and authority: __________________________________

request bounds:
  items / bytes / nesting / filter complexity: _________________
  candidates / dependency calls / service work: _______________
  default, maximum, coercion or rejection: _____________________
  tenant and traffic class source: _____________________________

sequence and response:
  calls and serial depth per journey: __________________________
  projection / expansion / aggregate owner: ___________________
  pagination ordering, cursor binding, expiry: _________________
  complete / partial / freshness / confidence: _________________

completion:
  end-to-end deadline and cancellation owner: _________________
  idempotency scope, hash, retention, replay: __________________
  accepted / committed / pending / unknown states: _____________
  overload code, retry safety, delay advice: ___________________

evolution:
  old/new writer-reader matrix: ________________________________
  absence, default, unknown-field behavior: ____________________
  expand/migrate/contract and backfill budget: _________________
  delayed clients, stored data, replay, rollback: ______________

evidence:
  calls, bytes, work, queue, cost and outcome telemetry: _______
  adversarial test and revisit trigger: ________________________

Reject the contract if a caller-controlled dimension can grow without a corresponding bound, or if a timeout/retry cannot be mapped to an authoritative operation outcome.

Contract drills

Redesign the unbounded search. Begin with the opening endpoint. Specify default and maximum page sizes, stable ordering, cursor binding, supported filters, projection, expansion cardinality, candidate and service-work budgets, response bytes, completeness/freshness, and exact-total behavior. Calculate the modeled 41-call versus two-call path and 204,800-byte versus 36,840-byte payload. Then add one adversarial tenant with a 100,000-ID filter and a request that changes sort order while reusing a cursor. Decide which requests are rejected, coerced, or moved to a job.

Make a reservation write safe to repeat. Define tenant/operation key scope, canonical request hash, retention, in-progress behavior, commit acknowledgment, deadline propagation, cancellation, unknown outcome, replay, conflict, overload, and status retrieval. Add a client timeout after durable commit but before response. Show why a new key could create a second reservation and why querying the original identity is safe.

Plan a mixed-version schema change. Add a quality_tier field whose absence must preserve the old behavior. Walk all four old/new writer-reader cells, a binary-to-JSON proxy, stored data, a delayed event replay, and rollback. Budget the 2.4-billion-record backfill with foreground and recovery headroom. State the evidence required before contraction.

Durable interface rules

1. Define the useful operation and its authority before choosing endpoint granularity or transport.
2. Bound every caller-controlled multiplier: items, bytes, nesting, filters, candidates, expansions, dependencies, duration, and side effects.
3. Treat pagination as a stable authorized walk with opaque continuation state, not a response-splitting trick.
4. Eliminate N+1 with bounded batching, owned joins, or explicit derived views; never hide it behind unlimited expansion.
5. Carry deadline, cancellation, idempotency, tenant, priority, and outcome semantics across the whole operation.
6. Evaluate compatibility across mixed readers, writers, stored data, translators, defaults, unknowns, and meaning—not only parser success.
7. Budget migration, replay, and bulk work as production demand with checkpoints, fairness, and recovery controls.
8. Make overload, partial completion, freshness, confidence, pending, and unknown outcomes machine-actionable.
9. Compare estimated and actual work per logical operation, then revisit bounds before clients depend on an unsafe shape.

The contract is now bounded, but the system still has to decide where each operation runs and which dependencies remain on its wait path. Part IV begins with that topology question: every new service boundary buys some combination of ownership, isolation, scaling, or change freedom by adding latency, failure, state, and coordination cost.

Evidence and transfer limits

• RFC 9110, HTTP Semantics defines resource, method, safety, idempotency, and Retry-After semantics. It does not define application idempotency-key retention, business completion, or query budgets.
• RFC 6585 defines 429 Too Many Requests, and RFC 9457 defines problem details. An API still needs stable domain codes, retry safety, scope, and authoritative status retrieval.
• gRPC deadlines and gRPC cancellation document one RPC ecosystem’s propagation and cooperative cancellation behavior. Transport cancellation does not undo a committed effect.
• AIP-158 and AIP-151 provide concrete pagination and long-running-operation patterns. Their field names and policy choices are ecosystem guidance, not universal protocol law.
• The Protocol Buffers proto3 language guide and best practices document wire evolution, presence, defaults, unknown fields, and mixed-version hazards for that format. Semantic compatibility and translation behavior still require application tests.
• The executable fixture in examples/performance-engineering-system-design-handbook/part-03/api-contracts/ reproduces 41 versus two modeled calls, 69 versus 38 ms teaching paths, 204,800 versus 36,840 bytes, 36.2 ms of modeled search work, a 37.04-hour backfill at 18,000 records/s, 1.296 TB of amplified teaching traffic, and 4,000 bulk chunks. These values are deterministic models, not observed API, search, storage, or network performance.