Appendix A: Glossary of Production Data Systems

How to Use This Glossary

Use this glossary when a familiar word is hiding an unspecified promise. The definitions are grouped by the work they do in a production data system. Each gives the term’s ordinary meaning in this book, then names the consequence worth carrying into design or incident review. For the mechanism and its trade-offs, return to the chapter where the term is developed.

Correctness and Transaction Terms

ACID: Four properties used to reason about transactions: atomicity, consistency, isolation, and durability. Here, consistency means that a transaction takes the database from one valid state to another under the constraints it actually enforces; it does not name a distributed consistency model. In design review, ask which operations need which guarantees and whether the chosen store provides them at the required boundary.

Consistency model: The rules that describe what reads may observe after writes, especially under replication, concurrency, or failure. Do not accept “eventual consistency is fine” until the temporarily inconsistent facts are named.

Durability: The expectation that committed data survives process crashes, node failures, or other defined failure events. Durability is only meaningful when the failure boundary and recovery procedure are known.

Exactly-once effect: A user-visible or business-visible effect that occurs once even when delivery, retries, or worker execution occur more than once. It is an end-to-end property, usually assembled from idempotency, deduplication, transactions, and reconciliation rather than inherited from a transport guarantee.

Idempotency: The property that repeating an operation with the same semantic key produces the same intended effect. A good idempotency key describes the business action, not merely the transport message.

Invariant: A fact that must remain true across allowed system states. Invariants should have an enforcement location, monitor, owner, and repair path.

Isolation level: The degree to which concurrent transactions are protected from seeing or creating each other’s intermediate states. Isolation level matters when lost updates, write skew, phantom reads, or cross-row invariants can harm correctness.

Quorum: A minimum number of participants required to accept or observe a distributed operation. Quorum choices affect latency, availability, durability, and stale-read behavior.

Saga: A sequence of local transactions coordinated across services, often with compensating actions after failure. Compensation is another business action, not a guaranteed reversal of time: an email cannot be unsent and an observed price cannot be unseen. A saga therefore needs explicit state, retries, idempotency, and repair for outcomes compensation cannot erase.

Modeling and Query Terms

Aggregate: A cluster of related facts treated as a consistency or update boundary. Aggregate design should follow invariants and write patterns, not just object structure.

Cursor pagination: Pagination that advances from a position in a stable ordering, such as creation time plus a unique ID, rather than from a row offset. It avoids the growing scan cost of large offsets and reduces skips or duplicates as the result set changes, provided the ordering, direction, and tie-breaker are explicit.

Index selectivity: How effectively an index narrows a query. Low-selectivity indexes may add write cost without improving read performance enough to justify them.

Materialized view: A stored result derived from source data for faster reads or alternate access patterns. It needs freshness, rebuild, backfill, and ownership rules.

MVCC: Multi-version concurrency control, a mechanism that lets readers and writers operate over different record versions. MVCC improves concurrency but creates vacuum, version-retention, and long-transaction concerns in some systems.

Partition key: The field or fields used to distribute data across partitions or shards. A poor partition key creates hot partitions, cross-partition transactions, or expensive fan-out reads.

Read model: A derived representation optimized for reads, often separate from the write model. It should declare source truth, freshness, rebuild behavior, and deletion propagation.

Shard: A horizontal slice of data placed on a subset of infrastructure. Sharding changes routing, balancing, backup, restore, and cross-shard query behavior.

Source of truth: The authoritative system for a particular fact. A system can be the source of truth for one fact and a derived copy for another.

Movement, Streaming, and Derived Data Terms

Append-only log: A sequence of records written by appending rather than updating in place. Logs are useful for audit, event propagation, replay, and ordered processing, but they require retention and schema discipline.

Backfill: A controlled process that fills missing or newly derived data for existing records. Backfills need throttling, idempotency, observability, rollback, and correctness checks.

CDC: Change data capture, a way to publish data changes from a source system to consumers. CDC needs ordering assumptions, schema compatibility, snapshot handling, and replay rules.

Checkpoint: A recorded processing position that lets a consumer resume after interruption. Checkpoints should match the system’s idempotency and replay model.

Compaction: A storage-maintenance process that rewrites or merges segments, often discarding superseded records or reclaiming deleted space while preserving required state. It can reduce storage and read amplification, but consumes I/O and may remove history needed for audit or rebuild.

Consumer group: A set of consumers that share work over partitions or messages. Consumer group behavior affects ordering, parallelism, retry, and failure handling.

Data contract: An agreement about schema, semantics, freshness, quality, ownership, and compatibility between producer and consumer. It should include change policy and incident ownership.

Data lineage: The trace of where data came from, how it changed, and where it went. Lineage is essential for debugging wrong reports, privacy obligations, and impact analysis.

Dead-letter queue: A queue for messages that could not be processed after allowed attempts. It is useful only when it has an owner, triage policy, replay safety, and age limits.

Derived data: Data copied, transformed, indexed, cached, or aggregated from another authority. Derived data needs freshness, deletion, rebuild, and reconciliation rules.

Event time: The time when a business event occurred, distinct from processing time. Event-time handling matters for late data, windows, reports, and replay.

Freshness: The age or lag of a derived view relative to source truth. Freshness should be visible when users or systems can make bad decisions from stale data.

Operations and Reliability Terms

ADR: Architecture decision record. An ADR should capture context, decision, alternatives, consequences, review triggers, and reversal plan.

Backup: A recoverable copy of data. A backup is not proven until restore has been tested against the recovery target.

Bounded staleness: A guarantee or expectation that reads may be stale by no more than a stated amount. It needs measurement and user-facing implications.

Cache-aside: A caching pattern where application code reads from cache, loads from source on miss, then writes the cache. It needs invalidation, TTL, stampede control, and source-truth discipline.

Cardinality: A count whose meaning depends on context: rows in a relation, distinct values in a column, or distinct series created by a set of metric labels. State which one you mean. In indexing and observability, the number and distribution of distinct values can dominate memory, storage, and query cost.

Error budget: The amount of allowed unreliability implied by an SLO over its measurement window. The remaining budget can guide release and reliability decisions, but only when the SLI represents user harm and the measurement is timely enough to change action.

Hot partition: A partition receiving disproportionate load. Hot partitions cause tail latency, throttling, uneven cost, and sometimes correctness risk under retries.

Point-in-time recovery: The ability to reconstruct a store near a chosen time by restoring a base backup and replaying retained changes. It is useful after corruption, accidental deletion, or a bad migration only if the required logs are intact, the recovery point is granular enough, and the restore completes inside the recovery target.

Replica lag: Delay between a primary write and its visibility on a replica. Replica lag affects freshness, read-your-writes behavior, failover, and analytical correctness.

RPO: Recovery point objective, the maximum tolerable gap between the last recoverable state and the failure. It is commonly expressed as time, but should be translated into the business work that could be lost.

RTO: Recovery time objective, the maximum tolerable time to restore a named service or capability after disruption. The objective is incomplete unless the restored capability and its required data state are explicit.

SLI: Service-level indicator, a measured signal of service behavior such as successful writes, read latency, freshness, restore success, or data correctness.

SLO: Service-level objective, a target for an SLI over a window. A useful SLO maps to user or business impact and has an owner.

Governance, Security, and Lifecycle Terms

Data lifecycle: The path from creation through use, movement, retention, archival, deletion, and audit. Lifecycle design prevents stale, illegal, expensive, or unowned data copies.

Data residency: Requirements or promises about where data is stored, processed, backed up, logged, and accessed. Residency must cover derived systems, not only the primary database.

Least privilege: Access design that grants only the permissions needed for a role or system. It should apply to data stores, dashboards, exports, support tools, and operational scripts.

Retention: The period and form in which data is kept. Retention should follow legal, product, operational, analytical, and cost requirements rather than one default.

Schema evolution: Controlled change to data shape and meaning over time. Safe schema evolution accounts for old writers, old readers, backfills, compatibility, and rollback.

Tombstone: A deletion marker used to propagate or preserve deletion state. Tombstones need retention long enough for derived systems to observe and act on them.

Transactional outbox: A pattern that writes domain state and a message record in the same local transaction, then publishes the message asynchronously. It reduces dual-write risk but still needs publisher idempotency, ordering expectations, monitoring, and replay behavior.

Write amplification: Extra writes caused by indexes, replicas, materialized views, logs, compaction, denormalized copies, or derived systems. It affects latency, cost, storage wear, migration time, and incident blast radius.

Design Review Questions

When someone asks for “strong consistency,” name the invariant and the operations that can break it, then locate enforcement. When a design introduces derived data, trace source truth, freshness, rebuild, reconciliation, and deletion. For a stream or queue, walk through replay, duplicate delivery, reordering, late data, and a message that never succeeds.

Operational claims deserve the same conversion. Replace “we have backups” with the last successful restore, its duration, its recovered point, and its owner. Trace residency and access through every copy rather than stopping at the primary store. Replace “cost should be manageable” with the storage, traffic, query, label, index, replica, retention, or recovery dimension that grows the bill.

Field Reference

The glossary has done its job when the noun turns into evidence: an invariant with an enforcement point, a source of truth with a rebuild path, a freshness or recovery target with a measurement, an access boundary with an owner, a failure mode with a response, or a cost driver with a limit. If the term changes neither a design decision nor an operational check, it is still only vocabulary.