Skip to content
PracticalHandbooksSoftware Handbooks
Library

Practical handbook

Production Data Systems Handbook

A practical field manual for choosing, combining, evolving, debugging, and operating production data systems without cargo-culting database choices.

Start Reading Listen All books
A layered production data landscape connecting durable facts, storage, streams, analytics, and recovery.

Reading order

Contents

Part 1

Orientation: Stop Choosing Databases by Vibes

5 entries
  1. 01 The Real Shape of a Production Data System
  2. 02 Workload Fingerprints: Requirements Before Products
  3. 03 Invariants: The Facts That Must Survive
  4. 04 The Cost of a Database Choice
  5. 05 Data Models and Query Shapes

Part 2

First Principles: What Data Systems Are Really Doing

7 entries
  1. 06 Storage Engines: B-Trees, LSM Trees, Column Stores, and Object Storage
  2. 07 Indexes and Access Paths
  3. 08 Transactions, Isolation, and Concurrency Anomalies
  4. 09 Replication, Consensus, Sharding, and Multi-Region Reality
  5. 10 Time, Ordering, Idempotency, and Exactly-Once Myths
  6. 11 Caches, Derived Data, and Materialized Views
  7. 12 Schemas, Contracts, Metadata, and Meaning

Part 3

Choosing and Combining Systems: Fit the Tool to the Workload

10 entries
  1. 13 The Decision Room: A Repeatable Database Selection Process
  2. 14 Relational Systems: The Default You Should Understand Before Replacing
  3. 15 Key-Value, Document, and Wide-Column Stores
  4. 16 Search Systems: Text, Relevance, and Retrieval Are Not Just Filters
  5. 17 Event Logs, Queues, and Streams
  6. 18 Analytical Stores: Warehouses, Lakes, Lakehouses, and OLAP Serving
  7. 19 Time-Series Systems and Observability Stores
  8. 20 Graph Systems
  9. 21 Vector Stores, Feature Stores, and AI-Adjacent Data Systems
  10. 22 Polyglot Persistence Without Distributed Regret

Part 4

Designing Production Data Paths

7 entries
  1. 23 Modeling Writes: Commands, Transactions, Events, and Side Effects
  2. 24 Modeling Reads: Query APIs, Read Models, and Freshness
  3. 25 Data APIs and Service Boundaries
  4. 26 Multi-Tenancy, Identity, Authorization, and Data Isolation
  5. 27 Ingestion, CDC, Outbox, and Data Movement
  6. 28 Batch, Streaming, Reprocessing, and Reconciliation
  7. 29 Capacity Planning, Load Testing, and Cost Modeling

Part 5

Evolution and Migration

4 entries
  1. 30 Schema and Index Migrations Without Downtime
  2. 31 Data Migrations, Backfills, and Cutovers
  3. 32 Evolving Data Models and APIs Under Consumer Pressure
  4. 33 Retiring, Archiving, and Deleting Data Systems

Part 6

Debugging and Performance

6 entries
  1. 34 Observability for Data Systems
  2. 35 Query Performance and Execution Plans
  3. 36 Locks, Contention, Hot Keys, and Tail Latency
  4. 37 Data Correctness Bugs, Drift, and Reconciliation
  5. 38 Distributed Failure Debugging
  6. 39 Testing Data Systems: Properties, Faults, and Production Confidence

Part 7

Operations, Reliability, Security, and Governance

8 entries
  1. 40 SLOs, SLIs, and Error Budgets for Data Systems
  2. 41 Backup, Restore, Disaster Recovery, and Business Continuity
  3. 42 On-Call, Runbooks, Incidents, and Postmortems
  4. 43 Security, Privacy, Compliance, and Abuse Resistance
  5. 44 Data Quality, Governance, Lineage, and Ownership
  6. 45 Building a Data Platform: Paved Roads, Guardrails, and Escape Hatches
  7. 46 FinOps for Data Systems: Cost as a Design Constraint
  8. 47 Ethics, User Trust, and Organizational Decision Quality

Part 8

Case Studies: Applying the Framework

7 entries
  1. 48 Case Study: E-Commerce Catalog, Inventory, Cart, and Search
  2. 49 Case Study: Payments, Ledger, and Financial Correctness
  3. 50 Case Study: Event-Driven Notifications and Workflow
  4. 51 Case Study: Metrics, Logs, Traces, and Time-Series Explosion
  5. 52 Case Study: Analytics Warehouse and Revenue Reporting
  6. 53 Case Study: Multi-Region SaaS and Tenant Isolation
  7. 54 Case Study: AI/RAG Knowledge Base with Vector Search

Reference section

Appendices

10 entries
  1. Appendix A Appendix A: Glossary of Production Data Systems
  2. Appendix B Appendix B: Workload Fingerprint Worksheet
  3. Appendix C Appendix C: Invariant Register Template
  4. Appendix D Appendix D: Data System ADR Template
  5. Appendix E Appendix E: Data Contract Template
  6. Appendix F Appendix F: Migration Runbook Template
  7. Appendix G Appendix G: Data Incident Runbook Template
  8. Appendix H Appendix H: Recovery Plan Template
  9. Appendix I Appendix I: Comparison Matrices
  10. Appendix J Appendix J: Drill Library