Caches, Derived Data, and Materialized Views

The Price Changed, but Where?

At 14:00, a pricing service changes a product from $49 to $59. Its transactional database commits the new price and revision 184. The write is correct. The system is not yet correct.

The product page has a cached response. Search has an indexed price for filtering and display. Recommendations use a price-band feature. Finance has a materialized daily total. A support export was generated before the change. Each copy can continue returning $49, and each can be doing exactly what it was built to do.

The copies do not all need to change at the same instant. Search may lag for a minute without doing harm if the detail page verifies the current price. A recommendation can tolerate a slower feature refresh. Yesterday’s orders must keep the price at which they were committed. Checkout cannot silently charge $59 after promising $49 unless the product has an explicit policy for that change.

This is the real design problem. A cache is not merely a faster place to find bytes. It is a derived data store with a bounded license to disagree with its source. Search indexes, read models, analytical aggregates, feature tables, and materialized views have the same obligation. For every copy, the team must be able to say which facts it represents, how wrong it may be, for whom, for how long, and how it becomes trustworthy again.

Authority Belongs to Facts, Not Databases

Calling one database “the source of truth” is rarely precise enough. In the price example, the pricing service owns the current catalog price. The order ledger owns the amount accepted for a completed order. A promotion service may own a temporary discount, while a tax service owns a jurisdiction-specific estimate. A rendered product response combines these authorities but does not replace any of them.

This distinction prevents a common category error. Derived data may reshape, denormalize, filter, rank, or summarize facts. It should not quietly acquire the authority to invent them. A product-page cache can repeat a price; it cannot decide which price a customer owes. A finance aggregate can summarize committed order lines; it should not reconstruct revenue from the mutable catalog price.

The same care applies to whole-response caching. A product response may contain title, price, availability, promotion eligibility, seller rating, localized tax, and personalized recommendations. Putting the response under one product:{id} key merges facts with different owners and different tolerances for age. The key gives no clue that an entitlement or legal-suppression change is more urgent than a corrected description.

Before choosing a cache product or refresh interval, split the response into facts. Decide which may travel together, which need a source version, which must be confirmed at a final action, and which should never be served from the same stale envelope.

Follow Revision 184

Suppose the price write and an outbox record commit together. The outbox event carries the product identity, price revision 184, the new amount, and the time semantics consumers actually need. From there, the revision takes several paths.

The product-detail path uses cache-aside. A request checks the cache, reads pricing on a miss, then stores the result. This is attractive because the cache is disposable and the application controls the miss. It also puts correctness in application code: the key dimensions, negative entries, concurrent loaders, serialization failures, and source behavior on a miss all belong to the design.

An explicit invalidation for revision 184 removes the old entry. A short time-to-live remains as a backstop in case that signal is lost. Neither mechanism is magic. The invalidation must name every affected key, and the TTL permits the old value until it expires. If price appears on product, category, merchant, and promotion pages, updating only product:42 leaves orphaned copies.

Versioned keys offer another boundary. A lookup for product:42:price:184 cannot accidentally return revision 183. But the reader must learn revision 184 before forming that key, and old versions still need cleanup. A generation can invalidate a wider group, such as all prices for one tenant or catalog release; changing it can also turn a warm cache into a synchronized wave of misses.

The search path consumes the price event and updates a document. This is a read model: a representation shaped for a query the write model does not serve well. The consumer must tolerate duplicate delivery, reject or quarantine an older revision, propagate deletions, and expose its lag. Event delivery changes the update mechanism, not the need for an invalidation rule. Revision 184 makes the indexed value for revision 183 untrusted.

The analytical path may maintain a materialized view: a stored result of a query or transformation. It may refresh after each committed order, on a schedule, or from a stream. Between refreshes it describes a particular cut of history, not “the data now.” A feature table is another materialization, shaped for model training or serving. Its contract must say which observation time a value represents and whether training and serving compute the feature by compatible rules.

Revision 184 should not rewrite completed orders. Finance derives revenue from committed order lines, not current catalog state. The same product therefore has two legitimate prices in the system: its current offered price and a historical transaction price. Chapter 10’s identity and time rules are what let these projections distinguish a correction from a new fact. Chapter 12’s schema and meaning rules will determine whether consumers continue to understand that distinction.

Choose an Update Path by Its Failure

Pattern names become useful once the consumer promise is known.

Cache-aside keeps loading in the application and works well for values that can be recomputed and may be briefly stale. Its characteristic danger is the miss path: many callers can reload the same hot key or turn source failure into repeated work.

Read-through puts loading behind the cache interface. It centralizes the behavior, but it can conceal whether one miss causes one source request, many retries, or a long wait. The source load and failure semantics still need to be visible to operators.

Write-through updates a copy as part of the source write path. Reads can become fresh sooner, at the cost of latency and coupling. If the source commits and the copy update fails, repair is required. If the copy changes before the source transaction commits, it may advertise a fact that never became authoritative.

Write-behind accepts a write into a cache or buffer and applies it to the eventual authority later. It can absorb bursts and shorten apparent write latency, but it changes the durability boundary. That trade can be reasonable for lossy telemetry counters and reckless for payment capture, inventory reservation, or legal deletion.

Refresh-ahead reloads popular values before expiration. It can smooth user latency, provided refresh concurrency and source load are bounded. Without those controls, popularity becomes an invisible background workload competing with live requests.

Batch and event projections inherit the failures of their transport. A batch can finish partially, miss a window, or make a backfill contend with production. An event consumer can see duplicates, gaps, disorder, poison messages, and replay. A synchronous update inherits the latency and rollback cases of the write path. Choose the path whose failures the consumer can survive and the team can observe, not the one with the most reassuring name.

Freshness Is a Consumer Promise

“Eventually consistent” says too little. The useful question is: which stale answer may this consumer receive, for how long, and what happens after the bound?

For revision 184, product browsing might tolerate revision 183 for a minute. Search might tolerate two minutes of lag if selecting a result leads to a fresher product page. Recommendations might use a feature computed hours ago. Checkout needs a different rule: revalidate against the price authority, honor revision 183 for a bounded reservation window, or ask the customer to accept revision 184 before committing. Finance may publish daily, but its report should state the completed interval and its correction policy.

Security, privacy, and deletion narrow the license further. A revoked permission, suppressed record, fraud block, or legal deletion cannot inherit a casual five-minute TTL merely because ordinary profile changes can. The derived store may need push invalidation, a source check at the protected action, or fail-closed behavior when freshness cannot be proven.

Read-your-writes is also a promise to a particular consumer. The actor who changed a value may need to see revision 184 immediately even while other users can receive revision 183. A session can carry the committed revision, bypass the cache until it observes that revision, or route the actor to a fresh path. These mechanisms are more precise than pretending every copy is instantly consistent.

Freshness must be observable. Hit rate alone cannot reveal whether successful hits are wrong. Operators need the age or source revision of served data, projection lag, event backlog, refresh failures, source load on misses, and the fraction of comparisons that drift. If users are the first lag detector, the promise is not being operated.

Empty, Cold, and Wrong

An empty cache is not automatically safe. If one hot key expires and thousands of requests load it together, the result is a stampede. Single-flight loading or request coalescing lets one bounded loader do the work while peers wait or receive a controlled fallback. Jittered expirations, per-key limits, and backpressure keep one key from overwhelming the source.

When many keys disappear together, the system faces a thundering herd. A broad flush, generation change, deployment, failover, or new region can transfer the full read workload to the source at the moment capacity is already constrained. Warmup, staggered expiration, load shedding, priority traffic, and a tested cold-path capacity limit matter more than a high steady-state hit rate.

A poisoned cache is more treacherous because it is fast and full. Omitting tenant, permission, currency, locale, experiment, or source revision from a key can serve a valid value to the wrong request. Caching a partial response, a transient error as “not found,” or data from an unsafe replica has the same shape. Success and latency graphs may improve while correctness gets worse.

Negative entries deserve particular care. Caching “not found” can hide a newly created object; caching “not allowed” can delay a permission grant; caching a source timeout as absence can turn an outage into apparently authoritative emptiness. Negative caching is useful only when absence has a precise meaning, key, and usually a shorter lifetime.

An inconsistent projection may be complete and still disagree with its source because an update was dropped, applied out of order, transformed under a changed schema, or overwritten during a partial backfill. A deletion leak is the high-consequence version: the source suppresses a record while search, analytics, exports, or model features retain it. Deletion and permission changes are first-class inputs to every derived path, not cleanup work for later.

These failures suggest drills that reveal the actual capacity and correctness model. Expire the hottest key under load. Empty the cache during a source slowdown. Delay revision 184 in the event path, then deliver revision 183 afterward. Insert a tenant-poisoned value. Remove a product from the source during a projection rebuild. For each drill, predict what consumers see and which limit prevents repair traffic from becoming the next incident.

Rebuild Is Part of the Write Path

A copy is safely derived only if the team can recreate it from facts that still exist. If a materialized table contains the only durable record of a computed decision, it has become authoritative for that decision, however disposable its name sounds.

A backfill is a production write workload. It needs deterministic or versioned transformation logic, idempotent writes, ordering rules, checkpoints, resource limits, progress visibility, validation, pause and resume behavior, and rollback. It must not resend emails, charges, webhooks, or other external effects merely because historical inputs are being replayed.

For a large projection, a common rebuild combines a snapshot with a change stream. Take a consistent source snapshot at a known position, load it into a separate destination, then apply changes after that position until the new projection catches up. The cutover is safe only if the team can show there is no gap, duplicates are harmless, newer revisions cannot be overwritten by older ones, and readers switch to a validated destination.

Drift detection supplies that validation over time. Depending on the contract, it may compare sampled records, revisions, counts, totals, checksums, tombstones, age distributions, or domain invariants. A mismatch must lead somewhere: rebuild a key, replay a range, quarantine the projection, suppress a consumer, block a cutover, or page an owner. A dashboard that reports drift without assigning a repair action is only a display of accumulating doubt.

Write the Derived Data Contract

The design record for a copy should be short enough to use and exact enough to operate. Write it around these decisions:

• Authority and contents: Name the owner of each source fact and the fields, filters, tenant boundaries, permissions, versions, tombstones, and sensitivity classes carried into the copy.
• Consumer promise: State completeness, staleness, ordering, and read-your-writes behavior per surface or decision. Say what happens when the promise cannot be met.
• Update and invalidation: Name the load, write, event, schedule, generation, source version, or hybrid trigger. Identify every source change that makes an existing value unsafe.
• Miss and failure behavior: Decide whether an absent, expired, corrupt, loading, or unverifiably stale value causes bypass, degradation, partial results, retry, queuing, fail-open, or fail-closed behavior.
• Load protection: Bound hot-key reload, broad invalidation, cold start, refresh work, replay, and backfill pressure against the source.
• Rebuild and drift: Record the snapshot boundary, replay range, checkpoints, transformation version, validation, drift signal, and concrete repair action.
• Security and lifecycle: Propagate permission changes, tenant isolation, suppression, deletion, retention, and privacy requirements to every copy.
• Ownership: Name the team that responds when the copy is slow, stale, wrong, expensive, unrebuildable, or unsafe.

Return to revision 184 and fill in that record for the product cache, search index, finance view, and recommendation feature. The contracts should differ. If they do not, the design has probably confused shared input with shared consequences.

Break One Copy on Purpose

Choose one derived store in the price system. Begin with the consumer and decide which stale answers it can accept. Then specify its key dimensions, update trigger, invalidation rule, source version, miss behavior, overload limit, drift monitor, rebuild procedure, and owner.

Run three thought experiments: flush it during peak traffic; delay the price event until after the freshness bound; and inject revision 183 after revision 184. Add a fourth if the store carries tenants, permissions, or personal data: let an invalidation reach every copy except one.

For each case, name what the user sees, what the operator observes, which fact remains authoritative, and what action restores trust. A derived store is ready when being empty, late, or wrong leads to an intentional system response rather than surprise. The next chapter takes up the contract underneath that response: whether every producer and consumer still agrees on what the copied facts mean.