Caching and Content Delivery

The cache is fast while the product is wrong

At noon, a limited-release product goes on sale. The product page is already warm in browsers, at the CDN, and in the application cache. The first minute looks excellent: 98 percent of requests are cache hits.

Yet customers report three different failures. Some see yesterday’s price. Some see the right product in the wrong currency. At the turn of the minute, the database saturates even though the cache dashboard still shows a high aggregate hit rate.

These are not three unrelated cache bugs. The system never made a precise promise about reuse. It cached “the product page” without deciding which parts of the answer could be shared, which request inputs changed them, how a write displaced old copies, or how one popular miss would reach the origin.

A cache stores an answer produced in the past and offers it to a request in the future. Its contract therefore has six parts:

object + key + visibility + freshness + invalidation + failure behavior

Speed is the result when that contract is sound. It is not part of the proof that the answer is safe to reuse.

One page contains several kinds of truth

Before choosing a cache, split the response by ownership and consequence. The launch page contains at least five different objects:

• content-hashed JavaScript and CSS, which never change at the same URL;
• public images, which can live near users for a long time;
• descriptive copy, which may be several minutes old without causing harm;
• a localized price, which varies by market and must change promptly;
• availability and checkout state, which must not authorize a sale from a stale observation.

One page-wide TTL cannot express those promises. The static assets can be immutable. The description can be a public CDN object. The price needs market in its key and a deliberate invalidation path. The cart is private. The final reservation belongs to the source of truth protected by the transaction rules from the previous chapter.

This separation is the first important caching decision. A response should become one cached object only when its fields share the same visibility, freshness, and failure policy. Otherwise, compose the page from objects with different contracts or keep the sensitive part out of shared caches.

Follow a read through every layer

Suppose a customer in Kenya requests product 42. The request can encounter four independent reuse decisions:

browser -> CDN edge -> application cache -> product service and database

The browser may reuse a private copy without making a network request. The CDN may serve a shared response from Nairobi. The application may find a serialized product projection in a distributed cache. Only after all three misses does the product service read its database and rebuild the answer.

The word “hit” is incomplete without a layer. A CDN hit prevents work at the application. An application-cache hit still consumes CDN-to-origin capacity. A browser hit is invisible to server request metrics. During an incident, trace the request from the outermost cache inward and ask at each layer:

• What exact bytes or object are stored?
• Which requests are allowed to share them?
• How does this layer know the copy is fresh?
• What makes the copy unreachable after a change?
• What happens when this layer misses or cannot answer?

The source of truth must remain capable of rebuilding every disposable cache. “Disposable” describes ownership, not operational impact. Losing a cache can still bring down the database if every request falls through at once.

The key decides who shares an answer

A cache key is a compressed description of everything that changes the response. For the public product description, a useful key might begin as:

product-description:{product_id}:{locale}:{content_version}

For a regional price it may need market and currency:

product-price:{product_id}:{market}:{currency}:{price_version}

A key of product:42 aliases those answers. It can serve the Kenyan price to a customer in another market, or preserve a value from before a price change. Adding every request property is not the solution either. Keying a CDN entry by the entire cookie header or by unnormalized tracking parameters produces thousands of objects that contain the same answer and destroys reuse.

The design task is to include every semantic input and exclude incidental noise. Common semantic inputs include tenant, user, permission version, locale, currency, experiment bucket, query filters, pagination position, and representation format. Query order and irrelevant tracking parameters can be normalized before lookup. A Vary response header tells an HTTP cache which request headers select different representations, but it cannot rescue an application that has not identified those dimensions correctly.

For authenticated data, key design is also access control. A profile cached as profile:{user_id} is unsafe if the same user belongs to multiple tenants or if a role change alters the response. Possible answers include a key that contains tenant and permission version, a private device cache, or no cache at all. One cross-tenant response is not an acceptable price for a better hit rate.

Freshness is a product promise

A time-to-live answers one narrow question: how long may this layer reuse an entry without checking elsewhere? It does not prove that the value remains correct for that long.

The release page illustrates several useful freshness mechanisms. A hashed asset URL such as /assets/app.7f3a91c.js is versioned by content, so it can have a long immutable lifetime. The product description may have a five-minute TTL plus a content-update-triggered purge. A price can carry an explicit version and be invalidated immediately after the committed change. Checkout can bypass the page cache and ask the reservation owner for a current decision.

Validation is another option. An HTTP cache can send an ETag in a conditional request; the origin returns a small “not modified” response when the representation still matches. Last-Modified provides a time-based validator when its precision and semantics are sufficient. Validation saves transfer, although it still requires a round trip and some origin work.

Two HTTP directives are often confused. no-store forbids storage and is the right starting point for secrets, tokens, and other highly sensitive responses. no-cache permits storage but requires successful revalidation before reuse. private prevents shared caches from storing the response while still allowing a user’s private cache to do so. These instructions apply only when every intermediary honors them; application and CDN configuration must agree with the response headers.

Staleness may be a feature when its boundary is explicit. A CDN can serve an old public description briefly while one request revalidates it. The same policy is usually unsafe for authorization, quota enforcement, money movement, or an irreversible command. “Serve stale” needs both a duration and a list of paths where the product permits it.

A write must defeat every old copy

Consider the price changing from 100 to 90. The database transaction commits the new price; then the write path deletes the application key and purges the CDN object. That is better than waiting for TTL, but the order still contains a race:

R1 misses the cache and reads price 100
W1 commits price 90 and deletes the old cache entry
R1 finishes its slow work and fills the cache with price 100

The invalidation happened, yet an older read survived it. A cache-aside design must account for fills racing with writes, not merely remember to delete a key.

Versioned keys are one strong answer. The write advances the authoritative price version; a reader can publish its result only under the version it read, and new requests select the new version. The old entry may remain until eviction without being reachable. Other systems use compare-and-set on a versioned cache record, serialize fills for the key, update the cache in the write path, or accept a bounded stale window because the product consequence is small. The mechanism must match the promise.

The common application patterns differ mainly in ownership and timing:

• In cache-aside, the application reads the source on a miss and fills the cache. It is simple and keeps the database authoritative, but it exposes miss stampedes and fill-versus-write races.
• In write-through, the write path updates the source and cache as part of its visible work. Read-your-writes can improve, but mutation latency and failure handling now include the cache. Without one atomic boundary, the system still needs an order and a repair path for partial success.
• In refresh-ahead, background work rebuilds predictable hot entries before expiry. It moves latency away from users while spending capacity on guesses.
• In write-behind, the cache accepts a change before durable persistence. That is no longer merely a read optimization: loss, ordering, replay, and recovery become part of the write contract.

Negative results also need a contract. Caching “product not found” for a short period can protect the database from repeated invalid IDs. A long negative TTL can hide a product created seconds later. Cache absence only when its lifetime and invalidation are understood.

One expired object can become a load event

Return to noon. The launch object has a 60-second TTL, so every copy created during prewarming expires at almost the same instant. Thousands of requests miss. Each begins the same database query and rendering work. The cache is empty precisely while the origin has the least spare capacity.

This is a cache stampede. A per-key single-flight mechanism lets one request rebuild while other requests wait for that result. If the product permits it, stale-while-revalidate lets those other requests receive the previous copy instead. TTL jitter spreads unrelated expirations over time. Scheduled warming can prepare a known launch object, and a CDN origin shield can ensure that many edges share one upstream fill.

Every control needs a failure decision. If the one fill times out, do waiters receive stale content, a reduced page, a bounded error, or another attempt? If the lock holder dies, how does the fill lease expire? If rebuilding takes longer than the TTL, the policy is unstable even though it looks protected on paper.

A hot key is related but not identical. It receives disproportionate traffic even while warm. One distributed-cache partition, network path, or application process can saturate under that single key. Local read-only copies, replication, request coalescing, CDN shielding, and product degradation can help, but only if updates and memory cost remain manageable.

Aggregate hit rate conceals both failures. A service can report 98 percent hits while the remaining 2 percent are simultaneous, expensive reads for the same object. Measure origin requests, fill concurrency, fill latency, stale serves, and errors by route and hot key. A cache protects an origin only when the origin’s load confirms it.

Capacity and eviction complete the picture. When memory is full, the cache evicts entries according to its policy; a workload larger than useful capacity can churn so quickly that fills consume more work than reuse saves. Track evictions, memory pressure, object size, and hit rate together. Adding cache nodes does not repair an unsafe key or a synchronized expiry.

Browser and CDN caches extend the deployment boundary

Browsers and CDNs are valuable because they avoid both distance and origin work. They are also outside the application’s immediate control, so URLs and headers must make old and new deployments coexist safely.

Static assets should normally use content-hashed names:

/assets/app.7f3a91c.js   Cache-Control: public, max-age=31536000, immutable
/index.html              Cache-Control: no-cache

Changing the JavaScript creates a new URL. Old HTML can still load the old asset, and new HTML points to the new one. Replacing /app.js in place while giving it a year-long TTL traps some clients on bytes the server no longer intends to serve.

A CDN’s cache key commonly includes host, path, selected query parameters, and selected headers. Its configuration must decide how cookies, authorization, compression variants, locale, and redirects affect reuse. Purge-by-tag or surrogate key can invalidate a group such as all fragments of product 42, but purge propagation is a distributed operation. Immediate global correction may require a versioned URL or an authoritative path rather than an assumption that every edge has already heard the purge.

Origin shielding reduces duplicate fills from many edges. Serving stale on origin failure can keep public content available. Neither should extend to a personalized or correctness-critical response merely because the CDN supports the feature.

Edge computation should narrow the request, not own the truth

Code at the edge can normalize query parameters, choose a public image variant, redirect an old URL, attach routing metadata, reject obvious abusive traffic, or decide whether a public cached representation matches the request. These operations benefit from proximity and need little coordinated state.

The edge is a poor owner for a final authorization decision, inventory reservation, payment transition, or workflow that spans several mutable records. Region-local state, restricted runtimes, deployment propagation, and limited observability make strong coordination harder. An edge function may verify a self-contained credential or reject an invalid request early, while the authoritative service still owns current entitlement and mutation rules.

“Move it to the edge” is therefore a placement proposal, not a correctness argument. Name which data the function reads, how fresh it is, which secrets it needs, how code and configuration roll out, and what happens when one region runs a different version.

Decide what happens without the cache

The launch design is incomplete until it handles four conditions separately:

• an ordinary miss, which may read and fill from the owner;
• a hot miss, which needs bounded fill concurrency;
• a cache timeout or outage, which must not redirect unlimited load inward;
• an origin failure, which may permit stale public content but not stale authorization or sale decisions.

Fail-open and fail-closed are consequences, not cache defaults. Serving an old product photograph during an origin outage is a useful degradation. Serving an old permission decision may disclose data. Treating a rate-limit cache as absent may admit an attack; treating every user as over limit may create an outage. The owning product and security rule decides.

Protect the source with short cache-operation deadlines, bounded origin concurrency, admission control, and a degraded response that has been chosen before the incident. Do not let each application instance independently retry an unhealthy cache and then an unhealthy database. Retries can turn the loss of an optimization into a system-wide amplification loop.

Useful evidence connects reuse to product behavior: hit and miss rates per layer; origin request rate; cache latency and timeouts; fill duration and concurrency; purge delay; stale responses; evictions and memory pressure; and the age or version of values users actually receive. Synthetic tests should also request the same route as two tenants, markets, and permission states. Performance telemetry cannot discover a privacy alias by itself.

Reconstruct the launch before adding a cache

The launch incident can now be explained without naming a vendor. The old price survived because the write did not defeat every reachable version. The wrong currency appeared because the key omitted a semantic input. The database spike came from synchronized expiry and unbounded fills for one hot object. The high hit rate was true and still misleading.

For any proposed cache, write the contract in one paragraph: the exact object, the inputs in its key, who may share it, its freshness rule, the write or version that invalidates it, and the behavior on a miss, hot miss, cache failure, and origin failure. Then trace one read and one concurrent write through every layer. If the answer becomes vague at a boundary, that boundary is where the design work remains.

Caches make old answers useful. The next chapter considers a different move across time: handing new work and durable facts to queues, logs, and stream processors without losing ownership of delay, duplication, or failure.