Invariants: The Facts That Must Survive

Twelve Units, Fourteen Reservations

A warehouse has twelve units of a product left. Two checkout requests arrive together, each asking for seven. Both workers read twelve available units. Both decide that seven can be reserved. If they commit independently, the system has accepted fourteen reservations against twelve units.

The problem is not that the database was insufficiently “consistent” in the abstract. The design failed to protect a specific fact:

Confirmed reservations for a SKU must never exceed its sellable stock.

That fact is an invariant: a condition that must hold across every state the system is allowed to commit. It has to survive concurrency, retries, worker crashes, imports, backfills, support corrections, and failover—not only the ordinary request path.

Once the fact is visible, the design conversation becomes useful. The team can ask where competing reservations meet, whether one authority can serialize them, whether stock can be partitioned by SKU, and what evidence will reveal corruption caused by a bypass path. “Use strong consistency” answers none of those questions by itself.

Other invariants protect different kinds of truth. A username belongs to at most one active account. A settled ledger transaction has equal debit and credit effects. An order cannot ship before payment is captured or explicitly waived. A deletion request must reach every governed copy by the promised deadline. Consistency requirements follow from such facts; they do not precede them.

Write the Fact Before the Mechanism

An invariant should be written as a state claim, not as an implementation wish. “Use transactions” is not an invariant. “Never oversell a confirmed SKU” is. “Use Kafka exactly once” is not an invariant. “A payment capture command may create at most one settled charge for an order” is.

A useful statement identifies its subject and rule, then makes three limits explicit. The boundary says which records, partitions, services, regions, or copies participate. The time condition says whether the rule must hold at commit, may converge within a bounded window, or must be satisfied by a deadline. The harm says what happens when it is violated.

“A payment capture command creates at most one settled charge for an order” is much better than “payments must be consistent.” It identifies the order and charge, names the forbidden duplicate effect, and gives the team something to test under retries and timeouts. The team still has to add the boundary: its own records, the provider request, incoming webhooks, and settlement data all participate.

Mechanisms are not interchangeable. A NOT NULL constraint, serializable transaction, partition-local conditional write, idempotency record, reconciliation job, and human review queue protect different shapes of fact. Write the fact first so that the mechanism has something precise to answer.

The Boundary Decides What Can Enforce the Rule

Local invariants fit inside one record or one aggregate. An email field must be normalized. A status must move through allowed states. A timestamp must not be earlier than the record’s creation time. These can often be enforced with schema constraints, validation, state-machine checks, or conditional writes.

Cross-entity invariants span rows, documents, records, or partitions. A user can have only one active trial. An order total equals its line items and adjustments. Confirmed reservations cannot exceed stock across competing carts. These facts may require a transaction, unique constraint, lock, compare-and-swap operation, escrow-like allocation, or partition design that puts the contested state under one authority.

Cross-system invariants span services or asynchronous flows. A payment captured by one provider must correspond to one order. A shipped order must have a fulfillment record. A user deletion request must remove or anonymize data across operational stores, exports, logs, and analytical tables. These invariants rarely fit inside one database transaction. They need idempotency, outbox patterns, reconciliation, audit trails, and sometimes human review.

Geography enlarges the boundary again. A unique coupon redemption, booking slot, username, or account balance becomes harder when several regions accept writes independently. The design might route writes for that fact through one authority, partition the fact so that it becomes tenant-local or SKU-local, reserve portions of capacity in advance, or accept a named conflict with a repair path. The invariant should shape the topology; the topology should not quietly redefine the invariant.

The boundary of an invariant is not the same as the boundary of a team or service. If support tooling, migration scripts, batch imports, or analytics exports can change or expose the fact, they are inside the invariant’s operating boundary even when they live outside the main application.

Decide What May Be Temporarily Wrong

Not every undesirable state needs synchronous prevention. The distinction depends on harm and time, not on a general preference for stronger or weaker consistency.

Some states are forbidden. A ledger entry that does not balance is not merely stale. Two accounts claiming the same external identity can break authorization, billing, and support. These facts belong on the write path, where a constraint, transaction, conditional write, lock, or single authority can reject the invalid transition.

Other observations may be temporarily old while the underlying authority remains correct. A search result can lag an order update, a dashboard can wait for late events until its watermark closes, and a recommendation system can use yesterday’s features during a backfill. This is a bounded-staleness contract only when the window is explicit, visible, and prevented from influencing decisions that require current truth.

Some violations are compensatable. A notification can be resent. A duplicate event can be neutralized before it creates a second business effect. A shipment exception can enter a review queue. A customer can receive a credit after an overcharge if the business has deliberately accepted that harm and the correction is reliable. Compensation is not permission to be vague: it requires detection latency, repair authority, evidence, and an owner.

Finally, some rules are advisory. They improve quality but do not protect money, access, safety, compliance, or trust. Keep them as validation or cleanup rules rather than inflating them into production-critical invariants.

“Eventual consistency is fine” becomes a decision only after the team can name what may differ, who can observe the difference, how long it may last, which action must still consult current truth, and what happens if convergence fails.

Prevention Is Not Detection, and Detection Is Not Repair

Enforcement prevents an invalid state from being committed. Detection proves that the system has stayed inside the allowed state space or notices when it has drifted. Repair moves the system back to an acceptable state and leaves enough evidence that operators, auditors, and users can trust what happened.

Keeping those jobs separate prevents a common design error: treating a monitor as if it were a lock. An alert that reports negative inventory after orders have shipped may be useful for incident response, but it did not protect the invariant. Conversely, a database constraint may prevent corruption but still need detection around bypass paths, failed migrations, and derived stores that can drift.

Prevention belongs as close as practical to the state being protected. Database constraints, foreign keys, exclusion constraints, transaction isolation, conditional updates, compare-and-swap, unique indexes, state-machine guards, and single-writer ownership can all make invalid writes fail.

Detection belongs where drift can appear. Reconciliation queries, duplicate-effect checks, control totals, source-to-derived comparisons, queue lag, watermark freshness, audit trails, and sampled workflow checks tell the team whether the fact still holds across time and copies.

Repair belongs in an explicit workflow. Automated compensation, replay, rebuild, rollback, data correction scripts, shipment holds, refund flows, access revocation, and human review queues should have owners and safety checks. A repair path that exists only in an experienced engineer’s memory is not part of the system.

One Checkout, Several Contracts

The twelve-unit race is only one part of checkout. A confirmed order needs one stable order ID, even if the client retries after losing the response. That fact is local or partition-local: generate the ID once, enforce uniqueness, and make the command idempotent.

Inventory has a different boundary. Confirmed reservations compete over sellable stock for a SKU, so the write path needs a transaction, conditional decrement, or partition-local allocator. An abandoned cart may hold stock temporarily, but its reservation needs an expiry time and a release job. The system protects the confirmed-sale invariant without pretending that every cart is permanent.

Payment crosses a boundary the order database cannot lock. The desired effect is still strong—one order must not produce two settled charges—but protection is assembled across the provider request ID, idempotency key, durable handoff, webhook deduplication, audit record, and settlement reconciliation. No single component can prove the entire fact.

Shipment introduces a transition rule: an order cannot move from paid to shipped without a fulfillment decision. The state machine and authorization guard can reject an invalid transition, while an audit trail can show who or what made the decision.

Search is different again. Its order status may lag, provided the lag is measured and critical workflows read authoritative state. Daily revenue reporting may accept late arrivals until a named close, then compare warehouse totals with settled payments and route differences to finance review.

Calling the whole checkout system “strongly consistent” would obscure these choices. Calling it “eventually consistent” would be worse. The system contains several facts, each with its own boundary, time condition, harm, enforcement, detection, and repair.

The Application Check Is Only One Writer

Application code often has to participate in invariant protection. Domain rules live there, and many databases cannot express every business transition. The risk is pretending that a check in application code is enforcement when other writers can race or bypass it.

The classic example is uniqueness. Two workers both execute SELECT and find that a username is free. Both then insert. The preflight check was useful for a friendly error message, but it was not the invariant. The unique index or equivalent serialized write is the invariant boundary.

The same pattern appears in more expensive forms:

• A service checks account balance before debiting, but concurrent debits commit in separate transactions.
• A job emits an event after a database write, but the process crashes between commit and publish.
• A consumer checks whether it has processed an event, but the check and side effect are not atomic.
• A support tool edits state without the state-machine transition used by the application.
• A migration script backfills derived columns without running the invariant checks used by live writes.

Application validation is still valuable. It improves user feedback, keeps bad requests away from storage, and expresses domain intent. It becomes production-grade only when the final write path, retry path, administrative path, and repair path preserve the same fact.

Keep a Register That Can Change the Architecture

An invariant register is the bridge between the workload fingerprint and technology choice. It is not a catalog of every validation rule in the codebase. It records the facts whose violation would cause money loss, security exposure, legal risk, customer harm, broken reporting, or operational confusion that cannot be repaired casually.

Use a short record that can be copied into a design review and challenged there:

Entity or flow: Checkout inventory reservation
Invariant: Confirmed reservations cannot exceed sellable stock for a SKU
Scope: SKU-local inventory authority and active confirmed reservations
Time and harm: Forbidden at confirmation; oversell creates fulfillment failure
Enforcement: Conditional update plus an idempotent reservation command
Break paths: Concurrent checkout, retry, import, manual correction, expiry bug
Detection: Negative-stock query and reservation-to-stock reconciliation
Repair: Hold shipment, release invalid reservation, audit stock correction
Owner: Inventory service owner; commerce on-call escalation
Test evidence: Concurrent reservations, duplicate command, import, repair drill

A serious register usually has fewer rows than a requirements document and more operational detail than an architecture diagram. It should affect the design. If the register says a fact is strong but the proposed data store cannot enforce it without cross-partition races, the architecture review has found a real trade-off. If the register says a fact is compensatable but there is no reconciliation query or repair owner, the system is accepting drift it cannot manage.

Challenge the Paths Around the Main Path

Before approving a transaction, messaging, cache, or multi-region design, try to break each registered fact. Send two writes together. Repeat a command after its response disappears. Crash the worker between the database commit and the external effect. Run a stale import. Bypass the service with a support tool. Fail halfway through a migration. Restore a backup that predates a correction. Let a derived view stop consuming updates.

Then ask three questions. What prevented the invalid state? What would detect it if prevention failed? What authorized, observable action would restore trust? A dashboard is not enforcement. “We can fix it manually” is not repair unless the operation has known inputs, safety checks, audit, and an owner.

Do not over-strengthen facts that do not need it. Making every read globally fresh can add latency, cost, and outage sensitivity without protecting the consequential state. A profile badge may tolerate lag; a permission revocation may not. Precision is knowing which is which.

Practice: Make One Fact Change the Design

List five checkout facts covering inventory, payment, order status, shipment, and refunds. For each, write its boundary, time condition, harm, enforcement, detection, repair, owner, and one failure test. If the only enforcement is “the service checks before writing,” add the concurrent write, retry, migration, or operator-tool path that defeats it.

Choose the most consequential fact and complete one invariant register. The exercise is finished only when the record changes the proposed system: a constraint is added, a transaction boundary moves, writes are routed through one authority, a derived store is demoted from authority, a reconciliation job gains an owner, or a database option is rejected.

That last possibility matters. Preserving an invariant has a cost shape: latency, coordination, operational burden, recovery work, and design gravity. Once the facts are explicit, the next architecture question is no longer whether correctness matters. It is what each viable way of protecting correctness will cost.