Evolving Data Models and APIs Under Consumer Pressure

The Field That Looked Redundant

A billing platform emits a subscription event with both plan_name and plan_id. The name predates the identifier. It is a display label, it changes occasionally, and it should never have become a join key. The producing team has corrected its own code and now wants to remove it.

The diff is one line. The change is not.

Entitlements, revenue recognition, lifecycle email, fraud detection, and a warehouse sink consume the event. Dashboards group warehouse rows by plan_name. A support script prints the name beside disputed invoices. A monthly compliance export reads the corresponding warehouse column through a shared service account. Some consumers use the name only for display; others have quietly treated it as identity.

Chapter 31 ended with a demand for attributable silence before an old path can disappear. This chapter begins where that demand becomes uncomfortable. A published data shape does not belong only to the team that emits it. Its consumers may include services, mobile clients, dashboards, notebooks, alerts, support tools, search pipelines, exports, and replay jobs. The producer sees cleanup. Each consumer sees a promise on which some decision or operation may depend.

That promise is larger than field names and types. It includes meaning, units, identity, null behavior, freshness, ordering, authorization, error behavior, and the point in time at which a value becomes reliable. A field can keep the same name and type while changing from “logged in during the last 30 days” to “performed a billable action during the last 30 days.” Every schema check passes; retention reports and lifecycle campaigns quietly answer a different question.

Safe evolution therefore starts with four populations, not one deployment: old and new producers, and old and new consumers. They coexist during rolling releases, delayed messages, retries, replay, regional failover, cache refill, backup restore, and slow client upgrades. The work is to preserve the business promise across those combinations, discover who relies on it, help them move, and prevent fresh dependencies on the contract being retired.

Name the Promise Before Changing It

For the billing change, “remove plan_name” is not yet a useful contract description. The field appears in a subscription event, the warehouse table derived from it, a customer billing API, and a finance export. Those surfaces have different readers, release cycles, and enforcement mechanisms.

The producer writes the intended meanings before choosing a versioning strategy:

• plan_id is the stable identifier of the billable plan and is safe for joins;
• plan_name is a mutable display label and is not identity;
• an event must identify the plan that governed the subscription at the event’s effective time;
• retries of the same business event must preserve that identity;
• consumers that need display text must resolve the identifier against a versioned plan dimension or an explicitly historical label.

This description exposes a problem that syntax alone concealed. A dashboard grouped by plan_name may merely need display text. Revenue recognition may have used it to identify a plan, which makes the migration a correction to a business invariant. The two consumers cannot be moved by the same search-and-replace.

Do the same work for any surface under change. State which fields and values may be absent; what null means; which identifier survives merges, deletes, and restores; whether a timestamp is event time, processing time, or effective time; what ordering and freshness are promised; how authorization is applied; and how a consumer distinguishes missing data from producer failure. “The schema” is too vague for production review.

The surface also determines how much control the producer has. A typed API can reject an incompatible request. A schema registry can block some incompatible events. A warehouse table read by copied SQL and shared accounts needs query attribution, naming discipline, and access policy. A downloaded CSV has almost no technical control once it leaves the system, so its replacement and communication path must carry more of the burden.

Discovery Must Outlast the Quiet Period

The billing team begins with the consumers it knows and then tries to disprove its own list.

Event infrastructure supplies consumer groups, offsets, lag, dead-letter queues, connector configuration, and replay jobs. These records find stable subscribers, but not necessarily batch readers that wake briefly or old jobs that replay from object storage. API logs add authenticated client identifiers, endpoint metrics, traces, user agents, and error reports. Warehouse query history adds query fingerprints, dashboard refreshes, scheduled extracts, notebooks, and downstream tables. Repository search, orchestration metadata, service-account activity, runbooks, support scripts, and credential inventories reveal paths that no lineage catalog captured.

Each signal has a blind spot. Shared warehouse accounts hide owners. Sampling can erase quiet API clients. A copied query loses its lineage. A consumer group says that something reads, not what business obligation it serves. A dashboard with no views may still feed an executive export. Discovery should therefore produce an accountable owner, business purpose, cadence, consequence of failure, migration state, and exception decision—not merely a technical identifier.

This is how the monthly compliance export changes the plan. Three quiet weeks could look like zero use. The export’s business rhythm is monthly, and its consequence is larger than its traffic. The observation window must cover that cycle or the team must obtain positive evidence from its owner. Disaster-recovery replayers, quarter-end reports, annual tax jobs, incident tools, and partner clients deserve the same skepticism. The more irreversible the removal, the less “we saw no traffic” can stand on its own.

Unknown ownership is a finding, not permission. A key named legacy-client or an account named svc_analytics is evidence of an unresolved consumer. The producer can trace provisioning records, query fingerprints, repositories, and team contacts; isolate the credential; or use a reversible warning or denial to elicit an owner. It cannot turn missing attribution into confidence by renaming it “no known consumers.”

Reason Through Mixed Versions

Compatibility names which combinations may coexist safely.

Backward compatibility means a new producer can still serve an old consumer. Adding an optional event field is often backward compatible when old readers ignore unknown fields. It is not when they reject unknown fields, assume an exact field set, fail on larger payloads, or interpret missing and null values differently.

Forward compatibility means a new consumer can still read old output. A new consumer that requires plan_id will fail on old events, partially backfilled rows, delayed messages, restored snapshots, or old producer instances unless it has a fallback. Adding a field does not make requiring it safe.

Full compatibility means old and new producers and consumers can all coexist. Production often needs this during a migration because there is no instant when every binary, row, cache entry, event, export, and backup changes together. A breaking change is one for which the coexistence cannot be made safe. Breaking work may be justified, but it needs an explicit boundary, migration, enforcement sequence, and risk owner.

For the billing event, walk the four combinations:

1. The old producer and old consumer preserve today’s behavior, including any misuse of plan_name that the migration must expose.
2. The new producer must continue emitting the old field while old consumers migrate, even though its own code no longer needs it.
3. The new consumer must tolerate old events without plan_id, either by a bounded historical mapping, a backfill, or an explicit rejection path that prevents wrong accounting.
4. The new producer and new consumer must agree that plan_id represents the plan effective for that subscription event, not merely the plan’s latest value.

The off-diagonal combinations usually carry the surprise. A replay job may send five-year-old events to new code. A backup restore may reintroduce rows created before the new field. A mobile client may be unable to understand a newly required response shape. A new producer may emit an enum value that an old consumer treats as impossible. Compatibility review follows data through time, not merely services through deployment.

Meaning changes deserve the presumption that they are breaking. If a reporting column named activated_at changes from “created a first project” to “invited a teammate and completed setup,” dashboards continue to render while experiments, campaigns, and forecasts move. The safer design introduces fields whose names preserve both meanings, runs important reports side by side, and lets a domain owner decide which measure governs future decisions. An error is easier to notice than a plausible chart with a changed denominator.

Choose the Smallest Version Boundary That Tells the Truth

Versioning identifies the contract a consumer is using; it does not discover consumers or make incompatible behavior safe.

Field-level evolution fits the billing event because the new identity can be introduced additively. The producer emits plan_id before any consumer requires it and preserves plan_name during migration. A semantic name such as billing_plan_id will age better than plan_id_v2; migration history is not a durable business meaning.

A new event type or event version is justified when meaning changes so far that readers need explicit handling. It brings a cost: producers, consumers, replay, backfills, observability, and retirement must now account for both contracts. Endpoint versions are useful when clients cannot move together, but every active version needs an owner, support window, usage telemetry, and removal criteria. A major semantic-version number by itself protects nothing if the old endpoint vanishes before its consumers move.

Version warehouse tables or views when old and new definitions need to coexist for historical reporting, but do not let compatibility views become permanent through neglect. A schema registry is valuable when it blocks incompatible publication, records ownership, and evaluates the compatibility mode consumers actually need. A catalog that stores schemas but cannot connect them to usage is documentation, not consumer governance.

Before adding any version, ask what it identifies, whether mixed versions can coexist, who operates each supported form, how its use is measured, how consumers test it, and what evidence permits its retirement. Version proliferation is often ownership deferred into architecture.

Deprecation Changes the Evidence, One Step at a Time

The billing team can now run a controlled deprecation rather than choose a deletion date and hope.

First, it introduces plan_id, documents both meanings, publishes representative fixtures, and keeps plan_name. Contract tests assert presence, null behavior, retry identity, and effective-time semantics. The warehouse sink carries both fields into a plan dimension that can supply display labels.

Next, it observes use by consumer group, query fingerprint, dashboard, export, and credential. The list joins technical evidence to owners and business purpose. Revenue recognition dual-reads the old and new identities and compares plan totals. Entitlements moves directly to plan_id. Dashboards join against the dimension. The support script keeps a label for display but stops using it as identity. The monthly compliance export receives a named replacement and must complete one real cycle on it.

Then the team freezes new use. Documentation and examples lead with plan_id; schema policy marks plan_name deprecated; new dashboards cannot select the old warehouse column; repository checks reject fresh joins on it. Without this step, new consumers arrive while old ones leave. A deprecation without a freeze is a treadmill.

Enforcement becomes progressively more revealing while it remains reversible. Non-production consumers receive failures. Production warnings carry consumer identifiers. A canary producer omits plan_name for a bounded internal path. Dead-letter queues, consumer errors, finance totals, support reports, and query history are watched together. If an unexpected reader appears, the team restores the field for that path, identifies the owner, and returns it to the migration list. The mechanism is doing discovery, not punishing lateness.

Only then does the producer stop writing the old field and remove it. Cleanup includes event schemas, producer and consumer code, warehouse columns or compatibility views, documentation, fixtures, dashboards, alerts, examples, lineage edges, and deprecated-field policies. An unused field left visible invites a new dependency and leaves future operators unsure whether it is safe.

Calendar time still matters: consumers need a support window, and external clients may have contractual dates. But elapsed time is not completion evidence. Removal rests on attributable usage, successful consumer behavior, closed exceptions, a reversible disable result, and an owner willing to accept what remains.

Put Safety Nets Where the Change Happens

Different contract surfaces need different protection.

Consumer and producer contract tests should assert behavior, not only syntax: required fields, allowed nulls, units, identity, ordering, defaults, permission filtering, and errors. A registry can stop an incompatible event before production when its configured compatibility mode reflects replay and rolling-deploy needs. A shadow pipeline can compare counts, business totals, freshness, and selected records while an old and new reporting model coexist. A canary consumer can exercise the new contract with one tenant, service, dashboard, group, or region and fail with evidence useful for diagnosis.

None of these proves that the consumer list is complete. Lineage misses downloads and copied SQL. Registries miss ad hoc tables and exports. Contract tests cover the consumers that participate. Combine enforcement with access evidence and accountable owners.

Controls closest to the change path are easiest to trust. A wiki warning is weaker than a repository check on the deprecated field. That check is weaker than a schema policy that blocks an incompatible producer deployment. For warehouse data, attributed query history and a policy preventing new dashboards on a deprecated view can turn a social request into an enforceable transition.

The machinery should stay proportional. An optional field with clear semantics may need an automated compatibility check and documentation update. A required field should begin optional, survive mixed producers, and become required only after backfill or rollout evidence. A rename needs additive introduction, dual writing, migration, freeze, and removal. A meaning change usually deserves a new field or version. Changes to identity, authorization, money movement, personal data, regulatory reports, or external clients demand deeper shadowing, rollback, and named risk acceptance.

Ownership follows the work rather than a standing committee. The producer owner defines and operates the contract through the support window. Consumer owners validate replacement behavior and migrate their paths. A platform owner supplies registries, attribution, lineage, and policy. A data steward or domain owner resolves contested meanings and units. When the evidence cannot be made complete, a change approver names the residual risk being accepted. Most changes should travel a fast automated path; higher consequence should make the evidence heavier, not the ceremony more ornate.

A Deprecation Record That Can Stop the Change

Keep one live record for the change. It should be short enough to update and specific enough to prevent an unsafe removal. For the billing event, a useful record might read:

contract: subscription event plan identity; warehouse subscription fact;
          billing API display label; monthly compliance export
old promise: plan_name is a mutable display label, historically misused as identity
new promise: billing_plan_id is stable identity at the event's effective time;
             display text resolves through the versioned plan dimension

mixed versions: old consumers ignore billing_plan_id; new consumers accept old events
                through bounded historical mapping; replay fixture covers both shapes
discovery: five event groups owned; warehouse query fingerprints attributed;
           support script migrated; compliance export observed on replacement

migration evidence: entitlement contract tests clean; revenue totals match in shadow;
                    named dashboards use plan dimension; replay and restore drills clean
freeze: old field absent from new examples; new warehouse dependencies denied;
        repository policy rejects new identity joins on plan_name

enforcement: non-production omission complete; internal canary complete;
             production warnings show no unresolved consumer for one monthly cycle
fallback: restore dual emission; restore compatibility view; keep mapping through window
removal: producer, registry, warehouse view, docs, fixtures, alerts, lineage, and policy listed
owners: producer, each consumer, platform contact, domain steward, and risk approver named

Every green claim should link to evidence: a query, owner, fixture, comparison result, canary, export run, or accepted exception. “Notify downstream users,” “should be fine,” and “no known consumers” do not belong in the completed record.

During review, ask what exact promise changes and what decision could become wrong without an error. Ask which producer-consumer combinations coexist during rollout, retry, replay, restore, and failover. Separate proven consumers from suspected ones. Identify the automated check that stops an incompatible producer, the signal that proves each consumer moved, and the control that prevents new use of the old shape. Name the first reversible enforcement step, the exception owner, the fallback, and every artifact that must disappear after removal.

Evolution Drill

Choose an event, API response, or reporting table with at least three consumers. Remove a field, change its meaning, or tighten an allowed value. Write its old and new promise, identify the four producer-consumer combinations, and build a deprecation record with discovery signals, owners, version boundary, fixtures, telemetry, freeze, enforcement, fallback, and cleanup.

Then make the record survive three consumers that tidy plans tend to miss:

1. a replay job processes old events with new code;
2. a monthly export wakes after three quiet weeks;
3. a new dashboard tries to adopt the deprecated column during the migration.

If the plan cannot detect or contain one of them, the contract is not ready to shrink. The discipline is not avoiding change. It is changing a production promise while its consumers still trust the system—and leaving the next chapter with a dependency that is known well enough to retire.