Skip to content

Production Data Systems Handbook / Chapter 25

Data APIs and Service Boundaries

Protect data ownership with explicit command, query, event, export, and analytical interfaces instead of shared database access and accidental coupling.

The Query That Owned Another Service

The billing service decides whether to send an invoice by reading three tables owned by identity: accounts, account_roles, and suspensions. The query began as a harmless shortcut. Then it learned that a suspended account is not billable, that only some roles may receive invoices, and that a pending account becomes eligible only after verification.

Identity now wants to replace roles with a policy model. Its own code is ready, but billing’s SQL still depends on the old tables, joins, null behavior, and status values. A read-only connection has acquired veto power over another service’s design.

Chapter 24 treated a read as a contract about what its caller may believe. Here the question moves outward: who is responsible for the meaning of the answer, and through which boundary may others depend on it? If identity owns billing eligibility, it must preserve the rule, authorize access to the result, evolve its representation, and repair the interface when it fails. Billing should depend on that supported meaning, not reconstruct it from private storage.

Database-per-service is useful when it expresses that responsibility. It is empty ritual when every team receives a separate database but still exposes table-shaped APIs, unmanaged replicas, and mutation events. A modular monolith can have strong ownership through module APIs and migration rules. A distributed system can have none while every box on the diagram looks independent.

The practical boundary test is simple: can the owner change its storage and internal model without silently changing what consumers are entitled to believe?

Name the Fact Before Choosing the Interface

The first replacement for billing’s query is not a technology choice. Identity and billing must name the fact: this account is eligible for this billing action under policy version P at time T. Identity owns the eligibility decision. Billing owns what it does with eligible, ineligible, pending, and unavailable results.

That division exposes the invariant. If an invoice must not be sent for an account that identity considers ineligible at the moment billing commits to the action, a query followed by a billing write contains a race: eligibility can change between the two. Identity may need to accept a authorize billing command, issue a short-lived decision token that it can revalidate, or define a weaker point-in-time promise that the business can tolerate. Naming the fact prevents “current” from concealing that choice.

It also exposes several distinct consumer decisions that the old SQL blurred together:

  • invoice creation needs a current decision for one account;
  • nightly planning needs a broad set of likely candidates and can tolerate delay;
  • finance needs governed counts by status, region, and segment;
  • migration engineers need to compare the old and new definitions for a limited time.

One generic data endpoint would make these consumers compete over shape, freshness, cost, and failure behavior. One event stream would be wrong for the current invoice decision. One synchronous endpoint called in a loop would be wasteful for finance. The boundary becomes useful when each interface fits the decision crossing it.

Let the Work Produce the Interface

For invoice creation, identity might expose a narrow query such as get billing eligibility. The response includes the decision, a stable reason code, the evaluated account version or decision version, and the freshness promise. That is sufficient only when billing may act on a declared point-in-time decision. If identity must prevent eligibility from changing before the action commits, the boundary needs a command, reservation, or revalidated token rather than a stronger adjective on the query. Identity owns authorization and semantic compatibility. Billing sets a timeout and decides whether uncertainty should delay invoice creation. Because a stale positive answer can charge the wrong account, billing cannot quietly substitute yesterday’s cache.

For nightly planning, identity publishes committed domain facts: account suspended, verification completed, billing eligibility changed. Billing consumes them into a local candidate projection. It now owns deduplication, replay, lag measurement, and rebuilding that projection. The projection makes planning independent of identity’s request-time availability, but it does not become the authority for invoice creation. Billing rechecks the current decision before sending the invoice.

For finance and operations, identity publishes an analytical contract with declared grain, dimensions, reason-code definitions, lineage, freshness, quality checks, backfill behavior, and a semantic owner. Finance queries that governed dataset instead of calling the operational endpoint once per account. The slower path is not inferior; it answers a different question with a meaningful snapshot.

For the migration, the old read-only access remains briefly as a comparison instrument. Dual reads compare the private SQL result with the new query response, and mismatches include enough version and reason evidence to investigate. The credential is narrowed, logged, approved, and assigned an expiry. The migration is complete only when billing has cut over and the direct credential has been revoked.

Other work produces other interfaces. A command API asks the owner to change state when its invariants must decide whether the change is valid: suspend an account, grant access, or post a payment. It accepts business intent rather than a patch to stored fields and makes authorization, idempotency, acceptance, rejection, and pending outcomes explicit.

A bulk export fits a large snapshot for reconciliation, a backfill, an audit, a partner delivery, or a migration. It needs snapshot time, schema meaning, scope, manifest, counts or checksums, encryption, delivery status, retention, and deletion. A file dropped in object storage without those properties is not a contract; it is an unmanaged copy.

A service-boundary map places an owned database behind command and query APIs, an event stream, and an analytical export. Consumers use those interfaces while direct database access is crossed out; callouts name invariants, coupling, freshness, versioning, deprecation, and a controlled emergency exception.
The map is an interface portfolio, not a requirement to deploy four mechanisms. Each supported path gives consumers a particular kind of access while the owner retains responsibility for meaning and change.

Wrapped Storage Is Still Shared Storage

Direct SQL makes the leak visible. A consumer learns table names, joins, nullable fields, indexes, and incidental enum values. It can surprise the owner with a full scan, treat a lagging replica as current, bypass row or field authorization, and break when a migration changes something the owner did not know was public.

Generic query surfaces can create the same leak behind HTTP, RPC, GraphQL, or a data gateway. If callers can request arbitrary fields, unstable joins, unbounded filters, storage-shaped pagination, or internal status values, the owner has wrapped its database without recovering freedom to change it.

Useful flexibility still has a boundary. A platform query service can publish stable schema meaning, allowed relationships, query budgets, result limits, authorization rules, consumer identities, usage telemetry, and deprecation policy. An analytical dataset can expose broad access because its governed grain and semantics are the product. The issue is not how many fields a consumer may request. It is whether somebody deliberately supports their meaning, cost, access, and lifecycle.

Shared storage has the same test. Modules in a monolith may use one physical database while one module owns writes, migrations, and supported read functions or views. A reporting replica may be acceptable when the owner controls its exposed schema, credentials, workload limits, freshness, migration compatibility, and access review. An internal platform dataset may be a healthy shared surface when it has a catalogued meaning, lineage, quality checks, support owner, and retirement path.

Without those controls, “internal” only describes where the surprise will happen.

Choose by Consequence, Then by Pressure

Start with the consumer’s action and the harm of a wrong, late, duplicated, or unavailable answer. A state change protected by owner invariants points toward a command. A user workflow needing a bounded current view points toward a query. Many consumers reacting after a committed fact point toward events. A large point-in-time transfer points toward an export. Shared reporting meaning points toward an analytical contract. An urgent need that cannot wait for a supported path may justify a time-bound exception.

Then disturb the first choice.

Latency may require a synchronous query during checkout, but it also makes identity part of billing’s availability path. The design needs a timeout and a safe response. If the answer may arrive later, events reduce request-time coupling but move complexity into lag, ordering, replay, and consumer repair.

Freshness is independent of response time. A fast projection can still be stale; a slower export can be authoritative for its declared snapshot. The contract should name the observation—source version, event position, snapshot time, or maximum lag—rather than promising vaguely “real-time” data.

Consumer count changes the burden. One narrow workflow may deserve one narrow query. Ten teams asking for related slices may justify stable events or a governed projection. Dozens of small endpoints may be evidence that the owner has never named the shared data product.

Volume can make a correct interface operationally wrong. An API that works for one account should not be called ten million times to reconstruct an export. An event stream that publishes every storage mutation may impose private churn on every subscriber. The unit of the interface should follow the fact and the work, not the easiest row or transaction to emit.

Failure behavior is often decisive. A synchronous dependency can stop the caller. An event consumer can continue on an old projection while lag grows. An export can be retried and checksummed but may leave an overbroad copy behind. The design is incomplete until the consumer says what it will do during each failure.

The Contract Includes the Bad Day

An interface that cannot be observed or repaired is only an optimistic promise.

For the eligibility query, operators need latency, timeouts, decision versions, reason distribution, authorization denials, result size, and dependency health. Identity owns the endpoint and its semantic rule. Billing owns retry behavior and the decision to delay an invoice. Traces or protected logs should let an incident responder correlate one invoice attempt with the eligibility decision without placing sensitive payloads in metrics.

For the event path, identity owns publication from committed state and schema compatibility. Billing owns consumer lag, deduplication, dead letters, projection drift, replay position, and rebuild. If the candidate projection is stale, billing should be able to suppress the batch without blocking identity’s writes.

For the analytical contract and exports, evidence includes snapshot time, freshness, completeness, reconciliation checks, row counts, manifests, delivery status, access logs, and deletion confirmation. For an exception, it includes approval, query or access logs where appropriate, expiry, and proof that access was removed.

This ownership split should survive an incident. “The data team owns it” is not enough. Operators need to know who can repair publication, who can rebuild a projection, who adjudicates the meaning of eligibility, who contacts consumers, and who may retire the old path.

Compatibility Is About Meaning

Every successful interface becomes a dependency. Additive fields and new event types often allow evolution, but schema shape is only part of compatibility. Changing what eligible means, when an event is emitted, the grain of a metric, the age of a snapshot, or the behavior on timeout can break a consumer while every parser remains green.

Owners therefore need consumer discovery. Client identities, access grants, subscription metadata, schema-registry use, catalog lineage, delivery lists, and support ownership can reveal who depends on an interface. If consumers cannot be named, deprecation is guesswork.

Compatibility tests should exercise representative meaning before deployment: query outcomes for known policy cases, event replay against consumer fixtures, export parsing and manifest checks, and analytical reconciliation around a definition change. A new version is justified when old and new meanings cannot coexist safely, not merely because a team prefers a cleaner URL.

Deprecation names an end condition. Support may last until a date, until every registered consumer migrates, or until observed use reaches an agreed threshold and owners approve removal. A security emergency can force immediate withdrawal, but that is an incident response, not a normal versioning strategy.

The direct identity read illustrates the full exit: discover billing as a consumer, supply replacement contracts, compare outcomes, cut each workload over, observe the new paths, revoke credentials, and prove that identity can change its policy model without billing’s permission.

Use Exceptions Without Lying About Them

Emergency reporting, incident investigation, migration comparison, and one-time repair can require access that the supported interfaces do not yet provide. Refusing to acknowledge that need encourages engineers to create an invisible route.

A defensible exception is narrower than the normal interface and shorter than the problem. It identifies the decision that cannot wait, the exact data and operations allowed, the approving owner, the authorization scope, workload guardrails, logging, output handling, expiry, and removal evidence. It should also leave a decision about whether a durable interface is needed.

An exception without an end condition is a product interface whose design review was skipped.

Write the Exit Before Approval

A boundary review should end by naming the private dependency it removes and the change the owner regains. This record gives architecture reviewers, migration engineers, and operators enough detail to challenge the proposed interface:

Owned fact and semantic owner:
Invariant and enforcement point:

For each consumer:
  Decision or action:
  Required latency, freshness, volume, and failure tolerance:
  Interface: command, query, event, export, analytical, or exception:
  Contract shape, limits, authorization, and cost controls:
  Behavior on timeout, lag, duplication, partial delivery, or stale data:
  Consumer identity, compatibility test, and deprecation rule:
  Producer and consumer operating owners:
  Telemetry, repair, replay, or reconciliation path:

Private dependency being removed:
Dual-run or comparison signal:
Cutover and rollback condition:
Credential, subscription, copy, or old-version removal proof:
Change the owner must be free to make afterward:

Apply it first to the billing dependency. Delay the eligibility endpoint after the invoice transaction begins. Let the candidate projection fall six hours behind. Replay an event. Change the meaning of pending without changing its field type. Deliver an analytical snapshot with a missing region. Allow the comparison credential to reach its expiry date.

For each disturbance, decide what billing or finance sees, which action remains safe, who is paged, and how the system returns to a known state. Then take a second direct read from your own system and replace it on paper. If the consumer only needs governed reporting, test whether a controlled replica is sufficient; name the schema surface, freshness, query budget, migration contract, access review, and shutdown rule that would make it a supported interface.

The boundary has succeeded when consumers can trust the fact and the owner can still change how that fact is stored. Chapter 26 makes that freedom harder: once the same paths carry facts for many tenants, ownership must also enforce who may see them, how much shared capacity they may consume, and how one tenant can be moved or restored without crossing another’s boundary.