Secure System Decomposition and Trust Boundaries

The refund service from the previous chapter has passed architecture review with a simple promise: large refunds require independent approval. On its system diagram, that promise appears to belong inside a box labeled billing.

Now follow one refund. A support operator sends a request through the console. An API checks policy and writes a decision. A worker reads an event and calls the payment provider. Reconciliation later compares provider and internal state. Which component prevents the worker from acting on a forged event? Does the approval authorize a particular amount, recipient, and time window, or merely say “approved”? What happens when the provider completes the refund but the worker times out?

The box cannot answer. The answers live at crossings: where identity changes, data changes hands, authority is granted, or another organization takes control. Secure decomposition makes those crossings explicit enough to enforce and observe.

For every meaningful flow, name its boundary, identity, data class, policy, enforcement point, observation point, and failure behavior. Anything left unnamed is an assumption the design has not yet earned.

Draw the action, then widen the frame

Begin with the action whose misuse would matter, not with the deployment topology. For the refund path, draw the operator, console, decision service, decision store, event stream, payment worker, payment provider, and reconciliation job. Put arrows on the request and response paths. An arrow is a claim that one component can cause another component to do something; give it a direction even when the transport appears bidirectional.

Then widen the frame until the systems that create or recover authority are visible. Add the identity provider that authenticates the operator. Add the support case from which the refund derives its purpose. Add the deployment pipeline and artifact registry that can replace the decision service. Add logs, alerts, backup, incident tooling, and the people who can invoke emergency access. A context view should include users, administrators, services, jobs, stores, external networks, cloud accounts, and third parties, but only at the detail needed to expose a change in trust.

This is why a production diagram that shows pods and subnets can still be useless for security. It may omit the support tool, CI identity, export job, or vendor webhook that holds more consequential authority than an ordinary service call.

Find boundaries by changes in authority

A firewall is one kind of boundary. It is not the definition. A trust boundary exists wherever the receiving side must not accept the sender’s claim without enforcing a new decision.

The public request crosses from the internet to the edge, then from a user session into the application. A call from the console backend to the decision service crosses an application-to-service boundary; the service must authenticate the workload and preserve, constrain, or replace the user’s authority. Database access crosses another boundary because a service identity gains read or mutation rights over a data class. The payment call crosses into a third party whose availability, identity model, and state are outside the team’s control.

Other boundaries may occupy the same arrow. Tenant selection changes which customer’s objects are reachable. A support role entering an admin plane gains powers unavailable to an ordinary customer. A workload moving between runtime environments or cloud accounts meets different credentials and control owners. A prompt entering a model, retrieval system, agent, or tool broker crosses an AI tool boundary whenever untrusted content can influence data access or action. Build and deployment paths cross into production authority even though they carry artifacts rather than customer requests.

Mark each kind that changes the decision. Do not decorate every arrow with every possible label. The test is whether compromise on one side should automatically confer the same identity, data access, or power on the other. If not, there is a boundary and it needs an enforcement point.

Turn each arrow into a flow record

The diagram helps people see the system; the flow record makes its security claims reviewable. Keep the notation compact enough to maintain beside the design:

Flow:       named source -> named destination (and response path)
Transport:  protocol, message, file transfer, model API, or tool call
Boundary:   network / tenant / privilege / runtime / cloud / third party / AI tool
Identity:   human, workload, job, vendor, model, agent, or administrator
Data:       classification plus the sensitive fields actually crossing
Policy:     allowed action and object, decision owner, and approval if required
Enforce:    gateway, service, database, consumer, broker, sidecar, or tool boundary
Observe:    allow, deny, mutation, failure, alert owner, and retention
Failure:    deny, degrade, retry, queue, drop, reconcile, roll back, or intervene

Consider the event from the refund decision service to the payment worker. “Internal billing event” is not enough. A useful record names the producer workload; the refund ID, account, amount, currency, provider, decision version, approver, and expiry carried by the event; the worker’s authorization to consume it; and the validation performed before money moves. It names the durable decision and consumer logs. It also says how duplicate delivery, stale approval, invalid schema, missing policy, and an ambiguous provider response are handled.

That record reveals a structural requirement: the worker must not treat possession of a queue message as authority. It should verify that a trusted decision granted this exact mutation, reject altered or expired decisions, and make provider execution idempotent. The queue is transport, not authorization.

Put controls where the crossing occurs

Controls become architecture only when attached to an enforceable location. Use the matrix to challenge each sensitive flow; not every cell needs a separate product, but every applicable question needs an answer.

For the refund path, operator authentication belongs at the console, but authorization for the money movement belongs at the authoritative decision service and must be honored again by the worker. Rate limits at the edge may slow abuse; aggregate refund limits at the decision point constrain valid-looking misuse. Encryption protects the crossing but does not decide whether it should occur. Logs at each hop must share enough stable identifiers to reconstruct one attempt without copying secrets or unnecessary payment data into telemetry.

Failure behavior is part of the boundary

The normal path can hide an accidental grant of authority. Walk the same flow with dependencies missing and outcomes uncertain.

If authentication or policy lookup is unavailable during a privileged refund, deny the new action and retain enough request context to investigate. If a key service cannot provide required protection, stop processing or use a documented, bounded break-glass path. If security logging fails, the design needs an explicit choice: buffer the evidence, stop the sensitive action, or accept a stated risk and alert an owner. “Continue as usual” is still a choice, merely an unreviewed one.

The payment provider poses a harder case. A timeout does not mean no refund occurred. Retrying without a stable idempotency key can duplicate the mutation; failing closed without reconciliation can leave financial state unknowable. The boundary record therefore needs a durable attempt identifier, provider lookup or reconciliation, and a manual path for ambiguous outcomes. Security here depends on distinguishing denied, not attempted, attempted, completed, and unknown—not on choosing one universal failure slogan.

The same reasoning applies to an AI tool call. Uncertain model output must not become privileged authority. Validate the proposed action at a tool broker, use a scoped identity, require human approval where consequence warrants it, and record both the decision and result without treating the model’s confidence as proof.

Audit the diagram for hidden trust

Once the main flow is labeled, look for paths that inherited trust by convenience.

An “internal” service that accepts any reachable caller turns network access into authorization. A flat network enlarges that mistake. A shared admin interface blends support, operations, and engineering privileges until neither least privilege nor attribution survives. One broad service account lets the compromise of a minor worker reach unrelated stores and actions. An event consumer that validates syntax but not producer, freshness, tenant, object, and authority turns the queue into a command injection surface. An unprotected workload metadata endpoint can convert runtime access into credentials. An AI agent with a broad token allows prompt content to reach far beyond the intended task.

The repair is not to rename these paths trusted. Give workloads and people named identities, scope them to the action, separate administrative and runtime planes, validate messages at consumption, restrict metadata and tool access, and log the decisions. “Internal” describes exposure. It does not enforce policy.

The trust-boundary review

Use this pass while the design can still change, then repeat it against the implemented system. Real systems acquire queues, exports, debug routes, support tools, scheduled jobs, and recovery paths that were absent from the first drawing.

1. Choose one consequential action and trace it from initiating actor to final effect and recovery.
2. Widen the context to include identities, services, stores, vendors, build paths, operations, observability, and incident response.
3. Label every flow with direction, transport, data class, and the human or workload identity presented.
4. Mark every change in network, tenant, privilege, data class, runtime, cloud account, external ownership, administration, or AI/tool authority.
5. Name the policy and the component that enforces it; test whether another API, worker, message, file, administrator, or maintenance path bypasses that point.
6. Attach evidence and alert ownership to sensitive crossings. Make identifiers sufficient to follow one action end to end.
7. Walk dependency failure, replay, partial completion, ambiguous response, logging loss, and emergency access.
8. Remove, isolate, or redesign any flow whose authority cannot be enforced, observed, revoked, and recovered.

A system is decomposed far enough for security review when every network request, API call, message, file movement, administrative action, and AI or tool call can be recorded this way. The result is more than a detailed picture. It is a map from each dangerous crossing to the exact place the system must decide, the evidence that decision leaves, and the behavior when certainty disappears.