Architecture Patterns for AI Systems

The Refund That Should Not Have Happened

A support team wants an assistant to handle refund requests. The first design sounds efficient: give a language model the complaint and order history, let it decide eligibility, call the refund API, and send an explanation to the customer.

Now consider one ordinary complication. The customer asks about two orders in one message. One is refundable; the other was already refunded to a different payment method. The model reads the policy correctly but attaches the right rule to the wrong order. If its output is a draft, an agent can catch the mix-up. If the same output becomes arguments to a tool, money moves before anyone sees the mistake.

The model did not become more or less accurate between those designs. Its authority changed.

AI architecture is the placement of uncertain judgments in a path of evidence, decisions, and actions. A reliable design makes each judgment visible, gives it no more authority than it has earned, and provides a route to detect and recover from failure.

Pattern names help only after the action path is understood. “Copilot,” “agent,” and “assistant” are product labels. Architecture begins with what the system can cause.

Trace the Consequence Before Choosing the Pattern

Follow one request from input to final state. At every boundary, ask:

• What evidence enters, and can it be stale, hostile, incomplete, or unauthorized?
• Which component makes a probabilistic judgment?
• What receives that judgment: a person, another model, deterministic logic, or an external system?
• What permission does the receiving component have?
• When can an error first be observed, and can its effect still be reversed?
• Which versions and identifiers would reproduce the path during an incident?

For the refund assistant, a complaint enters from an untrusted customer. Order and policy records enter from controlled systems. A model may infer which order the complaint concerns, extract facts, interpret the policy, and propose an action. The refund service moves money; the messaging service creates an external commitment. Those last two boundaries deserve stronger control because their consequences survive the model invocation.

This trace also reveals a useful design move: place AI earlier in the path when possible. Let it discover, extract, compare, route, or draft while deterministic services and empowered people retain consequential authorization. That does not mean “always require a human.” It means the action boundary must be deliberate, testable, and proportionate to harm and reversibility.

Build the Refund Path One Role at a Time

The original proposal gives one model four different jobs. Separating them exposes different evidence, controls, and tests.

Retrieval finds applicable evidence

The first role is AI as a search or retrieval layer. It finds the relevant order, payment event, and policy passage; it does not decide or act. Retrieval fits when the knowledge exists in a governed corpus and discovery is the bottleneck. It is a poor fit when sources are not authoritative, permission boundaries cannot be preserved, or the answer depends mainly on evidence outside the corpus.

The system filters by customer identity and order access before passages enter model context. It carries source, version, and effective date with every result, permits an empty result, and lets the agent reach the source. Retrieval recall, ranking quality, freshness, and permission violations matter more here than the elegance of a generated answer. A fluent synthesis cannot repair a missing or unauthorized source; Chapter 21 follows this evidence path in depth.

Transformation creates a checkable artifact

Next, AI as a content transformer extracts order references, claimed events, dates, and requested remedies into a typed proposal. Transformation suits summarization, translation, extraction, and reformatting when the output can be checked against a supplied input. It becomes unsafe when the task silently requires facts or judgments the input cannot support.

Each extracted field points back to its source span. A schema rejects missing types and impossible formats, while business rules check order identity and state. Evaluation looks for fidelity, severe omissions, field accuracy, and terminology—not merely readable prose. The recurring failure is unsupported completion: the model fills a gap because a complete record looks more plausible than an honest blank.

A copilot keeps judgment in the workflow

The extracted facts and policy evidence can feed AI as a copilot. It drafts an eligibility explanation and a customer response, but the service agent sees the underlying order state, edits the draft, and chooses whether to proceed. This pattern fits decomposable work in which a qualified user can inspect evidence and still controls the action. It fails when review capacity is fictional or when accepting the proposal is easier than forming an independent view.

The interface therefore separates evidence from recommendation, makes accept and reject equally available, and has an explicit approval step. Team decision quality, correction severity, downstream rework, reliance calibration, and completion time show whether the combined workflow improved. Adoption rate alone rewards the very automation bias the design is meant to contain.

An AI as suggestion pattern is the smaller version of this relationship: the system proposes one option and a person decides and acts. A copilot may support several steps and maintain working context; a suggestion need not. Neither name guarantees human authority. If a default approval silently triggers a side effect, the actual architecture is execution with a ceremonial click.

An executor receives narrow authority

Only after approval does AI as a constrained workflow executor enter. It may select and invoke tools, but its action space is deliberately small. The refund tool accepts typed order, amount, currency, reason, and idempotency fields. The refund service—not the model—enforces account ownership, order state, role, amount limits, duplicate prevention, and allowed payment routes. The message tool remains unavailable until the service returns a reconciled result.

Execution is suitable when permissions can be narrow, state is observable, and every consequential step is reversible or approval-gated. It is unsuitable when a natural-language goal implies broad credentials, when success cannot be verified, or when failure has no compensating action. Test tool choice, argument validity, unnecessary steps, permission violations, budget use, side-effect severity, and recovery. A goal-completion rate that ignores unauthorized or duplicate actions is not a success measure.

High-value, repeated, identity-mismatched, and policy-conflict requests leave the automated path. Over time, evidence may justify automatic refunds for a tightly bounded class of low-value, reversible cases. The unit of earned autonomy is that action class, not the product as a whole.

Other Patterns Change Attention, Not Just Content

The refund path does not need every available pattern. The remaining ones become easier to choose once their effect on human or system attention is explicit.

AI as classifier or scorer assigns a class, probability, or risk score. It fits when the target label is defensible, thresholds reflect unequal error costs, and outcomes can be measured. Calibration, temporal validation, segment tests, an abstention region, and a correction path surround the score. The dangerous transition occurs when a prediction becomes an objective fact—or a consequential decision—without a separately authorized policy.

AI as triage uses a classification or ranking judgment to route or prioritize work while a later process makes the substantive decision. It is useful when volume is high and delay costs differ, provided misroutes are detectable and recoverable. Urgent-case recall, queue age, workload distribution, and misroute severity reveal more than average accuracy. A catch-all queue, maximum waiting time, sampled low-priority cases, and route correction prevent “low priority” from becoming silent denial.

AI as recommender or ranker orders options for a person or downstream process. It fits when a smaller candidate set helps and several outcomes could be acceptable. Its objective deserves architectural scrutiny: ranking for clicks, margin, short-term completion, or user benefit creates different systems. Measure coverage, diversity, exposure, long-term outcomes, and harmful-content rates alongside ranking quality. Otherwise the ranker can narrow choice or repeatedly hide the same people, products, or ideas while appearing individually reasonable.

These three roles often feed one another, but their boundaries should remain visible. A score may inform a queue; a queue position may determine whether anyone looks at a case; lack of attention may then change the data used to retrain the score. The feedback loop is part of the architecture.

Monitoring and Simulation Produce Evidence

Two patterns sit beside the action path rather than on its ladder of authority.

AI as a monitoring layer detects anomalies, unsafe content, policy violations, drift, or suspicious tool sequences. A monitor is useful only when detection leads to staffed review or deterministic containment and missed events can be studied. Track recall, precision, alert burden, time to detect, and time to contain. Give it independent signals where possible: a monitor built from the same model, prompt, and data as the primary system may share the failure it is supposed to catch.

AI as a simulator generates scenarios or synthetic interactions for testing, training, and planning. It can broaden a test set with refund combinations the team has not yet observed, but its output is a hypothesis about what might happen, not evidence that it does happen. Label simulated data, separate it from production truth, set coverage goals, use domain review, and vary assumptions. Judge the simulator by whether it exposes useful risk and supports decisions, not by whether its invented cases look impressively realistic.

Monitoring observes a running system; simulation probes possible behavior before or alongside operation. Neither proves the system safe. Together they help the team find questions that production averages conceal.

Put a Deterministic Shell Around Uncertainty

Separating roles is not enough. A probabilistic component needs a shell that constrains what may enter, what may leave, and what consequence may follow. Depending on the path, that shell includes:

• identity and authorization before data access;
• input type, size, and content checks;
• separation between instructions and retrieved or uploaded data;
• structured output plus schema and business-rule validation;
• policy decisions enforced outside the model;
• allowlisted tools, narrow credentials, quotas, timeouts, and termination rules;
• safe abstention, fallback, and escalation;
• idempotency, post-action reconciliation, and compensating actions;
• versioned traces, incident evidence, feature flags, kill switches, and rollback.

These controls do not make a model deterministic. They limit the number of ways its uncertainty can become system state.

Never send generated text directly into SQL, shell, payments, messages, or configuration. Parse it as an untrusted proposal. Validate its shape and meaning, enforce authorization, preview the consequence where useful, seek approval when required, execute idempotently, and reconcile the observed result with the intended one.

Compose Patterns at Named Trust Boundaries

A production application usually combines patterns. Record each handoff in one line:

producer -> artifact -> consumer -> validation -> permission -> failure route -> trace

For the refund system, one edge might read:

extractor -> typed refund proposal -> agent workspace
-> schema, order identity, and source-span checks
-> no external side effect
-> missing or conflicting fields require review
-> order, source, policy, and component versions recorded

Another edge—from the approved proposal to the refund service—has permission to move money, so it adds role authorization, amount limits, duplicate prevention, approval evidence, an idempotency key, and reconciliation. The model can propose the arguments. It cannot grant itself the right to use them.

This boundary view keeps component evaluation honest. Retrieval is tested for evidence delivery, extraction for fidelity, the copilot for team judgment, execution for correct and authorized state change, and the composed system for customer outcomes and recovery. One end-to-end score cannot identify which boundary needs repair.

Write the Decision So It Can Be Reopened

An architecture decision record should explain the use case and non-AI baseline; chosen pattern composition and rejected alternatives; evidence, data, instruction, and trust boundaries; permissions and reversibility; deterministic rules; human authority and escalation; component and end-to-end evaluation; observability and privacy limits; dependency failure and safe mode; residual risk and owners; and the changes that force reconsideration.

The record is not a claim that the design is permanently safe. It preserves why this amount of authority was justified for this version of the system under these operating conditions.

Try the Choice Before Naming It

Take this proposal: “Read incoming vendor invoices, resolve discrepancies, update the ledger, and email the vendor.” Before assigning a pattern, draw the action path. Mark where identity, invoice data, purchase orders, policy, and messages enter; where a mistaken judgment becomes financial state; who can detect it; and what can be undone.

Then redesign the proposal so the AI may retrieve records, extract fields with source spans, classify discrepancies, and draft a reconciliation, while a person resolves material conflicts and a deterministic ledger service enforces accounting invariants. Decide whether any narrow, repeated case deserves constrained execution. State the evidence that would earn that change and the signal that would revoke it.

The useful answer is not “make it a copilot.” It is a boundary map showing which judgments remain uncertain, which actions remain authorized elsewhere, and how failure returns the system to a known state.

Architecture allocates authority. Put uncertainty where it can be observed and challenged; put consequence behind controls that can be tested, owned, and reversed.

Source Notes

• NIST AI RMF Core, system-level guidance for mapping context, measuring risks and controls, and managing responses across the lifecycle; verified 2026-07-10.
• NIST SP 800-218A, secure development guidance that extends the Secure Software Development Framework for producers and acquirers of generative AI models and systems; verified 2026-07-10.
• NIST AI RMF Generative AI Profile, companion guidance for generative AI risks and actions across governance, mapping, measurement, and management; verified 2026-07-10.
• Microsoft Guidelines for Human-AI Interaction, interaction guidance used here for communication, correction, control, and failure handling around AI components; verified 2026-07-10.