Reliability, Resilience, and Fallback Design

Available, Fast, and Unusable

A claims assistant meets its 99.9% availability target during a model-provider disruption. Requests return HTTP 200, latency remains low, and the dashboard stays green. Yet the provider’s degraded model produces weak evidence links, so reviewers must open every source manually. The queue doubles, urgent claims wait, and staff begin copying drafts into an unapproved public tool.

The service was technically available but operationally unreliable. Its objective measured response delivery, not whether people could complete the task safely.

Reliability is the ability to deliver an acceptable user outcome within defined operating boundaries. Resilience is the ability to detect disturbance, contain it, degrade deliberately, and recover without transferring hidden risk to users or operators.

Measure the Outcome That Must Survive

The claims team begins with a seemingly obvious objective: 99.9% of requests return successfully. The disruption satisfies it. The useful question is harder: what must remain true for a reviewer to finish a claim safely?

The answer crosses four parts of the system. The service must respond within a useful time. Its behavior must produce adequately supported drafts or abstain. Its controls must preserve authorization, approval, and audit boundaries. The human workflow must keep urgent work moving without exhausting reviewers or pushing them toward unsafe tools. A model endpoint can be healthy while retrieval is stale; an answer can look good while its sources are hidden; an agent can finish a task while violating an approval boundary.

Choose a few service level indicators (SLIs) that represent that outcome. A service level objective (SLO) sets a target or range for an SLI over a defined population and window. An error budget states how much failure the team will tolerate and what it will do when that allowance is being consumed. For this assistant, an objective might require 99% of eligible requests over 28 days to receive either a policy-conforming draft or an explicit safe abstention within four seconds. Separate floors could require at least 95% support for cited claims in an audited sample, no authorization violations, and 90% of urgent escalations to reach a qualified reviewer within 15 minutes during staffed hours.

Those figures are illustrative, not defaults. Each needs a numerator, denominator, exclusions, measurement point, segment cuts, owner, and response. The team also needs to say what cannot be traded. Fast abstention may satisfy the response-time objective while overwhelming the human queue. High average groundedness cannot excuse an authorization failure or conceal collapse in one language. An SLO becomes operational when crossing it changes a release, traffic, staffing, or recovery decision.

Turn Failure Into State Transitions

The provider disruption first appears as timeouts and capacity rejections. The application cancels generation after two seconds and moves to permission-aware search. If the error rate persists, a circuit breaker stops new generation attempts instead of allowing retries to consume the remaining latency budget. Urgent claims enter a staffed queue. Routine drafting pauses when queue age approaches its own objective.

This sequence is a fallback ladder: full AI service gives way to constrained AI, then to deterministic search or rules, a human service, and finally a safe pause. It is not necessary for every system to have every rung. A translation assistant might use an independently evaluated model; a clinical drafting tool might move directly to human work; an autonomous payment action might have to stop. The design question is what capability can remain without disguising a weaker level of evidence or control.

Each transition is a small operating contract. It names the signal and authorized activator; the capability preserved and actions prohibited; what the user will see; the security, privacy, and accessibility controls that remain; the mode’s capacity and maximum duration; and the evidence required to leave it. “Best effort” is not a mode when the user cannot see that quality or authority has changed.

Now test the ladder against correlated failure. Permission-aware search is useless if the outage also took down the evidence store. Human review is fictitious capacity if reviewers need that same store or if the queue has no priority rule. A second model route is not independent when it shares a provider account, region, credential, or mutable upstream policy. Cached content can preserve availability while serving expired or newly forbidden evidence. The failure inventory should therefore follow dependencies and people, not product names: detection, containment boundary, blast radius, safe state, recovery point, recovery time, reconciliation, owner, and drill cadence.

The fallback itself can become the next failure. Watch queue age, abandonment, authorization checks, search success, reviewer error, and work displaced to other channels. Limit intake before rushed review turns resilience into hidden harm. The safe pause remains a designed state: preserve the claim, explain the limitation, identify an alternative route, and say when the user should expect another update.

Engineer Dependency Failure

The model provider is uncertain, but the surrounding software does not get to be vague. Use ordinary distributed-systems discipline around it:

• set end-to-end latency budgets and shorter dependency timeouts;
• retry only operations that are safe to repeat, with bounded attempts, backoff, and jitter;
• attach idempotency keys and reconcile uncertain tool outcomes before retrying;
• isolate resources with bulkheads so one tenant, route, or workload cannot exhaust all capacity;
• use circuit breakers to stop amplifying a failing dependency;
• cap agent steps, time, tokens, cost, and tool calls;
• queue work only when deadlines, capacity, privacy, and cancellation behavior are explicit;
• pin and verify provider/model versions where the service permits it;
• preserve a tested known-good release bundle and migration path;
• avoid caching outputs whose evidence, permission, or validity expires sooner than the cache.

Redundancy must diversify the failure mode. A second provider can improve availability, but it also introduces behavior differences, data transfer, contracts, evaluation work, and operational complexity. The route is useful only after its complete operating bundle meets its own acceptance criteria; silently sending traffic to a cheaper unevaluated model changes the system during an incident.

Test Recovery, Not Just Failure Detection

The claims team next restores the provider route. Reachability is only the beginning. It runs drills that exercise real decisions:

• model and embedding provider outage;
• increased latency and partial timeouts;
• model alias or safety-policy behavior change;
• stale, corrupted, or permission-broken corpus;
• unavailable tool with uncertain side effect;
• monitoring loss during high autonomy;
• reviewer queue overload;
• regional failure or credential revocation.

During each drill, someone must decide whether to change modes, while observers verify user messaging, escalation, capacity, data integrity, rollback, evidence preservation, and recovery. An exercise that proves an alert fires but never puts real work through the fallback has tested detection, not resilience.

Define a recovery point objective where state or data loss matters, and a recovery time objective for reaching an acceptable mode—not necessarily full service. The claims assistant reconciles requests whose completion is uncertain, verifies corpus freshness and authorization, and evaluates the restored release bundle. It then sends internal traffic, opens a small cohort, and widens exposure only while service, behavior, control, and reviewer-workload indicators remain stable.

This return path prevents the outage from ending merely because the dashboard turns green. Recovery ends when the claims workflow is acceptable again and the accumulated human queue is under control.

AI Reliability Plan

• Outcome boundary: user task, affected people, dependencies, and unacceptable outcomes.
• Objectives: service, behavior, control, and human-outcome SLIs with SLOs and error-budget policy.
• Failure inventory: triggers, correlated causes, blast radius, containment, and evidence.
• Fallback ladder: mode, preserved capability, prohibited action, capacity, disclosure, and owner.
• Dependency rules: timeout, retry, idempotency, circuit breaker, isolation, queue, and cache policy.
• Recovery: known-good bundle, recovery point, recovery time, reconciliation, evaluation, and staged return.
• Exercises: scenario, cadence, participants, observations, repairs, and verification.

Design the Outage Before It Happens

For a customer-support assistant that depends on a hosted model, choose the first degraded mode for a provider outage. State what users can still accomplish, what the system must refuse, and the signal that moves it down another rung. Then introduce two complications: the human queue has half its expected staff, and the alternate provider fails the refusal floor for one language. Revise the ladder without averaging either failure away.

Finish by writing four objectives: user-visible response time, groundedness, a hard safety or authorization floor, and escalation time. Give each a population, window, measurement point, owner, and operational consequence. Finally, describe the evidence that would justify moving the restored service upward again. If that evidence only proves endpoint health, the recovery design is unfinished.

Source Notes

• Google SRE, Service Level Objectives, defines SLIs, SLOs, and error budgets and emphasizes a small set of indicators tied to user-relevant behavior and operational decisions; verified 2026-07-20. The examples in this chapter are author-designed and must be adapted to the use case.
• NIST AI Risk Management Framework Core, voluntary guidance for evaluating reliability and resilience and for monitoring, response, recovery, communication, and lifecycle risk management; verified 2026-07-20. NIST notes that AI RMF 1.0 is being revised, so teams should verify the current version when adopting it.
• See Observability, Monitoring, and Drift for actionable indicators and Incident Response for AI Systems for containment and evidence preservation.