Case Study 6: Autonomous IT Remediation Agent

The Fix That Multiplied the Outage

At 02:13, an operations assistant sees checkout errors rising while the application tier exhausts its database connections. It proposes restarting an unhealthy instance. The restart briefly lowers the local error rate, so the agent repeats the action across the fleet. Each new instance opens fresh connections. The database begins refusing them, healthy application capacity disappears, and an incident that needed one controlled rollback becomes a regional outage.

The agent found a real symptom and chose an action from the runbook. None of that made the action safe. Its production credential covered the whole application tier. Its success test watched one service metric for ninety seconds. It did not inspect database saturation, notice that its first restart had increased connection churn, or ask whether another actor had changed the deployment. A plausible diagnosis had crossed directly into broad, repeatable authority.

The team pauses automated writes and redesigns the system around one rule: autonomy belongs to an action under stated conditions, not to an agent as a general property. The repaired agent may observe broadly. It earns permission to change one resource only after policy, current state, expected effect, verification, and recovery have all become explicit.

Reconstruct the Baseline Before Adding Autonomy

Before the assistant, an alert pages the on-call engineer. The engineer checks recent deployments, reads service and dependency dashboards, chooses a versioned runbook step, opens a change record when required, watches the result, and hands control to incident command when the diagnosis remains uncertain. The process can be slow and inconsistent, but it contains useful separations: observation is not execution, a runbook is not permission, and one operator’s action remains visible to the others.

The agent must improve that whole response path. Time to the first plausible hypothesis is useful, but time to a stable recovery matters more. The comparison also includes unnecessary changes, operator correction, repeated actions, rollback, collateral load, handoff quality, and whether responders can reconstruct the state after the agent stops. Automating the click while weakening those properties would be a faster route to a worse incident.

The team replays the 02:13 trace against the baseline. The database connection limit was visible. A deployment had changed connection-pool behavior eleven minutes earlier. The runbook described instance restarts for a different failure family and warned against fleet-wide restart when database headroom was falling. The evidence existed; the original design gave the model no enforced reason to join it before acting.

Turn a Plan Into a Typed Action Contract

The repaired system separates four trust levels. A read-only investigator gathers approved telemetry, dependency state, recent changes, active incident records, and runbook evidence. A planner turns that evidence into a proposal. A deterministic policy service decides whether the proposed action is eligible. Only then can a narrow executor obtain a short-lived credential and attempt the exact operation authorized.

The proposal for the replay is not “restart checkout.” It is a typed contract for restart_canary that identifies one inventory-resolved instance and the current release version. It records the observations supporting the diagnosis, the database-headroom and concurrency preconditions, the expected change in checkout errors, the metrics that could reveal collateral harm, the verification window, the maximum of one attempt, and the rollback or handoff state. Its idempotency key lets the executor reconcile an ambiguous result instead of blindly repeating the write.

Free-form model output never becomes shell, SQL, or a generic cloud API call. Parameters are range-checked against current inventory. The executor rejects a resource version that changed after the plan was prepared. It also rejects an expired contract, an action already in flight on the same dependency path, or a request whose verification and recovery fields are absent. A valid schema proves only that the proposal is well formed; identity and policy still decide whether it is allowed.

This boundary changes the meaning of permission. The investigator may read enough of the service graph to form a useful diagnosis, subject to data and query limits. The planner may draft a ticket without changing production. A reversible canary action may run automatically only inside a preapproved service envelope. Rolling back a release, scaling a fleet, or changing production routing requires a person to approve the exact current plan. Deleting data, changing access policy, rotating broad credentials, disabling monitoring, or altering the policy service stays outside the agent’s authority and enters a separate controlled workflow.

No standing administrator credential waits inside the agent loop. The policy service issues a task-scoped token after the checks pass, and the executor cannot use it on another resource or action. The agent cannot mint credentials, approve its own proposal, widen the contract, erase the trace, or move from test to production. Its kill switch and credential revocation path sit outside every permission the agent holds.

Approval Must Bind to the State a Person Saw

For a material production action, the on-call engineer sees the diagnosis and its source evidence, the affected resources, current dependency health, uncertainty, expected effect, maximum blast radius, verification window, and recovery path. An “Allow” button without those facts merely transfers the model’s confidence to a person.

Approval binds to the contract hash and the state version used to create it. If connection headroom falls, another responder changes the target, the deployment advances, or the agent revises an argument, the approval expires. The agent must investigate again or present the changed proposal. Silence, a message found in a runbook, and approval for an earlier action are not consent for the new one.

This also gives incident command a coherent view of concurrent work. The action ledger shows proposals, denials, approvals, credentials, execution results, state changes, and cooldowns. A human intervention can cancel a pending action before the executor acquires authority. If the executor loses contact with policy or the stop channel, loss of control causes a halt, not continued operation under the last remembered instruction.

Give the Harness a Database That Can Get Worse

A stateless prompt test would reward the original plan: exhausted connections are often associated with unhealthy application instances, and restart is a real operation. The failure appears only when the environment remembers what each action changed.

The evaluation harness therefore recreates a small checkout service, connection pool, database limit, deployment history, alert stream, action ledger, and clock. In the seeded 02:13 incident, restarting one instance briefly improves a local error metric while increasing connection churn and database saturation. The correct trajectory is to notice the dependency violation, stop after the canary, preserve the resulting state, and hand off or propose rollback of the recent release. Resolution by repeated restart is a forbidden trajectory even if errors happen to fall in one run.

Variations make the same case resist easy memorization. Telemetry arrives late. A restart call succeeds but times out before acknowledging it. An operator rolls back the deployment while the agent is planning. Two hostnames differ by one character. A runbook contains instruction-like text asking the agent to reveal a secret or bypass a denial. The policy service becomes unavailable. The stop signal disappears. Each run begins from a known fixture and ends by inspecting resources, connections, credentials, locks, and unresolved actions.

The record keeps outcome and conduct separate. Did the service recover? Did the agent choose an eligible tool and current target? Did it respect approval, concurrency, attempts, time, cost, and blast radius? Did it reconcile ambiguous writes? Did it stop when its evidence changed? Did rollback restore the expected state? One successful recovery cannot average away an unauthorized production write, a leaked credential, or a destructive action after loss of the stop channel.

Repeated runs expose severe tails as well as common behavior. The release bundle names the model, prompt, tool and policy versions, fixture, injected fault, and evaluator logic. That provenance matters because a model or planner change may alter the trajectory even while the executor remains unchanged.

Release Authority One Action at a Time

The agent returns to production first as an investigator. Operators compare its evidence and hypotheses with the existing on-call path, including cases in which the right answer is uncertainty. Dry-run mode then sends contracts through current policy and inventory without issuing credentials. Denials and operator corrections are product evidence: they reveal stale runbooks, missing invariants, confusing approval views, and proposals that are persuasive but ineligible.

Only after the 02:13 failure family passes in the state-bearing harness does the agent receive authority to restart one canary in one service class. The production contract retains the tested preconditions, one-resource cap, single attempt, verification window, cooldown, and automatic handoff. A passing canary does not authorize fleet repetition. Broader reach is a new release decision supported by its own failure cases and operating evidence.

During the first live canary, checkout errors fall but database saturation rises. The dependency precondition crosses its limit, so the executor revokes the pending action token, records the canary state, and hands the trace to on-call. It does not attempt to repair the unexpected condition with another unapproved action. This is a successful containment result even though the agent did not resolve the incident.

Operations disables agent writes and incident command restores the known worker and release state. The preserved observation-plan-policy-tool ledger lets the review locate the weakness: simulation modeled restart success but not reconnection pressure, and the verification rule privileged checkout errors over the database invariant. The team adds that dependency behavior to the fixture, policy precondition, and production alert. Recovery becomes a new bounded release; it is not permission to turn the agent back on after a prompt edit.

Monitoring follows the same authority argument. Operators watch proposal correction and denial, stale-state rejection, action duplication, attempts beyond scope, dependency precondition failures, rollback and handoff, resource reach, time and tool budgets, and the severity of near misses. A denied attempt to widen authority deserves review even when no production write occurred.

The Authority Release Record

Before enabling a new action, one compact record should let an incident commander or independent reviewer reconstruct what was granted and why:

• the named action schema, eligible services and environments, resource cap, credential scope, concurrency rule, and expressly prohibited operations;
• the required evidence, state freshness, policy invariants, approval boundary, contract binding, attempt budget, verification window, stop conditions, rollback, and handoff state;
• the state-bearing fixtures, injected failures, repeated-run results, severe release blockers, simulation limits, and production canary evidence;
• the versioned model, prompt, tools, policy, executor, runbook, inventory, evaluator, and ledger retention needed for replay;
• the service owner responsible for invariants, platform owner for executor and rollback, security owner for identity and adversarial testing, operations owner for approval and incident command, and AI system owner for evaluation and change review; and
• the kill switch, credential revocation, evidence-preservation route, recovery authority, monitoring owner, and changes that require a new decision.

The strong implementation shortens investigation and earns a narrow set of reversible actions. The weak one gives a persuasive planner standing privilege and calls a human after the blast radius is spent.

The hiring case before this one showed that authority can hide in control over attention. Here it is visible in credentials, state transitions, and service damage. The benefits-triage case that follows returns to a quieter form of power: an ordering decision that changes how long a person waits for public service.

See Agents and Tool-Using Systems, Security Architecture, and Incident Response.