Appendix F: AI Evaluation Plan Template

Begin With the Decision, Not the Metric

A team reports that its internal knowledge assistant achieved a groundedness score of 0.86. The release meeting stalls. Nobody can say whether 0.86 is sufficient, which sources counted as support, how performance varies by language, or what the assistant does when the corpus has no answer. The team can rerun the metric, but the number has no path to a decision.

An evaluation plan is a precommitment: before seeing favorable results, the team specifies what decision is being made, what evidence represents the deployment context, which failures matter, what thresholds apply, and how mixed results will be resolved. The plan prevents metric shopping and turns “test the model” into an assurance argument for the AI-enabled system.

Plan Before Running the Study

The product owner states the decision and business outcome; domain and risk owners identify consequential failures; evaluators design samples and measures; technical owners freeze the system version; operators define service evidence and re-evaluation triggers. For a consequential system, someone outside the delivery team should challenge whether the evidence can really support the proposed authority.

Keep exploratory analysis separate from confirmatory acceptance. Exploratory work may discover failure categories and tune the system. Acceptance must use protected data or procedures that were not consumed by tuning.

Build an Evidence Portfolio

No single score establishes readiness. Automated checks can repeat known assertions; expert review can judge domain correctness; user studies can reveal whether people complete the task; adversarial tests can probe abuse; and operational tests can expose service limits. Production evidence later tests whether those claims survive real work. Name what each method can establish and what it cannot. Agreement among weak proxies is not independent confirmation.

AI Evaluation Plan

DECISION AND SCOPE
Use case and system boundary:
Decision: reject / redesign / limited pilot / launch / expand / change / retire
Decision owner and required reviewers:
System version: model, prompt, retrieval corpus, tools, policy, code, data pipeline
Deployment context: users, affected people, regions, languages, channels, volume
Evaluation date and plan version:

CLAIMS AND RISK HYPOTHESES
Primary capability claim:
Business or user outcome claim:
Safety, security, privacy, fairness, and rights claims:
Failure modes and affected parties:
Most consequential uncertainty:
Evidence outside this evaluation:

BASELINES
Current human or operational process:
Deterministic or non-AI alternative:
Previous or simpler model/system:
Why each is a fair comparison:

EVALUATION DATA
Dataset name and immutable version:
Source, dates, sample size, and sampling design:
Deployment population and context coverage:
Required segments and minimum sample support:
Edge, rare, and high-severity cases:
Known gaps, contamination, and leakage checks:
Privacy, access, retention, and deletion controls:

REFERENCE OR GROUND TRUTH
Reference source and operational definition:
Labeling or adjudication process:
Rater qualifications and blinding:
Ambiguity and disagreement handling:
Agreement and quality checks:
Cases where no reliable reference exists:

MEASURES
Primary task metric and why it supports the decision:
Secondary and diagnostic metrics:
Safety and harm measures:
Fairness and segment measures:
Security and abuse measures:
Privacy measures:
Human factors and user task measures:
Service, latency, cost, and reliability measures:
Business outcome and counterfactual measure:
Uncertainty intervals or sensitivity analysis:

THRESHOLDS AND DECISION RULE
Minimum thresholds:
Safety or rights floors that cannot be traded off:
Segment floors:
Service and cost limits:
Stretch targets:
Rule for mixed results:
Who may accept residual risk:

TEST SUITES
Normal and representative cases:
Boundary and rare cases:
Perturbation and distribution-shift tests:
Adversarial and misuse cases:
Prompt injection / tool misuse / data-exposure tests as applicable:
Failure, fallback, abstention, rollback, and recovery tests:
Accessibility and multilingual tests:

HUMAN EVALUATION
Question and rubric version:
Rater population, training, and calibration:
Blind or comparative design:
Assignment and order randomization:
Inter-rater agreement and adjudication:
Rater wellbeing and sensitive-content controls:

ERROR ANALYSIS
Sampling strategy for errors and near misses:
Taxonomy and severity scale:
Segment and systemic-pattern review:
Root-cause owners:
Fix verification and regression-set process:

EXECUTION AND REPRODUCIBILITY
Evaluation harness and code version:
Configuration, seeds, environment, and dependencies:
Raw output and evidence retention:
Quality-control checks:
Responsible evaluator and independent reviewer:

DECISION RECORD
Result by threshold and segment:
Material failures and uncertainty:
Decision: launch / limited pilot / revise / block
Conditions, owners, and due dates:
Evidence communicated to users or operators:

RE-EVALUATION
Cadence:
Production signals that trigger re-evaluation:
Model, prompt, data, retrieval, tool, policy, or workflow change triggers:
Protected regression set and refresh policy:
Retirement or invalidation conditions:

Write One Plan Backward

For the knowledge assistant, “launch” is still too vague. The proposed decision is to admit a fixed release bundle into a four-week pilot for employees in three supported languages. The assistant may answer from the approved internal corpus, cite the supporting passage, abstain when support is missing, and route access-sensitive questions to the existing help desk. It may not change permissions, answer from another tenant’s material, or turn an unsupported answer into advice. The product owner may approve the pilot only after security and domain reviewers sign the result.

That authority determines the evidence. The team freezes the model identifier, system prompt, retrieval index, corpus snapshot, access policy, citation renderer, interface, and fallback route. It samples routine and difficult questions from the work the pilot will receive, with separate support for each language and for access-sensitive cases. Developers may use an exploratory set while they repair retrieval and prompting; the release gate uses a protected set whose cases and reference judgments were not exposed during tuning.

The current search workflow is the baseline because the pilot is meant to improve the employee’s task, not merely the assistant’s score. Reviewers therefore judge whether a person reaches a correct supported answer, how much correction is required, and whether abstention and escalation leave the task recoverable. Citation checking traces each material claim to the cited passage. The security suite probes cross-tenant retrieval and disclosure. Repeated runs, paraphrases, missing-document cases, degraded retrieval, and supported-language variants test whether a single favorable output survives ordinary variation.

Before opening the protected results, the team writes the gate. Citation correctness must be at least 97 percent; employee task success must be no worse than current search; every supported language must meet its groundedness floor; latency and cost must remain inside the pilot budget; and any critical cross-tenant disclosure blocks release. A language below its floor can be excluded only if routing reliably keeps that language on the current search path. A quality miss returns the candidate for revision. The security floor cannot be exchanged for stronger average quality, and the decision owner cannot waive a result that another required reviewer has not seen.

Let a Mixed Result Narrow the Release

The results are mixed. Overall quality and task success clear their gates, but one language misses its groundedness floor. The security suite finds no critical disclosure. The plan does not convert that pattern into a universal pass. The pilot is limited to the two languages whose floors were met; the third remains on current search, and the language gate, route count, and fallback success become monitored signals. Expanding the language, changing the corpus, retrieval policy, prompt, model, citation behavior, or action path requires proportionate re-evaluation against the protected and regression sets.

Read a completed plan backward before accepting it. Start with the recorded decision and ask which threshold authorized each part of it. For every threshold, find the evidence, population, method, and frozen system version that produced the result. Then follow each material failure to an exclusion, fallback, repair, regression case, or named acceptance of residual risk. If any link depends on an explanation from the evaluation team rather than the record, the plan is not yet ready to govern a release.

Pair the plan with Evaluation Mindset, Evaluating Generative AI, and the Generative AI Evaluation Rubric.