Appendix O: Glossary

Define the Term at the Decision Point

One team says an assistant is “accurate” because its answers sound correct. Another means that cited claims are supported. A third means that a classifier exceeds an aggregate test-set target. They can agree in a meeting while approving three different properties.

This glossary uses operational definitions: each term should help a reader identify a system property, piece of evidence, decision, or owner. When a term affects acceptance or risk, replace the shorthand with a measurable definition in the system’s own documentation.

A

Abstention: A deliberate system outcome in which the system does not answer, predict, recommend, or act because evidence, confidence, permission, policy, or operating conditions are insufficient. Define the fallback and measure abstention quality, not only its rate.

Accuracy: Agreement with a defined reference under stated conditions. Name the unit being judged, the reference or ground truth, the population, the measure, and the cost of disagreement; apparent correctness, citation support, classification accuracy, and usefulness are different claims.

Agent: An AI-enabled system that can choose or sequence actions toward an objective, often by using tools, APIs, memory, or external services. The term does not establish autonomy, reliability, or accountability; specify permissions, limits, approvals, and recovery.

AI-enabled system: The model plus the data, software, prompts, retrieval, tools, interface, workflow, people, controls, monitoring, suppliers, and institutional decisions required to produce and govern an outcome.

AI governance: The allocation of decision rights, accountability, policies, evidence requirements, review forums, controls, records, escalation, and assurance across the AI lifecycle.

AI literacy: The knowledge and practiced judgment a person needs for their role in selecting, building, using, reviewing, procuring, governing, or being affected by an AI system. Awareness training alone is not demonstrated competence.

Automation bias: The tendency to favor an automated suggestion even when conflicting evidence or context is available. Interface design, workload, incentives, and perceived authority can strengthen it.

B

Baseline: The non-AI process, simple rule, previous model, human workflow, or other reference against which a proposed system is compared. A baseline makes incremental value and additional risk visible.

Blast radius: The maximum credible scope and severity of consequences an error, compromise, or unintended action can cause before detection and containment. Permissions, rate limits, transaction limits, segmentation, and reversibility shape it.

C

Calibration: The relationship between a model’s confidence score and observed outcome frequency for a defined population and period. A score is useful only when its meaning is tested in the decision context.

Concept drift: A change in the relationship between inputs and the outcome or target the system is meant to predict. A stable input distribution does not rule it out.

Confidence score: A model-produced or derived value associated with an output. It is not automatically a probability of correctness, a safety score, or a sufficient basis for automation; validate its interpretation.

Context window: The bounded input sequence a generative model can process for one interaction. Inclusion in the context does not guarantee attention, faithful use, or correct reasoning.

Control: A preventive, detective, corrective, or compensating mechanism intended to reduce the likelihood or consequence of a defined risk. Control existence is not control effectiveness; test the mechanism in context.

D

Data drift: A change in the distribution, quality, coverage, provenance, or availability of production inputs relative to a reference population or period.

Data leakage: Improper information flow that makes development or evaluation results misleading, or exposes information across boundaries. Examples include target information entering features, test examples entering training, or one tenant’s content reaching another tenant.

Datasheet for a dataset: Structured documentation of a dataset’s purpose, composition, collection, provenance, processing, known gaps, access, maintenance, and appropriate uses.

Decision support: A system role in which AI supplies information, predictions, options, or drafts while a human or accountable process retains the decision. The label is credible only when the human has usable evidence, time, authority, and fallback.

Decommissioning: The controlled retirement of a model or AI-enabled system, including traffic removal, dependency handling, data and credential disposition, record retention, user communication, replacement or fallback, and verification that unintended use has stopped.

Deterministic software: Software intended to produce outputs from explicit rules and state rather than learned statistical behavior. Deterministic components can still contain defects and uncertainty in inputs, but they often provide stronger repeatability for stable rules.

E

Embedding: A numerical representation that places items such as text, images, users, or products in a learned vector space. Similarity in that space reflects the model and training objective, not necessarily factual, causal, legal, or human equivalence.

Evaluation: A structured process that tests defined system claims using representative data, scenarios, measures, reviewers, thresholds, and decision rules. Evaluation is broader than calculating a model metric.

Evaluation leakage: Contamination that makes measured performance more favorable than expected performance on genuinely unseen conditions, such as overlap with training data or adapting prompts repeatedly to a fixed test set.

Explainability: Methods or artifacts intended to help a defined audience understand relevant aspects of a model or system behavior. Specify the audience, question, fidelity, limitations, and decision the explanation supports.

F

Fairness: A context-dependent property concerning the distribution of benefits, burdens, errors, access, treatment, or opportunity across people or groups. It requires a normative choice, affected-party context, and segment evidence; no single metric settles it universally.

Fallback: A safer alternate process used when the AI path is unavailable, uncertain, prohibited, or outside its operating envelope. A fallback must be staffed, tested, reachable, and monitored.

Feature: An input variable or derived representation supplied to a predictive model. Its definition, availability, timing, provenance, and relationship to the target can create leakage or inequity.

Fine-tuning: Additional model training that adapts behavior using selected examples or feedback. It may improve task fit but creates versioning, data governance, evaluation, and regression obligations.

Foundation model: A broadly capable model trained on large and varied data that can be adapted or prompted for multiple downstream uses. Its generality does not remove use-case-specific evaluation and controls.

G

Generative AI: Models and systems that produce content such as text, images, audio, video, code, or structured data. Output plausibility must not be confused with factual support, authorization, or fitness for action.

Groundedness: The degree to which claims in an output are supported by the evidence sources the system is instructed or expected to use. Groundedness does not by itself establish that sources are correct, current, complete, or authorized.

Guardrail: A colloquial term for a control around AI inputs, outputs, actions, or workflow. Name the specific mechanism, threat or harm addressed, coverage, failure mode, and response rather than relying on the label.

H

Hallucination: A generated output that is unsupported, fabricated, or inconsistent with the required evidence or context. Because usage varies, define the observable failure—such as unsupported citation, invented entity, or contradicted claim—in evaluation records.

Human-in-the-loop: A workflow in which a human review or approval step occurs before a specified output or action proceeds. The phrase does not prove meaningful oversight.

Human-on-the-loop: A workflow in which a human supervises operation and may intervene while some outputs or actions proceed without prior approval. Response time, observability, authority, and containment determine whether supervision is effective.

Human oversight: A designed control in which people have the information, competence, time, authority, independence, escalation route, and fallback needed to influence consequential system behavior.

I

Impact assessment: A structured examination of intended benefits, affected people, plausible harms, rights and obligations, distributional effects, controls, evidence gaps, ownership, and residual risk before and during use.

Inference: Use of a trained model to produce an output from new input. Inference behavior also depends on preprocessing, prompts, retrieval, tools, configuration, and serving infrastructure.

Input validation: Checks that input has acceptable format, provenance, authorization, size, content, and context before processing. It is one control layer, not proof that the resulting output or action is safe.

L

Large language model (LLM): A model trained over token sequences to predict or generate language-like outputs and often adapted for instruction following. An LLM is a component, not a complete product or accountable actor.

Latency: The elapsed time between a defined request and response point. Report the distribution and conditions—such as p50 and p95 by route—not only an average.

Least privilege: Granting a person, model-mediated workflow, service, or tool only the access and action authority necessary for a defined task and duration.

M

Machine learning (ML): Methods that fit model behavior from data, objectives, or interaction rather than specifying every decision rule directly.

Material change: A change capable of altering an approved purpose, population, data use, behavior, authority, control, evidence claim, risk tier, or credible consequence enough to require renewed review. Organizations should define triggers for their context.

Model: A learned computational component that maps inputs or context to outputs. Its properties do not automatically describe the reliability, fairness, safety, or usefulness of the surrounding system.

Model card: Structured documentation of a model’s identity, intended uses, development, evaluation, limitations, ethical or risk considerations, and maintenance. A system using the model needs additional system-level documentation.

Model drift: A broad operational label for degrading or changing model performance over time. Diagnose the mechanism—data drift, concept drift, upstream change, feedback, dependency change, or another cause—before choosing a response.

Multimodal model: A model that processes or produces more than one modality, such as text, image, audio, or video. Cross-modal inputs create additional provenance, privacy, evaluation, and accessibility concerns.

O

Offline evaluation: Evaluation performed before or outside live production interaction, often on fixed datasets, simulations, red-team cases, or recorded scenarios. It supports but cannot fully replace production evidence.

Online evaluation: Evaluation using live or production-like interaction, such as shadow operation, canaries, pilots, experiments, monitoring, or structured feedback. Exposure and stop conditions must match the risk.

Operating envelope: The approved conditions under which a system is expected and permitted to operate, including users, tasks, data, regions, languages, volume, dependencies, authority, performance, and control availability.

Out-of-distribution input: Input that differs meaningfully from the data or conditions represented in development and evaluation. The relevant difference depends on the system claim and may not be captured by a single distance measure.

P

Precision: Among items predicted positive, the proportion that are actually positive under the evaluation definition. It answers a different question from recall and can vary by threshold and segment.

Prompt: Instructions and context supplied to a generative model. Prompts are versioned behavioral components and may include system, developer, user, retrieved, tool, and conversational content.

Prompt injection: Instructions or content from an untrusted or lower-trust source that influence a model-mediated workflow contrary to intended policy. Treat external content as data, constrain authority outside the model, and test end-to-end consequences.

Provenance: Evidence about the origin, custody, transformation, permissions, and version history of data, models, content, or artifacts.

R

Recall: Among items that are actually positive, the proportion the system predicts positive under the evaluation definition. Improving recall can change false positives and workload; assess the decision trade-off.

Red teaming: Structured adversarial examination intended to discover vulnerabilities, harmful behaviors, misuse paths, and control failures. Findings should become fixes, restrictions, monitoring, or regression tests.

Regression test: A repeatable test preserved to detect recurrence of a known failure or unintended behavior change. A strong test names the input, context, expected property, version, and decision rule.

Residual risk: Risk remaining after controls are applied and their effectiveness is considered. Acceptance requires a named owner with authority and evidence, not a checkbox.

Retrieval-augmented generation (RAG): A pattern in which a system retrieves information and supplies selected material as context for generation. Quality and safety depend on source governance, authorization, indexing, ranking, context construction, generation, citation, and fallback.

Robustness: The ability of a defined system property to remain acceptable under specified variation, stress, perturbation, failure, or adversarial conditions. Always name the property and conditions.

S

Safety case: A structured argument that a system is acceptably safe for a defined use, supported by claims, evidence, assumptions, controls, and unresolved limitations. It is not a generic declaration that the system is safe.

Segment evaluation: Measurement broken down by meaningful populations, conditions, languages, regions, workflows, or other slices to reveal concentrated failure hidden by an aggregate.

Shadow mode: Operation in which a candidate system receives production-like inputs but its outputs do not drive the live decision or action. Shadow mode reduces some exposure but may still create privacy, security, cost, and operational risk.

System card: Structured documentation of an AI-enabled system’s purpose, components, versions, data, architecture, users, evaluations, controls, limitations, operations, governance, and change history.

System prompt: High-priority instructions supplied by an application to shape model behavior. It is a behavioral component, not an enforceable security boundary by itself.

T

Threshold: A decision boundary that converts a score or measure into an action, class, escalation, or abstention. Choose it from consequences, capacity, segment effects, and operating conditions—not model performance alone.

Tool use: A model-mediated request to invoke software, data, or an external service. Authorization, validation, limits, confirmation, idempotency, logging, and recovery must be enforced by the surrounding system.

Training data: Data used to fit model parameters or behavior. Record provenance, rights, coverage, quality, transformations, and relevant exclusions.

V

Validation set: Data used during development for model selection, tuning, threshold choice, or other decisions. Repeated adaptation to it can make results optimistic.

Version: An identifier for a reproducible configuration or artifact. An AI system release may need versions for model, prompt, policy, retrieval corpus, index, data pipeline, tools, code, and interface—not one application tag.

Use the Glossary as a Team Control

For a design, evaluation, launch, incident, or audit meeting, select the five terms most likely to hide disagreement. Ask each owner to connect the term to an artifact and decision: Which accuracy property? Which system boundary? What evidence demonstrates meaningful oversight? Which version can be reproduced? What exact event triggers fallback?

The exercise is successful when the team replaces an ambiguous noun with an inspectable claim, not when everyone memorizes a definition.

Vocabulary Quality Gate

• Terms used in acceptance criteria have a local measurable definition.
• Model properties are not silently promoted to system properties.
• “Human in the loop,” “guardrail,” and “safe” name actual mechanisms and evidence.
• Scores identify their population, conditions, threshold, and consequence.
• System records distinguish model, prompt, data, retrieval, policy, tool, and release versions.
• Risk terms name affected people, credible consequence, controls, and owner.
• Legal or policy terms are defined from the applicable authority rather than inferred from this glossary.

Return to What AI Is, and What AI Systems Are, Evaluation Mindset: From Demo to Evidence, or Governance as an Operating System when a definition needs its full design and operating context.