Appendix D: Dataset Datasheet Template

Make the Dataset Possible to Refuse

A team assembles 2,400 resolved support conversations to evaluate draft replies. The extract is tidy, the labels are complete, and the sample is large enough to look persuasive. Only after another group proposes reusing it to score support agents does anyone ask what is absent: voice and chat contacts, deleted accounts, unresolved cases, policy changes during the quarter, and customers who abandoned the support flow before opening a ticket.

The problem is not that the dataset lacks a longer description. It lacks a boundary another team can inspect before accepting or refusing it. A datasheet is a versioned claim about where data came from, what it represents, what transformations changed it, and where it is unfit for use. Reviewers should be able to trace every material fitness claim to evidence and every limitation to a product, evaluation, or operating control.

Complete It From the Released Snapshot

The data owner leads completion with collection, labeling, privacy, security, domain, and model representatives as relevant. Describe the exact snapshot used by the system. Link queries, manifests, schemas, label guides, quality reports, consent or license records, and approvals rather than summarizing them from memory.

Create a new version when records, labels, filters, joins, feature logic, deduplication, rights, retention, or intended use changes materially. Record downstream consumers as well as upstream sources. A mutable “latest” datasheet cannot support reproduction, correction, deletion, or incident investigation.

Separate Description From Fitness

The datasheet records facts about the dataset; an approval records whether those facts support a particular use. The same records may be fit for aggregate service analysis and unfit for individual eligibility decisions. State the task, population, deployment context, error exposure, and required controls with every fitness decision. Never turn broad access permission into evidence of representativeness or permission for a new purpose. A useful datasheet lets two consumers reach different answers from the same facts.

Dataset Datasheet

IDENTITY
Dataset name:
Dataset version / immutable snapshot ID:
Datasheet version:
Created / last reviewed:
Data owner and steward:
Technical custodian:
Contact and incident route:
Linked systems, models, and evaluations:

CONSUMERS AND SERVICE
Current consumers and named owners:
Delivery cadence and freshness objective:
Schema, key, unit, null, and event-time semantics:
Support route and response expectation:
Change notice, compatibility, and deprecation policy:

PURPOSE AND SCOPE
Why the dataset was created:
Decisions or tasks it is intended to support:
Unit of observation:
Target or label, if any:
Intended populations, contexts, regions, and time periods:
Explicitly prohibited or unsupported uses:

PROVENANCE AND COLLECTION
Original sources and source owners:
Collection method, dates, settings, and participants:
Sampling or inclusion process:
Inclusion and exclusion criteria:
Incentives, measurement conditions, or observer effects:
Upstream versions and lineage references:
Policy, interface, or instrumentation changes during collection:

RIGHTS, PRIVACY, AND CONFIDENTIALITY
Legal or organizational basis for collection and use:
Licenses, contracts, consent, notice, and purpose limits:
Personal, sensitive, confidential, or regulated fields:
Access roles and approved environments:
Retention, deletion, correction, and subject-request process:
Cross-border or third-party restrictions:
Secondary-use review required:

COMPOSITION
Record and entity counts:
Fields, types, units, schema version:
Population, geography, language, device, channel, and time coverage:
Class or outcome distribution:
Missingness and censoring:
Duplicates and entity overlap:
Train / validation / test split method and leakage controls:
Excluded records and reasons:

LABELING AND GROUND TRUTH
Label source and operational definition:
Annotator qualifications and context:
Instructions and rubric version:
Blinding and ordering:
Quality controls and gold items:
Disagreement measure and resolution:
Ambiguous or unlabelable examples:
Known label delay, noise, or proxy limitations:

TRANSFORMATIONS
Cleaning, normalization, imputation, filtering, joins, and deduplication:
Derived fields or features:
Redaction, de-identification, or aggregation:
Rows or fields removed and why:
Transformation code / pipeline version:
Reversibility and audit trail:

QUALITY AND FITNESS EVIDENCE
Completeness:
Accuracy:
Consistency and validity:
Timeliness and freshness:
Uniqueness and entity resolution:
Representativeness by relevant segment:
Comparison with deployment distribution:
Tests, thresholds, results, date, and reviewer:

KNOWN GAPS AND LIMITATIONS
Underrepresented or absent populations and contexts:
Historical, measurement, selection, or survivorship bias:
Proxy variables and sensitive correlations:
Feedback loops or policy effects:
Likely distribution shifts:
Consequences for performance, fairness, safety, and use:
Required mitigations or restrictions:

MAINTENANCE
Refresh trigger and cadence:
Freshness service level:
Schema and drift monitoring:
Change approval and downstream notification:
Versioning and rollback:
Retention and end-of-life date:
Owner if source becomes unavailable or rights change:

DOWNSTREAM LINEAGE AND RESPONSE
Extracts, features, indexes, models, evaluations, reports, and vendor copies:
How corrections, deletions, rights changes, and incidents propagate:
Consumer notification and impact-assessment owner:
Retirement verification and unresolved copies:

REVIEW AND APPROVAL
Fitness decision: approved / conditional / rejected
Approved use and deployment context:
Conditions and controls:
Residual limitations accepted by:
Privacy / security / domain review as required:
Next review date and change triggers:

Two Decisions From the Same Support Data

The team completes the record for the 2,400 conversations. The immutable snapshot contains English-language email cases marked resolved during one quarter. It excludes deleted accounts, voice and chat contacts, unresolved cases, and customers who never completed the intake flow. Final queue and resolution codes were entered by agents; on a stratified audit sample, expert reviewers disagreed with the recorded resolution code 11% of the time. Customer identifiers are tokenized, but free text can still contain personal information. The datasheet links the extraction query, policy calendar, label audit, access controls, and deletion-propagation job.

A reply-assistance team wants to use the snapshot for offline evaluation. The record supports a conditional yes: evaluation must remain within the sampled channel, language, period, and policy versions; privacy-controlled access is required; reply quality needs direct human assessment because the resolution code is not a quality label; and multilingual, voice, chat, unresolved-case, and abandonment claims remain blocked until those contexts receive their own evidence.

A workforce-analytics team wants to rank agents. The same datasheet supports a no. The sample contains only cases marked resolved, the label partly reflects queue design and agent-entered codes, contact difficulty and reassignment are uneven, and the snapshot says nothing reliable about work completed outside the email system. Access to the records does not make agent-performance scoring an approved or evidentially supportable use.

The two decisions are not inconsistent. They show why the unit of approval is dataset version plus proposed use, population, context, and controls—not a dataset in isolation.

Read the Decision Backward

Start with the approved, conditional, or rejected use. Can a reviewer trace that decision to the released snapshot, generating process, rights, composition, labels, transformations, and measured gaps? Has every material limitation become a restriction, a test, or an operating control? Can a correction, deletion, rights change, or incident be followed into downstream copies? Are refresh, change notice, expiry, and retirement owned?

If the answers depend on memory, the record is unfinished. If the evidence describes only row counts and field quality, it has not yet described the people, processes, and conditions that produced the observations. If every proposed consumer receives the same answer, the fitness boundary is probably still too broad.

Pair this datasheet with Data as Product, Evidence, and Liability, the Model Card Template, and the Evaluation Plan Template.