Chapter 3 — Ship Proof, Not Promises

The repository that arrived three days late

Composite. Built from recurring patterns across several launches; no identifying detail.

The demo worked. On a conference stage, a staff engineer ran a four-command sequence against a live cluster, the dashboard filled with traces, and the room made the noise rooms make. Someone in row six wrote the repository URL on the back of a badge.

The repository appeared three days later. It needed two environment variables nobody had documented, an account tier the free tier does not include, and a client library two minor versions behind the one that shipped that week. The engineer who wrote it in the two evenings before the talk had also, sensibly, deleted the test project it ran against. The person from row six opened an issue on a Thursday. It sat.

Six weeks after the keynote the launch had roughly four thousand stars, a lively discussion thread, and — as far as anyone could later establish — no developer outside the authoring team who had reproduced the result shown on stage. The excitement was real. It had been converted into nothing a person could carry back to work on Monday.

Here is the question the event never answered, and the one this chapter is about: when you claim your system does something, what do you hand a developer so they can find out for themselves?

The claim is the cheap part

Every launch makes a claim. Two-second cold starts. Integrate in five minutes. Handles a million events a second. A claim costs nothing to make and nothing to repeat, which is exactly why it moves no one who has been disappointed before. The developer’s problem is not that they doubt your honesty. It is that your claim, as stated, is not something they can act on. It carries no version, no environment, no failure behavior, and no boundary. They cannot tell whether it holds for their load, their region, their data volume, or their budget.

The common approach treats proof as a communications deliverable — something produced after the decision to launch, sized to the marketing calendar, and finished when the post goes live. That sequencing produces artifacts optimized for the moment of announcement rather than for the weeks in which developers actually evaluate things. A sample built the week before a keynote demonstrates that the feature can work under conditions its author controlled. It rarely survives contact with a fresh account.

The strongest thing developer relations can publish lets a developer reproduce the claim, see where it stops holding, and keep going without you in the room. Those three properties — reproducible, bounded, self-sufficient — are what separates evidence from assertion, and each one is a design decision you make before you write a line of the sample.

The evidence ladder

Not every claim needs a laboratory. Proof has rungs, and the useful skill is matching the rung to the risk the developer is taking.

1. Assertion. “It scales.” Costs nothing, proves nothing, and is appropriate only for claims nobody disputes.
2. Polished screenshot. Shows an end state. Says nothing about how it was reached or whether it can be reached again.
3. Recorded demonstration. Shows sequence and timing, which is a genuine step up. It also shows only the take that worked.
4. Readable code excerpt. The reader can now evaluate mechanism and judge whether the approach fits their codebase. They still cannot run it.
5. Runnable sample. The reader executes it. This is the first rung where the claim leaves your hands.
6. Guided lab with expected output and troubleshooting. The reader executes it, knows at each checkpoint whether they are on track, and can recover when they are not. Most launches should aim here.
7. Independent reproduction. Someone with no stake in the result got the result. This is the rung where a performance number becomes credible.
8. Community adaptation or contribution. People have modified it for cases you did not anticipate and pushed the changes back. Your proof now has a life independent of your roadmap.

The rungs are cumulative in transferability and in cost. They are also cumulative in obligation, which is the part teams underestimate: a screenshot has no maintenance consequences, and a guided lab is a commitment to keep something working across versions you have not shipped yet.

Match the rung to what the developer risks by believing you. A design principle or an architectural stance can be carried by explanation alone. A latency figure needs stated conditions and a way to re-run them. Anything a team will build on for two years — a migration path, a client library, a data model — needs at minimum a guided lab, and ideally one that somebody outside your team has completed.

Study the artifact: Kubernetes The Hard Way, read as proof

Chapter 1 looked at Kelsey Hightower’s Kubernetes The Hard Way as a credibility artifact — something whose existence tells you what its author knows. Read a second time, with the question what makes this evidence?, it turns out to be built almost entirely out of the properties this chapter argues for.

Artifact. Kubernetes The Hard Way, a public step-by-step guide to bootstrapping a Kubernetes cluster by hand, maintained on GitHub across many Kubernetes versions since 2016.

Observable mechanics. The guide is a sequence of visible operations. Each lab performs a small number of concrete acts — generate this certificate authority, distribute these keys, start this component with these flags — and then verifies. Verification is not decorative. The reader runs a command and the guide states the output they should see, so the reader knows which step broke rather than discovering three labs later that something is wrong. The prerequisites are enumerated up front: the tools, the compute footprint, the cloud project. The component relationships emerge from the ordering, because you cannot start the scheduler before the thing it talks to exists, and doing it in sequence teaches the dependency graph more durably than a diagram would. The guide also ends: there is a cleanup lab that deletes what the reader created.

The boundary is doing work. The document states plainly that it is optimized for learning and that the resulting cluster is not suitable for production. That sentence is easy to read as modesty. It is closer to a specification. It tells the reader what the artifact proves — you can understand these components by assembling them — and what it does not, which is the thing almost every vendor sample gets wrong in the other direction. A proof that overstates its scope fails the first time a reader takes it at its word.

Transferable method. State the learning objective, decompose the claim into operations the reader performs, verify after each, pin the environment, provide cleanup, and name what the result is not good for. None of that requires a famous author. All of it is checkable by someone reviewing a draft.

Context and limit. This is a deliberately demanding lab for practitioners who want a mental model, not an on-ramp for someone evaluating your product on a Tuesday afternoon. Its maintenance burden is real and visible in its own commit history: pinned versions age, and somebody has to re-run the whole thing. Copy the verification discipline, not the difficulty.

Research note. Observation of the public repository and its documented structure; the version pinning and cleanup labs are inspectable in the repository itself. The claim about maintenance cost is inference from the repository’s version-bump history, not a statement from its maintainers. Verify current structure and version before citing specifics.

The Proof Pack

A proof pack is the set of things that ship alongside a launch, a major concept, or a migration so that the claim can be tested by a stranger. Ten parts, and the useful discipline is that each one has an owner and none of them is optional for a claim a team will build on.

1. Claim. One sentence, precise enough to be wrong. “Faster cold starts” cannot be tested. “Cold start under 300 ms at p50 for a 20 MB Node.js function in a single region, measured from invocation to first byte” can be. Write this first; it determines everything below it.

2. Audience. Who has the prerequisites, stated as prerequisites rather than as a persona — an existing account, a language version, familiarity with a protocol, an ability to create cloud resources. Say who this is not for. A reader who self-selects out at minute one is a reader you did not disappoint at minute forty.

3. Environment. Versions of runtime, client library, and CLI, pinned or bounded. Operating systems tested. Architecture, if it matters, and it usually matters more than teams expect. Account tier. Expected cost in currency, not in vibes. Expected time, from a real measurement rather than from the author’s second run.

4. Runnable path. The smallest sequence that produces a meaningful result. Resist adding the second feature. Every optional branch you include is a place for the reader to guess wrong, and a quickstart with three configuration choices in it is a quickstart with three ways to fail.

5. Expected output. After each significant step, what success looks like — the log line, the status field, the row count, the HTTP code. This is the highest-leverage part of the pack and the most frequently omitted. Without it, a reader who has silently diverged keeps going.

6. Failure map. The four or five errors people actually hit, each with the diagnostic that identifies it and the recovery. Build this from support tickets and from watching people run the path, not from imagination. The failure map is also your best evidence of where the product itself needs a fix.

7. Boundary. Unsupported cases, security caveats, data-handling constraints, and what would have to change before this is safe in production. If the sample disables certificate verification or hardcodes a credential to stay short, say so at the point where it happens and say what the production form looks like.

8. Cleanup or rollback. How to delete every resource created, or reverse the change. Test it. An untested teardown script is how a tutorial becomes a bill.

9. Source and change history. Repository URL, license, version, commit or tag, and the date the path was last verified end to end. A visible last-verified date lets a reader decide how much to trust the sample without opening an issue to ask.

10. Support and feedback. Where questions go, who reads them, and how updates will be communicated. A repository with no issue path is a repository whose author has decided not to learn anything from its readers.

The sequencing matters more than the list. Write the claim, then the boundary, then the environment — before the code. Teams that write the code first end up with a claim shaped by what happened to work.

A live demo is a proof pack under time pressure

A demo is the same argument compressed into eight minutes in front of people who cannot pause you. The compression is the difficulty: everything the written pack states, the demo has to show or admit.

A demo that functions as proof has one visible state transition — something is in a condition, you act, it is in a different condition, and the audience can see the difference without being told. It answers a narrow technical question rather than surveying a product. The output is observable on screen, not narrated. Hidden setup is minimal, and whatever is hidden is stated: “I’ve pre-created the account and the network, and the script that did it is in the repo.” The public artifact is released before or with the talk, so the person in row six leaves with a URL that already works rather than a promise.

And you prepare for failure, out loud. Not a fallback recording swapped in silently, but an explanation you can give while looking at the actual error. A demo that breaks and gets diagnosed calmly in front of the room often builds more trust than one that runs clean, because the audience watches someone who understands the system rather than someone who memorized a path through it. What destroys trust is a recording presented as live. People find out, and what they learn is not about the recording.

Prepare a backup that preserves the learning rather than the appearance. If the cluster is unreachable, walk the audience through the terminal history from the rehearsal run and say what you would expect to see. You lose the drama and keep the transferable content, which is the part they came for.

Publication creates an obligation

A sample is not a post. It keeps running after you stop paying attention, and the developers who find it in month nine have no way to know it was accurate in month one.

Four commitments make that manageable. Every proof artifact needs a named owner, and “the DevRel team” is not a name. It needs a verification cadence — quarterly for most, and tied to release trains for anything that tracks an API. It needs a stated compatibility policy: which versions this path is known to work against, and what happens when the next major release lands. And it needs an archive rule, because the alternative to retiring a sample is not neutrality.

Stale samples do active harm. They generate support load from readers debugging your example instead of their problem. They teach patterns that were reasonable in 2023 and are unsafe now — an authentication flow you have since deprecated, a permission scope far wider than the task requires. Search engines and coding assistants do not know which of your repositories you still believe in. When a sample reaches the end of its life, mark it: a banner in the README naming the last version it was verified against, the reason it was retired, and a link to the supported alternative. Archiving with a pointer is a service. Silent decay is a liability with your name on it.

Words that move

Situation. A quickstart is shipping with a launch. The product marketing draft says “integrate in five minutes.” Your five-user test says otherwise.

Objective. Keep the invitation, replace the promise with something a reader can check, and name the gaps before a reader finds them.

Promise:

You can integrate in five minutes.

Proof-oriented:

The quickstart completes one signed API request in a fresh test account. Median completion in our five-user test was twelve minutes; the fastest was seven. It does not cover production key rotation or retry behavior — those are the next two guides, owned by @maya, due before the March release. The repository pins client version 4.2, prints expected output after each step, and includes teardown.

The evidence it points to is the repository itself: pinned versions, expected output, teardown, and a distribution of real completion times rather than a single number.

What the language deliberately avoids: the word just; a single best-case time presented as typical; and any suggestion that the uncovered cases are unimportant. Naming the two missing guides with an owner and a date converts a gap from something you concealed into something you scheduled. A developer who reads “median twelve minutes, fastest seven” and finishes in fourteen has had an accurate experience of your team. That is worth more than the five-minute version, which buys attention now and spends credibility later.

Change smells

• The sample lives only in a presenter’s personal repository.
• Setup time is excluded from “time to first success,” so the published number describes a step no reader can start at.
• The demo depends on hidden state, a warmed cache, or data corrected by hand between runs.
• Security and cost caveats were cut because they dampened the announcement.
• Expected output is absent, so a reader who diverges silently keeps going.
• No owner, no license, no last-tested date.
• The public sample proves a narrower or simply different claim than the keynote made — the demo showed the streaming path, the repository implements the batch one.
• Nobody outside the authoring team has completed the path, and nobody was asked to.

Field tool: the Proof Pack release checklist

Run this before the launch communication is scheduled, not after. Every unchecked line is either work or a claim you are about to soften.

• The claim is one sentence, precise enough to be falsified.
• Audience and prerequisites are explicit, including who this is not for.
• Versions and environment are pinned or bounded; cost and time come from measurement.
• The first meaningful result is verifiable by the reader without contacting anyone.
• Expected output appears after every significant step.
• The four or five most common failures have a diagnostic and a recovery.
• Security, cost, data-handling, and production caveats are visible at the point they apply.
• Cleanup or rollback exists and has been executed successfully.
• The repository has a named owner, a license, an issue path, and a last-verified date.
• A person outside the authoring team completed the path in a fresh account, and their transcript or notes are recorded.
• The launch communication links directly to the artifact, not to a landing page.
• Archive criteria are written down: what triggers retirement, and where readers are sent.

Two of these carry most of the weight. The outside completion is the only line that tests the pack rather than your belief about the pack — schedule it a week before launch, watch without helping, and write down where they stopped. And the archive criteria are the line teams skip, which is why organizations accumulate hundreds of samples nobody will admit to owning.

If you can only do a short version, do the claim, the expected output, the outside completion, and the owner with a date. Those four catch most of what goes wrong.

Measure it

Four signals tell you whether proof is transferring. Independent completion rate: of people outside your team who start the path, how many reach the verified result. Median and distribution of time to verified success — the distribution matters more than the median, because a long tail means the path works for people who already know your system. Failure concentration by step, from telemetry, tickets, or a watched session, which turns a vague complaint about onboarding into a specific broken step with an owner. And the number and quality of community adaptations: forks that changed something, pull requests fixing your sample, ports to a language you did not cover. Those are the top rung of the ladder appearing on its own.

Countermetric: stars without evidence of execution. Stars measure the intent to look later. A repository with four thousand stars, no fork that diverged, and no issue harder than a typo report has been bookmarked rather than run.

Next

Proof answers is this real? A developer who has run your sample and watched the expected output appear now believes your claim, which is a different thing from believing they can succeed with it. The next question is quieter and harder to hear: can someone like me, with my prerequisites and my thirty minutes, get to a first win — and does the path we built for them teach anything, or does it just end?