State Machines, Typestate, and Protocols

The session ID arrived in the wrong state

A ledger relay restores this record after a restart:

state=disconnected attempts=3 session=41

Every field is individually valid. The record is not. A disconnected relay cannot own an established session, and accepting session 41 next would skip negotiation. The preceding chapters can give each field an honest type and every branch an exhaustive match; this failure exists between snapshots. Time has added another invariant.

The relay needs two kinds of evidence. Events from storage, the network, and operators must be checked against runtime state. Inside one Rust call sequence, the compiler can prevent a handle from exposing a session before acceptance. Neither kind proves that the remote peer still agrees.

This division is the governing rule: keep recoverable, externally supplied state as data; use typestate for a short local sequence when consuming one phase should make the next API available.

Let the runtime machine reject and return

The relay has four phases. Disconnected may begin negotiation. Negotiating may accept a nonzero session, time out to Disconnected, or close. Established may close. Closed is terminal. The phase owns the evidence meaningful within it:

#[derive(Clone, Debug, Eq, PartialEq)]
enum HandshakeState {
    Disconnected { attempts: u8 },
    Negotiating { attempts: u8 },
    Established { session: NonZeroU16 },
    Closed { clean: bool },
}

#[derive(Clone, Copy, Debug, Eq, PartialEq)]
enum HandshakeEvent {
    Connect,
    Accept(NonZeroU16),
    Timeout,
    Close { clean: bool },
}

This enum eliminates combinations such as “disconnected with a session.” It does not eliminate illegal events. A decoded Accept(41) can still arrive while the relay is disconnected, so the transition remains fallible.

An owned transition should not destroy the last good state when it rejects an event. Return both pieces to the caller:

#[derive(Debug, Eq, PartialEq)]
struct RejectedTransition {
    state: HandshakeState,
    event: HandshakeEvent,
}

impl HandshakeState {
    fn apply(self, event: HandshakeEvent) -> Result<Self, RejectedTransition> {
        let next = match (&self, event) {
            (Self::Disconnected { attempts }, HandshakeEvent::Connect) => {
                Self::Negotiating {
                    attempts: attempts.saturating_add(1),
                }
            }
            (Self::Negotiating { .. }, HandshakeEvent::Accept(session)) => {
                Self::Established { session }
            }
            (Self::Negotiating { attempts }, HandshakeEvent::Timeout) => {
                Self::Disconnected { attempts: *attempts }
            }
            (Self::Negotiating { .. } | Self::Established { .. },
                HandshakeEvent::Close { clean }) => Self::Closed { clean },
            _ => return Err(RejectedTransition { state: self, event }),
        };

        Ok(next)
    }
}

Matching on &self delays the move until the transition is known to be legal. On rejection, the scheduler can keep the prior state, count the offending event, and decide whether to retry, quarantine the record, or close the transport. A bare ProtocolError::IllegalTransition would identify the class of failure but discard the value required for recovery unless the state were cheaply copied elsewhere.

The saturating_add above is a declared telemetry policy, not a general answer to overflow: attempts beyond u8::MAX remain represented as u8::MAX. If the count controls retry permission or billing, it needs a wider type or a checked transition instead. State design does not excuse numeric policy.

Move the proof into method availability

Suppose one component creates a connection, negotiates it, and hands an established transport to a request loop. No decoded state enters during those calls. Here the sequence itself can become part of the type:

struct Disconnected;
struct Negotiating;
struct Established;

struct Connection<State> {
    peer: String,
    session: Option<NonZeroU16>,
    state: PhantomData<State>,
}

impl Connection<Disconnected> {
    fn connect(self) -> Connection<Negotiating> {
        Connection {
            peer: self.peer,
            session: None,
            state: PhantomData,
        }
    }
}

impl Connection<Negotiating> {
    fn accept(self, session: NonZeroU16) -> Connection<Established> {
        Connection {
            peer: self.peer,
            session: Some(session),
            state: PhantomData,
        }
    }
}

impl Connection<Established> {
    fn session(&self) -> NonZeroU16 {
        self.session.expect("accept constructs established sessions")
    }
}

Connection<Disconnected> has no session method. connect(self) consumes the disconnected handle, so safe callers cannot continue to use it after negotiation begins. accept both installs the session and changes which methods exist. The marker occupies no storage, but the proof still depends on the private representation: if some escape hatch can construct Connection<Established> with session: None, the expect becomes reachable.

This API proves a local ownership history. It does not prove that connect sent a packet, that the peer accepted session 41, that a lease remains current, or that a replayed response is authentic. Those claims need runtime protocol evidence. The type should not promise more than this process established.

Consuming transitions are strongest when resources change with the phase. A negotiation buffer can be consumed into an established transport; a draft transaction can become a receipt while making further edits impossible. Changing only PhantomData while leaving every resource accessible through shared global state creates a ceremonial proof.

Recovery ends the compile-time history

A decoder reconstructs values, not the Rust calls that once produced them. Deserializing bytes directly into Connection<Established> would let untrusted input manufacture the conclusion that accept ran.

Restore into a wire representation first. Validate its version, state tag, required fields, attempt policy, and state-specific combinations. The result should be the runtime HandshakeState, because the service must still store mixed phases, apply operator events, migrate old records, and report rejected transitions.

Only after validation should a narrow adapter create a typed handle for a local operation. That conversion is a trust boundary. Keep its constructors private, test every runtime variant, and decide what happens to states removed by a later version. A persisted Negotiating record may be resumed, timed out, or deliberately downgraded to Disconnected; silently constructing whichever marker makes current code convenient erases recovery policy.

The same boundary appears in long-running work. A scheduler needs Vec<HandshakeState> because connections occupy different phases. A request handler may briefly borrow an established entry and receive a façade that exposes established-only operations. Erasing state at the system-of-record boundary is not surrendering type safety; it is representing information that is genuinely dynamic.

Builders have a shorter clock

A builder is another state machine, but its lifetime changes the decision. For a Rust-only API with two required steps, RequestBuilder<NeedsEndpoint> becoming RequestBuilder<Ready> can make omission unrepresentable. When configuration is assembled from a file, environment variables, and optional overrides, the order is not controlled by the caller’s Rust types. A mutable builder with build() -> Result<Request, ConfigError> can accumulate inputs and report missing or conflicting fields together.

Do not generate a marker type for every optional flag combination. The useful question is whether method availability helps a caller choose the next legal action. If it merely transfers runtime validation into a large generic vocabulary, the compiler messages, documentation surface, compile time, and generated code become part of the API’s cost.

Public libraries also need to consider storage and abstraction. Different Connection<State> instantiations cannot share a homogeneous collection without an enum, trait object, or state erasure. Generic transition code may be monomorphized for several states, though the actual code-size effect depends on optimization and should be measured. A dynamic plugin or FFI boundary may be clearer with one stable runtime representation than with types callers cannot name conveniently.

Close must survive every representation

Protocols are often designed around their successful path: connect, accept, use. Production design begins to show its quality when negotiation times out, persistence contains an older state, shutdown arrives during recovery, or the peer disappears after local acceptance.

The runtime machine therefore needs explicit rejection and recovery policy. The typed façade needs cancellation or a way to return ownership to the runtime machine from every live phase. If close exists only on Connection<Established>, typestate has made cleanup less safe. If reconnection manufactures Established from a cached session without runtime validation, it has made the happy path look safer by hiding the dangerous path elsewhere.

Observability follows the same split. Runtime state can expose phase residence time, attempt saturation, rejected events, and migration outcomes. Compile-time rejection produces no incident telemetry; it prevents a class of local misuse before deployment. A useful hybrid does not ask either mechanism to do the other’s job.

Protocol review

• Can a reviewer enumerate every legal state and event without following boolean combinations?
• Does a rejected owned transition return enough state and event data to recover and observe it?
• Which transitions consume ownership, and which phase-specific resources move with them?
• Which facts enter from wire, storage, configuration, FFI, plugins, or operators?
• Where are those facts validated before a typed façade is constructed?
• Can every live phase be cancelled, closed, persisted, migrated, or deliberately abandoned?
• Will callers need heterogeneous storage or state erasure?
• Does typestate remove a consequential caller mistake, or only multiply types?
• Have compile time, diagnostics, documentation surface, and code size been inspected for the real API?

Exercise: let requirements break both designs

Implement the handshake first as HandshakeState::apply(event). Preserve the prior state and rejected event on failure. Add timeout, graceful close, and a restored record with an impossible state-field combination; write runtime tests that show where decoding and transition validation differ.

Then implement the local happy path as Connection<State>. Capture compiler evidence that session() is unavailable before accept, but do not stop at that success. Add a requirement to store ten thousand connections in mixed phases, resume a negotiating connection after restart, and close any live phase. Decide where the typed path must return to runtime data.

Write the boundary decision as part of the exercise result: which representation is the system of record, which local sequence earns typestate, which constructors remain private, and what evidence a marker does not provide. A solution that lets persistence or callers manufacture marker states has avoided the central problem.

Durable takeaways

1. An enum can make legal runtime states explicit while a transition function validates events.
2. A consuming typestate API proves local method order and can move phase-specific resources.
3. Persistence, remote agreement, recovery, plugins, and operator actions remain runtime facts.
4. Rejected owned transitions should preserve the state and event needed for recovery.
5. A dynamic system of record with a narrow typed façade is often the honest hybrid.

Part III has moved from legal values through exhaustive decisions and explicit exits to legal changes over time. One representation hazard remains: a state transition can be structurally correct while its counter wraps or its encoded length narrows. The next chapter makes those numeric policies visible.

Sources and version notes

• Rust Reference: struct and enum types and enumerated types
• std::marker::PhantomData
• Rust API Guidelines: type safety
• Runtime and consuming-transition examples compile on Rust 1.93.1. The chapter does not claim language-level session types or remote protocol correctness.