Pinning and Self-Referential Constraints

The cache survived; its pointer did not

A routing cache stored a parsed key inline and, to avoid searching it again, retained a raw pointer to that field. Construction succeeded. Lookups succeeded in unit tests. A later refactor put cache records into a vector, sorted them, and returned one from a helper. The records remained valid Rust values, but an internal pointer still named an earlier address. The first visible symptom was not a move error. It was memory corruption inside an unsafe lookup.

The design had crossed two different contracts:

• an ordinary Rust value may be moved by assignment, return, container operations, or optimizer transformations whenever its semantics permit;
• an address-sensitive value needs some relationship to remain valid at one memory location after a particular state begins.

An ordinary move transfers the value. At the language level, engineers should reason as if its bytes may be relocated, without assuming that every source move produces a machine-code copy. Source and destination bindings are not stable identity anchors. A raw pointer into the old location is not automatically rewritten.

Pin exists so safe APIs can uphold an address-stability invariant required by a narrow class of abstractions. It is not a general “do not mutate” marker, a heap marker, or a way to extend a reference’s lifetime.

Once a value that relies on pinning has entered its pinned state, it must remain valid at that location until its pinned destruction obligation has been discharged. Safe code must not obtain an operation that moves or invalidates it there.

Address sensitivity begins at a state transition

Many values are constructed unpinned and moved freely before becoming address-sensitive. A future returned by an async function can be passed through several combinators before an executor pins and polls it. An intrusive node may be assembled before being linked. A self-referential record must reach final storage before its internal pointer is initialized.

That transition matters more than the type’s creation time. Review it as a small protocol:

1. Allocate or otherwise choose the final storage.
2. Establish pinning with a sound pointer constructor.
3. Initialize address-dependent state without moving the pointee again.
4. Expose only operations that preserve the invariant.
5. On destruction, detach dependants or otherwise satisfy the pinned drop contract before storage is reused.

Pinning too early can make initialization needlessly awkward. Pinning too late allows a derived pointer or registration to outlive the address it describes. A safe constructor should make the transition indivisible to callers.

The example crate’s PinnedRecord::new first calls Box::pin, then stores a pointer to its own label field. Its public result is Pin<Box<PinnedRecord>>. The contained unsafe blocks are small, carry proof comments, and are not caller obligations. A PhantomPinned field prevents the record from automatically implementing Unpin:

pub struct PinnedRecord {
    label: String,
    label_field: NonNull<String>,
    _pin: PhantomPinned,
}

pub fn new(label: impl Into<String>) -> Pin<Box<PinnedRecord>> {
    let mut record = Box::pin(PinnedRecord {
        label: label.into(),
        label_field: NonNull::dangling(),
        _pin: PhantomPinned,
    });

    let field = NonNull::from(&record.as_ref().get_ref().label);
    // SAFETY: the pointee now has its final address, and the API never moves
    // or mutably exposes `label` after this pointer is installed.
    unsafe { record.as_mut().get_unchecked_mut().label_field = field };
    record
}

This is evidence about one abstraction, not a self-reference template. The safer production repair is often to remove the pointer: store an index, byte range, key, or separate owned allocation; recompute a cheap derived view; or arrange the owner so the referenced data has an independent stable address. Every removed unsafe relationship reduces the proof surface.

Pin<P> wraps a pointer, then constrains access to its pointee

Pin<P> is a wrapper around a pointer-like P. Examples include Pin<&mut T>, Pin<&T>, and Pin<Box<T>>. The wrapper does not pin its own stack slot. Moving a Pin<Box<T>> handle is ordinarily harmless because moving the box handle does not relocate its heap pointee. Similarly, reborrowing a Pin<&mut T> produces another pointer value to the same location.

This pointer/pointee distinction continues Chapter 30’s model. The metadata or wrapper is a sized value that may move. The contract concerns the value reached through it.

Nor does Pin promise:

• physical immutability: interior mutability may still change state;
• unique access in every form: Pin<&T> is shared, and APIs may create multiple shared pins;
• 'static lifetime: a locally pinned value remains bounded by its storage lifetime;
• heap allocation: pin! can pin a local value without requesting a new heap allocation;
• that a type cannot be leaked: safe Rust permits mem::forget, so soundness may not depend on every destructor running;
• that all fields are pinned: structural projection is an API decision.

For T: Unpin, pinning deliberately imposes no additional move restriction. Pin::new and Pin::get_mut are safe in that case because moving T cannot violate an address-sensitive invariant. Most ordinary Rust types are Unpin, including many pointer handles. Unpin is an auto trait and a statement about whether pinning restrictions matter for the pointee, not a statement that the current value happens to be on a movable stack.

A negative marker such as PhantomPinned opts a type out of automatic Unpin. Manually implementing Unpin for an address-sensitive type is a safety-critical claim even though the implementation syntax itself is safe. It can re-enable mem::replace through a pinned mutable reference and invalidate the entire abstraction.

Local pinning and heap pinning answer different ownership questions

Box::pin(value) gives ownership plus stable heap storage for the pointee. It is a good fit when the pinned value must be returned, stored heterogeneously, or outlive the constructing scope. The allocation and indirection are part of its cost.

The standard pin! macro constructs a Pin<&mut T> for a local value. It does not itself request a heap allocation and is useful for manually polling a future or using a pinned value within a bounded scope. “Local pinning” is more precise than “stack pinning”: inside an async state machine, a local that crosses .await resides in the future’s storage, which the executor may itself keep on a stack or heap.

use std::pin::pin;

let record = PinnedRecord::new("heap owned");
assert_eq!(record.as_ref().label(), "heap owned");

let future = async { 7_u32 };
let mut local = pin!(future);
// An executor can poll `local.as_mut()` while this storage remains alive.

Do not construct Pin<&mut T> with Pin::new_unchecked merely because the current address looks stable. The caller must prove that the pointee will remain valid at that address for the entire promised duration and that the pointer type cannot later expose a moving operation. Pinning through a pointer whose DerefMut implementation can move or swap its target is unsound.

Projection decides which fields inherit pinning

Given Pin<&mut Outer>, code often needs to call a pinned operation on an inner field. Converting that outer pin into Pin<&mut Field> is projection. It is not ordinary field borrowing because the projection promises the field cannot be moved independently while the outer value remains pinned.

An abstraction chooses whether each field is structurally pinned:

• A nested future in a future combinator generally must be structurally pinned so it can be polled with Pin<&mut InnerFuture>.
• A counter, cached status, or waker slot may be unpinned state and safely accessible by &mut if changing it cannot violate the outer address contract.
• A String field referenced by an internal pointer must not be replaceable merely because String: Unpin; the outer API has chosen to make that field part of its address-sensitive structure.

Hand-writing projection uses unsafe operations because the compiler cannot infer those choices. Established projection libraries can generate the boilerplate and align Unpin and pinned-drop behavior with declared fields, but the design proof remains yours. Application code should prefer a safe library abstraction or eliminate address sensitivity rather than scatter map_unchecked_mut calls.

Projection also constrains evolution. Adding a field, changing which fields are structurally pinned, or adding a manual Unpin implementation can invalidate earlier proof comments. Treat projection declarations as part of the type’s safety case and test them across supported feature combinations.

Destruction is part of the address contract

Pinning promises more than “no move.” The pinned location must not be invalidated or repurposed before the value’s destruction obligation is handled. Custom storage that frees or reuses memory must run drop_in_place first where the pin contract requires it. An intrusive node may need to unlink itself while its address is still meaningful.

For a type that may be pinned and depends on that fact, reason about Drop::drop(&mut self) as if it received a pinned mutable receiver. The ordinary signature must not be used to move structurally pinned fields. A projection facility with pinned-drop support can enforce the intended access shape.

Two qualifications prevent dangerous overclaims:

1. Safe Rust can leak values with mem::forget, and process abort or exit can bypass ordinary cleanup. A memory-safety proof cannot require that every destructor eventually runs.
2. Panic during destruction changes which cleanup completes. Unsafe intrusive or registered structures need unwind analysis, and production services need a clear panic strategy.

Pinned set-style replacement is possible through APIs that first drop the old value in place and then initialize a new value at the same address. That is not permission to overwrite storage with ptr::write while dependants still believe the old pinned value exists.

Why Future::poll takes Pin<&mut Self>

The Future trait’s central method is:

fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output>;

An async function is compiled into a state machine whose locals may live across suspension points. Some generated futures can contain relationships that become address-sensitive once polling starts: one state may refer to data stored elsewhere inside the same future. The executor must be able to move the future before it begins polling, then keep its pointee in place while repeated polls advance that state.

The receiver supports both kinds of future. A future that is Unpin can be accessed much like an ordinary mutable value. A !Unpin future can rely on address stability after pinning. This does not mean every future is self-referential, that async values are always heap allocated, or that Pin schedules anything. Futures are inert until polled; the context and waker protocol determine when polling should resume.

Combinators that store nested futures commonly require structural projection to poll them. Executors frequently own tasks through pinned boxes, but local executors and scoped polling can choose other storage. For async borrowing, Send, cancellation, and runtime architecture, see Chapter 59; the durable point here is the receiver contract.

Cancellation connects polling to destruction. Dropping a pending future destroys whichever state is currently active. Resources owned by initialized fields are dropped, but application-level rollback is not automatic, destructors are not async, and external side effects already performed remain performed. Address safety and transactional safety are separate requirements.

Patterns that compile and still weaken the design

Reject these in review:

• Pinning Box<T> with Pin<&mut Box<T>> when the intended contract concerns the heap T; the wrong pointee has been pinned.
• Creating Pin<&mut T> unsafely from a temporary or container element that can later move or reallocate.
• Assuming Pin<Box<T>> cannot be moved; the handle can move while T remains in place.
• Exposing &mut to a structurally pinned field through a convenience accessor.
• Implementing Unpin to make an error disappear without proving address insensitivity.
• Initializing an internal pointer before the outer value reaches final pinned storage.
• Pointing into a String or Vec buffer and then allowing operations that may reallocate that buffer. Pinning the owner object does not pin a separately allocated buffer’s elements against its own mutation policy.
• Treating a raw pointer as a lifetime extension. It remains subject to validity, aliasing, initialization, and provenance requirements.
• Depending on Drop to run for memory safety or to complete external rollback.
• Using self-reference where an offset, key, handle, separate box, or recomputation would remove unsafe code.

The cache incident’s strongest repair stores a byte range into an immutable backing buffer. A method validates the range and creates a borrow on demand. Moving the cache moves integers, not a pointer whose validity depends on its old enclosing address. If the backing buffer itself may move independently, own it behind an appropriate stable allocation or shared owner.

Four alternatives to a self-reference

Before approving a pinned self-reference, compare four designs against the operation that supposedly needs it.

Reborrow from an owner

Keep the owner and derive a view only for the duration of a method call:

struct ParsedKey {
    bytes: Box<[u8]>,
    key_range: std::ops::Range<usize>,
}

impl ParsedKey {
    fn key(&self) -> &[u8] {
        &self.bytes[self.key_range.clone()]
    }
}

The range survives moves, the box handle may move without relocating its byte allocation, and the returned borrow cannot outlive self. Construction must validate the range once, and mutation that changes buffer length must remain unavailable. This design is often a stronger “zero-copy” result than an internal raw reference because callers get the same borrowed view without inheriting an unsafe address proof.

Store an offset or handle

Offsets work when the backing sequence remains logically stable but may relocate. Handles work when a collection may reuse storage and needs a generation check. Resolution adds bounds or generation validation, usually a small price for explicit identity and better diagnostics. It also makes serialization possible if the format defines offsets independently of process addresses.

An offset is not automatically valid. Arithmetic must be checked, the range must lie on required character or element boundaries, and any mutation that changes interpretation must invalidate or update it. The difference is that these are ordinary value invariants rather than pointer-provenance and lifetime obligations.

Give the referent independent ownership

Place the referenced object in Box, Arc, or another owner whose pointee address and lifetime match the use. The outer record can move while the separately allocated pointee stays put. Arc adds atomic reference counting and potentially shared lifetime; it does not make mutation safe or eliminate cycles. Box adds one allocation but keeps unique ownership simple.

This approach is particularly credible when the referent already has a domain identity or is shared by multiple records. It is weaker when it introduces thousands of small allocations solely to avoid a cheap range calculation.

Recompute the derived value

A cache pointer often exists to avoid work that was never measured. Recomputing a hash, slice, parsed flag, or table lookup may be cheaper than an allocation or the maintenance burden of unsafe self-reference. If computation is expensive, cache an owned derived result rather than a borrowed pointer when its size is bounded.

Compare the alternatives on allocation count, lookup cost, memory footprint, invalidation complexity, serialization, and audit surface. The pinned design should win on a named constraint, not on aesthetic closeness to a C or C++ representation.

Audit the unsafe boundary as a proof graph

A sound constructor is necessary but insufficient if another method later breaks the relationship. Build a proof graph whose nodes are unsafe operations and whose edges are the invariants they consume or establish.

For PinnedRecord, allocation establishes stable storage; pointer initialization consumes that fact and establishes “label_field names label”; the accessor consumes pointer validity; and every future API must preserve the field in place. Adding fn label_mut(&mut self) -> &mut String, even in safe code, would allow replacement or buffer reallocation that may invalidate a different form of internal pointer. Adding serialization code that reconstructs raw address bits would bypass construction entirely.

Review at least these mutation paths:

• mem::replace, mem::take, swaps, and pattern moves;
• container reallocation before and after pinning;
• projection to fields, including generic blanket implementations;
• Drop, panic during initialization, and replacement in place;
• deserialization, cloning, and conversion constructors;
• feature-gated methods and downstream trait implementations;
• thread transfer and interior mutation.

Run Miri over successful and failing state transitions where its model covers the operations. Use compile-fail tests for forbidden safe access, but remember that compiler rejection of one expression does not prove all expressions are rejected. The safety documentation should state the validity invariant, not merely narrate the current unsafe line.

Operational review: pinning has costs beyond syntax

Heap pinning introduces allocation and indirection where local pinning might not. Boxing many small futures can affect allocator contention and locality; embedding very large futures can increase task footprint. Measure task sizes, allocation counts, poll latency, cancellation cleanup, and queue retention rather than attributing cost to dynamic dispatch or pinning by inspection.

Unsafe pin abstractions should expose diagnostics that do not reveal addresses as stable identifiers. Addresses can be useful in debug logs during a controlled investigation, but allocator reuse makes them misleading identifiers and address disclosure may be a security concern. Prefer stable node IDs, task IDs, and state names.

Maintenance evidence should include:

• tests that move pointer handles while confirming the pointee relationship remains valid;
• compile-fail coverage for prohibited move-out operations;
• Miri or other appropriate dynamic checking for unsafe pointer paths;
• panic and cancellation tests around partially initialized states;
• proof comments adjacent to each unsafe operation;
• documentation of which fields are structurally pinned and why.

The included pinning-allocation-lab verifies safe construction and handle movement on Rust 1.97.0 and the declared Rust 1.85 MSRV. Its doctest confirms that moving a PinnedRecord out of its pinned box is rejected. That compiler evidence supports the example; it does not replace independent unsafe-code review.

Exercise: remove address sensitivity before approving it

Review a cache proposed with these fields: an owned byte buffer, a parsed header containing raw pointers into the buffer, an intrusive expiry link, and a cached future that refreshes the entry.

Produce two designs. First, eliminate self-reference where possible: use validated byte ranges, a generational expiry handle, and a refresh operation owned separately from the cache entry. Second, if any address-sensitive state remains, provide a safety case covering:

1. the exact transition at which address sensitivity begins;
2. final storage and the safe constructor sequence;
3. every structurally pinned and unpinned field;
4. operations that could reallocate an inner buffer;
5. destruction, leak, panic, cancellation, and partial initialization;
6. thread-safety and aliasing;
7. tests, compile-fail fixtures, Miri coverage, and supported toolchains;
8. allocation, locality, and maintenance costs versus the handle/range design.

Approve pinning only if the second design earns a material benefit that the first cannot provide. Record that benefit as a measurable requirement rather than “zero-copy” or “faster” by assumption.

Pinning review card

• Which relationship becomes invalid if this pointee moves, and when does that state begin?
• Is the pointer wrapper being confused with its pointee?
• Does T: Unpin make pinning restrictions irrelevant?
• Is local pinning sufficient, or is owned heap storage required?
• Does construction pin before installing address-dependent state?
• Which fields are structurally pinned, and how is projection generated and reviewed?
• Can an inner String, Vec, or allocation still reallocate independently?
• Does destruction satisfy the pin contract without assuming cleanup always runs?
• What happens on panic, cancellation, leak, and partial initialization?
• Could an offset, handle, separate allocation, or recomputation delete the unsafe relationship?

The contract to carry forward

Ordinary Rust moves do not preserve a source address. Pin<P> constrains how safe code may access the pointer’s pointee after pinning; it does not freeze the pointer handle, create 'static lifetime, require the heap, or make every field structurally pinned. Unpin identifies types for which those move restrictions are unnecessary.

Address-sensitive abstractions must align construction, projection, and destruction around one proof. Future::poll uses a pinned receiver so executors can support state machines that may require stability without burdening futures that are Unpin. Most application data should avoid self-reference. Handles, ranges, ownership separation, and recomputation usually produce a smaller safety case.

Chapter 32 turns from the stability of one address to the policy for obtaining and reusing many storage locations. Once allocation strategy changes ownership, identity, or reset behavior, it has become an architecture decision.

Sources and version notes

• Standard library: std::pin module and pinning contract
• Standard library: Pin, Unpin, and pin!
• Standard library: Future::poll
• Rust Reference: destructors
• Async Book: pinning
• The dependency-free pinning-allocation-lab uses Rust 2024, pinned Rust 1.97.0, and MSRV 1.85. Its unsafe code is limited to establishing and reading one internal self-reference behind a safe constructor. The independent editor must audit that safety case and revalidate compiler-sensitive evidence.