C ABI and FFI: Ownership, Errors, Callbacks, and Unwinding

Start a C API review with a table, not an extern declaration. For every pointer crossing the boundary, write who owns it before the call, who owns it after the call, how long it remains valid, whether it may be null, which thread may use it, and which function eventually releases it. If any cell says “obvious,” the boundary is underspecified.

Rust’s types normally carry these facts through moves, borrows, lifetimes, Option, Result, and destructors. A C signature reduces them to addresses, integers, and calling convention. The foreign caller cannot infer that *const u8 is borrowed for one call, that an opaque pointer must be released exactly once by Rust, or that an output is initialized only on success. A sound C ABI makes ownership and failure semantics explicit in the protocol around the signature.

This chapter designs that protocol for wireview, the handbook’s binary parser. The boundary accepts a caller-owned record, stores only a parsed value in an opaque Rust-owned handle, reports stable status codes, invokes a synchronous callback without retaining it, and contains Rust panics. It is intentionally smaller than a binding system. Cross-language wrappers, managed runtimes, packaging, and process isolation follow in the next chapter.

Fix the ABI before exporting symbols

An ABI specifies low-level calling details: argument and result passing, register use, stack discipline, data layout assumptions, symbol linkage, and unwind behavior. extern "C" asks Rust to use the platform’s C calling convention for the function. It does not turn arbitrary Rust types into C-compatible types, stabilize Rust’s native ABI, choose allocation ownership, or make a panic safe to cross the boundary.

Use C-compatible surface types:

• fixed-width integers from <stdint.h> when width is part of the contract;
• size_t/Rust usize for in-process object sizes where the target C ABI agrees;
• raw pointers plus explicit length for borrowed buffers;
• #[repr(C)] structs or enums only after checking their precise cross-language representation;
• opaque incomplete C structs for Rust-owned state;
• extern "C" function pointers for callbacks;
• explicit status values rather than Rust Result, references, slices, strings, trait objects, or generic types.

The Rust 2024 spelling for an exported unmangled symbol is explicit about the unsafe attribute:

#[unsafe(no_mangle)]
pub extern "C" fn wireview_abi_version() -> u32 {
    1
}

no_mangle makes a predictable external symbol name; it also enters a global namespace where collisions can be unsound. Prefix public symbols with a library identifier. Control symbol visibility and exported symbol sets through the target linker and packaging process rather than assuming every pub Rust item is part of the ABI.

Some platforms prefix symbols, use import libraries, or require export annotations. Calling conventions also vary. If the API is only supported on selected targets, state and test that matrix. If a platform requires another ABI string, gate it explicitly rather than assuming C means one universal machine contract.

repr(C) provides selected layout facts, not a complete boundary

The lab status enum uses #[repr(C)] so its discriminant representation follows the applicable C-compatible enum rules. The checked-in header assigns the same named integer values. For a long-lived ABI, many teams prefer a fixed-width integer typedef with constants because C enum underlying types and compiler flags can complicate size assumptions. Whichever representation you choose, assert sizes where required and test the header with every supported compiler family.

#[repr(C)] on a struct gives a C-compatible field layout for the target; it does not make every field type FFI-safe, initialize padding, validate pointers, define byte order, or produce a portable file format. Both sides must compile matching declarations for the same target ABI. Bitfields, packed structures, flexible array members, long double, platform-dependent long, and C++ types need target-specific attention.

Never expose a Rust default-layout struct just because today’s compiler places fields as expected. Do not put String, Vec<T>, Box<T>, references, slices, Option without a documented FFI representation, or trait objects directly in a C header. Their layout or validity is Rust-specific, and their allocation and drop behavior is not available to C.

For wire data, keep object layout separate from protocol encoding. wireview_parse treats the input as bytes, checks the tag, and decodes a big-endian integer. It does not cast the input pointer to a Rust header. That copy is four bytes, independent of alignment, and gives the parser an owned valid u32. A borrowed typed view would need additional alignment, validity, aliasing, lifetime, and format-version proofs.

Opaque handles preserve representation freedom

The header declares, but does not define, the handle:

typedef struct wireview_handle wireview_handle;

wireview_status wireview_new(wireview_handle **out_handle);
void wireview_free(wireview_handle *handle);

Rust returns a pointer created by Box::into_raw. C holds that pointer as an opaque token and passes it back to library functions. It cannot depend on Rust field layout. wireview_free reconstructs the matching Box and destroys it exactly once.

The ownership protocol is:

This table is part of the API. Put equivalent statements in the public header, generated documentation, and language bindings.

Allocator matching is non-negotiable. C must not call free(handle) on memory allocated by Rust’s Box, and Rust must not reconstruct a Box from memory allocated by arbitrary C code. Provide paired creation and destruction functions in the same library version. The rule also applies to returned strings and arrays: either the caller supplies the buffer, or the library returns an object with a matching library release function.

Opaque handles need a concurrency contract. The lab excludes concurrent calls to one handle and invokes callbacks on the calling thread. The handle is not advertised as thread-safe. A production API can provide separate handles, internal synchronization, or explicit ownership transfer, but “opaque” does not mean synchronized. Document whether functions may be called concurrently on different handles and on the same handle.

Null handling should be boring and consistent. The constructor rejects a null out pointer. Operations reject null handles, buffers, or result slots. The free function accepts null as a no-op, matching common C cleanup style. After a successful free, the old non-null value is dangling; passing it again is not a benign null case. Encourage callers to set their variable to null after release, and make generated wrappers do so.

The diagram says “owned by Rust” inside the allocation while C owns the opaque token. These are two views of the same protocol: Rust controls representation and deallocation; C controls when the one live token is returned. Neither side may duplicate logical ownership.

Buffers need pointer, length, initialization, and retention rules

A (data, len) pair is not a slice until the Rust entry point proves the conditions required by slice::from_raw_parts: the pointer is non-null and aligned for the element type, the range contains len initialized elements within one allocation, the total size is representable, and the memory is not mutated incompatibly for the borrow duration. Alignment is trivial for u8, but the other requirements remain caller obligations.

Check null before constructing a slice. Check the minimum length before indexing. Avoid pointer.add or slice creation merely to validate a value that can be rejected earlier. The lab constructs the slice after null and length checks, then validates a tag before decoding.

Define zero-length behavior explicitly. Rust slice construction has pointer validity requirements even for length zero; C APIs often allow null with zero. An entry point can special-case len == 0 and avoid constructing a slice, or require a non-null dangling-safe pointer. Pick one policy and test it. The lab requires a non-null buffer and then returns SHORT_INPUT for lengths below five.

Length and capacity pairs require more care than input slices. If Rust receives ownership of a C allocation, it needs the exact allocator and layout contract—not just a pointer, length, and capacity that resemble Vec raw parts. If C receives a Rust vector’s buffer, the API must preserve the original capacity and element layout for the matching Rust release function. Do not invite callers to use realloc on Rust storage.

For caller-provided output buffers, a common protocol is:

1. accept pointer and capacity;
2. return required length when capacity is insufficient;
3. write at most capacity initialized bytes;
4. report the number actually written;
5. define whether a terminating NUL is included;
6. never read uninitialized output storage.

Integer overflow matters when computing byte counts. Validate count * size_of::<T>(), offsets, and cumulative lengths with checked arithmetic before pointer construction. C’s size_t maps naturally to usize only within the same target process; serialized lengths need a fixed-width format and range conversion.

Strings are encodings plus ownership

C strings conventionally use a terminating NUL and have no inherent length. Rust str is length-delimited, valid UTF-8, and may contain interior NUL bytes. A pointer called name does not settle these differences.

Choose one contract:

• borrowed NUL-terminated bytes, validated for termination within a stated maximum;
• pointer plus length bytes, with an explicit encoding such as UTF-8;
• caller-provided output buffer;
• library-owned immutable string valid until a named event;
• newly allocated result with a matching library free function.

Never call CStr::from_ptr unless the caller guarantees a readable NUL occurs before the accessible allocation ends. Searching unbounded foreign memory is not validation. Prefer a length-bearing API when the source can provide it. Convert invalid text according to the domain: reject, preserve bytes, or replace lossily. Do not silently treat platform path bytes as UTF-8.

Returned error messages require a lifetime policy. Thread-local “last error” buffers are convenient but can be overwritten by the next call and complicate nested calls. Caller-provided buffers or owned error objects are more explicit. Status codes should remain usable without requiring an English message.

Errors belong in status codes and outputs

Rust Result<T, E> has no general C ABI representation. The lab returns a stable wireview_status and writes the result through an out parameter only on success:

wireview_status wireview_parse(
    wireview_handle *handle,
    const uint8_t *data,
    size_t len,
    uint32_t *out_value
);

The caller can test the status before reading out_value. On error, the lab leaves the existing slot unchanged. That rule prevents C from observing uninitialized or misleading partial output.

Separate categories that callers act on differently: invalid arguments, malformed input, buffer too small, resource exhaustion, version mismatch, busy/reentrant state, and internal failure. Do not expose Rust enum discriminants or io::ErrorKind values without an explicit stable mapping. Reserve ranges or an unknown code path so newer libraries can report errors to older callers without making their switch statements undefined.

Document retry behavior. A parse error for a fixed input is permanent; allocation failure may be transient but retrying in a loop can worsen pressure; a busy status may be retriable after a callback returns. State whether the handle remains usable after each error and whether partial state was committed.

Out parameters should be initialized transactionally. Compute and validate into Rust locals, update handle state according to the operation’s panic guarantee, then write foreign outputs at the commit point. If multiple outputs must be consistent, either initialize all on success or return one C-compatible result struct with a documented validity rule.

Map sensitive errors deliberately. Parser offsets can help debugging but may expose data shape. System error strings can reveal paths. Panic payloads must not be formatted across a hostile boundary. Keep stable machine-readable status small, and provide opt-in diagnostic facilities with size, lifetime, redaction, and thread rules.

Callbacks are borrowed code plus borrowed state

A C callback usually combines a function pointer and void *userdata:

typedef void (*wireview_record_cb)(uint32_t value, void *userdata);

The userdata restores the environment that a closure would normally capture. Its type, lifetime, thread affinity, and aliasing are entirely contractual. Rust cannot inspect the pointer and discover whether it denotes a live u32, a reference-counted context, or a stack frame that already returned.

The lab makes the narrowest useful promise: wireview_visit invokes the callback synchronously, at most once, on the calling thread; it does not retain the function or userdata; both are invalid for Rust to use after the call returns. The caller must supply an ABI-compatible function and valid userdata for the duration of invocation.

That protocol permits stack-backed userdata:

uint32_t captured = 0;
wireview_visit(handle, capture, &captured);

It would become a use-after-return if the library queued the callback. An asynchronous registration API needs a different design: owned registration handle, explicit unregister/drain operation, rules for callbacks already in flight, thread-safe userdata ownership, and destruction only after quiescence. A mere set_callback(ptr, userdata) is insufficient.

Nullability has two dimensions. The callback function can be optional, which Rust can represent at this ABI as Option<unsafe extern "C" fn(...)> where the platform’s documented nullable-function-pointer representation applies. Userdata may legitimately be null if the callback contract allows it. Do not reject null userdata reflexively or dereference it without a callback-specific guarantee.

Thread affinity must be stated even for synchronous callbacks. The calling thread may be a worker unknown to C; foreign UI or runtime APIs may require an attached or main thread. If Rust chooses the callback thread, provide dispatch hooks or require the caller to marshal events. If callbacks can overlap, userdata needs synchronization and the API must describe ordering.

Reentrancy is distinct from concurrency. The lab marks in_callback and rejects nested visits so a callback cannot recursively enter the same callback protocol. A production parser must decide which other functions are permitted during a callback. “Do not call the library” is easy to state but difficult for logging, destruction, and error hooks; enumerate allowed operations or design callbacks from immutable snapshots.

Never hold an internal mutex while calling unknown foreign code unless lock ownership is part of a carefully constrained contract. The callback may block, call back, acquire locks in reverse order, or trigger destruction. Copy the payload, restore invariants, release locks, then call outward when semantics permit.

Panics and foreign exceptions need containment

An unwinding Rust panic must not escape an extern "C" function. With Rust’s unwinding panic runtime, reaching that non-unwinding ABI boundary aborts the process; a native foreign unwind entering through the same kind of boundary is undefined behavior. More importantly, a foreign caller has no Rust panic protocol. Each exported entry point needs a clear policy.

The lab wraps entry-point work with catch_unwind(AssertUnwindSafe(...)) and maps an unwinding panic to WIREVIEW_PANIC. This is a containment belt, not the first line of soundness. Before a panic is caught:

• internal unsafe code must remain valid on unwind;
• guards must restore or poison partial state;
• no Rust reference may outlive foreign storage;
• owned outputs must not have been published halfway;
• panic payload destruction must remain safe.

catch_unwind catches unwinding panics, not builds configured to abort. It also does not catch segmentation faults, C longjmp, hardware exceptions, or arbitrary foreign exceptions. The UnwindSafe traits are advisory for this boundary; AssertUnwindSafe records a manual judgment that internal state has an acceptable unwind story.

Do not let a C++ exception unwind through Rust frames using a plain C ABI. ABI variants that support unwinding exist in specific circumstances, but cross-language exception interoperability remains target- and toolchain-sensitive. The robust baseline translates exceptions and panics into status values at their native boundary. If a C callback can throw or long-jump, require a C/C++ shim that catches before returning to Rust.

Destructors called during containment must not panic. Logging the panic should avoid allocation-heavy or reentrant infrastructure where possible. Return a generic internal status and record diagnostics through a separately designed channel. If continuing could expose corrupt application state, aborting may be safer than pretending the operation failed normally; make that component policy explicit.

extern declarations import obligations too

Calling C from Rust is the mirror image of exporting Rust. In Rust 2024, foreign declarations live in an unsafe extern block because the compiler cannot validate the declaration against the linked symbol:

unsafe extern "C" {
    safe fn stable_library_version() -> u32;
    unsafe fn parse_into(data: *const u8, len: usize, out: *mut u32) -> i32;
}

Marking an individual foreign function safe is a strong assertion: every Rust caller satisfying its ordinary type signature can call it without extra safety preconditions. Most pointer-bearing functions remain unsafe and should be wrapped immediately in a safe Rust API that validates lengths, lifetimes, return codes, and ownership.

The declaration must match the actual header in symbol name, ABI, parameter and return representation, constness-relevant access, and variadic status. A mismatched signature can corrupt registers or stack before application validation runs. Generate bindings where appropriate, pin the header version, and compile a C or C++ smoke program against the produced library. Link success alone does not prove matching semantics.

Foreign global variables are especially hazardous because mutation, initialization order, thread safety, and lifetime may be unclear. Prefer accessor functions. Variadic functions have additional promotion and ABI rules; hide them behind a typed shim rather than exposing them to ordinary application code.

Headers are versioned build artifacts

The checked-in wireview.h is an inspectable artifact, not proof that header and Rust can never drift. Production options include generating C headers from annotated Rust, generating Rust declarations from authoritative C headers, or maintaining a small manual ABI with automated structural tests. Choose one source of truth and make drift fail CI.

Verification should include:

• compile the Rust library for every supported target and profile;
• compile the public header as C and C++ with supported compilers and strict warnings;
• link and run a smoke caller against the actual shared or static artifact;
• compare sizes, alignments, constant values, and calling behavior;
• inspect exported symbols and visibility;
• test null, zero-length, malformed, allocation-failure, repeated-free misuse in an isolated process, callback, reentrancy, and panic paths;
• run sanitizers on both sides where supported;
• verify oldest supported header with newest compatible library and the reverse combinations promised.

Header generators are development dependencies with versions and release notes. A generator upgrade can rename types, change enum strategy, or alter platform annotations without changing Rust source. Review generated diffs and include the exact generator command in release automation.

Keep source packages, headers, import libraries, shared objects, debug symbols, licenses, and target triples aligned. Dynamic loader search behavior differs across Linux, macOS, and Windows. The ABI may be semantically correct while packaging loads an older library at runtime. Export and log the ABI/library version so incidents can identify the actual binary.

Version the protocol, not just the package

SemVer describes package releases; a C ABI needs concrete compatibility rules. Expose an ABI version function or symbol. Prefix symbols. Decide whether structs are fixed-size by value, caller-sized with a struct_size field, or opaque. Opaque handles preserve the most evolution freedom.

Adding a new function is commonly backward compatible if old symbols remain. Changing an existing signature, enum size, struct field order, calling convention, ownership rule, or callback timing is not. Adding a status code can break callers that treat unknown values as impossible, so document a default branch. Increasing alignment can break caller allocation.

For extensible configuration, use a versioned constructor plus a C-layout options struct carrying its size. The library reads only fields present in the supplied size and requires reserved fields to be zero. This pattern needs careful bounds checks but allows newer libraries to accept older callers. Avoid passing large mutable structs in both directions when an opaque builder or named setter makes ownership clearer.

Do not promise ABI stability across targets you do not test. A library can maintain source API stability while requiring relinking, or maintain a stable C ABI for a precise target matrix. State which one. Rust dependencies behind an opaque implementation can change freely as long as exported behavior, symbols, and allocation protocol remain compatible.

When compatibility becomes too expensive, introduce versioned symbol prefixes or a new constructor rather than silently reinterpret old pointers. A process boundary with a versioned wire protocol may be a better isolation and upgrade boundary for independently deployed components, untrusted inputs, or incompatible runtimes.

Production review: trace one call end to end

For wireview_parse, begin in C. The caller owns a live opaque handle, at least five readable bytes, and a writable uint32_t. It invokes the symbol with the platform C convention. Rust checks pointers before reference or slice construction, checks length before indexing, validates the tag, decodes explicit big-endian bytes, updates internal state, writes the output only at commit, and returns a stable status. No pointer is retained. A later matching free returns handle ownership to Rust exactly once.

Now trace failures. Null produces NULL; short data produces SHORT_INPUT; a wrong tag produces BAD_TAG; none writes the out value. An unwinding internal panic becomes PANIC after unwind-safe cleanup. An abort terminates. Invalid caller claims—dangling pointers, incorrect length, data races, double free, wrong function pointer—remain outside what entry-point checks can fully detect and must be prominent in the C contract.

Trace the callback separately. The handle is live, the function pointer uses the C ABI, userdata is valid for the invocation, and the callback runs on the current thread. The internal state marks callback entry; an RAII guard clears it on return or Rust unwind. The callback and userdata are not retained. Foreign non-local exit is forbidden. The API does not claim callback concurrency or asynchronous registration.

That end-to-end trace is more valuable than a catalogue of FFI types because it exposes where each guarantee changes owners.

Draft the boundary, then try to break it

Design a C API for creating a parser, parsing a record, retrieving a diagnostic, visiting decoded fields, and freeing all resources. Submit these artifacts:

1. Header: use prefixed symbols, fixed-width semantic integers, opaque handles, explicit status codes, and documented nullability.
2. ownership table: cover every pointer at entry, success, failure, callback, and release; name matching allocators.
3. buffer contract: state readable/writable ranges, encoding, length units, zero-length behavior, overflow checks, retention, and output initialization.
4. callback sequence: state thread, ordering, overlap, reentrancy, userdata lifetime, registration/unregistration, and in-flight drain behavior.
5. failure map: translate parser errors, resource failure, panic, foreign exception, cancellation, and unknown future status.
6. compatibility plan: define ABI version discovery, struct evolution, symbol policy, generator pin, supported target/compiler matrix, and old/new tests.
7. executable evidence: build a C caller with strict warnings, run success and error cases, inspect exports, and run available cross-language sanitizers.

Red-team the design with a null pointer and nonzero length, zero pointer and zero length, SIZE_MAX, an interior-NUL string, callback-triggered free, callback after unregister, two threads using one handle, a Rust panic, a C++ exception, an unknown status code, and a caller linked to an older shared object. For each case, decide whether the API safely rejects it, defines it, or names it as caller undefined behavior.

A good FFI API does not make misuse impossible in C. It makes correct use compact, incorrect ownership conspicuous, and wrapper generation faithful. Its documentation is part of the safety boundary.

Security review should also treat foreign pointers as untrusted claims even when the caller is in-process. Validate all values that can be validated before pointer access, cap allocations derived from foreign lengths, and avoid diagnostic formatting that scans unbounded memory. A C ABI is not a privilege boundary—the caller can usually corrupt its own process—but defensive limits turn accidental misuse and compromised plugins into more diagnosable failures. When callers are mutually untrusted or need independent crash recovery, move parsing behind a process boundary.

Finally, inspect the actual export surface. Rust source can look narrow while link settings expose allocator shims, dependency symbols, or unintended functions. Record an allowlist of public wireview_* symbols and compare the built artifact in CI. Symbol visibility, header declarations, and ownership documentation should describe the same API version.

Treat ABI review as a release gate even when Rust implementation changes seem internal. Changing panic strategy, link-time optimization, allocator configuration, minimum operating system, or C compiler flags can alter failure or loading behavior without modifying the header. Re-run the real caller against the packaged artifact, not merely the Rust unit tests.

Make the caller test assert protocol details, not only a zero exit code: version equality, one successful allocation, exact endian decoding, unchanged outputs on rejected input, callback value and thread, and one matching release. Keep negative misuse that may crash—double free, stale handle, invalid ranges—in isolated subprocess or sanitizer jobs so one expected fault cannot corrupt the test runner. This separation makes the supported contract executable without presenting undefined caller behavior as a recoverable library feature.

Field checklist

• Is the ABI and supported target/compiler matrix explicit?
• Are exported names prefixed and visibility controlled?
• Does every aggregate have a documented C representation?
• Is each pointer’s ownership, lifetime, range, nullability, aliasing, and retention stated?
• Are allocation and release paired in the same library contract?
• Are lengths counted in bytes or elements, and are arithmetic overflows checked?
• Are strings given an encoding and termination rule?
• Are outputs initialized only under a documented status?
• Can callers handle unknown future statuses?
• Are callback ABI, thread, ordering, reentrancy, userdata, and unregister rules explicit?
• Are Rust panic and foreign exception paths contained?
• Does imported foreign code sit behind a narrow safe wrapper?
• Is header generation or conformance checked mechanically?
• Are old/new ABI combinations and actual shared-library loading tested?
• Would IPC provide a safer lifecycle, security, or compatibility boundary?

The next design question is broader than C syntax. C is often the lowest common denominator beneath C++, Python, JVM/.NET, JavaScript, and generated binding layers, but each runtime adds its own ownership, thread-affinity, exception, garbage-collection, async, and packaging rules. The stable C core remains valuable only if those bindings preserve rather than blur its contracts.

Sources and version notes

The lab targets Rust 2024 and declares Rust 1.85 as its MSRV. The language and library claims were reviewed against current Rust 1.97.1 documentation; the fixture was executed on Rust 1.93.1 because the declared MSRV and documentation toolchain were not installed locally. Primary references are the Rust Reference on external blocks, external functions, type layout, unsafe attributes, and unwinding; and standard-library documentation for slice::from_raw_parts, Box::into_raw, Box::from_raw, CStr, and catch_unwind. Platform ABI, linker, exception, and packaging claims must be verified against each supported toolchain; the C ABI is target-specific, not a universal serialization format.