Appendix G — Numeric and Bitwise Correctness Checklist
Audit Rust numeric and bitwise code for explicit width, conversion, overflow, endian, shift, floating-point, nonzero, and flag policies.
Eight bytes arrive from a peer:
01 05 00 03 00 00 01 02
The relay protocol assigns them this shape:
byte 0 version
byte 1 flags; only bits 0..=2 are defined
bytes 2..4 nonzero payload length, big-endian u16
bytes 4..8 sequence number, big-endian u32
Reading the fields is the easy part. Correctness depends on the questions around each read. Is the slice long enough? Are reserved flag bits rejected or preserved? Is zero a valid length? Can adding the header offset and payload length overflow usize? Does the peer’s byte order become the host’s byte order explicitly? What happens when the sequence wraps?
Numeric review is therefore not a search for arithmetic operators. It is an operation-by-operation policy audit. Every representation, conversion, arithmetic operation, shift, float admission, and bit merge needs a meaning for values at and outside its boundary. Rust supplies several precise mechanisms; it cannot choose the domain policy.
The operation-policy matrix
Print this table for reviews. Fill the policy column before choosing the method.
| Operation | Questions that must be answered | Usually explicit Rust mechanism | Evidence to require |
|---|---|---|---|
| external bytes to integer | width, signedness, byte order, short input | fixed-size slice/array plus from_be_bytes or from_le_bytes |
golden byte vectors and truncated input |
| integer narrowing or sign change | accepted range, rejection behavior | TryFrom / try_into |
minimum, maximum, one below, one above |
| in-memory size/index | address-space dependence, addition/multiplication overflow | usize only at the memory boundary; checked_* for hostile extents |
maximum-length and overflow cases |
| domain addition/subtraction | is excess invalid, clamped, modular, or separately reported? | checked_*, saturating_*, wrapping_*, or overflowing_* |
boundary transition and downstream consequence |
| shift | valid count, invalid-count policy, shift versus rotation | checked_shl/checked_shr, rotate_left/rotate_right |
zero, BITS - 1, BITS, and hostile count |
| floating input/result | NaN, infinity, signed zero, subnormal, range, precision | admission newtype or validation using classification methods | every rejected class and serialization round trip |
| nonzero value | is zero impossible or merely unusual? | NonZero*::new at the boundary |
zero rejection and Option behavior |
| flags | known mask, unknown-bit compatibility, mutually exclusive bits | newtype plus named masks and constructor policy | each named bit, combinations, unknown bits |
The choice belongs to the operation, not the primitive type. A single u64 can be a checked byte count, a saturating dashboard total, a wrapping protocol sequence, or one half of an overflowing arithmetic primitive. Naming the type does not name the policy.
For every row, record the expected result with overflow checks enabled and disabled. If that result changes, an ordinary operator is still standing in for an unstated domain policy.
Decode representation before doing arithmetic
Protocol and file formats should use fixed-width integers because their representation must not change with the target. u16, i32, and u64 state a bit width. usize and isize state a pointer-sized machine quantity; they are appropriate for indexing and allocation sizes after validation, not for a portable wire field.
The companion fixture decodes the frame without alignment assumptions or native-endian dependence:
let payload_len = u16::from_be_bytes([header[2], header[3]]);
let sequence = u32::from_be_bytes([
header[4], header[5], header[6], header[7],
]);
from_be_bytes describes the protocol. from_ne_bytes would describe whichever machine happened to run the process. to_be_bytes and to_le_bytes do the inverse when encoding. Use swap_bytes, to_be, and to_le when the input is already an integer whose byte interpretation is known; do not scatter conditional target-endian branches through a parser.
Convert to usize only when entering a slice or allocation operation. A u16 always fits in usize on Rust’s supported targets, so usize::from(payload_len) is infallible. Wider protocol lengths may not fit; use usize::try_from and return a representation error. After conversion, addition can still overflow:
let frame_end = header_offset
.checked_add(FrameHeader::ENCODED_LEN)
.and_then(|start| start.checked_add(usize::from(payload_len)))
.ok_or(NumericError::Overflow)?;
This sequence separates three claims: the field decoded, the field fits the address-space type, and the complete extent fits. A later bounds check such as bytes.get(..frame_end) establishes that the input actually contains that extent.
Conversion review
Use From only when the conversion cannot fail and does not discard information relevant to the contract. Use TryFrom when range or validity can reject the value. as casts have specified numeric behavior, but their compactness hides the design decision: integer narrowing truncates, signedness changes reinterpret modulo the destination width, and float-to-integer casts saturate with NaN becoming zero. Those rules are occasionally the policy; they should not become it accidentally.
For each cast, write one sentence:
Convert ___ to ___; values outside ___ are ___ because ___.
If “truncated” or “saturated” is the answer, name that behavior in a helper and test it. For identifiers, lengths, money, quotas, and timestamps, silent narrowing is usually rejection-worthy. For extracting a documented low byte from a word, masking followed by a cast can be exact and reviewable.
Select overflow semantics by domain
The ordinary integer operators are a poor way to encode deliberate overflow policy. Rust’s overflow checks can differ with compiler settings: debug builds normally panic on overflow, while builds with checks disabled produce wrapped two’s-complement results. Division of the minimum signed value by -1 and invalid shifts retain special checks. A correctness argument that says “tests panic” or “release wraps” has coupled domain behavior to build configuration.
Use the named families instead:
| Family | Result on overflow | Appropriate when | Misuse signal |
|---|---|---|---|
checked_* |
None |
excess is a recoverable invalid operation | caller immediately unwraps external-data arithmetic |
saturating_* |
clamps at numeric bound | the domain explicitly accumulates no further past a cap | saturation hides lost money, bytes, or sequence identity |
wrapping_* |
modular result | modular arithmetic is the contract | wrap is used merely to avoid a panic |
overflowing_* |
modular result plus overflow flag | algorithm needs both pieces | flag is ignored |
strict_* |
always panics on overflow | overflow proves an internal invariant bug and panic policy permits it | used on untrusted or normal boundary input |
The strict_* integer family is stable in the current standard library, but its minimum supported Rust version must be checked before using it in an MSRV-bound public fixture. This appendix’s executable fixture uses checked_*, which is available at its declared Rust 1.85 baseline.
Consider four counters:
- A buffer extent uses
checked_add; overflow means the frame cannot be represented. - A rate-limit token count may use
saturating_addif “at capacity” and “beyond capacity” are intentionally identical. - A protocol serial number may use
wrapping_addif the protocol defines comparison across modular sequence space. - A multiprecision limb operation may use
overflowing_addbecause the carry bit feeds the next limb.
None is the universal “safe” method. Safety comes from matching behavior to meaning and testing the transition.
Signed arithmetic needs additional cases: negating or taking the absolute value of the minimum signed integer cannot produce a positive value of the same type. Division by zero is not an overflow policy and still fails. Multiplication used to compute capacity deserves the same checked reasoning as addition. When subtracting timestamps or counters, decide whether reversal is an error, a signed result, or modular distance.
Shift counts and rotations are different operations
An external bit position is data. For a u32, positions 0..32 are valid; 32 is not. Encode that boundary:
fn bit_at(position: u32) -> Result<u32, NumericError> {
1_u32
.checked_shl(position)
.ok_or(NumericError::InvalidShift { shift: position })
}
wrapping_shl reduces the count modulo the width. That is useful only when modular shift counts are deliberate. overflowing_shl returns a value and a flag indicating that the count exceeded the width; ignoring the flag hides the same issue. An unchecked shift makes the valid-count precondition a safety obligation and does not belong at an unvalidated boundary.
A rotation is not a shift with a convenient overflow behavior. Rotation preserves every bit and moves bits leaving one end back into the other. Use rotate_left or rotate_right for cryptographic primitives, hash mixing, and circular fields whose specification calls for rotation. Use a shift when discarded bits are meant to disappear, then review whether discarded nonzero bits matter.
Admit floating-point values into a smaller domain
f32 and f64 include finite normal numbers, subnormal numbers, positive and negative infinity, multiple NaN bit patterns, and positive and negative zero. NaN != NaN, ordinary comparisons with NaN are unordered, and -0.0 == 0.0 even though their sign bits differ. A floating-point field is therefore larger than most application domains.
Validate at admission:
let ratio = numerator as f64 / denominator as f64;
if !ratio.is_finite() {
return Err(NumericError::NonFiniteRatio);
}
Then decide the remaining policy:
- Is zero allowed, and are both signs of zero normalized?
- Are negative values valid?
- Are subnormal values accepted, flushed by an external platform, or operationally meaningless?
- What closed or open range applies?
- Is exact decimal behavior required? Binary floating point is usually the wrong representation for exact monetary decimals.
- Does serialization preserve the required value model and special values?
Do not use a blanket abs(a - b) < EPSILON. f64::EPSILON describes spacing near 1.0, not a domain tolerance for every magnitude. Choose an absolute tolerance, relative tolerance, units-in-last-place method, or exact bit comparison according to the computation and scale. State how accumulated error is bounded.
Rust floats implement PartialEq and PartialOrd, not Eq and Ord. If a sort must include every bit pattern, total_cmp provides a total ordering, including distinct positions for signed zeros and NaNs. That ordering is a mechanism, not a claim that NaN is valid in a business key. Often the stronger design is a validated finite newtype whose equality and ordering contract is documented.
Put zero and flag policy in constructors
NonZeroU16 makes zero unrepresentable after construction:
let payload_len = NonZeroU16::new(decoded)
.ok_or(NumericError::ZeroPayload)?;
This is useful when every later operation relies on nonzero: divisors, identifiers, capacities, and protocol lengths are common examples. The standard library guarantees useful layout properties for NonZero and Option<NonZero<_>>, but choose the type for its invariant first. Do not introduce it merely because an optimization might occur, and never call new_unchecked unless a local proof establishes nonzero at that exact unsafe boundary.
Bitflags need a compatibility policy. Suppose only 0b0000_0111 is defined:
let unknown = bits & !KNOWN_FLAGS;
if unknown != 0 {
return Err(NumericError::UnknownFlags { unknown });
}
Rejection is right for a closed protocol version where unknown behavior is unsafe. Other boundaries may preserve unknown bits when decoding and re-encoding to support forward compatibility, or ignore them for behavior while retaining the raw word for telemetry. Clearing unknown bits silently destroys round-trip information. Treating every bit as known silently opts into behavior the implementation does not understand.
Also record:
- whether named flags may be combined;
- whether fields are masks rather than independent booleans;
- whether two bits are mutually exclusive;
- which value means “none” or “default”;
- whether bitwise NOT is restricted back to the known mask;
- how new bits change version negotiation and public APIs.
Use a newtype with named constants and controlled constructors. A bare integer invites arbitrary masks into every caller.
Packet-boundary audit exercise
Audit a telemetry frame containing a u32 payload length, u64 timestamp, i16 temperature, f32 utilization, eight flag bits, and a sequence counter.
- Write the exact byte order and width of each field. Reject “machine endian.”
- Mark the first conversion into
usize; prove both conversion and extent arithmetic. - Choose checked, saturating, wrapping, overflowing, or strict behavior for every arithmetic operation. Give the domain reason.
- Specify admissible float classes, range, zero normalization, and comparison policy.
- Define known flags and the unknown-bit compatibility rule.
- Add tests at zero, both integer bounds, one outside every accepted range, a truncated frame, NaN, both infinities, signed zero, every known flag, an unknown flag, shift counts 0, width minus one, and width.
- Rehearse the same tests with overflow checks explicitly enabled and disabled. Named policy should keep results unchanged.
Finish by recording five decisions in the design ledger:
| Value | Representation | Boundary policy | Arithmetic policy | Failure/result |
|---|---|---|---|---|
| counter | ||||
| quota | ||||
| protocol length | ||||
| timeout | ||||
| flag word |
Numeric and bitwise review card
- Wire, file, database, and ABI fields use explicit widths and signedness.
- Byte order is named at every serialization boundary.
- Every narrowing or signedness change states range and rejection behavior.
-
usizeappears only where address-space sizing or indexing is intended. - Addition, subtraction, multiplication, negation, absolute value, division, and remainder have boundary tests where used.
- Overflow behavior is named and independent of debug/release settings.
- External shift counts are validated; rotations are used only when specified.
- Float admission addresses NaN, infinities, signed zero, subnormals, range, precision, equality, and ordering as relevant.
-
NonZerorepresents a domain invariant established by a checked constructor. - Bit masks define known bits, legal combinations, and unknown-bit handling.
- Tests include hostile values and observable downstream effects, not only arithmetic results.
- Version-sensitive methods are checked against stable and the declared MSRV.
The audit ends when every operation has one intentional policy and evidence at its boundary. The next question is no longer whether malformed data fails, but how that failure should travel through the system. Appendix H supplies that error and panic design contract.
Sources and version notes
- The Rust Reference overflow rules define ordinary integer overflow and invalid shift behavior.
- The standard library’s
u32documentation documents checked, saturating, wrapping, overflowing, strict, endian, shift, and rotation operations. Check each method’s stabilization version against the project MSRV. TryFromand the Reference’s numeric cast semantics define explicit conversion mechanisms.f64documents IEEE-754 classes, classification, byte conversion, andtotal_cmpbehavior.NonZerodocuments its validity and guaranteed layout properties.- The companion fixture is
examples/rust-engineering-handbook/appendices/numeric-error-policy-lab/. It targets Rust 2024, declares Rust 1.85 as MSRV, and deliberately avoids newer helpers where MSRV availability is uncertain. Publication stable and the MSRV require independent revalidation.
Continue reading
Full table of contents