I tried to recreate C++ bit fields in Rust

Introduction

This is a story from when I used to do game development. To reduce the size of packets sent over the network, I had packed multiple values into C++ bit fields. I was curious whether the same thing could be done in Rust, so I decided to try it out.

One might think that adding #[repr(C, packed)] would pack a struct at the bit level and allow it to be sent directly over the network. However, this time I decided to pack values into 2 bytes by explicitly converting them using shifts and masks, without using repr(packed).

What is Rust

Rust is a systems programming language that aims to achieve both memory safety and runtime performance. It manages memory through an ownership and borrowing system without using a garbage collector. The compiler detects at compile time many operations that could lead to use-after-free memory references or data races.

Rust also has low-level features for handling memory layout, alignment, pointers, and more. Operations whose safety cannot be verified are explicitly marked as unsafe. However, for the bit packing covered this time, we will build the communication format without using unsafe, relying only on integer bitwise operations and conversion to byte sequences.

Verification Environment

• macOS 26.6.2, arm64
• rustc 1.98.1, host aarch64-apple-darwin
• Apple clang 21.0.0, target arm64-apple-darwin25.6.0
• External Rust crates: none

Target Audience

• Those who have experience using bit fields or struct packing in C/C++
• Those who want to implement binary protocols in Rust
• Those who want to reduce communication volume in games, embedded systems, IoT, etc.

References

• Type layout - The Rust Reference
• Behavior considered undefined - The Rust Reference
• u16 - Rust
• std::ptr::read_unaligned - Rust
• Bit-fields - C++ working draft

Verification

Reproducing C++ Bit Fields

This time, we pack the following 4 fields into 16 bits. The byte order of the packet is big-endian.

Arranging the representative values as specified gives the following bit string.

101 1 1010 01111111

In hexadecimal, this is 0xba7f, and the byte sequence sent over the network is ba 7f.

First, I tried to reproduce the method I used in the past.

#include <array>
#include <bit>
#include <cstddef>
#include <cstdint>
#include <iomanip>
#include <iostream>

struct Header {
  std::uint16_t message_type : 4;
  std::uint16_t compressed : 1;
  std::uint16_t version : 3;
  std::uint16_t payload_len : 8;
};

int main() {
  Header header{.message_type = 10,
                .compressed = 1,
                .version = 5,
                .payload_len = 0x7f};
  const auto bytes = std::bit_cast<std::array<std::byte, sizeof(Header)>>(header);

  std::cout << "size=" << sizeof(Header) << " alignment=" << alignof(Header)
            << '\n';
  std::cout << "fields=" << header.version << ',' << header.compressed << ','
            << header.message_type << ',' << header.payload_len << '\n';
  std::cout << "object bytes=";
  for (const auto byte : bytes) {
    std::cout << ' ' << std::hex << std::setw(2) << std::setfill('0')
              << std::to_integer<unsigned int>(byte);
  }
  std::cout << '\n';
}

Compiling and running produced the following output.

c++ -std=c++20 -Wall -Wextra -pedantic bitfield.cpp -o cpp-bitfield
./cpp-bitfield

size=2 alignment=2
fields=5,1,10,127
object bytes= ba 7f

The struct turned out to be 2 bytes as expected, and the object representation also matched the communication format we defined, ba 7f.

Rust Struct Layout

Next, I created a struct in Rust holding the same 4 logical values. Each field is declared as u8 or bool, and I compare the default repr(Rust), repr(C), and repr(C, packed).

use std::mem::{align_of, size_of};

struct RustHeader {
    version: u8,
    compressed: bool,
    message_type: u8,
    payload_len: u8,
}

#[repr(C)]
struct CHeader {
    version: u8,
    compressed: bool,
    message_type: u8,
    payload_len: u8,
}

#[repr(C, packed)]
struct PackedHeader {
    version: u8,
    compressed: bool,
    message_type: u8,
    payload_len: u8,
}

fn print_layout<T>(name: &str) {
    println!(
        "{name}: size={} bytes, alignment={} byte(s)",
        size_of::<T>(),
        align_of::<T>()
    );
}

fn main() {
    print_layout::<RustHeader>("repr(Rust)");
    print_layout::<CHeader>("repr(C)");
    print_layout::<PackedHeader>("repr(C, packed)");
}

The execution result is as follows.

repr(Rust): size=4 bytes, alignment=1 byte(s)
repr(C): size=4 bytes, alignment=1 byte(s)
repr(C, packed): size=4 bytes, alignment=1 byte(s)

Even with repr(C, packed), the size did not become 2 bytes. Since each field is 1 byte, even removing the padding between fields results in a total of 4 bytes.

This result occurs because repr(packed) does not compress the fields themselves at the bit level. repr(packed) is intended to lower the alignment of a type and reduce padding between fields. Also, specifying repr(packed) alone does not guarantee the field order.

References to Fields in Packed Structs

In addition to the size issue, repr(packed) also requires caution regarding unaligned access. To illustrate the problem of unaligned access, I create a minimal example placing a u16 after a u8.

#[repr(C, packed)]
struct PackedHeader {
    tag: u8,
    payload_len: u16,
}

fn main() {
    let header = PackedHeader {
        tag: 1,
        payload_len: 256,
    };
    let payload_len = &header.payload_len;

    println!("{payload_len}");
}

This code does not compile.

error[E0793]: reference to field of packed struct is unaligned
  --> packed-field-reference.rs:12:23
   |
12 |     let payload_len = &header.payload_len;
   |                       ^^^^^^^^^^^^^^^^^^^

u16 normally requires alignment to a 2-byte boundary. However, in a packed struct, it is not guaranteed to be placed at that boundary. Since creating a regular reference to an unaligned location would be undefined behavior, Rust rejects this at compile time.

Implementation

Within the program, each value is held as a readable Header. Only during communication is this struct converted to and from [u8; 2].

#[derive(Debug, PartialEq)]
pub struct Header {
    pub version: u8,
    pub compressed: bool,
    pub message_type: u8,
    pub payload_len: u8,
}

#[derive(Debug, PartialEq)]
pub enum EncodeError {
    VersionOutOfRange(u8),
    MessageTypeOutOfRange(u8),
}

impl Header {
    pub fn encode(&self) -> Result<[u8; 2], EncodeError> {
        if self.version > 0b111 {
            return Err(EncodeError::VersionOutOfRange(self.version));
        }
        if self.message_type > 0b1111 {
            return Err(EncodeError::MessageTypeOutOfRange(self.message_type));
        }

        let packed = (u16::from(self.version) << 13)
            | (u16::from(self.compressed) << 12)
            | (u16::from(self.message_type) << 8)
            | u16::from(self.payload_len);

        Ok(packed.to_be_bytes())
    }

    pub fn decode(bytes: [u8; 2]) -> Self {
        let packed = u16::from_be_bytes(bytes);

        Self {
            version: ((packed >> 13) & 0b111) as u8,
            compressed: ((packed >> 12) & 0b1) != 0,
            message_type: ((packed >> 8) & 0b1111) as u8,
            payload_len: (packed & 0xff) as u8,
        }
    }
}

In encode, each value is left-shifted to its specified position, then combined into a single u16 using OR. Finally, to_be_bytes is called to fix the byte order to big-endian.

Since version and message_type are u8 in Rust, they can hold values that exceed 3 bits and 4 bits respectively. Silently truncating the excess would make it impossible to detect caller errors, so the value range is validated before encoding.

Converting the representative values produced the expected result according to the specification.

Header: size=4 bytes, alignment=1 byte(s)
wire length: 2 bytes
wire bytes: [ba, 7f]
wire bits:  1011101001111111
decoded: Header { version: 5, compressed: true, message_type: 10, payload_len: 127 }

Testing Fixed Byte Sequences and All Combinations

Simply encoding a value and then decoding it with the same implementation to verify that it returns to its original form would allow tests to pass if both contain the same bit-position error.

Therefore, the golden value [0xba, 0x7f] derived from the specification at the outset was fixed as a constant.

#[test]
fn encodes_known_header_to_expected_bytes() {
    let header = Header {
        version: 5,
        compressed: true,
        message_type: 10,
        payload_len: 0x7f,
    };

    assert_eq!(header.encode(), Ok([0xba, 0x7f]));
}

In addition to the golden value, we verified minimum values, maximum values, and rejection of out-of-specification inputs. The number of possible value combinations under this specification is 8 × 2 × 16 × 256 = 65,536. We also tested that all combinations return to their original values after encoding followed by decoding.

#[test]
fn round_trips_all_valid_headers() {
    for version in 0..=0b111 {
        for compressed in [false, true] {
            for message_type in 0..=0b1111 {
                for payload_len in 0..=0xff {
                    let header = Header {
                        version,
                        compressed,
                        message_type,
                        payload_len,
                    };

                    assert_eq!(Header::decode(header.encode().unwrap()), header);
                }
            }
        }
    }
}

All tests passed in the local environment.

running 6 tests
test result: ok. 6 passed; 0 failed; 0 ignored

Discussion

Why Didn't repr Produce 2 Bytes

Rust's repr is a feature that specifies how a struct is laid out in memory. What repr(packed) changes is the alignment of each field and the resulting padding between fields; it does not change the size of u8 or bool themselves. Changing the bit width would also change the value range and how values of the same type are handled, so it seems likely that repr is designed to change only the memory layout while preserving the type.

Could What Was Done in C++ Be Done in Rust?

The goal of packing multiple values into a specification-compliant 2 bytes can be said to have been achieved in Rust as well. However, since Rust does not have built-in bit fields equivalent to those in C++ that could be used this time, the communication format is explicitly constructed using shifts and masks. Rather than providing the same mechanism, decoupling the communication format from the implementation's memory layout may be more in line with Rust's design, which prioritizes portability and safety.

Summary

A 2-byte packet header that was created using C++ bit fields could also be created in Rust. Unlike bit fields, repr(packed) could not reduce this struct to 2 bytes. In Rust, the same 2 bytes could be generated by explicitly specifying the communication format using shifts and masks. I hope this serves as a useful reference when migrating packet processing from C++ to Rust.