protozoa/encode
Protocol Buffer Encode Module
This module provides functions for encoding Gleam values into Protocol Buffer binary format. It handles the conversion from typed Gleam data structures to the compact binary wire format used by Protocol Buffers for efficient storage and network transmission.
Design Philosophy
- Correctness: Produces valid Protocol Buffer binary data compatible with other implementations
- Efficiency: Optimized encoding with minimal memory allocations and copies
- Composability: Field-level encoders that can be combined for message encoding
- Type safety: Compile-time guarantees about the structure of encoded data
- Deterministic output: Consistent binary output for the same input data
Capabilities
- All proto3 types: Scalars (int32, int64, string, bool, bytes), messages, enums
- Advanced features: Repeated fields, maps, oneofs, nested messages
- Wire format compliance: Correct tag encoding, varint encoding, length-delimited data
- Field-level encoding: Individual field encoders for fine-grained control
- Message-level encoding: Complete message encoders with proper field ordering
- Size calculation: Efficient pre-calculation of encoded message sizes
Usage Pattern
Generated code uses this module to create message-specific encoder functions:
pub fn encode_user(user: User) -> BitArray {
encode.message([
encode.string_field(1, user.name),
encode.int32_field(2, user.age),
])
}
Wire Format
The module correctly implements the Protocol Buffer wire format:
- Varint encoding for integers and field tags
- Length-delimited encoding for strings, bytes, and messages
- Fixed-width encoding for fixed32/fixed64 types
- Proper field tag calculation with field numbers and wire types
Performance
Encoding is optimized for performance with:
- Pre-calculated message sizes to avoid buffer reallocations
- Efficient bit array operations for binary data construction
- Minimal intermediate allocations during encoding
Values
pub fn bool_field(field_number: Int, value: Bool) -> BitArrayEncodes a boolean field with tag and value. True is encoded as 1, False as 0.
pub fn bytes(field_number: Int, data: BitArray) -> BitArrayEncodes a bytes field with tag and length-delimited data.
pub fn double(value: Float) -> BitArrayEncodes a double as a 64-bit IEEE 754 value in little-endian format.
pub fn double_field(field_number: Int, value: Float) -> BitArrayEncodes a double field with tag and value.
pub fn field(
field_number: Int,
wire_type: wire.WireType,
value_encoder: BitArray,
) -> BitArrayEncodes a complete field with tag and value. Combines the tag (field number and wire type) with the encoded value.
pub fn fixed32(value: Int) -> BitArrayEncodes an integer as a fixed 32-bit value in little-endian format. Used for fixed32 and sfixed32 fields.
pub fn fixed32_field(field_number: Int, value: Int) -> BitArrayEncodes a fixed32 field with tag and value.
pub fn fixed32s(field_number: Int, values: List(Int)) -> BitArrayEncodes a packed repeated fixed32 field. All values are encoded together and length-delimited.
pub fn fixed64(value: Int) -> BitArrayEncodes an integer as a fixed 64-bit value in little-endian format. Used for fixed64 and sfixed64 fields.
pub fn fixed64_field(field_number: Int, value: Int) -> BitArrayEncodes a fixed64 field with tag and value.
pub fn fixed64s(field_number: Int, values: List(Int)) -> BitArrayEncodes a packed repeated fixed64 field. All values are encoded together and length-delimited.
pub fn float(value: Float) -> BitArrayEncodes a float as a 32-bit IEEE 754 value in little-endian format.
pub fn float_field(field_number: Int, value: Float) -> BitArrayEncodes a float field with tag and value.
pub fn int32_field(field_number: Int, value: Int) -> BitArrayEncodes an int32 field with tag and value.
pub fn int64_field(field_number: Int, value: Int) -> BitArrayEncodes an int64 field with tag and value.
pub fn length_delimited(data: BitArray) -> BitArrayEncodes data with a length prefix. The length is encoded as a varint followed by the data. Used for strings, bytes, and nested messages.
pub fn message(fields: List(BitArray)) -> BitArrayCombines multiple encoded fields into a complete Protocol Buffer message.
pub fn message_field(
field_number: Int,
message: BitArray,
) -> BitArrayEncodes a nested message field with tag and length-delimited message data.
pub fn repeated_bool_field(
field_number: Int,
values: List(Bool),
) -> BitArrayEncodes a packed repeated boolean field. All values are encoded together and length-delimited.
pub fn repeated_double_field(
field_number: Int,
values: List(Float),
) -> BitArrayEncodes a packed repeated double field. All values are encoded together and length-delimited.
pub fn repeated_float_field(
field_number: Int,
values: List(Float),
) -> BitArrayEncodes a packed repeated float field. All values are encoded together and length-delimited.
pub fn repeated_int32_field(
field_number: Int,
values: List(Int),
) -> BitArrayEncodes a packed repeated int32 field. All values are encoded together and length-delimited.
pub fn repeated_int64_field(
field_number: Int,
values: List(Int),
) -> BitArrayEncodes a packed repeated int64 field. All values are encoded together and length-delimited.
pub fn repeated_sint32_field(
field_number: Int,
values: List(Int),
) -> BitArrayEncodes a packed repeated sint32 field with zigzag encoding. All values are encoded together and length-delimited.
pub fn repeated_sint64_field(
field_number: Int,
values: List(Int),
) -> BitArrayEncodes a packed repeated sint64 field with zigzag encoding. All values are encoded together and length-delimited.
pub fn sfixed32_field(field_number: Int, value: Int) -> BitArrayEncodes an sfixed32 field (signed fixed32) with tag and value.
pub fn sfixed64_field(field_number: Int, value: Int) -> BitArrayEncodes an sfixed64 field (signed fixed64) with tag and value.
pub fn sint32_field(field_number: Int, value: Int) -> BitArrayEncodes a sint32 field (signed int32 with zigzag encoding).
pub fn sint64_field(field_number: Int, value: Int) -> BitArrayEncodes a sint64 field (signed int64 with zigzag encoding).
pub fn string(value: String) -> BitArrayEncodes a string as length-delimited UTF-8 bytes.
pub fn string_field(field_number: Int, value: String) -> BitArrayEncodes a string field with tag and value.
pub fn tag(
field_number: Int,
wire_type: wire.WireType,
) -> BitArrayCreates a Protocol Buffer tag from a field number and wire type. The tag is encoded as a varint.
pub fn uint32_field(field_number: Int, value: Int) -> BitArrayEncodes a uint32 field with tag and value.
pub fn uint64_field(field_number: Int, value: Int) -> BitArrayEncodes a uint64 field with tag and value.