Send Half the Bytes Over the Network: Cut Kafka Integer Payloads with Bijou64

If you produce millions of Kafka messages where the value is just a number — counters, IDs, timestamps, sequence numbers — you’re almost certainly paying for 8 bytes per message whether the value is 1 or 1_000_000_000.

Kafka’s built-in LongSerializer is simple and correct. But simple also means fixed-width: every long becomes 8 bytes on the wire and on disk. On high-volume, integer-heavy topics, that adds up fast in broker storage, replication traffic, and egress costs.

That’s the problem Bijou64 solves. It’s a compact, variable-length encoding for unsigned 64-bit integers from Ink & Switch. I wrapped it as a Java library with Kafka serializers so you can swap it in without changing your application logic.

GitHub repo: github.com/Joel-hanson/bijou64

What Is Bijou64?

bijou64 (pronounced “bee-zoo-sixty-four”) stands for BIJective Offset U64. It’s a way to encode integers into 1–9 bytes instead of always using 8.

The idea is straightforward:

Small numbers stay small. Big numbers grow only as much as they need to. And unlike some varint formats, bijou64 is canonically unique — each number has exactly one valid encoding, so you don’t get ambiguous “overlong” forms.

If you want the deep dive on the wire format, Ink & Switch’s bijou64 write-up and the upstream spec are excellent.

Why Should Kafka Users Care?

Think about topics like:

• Event counters and metrics
• Monotonic sequence numbers (1, 2, 3, …)
• Numeric IDs that start small and grow over time
• Timestamp-like values where most records cluster in a range

For these workloads, most values are small. Bijou64 encodes them in 1–3 bytes instead of 8.

What you gain

• Smaller payloads → less disk on brokers, less data to replicate
• Less network I/O → especially on producer-heavy pipelines
• Potential throughput bump → less bytes to serialize, compress, and ship

What you don’t get

Bijou64 is not a replacement for Avro, Protobuf, or JSON. It’s not for arbitrary structured objects. If your values are complex records, stick with schema-based serialization (and enable Kafka compression first).

When to Use It (and When Not To)

Use Bijou64 when:

• Your Kafka values are mostly numeric (Long / Integer)
• The topic is high volume and storage or egress costs matter
• You control both producer and consumer serialization

Skip Bijou64 when:

• Payloads are JSON, Avro, Protobuf, or other structured formats
• Producers and consumers can’t share the same serializer/deserializer pair
• Topics are low volume and the savings aren’t worth the migration

Rule of thumb: if you’d reach for LongSerializer today and your numbers are often small, Bijou64 is worth a look.

Know the Limits (Quick Gist)

Bijou64 is narrow on purpose — that’s what makes it fast and simple:

• Range: unsigned 0 → 2⁶⁴ − 1 (Java long is treated as unsigned on the wire)
• Size: 1–9 bytes per value — small ints save the most; very large values can use 9 bytes (slightly more than Long’s fixed 8)
• Signed negatives: if you rely on negative Long values, stick with LongSerializer — Bijou64 does not preserve signed semantics the same way
• Kafka contract: producer and consumer must both use Bijou64Serializer / Bijou64Deserializer on the same field
• Payload shape: one integer per message — not JSON, Avro, Protobuf, or nested objects
• Migration: existing topics written with LongSerializer need a coordinated rollout or a new topic

If your topic is counters, sequence numbers, or IDs that stay non-negative and mostly small, you’re in the sweet spot for cutting network transfer.

How It Works in Java

The library exposes a simple encode/decode API:

// Encode 300 → [0xF8, 0x34]
byte[] encoded = Bijou64.encode(300);

// Decode back to 300
long value = Bijou64.decode(encoded);

Under the hood there are two implementations:

• Native (JNI) — wraps the upstream Rust crate for maximum speed
• Pure Java fallback — used automatically if the native library isn’t available

Force pure Java if you need it:

bijou64.useJava=true

Drop-In Kafka Integration

This is the part that matters for day-to-day use. Swap your serializers — no schema registry changes, no payload restructuring.

Producer:

key.serializer=org.bijou64.perf.kafka.Bijou64Serializer
value.serializer=org.bijou64.perf.kafka.Bijou64Serializer

Consumer:

key.deserializer=org.bijou64.perf.kafka.Bijou64Deserializer
value.deserializer=org.bijou64.perf.kafka.Bijou64Deserializer

Maven:

<dependency>
  <groupId>org.bijou64</groupId>
  <artifactId>bijou64</artifactId>
  <version>0.2.0</version>
</dependency>

<dependency>
  <groupId>org.bijou64</groupId>
  <artifactId>bijou64-kafka-serializers</artifactId>
  <version>0.2.0</version>
</dependency>

Build from source (includes the Rust native library via submodule):

git clone https://github.com/Joel-hanson/bijou64.git
cd bijou64
git submodule update --init --recursive
mvn -B clean package

Does It Actually Help? Benchmark Numbers

I run an end-to-end Kafka producer benchmark in CI on every push to main: 50,000 sequential integers (1..N), Kafka 4.3 (KRaft), Java 17, comparing LongSerializer vs Bijou64 (JNI and pure Java), with and without zstd compression.

Results from the 2026-06-06 CI run:

Takeaways:

• 62% smaller payloads on sequential integers (3 vs 8 bytes)
• ~11–14% higher producer throughput vs LongSerializer without compression
• Pure Java is competitive with JNI on the CI runner — you won’t necessarily need native libs everywhere

These are full producer runs (client + broker + network), not isolated microbenchmarks. Run the same test locally for your hardware:

cd perf/kafka
docker compose up -d
./scripts/compare-benchmarks.sh --ci --distribution sequential 50000 2 \
  bijou64-benchmark-topic localhost:9092

For encode/decode-only numbers:

./scripts/run-jmh.sh

A Mental Model

Standard LongSerializer          Bijou64
─────────────────────────        ─────────────────────────
value: 42                        value: 42
wire:  [8 fixed bytes]           wire:  [1 byte]

value: 300                       value: 300
wire:  [8 fixed bytes]           wire:  [2 bytes]

value: 1_000_000                 value: 1_000_000
wire:  [8 fixed bytes]           wire:  [3 bytes]

You’re not changing what you send — just how many bytes it takes to represent the same number.

Try It in 5 Minutes

Artifacts are published to GitHub Packages on every release tag (v0.2.0, etc.). You can pull them into an existing Kafka app without cloning the repo.

1. Authenticate with GitHub Packages

GitHub Packages requires a personal access token, even for public repos.

1. Create a classic PAT with the read:packages scope (or a fine-grained token with Packages: Read on Joel-hanson/bijou64).

2. Add credentials to ~/.m2/settings.xml:

   <settings xmlns="http://maven.apache.org/SETTINGS/1.2.0"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.2.0 https://maven.apache.org/xsd/settings-1.2.0.xsd">
     <servers>
       <server>
         <id>github</id>
         <username>YOUR_GITHUB_USERNAME</username>
         <password>YOUR_GITHUB_PAT</password>
       </server>
     </servers>
   </settings>
   

   The <id>github</id> must match the repository <id> in your pom.xml below.

2. Add the dependency to your Maven project

Point Maven at the package registry and pull in the Kafka serializers (which transitively includes the core library):

<repositories>
  <repository>
    <id>github</id>
    <url>https://maven.pkg.github.com/Joel-hanson/bijou64</url>
  </repository>
</repositories>

<dependencies>
  <dependency>
    <groupId>org.bijou64</groupId>
    <artifactId>bijou64-kafka-serializers</artifactId>
    <version>0.2.0</version>
  </dependency>
</dependencies>

If you only need encode/decode outside Kafka, depend on bijou64 instead of bijou64-kafka-serializers.

Gradle (Kotlin DSL):

repositories {
    maven {
        url = uri("https://maven.pkg.github.com/Joel-hanson/bijou64")
        credentials {
            username = providers.gradleProperty("gpr.user").get()
            password = providers.gradleProperty("gpr.key").get()
        }
    }
}

dependencies {
    implementation("org.bijou64:bijou64-kafka-serializers:0.2.0")
}

Set gpr.user and gpr.key in ~/.gradle/gradle.properties to your GitHub username and PAT.

3. Wire up your Kafka producer and consumer

Swap the serializers in your existing app — no changes to your Long business logic.

producer.properties:

bootstrap.servers=localhost:9092
key.serializer=org.apache.kafka.common.serialization.StringSerializer
value.serializer=org.bijou64.perf.kafka.Bijou64Serializer

consumer.properties:

bootstrap.servers=localhost:9092
group.id=my-app
key.deserializer=org.apache.kafka.common.serialization.StringDeserializer
value.deserializer=org.bijou64.perf.kafka.Bijou64Deserializer
auto.offset.reset=earliest

Or in Java:

Properties props = new Properties();
props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, Bijou64Serializer.class.getName());

try (KafkaProducer<String, Long> producer = new KafkaProducer<>(props)) {
    producer.send(new ProducerRecord<>("counters", "page-views", 42L));
}

Both producer and consumer must use Bijou64 serializers on the same topic. Mixing LongSerializer on one side and Bijou64Serializer on the other will corrupt your data.

4. Run it

mvn clean package
# start your app, or use kafka-console-producer/consumer with the properties above

The library falls back to pure Java automatically if the native JNI library isn’t on your classpath — no extra setup required for most deployments.

Want to run the benchmarks locally?

Clone the repo, start Kafka with Docker, and compare LongSerializer vs Bijou64:

git clone https://github.com/Joel-hanson/bijou64.git
cd bijou64
git submodule update --init --recursive
mvn -B clean package

cd perf/kafka
docker compose up -d
./scripts/compare-benchmarks.sh sequential 10000 2 \
  my-benchmark-topic localhost:9092

Credits

This project implements the bijou64 wire format from Bijou by Ink & Switch. The native path uses their Rust crate via a git submodule and JNI. My repo adds the Java/Kafka layer, benchmarks, and CI.

Final Thoughts

Kafka compression (lz4, zstd, gzip) is the first knob most teams turn. That’s the right default for general payloads. But when your topic is integer-heavy, you’re compressing 8-byte values that didn’t need to be 8 bytes in the first place.

Bijou64 attacks the problem at the serialization layer: same semantics, fewer bytes, drop-in Kafka integration. If your pipeline looks like counters, IDs, or sequential numbers at scale, it’s worth benchmarking on your own cluster.

Star the repo if it’s useful → github.com/Joel-hanson/bijou64

For more Kafka tips and open-source tools, follow the blog series