Single-node · In-memory · gRPC

Heads up: I ended up switching to NATS + PostgreSQL. If you need consumer groups, bare NATS won't help — you need JetStream, which requires persistence. tinybroker fills the gap when you want in-memory consumer groups with no persistence overhead. The code works and is free to use; just know why you're reaching for it.

vs Apache Kafka

Apache Kafka is the de-facto standard for high-throughput, durable, ordered event streaming at scale. It excels at replay, long-term retention, and audit trails.

Use Kafka when

  • You need messages to survive indefinitely or be replayed from any point in history.
  • You need strict ordering within a partition across millions of events.
  • You are building a data pipeline that feeds analytics, data warehouses, or event sourcing systems.
  • You need guaranteed at-least-once or exactly-once semantics with durable state.
  • You run a large organisation where multiple teams consume the same event streams independently.

Use tinybroker when

  • You need dynamic topic names that are created by publishers at runtime, without prior configuration. Kafka topics must be created explicitly (or auto-creation must be enabled, which makes management harder). tinybroker topics are implicit — a topic exists as soon as someone publishes to it.
  • You need wildcard subscriptions. Kafka has no native support for topic name pattern matching — a consumer specifies exact topic names or uses a regex that must be evaluated against the full topic list at connection time, not dynamically. tinybroker subscribers specify glob patterns that are matched per-message as topics appear.
  • Your fleet is small (single node or a handful of replicas) and Kafka’s minimum viable cluster (3 brokers + ZooKeeper/KRaft) is disproportionate.
  • Messages can be lost on restart and you want to avoid the operational cost of brokers, storage management, partition leadership, and consumer group offset tracking.

The core trade-off

Kafka is a persistent ordered log with a fixed schema of named, configured topics. tinybroker is a dynamic, schema-free pub/sub bus. They solve different problems — Kafka’s strength (durability, ordering, retention) is precisely what tinybroker omits to stay small.