Single-node · In-memory · gRPC

Heads up: I ended up switching to NATS + PostgreSQL. If you need consumer groups, bare NATS won't help — you need JetStream, which requires persistence. tinybroker fills the gap when you want in-memory consumer groups with no persistence overhead. The code works and is free to use; just know why you're reaching for it.

Consumer Groups (balanced)

Multiple subscribers share the same subscription_id and consumer_group. The broker delivers each message to exactly one member of the group, cycling through active members round-robin. This turns tinybroker into a work-queue dispatcher.

Consumer groups require DELIVERY_MODE_BALANCED and a non-empty consumer_group.

When to use

  • Distribute background jobs across worker replicas so each job runs once.
  • Shard inbound webhook events across processors without a separate queue.
  • Rate-limit expensive downstream calls by controlling replica count.

Example

Three workers all join group "image-resizer". Fifty thumbnail jobs arrive. Each job goes to exactly one worker; no job is processed twice.

Publishers         tinybroker          Worker A
    │                  │                   │
    │── Publish ───────►                   │
    │    "jobs.thumb.1"                    │
    │                  │── Message ────────►
    │── Publish ───────►    "jobs.thumb.1" │
    │    "jobs.thumb.2"  
    │                  │               Worker B
    │                  │── Message ────────►
    │                       "jobs.thumb.2"

Protocol

All workers open with the same subscription_id and consumer_group:

OpenSubscription {
  subscription_id: "image-resizer-group"
  topic_patterns:  ["jobs.thumb.*"]
  mode:            DELIVERY_MODE_BALANCED
  consumer_group:  "image-resizer"
}

After processing, each worker calls Ack with the system_id to unblock delivery of the next message:

Ack {
  subscription_id: "image-resizer-group"
  system_id:       "<system_id from the delivered Message>"
}

Scaling

Add or remove workers at any time. The broker detects disconnections and redistributes in-flight unacked messages to remaining members. There is no rebalance delay — members that are present receive messages immediately.