A cloud native streaming platform,
and it's Kafka compatible.
Manifester is a Kafka-compatible streaming engine that keeps every byte in object storage, so you stop paying the cloud tax. No broker limits, no cross-AZ traffic, no disks to babysit.
The 3-AZ tax, in numbers
Textbook Kafka at 100 MB/s of ingest: replicated 3 times, one replica per zone, one consumer group. Traditional Kafka on the left, Manifester on the right.
Clients talk only to brokers in their own zone, and brokers write straight to object storage.
No broker-to-broker replication. S3 already copies your data across zones, and EC2-to-S3 in-region is free.
181 TB of replicated EBS gp3 vs. 60 TB stored once in S3 Standard.
Object Store Backed Kafka
Everything durable lives in object storage. Everything else is stateless and rebuildable.
S3 is the log
Not a tier behind the log. A produce is acked only once it's durable in object storage and its offsets are committed. No ISR, no under-replicated partitions.
Stateless brokers
No broker owns data, so any broker serves any partition. Scale out: start a broker. Scale in: stop one. Crash: clients reconnect. Stateless brokers love spot.
Zone-aligned routing
Clients put their AZ in their client ID, and every metadata response points them at a broker in that zone. The cross-AZ data path disappears.
Cheap, deep retention
Retention is priced like object storage, not like replicated disks. Keep days or weeks of history without buying brokers just to hold it.
Built for throughput, not microseconds
Manifester is not a low-latency Kafka. Writes land in S3 before they're acked, so end-to-end latency is in the hundreds of milliseconds. For a lot of workloads, that's a great trade for a much smaller bill.
A great fit
- Log and event pipelines
- Change data capture (CDC)
- Analytics ingestion
- Observability
- ML feature pipelines
Not yet
- Single-digit-ms latency (S3 Express One Zone quorum is planned)
- Exactly-once producer transactions
- Compacted topics
Why manifester
Object native
Durability comes from the bucket, not from copying bytes between brokers. Storage scales with retention; compute scales with traffic.
Kafka wire protocol
Your existing Kafka clients, connectors and tooling just work. Written in Go, built to keep hundreds of S3 requests in flight.
Open source
Apache 2.0. Run it in your own account, your own VPC, or an air-gapped box. No license keys, no phone-home.
Self-host in minutes
Run Manifester in your own account, against your own bucket. Then point any Kafka client at it.
# run a manifester broker against your own S3 bucket
$ docker run -p 9092:9092 \
-e S3_BUCKET=my-topics ghcr.io/manifesterio/manifester
# use any Kafka client against it
$ kafka-console-producer.sh --bootstrap-server localhost:9092 --topic events