Design: a Kafka-style log
The core idea
A publishing service needs search and analytics to receive the same post changes.
Search may be current while analytics is an hour behind.
In this lesson
- What are we promising?
- Partition the order we need
- Follow one accepted publication
- Replace a failed leader safely
- Handle producer uncertainty
- Read, apply, then save progress
- Store batches, expire segments
- Size the recovery work
- Run the smaller recovery experiment
- Check a failure, not just a happy path
Continue reading
Pro unlocks video lessons, visual walkthroughs, full notes, diagrams, case studies, and implementations across the System Design archive.