What is fan-out? Fan-in, feeds and tail latency
Fan-out is one request, event or message turning into many downstream operations: a search query sent to every shard of an index, an order event delivered to every service that subscribes to it, a post copied into each follower's feed. Fan-in is the reverse step, where the many results are gathered back into one answer.
A wider fan-out spreads work across more machines, and it also makes each request depend on more of them, so the slowest machine in the group starts deciding how fast the whole request is. The sections below cover request fan-out, fan-in, message fan-out and feeds, then the other things the word means.
Request fan-out and tail latency
A service that splits its data across many servers has to ask all of them. A search query against an index sharded over 100 machines becomes 100 parallel calls, and the answer is ready only when the last of them returns.
Suppose the calls are independent and each is slow 1% of the time, that is, slower than its 99th-percentile latency. The chance that a request with N parallel calls waits on at least one slow call is 1 − 0.99ᴺ, one minus 0.99 multiplied by itself N times:
- 1 call: 1%
- 10 calls: 9.6%
- 100 calls: 63.4%
At 100 calls most requests wait on a straggler, so the per-call p99 has become the request's typical latency. Jeffrey Dean and Luiz André Barroso worked through this arithmetic in The Tail at Scale (Communications of the ACM, 2013), where one slow call in 10,000 still makes almost one request in five slow once a request touches 2,000 servers.
A hedged request sends a second copy of a call that is running late, for example past its 95th-percentile latency, and takes whichever answer arrives first. A tied request queues the call on two servers at once, and the server that starts it first tells the other to drop its copy. Where the design allows it, fewer and larger calls shrink the fan.
To keep requests as reliable as single calls, each call has to get much better. With 100 calls, holding slow requests to 1% means each call can be slow only about 0.01% of the time: the per-call p99.99 has to meet the latency the request promises. The calculator after this article runs both numbers for other widths.
Fan-in: gathering the results
Fan-in is the step that collects what the fan-out produced: merging search hits by score, adding up partial counts, or waiting for every map task before a reduce task starts. It decides two things the fan-out leaves open: how long to wait, and what to do with a partial answer.
A fan-in with a deadline returns whatever has arrived when time runs out, trading completeness for latency. The Tail at Scale calls these good-enough results: in a large search system, the chance that one particular leaf server holds the best result is small, so answering without the last few servers rarely changes what the user sees. A fan-in that waits for every branch is exact, and it inherits the slowest one.
In messaging, fan-in also names the opposite shape: many producers writing into one queue or topic that a smaller set of consumers drains.
Message fan-out: one event, many consumers
In publish-subscribe messaging, a producer publishes an event once and the broker hands a copy to every subscriber. AWS documents this as fanout: one Amazon SNS topic delivering each message to several Amazon SQS queues, each read by a different service (AWS documentation). An order event can reach billing, email and analytics without the producer knowing any of them.
Apache Kafka gets the same effect another way. Each consumer group reads the whole topic at its own pace, so adding a consumer group adds a reader without copying anything at publish time.
The producer's cost stays at one write. The broker does the multiplying, and each subscriber now owns its own backlog. With standard SNS/SQS or at-least-once Kafka processing, consumers must handle duplicates; delivery guarantees depend on configuration.
Fan-out on write vs fan-out on read
Feeds and inboxes face the same choice inside the database. Fan-out on write, also called push, copies a new post into each follower's feed when it is published, so opening a feed is one cheap read. Fan-out on read, or pull, stores the post once and builds each feed when someone opens it, by fetching recent posts from every account they follow and merging them.
- Fan-out on write does the work at publish time, one write per follower. It struggles when one account has millions of followers.
- Fan-out on read does the work at read time, one fetch per followed account plus a merge. It struggles when people follow many accounts or open their feed often.
Large services mix the two. Twitter's 2013 talk on its timeline service, summarised by High Scalability, described fanning tweets out on write into in-memory timelines, and doing more of the work at read time for accounts with very large followings.
Fanout's System Design course goes further in its Fanout Patterns lesson (Fanout Pro), which covers queueing, delivery guarantees and hybrid designs.
Other meanings of fan-out
In digital electronics, fan-out is the number of gate inputs that one gate's output drives. The maximum fan-out is set by how much current the output can source and sink while keeping valid logic levels: roughly 2 to 10 inputs for TTL gates, and far more for CMOS (Wikipedia).
In AI search, query fan-out is a model splitting one question into many related searches and combining the results. Google says AI Mode and AI Overviews use it (Google Search Central).
In everyday English, to fan out is to spread out from one point, the way a search party fans out across a field.
Where to go next
Fan-out turns per-call latency and timeouts into request-level problems. Timeout Architect checks whether a retry plan fits a request deadline, and Latency Numbers lists the latencies each extra call adds.