TensorFlow: A System for Large-Scale Machine Learning

TensorFlow made one machine-learning program portable across heterogeneous hardware by representing primitive computation, mutable state, and update rules in one deferred dataflow graph. Follow how the runtime prunes, places, partitions, and connects that graph across CPU, GPU, TPU, and machine boundaries.

Reading focus: Why putting mutable Variables and their update operations inside the graph was more flexible than a fixed parameter-server interface. How feeds and fetches select a subgraph, placement assigns operations to devices, and cross-device edges become Send and Recv operations. What the 2016 performance evidence establishes, and why static graphs, weak checkpoint consistency, and incomplete placement policy bound the result.

OSDI 2016. Martín Abadi et al.. 13 min read, medium difficulty.