--- title: "Technical field notes from Fanout" description: "Practical writing on AI research, system design, inference engineering, and machine learning mathematics from the work behind Fanout." canonical_url: "https://fanout.sh/blog" md_url: "https://fanout.sh/blog.md" access: "public" --- # Technical field notes from Fanout Practical writing on AI research, system design, inference engineering, and machine learning mathematics from the work behind Fanout. ## Public overview Fanout field notes connect AI research, system design, inference engineering, and machine learning mathematics to practical engineering decisions. Each note links to its primary sources and continues into a related Fanout course, roadmap, or interactive lab. ## Published field notes ### [Token choice vs expert choice routing in MoE](https://fanout.sh/blog/token-choice-vs-expert-choice-routing.md) How token choice and expert choice routing split MoE work: who picks whom, load balance, dropped tokens, and why decoder LLMs route by token. - Author: Suraj Gaud - Published: 2026-08-19 - Track: AI research ### [What is a good cross entropy loss value](https://fanout.sh/blog/what-is-a-good-cross-entropy-loss-value.md) Judge cross entropy loss against the ln C uniform baseline and the data's entropy floor, with worked numbers from coin flips to GPT-2. - Author: Suraj Gaud - Published: 2026-08-19 - Track: ML mathematics ### [AI inference engineering, explained with numbers](https://fanout.sh/blog/ai-inference-engineering-guide.md) What AI inference engineering is, derived from one GPU ratio: bandwidth ceilings, batching, caching, quantization, and when self-hosting pays. - Author: Suraj Gaud - Published: 2026-08-18 - Track: Inference engineering ### [FlashAttention vs PagedAttention: what each fixes](https://fanout.sh/blog/flashattention-vs-pagedattention.md) FlashAttention cuts attention kernel memory traffic. PagedAttention cuts KV cache waste. Which one is your bottleneck, and why engines run both. - Author: Suraj Gaud - Published: 2026-08-18 - Track: Inference engineering ### [W8A8 vs W4A16 quantization: how to choose](https://fanout.sh/blog/w8a8-vs-w4a16-quantization.md) Choose W8A8 or W4A16 quantization from prefill compute, decode bandwidth, and memory-fit arithmetic, with the batch-size crossover that flips the answer. - Author: Suraj Gaud - Published: 2026-08-18 - Track: Inference engineering ### [Expert parallelism vs tensor parallelism for MoE](https://fanout.sh/blog/expert-parallelism-vs-tensor-parallelism.md) Choose expert or tensor parallelism for MoE serving from weight placement, all-to-all versus all-reduce arithmetic, expert size, and batch occupancy. - Author: Suraj Gaud - Published: 2026-08-17 - Track: Inference engineering ### [When KV cache quantization slows inference](https://fanout.sh/blog/when-kv-cache-quantization-slows-inference.md) When FP8 or INT4 KV cache quantization slows LLM inference: unsupported attention backends, short contexts, low concurrency, and conversion kernels. - Author: Suraj Gaud - Published: 2026-08-17 - Track: Inference engineering ### [Disaggregated prefill and decode with numbers](https://fanout.sh/blog/disaggregated-prefill-and-decode.md) Calculate KV transfer bytes and time, size prefill and decode pools independently, and decide when disaggregation beats chunked prefill. - Author: Suraj Gaud - Published: 2026-08-16 - Track: Inference engineering ### [Write-ahead log explained through one crash](https://fanout.sh/blog/write-ahead-log-explained.md) Follow one transfer through WAL records, dirty pages, commit, crash recovery, redo, undo, checkpoints, and group commit. - Author: Suraj Gaud - Published: 2026-08-16 - Track: System design ### [Consistent hashing explained simply with 10 keys](https://fanout.sh/blog/consistent-hashing-explained-simply.md) Trace 10 keys through modulo hashing and a hash ring, count exactly what moves, then add virtual nodes, replication, and failure limits. - Author: Suraj Gaud - Published: 2026-08-15 - Track: System design ### [Softmax temperature explained with numbers](https://fanout.sh/blog/softmax-temperature-explained.md) See softmax at T=0.5, 1, and 2, then separate temperature's roles in sampling, calibration, training, and knowledge distillation. - Author: Suraj Gaud - Published: 2026-08-15 - Track: ML mathematics ### [FP8 vs INT8 vs AWQ vs GPTQ](https://fanout.sh/blog/fp8-vs-int8-vs-awq-vs-gptq.md) FP8 and INT8 are number formats. AWQ and GPTQ are weight quantization methods. Compare memory, kernels, quality, and the workloads each fits. - Author: Suraj Gaud - Published: 2026-08-14 - Track: Inference engineering ### [Mixture of experts routing explained](https://fanout.sh/blog/mixture-of-experts-routing-explained.md) Follow one token through MoE router logits, top-k softmax, expert outputs, and capacity limits. See why routing balance is a systems constraint. - Author: Suraj Gaud - Published: 2026-08-14 - Track: AI research ### [KV cache quantization: memory savings by bits](https://fanout.sh/blog/kv-cache-quantization.md) Calculate how much KV cache quantization saves after scale metadata and residual tokens, then decide when lower precision is worth the tradeoff. - Author: Suraj Gaud - Published: 2026-08-13 - Track: Inference engineering ### [Why is FlashAttention faster?](https://fanout.sh/blog/why-is-flash-attention-faster.md) FlashAttention is faster because it avoids writing full attention matrices to HBM. Follow the 4K-token memory traffic and exact softmax method. - Author: Suraj Gaud - Published: 2026-08-13 - Track: Inference engineering ### [How Grok 4.6 happened: Cursor's data flywheel](https://fanout.sh/blog/grok-4-6-cursor-data-flywheel.md) The six-month story behind Grok 4.6: the $60B Cursor deal, trillions of trace tokens, and a post-training recipe where Grok 4.5 built its successor. - Author: Suraj Gaud - Published: 2026-08-12 - Track: AI research ### [Tensor parallelism vs pipeline parallelism](https://fanout.sh/blog/tensor-parallelism-vs-pipeline-parallelism.md) Choose tensor or pipeline parallelism for LLM inference using memory fit, interconnect topology, latency goals, and worked communication arithmetic. - Author: Suraj Gaud - Published: 2026-08-12 - Track: Inference engineering ### [What is chunked prefill?](https://fanout.sh/blog/what-is-chunked-prefill.md) Chunked prefill splits long prompts across scheduler iterations. See the token-budget arithmetic and the TTFT, ITL, throughput, and memory tradeoffs. - Author: Suraj Gaud - Published: 2026-08-12 - Track: Inference engineering ### [Argmax vs max: choice and value explained](https://fanout.sh/blog/argmax-vs-max.md) Learn why max returns the best value while argmax returns the choice that produced it, including ties, types, axes, and an interactive sandbox. - Author: Suraj Gaud - Published: 2026-08-09 - Track: ML mathematics ### [How to read equations in AI research papers](https://fanout.sh/blog/how-to-read-equations-ai-research-papers.md) A practical method for reading notation, scope, types, and meaning in AI research-paper equations, with an interactive reinforcement-learning example. - Author: Suraj Gaud - Published: 2026-08-09 - Track: ML mathematics ### [KV-cache memory formula for LLM inference](https://fanout.sh/blog/kv-cache-memory-formula-llm-inference.md) Derive KV-cache bytes from tensor shapes, compare MHA, GQA, and MQA, test scaling laws, and separate logical payload from reserved GPU memory. - Author: Suraj Gaud - Published: 2026-08-09 - Track: Inference engineering ### [Softmax and cross-entropy from logits](https://fanout.sh/blog/softmax-cross-entropy-from-logits.md) Follow logits through stable softmax and categorical cross-entropy, with exact invariants, edge cases, API assumptions, and an overflow demonstration. - Author: Suraj Gaud - Published: 2026-08-09 - Track: ML mathematics ### [The Roofline model for AI inference](https://fanout.sh/blog/roofline-model-ai-inference.md) Use operational intensity, bandwidth, and peak compute to reason about inference bottlenecks without mistaking a bound for a latency prediction. - Author: Suraj Gaud - Published: 2026-08-09 - Track: Inference engineering ### [The scaled dot-product attention equation](https://fanout.sh/blog/scaled-dot-product-attention-equation.md) Decode query-key scores, square-root scaling, softmax, value mixing, tensor shapes, and the assumptions behind the Transformer attention equation. - Author: Suraj Gaud - Published: 2026-08-09 - Track: ML mathematics ### [What the vertical bar means in mathematics](https://fanout.sh/blog/what-does-vertical-bar-mean-in-mathematics.md) Resolve conditioning, magnitude, cardinality, determinants, set-builder notation, divisibility, and evaluation by reading the mathematical frame. - Author: Suraj Gaud - Published: 2026-08-09 - Track: ML mathematics ### [A robotics roadmap for software engineers](https://fanout.sh/blog/robotics-roadmap-software-engineers.md) A practical path from software engineering to robotics, covering math, ROS 2, simulation, controls, perception, planning, and real hardware. - Author: Suraj Gaud - Published: 2026-08-06 - Track: System design ### [How to estimate LLM API costs](https://fanout.sh/blog/llm-api-cost-estimation.md) A workload-first method for estimating LLM API spend across input, output, caching, tools, retries, batch jobs, traffic growth, and quality. - Author: Suraj Gaud - Published: 2026-08-06 - Track: Inference engineering ### [How to use AI to study for exams](https://fanout.sh/blog/how-to-use-ai-to-study-for-exams.md) Build an exam plan with AI that uses active recall, spaced practice, mixed problems, error logs, and timed tests without outsourcing answers. - Author: Suraj Gaud - Published: 2026-08-06 - Track: AI research ### [How to use AI to study without losing the work](https://fanout.sh/blog/how-to-use-ai-to-study.md) Use AI as a tutor, question writer, and feedback partner while keeping retrieval, explanation, practice, and judgment in your own hands. - Author: Suraj Gaud - Published: 2026-08-06 - Track: AI research ### [LLM inference interview questions that matter](https://fanout.sh/blog/llm-inference-interview-questions.md) Twenty practical LLM inference interview questions with answer rubrics covering latency, KV cache, batching, GPUs, parallelism, and reliability. - Author: Suraj Gaud - Published: 2026-08-06 - Track: Inference engineering ### [OpenAI vs Anthropic API pricing](https://fanout.sh/blog/openai-vs-anthropic-api-pricing.md) A dated comparison of OpenAI and Anthropic API token prices, cache costs, batch discounts, long-context rules, and workload-level cost. - Author: Suraj Gaud - Published: 2026-08-06 - Track: Inference engineering ### [Speculative decoding for faster LLM inference](https://fanout.sh/blog/speculative-decoding-llm-inference.md) How speculative decoding proposes and verifies tokens, why it can preserve model outputs, and when draft models make latency worse. - Author: Suraj Gaud - Published: 2026-08-06 - Track: Inference engineering ### [The LLM inference engineer roadmap](https://fanout.sh/blog/llm-inference-engineer-roadmap.md) A practical roadmap through GPU fundamentals, transformer decoding, KV cache memory, batching, parallelism, serving, and production measurement. - Author: Suraj Gaud - Published: 2026-08-06 - Track: Inference engineering ### [The open-source robotics stack, explained](https://fanout.sh/blog/open-source-robotics-stack.md) How ROS 2, Gazebo, MuJoCo, MoveIt, ros2_control, LeRobot, and open hardware fit together in a modern robotics project. - Author: Suraj Gaud - Published: 2026-08-06 - Track: System design ### [Vision-language-action models for robotics](https://fanout.sh/blog/vision-language-action-models-robotics.md) How VLA models turn images and instructions into robot actions, what OpenVLA changed, and where evaluation and deployment still break. - Author: Suraj Gaud - Published: 2026-08-06 - Track: AI research ### [100 papers to understand software and computing](https://fanout.sh/blog/100-papers-to-understand-software-and-computing.md) A foundational reading list for getting started with programming, databases, networks, distributed systems, interfaces, and machine learning. - Author: Suraj Gaud - Published: 2026-08-05 - Track: System design ### [Build an AI homelab that works like one computer](https://fanout.sh/blog/ai-homelab-one-computer.md) Connect every Mac, home server, and cloud worker into one secure AI software factory, with the tools that matter and the order to add them. - Author: Medhansh - Published: 2026-08-05 - Track: System design ### [Continuous batching for LLM inference](https://fanout.sh/blog/continuous-batching-llm-inference.md) How iteration-level scheduling lets LLM requests join and leave a live batch, and how token budgets, KV memory, and preemption shape latency. - Author: Suraj Gaud - Published: 2026-08-03 - Track: Inference engineering ### [PagedAttention: how vLLM manages the KV cache](https://fanout.sh/blog/pagedattention-vllm-kv-cache.md) A systems guide to PagedAttention, KV blocks, block tables, copy-on-write, fragmentation, and what vLLM’s memory manager changes. - Author: Suraj Gaud - Published: 2026-08-03 - Track: Inference engineering ### [Prefill vs decode in LLM inference](https://fanout.sh/blog/prefill-vs-decode-llm-inference.md) A practical guide to prefill, decode, TTFT, inter-token latency, chunked prefill, and why one LLM request creates two serving workloads. - Author: Suraj Gaud - Published: 2026-08-03 - Track: Inference engineering ### [How AI memory works: five systems, not one](https://fanout.sh/blog/how-ai-memory-works-five-systems.md) A technical map of model weights, context windows, KV caches, recurrent state, retrieval, and the limits of what AI systems remember. - Author: Suraj Gaud - Published: 2026-07-29 - Track: AI research ### [Kimi K3 architecture: KDA, MLA and MoE](https://fanout.sh/blog/kimi-k3-architecture-kda-mla-moe.md) A source-led guide to Kimi K3’s hybrid attention, Attention Residuals, Stable LatentMoE, cache design, and deployment tradeoffs. - Author: Suraj Gaud - Published: 2026-07-29 - Track: Inference engineering ### [KV cache formula for LLM inference memory](https://fanout.sh/blog/kv-cache-formula-llm-inference-memory.md) Derive KV-cache memory from layers, KV heads, head dimension, context, batch, and precision, then connect the result to GPU capacity. - Author: Suraj Gaud - Published: 2026-07-29 - Track: Inference engineering ### [Daily paper summaries for AI engineers](https://fanout.sh/blog/daily-paper-summaries-for-ai-research-engineers.md) How AI research engineers can use daily paper summaries for discovery, verification, durable notes, experiments, and team decisions. - Author: Suraj Gaud - Published: 2026-07-25 - Track: AI research ### [Deep learning for computer vision engineers](https://fanout.sh/blog/deep-learning-course-for-computer-vision-engineers.md) How to choose and sequence computer vision learning across foundations, current architectures, experiments, evaluation, and deployment. - Author: Suraj Gaud - Published: 2026-07-25 - Track: AI research ### [Machine learning roadmap for backend engineers](https://fanout.sh/blog/machine-learning-roadmap-for-backend-engineers.md) A backend-first machine learning roadmap covering model behavior, data, evaluation, serving, observability, and inference systems. - Author: Suraj Gaud - Published: 2026-07-25 - Track: Inference engineering ### [ML interview prep for software engineers](https://fanout.sh/blog/how-to-prepare-for-ml-engineer-interviews-as-a-software-engineer.md) A role-specific ML engineer interview plan for software engineers covering coding, mathematics, modeling, systems, projects, and mocks. - Author: Suraj Gaud - Published: 2026-07-25 - Track: AI research ### [System design prep for senior engineers](https://fanout.sh/blog/system-design-prep-for-senior-engineer-interviews.md) Senior-level system design preparation focused on scope, trade-offs, evolution, failure recovery, operations, and clear technical leadership. - Author: Suraj Gaud - Published: 2026-07-25 - Track: System design ### [AI papers for system design engineers](https://fanout.sh/blog/ai-research-papers-for-system-design-engineers.md) A focused paper-reading path from distributed systems foundations to modern AI serving, with a practical protocol for each paper. - Author: Suraj Gaud - Published: 2026-07-24 - Track: System design ### [ML engineer roadmap for career switchers](https://fanout.sh/blog/machine-learning-engineer-roadmap-for-career-switchers.md) A career-switch roadmap that maps existing engineering strengths to ML foundations, model development, production systems, and credible projects. - Author: Suraj Gaud - Published: 2026-07-24 - Track: AI research ### [ML math roadmap without a math degree](https://fanout.sh/blog/ml-math-roadmap-for-engineers-without-a-math-degree.md) A dependency-based ML math roadmap for engineers covering notation, linear algebra, calculus, probability, optimization, and practice. - Author: Suraj Gaud - Published: 2026-07-24 - Track: ML mathematics ### [Paper-reading habit for staff engineers](https://fanout.sh/blog/research-paper-reading-habit-for-staff-engineers.md) A sustainable paper-reading system for staff engineers built around work questions, triage, durable notes, discussion, and application. - Author: Suraj Gaud - Published: 2026-07-24 - Track: AI research ### [System design labs for distributed systems](https://fanout.sh/blog/system-design-labs-for-distributed-systems-engineers.md) A lab framework for distributed systems engineers that tests invariants, failure recovery, observability, performance, and design judgment. - Author: Suraj Gaud - Published: 2026-07-24 - Track: System design ### [Deep learning roadmap for self-taught engineers](https://fanout.sh/blog/deep-learning-roadmap-for-self-taught-engineers.md) A practical deep learning roadmap that connects mathematics, model building, evaluation, systems work, and research reading. - Author: Suraj Gaud - Published: 2026-07-23 - Track: AI research ### [ML interview prep for data scientists](https://fanout.sh/blog/ml-interview-prep-for-data-scientists.md) A focused ML engineering interview plan for data scientists covering coding, modeling, system design, and research communication. - Author: Suraj Gaud - Published: 2026-07-23 - Track: AI research ### [ML math for software engineers](https://fanout.sh/blog/machine-learning-math-for-software-engineers.md) The machine learning mathematics software engineers need for model code, training behavior, evaluation, and production debugging. - Author: Suraj Gaud - Published: 2026-07-23 - Track: ML mathematics ### [System design prep for backend engineers](https://fanout.sh/blog/system-design-interview-prep-for-backend-engineers.md) A system design interview plan that turns backend experience into clear requirements, trade-offs, failure analysis, and operational judgment. - Author: Suraj Gaud - Published: 2026-07-23 - Track: System design ### [System design prep for new grads](https://fanout.sh/blog/system-design-interview-prep-for-new-grads.md) A beginner-friendly system design interview framework for new grads, with estimation, core components, failure reasoning, and mock practice. - Author: Suraj Gaud - Published: 2026-07-23 - Track: System design ### [SLMs are more powerful than you think](https://fanout.sh/blog/slms-are-more-powerful-than-you-think.md) Why small language models might be the secret sauce your AI stack is missing. - Author: Suraj Gaud - Published: 2025-05-18 - Track: AI research --- This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included. Browse the public content map: https://fanout.sh/sitemap.md