https://fanout.sh/blog Practical field notes across AI research, system design, inference engineering, and machine learning mathematics. en-US Wed, 19 Aug 2026 12:00:00 GMT https://fanout.sh/blog/token-choice-vs-expert-choice-routing https://fanout.sh/blog/token-choice-vs-expert-choice-routing Wed, 19 Aug 2026 12:00:00 GMT Suraj Gaud AI research How token choice and expert choice routing split MoE work: who picks whom, load balance, dropped tokens, and why decoder LLMs route by token. https://fanout.sh/blog/what-is-a-good-cross-entropy-loss-value https://fanout.sh/blog/what-is-a-good-cross-entropy-loss-value Wed, 19 Aug 2026 12:00:00 GMT Suraj Gaud ML mathematics Judge cross entropy loss against the ln C uniform baseline and the data's entropy floor, with worked numbers from coin flips to GPT-2. https://fanout.sh/blog/ai-inference-engineering-guide https://fanout.sh/blog/ai-inference-engineering-guide Tue, 18 Aug 2026 12:00:00 GMT Suraj Gaud Inference engineering What AI inference engineering is, derived from one GPU ratio: bandwidth ceilings, batching, caching, quantization, and when self-hosting pays. https://fanout.sh/blog/flashattention-vs-pagedattention https://fanout.sh/blog/flashattention-vs-pagedattention Tue, 18 Aug 2026 12:00:00 GMT Suraj Gaud Inference engineering FlashAttention cuts attention kernel memory traffic. PagedAttention cuts KV cache waste. Which one is your bottleneck, and why engines run both. https://fanout.sh/blog/w8a8-vs-w4a16-quantization https://fanout.sh/blog/w8a8-vs-w4a16-quantization Tue, 18 Aug 2026 12:00:00 GMT Suraj Gaud Inference engineering Choose W8A8 or W4A16 quantization from prefill compute, decode bandwidth, and memory-fit arithmetic, with the batch-size crossover that flips the answer. https://fanout.sh/blog/expert-parallelism-vs-tensor-parallelism https://fanout.sh/blog/expert-parallelism-vs-tensor-parallelism Mon, 17 Aug 2026 12:00:00 GMT Suraj Gaud Inference engineering Choose expert or tensor parallelism for MoE serving from weight placement, all-to-all versus all-reduce arithmetic, expert size, and batch occupancy. https://fanout.sh/blog/when-kv-cache-quantization-slows-inference https://fanout.sh/blog/when-kv-cache-quantization-slows-inference Mon, 17 Aug 2026 12:00:00 GMT Suraj Gaud Inference engineering When FP8 or INT4 KV cache quantization slows LLM inference: unsupported attention backends, short contexts, low concurrency, and conversion kernels. https://fanout.sh/blog/disaggregated-prefill-and-decode https://fanout.sh/blog/disaggregated-prefill-and-decode Sun, 16 Aug 2026 12:00:00 GMT Suraj Gaud Inference engineering Calculate KV transfer bytes and time, size prefill and decode pools independently, and decide when disaggregation beats chunked prefill. https://fanout.sh/blog/write-ahead-log-explained https://fanout.sh/blog/write-ahead-log-explained Sun, 16 Aug 2026 12:00:00 GMT Suraj Gaud System design Follow one transfer through WAL records, dirty pages, commit, crash recovery, redo, undo, checkpoints, and group commit. https://fanout.sh/blog/consistent-hashing-explained-simply https://fanout.sh/blog/consistent-hashing-explained-simply Sat, 15 Aug 2026 12:00:00 GMT Suraj Gaud System design Trace 10 keys through modulo hashing and a hash ring, count exactly what moves, then add virtual nodes, replication, and failure limits. https://fanout.sh/blog/softmax-temperature-explained https://fanout.sh/blog/softmax-temperature-explained Sat, 15 Aug 2026 12:00:00 GMT Suraj Gaud ML mathematics See softmax at T=0.5, 1, and 2, then separate temperature's roles in sampling, calibration, training, and knowledge distillation. https://fanout.sh/blog/fp8-vs-int8-vs-awq-vs-gptq https://fanout.sh/blog/fp8-vs-int8-vs-awq-vs-gptq Fri, 14 Aug 2026 12:00:00 GMT Suraj Gaud Inference engineering FP8 and INT8 are number formats. AWQ and GPTQ are weight quantization methods. Compare memory, kernels, quality, and the workloads each fits. https://fanout.sh/blog/mixture-of-experts-routing-explained https://fanout.sh/blog/mixture-of-experts-routing-explained Fri, 14 Aug 2026 12:00:00 GMT Suraj Gaud AI research Follow one token through MoE router logits, top-k softmax, expert outputs, and capacity limits. See why routing balance is a systems constraint. https://fanout.sh/blog/kv-cache-quantization https://fanout.sh/blog/kv-cache-quantization Thu, 13 Aug 2026 12:00:00 GMT Suraj Gaud Inference engineering Calculate how much KV cache quantization saves after scale metadata and residual tokens, then decide when lower precision is worth the tradeoff. https://fanout.sh/blog/why-is-flash-attention-faster https://fanout.sh/blog/why-is-flash-attention-faster Thu, 13 Aug 2026 12:00:00 GMT Suraj Gaud Inference engineering FlashAttention is faster because it avoids writing full attention matrices to HBM. Follow the 4K-token memory traffic and exact softmax method. https://fanout.sh/blog/grok-4-6-cursor-data-flywheel https://fanout.sh/blog/grok-4-6-cursor-data-flywheel Wed, 12 Aug 2026 12:00:00 GMT Suraj Gaud AI research The six-month story behind Grok 4.6: the $60B Cursor deal, trillions of trace tokens, and a post-training recipe where Grok 4.5 built its successor. https://fanout.sh/blog/tensor-parallelism-vs-pipeline-parallelism https://fanout.sh/blog/tensor-parallelism-vs-pipeline-parallelism Wed, 12 Aug 2026 12:00:00 GMT Suraj Gaud Inference engineering Choose tensor or pipeline parallelism for LLM inference using memory fit, interconnect topology, latency goals, and worked communication arithmetic. https://fanout.sh/blog/what-is-chunked-prefill https://fanout.sh/blog/what-is-chunked-prefill Wed, 12 Aug 2026 12:00:00 GMT Suraj Gaud Inference engineering Chunked prefill splits long prompts across scheduler iterations. See the token-budget arithmetic and the TTFT, ITL, throughput, and memory tradeoffs. https://fanout.sh/blog/argmax-vs-max https://fanout.sh/blog/argmax-vs-max Sun, 09 Aug 2026 12:00:00 GMT Suraj Gaud ML mathematics Learn why max returns the best value while argmax returns the choice that produced it, including ties, types, axes, and an interactive sandbox. https://fanout.sh/blog/how-to-read-equations-ai-research-papers https://fanout.sh/blog/how-to-read-equations-ai-research-papers Sun, 09 Aug 2026 12:00:00 GMT Suraj Gaud ML mathematics A practical method for reading notation, scope, types, and meaning in AI research-paper equations, with an interactive reinforcement-learning example. https://fanout.sh/blog/kv-cache-memory-formula-llm-inference https://fanout.sh/blog/kv-cache-memory-formula-llm-inference Sun, 09 Aug 2026 12:00:00 GMT Suraj Gaud Inference engineering Derive KV-cache bytes from tensor shapes, compare MHA, GQA, and MQA, test scaling laws, and separate logical payload from reserved GPU memory. https://fanout.sh/blog/softmax-cross-entropy-from-logits https://fanout.sh/blog/softmax-cross-entropy-from-logits Sun, 09 Aug 2026 12:00:00 GMT Suraj Gaud ML mathematics Follow logits through stable softmax and categorical cross-entropy, with exact invariants, edge cases, API assumptions, and an overflow demonstration. https://fanout.sh/blog/roofline-model-ai-inference https://fanout.sh/blog/roofline-model-ai-inference Sun, 09 Aug 2026 12:00:00 GMT Suraj Gaud Inference engineering Use operational intensity, bandwidth, and peak compute to reason about inference bottlenecks without mistaking a bound for a latency prediction. https://fanout.sh/blog/scaled-dot-product-attention-equation https://fanout.sh/blog/scaled-dot-product-attention-equation Sun, 09 Aug 2026 12:00:00 GMT Suraj Gaud ML mathematics Decode query-key scores, square-root scaling, softmax, value mixing, tensor shapes, and the assumptions behind the Transformer attention equation. https://fanout.sh/blog/what-does-vertical-bar-mean-in-mathematics https://fanout.sh/blog/what-does-vertical-bar-mean-in-mathematics Sun, 09 Aug 2026 12:00:00 GMT Suraj Gaud ML mathematics Resolve conditioning, magnitude, cardinality, determinants, set-builder notation, divisibility, and evaluation by reading the mathematical frame. https://fanout.sh/blog/robotics-roadmap-software-engineers https://fanout.sh/blog/robotics-roadmap-software-engineers Thu, 06 Aug 2026 12:00:00 GMT Suraj Gaud System design A practical path from software engineering to robotics, covering math, ROS 2, simulation, controls, perception, planning, and real hardware. https://fanout.sh/blog/llm-api-cost-estimation https://fanout.sh/blog/llm-api-cost-estimation Thu, 06 Aug 2026 12:00:00 GMT Suraj Gaud Inference engineering A workload-first method for estimating LLM API spend across input, output, caching, tools, retries, batch jobs, traffic growth, and quality. https://fanout.sh/blog/how-to-use-ai-to-study-for-exams https://fanout.sh/blog/how-to-use-ai-to-study-for-exams Thu, 06 Aug 2026 12:00:00 GMT Suraj Gaud AI research Build an exam plan with AI that uses active recall, spaced practice, mixed problems, error logs, and timed tests without outsourcing answers. https://fanout.sh/blog/how-to-use-ai-to-study https://fanout.sh/blog/how-to-use-ai-to-study Thu, 06 Aug 2026 12:00:00 GMT Suraj Gaud AI research Use AI as a tutor, question writer, and feedback partner while keeping retrieval, explanation, practice, and judgment in your own hands. https://fanout.sh/blog/llm-inference-interview-questions https://fanout.sh/blog/llm-inference-interview-questions Thu, 06 Aug 2026 12:00:00 GMT Suraj Gaud Inference engineering Twenty practical LLM inference interview questions with answer rubrics covering latency, KV cache, batching, GPUs, parallelism, and reliability. https://fanout.sh/blog/openai-vs-anthropic-api-pricing https://fanout.sh/blog/openai-vs-anthropic-api-pricing Thu, 06 Aug 2026 12:00:00 GMT Suraj Gaud Inference engineering A dated comparison of OpenAI and Anthropic API token prices, cache costs, batch discounts, long-context rules, and workload-level cost. https://fanout.sh/blog/speculative-decoding-llm-inference https://fanout.sh/blog/speculative-decoding-llm-inference Thu, 06 Aug 2026 12:00:00 GMT Suraj Gaud Inference engineering How speculative decoding proposes and verifies tokens, why it can preserve model outputs, and when draft models make latency worse. https://fanout.sh/blog/llm-inference-engineer-roadmap https://fanout.sh/blog/llm-inference-engineer-roadmap Thu, 06 Aug 2026 12:00:00 GMT Suraj Gaud Inference engineering A practical roadmap through GPU fundamentals, transformer decoding, KV cache memory, batching, parallelism, serving, and production measurement. https://fanout.sh/blog/open-source-robotics-stack https://fanout.sh/blog/open-source-robotics-stack Thu, 06 Aug 2026 12:00:00 GMT Suraj Gaud System design How ROS 2, Gazebo, MuJoCo, MoveIt, ros2_control, LeRobot, and open hardware fit together in a modern robotics project. https://fanout.sh/blog/vision-language-action-models-robotics https://fanout.sh/blog/vision-language-action-models-robotics Thu, 06 Aug 2026 12:00:00 GMT Suraj Gaud AI research How VLA models turn images and instructions into robot actions, what OpenVLA changed, and where evaluation and deployment still break. https://fanout.sh/blog/100-papers-to-understand-software-and-computing https://fanout.sh/blog/100-papers-to-understand-software-and-computing Wed, 05 Aug 2026 12:00:00 GMT Suraj Gaud System design A foundational reading list for getting started with programming, databases, networks, distributed systems, interfaces, and machine learning. https://fanout.sh/blog/ai-homelab-one-computer https://fanout.sh/blog/ai-homelab-one-computer Wed, 05 Aug 2026 12:00:00 GMT Medhansh System design Connect every Mac, home server, and cloud worker into one secure AI software factory, with the tools that matter and the order to add them. https://fanout.sh/blog/continuous-batching-llm-inference https://fanout.sh/blog/continuous-batching-llm-inference Mon, 03 Aug 2026 12:00:00 GMT Suraj Gaud Inference engineering How iteration-level scheduling lets LLM requests join and leave a live batch, and how token budgets, KV memory, and preemption shape latency. https://fanout.sh/blog/pagedattention-vllm-kv-cache https://fanout.sh/blog/pagedattention-vllm-kv-cache Mon, 03 Aug 2026 12:00:00 GMT Suraj Gaud Inference engineering A systems guide to PagedAttention, KV blocks, block tables, copy-on-write, fragmentation, and what vLLM’s memory manager changes. https://fanout.sh/blog/prefill-vs-decode-llm-inference https://fanout.sh/blog/prefill-vs-decode-llm-inference Mon, 03 Aug 2026 12:00:00 GMT Suraj Gaud Inference engineering A practical guide to prefill, decode, TTFT, inter-token latency, chunked prefill, and why one LLM request creates two serving workloads. https://fanout.sh/blog/how-ai-memory-works-five-systems https://fanout.sh/blog/how-ai-memory-works-five-systems Wed, 29 Jul 2026 12:00:00 GMT Suraj Gaud AI research A technical map of model weights, context windows, KV caches, recurrent state, retrieval, and the limits of what AI systems remember. https://fanout.sh/blog/kimi-k3-architecture-kda-mla-moe https://fanout.sh/blog/kimi-k3-architecture-kda-mla-moe Wed, 29 Jul 2026 12:00:00 GMT Suraj Gaud Inference engineering A source-led guide to Kimi K3’s hybrid attention, Attention Residuals, Stable LatentMoE, cache design, and deployment tradeoffs. https://fanout.sh/blog/kv-cache-formula-llm-inference-memory https://fanout.sh/blog/kv-cache-formula-llm-inference-memory Wed, 29 Jul 2026 12:00:00 GMT Suraj Gaud Inference engineering Derive KV-cache memory from layers, KV heads, head dimension, context, batch, and precision, then connect the result to GPU capacity. https://fanout.sh/blog/daily-paper-summaries-for-ai-research-engineers https://fanout.sh/blog/daily-paper-summaries-for-ai-research-engineers Sat, 25 Jul 2026 12:00:00 GMT Suraj Gaud AI research How AI research engineers can use daily paper summaries for discovery, verification, durable notes, experiments, and team decisions. https://fanout.sh/blog/deep-learning-course-for-computer-vision-engineers https://fanout.sh/blog/deep-learning-course-for-computer-vision-engineers Sat, 25 Jul 2026 12:00:00 GMT Suraj Gaud AI research How to choose and sequence computer vision learning across foundations, current architectures, experiments, evaluation, and deployment. https://fanout.sh/blog/machine-learning-roadmap-for-backend-engineers https://fanout.sh/blog/machine-learning-roadmap-for-backend-engineers Sat, 25 Jul 2026 12:00:00 GMT Suraj Gaud Inference engineering A backend-first machine learning roadmap covering model behavior, data, evaluation, serving, observability, and inference systems. https://fanout.sh/blog/how-to-prepare-for-ml-engineer-interviews-as-a-software-engineer https://fanout.sh/blog/how-to-prepare-for-ml-engineer-interviews-as-a-software-engineer Sat, 25 Jul 2026 12:00:00 GMT Suraj Gaud AI research A role-specific ML engineer interview plan for software engineers covering coding, mathematics, modeling, systems, projects, and mocks. https://fanout.sh/blog/system-design-prep-for-senior-engineer-interviews https://fanout.sh/blog/system-design-prep-for-senior-engineer-interviews Sat, 25 Jul 2026 12:00:00 GMT Suraj Gaud System design Senior-level system design preparation focused on scope, trade-offs, evolution, failure recovery, operations, and clear technical leadership. https://fanout.sh/blog/ai-research-papers-for-system-design-engineers https://fanout.sh/blog/ai-research-papers-for-system-design-engineers Fri, 24 Jul 2026 12:00:00 GMT Suraj Gaud System design A focused paper-reading path from distributed systems foundations to modern AI serving, with a practical protocol for each paper. https://fanout.sh/blog/machine-learning-engineer-roadmap-for-career-switchers https://fanout.sh/blog/machine-learning-engineer-roadmap-for-career-switchers Fri, 24 Jul 2026 12:00:00 GMT Suraj Gaud AI research A career-switch roadmap that maps existing engineering strengths to ML foundations, model development, production systems, and credible projects. https://fanout.sh/blog/ml-math-roadmap-for-engineers-without-a-math-degree https://fanout.sh/blog/ml-math-roadmap-for-engineers-without-a-math-degree Fri, 24 Jul 2026 12:00:00 GMT Suraj Gaud ML mathematics A dependency-based ML math roadmap for engineers covering notation, linear algebra, calculus, probability, optimization, and practice. https://fanout.sh/blog/research-paper-reading-habit-for-staff-engineers https://fanout.sh/blog/research-paper-reading-habit-for-staff-engineers Fri, 24 Jul 2026 12:00:00 GMT Suraj Gaud AI research A sustainable paper-reading system for staff engineers built around work questions, triage, durable notes, discussion, and application. https://fanout.sh/blog/system-design-labs-for-distributed-systems-engineers https://fanout.sh/blog/system-design-labs-for-distributed-systems-engineers Fri, 24 Jul 2026 12:00:00 GMT Suraj Gaud System design A lab framework for distributed systems engineers that tests invariants, failure recovery, observability, performance, and design judgment. https://fanout.sh/blog/deep-learning-roadmap-for-self-taught-engineers https://fanout.sh/blog/deep-learning-roadmap-for-self-taught-engineers Thu, 23 Jul 2026 12:00:00 GMT Suraj Gaud AI research A practical deep learning roadmap that connects mathematics, model building, evaluation, systems work, and research reading. https://fanout.sh/blog/ml-interview-prep-for-data-scientists https://fanout.sh/blog/ml-interview-prep-for-data-scientists Thu, 23 Jul 2026 12:00:00 GMT Suraj Gaud AI research A focused ML engineering interview plan for data scientists covering coding, modeling, system design, and research communication. https://fanout.sh/blog/machine-learning-math-for-software-engineers https://fanout.sh/blog/machine-learning-math-for-software-engineers Thu, 23 Jul 2026 12:00:00 GMT Suraj Gaud ML mathematics The machine learning mathematics software engineers need for model code, training behavior, evaluation, and production debugging. https://fanout.sh/blog/system-design-interview-prep-for-backend-engineers https://fanout.sh/blog/system-design-interview-prep-for-backend-engineers Thu, 23 Jul 2026 12:00:00 GMT Suraj Gaud System design A system design interview plan that turns backend experience into clear requirements, trade-offs, failure analysis, and operational judgment. https://fanout.sh/blog/system-design-interview-prep-for-new-grads https://fanout.sh/blog/system-design-interview-prep-for-new-grads Thu, 23 Jul 2026 12:00:00 GMT Suraj Gaud System design A beginner-friendly system design interview framework for new grads, with estimation, core components, failure reasoning, and mock practice. https://fanout.sh/blog/slms-are-more-powerful-than-you-think https://www.surajgaud.com/blog/slms-are-more-powerful-than-you-think Sun, 18 May 2025 12:00:00 GMT Suraj Gaud AI research Why small language models might be the secret sauce your AI stack is missing.