--- title: "AI, system design & ML math roadmaps" description: "Explore visual learning roadmaps for AI research, system design, and machine learning mathematics, with direct paths into Fanout courses." canonical_url: "https://fanout.sh/roadmap" md_url: "https://fanout.sh/roadmap.md" access: "public" --- # AI, system design & ML math roadmaps Explore visual learning roadmaps for AI research, system design, and machine learning mathematics, with direct paths into Fanout courses. ## AI Research The current Fanout path from mathematical intuition and frameworks to frontier models, operations, and publishing. ### Math Fundamentals The mathematical foundations you need for AI research — from functions and derivatives to information theory and SVD. - Functions - Derivatives - Vectors - Gradients - Matrices - Derivation Rules & Examples - The Chain Rule - Backprop in Python - The Jacobian Matrix - Hadamard Product (Element-wise Op) - Entropy & Information Theory - KL Divergence - Singular Value Decomposition (SVD) - Moving Averages (EMA) - More Math Lessons ### Core AI Intuitions Build deep intuition for the core operations that power all of AI — dot products, softmax, broadcasting, and norms. - Similarity With Dot Product - Softmax Probabilities - Tensor Broadcasting - L1 vs L2 Norms ### PyTorch Fundamentals Master the tensor operations that are the building blocks of every neural network implementation. - Creating Tensors - Matrix Multiplication - Transposing Tensors - flatten, reshape, view, squeeze, unsqueeze - Indexing and Slicing - cat, stack - Special Tensors (eye, rand, arange, linspace) - New Full Lesson - 7 PyTorch Tasks (Advanced) ### TensorFlow Fundamentals Learn TensorFlow from the ground up — linear models, CNNs, transfer learning, adversarial examples, NLP, reinforcement learning, and more. - Simple Linear Model - Convolutional Neural Network - Pretty Tensor - Layers API - Keras API - Save & Restore - Ensemble Learning - CIFAR-10 - Inception Model - Transfer Learning - Video Data - Fine-Tuning - Adversarial Examples - Adversarial Noise for MNIST - Visual Analysis - Visual Analysis for MNIST - Deep Dream - Style Transfer - TensorFlow GPU vs CPU - Reinforcement Learning - Estimator API - TFRecords & Dataset API - Hyper-Parameter Optimization - Natural Language Processing - Machine Translation - Image Captioning - Time-Series Prediction ### Neural Network from Scratch Build neural networks from the ground up — single neurons, layers, training loops, normalization, and optimization. - Single Neuron From Scratch - Building a Layer - Implementing a Network - RMSNorm - Learning Rate, Decay - Adam Optimizer - Neural Network From Scratch ### Transformers The architecture that changed everything — attention mechanisms, self-attention, and building GPT from scratch. - Attention Mechanism Explained - Self Attention from Scratch - GPT From Scratch ### Reinforcement Learning How agents learn from interaction — from basic environments to PPO and modern LLM reasoning techniques. - Agents & Environments - Policy Gradients (REINFORCE) - Deep Q-Learning (DQN) - PPO, LLM Reasoning, Importance Ratio, Advantage - Qwen 3 GSPO & DeepSeek GRPO — LLM Reasoning ### LLM From Scratch Build state-of-the-art large language models from scratch — LLaMA 4, DeepSeek V3, Qwen 3, and more. - Llama 4 From Scratch - DeepSeek V3 From Scratch - Qwen 3 From Scratch - Self-Study LLM ### Write Research Paper The complete workflow from coding experiments to writing and publishing an AI research paper. - Code, Write & Publish AI Research Paper ### How to Fine-Tune Models From LoRA to full fine-tuning — learn to adapt pre-trained models to your data and tasks. - Why Fine-Tune? - LoRA & QLoRA - Data Preparation - Training & Hyperparameters - Evaluation & Deployment ### Machine Learning Operations (MLOps) Deploy and maintain ML models in production — from Git and Docker to Kubernetes, CI/CD, and monitoring with Prometheus & Grafana. - Introduction to MLOps - Git & GitHub - Python OOP for MLOps - Data Versioning with DVC - ML Pipeline with DVC & AWS S3 - MLflow | Experiment Tracking - Continuous Integration - Docker - Project: Vehicle Insurance Domain - MongoDB Setup & Notebook Experiment - Data Ingestion Component - Data Validation & Transformation - Model Evaluation & AWS S3 - Building ML App with FastAPI - Complete CI/CD on AWS - Kubernetes Part 1 - Project: First Kubernetes Deployment - Prometheus & Grafana - Project: Prometheus-Grafana on Kubernetes - Capstone Project: End-to-End MLOps - Experiment Tracking with MLflow & DagsHub - App Building & Automation with DVC - CI/CD Implementation (Capstone) - EKS Cluster Deployment - Prometheus-Grafana on EKS ### Bonus Lessons Advanced topics — training dynamics, activation functions, and cutting-edge reasoning architectures. - Train LLM — Sequence Length vs Batch Size - SwiGLU — Better Neural Networks - 100x AI Reasoning — Tiny Recursive Model ## System Design Fourteen real course modules spanning requirements, data systems, coordination, information systems, and production reliability. ### Foundations Scalability basics, system design process, tradeoffs, and capacity planning. - System Design Interview: A Step-By-Step Guide - Horizontal vs. Vertical Scaling - Back-of-the-Envelope Estimation / Capacity Planning - Monolithic vs Microservice Architecture - System Design Primer: Distributed Systems - Scalability – Harvard CS75 Lecture ### APIs, Services & Protocols API design, load balancing, gRPC vs REST, retries, and backpressure. - API Architectural Styles: REST, SOAP, gRPC, GraphQL - Reverse Proxy vs API Gateway vs Load Balancer - What Is RPC? gRPC Introduction - Proxy vs Reverse Proxy (Real-world Examples) - What Is an API Gateway? - Idempotency in APIs Explained ### Data Modeling & SQL Relational database design, indexing, ACID, locking, WAL, and schema evolution. - 7 Database Paradigms - Secret to Optimizing SQL Queries – Understand the SQL Execution Order - Database Indexing Explained (with PostgreSQL) - ACID Properties in Databases With Examples - How Does Database Sharding Work? - Tree Indexes: B+Trees – CMU Intro to Database Systems ### NoSQL, Partitioning & IDs NoSQL databases, sharding, consistent hashing, Bloom filters, and ID generation. - How to Choose the Right Database? - Consistent Hashing – Algorithms You Should Know - Bloom Filters – Algorithms You Should Know - Distributed ID Generation – Twitter Snowflake - What Is Database Sharding? ### Caching & Fast Reads Caching strategies, eviction policies, Redis, CDN, and cache invalidation. - Caching Strategies: Write-Through, Write-Back, Cache Aside - Why Is Single-Threaded Redis So Fast? - Cache Systems Every Developer Should Know - What Is a CDN? How Does It Work? - Cache Invalidation Strategies Explained ### Distributed Coordination Distributed systems foundations, consensus, leader election, gossip protocols. - Designing for Understandability: The Raft Consensus Algorithm - CAP Theorem Explained in 5 Minutes - Distributed Systems 6.1: Consensus – Martin Kleppmann - Gossip Protocol – System Design Concepts - Leader Election in Distributed Systems - Distributed Locks – System Design Basics ### Storage Engines LSM trees, B-trees, SSTables, compaction, object storage, and write-ahead logs. - How Databases Store Data on Disk (B-Trees vs LSM Trees) - Architecting Amazon S3: Scalable Object Storage - Database Storage Models & Data Layout – CMU Advanced Databases - Event Sourcing and Stream Processing – Martin Kleppmann - Write-Ahead Logging: Crash Recovery & Durability Explained ### Async Work & Streams Message queues, Kafka, event-driven architecture, task scheduling. - Apache Kafka in 3 Minutes - System Design Interview – Distributed Message Queue - Event-Driven Architecture Explained in 7 Minutes - Kafka vs RabbitMQ - Distributed Task Scheduler – System Design Interview ### Search & Retrieval Inverted indexes, ranking algorithms, full-text search, autocomplete. - What's Elasticsearch Used For? Search Indexes Explained - Design a Basic Search Engine – System Design Interview - Elasticsearch from the Bottom Up - Design Autocomplete / Typeahead Suggestions - Design a Web Crawler – Systems Design Interview ### Analytics & Sketches Counting at scale, HyperLogLog, streaming analytics, probabilistic data structures. - HyperLogLog – Counting Distinct Values at Scale - Top K Problem – Heavy Hitters - How YouTube Counts Views – Lambda Architecture - Count-Min Sketch – Counting in Data Streams - Reservoir Sampling & Streaming Algorithms ### Realtime, Social & Feeds WebSockets, news feeds, social graph modeling, push vs pull fanout. - Short Polling vs Long Polling vs WebSockets - Designing a News Feed System - WhatsApp System Design: Chat Messaging Systems - HTTP Long-Polling vs WebSockets vs Server-Sent Events - Facebook / Instagram System Design ### Geo, Matching & Recs Geospatial indexing, geohash, proximity search, matching algorithms. - Design a Location Based Service (Yelp, Google Places) - Geohash: Deep Intuitive Understanding - System Design: Uber Lyft Ride Sharing Services - Design Tinder – System Design Interview - Design Google Maps – System Design Interview ### Media, Files & CDN CDN architecture, video transcoding, file uploads, signed URLs, live streaming. - Design YouTube – System Design Interview - How CDN Works – System Design - Design Google Drive / Dropbox – File Storage System - Amazon S3: Presigned URLs & Multipart Upload - Live Streaming Architecture: How YouTube & Twitch Work ### Reliability & Operations Observability, SLOs, deployment strategies, rate limiting, and incident response. - Design a Rate Limiter – System Design Interview - How Service Observability Works – System Design - SLAs, SLOs, and SLIs Explained - Deployment Strategies: Blue-Green, Canary, Rolling - Circuit Breaker Pattern – Fault Tolerant Microservices ## In-depth ML A rigorous machine-learning path built around mathematical understanding, classical model families, trustworthy evaluation, and research practice. ### Mathematics for Models The linear algebra, calculus, probability, and optimization that explain why models learn. - Linear algebra - Multivariable calculus - Probability & statistics - Convex optimization ### Data, Features & Preprocessing Understand the data-generating process before choosing a model or trusting a metric. - Sampling & data generation - EDA and missingness - Scaling & encoding - Feature engineering - Data leakage ### Learning Theory & Generalization Reason about what can be learned, why models overfit, and how regularization changes the hypothesis space. - Empirical risk - Bias–variance - Regularization - PAC intuition - Distribution shift ### Linear & Generalized Linear Models Build the most interpretable model family from its objective through calibration and diagnostics. - Linear regression - Logistic regression - GLMs - Regularized regression - Calibration ### Trees & Ensemble Methods Move from recursive partitions to bagging and boosting, with a focus on tabular performance. - Decision trees - Random forests - Gradient boosting - XGBoost & LightGBM - Feature importance ### Geometry, Margins & Kernel Methods Use geometry to understand maximum-margin learning and nonlinear decision boundaries. - Geometric classifiers - Support vector machines - The kernel trick - Kernel selection - Large-margin loss ### Probabilistic Models Model uncertainty explicitly with likelihoods, priors, latent variables, and approximate inference. - MLE & MAP - Bayesian inference - Naive Bayes - Gaussian mixtures - Graphical models ### Unsupervised Learning Find useful structure without labels through clustering, representation, and density estimation. - K-means & clustering - PCA & factorization - Manifold learning - Density estimation - Anomaly detection ### Evaluation, Selection & Error Analysis Design estimates you can trust and turn model failures into the next experiment. - Train/validation/test - Cross-validation - Metrics & thresholds - Hyperparameter search - Error analysis ### Neural Networks & Deep Learning Derive backpropagation, understand optimization dynamics, and build modern representation learners. - Backpropagation - Initialization & normalization - Optimizers - CNNs & sequence models - Attention ### Representation & Generative Models Learn latent representations and the major families used to generate or transform data. - Embeddings - Autoencoders - GANs - Diffusion models - Self-supervised learning ### Production ML & Research Practice Carry a model from a reproducible experiment into a monitored system that survives change. - Reproducible experiments - Experiment tracking - Serving & monitoring - Drift & retraining - Ablations & reporting --- This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included. Browse the public content map: https://fanout.sh/sitemap.md