---
title: "GPU programming resources"
description: "CUDA, Triton, and GPU architecture — curated resources for making models fast, from single kernels to distributed training."
canonical_url: "https://fanout.sh/ai/gpu"
md_url: "https://fanout.sh/ai/gpu.md"
access: "public"
---

# GPU programming resources

CUDA, Triton, and GPU architecture — curated resources for making models fast, from single kernels to distributed training.

## Public GPU resources

From parallel programming fundamentals to CUDA kernel optimization — everything you need to understand and program GPUs for deep learning.

## Start here

### [GPU Glossary](https://modal.com/gpu-glossary)

- Author: Modal
- Type: explainer

### [Making Deep Learning Go Brrrr From First Principles](https://horace.io/brrr_intro.html)

- Author: Horace He
- Type: explainer

## Parallel Programming

### [Is Parallel Programming Hard, And, If So, What Can You Do About It](https://mirrors.edge.kernel.org/pub/linux/kernel/people/paulmck/perfbook/perfbook.2023.06.11a.pdf)

- Author: Paul E. McKenney
- Type: book

### [Intro to Parallel Programming](https://www.youtube.com/playlist?list=PLAwxTw4SYaPnFKojVQrmyOGFCqHTxfdv2)

- Type: course

### [Programming Parallel Computers](https://ppc.cs.aalto.fi/)

- Author: Aalto University
- Type: course

### [Python Parallel Computing](https://www.youtube.com/playlist?list=PL30NBs02RsiUbmXVPDo56APsU0xa6gfL2)

- Type: course

### [NHR@FAU - Parallel Programming](https://www.youtube.com/playlist?list=PLxVedhmuwLq2Ie88ODuZufCGorawPO1AP)

- Type: course

## CUDA

### [How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance](https://siboehm.com/articles/22/CUDA-MMM)

- Author: siboehm
- Type: tutorial

### [CUDA Programming Course — High-Performance Computing with GPUs](https://www.youtube.com/watch?v=86FAWCzIe_4)

- Author: elliotarledge
- Type: course

### [GPU(CUDA) MODE](https://www.youtube.com/@GPUMODE)

- Type: video

### [Programming Massively Parallel Processors](https://github.com/h3ct0rjs/HighPerformanceComputing/blob/master/BookRef/Programming%20Massively%20Parallel%20Processors.pdf)

- Type: book

### [CUDA Toolkit Documentation](https://docs.nvidia.com/cuda/)

- Author: NVIDIA
- Type: tutorial

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md