---
title: "llama.cpp & Edge-Oriented Runtimes"
description: "How llama.cpp serves one user on one device: the GGUF file mapped into memory, quantization by default, layers as the unit of placement, and server slots."
canonical_url: "https://fanout.sh/inference-eng/curriculum/17-04"
md_url: "https://fanout.sh/inference-eng/curriculum/17-04.md"
access: "pro-preview"
---

# llama.cpp & Edge-Oriented Runtimes

How llama.cpp serves one user on one device: the GGUF file mapped into memory, quantization by default, layers as the unit of placement, and server slots.

- Access: Fanout Pro (public preview)

## Public preview

Every engine this module has read so far serves a machine whose reason to exist is many callers. The last seat belongs to the opposite deployment: one device, one user, and a model that has to fit before any other question matters. The TensorRT-LLM lesson handed that case over, and the change of target changes what the software must be…

## In this lesson

- The bet underneath
- One file, mapped
- Quantization as the default path
- Layers as the unit of placement
- A server with slots

## Continue learning

Pro unlocks the complete lesson, video, practice, and code where available.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md