---
title: "XGBoost: A Scalable Tree Boosting System"
description: "XGBoost asks how many small decision trees can correct one another without growing into an uncontrolled forest, and how that learning rule can survive sparse, billion-row data. Read it to connect additive leaf scores and regularized split gain to weighted candidate sketches, learned missing-value directions, cache-aware column blocks, and the system work behind a practical tabular-ML classic."
canonical_url: "https://fanout.sh/daily/2026-08-26-xgboost-scalable-tree-boosting"
md_url: "https://fanout.sh/daily/2026-08-26-xgboost-scalable-tree-boosting.md"
last_updated: "2026-08-26"
access: "public"
---

# XGBoost: A Scalable Tree Boosting System

XGBoost asks how many small decision trees can correct one another without growing into an uncontrolled forest, and how that learning rule can survive sparse, billion-row data. Read it to connect additive leaf scores and regularized split gain to weighted candidate sketches, learned missing-value directions, cache-aware column blocks, and the system work behind a practical tabular-ML classic.

## Paper details

- Authors: Chen and Guestrin

- Venue: KDD 2016

- Track: AI Research

- Difficulty: Easy

- Reading time: 35 min

- Original paper: https://arxiv.org/abs/1603.02754v3

## What you will learn

- How each new tree contributes a signed leaf correction to the ensemble's current prediction instead of replacing what earlier trees learned.

- Why a split survives only when its improvement clears a complexity cost, and how shrinkage and leaf-weight penalties restrain the corrections.

- How weighted split candidates, learned default directions for missing values, sorted column blocks, prefetching, compression, and sharding turn the same learner into a scalable system.

## Continue exploring

[Explore trees and ensemble methods](https://fanout.sh/roadmap?track=ai): Open Fanout's AI roadmap and continue through decision trees, random forests, gradient boosting, XGBoost, regularization, and the tabular-model tradeoffs surrounding this paper.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md