---
title: "Foundational AI papers"
description: "The papers that built modern AI, in reading order — attention, scaling laws, RLHF, diffusion — with context on why each one matters."
canonical_url: "https://fanout.sh/ai/papers"
md_url: "https://fanout.sh/ai/papers.md"
access: "public"
---

# Foundational AI papers

The papers that built modern AI, in reading order — attention, scaling laws, RLHF, diffusion — with context on why each one matters.

## Public papers

### [mHC: Manifold-Constrained Hyper-Connections](https://arxiv.org/abs/2512.24880)

Replaces the residual connection paradigm (x + f(x)) used in every major transformer for the past decade. Projects residual paths onto Birkhoff Polytope manifold — neither explode nor collapse at any depth.

- Authors: DeepSeek Research
- Category: Architecture

### [Less is More: Recursive Reasoning with Tiny Networks](https://arxiv.org/abs/2510.04871)

A 7M parameter model beats DeepSeek-R1, Gemini 2.5 Pro, and o3-mini on ARC-AGI benchmarks. Proves recursive reasoning can compensate for massive parameter counts.

- Authors: Jolicoeur-Martineau et al. (Samsung SAIL)
- Category: Architecture

### [Holographic Transformers](https://arxiv.org/abs/2509.19331)

Physics-inspired attention using wave interference — builds holographic memory with cosines, sines, and phase modulation instead of standard dot-product attention.

- Authors: Various
- Category: Architecture

### [Recursive Self-Aggregation for Deep Thinking](https://arxiv.org/abs/2509.26626)

Test-time scaling via evolutionary algorithms — combines parallel and sequential scaling. Enables Qwen3-4B to compete with DeepSeek-R1 on AIME and LiveCodeBench.

- Authors: RSA Team
- Category: Reasoning & Scaling

### [Agentic Context Engineering (ACE)](https://arxiv.org/abs/2510.04618)

Makes fine-tuning potentially irrelevant — evolves the context instead of weights. Delta updates and grow-and-refine mechanisms. 59.5% on AppWorld (+10.6pp).

- Authors: Stanford & SambaNova
- Category: Training & Optimization

### [Verbalized Sampling](https://www.analyticsvidhya.com/blog/2025/10/verbalized-sampling/)

Recovers 66.8% of base model diversity hidden by RLHF alignment. Single prompting technique: ask LLMs to generate multiple responses with probability estimates. 2x diversity, zero safety loss.

- Authors: Stanford
- Category: Training & Optimization

### [GEPA: Reflective Prompt Evolution Outperforms RL](https://arxiv.org/abs/2507.19457)

Prompt optimization via evolutionary search outperforms GRPO by 10-20% while using 35x fewer rollouts. Also beats MIPROv2 by 10%+.

- Authors: Various (ICLR 2026 Oral)
- Category: Training & Optimization

### [Drag-and-Drop LLMs: Zero-Shot Prompt-to-Weights](https://arxiv.org/abs/2506.16406)

Eliminates per-task fine-tuning — maps task prompts directly to LoRA weights via hyper-convolutional decoder. 12,000x lower overhead than full fine-tuning.

- Authors: Liang et al. (NUS, UT Austin, Oxford)
- Category: Efficiency

### [Categories of Inference-Time Scaling](https://magazine.sebastianraschka.com/p/categories-of-inference-time-scaling)

Overview of inference-scaling approaches — recursive language models, chain-of-thought, tree search, and their trade-offs for improved reasoning.

- Authors: Sebastian Raschka
- Category: Reasoning & Scaling

### [PageIndex: RAG Without Vector Databases (98.7% accuracy)](https://pub.towardsai.net/pageindex-the-rag-framework-that-threw-out-vector-databases-and-still-hit-98-7-accuracy-d194e0549478)

RAG approach that achieves 98.7% on FinanceBench without vector DBs, embeddings, chunking, or similarity search. Challenges the assumption that RAG = vector DB.

- Authors: Akshay Kalane
- Category: RAG & Retrieval

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md