Retrieval-Augmented Generation for…

Retrieval-Augmented Generation gives a language model an explicit, updateable memory outside its parameters. Read it to see how a neural retriever and a sequence generator work together, and why grounding generation in retrieved evidence can improve factual, knowledge-intensive answers.

Reading focus: How RAG combines a dense Wikipedia retriever with a pretrained sequence-to-sequence generator. Why parametric memory in model weights and non-parametric memory in a document index solve different parts of the task. How RAG-Sequence and RAG-Token differ in whether retrieved passages stay fixed for a response or can change for each token.

NeurIPS 2020. Lewis et al.. 40 min read, easy difficulty.