RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
RT-2 turns low-level robot actions into the same kind of tokens a vision-language model already writes. Follow the shared vocabulary, co-fine-tuning recipe, transfer evidence, cloud control loop, and the boundary between new semantic uses and new physical skills.
Reading focus: How eight discretized action values let one vision-language model share every weight across web and robot tasks. Why mixing the original web tasks into robot fine-tuning matters more than simply attaching a larger pretrained model. What the seen, unseen, and emergent-skill evaluations establish, and why scale, latency, and narrow robot motions bound the result.
CoRL 2023. Anthony Brohan et al.. 10 min read, medium difficulty.