Ai Research

Why GPT-3 Matters

Leo Gao's early analysis of why GPT-3 is qualitatively different from prior language models — written in May 2020 when GPT-3 was first announced. One of the clearer technical takes on what the scaling breakthrough meant, from someone who later worked on it.