STEM on Ryan Orban

Solving Quantitative Reasoning Problems with Language Models (Minerva)

me@ryanorban.com (Ryan Orban) — Wed, 17 Aug 2022 00:00:00 +0000

Summary

Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, and colleagues at Google Research (arXiv:2206.14858, Jul 2022) introduce Minerva, a large language model pretrained on general natural language data and then further trained on a curated corpus of technical content — scientific papers, mathematics textbooks, and course notes. The key design insight is that quantitative reasoning requires a different pretraining distribution than general language tasks, and that targeted continued pretraining can close a substantial capability gap without architectural changes.

U.S. Pushes for More Scientists, But the Jobs Aren't There

me@ryanorban.com (Ryan Orban) — Sun, 08 Jul 2012 00:00:00 +0000

Summary

This 2012 Washington Post investigation challenged the prevailing policy narrative that the United States faced a critical shortage of scientists and engineers. Federal agencies, universities, and technology companies were all pushing for more STEM education investment and increased PhD production — but the data on actual employment outcomes for science PhDs told a more complicated story. The article documented a structural oversupply in many scientific fields: more graduates than available positions, with consequences for both career trajectories and research culture.