Training Transparency

OPT Training Chronicles

Meta's OPT-175B training logbook — a day-by-day record of hardware failures, instabilities, loss spikes, and interventions during a ~2 month distributed training run. Rare public documentation of what large language model training actually looks like at scale.