Subject
2 entries
Generalization
Bookmarks
On the Paradox of Learning to Reason from Data
Zhang, Li, Meng, Chang, and Van den Broeck (UCLA) show that BERT achieves near-perfect accuracy on in-distribution logical reasoning problems while completely failing to generalize to other distributions over the same problem space. The explanation: BERT learned statistical features of the logical reasoning distribution, not the underlying reasoning function — a fundamental distinction between benchmark performance and genuine reasoning.
The Overfitted Brain: Dreams Evolved to Assist Generalization
A 2020 paper proposing that dreams evolved as a biological regularization mechanism — the brain 'trains' on noisy, hallucinated data during sleep to prevent overfitting to waking experience. A striking bridge between ML theory and sleep neuroscience.