Subject
3 entries
Self Supervised Learning
Bookmarks
Deep Anomaly Detection with Self-Supervised Learning and Adversarial Training
This paper combines self-supervised learning and adversarial training to improve deep anomaly detection, leveraging unlabeled normal data to learn representations that are robust to perturbations and more sensitive to out-of-distribution inputs. It matters because labeled anomaly data is rare in practice, making self-supervised approaches essential for real-world deployment.
Self-Supervised Learning: The Dark Matter of Intelligence
Yann LeCun and Ishan Misra's Facebook AI blog post arguing that self-supervised learning — learning from unlabeled data — is the key to human-level AI, analogous to the dark matter that makes up most of the universe's mass. Published ahead of a wave of self-supervised breakthroughs.
Generative Pretraining from Pixels (iGPT)
The iGPT paper from OpenAI (ICML 2020) showing that a GPT-2-scale transformer trained to autoregressively predict pixels learns strong image representations — 96.3% accuracy on CIFAR-10 with a linear probe. It's a direct transposition of NLP pretraining ideas to the image domain, predating CLIP and DALL-E.