Subject
2 entries
Regularization
Bookmarks
Overfitting, Regularization, and Hyperparameters
DS Walter's practitioner explainer on overfitting, regularization techniques, and hyperparameter tuning — covering L1/L2 penalties, dropout, and cross-validation. A clear introduction to the bias-variance tradeoff for working data scientists.
Regularizing Neural Networks with Dropout and DropConnect
FastML's comparison of Dropout and DropConnect — two techniques for regularizing neural networks by randomly zeroing activations or weights during training. Clarifies that DropConnect's CIFAR-10 SOTA came from model ensembling, not the technique itself.