Subject
33 entries
Neural Networks
Bookmarks
Understanding Neural Networks, Visually
An interactive web visualization that walks through how neural networks work — weights, activations, backpropagation — through direct manipulation. Good at building intuition rather than just showing diagrams.
Neural Networks — 3Blue1Brown
3Blue1Brown's neural networks playlist is the canonical visual introduction to how neural networks and backpropagation work — four episodes, starting from scratch and building to the chain rule. The most-watched math explainer for deep learning fundamentals.
MLGarden: Visual Neural Network Editor
MLGarden is an interactive visual editor for building and training neural networks without linear algebra — you construct computation graphs visually, watch backpropagation happen, and train on classification tasks. Built in C++ for performance, runs in the browser for accessibility.
Heinsen Routing: Vector Routing for Sequences and Capsule Networks
Heinsen routing is a novel algorithm for routing vectors in sequences and capsule networks — an alternative to Hinton's dynamic routing that generalizes across domains. A niche but foundational contribution to the capsule network research line.
Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation
NeurIPS 2022 paper investigating how to train very deep Transformers by removing shortcut connections (residual paths), which typically cause rank collapse in attention layers. The work has implications for understanding how information propagates through depth in Transformer architectures.
Editable Neural Networks
ICLR paper proposing 'editable neural networks' — a method for making targeted modifications to a model's behavior on specific inputs without disrupting performance elsewhere. An early formalization of the model editing problem.
Attention and Augmented Recurrent Neural Networks (Distill)
Distill's 2016 interactive visual explainer covering four augmented RNN variants — attention, memory networks, neural Turing machines, and neural programmers. A foundational reference for understanding how external memory and attention were grafted onto recurrent networks before the transformer era.
Colah's Blog — Chris Olah on Neural Networks
Chris Olah's personal blog is foundational reading for anyone trying to understand deep learning from first principles — his LSTM explainer, neural network topology post, and attention posts have shaped how a generation of researchers think about these models. Olah went on to co-found Anthropic's interpretability team, and the blog reflects his interest in mechanistic understanding.
Temporal Neural Networks
A paper on Temporal Neural Networks, architectures that encode time as a first-class computation dimension rather than implicitly through state, likely in the context of spiking neural networks or neuromorphic computing. Saved July 2022; specific paper identity unclear from the filename alone.
Deep Learning with PyTorch
Manning's 2020 practical guide to deep learning with PyTorch by Stevens, Antiga, and Viehmann, covering tensors through CNNs, RNNs, generative models, and production deployment. The go-to book for practitioners who want to understand PyTorch from first principles rather than copy-paste patterns.
DeepDream: How Alexander Mordvintsev Excavated the Computer's Hidden Layers
The story behind Google's DeepDream — how researcher Alexander Mordvintsev discovered that running gradient ascent on a convolutional network's hidden layers produces psychedelic imagery that reveals what features the network learned. A landmark moment in neural network interpretability.
Neuro-Fuzzy
Neuro-fuzzy systems combine neural network learning with fuzzy logic's ability to handle imprecise, rule-based reasoning — a hybrid AI approach popular in consumer electronics and control systems during the 1990s-2000s. Famous for appearing on washing machine panels before deep learning made these labels obsolete.
Fizz Buzz in TensorFlow
Joel Grus deliberately solves FizzBuzz using TensorFlow — a satirical demonstration of using a neural network where simple conditional logic would do. A sharp critique of ML overcomplexity disguised as a tutorial.
What a Deep Neural Network Thinks About Your Selfie
Andrej Karpathy trained a VGGNet on 2 million Instagram selfies to learn what makes a selfie 'good' — using likes-per-follower as the quality signal. Beyond the entertainment value, it's a sharp demonstration of how supervised learning can proxy for human aesthetic judgment at scale.
That Time When "Recurrent Neural Network" Was Replaced with "Dog"
The joke where replacing 'recurrent neural network' with 'dog' in ML papers produces sentences that remain plausible — a 2015 meme that captured real frustration with AI hype language. Still a useful test for vague ML claims.
Visualizing Representations: Deep Learning and Human Beings
Christopher Olah's essay on visualizing what neural networks actually learn — using dimensionality reduction to show how deep networks transform data into progressively more separable representations. One of the most important early pieces on deep learning interpretability.
Zipfian Academy Partners with Skymind to Teach Deep Learning
Zipfian Academy announces a partnership with Skymind to add deep learning to its data science curriculum in September 2014 — one of the first bootcamps to formally incorporate neural network training. Reflects the early excitement around deep learning reaching commercial viability.
10 Tips for Better Deep Learning Models
Laura Diane Hamilton's ten practical tips for improving deep learning model performance, covering data preparation, architecture choices, regularization, and training tricks. A snapshot of practitioner wisdom circa 2014, before the era of giant pretrained models made many of these tradeoffs less urgent.
The Mission to Bring Google's AI to the Rest of the World
Wired profiles Skymind, the startup founded by Adam Gibson to bring deep learning to enterprise Java shops — featuring Zipfian Academy's role in democratizing the field. A snapshot of 2014's optimism about deep learning accessibility before TensorFlow existed.
Google Turns to Machine Learning to Build a Better Data Centre
A 2014 report on Google applying machine learning to optimize data center cooling — one of the first public disclosures that Google was using neural networks to automate infrastructure decisions. DeepMind later published the full methodology in 2016.
ConvNetJS Demo: Classify Toy 2D Data
Andrej Karpathy's ConvNetJS demo for classifying 2D toy datasets — a real-time browser visualization of a neural network training on user-drawn boundaries. One of the first compelling interactive deep learning visualizations.
Introduction to Deep Learning on Hadoop (Hadoop Summit 2014)
A Hadoop Summit 2014 session proposal on deep learning at Hadoop scale — from the team behind DL4J (DeepLearning4J), a Java-native deep learning framework designed to run on Hadoop/Spark clusters. A snapshot of the moment distributed deep learning was being invented.
NIPS 2012 Paper #1338
A NIPS 2012 (NeurIPS 2012) conference paper — saved in the same timeframe as deep learning and distributed neural network research. Likely related to neural architecture or distributed/large-scale training given the surrounding bookmarks.
Multilayer Perceptron with Jobman
Deeplearning.net tutorial on training a multilayer perceptron using Jobman — a job scheduling system from the Montreal LISA Lab for running batches of hyperparameter experiments. Shows the infrastructure behind systematic deep learning research in 2014.
Computer Science: The Learning Machines
Nature News's January 2014 feature on machine learning and deep neural networks going mainstream — the moment the field started reaching a broader scientific audience. Documents the Hinton-LeCun-Bengio wave before it was inevitable.
ConvNetJS: Deep Learning in Your Browser
Andrej Karpathy's JavaScript library for training neural networks entirely in the browser — no install, no GPU, instant demos. In 2014 it made deep learning tangible for anyone with a web browser.
UFLDL Tutorial — Stanford
Andrew Ng's Stanford UFLDL Tutorial — the primary self-study resource for deep learning before MOOCs existed. Teaches sparse autoencoders, PCA, CNNs, and deep belief networks with mandatory from-scratch implementation exercises.
Regularizing Neural Networks with Dropout and DropConnect
FastML's comparison of Dropout and DropConnect — two techniques for regularizing neural networks by randomly zeroing activations or weights during training. Clarifies that DropConnect's CIFAR-10 SOTA came from model ensembling, not the technique itself.
Deep Learning Tutorials — DeepLearning.net
The canonical deep learning tutorial site from the Montreal group — code-first walkthrough of core architectures (RBM, DBN, CNN, LSTM) using Theano. Written by the lab around Yoshua Bengio and became the standard reference before fast.ai existed.
A Not-So-Basic Neural Network in Python
Daniel Rodriguez's practical tutorial on implementing a non-trivial neural network in Python from scratch — going beyond the toy perceptron examples to show backpropagation and training on real data. A mid-2013 hands-on coding reference.
Deep Learning 101
Introductory survey of deep learning for non-specialists — covers hierarchical representation learning, RBMs, autoencoders, and the four core obstacles. Written to help readers mentally filter hype from substance in 2013.
Deep Support Vector Machines
A video lecture on Deep Support Vector Machines from ROKS 2013 — hybrid architectures combining deep feature learning with SVM classification. A snapshot of the moment researchers explored whether SVMs and deep learning could coexist before end-to-end networks won out.
Variable Importance in Neural Networks
R-bloggers post on measuring variable importance in neural networks — techniques like the Garson algorithm and Olden's method for attributing prediction contributions to input features. An early attempt at neural network interpretability before SHAP, LIME, and modern explainability tools existed.