Rnns

Attention and Augmented Recurrent Neural Networks (Distill)

Distill's 2016 interactive visual explainer covering four augmented RNN variants — attention, memory networks, neural Turing machines, and neural programmers. A foundational reference for understanding how external memory and attention were grafted onto recurrent networks before the transformer era.