Subject
5 entries
Transformer
Bookmarks
LLM Visualization: Interactive 3D Transformer Walkthrough
An interactive 3D visualization of how LLMs work — walking through the transformer architecture token by token, layer by layer, with actual weight animations. The clearest visual explanation of attention, embeddings, and feedforward layers available.
Large Transformer Model Inference Optimization
Lilian Weng's comprehensive survey of transformer inference optimization techniques — covering quantization, distillation, pruning, efficient attention, speculative decoding, and hardware-level optimizations. The definitive reference for the topic as it stood in early 2023.
Flash Attention: Training Examples
Flash Attention's training examples from Hazy Research — demonstrating how to use the memory-efficient attention implementation for training large transformers. Flash Attention became one of the most impactful algorithmic contributions to LLM training efficiency.
The Annotated Transformer
The Annotated Transformer walks through the 'Attention Is All You Need' paper with working PyTorch code alongside every equation — the canonical resource for understanding transformer architecture from first principles. Published by Harvard NLP.
Transformer Models: An Introduction and Catalog
Xavier Amatriain's catalog of transformer models covers the full landscape of architecture variants and named models as of mid-2022 — encoder-only, decoder-only, encoder-decoder, and multimodal. A useful orientation map for the pre-ChatGPT era of rapid model proliferation.