Subject
9 entries
Pytorch
Bookmarks
Heinsen Routing: Vector Routing for Sequences and Capsule Networks
Heinsen routing is a novel algorithm for routing vectors in sequences and capsule networks — an alternative to Hinton's dynamic routing that generalizes across domains. A niche but foundational contribution to the capsule network research line.
How Hugging Face Accelerate Runs Very Large Models
Hugging Face's technical guide to running very large models using Accelerate — covers device_map, model parallelism across GPUs, CPU offloading, and the mechanics of loading models that don't fit in a single GPU's VRAM. Essential reading for anyone self-hosting large LLMs.
Ivy: The Unified Machine Learning Framework
Ivy is a unified machine learning framework that transpiles code between NumPy, PyTorch, TensorFlow, and JAX — write once, run anywhere across ML backends. An ambitious attempt to solve the framework fragmentation problem that plagues ML research and production.
The Annotated Transformer
The Annotated Transformer walks through the 'Attention Is All You Need' paper with working PyTorch code alongside every equation — the canonical resource for understanding transformer architecture from first principles. Published by Harvard NLP.
Colossal-AI: Distributed Deep Learning System
Colossal-AI is an open-source distributed deep learning framework that makes training very large models more accessible — cutting GPU memory requirements by up to 10x versus standard PyTorch. One of several systems research projects responding to the GPU memory wall problem in 2022.
CrypTen: Secure Multi-Party Computation Meets Machine Learning
CrypTen is a PyTorch-compatible framework from Facebook AI Research that wraps secure multi-party computation protocols behind a familiar tensor API, making private inference and training accessible to ML practitioners without cryptography expertise. The bet is that adoption bottlenecks for privacy-preserving ML are mostly about developer experience, not theoretical limits.
Deep Learning with PyTorch
Manning's 2020 practical guide to deep learning with PyTorch by Stevens, Antiga, and Viehmann, covering tensors through CNNs, RNNs, generative models, and production deployment. The go-to book for practitioners who want to understand PyTorch from first principles rather than copy-paste patterns.
LabML Neural Networks: Annotated Implementations
LabML Neural Networks is a collection of PyTorch implementations of ML papers with line-by-line annotations — making research papers readable by walking through the actual code. Covers transformers, diffusion models, GANs, and RL algorithms side-by-side with the paper math.
Explicit Recommender System — Matrix Factorization in PyTorch
A tutorial implementing explicit recommender systems via matrix factorization in PyTorch — using Embedding layers for user and item factors, trained with alternating gradient descent. Concrete implementation of collaborative filtering fundamentals.