Skip to main content
Ryan Orban

Ryan Orban

Subject
7 entries

Training

Bookmarks

  1. Building a Deep Learning Rig (Part 2)

    Part 2 of building a home deep learning rig: upgrading to a Threadripper 1920X to support three RTX 3090s with full PCIe bandwidth. Total cost €2,379. Key lesson: GPU peer-to-peer DDP failures were fixed by downgrading the NVIDIA driver, not hardware changes.

  2. OpenLLaMA: Open Reproduction of LLaMA

    OpenLLaMA is an open-source reproduction of Meta's LLaMA model trained on the RedPajama dataset, released under permissive Apache 2.0 licenses. It was one of the first serious attempts to produce a fully open LLaMA-quality model that anyone could use commercially.

  3. Colossal-AI: Open-Source ChatGPT Training Replication

    Colossal-AI released an open-source implementation of the ChatGPT training process (SFT + RLHF) that runs on a single GPU with 1.6GB memory — 7.73x faster than naive implementations. Made the ChatGPT training pipeline accessible to researchers without multi-GPU clusters.

  4. Google Research Deep Learning Tuning Playbook

    Google Research's Deep Learning Tuning Playbook — a systematic guide to maximizing model performance through hyperparameter optimization. Written by Braxton Osting and team, it covers the science and art of tuning learning rates, batch sizes, regularization, and the full training configuration.

  5. A Hitchhiker's Guide to Distributed Training of Deep Neural Networks

    Chahal, Grover, and Dey survey the algorithms and engineering techniques for distributed deep learning training, covering data parallelism, model parallelism, AllReduce strategies, gradient compression, and mixed precision. A practical reference for scaling training from single GPU to multi-node clusters.

  6. Cloudera Developer Class Links

    Cloudera's developer training class resource links page — a collection of documentation, tutorials, and reference materials for the Cloudera Hadoop Developer certification course. Saved as a training reference during the 2013 era of Hadoop skills development.

  7. Bay Area Motorcycle Training — San Francisco Calendar

    Bay Area Motorcycle Training's San Francisco course calendar — bookmarked for their MSF Basic RiderCourse. A personal reference bookmark for motorcycle license preparation.

All bookmarks