Deepmind on Ryan Orban

In-Context Reinforcement Learning with Algorithm Distillation

me@ryanorban.com (Ryan Orban) — Thu, 27 Oct 2022 00:00:00 +0000

Summary

Algorithm Distillation (AD), from DeepMind (Laskin et al., 2022), is a method for distilling entire reinforcement learning algorithms into a neural network by training it on sequences of RL learning histories. The core idea: if you expose a causal transformer to many episodes of an RL agent improving at a task, the transformer learns to predict the improving agent’s behavior — and this learned prediction generalizes to new tasks without any gradient updates.

Building Safer Dialogue Agents (DeepMind / Sparrow)

me@ryanorban.com (Ryan Orban) — Fri, 23 Sep 2022 00:00:00 +0000

Summary

DeepMind’s post describes Sparrow, their research dialogue agent designed to be safe, helpful, and accurate. Published in September 2022, it’s one of the first detailed public accounts of using RLHF (reinforcement learning from human feedback) combined with explicit rules to train conversational AI that avoids harmful outputs.

Introduction to Graph Neural Networks with JAX/jraph

me@ryanorban.com (Ryan Orban) — Thu, 04 Aug 2022 00:00:00 +0000

Summary

This Google Colab notebook from DeepMind’s educational materials is an interactive introduction to Graph Neural Networks (GNNs) using JAX and jraph — DeepMind’s graph neural network library built on JAX. The notebook covers the core GNN paradigm: message passing, where nodes aggregate information from neighbors iteratively, with the aggregated representations used for downstream tasks like node classification, edge prediction, or graph-level classification.

A Generalist Agent

me@ryanorban.com (Ryan Orban) — Fri, 13 May 2022 00:00:00 +0000

Summary

This DeepMind paper by Scott Reed, Konrad Zolna, Emilio Parisotto, and 17 co-authors including Nando de Freitas presents Gato, a single transformer model that operates as a generalist policy across over 600 diverse tasks. Gato uses a unified tokenization scheme to represent text, images, discrete actions, and continuous control signals (robot joint torques) as a flat sequence, then applies a standard autoregressive decoder to predict the next token. The same network weights handle Atari gameplay, real robotic arm manipulation, image captioning, question answering, and dialogue — selecting output type based on context.

Flamingo: A Visual Language Model for Few-Shot Learning

me@ryanorban.com (Ryan Orban) — Mon, 09 May 2022 00:00:00 +0000

Summary

Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, and colleagues at DeepMind introduce Flamingo, a family of Visual Language Models (VLMs) that achieve strong few-shot performance on vision-language tasks by efficiently bridging a frozen vision encoder with a frozen large language model through lightweight cross-attention layers. The core insight is that separately pre-trained vision and language experts can be kept frozen — protecting their learned representations — while only the connecting layers are trained on interleaved image-text data.