Bloom

BLOOM Training Corpus: 2D Embedding Visualization

A 2D UMAP visualization of 10 million text chunks from the BLOOM training corpus, encoded with all-distilroberta-v1. A rare window into the geometry of a frontier model's pretraining data.