Gossip

Asynchronous Decentralized SGD with Quantized and Local Updates

SwarmSGD is an asynchronous decentralized optimization algorithm that provably converges when combining gossip communication, gradient quantization, and local update steps simultaneously across heterogeneous data distributions. It achieves performance comparable to large-batch SGD on supercomputing systems while reducing communication overhead.