stalin-grad

A centrally planned distributed SGD framework for CIFAR-10. A hand-rolled parameter server, the Politburo, coordinates workers over two competing transports: TCP, the bourgeois baseline, and shared memory, the people's transport.

• Trains a 1.34M parameter CNN across N workers on one GPU, moving about 5.36 MB of gradients each way per round.
• Five experiments on scalability, transport latency, convergence, fault tolerance and batch size. The Central Committee guarantees results.