Edge0 runs 35B models in 2.5 GB of RAM by streaming from SSD
Edge0 just open-sourced a streaming MoE inference framework that treats storage like memory, running 35B models in roughly 2.5 GB of active RAM on iPhones and Macs. By predicting routing one step ahead to hide SSD read latency, it squeezes large models onto low-memory consumer devices without crushing performance.