How Apple squeezes big LLMs onto edge devices
Apple just detailed how it plans to fit heavy AI models onto everyday devices without blowing past hardware limits. In a paper titled 'LLM in a flash', researchers store parameters on flash memory and stream them to DRAM on demand, running models twice the size of available RAM while speeding up GPU inference by up to 25x. Why it matters: It lays the technical groundwork for reliable on-device generative AI.