An all-optical AI chip outruns Nvidia by 100x

Generating a batch of AI images consumes way more power than most people realize. In fact, a study found that generating just 1,000 images churns out carbon emissions equivalent to driving a gas-powered car for over four miles.

Now, researchers from Shanghai Jiao Tong University and Tsinghua University have built a fully optical generative AI chip that could blow standard silicon out of the water. Dubbed LightGen, the new chip was recently detailed in Science and claims to run generative video and image tasks over 100 times faster and with far greater energy efficiency than Nvidia's powerhouse A100 GPU.

Packing millions of neurons into a speck of light

Photonic computing uses light instead of electricity to process data, offering insane speed boosts and virtually zero electrical resistance. But until now, photonic chips were mostly stuck running simple tasks like basic text generation or image classification.

The real blocker has been density. Generative AI needs millions of parameters, yet prior optical designs could only cram a few thousand artificial neurons onto a chip.

The LightGen team tackled this bottleneck using 3D packaging. They squeezed more than two million artificial neurons onto a tiny device measuring just a quarter of a square inch.

That massive bump in density lets the chip process full 512-by-512 pixel images natively. Older optical systems had to slice high-res images into tiny patches, which tanked processing speeds and broke the subtle statistical connections across the image.

Brewing up an optical latent space

Generative models typically rely on a latent space, which is a mathematical map of compressed data that helps the system understand what it is looking at before dreaming up something new.

LightGen actually pulls this off completely through optics. An optical encoder built from specialized metasurfaces channels the image into an array of optical fibers, naturally filtering out irrelevant data and locking the compressed representation right into the fiber array.

On the other end, another set of swappable metasurfaces reads that optical latent space and pumps out a brand new, high-res image. The researchers even introduced a new training technique using probabilistic representations, allowing the hardware to master novel creation rather than just running pre-baked inferences.

In testing, the chip handled high-res animal generation, artistic style transfers, and even turned 2D images into 3D models. The catch is that it is strictly confined to the lab for now, relying on bulky external lasers, spatial light modulators, and bespoke fabrication methods that standard semiconductor fabs cannot currently produce.

Why it matters

We think LightGen represents a massive conceptual leap for next-generation hardware. It proves that all-optical architectures can move beyond lightweight classifiers and handle the beast that is modern generative AI.

The clever move here is integrating 3D packaging with an optical latent space. Instead of forcing optical data to constantly convert back into sluggish electrical signals, the researchers kept the pipeline almost entirely within light waves.

However, the engineering hurdles are massive. While the chip itself is smaller than a postage stamp, the external hardware required to power and feed it signals is bulky and impractical. Furthermore, relying on specialized metasurface fabrication means commercial scale is still years away.

For everyday developers and businesses, an A100 or H100 in the cloud remains your only practical choice today. But as the energy grid buckles under the weight of generative video and heavy inference workloads, LightGen shows what the post-silicon future might actually look like.