UCL researchers reconstruct video clips from mouse brain activity

For years, reconstructing what a subject sees meant cramming them into a loud, clunky fMRI machine and accepting blurry, abstract shapes generated from coarse blood-flow readings.

Now researchers have taken visual decoding down to the individual cell level. On March 10, a team led by Dr. Joel Bauer at UCL's Sainsbury Wellcome Centre published a paper in eLife showing they could reconstruct 10-second movie clips watched by mice solely from single-neuron activity in the visual cortex.

Reading the brain cell by cell

Instead of relying on fMRI scans that lump millions of neurons together, the UCL team turned to microscopic imaging that detects calcium level spikes to track individual brain cells firing in real time.

They started with a dynamic neural encoding model originally developed for the 2023 Sensorium Competition, which predicts neural responses to video frames. Crucially, the model also factored in the animal's physical state, tracking movement and pupil diameter. In real life, perception is never passive; whether a mouse is running or alert profoundly alters how its brain processes incoming light.

To pull movies out of brain activity, the researchers set up an iterative decoding process. They started with a blank canvas, calculated the difference between predicted neural activity for a blank screen and actual recorded cell activity, and used an algorithm to tweak pixels until the predicted neural response matched reality.

Once trained, the system decoded a 10-second video clip that the animal watched for the very first time. Pixel correlation showed the timing of actions in the reconstructed video matched the original with minimal drift. Better yet, fidelity scaled directly with the number of tracked neurons: the more single cells you listen to, the crisper the playback gets.

Dr. Bauer noted that the discrepancies between the original and reconstructed footage aren't necessarily algorithmic glitches. The brain's visual pipeline naturally skews and filters sensory input, meaning deviations from ground truth might reveal how visual reality is reshaped by biology.

Why it matters

This is a major technical benchmark for neural decoding. It shifts the state of the art away from fuzzy regional blood-flow maps toward cellular-precision movie playback, showing that modern deep learning models can reverse-engineer real-time visual streams directly from wetware.

The clever move here is treating the brain as an embodied system rather than an isolated camera sensor. By including pupil dilation and physical movement, the researchers eliminated noise that typically confounds vision decoders. Furthermore, optimizing pixels against a validated biological forward model avoids relying purely on generative AI hallucinations to fill in the blanks.

Don't expect commercial mind-reading gadgets anytime soon, though. The technique relies on invasive optical calcium imaging that cannot be casually transferred to humans. Resolution is still constrained, and 10 seconds remains the ceiling for now.

In our view, the immediate impact won't be consumer brain-computer interfaces, but computational neuroscience. By measuring exactly where the reconstructed video diverges from physical reality, researchers finally have a tool to quantify how sensory reality is modified, filtered, and augmented as it travels through biological circuits.