Google faked its most impressive Gemini demo

Google had barely finished taking a victory lap for its new Gemini multimodal model when the internet started picking apart its shiny debut.

The centerpiece of the announcement was an incredible video titled “Hands-on with Gemini: Interacting with multimodal AI.” It quickly racked up over a million views by showing what looked like the holy grail of AI: a model that watches real-time video, listens to human voice, comments on hand-drawn duck sketches, and even plays rock-paper-scissors without skipping a beat.

Except, the interaction you saw never actually happened.

The reality behind the curtain

Bloomberg’s Parmy Olson first flagged the gap between the video and reality, and TechCrunch promptly labeled the demo for what it was: faked.

In the video, Gemini feels like a responsive, intuitive companion reacting live to the camera. But Google’s own developer documentation tells a completely different story. There was no real-time video feed and no conversational voice input. Instead, researchers took still image frames from the footage and fed them into Gemini using carefully engineered text prompts.

Take the rock-paper-scissors sequence at the 2:45 mark. The on-screen actor makes three silent hand gestures, and Gemini immediately chirps that it knows the game.

In reality, Gemini couldn't infer the game from watching individual moves. Researchers had to place all three gestures into a single static image and prompt it with: “What do you think I’m doing? Hint: It’s a game.”

It was the same story with the planetary order test. The video makes it seem like the user simply asked “Is this the correct order?” In the actual written prompt, Google explicitly told Gemini to “Consider the distance from the sun and explain your reasoning.” In the cup-and-ball tracking game, the model had to be pre-trained and instructed on the rules via text beforehand.

Google also never clarified whether the demo was running on Gemini Pro, which launched that day, or Gemini Ultra, the unreleased flagship not arriving until next year.

Damage control from DeepMind

Google pushed back on the backlash, claiming the outputs shown were genuine and that it had merely reduced latency and shortened responses for time. A spokesperson even asked TechCrunch to remove the word “faked” from its headline.

Oriol Vinyals, Vice President of Research at Google DeepMind, took to social media to clarify that the video was meant to illustrate what user experiences built with Gemini “could look like,” designed primarily to “inspire developers.” He also shared background prompts showing that Gemini could handle the planet question if primed with an expert persona beforehand.

Still, calling a video “Hands-on” when it depicts an aspirational, heavily edited workflow crossed a line for many observers.

Why it matters

We think this demo misstep speaks volumes about the immense pressure Google feels to match OpenAI’s momentum. Demonstrating genuine multimodal capability by recognizing sketches and gestures from static images is already impressive engineering. Dressing it up as a seamless, telepathic live interaction, however, needlessly undermined that real progress.

For developers and everyday creators, the lesson is clear: don't expect Gemini to act as a turnkey, real-time visual assistant just yet. You can still get meaningful utility out of Gemini Pro for multimodal tasks, but expect to write detailed, structured prompts and feed it static images rather than relying on intuitive conversational banter.

Within the industry, Google ended up hurting its own credibility right when it needed it most. OpenAI’s GPT-4V may be slower and less cinematic, but developers know what they are actually getting. By over-promising on the user experience, Google didn’t just set unrealistic benchmarks for its tech; it practically handed its rivals an easy critique on enterprise trust.