Boston Dynamics' Spot cleans the house with Gemini

Watching an industrial robot dog meant for power plants pick up sneakers in a messy living room feels surreal.

Boston Dynamics showcased its quadruped robot Spot tackling light home chores, from sorting shoes onto a rack to picking up discarded soda cans. What made the demo turn heads wasn't just the dexterity, but the brain behind it: Google's Gemini Robotics-ER 1.5 visual-language model, running on natural language prompts instead of rigid state-machine code.

Replacing hardcoded logic with a conversational brain

The project originated during a 2025 internal hackathon at Boston Dynamics. Traditionally, programming an autonomous robot for a multi-step task meant mapping out extensive conditional logic. If the robot encountered an unexpected shoe placement or an odd angle, the whole workflow would fail.

Instead, engineers built a lightweight SDK translation layer between Gemini Robotics and Spot's existing APIs. They gave the model a restricted set of basic tools: navigate to a spot, capture images, identify objects, grasp, and place.

Basically, Gemini Robotics acts as a virtual human operator holding Spot's tablet controller. Humans give high-level goals on a whiteboard, like tidying the entryway. The model looks through Spot's cameras, figures out where the shoes are, calls the navigation API, and triggers the arm's grip routines. Spot's onboard software still handles all the heavy lifting of balance, locomotion, and kinematics.

The team noted that prompt engineering mattered immensely. They had to teach the model practical rules, such as always checking the arm gripper camera first upon arrival, and avoiding the low-slung front cameras when trying to place items on elevated shelves.

Guardrails and two-way feedback

The integration works as a two-way dialogue. When Gemini triggers an action, the tool sends back plain-text feedback. If Spot is already clutching a can, the system replies that its gripper is full, prompting the AI to find a trash bin before trying to grab another object.

Crucially, Gemini is kept on a tight leash. It cannot hallucinate hardware movements or bypass Spot's safety stack; it can only request predefined API calls. That keeps behavior predictable while granting enough autonomy to adapt to unscripted messes.

While the living room cleanup was an experimental showcase, Boston Dynamics noted that production applications are already rolling out. The company recently released AIVI-Learning for Spot and its Orbit fleet software, running on Google's newer Gemini Robotics ER 1.6 model for industrial visual inspection.

Why it matters

We think this demo highlights the most practical path forward for physical AI right now.

The robotics world has been fiercely debating whether to discard traditional control theory in favor of end-to-end neural networks. Boston Dynamics took a smarter, pragmatic shortcut. Instead of reinventing the wheel, they kept their world-class locomotion and manipulation platform completely intact, grafting frontier vision models onto the top layer purely for semantic reasoning and task planning.

For enterprise developers, this approach completely reshapes deployment economics. Setting up specialized inspection or retrieval tasks no longer requires weeks of hardcoding finite state machines. You write clear instructions, provide modular tool APIs, and let the multimodal model orchestrate the steps dynamically.

The catch? Latency and reliability. Handing real-time sequencing over to an external foundation model inevitably introduces delays and potential logic misfires, making it unviable for mission-critical millisecond reactions. Still, as a bridge between rigid industrial hardware and intuitive human instructions, this layered architecture looks like the real deal.