Inside Google DeepMind's Robotics Lab: Where AI Learns to Pack Your Lunch

December 21 2025 08:11
Back in 2021, researchers were still using privacy screens to control lighting conditions. The robots needed carefully managed environments just to function. This time, walking through the California facility with mathematician Hannah Fry and robotics director Kasch Karo, those screens were gone. The robots can now work in open labs with natural lighting and varied backgrounds.

This small detail reveals something bigger. In just four years, robotics has fundamentally changed. Not through incremental improvements, but through a complete reimagining of how robots learn and operate.


Forget the backflipping robots you've seen in viral videos. Those are impressive engineering feats, but they're fundamentally limited. Every movement is pre-programmed, choreographed in advance by human engineers. Google DeepMind is working on something different: robots that understand instructions and adapt flexibly to unlimited tasks.

The breakthrough comes from building robotics on top of large vision language models. These AI systems, which have transformed how computers understand images and text, turn out to have an excellent grasp of general world concepts. When you integrate that understanding into a physical robot, something remarkable happens. The robot doesn't just follow commands. It comprehends them.

Karo explained the technical innovation behind this. DeepMind developed VLAs, or vision language and action models. These systems treat physical actions the same way they treat vision and language tokens. The robot can model sequences of actions and figure out what to do in new situations. They call this action generalization, and the improvements have been massive.


The architecture works like a stack. At the foundation sits a large multimodal model that understands the world. On top of that, DeepMind has built additional layers that chain sequences of actions together to accomplish complex, long-horizon tasks.

The latest version, RT-2 1.5, includes two new capabilities: an agent component and a thinking component. The agent orchestrates smaller moves into longer sequences. Instead of just picking up an object and placing it somewhere else, the robot can now handle tasks like packing luggage for a trip. It can look up the weather at your destination, decide what clothes you need, and pack your bag accordingly.

The thinking component does exactly what it sounds like. Before taking an action, the robot outputs its thoughts. This might sound like a gimmick, but it works. The same technique improves performance in language models. Tell a language model to "take a deep breath" before answering, and you often get better results. Chain of thought prompting helps these systems reason through problems step by step.

The same principle applies to physical actions. Making the robot articulate its reasoning before moving improves both generalization and performance. For robotics, where basic manipulation tasks remain genuinely difficult, this thinking step helps the robot succeed more often.


In the lab, an Aloha robot attempt to pack a lunchbox. This demonstration showcases dexterity at its most challenging. The robot needs millimeter-level precision to grab a ziplock bag correctly, then carefully place a sandwich inside without crushing it.

The robot uses only visual servoing. No special sensors, no pre-programmed movements. Just cameras and the ability to understand what it sees. When it succeeded in opening the ziplock bag and placing items inside, the achievement was genuinely impressive. When it crushed a grape while trying to pack it, that failure was equally instructive. These robots are learning the same way humans do: through trial and error, guided by demonstration.

The training method for the Aloha robots relies on teleoperation. Human operators embody the robot and perform tasks while the robot learns from that perspective. It's not running millions of simulations or randomly trying actions. It's learning from human demonstration, building an understanding of how to manipulate objects through observation.

This approach has limitations. It requires substantial amounts of real-world robot data, which is expensive and time-consuming to collect. But it produces robots that can handle the nuanced, difficult manipulations that make up everyday tasks.


The real test of generalization came when Fry pulled out a stress ball she travels with. The robot had never seen this object before. The researchers had never included it in training data. Fry placed it next to a small container with a lid and gave the robot a series of commands: open the lid, place the pink blob inside, put the lid back on.

The robot succeeded. It correctly identified the stress ball as "the pink blob" and the container as "the green pear" based on visual similarity. It manipulated objects it had never encountered, completing a task it had never practiced. This demonstrates genuine generalization, not memorization.

The system combines RT-2's vision-language-action model with a Gemini layer on top that enables natural conversation. You can speak to the robot, and it responds while performing tasks. When Fry asked it to move a block "as Batman would," the robot politely declined to impersonate characters but completed the task anyway. This interaction reveals both capabilities and limitations. The robot understands context and can parse natural language requests, but it also maintains boundaries programmed into the system.


Individual actions matter less than sequences. A robot that can only pick up objects and put them down has limited usefulness. A robot that can break down a high-level goal into component steps and execute them in order becomes genuinely helpful.

DeepMind demonstrated this with a trash-sorting task. Fry asked the robot to look up San Francisco's waste sorting rules and then tidy up. The system used two models working together. A vision-language model handled the reasoning, searching for information and breaking down the task. The VLA executed the physical actions, sorting items into appropriate bins.

The robot explained the rules (recyclables, compostables, and trash go in separate color-coded bins), then proceeded to sort various items. It made mistakes. It hesitated. But it understood the overall goal and worked through the steps needed to accomplish it.

This architecture, with a reasoning model orchestrating a physical action model, represents a significant advance. Previous versions of these robots could only respond to step-by-step instructions. Now they can handle abstract goals and figure out the steps themselves.


The humanoid robot lab showcased another advance: transparency in decision-making. A humanoid robot sorted laundry, separating dark and light clothes. As it worked, a screen displayed its thoughts at each timestep.

"Red cloth. Do not put in white bin. Put in dark bin."

This thinking-and-acting model outputs both reasoning and actions in the same end-to-end system. There's no hierarchy, no separate planning module. The robot thinks and acts in a tightly coupled loop, similar to how Gemini outputs reasoning before generating a response.

This transparency serves multiple purposes. It helps researchers understand failures and improve the system. It makes the robot's behavior more predictable and trustworthy for users. And crucially, it improves performance through the same mechanisms that help language models.

The humanoid also demonstrated generalization with objects purchased the day before the demonstration. A plant, a bag of Doritos, various household items the robot had never encountered in training. It successfully manipulated most of them, though not without struggles and occasional failures.


Watching these robots work, you notice two things. First, they're slow. Second, they don't succeed 100% of the time. But you can see the intention behind their actions. They're genuinely trying to accomplish the tasks you give them, using understanding rather than pre-programmed routines.

Karo believes these are foundational blocks that will lead to general-purpose robotics. But he also thinks at least one more major breakthrough is needed. The current systems require enormous amounts of data to learn tasks. Robots need to learn more efficiently.

One hypothesis suggests that if we could collect robot data at the scale of internet text data, we'd solve the problem. Language models benefit from vast corpora of text. Robot models need vast corpora of physical interaction data. But that data doesn't exist at anything close to the required scale. Physical interactions aren't documented and shared the way text and images are.

The long-term solution might come from learning from human video data. Humans post millions of videos showing how to do everything imaginable. If robots could learn from that unstructured demonstration data, the available training corpus would expand dramatically. This remains an unsolved research problem, but it represents a clear path forward.


There's a tendency in technology coverage to either dismiss incremental progress as boring or hype modest advances as revolutionary. The truth about DeepMind's robotics work falls somewhere in between.

These robots are clumsy. They're slow. They make mistakes. They can't do most of the things a human child can do without thinking. But the semantic understanding, the contextual awareness, the ability to reason through complex tasks—this was inconceivable just a few years ago.

The progress here is real and it's limited primarily by data availability. The technical approaches work. The models generalize. The architecture scales. What's missing is enough examples of physical interactions in the real world to train these systems to human-level competence.

That's a solvable problem. It's expensive and time-consuming, but it's not a fundamental barrier. Once that data exists, or once researchers figure out how to learn efficiently from existing human video, the capabilities should improve rapidly.


We're not on the verge of robot butlers. We're not about to see humanoid robots in every home. The current systems are too slow, too unreliable, and too expensive for consumer deployment.

But we might be on the cusp of something significant. The fundamental approach—building robotic systems on top of large multimodal models—appears to work. The robots genuinely understand what they're seeing and what they're being asked to do. They can generalize to new objects and new tasks without specific training.

The remaining challenges are engineering problems, not insurmountable conceptual barriers. Make the robots faster. Improve their success rates. Collect more training data. Figure out how to learn from human video. None of these are easy, but they're all tractable.