PaLM-E brings 562B multimodal power to real-world robots
Researchers just introduced PaLM-E, an embodied multimodal language model that injects real-world sensor data directly into an LLM's embedding space. Scaling up to 562B parameters, it turns high-level prompts like "bring me the rice chips" into multi-step physical manipulation plans, even generalizing to objects the robot has never encountered before.