Generalist AI unveils GEN-1.5 model for rapid robot learning

Generalist AI has introduced GEN-1.5, a new robotic foundation model capable of learning physical tasks through “physical prompting.” Unlike traditional robotics that requires extensive retraining or fine-tuning for every new task, GEN-1.5 can observe a single demonstration lasting between 3 and 12 seconds and immediately attempt to replicate the action.

The model processes multimodal data, including video, language, and sensorimotor information, to understand the physics of the environment. In testing, the robot demonstrated an ability to improvise; for example, if a specific tool was missing, the robot could use alternative objects to complete a task, such as using a dustpan to sweep. In one instance, a robot successfully unzipped a purse and retrieved banknotes after watching a short video, even switching grippers to achieve a better angle.

CEO Pete Florence noted that the approach is inspired by the success of large language models like GPT-3, where a model can be prompted to perform new tasks without structural changes. While the initial success rate for a single demonstration was 59%, increasing the task-specific data to five minutes raised the success rate to 83%.