MIT and Nvidia researchers develop AI to reduce robot latency

Researchers from MIT, Nvidia, Caltech, and other leading universities have introduced a new AI method called VLASH (Vision-Language-Action with Scheduled Heuristics) to improve robotic efficiency. This system addresses a major limitation in existing vision-language-action (VLA) models, where robots typically must wait for AI to calculate the next set of commands after completing a previous action.

VLASH allows robots to perform tasks while simultaneously calculating the next steps. Instead of attempting to predict the entire future environment, the system estimates the future state of the robot itself based on its current position and planned actions. This approach has reduced response latency by up to 17.4 times and doubled the speed of certain tasks.

In practical tests involving grasping, stacking, and sorting, robots using VLASH completed color-based cube sorting twice as fast as control methods with approximately 90% accuracy. The technology was also successfully tested in high-speed reactive tasks, such as table tennis and Whac-a-Mole, where targets change during execution. Additionally, the researchers improved the training process, reducing training step time from 421 to 129 milliseconds.