Agility Robotics explains how to train a whole-body control foundation model - The Robot Report

Agility Robotics has developed a whole-body control foundation model for its Digit humanoid robot, designed to enable safe, stable, and versatile task execution in complex, human-centric environments. This model acts like a "motor cortex," integrating signals from different control layers to manage voluntary movements and fine motor skills. It is implemented as a relatively small LSTM neural network with fewer than one million parameters, trained extensively in NVIDIA’s Isaac Sim physics simulator. Remarkably, the model transfers directly from simulation to the real world without additional training, allowing Digit to perform tasks such as walking, grasping, and manipulating heavy objects with high precision and robustness to disturbances. The model can be prompted using various inputs, including dense spatial objectives and large language models, enabling Digit to execute complex behaviors like grocery shopping demonstrated at NVIDIA’s GTC event. Agility Robotics aims to provide an intuitive interface for humanoid robots similar to fixed-base robots, where users specify desired end-effector poses and the robot autonomously positions itself accordingly.