Ai2 says new MolmoAct 7B model brings AI into the physical world - The Robot Report

The Allen Institute for AI (Ai2) has introduced MolmoAct 7B, an embodied AI model designed to bring advanced artificial intelligence into the physical world by enabling robots to perceive and interact with their surroundings more intelligently. Unlike traditional models that convert language instructions directly into movements, MolmoAct processes 2D visual inputs to generate 3D spatial plans, allowing robots to understand spatial relationships and plan actions accordingly. This model emphasizes transparency, safety, and adaptability, providing step-by-step visual reasoning that lets users monitor and adjust robot behavior in real time. Ai2 describes MolmoAct as an “action reasoning model” (ARM) that interprets high-level natural language commands and breaks them down into a sequence of spatially grounded decisions, enabling complex tasks like sorting objects to be executed as structured sub-tasks. MolmoAct 7B was trained on an open dataset of approximately 12,000 robot episodes captured in real-world household environments, such as kitchens and bedrooms, showcasing diverse tasks