ToyBlocks: A New Open Simulation Framework for Robot Learning and Evaluation

Introduction of FoundationWorlds

A new framework called FoundationWorlds has been introduced to create controlled environments for robotics research. These environments pair a simulated world with tools for automatic task and demonstration generation, along with metrics for measuring robot behavior. The goal is to enable systematic investigation into how robots learn and why they fail.

ToyBlocks: The First FoundationWorld

ToyBlocks is the initial implementation of a FoundationWorld. It is an open simulation framework centered around a robot arm manipulating blocks on a tabletop. Despite its simple setting, the robot must perform complex reasoning about block interactions to push, flip, and stack them.

Automated Demonstration Generation

A key feature of ToyBlocks is its ability to automatically generate demonstrations of robots completing tasks in simulation. This eliminates the need for time-consuming teleoperation. Researchers can easily modify these training experiences, for example, by generating more pushing or flipping demonstrations, and then assess the robot's performance on tasks not included in the training data.

Design Principles

ToyBlocks adheres to four core requirements for a FoundationWorld: realistic physics and appearance, scalable automatic generation of tasks and demonstrations, challenges that combine reasoning with dexterity, and broad coverage of possible tasks within its domain. This framework builds on previous open robot learning efforts, including MolmoAct 2, MolmoBot, and MolmoSpaces.

Implications for Robotics Research

This framework provides a setting for studying how models improve with more training data and how to combine high-level reasoning with precise physical control. By offering a constrained yet challenging domain, ToyBlocks allows researchers to isolate variables and trace the causes of robot failures, which is crucial for advancing robot learning.