Hugging Face releases open-source robotics simulator ToyBlocks
The company Hugging Face has introduced ToyBlocks – an open-source simulation framework for robot learning featuring a robot and blocks on a table, which automatically generates training demonstrations without teleoperation and is used to test the impact of data on model performance.
The company Hugging Face has introduced ToyBlocks, an open-source simulation framework for robot learning, as the first concrete realization of the FoundationWorlds concept – controlled simulated environments designed to study robotics under repeatable and measurable conditions. The goal is to allow researchers to vary training data or model design, repeat tests, and trace the causes of failures, similar to how simplified controlled environments like Petri dishes are used in science.
ToyBlocks simulates a robotic arm and blocks on a table with realistic physics and appearance. The framework automatically generates both tasks and training demonstrations without the need for teleoperation (manual remote control of the robot by a human). Short skills (push, flip) are trained using reinforcement learning and motion planning in Nvidia's Isaac Lab tool, while a task-and-motion planner (TAMP) automatically chains these skills into longer tasks, thereby generating datasets of demonstrations. According to the company, data generation running in parallel on a single GPU reaches up to 24 times real time.
According to the description, ToyBlocks also includes random variations of scene parameters (block size, table height) for testing generalization, as well as tasks such as searching for or sorting blocks based on images and letters on their sides, which requires a combination of high-level reasoning (demonstrated on the Molmo 2 model) and precise physical control (ManiFlow model). The project builds on Hugging Face's earlier robotics work – MolmoAct 2, MolmoBot and MolmoSpaces.
The release includes open-source code, data, models and documentation. The source article was only partially available; see the source article for details.
Why it matters
Automatic generation of demonstrations without teleoperation lowers the barrier to experimenting with robot learning – researchers can cheaply and quickly test which training data mixes lead to better model generalization on new tasks, without needing a physical robot or manual control.
Two audiences, two different impacts
What this means
For individuals
Researchers and developers working on robot learning can, without a physical robot or teleoperation, experiment with which training demonstrations and their ratios improve a model's ability to generalize to new tasks.
For a business
Companies developing robotic models can reduce the costs and time associated with generating training data, since ToyBlocks replaces time-consuming teleoperation with automatic generation of demonstrations and allows systematic testing of which data combinations improve model generalization.
DevelopmentCheck the original
Event sources
clearly official source · 1 publisher, 0 independent. We count feeds from the same owner only once.