Recent advances in artificial intelligence are shifting the way robots learn to interact with the real world. Instead of relying on costly and time-consuming manual data collection, researchers are turning to virtual simulations. A standout example is Ai2’s MolmoBot, which uses synthetic data to train robotic systems, opening new doors for faster and more affordable AI development.

Moving Beyond Manual Data Collection

Traditionally, teaching robots to perform tasks involved gathering large amounts of real-world demonstrations. Projects like DROID collected over 76,000 teleoperated trajectories across multiple institutions, taking hundreds of hours of human effort. Similarly, Google DeepMind’s RT-1 used 130,000 episodes collected over more than a year. These efforts are expensive and limit the number of labs capable of participating in such research.

Ali Farhadi, CEO of Ai2, emphasizes the importance of making robotics a scientific tool that can help researchers explore new questions faster. He points out that demonstrating how AI can transfer from simulated environments to real-world settings is a key step in that direction. The goal is to build systems that can generalize well outside controlled labs and become accessible tools for scientists everywhere.

Introducing MolmoBot: Training with Virtual Data

Ai2 offers a different approach with MolmoBot, a suite of robot manipulation models trained entirely on synthetic data. Instead of manually collecting demonstrations, the team generates large amounts of virtual trajectories within a system called MolmoSpaces. This allows the creation of diverse and varied training data without human intervention.

The dataset, called MolmoBot-Data, includes 1.8 million expert manipulation trajectories. These were produced using the MuJoCo physics engine combined with aggressive domain randomization—changing objects, viewpoints, lighting, and dynamics—to make the virtual environment as varied as possible. This diversity helps the AI learn more robustly and reduces the need for real-world data collection.

Benefits of Virtual Simulation Data

By using powerful computing resources—such as 100 Nvidia A100 GPUs—the pipeline can generate about 1,024 episodes per GPU-hour. This means over 130 hours of robot experience can be produced in just one hour of wall-clock time. Compared to traditional methods, this approach yields nearly four times more data in less time, significantly speeding up development and deployment cycles.

Researchers tested the MolmoBot models on two platforms: the Rainbow Robotics RB-Y1 mobile manipulator and the Franka FR3 tabletop arm. The primary model, based on a vision-language backbone called Molmo2, processes multiple RGB observations over time to decide how the robot should act. This multi-modal approach helps the robot understand its environment more effectively, even in unfamiliar situations.

This shift from manual data collection to virtual environment design represents a major step forward. Instead of trying to close the sim-to-real gap by adding more real-world data, the focus is on creating more diverse and realistic virtual worlds. This makes the simulated data more useful and can lead to more adaptable and capable robots in the future.

Overall, the use of synthetic data for training physical AI offers a promising path forward. It reduces costs, accelerates research, and democratizes access to advanced robotic systems. As virtual simulation technology continues to improve, we can expect even more innovative solutions that bring AI-powered robots closer to everyday use in homes, factories, and scientific labs.

Inspired by

• https://www.artificialintelligence-news.com/news/ai2-building-physical-ai-with-virtual-simulation-data/

Sources

• physicalaiconference.com
• allenai.org
• iottechnews.com