Pantograph Develops Goal-Conditioned Model for Robotics in Minecraft

Training Robotics Models with Video Data

Pantograph is focusing on training general robotics models capable of autonomous actions for extended durations. A key challenge in robotics is acquiring diverse training data, and harnessing internet video content enables the scale of model training to surpass limitations imposed by small action datasets.

Goal-Directed Behavior from Pretraining

The team introduces a method where goal-directed behavior is integrated during the pretraining phase rather than waiting for a post-training setup. This change significantly enhances the model's capacity to achieve more complicated objectives, contributing to its adaptability.

Using Minecraft for Evaluation

Minecraft serves as a unique platform for testing these models due to its open-ended nature and support for a wide range of long-term goals. Pantograph's model, named Pan, can perform various tasks in the game, including combat, exploration, and construction. The model's performance indicates its ability to understand and generalize to unfamiliar environments.

Reinforcement Learning from Observational Data

The research proposes viewing internet-scale videos as trajectories suitable for reinforcement learning, focusing on observations devoid of explicit rewards or actions. This innovative perspective allows for the implementation of goal conditioning, employing hindsight relabeling to repurpose earlier failures into new goals, thus enhancing training efficacy.