HiPHI Dataset for Humanoid Robot Learning Released

Introduction of HiPHI Dataset

A new large-scale motion capture dataset named HiPHI has been introduced. This dataset aims to provide comprehensive data for training humanoid robots, addressing the current limitations of existing data sources like internet videos and smaller motion capture datasets. HiPHI focuses on providing precise physical states and diverse actions necessary for embodied AI and Physical AI research.

Dataset Specifics and Features

The HiPHI dataset comprises 617.5 hours of whole-body human motion, captured with sub-millimeter accuracy using optical motion capture technology. A significant portion, 245.7 hours, is dedicated to human-object interaction, featuring synchronized object trajectories and meshes. The dataset's organization is guided by FrameNet, a linguistic framework for human action, ensuring systematic coverage of a broad range of movements.

Addressing Data Gaps in Robotics

Current data sources for humanoid robot learning have limitations. Internet videos offer diverse behaviors but lack precise physical state information, while existing laboratory motion capture datasets are accurate but narrow in scope. HiPHI is designed to bridge this gap by providing both high precision and broad coverage of human motion and interaction, which is crucial for teaching robots complex real-world tasks such as carrying, pushing, and pulling.

Application and Benchmarking

The white paper also details how reinforcement learning policies trained on the HiPHI dataset demonstrate improved performance with scale. These policies have been successfully transferred from simulation to a physical Unitree G1 humanoid robot. A benchmark suite is also introduced to measure motion diversity and interaction grounding, providing a standardized way to evaluate robot learning progress using the dataset.