Transformer Transformer Model Designs Robots for Specific Motions

Motion-Conditioned Robot Co-Design

A new model named Transformer Transformer has been developed to automate the design of robots for specific tasks. Given a manipulation demonstration, the model generates a complete robot embodiment, including all physical components like links, joints, motors, and inertial properties, along with a controller. This process is referred to as motion-conditioned robot co-design.

Performance and Architecture

In a practical application, a robot designed by Transformer Transformer for cloth flinging on an ALOHA2 bimanual platform achieved a 73% reduction in tracking error and a 30% reduction in maximum joint speed compared to the original platform. The model is a diffusion transformer trained on RoboTokens, a unified tokenization system for robot embodiments, states, and actions. This architecture supports various robot types, such as wheeled bimanual, quadrupeds, and humanoids, and can be used for embodiment generation and cross-embodiment control.

Dynamics Self-Guidance

The model functions as a dynamics model, making reward-agnostic predictions that are converted into reward-specific value predictions during inference. These predictions then guide the embodiment diffusion process through a method called Dynamics Self-Guidance. Experiments across three design spaces demonstrated zero-shot optimization for unseen rewards and trajectories, improving performance and runtime over an evolutionary baseline.

Impact on Robot Design

The embodiment of a robot significantly influences its task performance. While efforts often focus on improving robot intelligence through data and algorithms, a suboptimal physical design can hinder even advanced policies. This research addresses this by providing a framework that reframes robot co-design into a three-step process: demonstrating desired end-effector motion, generating an optimized embodiment, and validating the design through direct control by the same model.