HomeBody Humanoid Robot Uses Spatial Targets for Task Execution

Connecting Reasoning to Action

The HomeBody humanoid robot employs spatial targets to bridge the gap between abstract task reasoning and concrete physical execution. Its Vision-Language Model (VLM) is responsible for selecting appropriate skills and their corresponding targets. This selection process considers the robot's current view, map context, gripper state, recalled observations, and the outcome of previous actions.

Structured Skill Execution

Once a skill and target are selected, the VLM passes this information via a structured tool call. This design allows the skill itself to handle the planning and execution of the motion, meaning the VLM does not need to be aware of the skill's low-level implementation details. This modularity simplifies the overall control architecture.

Detailed Picking Process

For picking tasks, the tool call specifies an image point, normalized to a range of 0–1000, and indicates which hand to use. This point triggers segmentation, and depth is estimated from D435i stereo images using Fast-FoundationStereo. Camera calibration projects the masked geometry into 3D, enabling an analytical prediction of the grasp. To achieve the desired pose, an arm planner constructs a spline reference with minimum-jerk timing, solves inverse kinematics along the path, and verifies collision clearance for the swept motion.

Diverse Skill Targets

Other skills within the HomeBody system utilize targets tailored to their specific actions. Navigation, for instance, takes a 2D goal and a facing point in map coordinates, measured in meters. A placing call specifies the hand to use, a 3D release target within the torso frame, and a release distance. This skill moves the held object to the target and then opens the hand. Complex actions like drawer opening integrate handle alignment, a hooking posture, and backward walking into a single skill, coordinating the transition from reaching to pulling.