The Joint-Embedding Predictive Architecture (JEPA) is an alternative to generative world models, focusing on predicting embeddings of future states rather than pixels. JEPA models consist of an encoder, a predictor, and a planner, operating in embedding space to learn relevant scene information for action decisions.
LeWM is a JEPA world model developed by a team including Yann LeCun, benchmarked on the Push-T simulation where a robot pushes a T-shaped block to a target position.
Experiments were conducted to assess LeWM's predictive accuracy by comparing its performance with a version that used perfect physics from the simulator. On long-horizon tasks with subgoals, vanilla LeWM achieved a 78% success rate, while the perfect physics version achieved 77%.
This indicates that LeWM's predictions are already as good as perfect physics for planning along the types of paths it was trained on.
Further experiments focused on improving how the trained LeWM architecture is used, rather than retraining the model. Four new planning methods were developed.
Two of these methods collectively increased LeWM's success rate on long-horizon tasks from 22% to 85%, matching the performance of a planner given ideal expert subgoals. Another method, a single change to goal scoring, raised the success rate by 60 points, a greater impact than swapping in perfect physics.
✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →
One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.
One email a day. Unsubscribe in one click, any time.
Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.
▶ Play today's briefNew every morning, and the back catalogue is archived by date.
A researcher conducted 600 experiments on LeWM, a JEPA world model, to evaluate its predictive capabilities and planning methods. The experiments revealed that LeWM's predictions are comparable to perfect physics for planning on trained paths, and new planning methods significantly improved its success rate on long-horizon tasks without retraining the model.