A new research paper introduces a method where a language model can rewrite its own weights. This process occurs without the need for gradients or traditional training runs. The model acts as its own optimizer, diagnosing its failures and applying changes to its internal parameters.
An open 27B parameter model was given its weights, graded answers, and the ability to execute itself. It successfully identified its own shortcomings and modified the relevant weights. This self-editing led to a 6.6 point increase in accuracy on problems it had not encountered before. The model also demonstrated continuous improvement over eight consecutive sessions, with accuracy consistently rising as it tested and submitted effective edits.
The model was able to repair a damaged copy of itself. This damaged copy initially solved almost no problems, but the self-rewriting model restored it to over 80% of its original accuracy on image tasks, without accessing the original weights or undergoing retraining. The researchers noted that this self-rewriting ability is dependent on scale; smaller models in the same family either failed to make edits or broke themselves, while only the largest model successfully improved and repaired itself.
Unlike test-time scaling, the gains from self-rewriting are written directly into the model's weights, incurring no additional cost per inference. This method uses only forward passes, similar to how models are served, distinguishing it from traditional training. The research suggests that the separation between training and inference may be a limitation of current tools rather than an inherent property of intelligence. This work is presented as a step towards recursive self-improvement, where models could continuously upgrade themselves throughout their operational life without external intervention or larger models.
✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →
One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.
One email a day. Unsubscribe in one click, any time.
Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.
▶ Play today's briefNew every morning, and the back catalogue is archived by date.
Researchers demonstrated a language model that can modify its own weights based on its reasoning, improving its accuracy without traditional training or gradient descent. This self-rewriting capability allowed a 27B parameter model to diagnose its failures and apply edits, increasing its accuracy by 6.6 points on unseen problems and repairing a damaged copy of itself.