← All stories
● Covered by 1 source · 1 reportMedium impact1 neutral

Language Model Rewrites Its Own Weights Without Gradients or Retraining

🔄 Updated 1d ago
New to BrevFeed? We gather this story from every outlet covering it into one summary — ranked by real-world impact, not just the latest headline — so you never miss what matters. What is BrevFeed? →

Key points

  • A 27B language model rewrote its own weights using its reasoning.
  • Improved accuracy by 6.6 points on new problems without retraining.
  • Repaired a damaged copy of itself to over 80% original accuracy.
  • Ability emerged with model scale; smaller models failed or broke.

Self-Rewriting Language Models

A new research paper introduces a method where a language model can rewrite its own weights. This process occurs without the need for gradients or traditional training runs. The model acts as its own optimizer, diagnosing its failures and applying changes to its internal parameters.

Demonstrated Capabilities

An open 27B parameter model was given its weights, graded answers, and the ability to execute itself. It successfully identified its own shortcomings and modified the relevant weights. This self-editing led to a 6.6 point increase in accuracy on problems it had not encountered before. The model also demonstrated continuous improvement over eight consecutive sessions, with accuracy consistently rising as it tested and submitted effective edits.

Self-Repair and Scale Dependence

The model was able to repair a damaged copy of itself. This damaged copy initially solved almost no problems, but the self-rewriting model restored it to over 80% of its original accuracy on image tasks, without accessing the original weights or undergoing retraining. The researchers noted that this self-rewriting ability is dependent on scale; smaller models in the same family either failed to make edits or broke themselves, while only the largest model successfully improved and repaired itself.

Implications for AI Development

Unlike test-time scaling, the gains from self-rewriting are written directly into the model's weights, incurring no additional cost per inference. This method uses only forward passes, similar to how models are served, distinguishing it from traditional training. The research suggests that the separation between training and inference may be a limitation of current tools rather than an inherent property of intelligence. This work is presented as a step towards recursive self-improvement, where models could continuously upgrade themselves throughout their operational life without external intervention or larger models.

✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →

The daily brief

One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.

One email a day. Unsubscribe in one click, any time.

Today's brief

Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.

~4 min · 3 stories · Nov 21

▶ Play today's brief Listen on Spotify

New every morning, and the back catalogue is archived by date.

Primary sources

arXiv 2611.21885

Reporting from

Researchers demonstrated a language model that can modify its own weights based on its reasoning, improving its accuracy without traditional training or gradient descent. This self-rewriting capability allowed a 27B parameter model to diagnose its failures and apply edits, increasing its accuracy by 6.6 points on unseen problems and repairing a damaged copy of itself.