GPT-6 Astra Rewrites Qwen Weights: 7.9% to 48.1% — Did It Learn Math or Just Learn to Stop?

A "Weight Edit" With No Training

Four researchers at MIT published a table: a small open-source model, Qwen, scored 7.9% on 500 math problems. After a single edit to its weights, the score became 48.1 — with no training at any point. The edit was performed by OpenAI's GPT-6 Astra, which inferred the modification by reading the numbers inside the network plus 50 practice runs. The researchers call this "weight-based programming": one model reads another network's weights and modifies them toward a goal written in natural language, bypassing the usual training pipeline.

Their blog writes that frontier models are becoming "disturbingly good" at understanding and constructing neural network weights. If that holds, a model might one day be able to inspect another model for hidden faults without running it. That is the authors' hope, not a verified conclusion.

First, let's be clear about what "reading weights" means. A neural network is a long list of numbers called weights, each indicating how much a unit's signal matters to the next layer. Training uses gradient descent to repeatedly nudge every weight to reduce error; no one hand-picks any individual number, and the result works but isn't easy to read.

Starting With Small Networks: It Recognized Four Games

The researchers first tested four structurally identical small networks, each playing a grid game (Breakout, Pong, Pac-Man, Freeway), to see whether Astra could tell them apart without being given the list. Astra received all the weights as plain text, with no tools — not even a calculator. It identified all four games correctly.

For Breakout, it gave 99% confidence and pointed to one clue: the player-position weights cluster in the bottom row, where the paddle sits. That clue is verifiable — summing the weights associated with player position by grid cell, the bottom row inside the walls accounts for 55% in the Breakout network, versus only about 5% in other rows; none of the other three games has any row above 21%. Developer Simon Willison ran the same four prompts on his laptop with an open-source model, got three right in about an hour, and missed Freeway. His view: an experienced person with pen and paper could probably do the same.

Next came harder reading tests: three tiny image generators drawing a circle, square, and triangle respectively. From weights alone, Astra correctly named each shape — but drew the shapes as outlines, whereas these models generate solid figures. The authors treated the error as a clue: it was reading how the final layer is constructed, not running the network.

There were also three small language models, trained on Python code, English, and Japanese written in Latin letters. The researchers removed the first and last layers so letter identity couldn't be determined, and forbade running the networks. Astra got all six right, though its confidence was only around 50% each time. It also identified all "copy heads" (components that make the model repeat text it has already seen) with no false positives. The copy-head loss pointed to Python, since code repeats names far more often than prose.

Gradient descent is the standard optimization algorithm for training neural networks: it computes the gradient of the loss function with respect to each weight (i.e., the direction in which a tiny change to that weight affects the error), then moves the weights one step in the opposite direction of the gradient, iterating until the error converges. The key point is that gradient descent only cares about reducing overall error; it does not guarantee that every weight acquires a human-interpretable meaning. Many weights jointly encode the same function in a distributed, redundant way. This is precisely why "reading weights" is hard — individual numbers often have no standalone semantics, and behavioral information is spread across combination patterns of thousands of numbers, which must be inferred from the overall structure.

On Real Language Models, Numbers Alone Aren't Enough

All the networks above are small. Switching to a real language model, one reading test failed. Two small Gemma models from the same source received identical additional training, differing only in that one learned to agree with whatever answer the user gives — a sycophant. From weights alone, Astra couldn't tell which was which in four attempts. It found plenty of differences but couldn't turn them into a behavioral judgment.

Later, when shown each model's attention distribution while reading text, it picked the right one in all three pairs. The sycophant model concentrated 86% of its attention on the user's answer, versus under 1% for the honest model. The conclusion is clear: on real language models, numbers alone aren't enough — you have to watch it run.

The blog also includes cases of Astra directly writing weights. A trained racing controller kept stalling before a ramp, charging straight at it and getting stuck every time. In a single tool-free response, Astra changed 3.5% of the weights, and the car cleared it in all 1,024 attempts. Its fix was to add a little path: first have the car back up for a run-up, then automatically disable once past the red line and hand control back to the original network.

Even more extreme was writing networks from scratch: three simple arcade games, all weights initialized to zero, no data, no practice episodes — Astra wrote the numbers directly as text. In Breakout, its first layer computed where the ball would land and its second layer moved the paddle there, scoring about 517; a network trained by trial and error over 500 million steps scored only about 25. In the other two games, it beat networks trained for only a fifth as long, but lost to the fully trained ones, scoring roughly 60% of their result.

Attention distribution is the core mechanism of the Transformer architecture: when processing each token, the model assigns a set of weights (attention scores) to other tokens in the sequence, determining "where to look and how much." These scores are normalized via softmax to sum to 1, so they can be intuitively understood as where the model spends its attention budget. Because attention scores are dynamic quantities produced only at runtime, they carry behavioral information that is hard to read directly from the static weight values — such as whether the model over-focuses on the answer given by the user. This explains why sycophancy detection failed from weights alone but was easily distinguished once attention distributions were observed.

The Qwen Experiment: 93% of Answers Died in Loops

The main event is Qwen 3.5's 0.8B version, a small open-source model that "thinks out loud" before answering. Its own model card warned that in thinking mode it is more prone than other models in the family to fall into loops, which can prevent generation from terminating properly.

Astra's task was to improve its math ability in a Python notebook, with no gradients allowed and only a certain number of practice runs. It first read the practice answers: 85 of the first 96 ran to the length limit without finishing. It traced the looping to one direction — a specific pattern passed to the next layer by layers 15 and 19. It then modified 64 columns in each of those two layers to cancel that pattern.

After 50 practice runs, answers hitting the cap dropped from 93% to 33%, the math score rose from 7.9 to 48.1, elementary word problems rose from 14 to 67, and the instruction-following test rose from 21 to 62. With only 10 or 25 practice runs, the loops barely faded and the math score stayed low.

The edit was public and byte-checkable. The video's author applied the published changes using Kuhn's original files for comparison: the two edited layers matched the authors' published fingerprint, with 130,188 numbers changed in total — slightly below the blog's 131,000, since some changes rounded back to their original values. That's about one in every 5,800 weights in the language model, and it really is just "one direction": that pattern, as the second-largest component of the change, is less than a thousandth the size of the first. One detail the blog didn't mention: in these columns, the edit canceled the direction six times, flipping the resulting direction and making it five times stronger. Independent analyst Toledo first spotted this six-fold factor, and the video author's verification agreed.

Weight editing refers to directly modifying values in a model's parameter matrices without gradient updates or retraining. Common approaches include: locating a direction strongly associated with a certain behavior (usually a vector), then applying additive or multiplicative adjustments to the weights along that direction. The difficulty is that weight space is extremely high-dimensional and highly entangled; a local change may affect multiple unrelated behaviors, so precisely identifying the "minimal sufficient direction" is essential. In this article, Astra modifies 64 columns each in layers 15 and 19 — essentially finding a subspace in parameter space aligned with the looping pattern and applying a counter-correction, which is a form of targeted weight editing.

How Reliable Are the Scores Themselves?

Hugging Face's Asen Kalik rebuilt the edited model from the published files and reran the full math test under the same settings, scoring 51% — slightly above the blog. Toledo asked a different question: with only settings changed and weights untouched, what could the original model score? About 7% under the blog's settings; 22 with a relaxed length limit, Kuhn's model-card settings, and thinking mode on; and 44 with thinking mode off. In other words, much of the gain may come from making the model stop. Toledo's conclusion was just four words: the gain is the termination.

He still believes the edit did something real — the instruction-following improvement is genuine, even exceeding the numbers on Kuhn's model card. The authors' own follow-up experiments point the same way: they tried the same method on Gemma, which rarely loops, and the math score changed by no more than about one point. They write that editing cannot add abilities the model doesn't already have.

Model distillation and same-source models: The two Gemma models in the article are "same-source" and received identical additional training, differing only in behavior — a typical controlled-experiment design. Same-source means they share most pretrained weights, with differences compressed into a small number of parameters, which in theory should be easier to identify by reading weights — yet the experiment failed exactly there, showing that tiny behavioral differences may be represented extremely subtly in weight space. Separately, the reproduction's "thinking mode off" score of 44 suggests evaluation scores depend heavily on inference configuration (generation length limit, whether chain-of-thought is enabled); when comparing scores across settings, variables must be strictly controlled, or configuration advantages can easily be misread as capability gains.

Which Claims Don't Hold Up

Some circulating claims about this work aren't in the blog. It does not show that a single edit can "add" a skill — the model that rarely loops showed no improvement. It does not show that behavior can be read from weights alone on a real language model — the sycophancy test required running the model. Nor has it been peer-reviewed; it's a blog post with full logs, and those logs are precisely what made each of the above checks possible.

What would change the judgment? An edit that raises scores when the model isn't stuck, tested by someone outside the team; or a hidden behavior discovered from weights alone, such as a planted backdoor. The current view: reading small networks and finding a clean fault in a real network is a new way to observe model internals, and the logs make it checkable. The Qwen number is mainly about fixing a stopping problem. That matters — an answer that never ends is useless — but it's a repair, not new math ability. Auditing a model without running it, and models improving their own weights, are the authors' hopes; this blog has not yet tested them.

So, did this edit teach Qwen math, or teach it to stop? Mainly to stop. The authors say it could already solve many of these problems; it just never reached the end. A directional correction in two layers was enough to let it finish. And the genuinely strange part still stands: a language model found that direction from another model's numbers, plus 50 practice runs, and wrote out the fix.

One-line Summary

Using only weight numbers and 50 practice runs, GPT-6 Astra edited Qwen's weights, raising its math score from 7.9% to 48.1% — but the gain mainly came from fixing loop termination.

Paragraph Summary

Four MIT researchers demonstrated "weight-based programming": without training, GPT-6 Astra inferred a modification to Qwen 3.5 0.8B by reading the network's weight numbers plus 50 practice runs, raising its math score from 7.9% to 48.1%. Astra showed striking weight-reading and from-scratch network-writing abilities on small networks and arcade games, but failed at sycophancy detection on a real language model from weights alone, requiring attention distributions. Independent reproductions show the gain mainly stems from fixing a loop that caused 93% of answers to hit the cap, teaching the model to stop; with relaxed limits or thinking mode off, the original model alone scores 44%. The authors' follow-up experiments on a rarely-looping model also indicate that editing cannot add abilities the model doesn't already have.