Weight-Space Programming II: what the critics and replications found

Three days ago, a post from MIT said an AI had edited another AI’s weights and lifted its maths score from 7.9% to 48.1%. The internet loved it. Then people started checking.

In Part I, I explained what the study did and why the headline needed care. This is Part II: what the critics argued, what the re-runs measured, and what still holds up.

Short version: the result is real. It is also narrower than the headline made it sound.

A quick recap of Part I

On 2 November 2026, four MIT CSAIL researchers published Sparks of Weight-Space Programming. Their headline: OpenAI’s GPT-6 Astra edited 131,072 weights inside the small open model Qwen3.5-0.8B, about 0.016% of it, and its MATH-500 score jumped from 7.9% to 48.1%.

The edit fixed one problem. Qwen kept getting stuck in reasoning loops and running out of room before it could answer. Once it stopped looping, it finished far more answers. Part I has the full story, with charts.

What the MIT post actually tested

The post’s summary says Astra did this “just by looking at the raw weights”. That phrase is where most of the argument starts, so it’s worth checking what each experiment allowed.

The post ran eight experiments. Their setups differ a lot:

None of this is hidden. The post’s main text describes the Python session and the test runs. But the “raw weights” framing in the summary only fits the smaller experiments, on networks up to about 172,000 weights.

Critique 1: it isn’t one technique

Matthew Honnibal, who created the spaCy language library, wrote the sharpest critique: the post is misleading. His argument, step by step:

1. He sorted the experiments by what they allowed, much like the table above.
2. The “no tools, just weights” setup only appears in experiments 1, 2 and 4, all on tiny networks.
3. Every experiment on a real language model used Python. The Qwen result also ran the model on practice problems.
4. The from-scratch game players beat a trained baseline on Breakout (517 against 25), but lost on Seaquest (53 against 89) and Asterix (53 against 55).

He accepts that the Qwen result shows Astra can be pointed at a real ML engineering task and do it. His objection is to presenting these experiments as a single new ability, “because it’s not a single technique”. That part is his judgement. The sorting of the experiments, though, anyone can check against the post.

Critique 2: a human could have done it

Sean Goedecke takes a different angle in LLMs are not superhuman at reading and writing neural network weights. His points:

• Experiment 1 is easier than it sounds. The game networks’ inputs are labelled. He shows that a few lines of Python measuring the first-layer weights get you most of the way.
• The Qwen fix follows a known recipe. Compare the model’s internal signals when it loops and when it doesn’t, find the direction that separates them, and remove it. That is close to “abliteration”, a technique people already use to steer open models.
• An analogy: editing weights by hand is like patching a compiled program. It works for targeted fixes, but the normal way to improve a model is still to train it.

These are mostly opinions, and he says so. He doesn’t run new experiments. His core claim, that a competent human ML researcher could have done every task, is reasonable for the Qwen fix. It is harder to judge for the from-scratch networks.

The re-run: 51.0% on MATH-500

The most useful new number comes from the edited model’s page on Hugging Face. Ahsen Khaliq rebuilt the edited checkpoint and re-ran the full benchmark:

• 500 MATH-500 problems, 4 answers each: 2,000 generations.
• The same sampling settings as the paper, one fixed random seed.
• The edited model scored 51.0%, and 31.6% of its answers hit the length limit.
• The two changed weight blocks were checked byte for byte against the paper’s files.

That is close to the paper’s 48.1%, so the edited model really does what the post says. Two caveats:

1. It’s not fully independent. The MIT post thanks Khaliq for rebuilding and re-running the model, even though the model card calls the check independent.
2. The original model wasn’t fully re-run. It got a 25-problem smoke test only, scoring 5.0% with 95% of answers hitting the limit.

One more thing a re-run can’t show: whether Astra would find the same fix again. The checkpoint was rebuilt from the logged edit. Astra wasn’t asked to rediscover it.

The baseline question

In Part I, I covered Carlos Toledo’s re-analysis. It is still the biggest open question, so here it is again in brief:

• With the paper’s sampling settings, the original Qwen scored 7.2% in his run, close to the paper’s 7.9%.
• With the non-thinking settings Qwen recommends on its own model card, the unedited model scored about 44.0%.
• He also found that 130,188 weights actually changed, not 131,072, because some rounded back to their old values.

So the “sixfold” jump compares the edited model against the original run in its worst setting. Against the original run in its recommended setting, the gap is a few points. The edit still has value: it rescues the model in the setting where it was broken. It just isn’t six times more maths.

Other people’s experiments

Chandan Singh: can Claude read a real model’s weights?

Chandan Singh’s open experiment on GitHub tests two things:

1. Writing weights from scratch. Claude, working as a coding agent, set every weight of small blank transformers by hand. Across 13 toy tasks it wrote 959 weight programs. All ten algorithmic tasks reached 100% accuracy within three attempts.
2. Reading a real model. Claude Opus 5.5, with no way to run code, looked at 420 weight edits to Qwen3.5-0.8B and predicted what each would do. Seeing the raw numbers improved its predictions only slightly: a correlation gain of 0.06 ± 0.02, or 0.13 ± 0.04 for the simplest edits.

The repo’s own verdict on the reading part is that the result “is mostly negative”. In other words, writing tiny networks by hand works well, and reading a real model’s raw weights mostly doesn’t, yet. This tests a different question from the MIT post, so it neither confirms nor refutes the Qwen result.

Simon Willison: a laptop model can read the game networks

Simon Willison ran the MIT post’s game-network prompts through Qwen 3.8 27B on his laptop. It named three of the four games correctly, taking between about 45 minutes and an hour each, and missed Freeway. That supports the easy end of the post: small networks with labelled inputs can be read without a frontier model.

What the authors have said since

As of 5 November, the MIT post shows no update, correction or reply to the critics. Its numbers match the Hugging Face page and the published logs. The authors say they are building a benchmark and a test harness for weight-space programming, which would let others compare methods fairly.

So what’s real?

Putting it together:

• Real: a general AI model, given Python and the ability to test, found and applied a small weight edit that stopped Qwen looping. The edited model does score about 48–51% in the paper’s setting.
• Real, but small: reading tiny, labelled networks from raw weights. A laptop model can do some of it too.
• Not shown: that models can read large models’ raw weights the way the summary implies. The one direct test of that came back mostly negative.
• Overstated: “six times better at maths”. Much of the gap disappears when the original model runs with its recommended settings.

That still leaves a genuinely new trick. An AI that can diagnose a model’s failure and patch it directly, in under a day, is useful even if a skilled human could do the same. What’s missing is evidence that it scales, and independent attempts to make Astra find a fix from scratch.

Open questions worth watching:

1. Can someone else get Astra, or another model, to find a fix like this from scratch?
2. Does the trick help models that don’t have an obvious bug like looping?
3. Will the authors’ benchmark make these claims easier to test?

Frequently asked questions

Sources

1. Takehi, R., Teoh, J., Zhang, A. and Khattab, O. (2026). Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights. MIT CSAIL, 2 November 2026.
2. Honnibal, M. (2026). The ‘Astra can read and write weights to improve Qwen’ post is misleading. 3 November 2026.
3. Goedecke, S. (2026). LLMs are not superhuman at reading and writing neural network weights. 3 November 2026.
4. Qwen3.5-0.8B-weight-space-programming: edited model and re-run results. Hugging Face.
5. Singh, C. weight-space-programming-autoresearch. GitHub.
6. Willison, S. (2026). Qwen 3.8 27B on my laptop can interpret model weights, too. 3 November 2026.
7. Toledo, C. (2026). Weight-Space Programming: the edit decided from its bytes, the scores re-graded. Independent analysis.
8. Tripathi, S. (2026). An AI edited another AI’s weights, and its math score jumped from 7.9% to 48.1% (Part I).