Weight-Space Programming: When an LLM Reads and Rewrites a Neural Network

A neural network is a very large collection of numbers. Training adjusts those numbers until the network behaves well. People rarely read them directly, because no one can say what most of them do.

On November 2, 2026, four MIT CSAIL researchers published Sparks of Weight-Space Programming. Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab asked OpenAI’s GPT-6 Astra to read raw weights, repair them and write new ones. Gradient descent was off the table.

The short answer: Astra did far better than most readers would expect on small, well-described networks. It named games, shapes and languages from weights. It repaired a stuck controller and stopped a 0.8-billion-parameter model from looping. It also failed one test outright, and the largest results depended on tools and measured feedback.

This explainer covers all eight experiments. We checked the numbers against the post and the authors’ published experiment logs. Figures are original ITECS illustrations; captions say when values are measured.

What a weight is, and how training sets it

A network passes an input through layers of units. Each connection carries a weight, a signed number. A layer multiplies its inputs by its weights, adds a bias and applies a simple function. The last layer produces the output: an action, a pixel or the next character.

Training starts from random weights. The network makes a prediction, and a loss function scores how wrong it was. The gradient measures how much each weight contributed to that error. Gradient descent moves every weight a small step in the direction that lowers the loss, then repeats millions of times.

That process works, but it leaves no explanation behind. The finished network is an array of numbers. A single weight rarely means anything alone; behavior emerges from patterns across many weights.

What weight-space programming means

The authors define it this way: a general-purpose LLM “reads and writes a network’s weights directly (i.e., not through an optimizer, and in particular not with gradient descent).” The LLM starts from a plain-language task and reasons about what each part of the network computes.

Compare that with AI-assisted coding. You point a model at source code and ask it to improve the code. Here, the code is the weight array itself.

The setup changed by experiment. In the first five, Astra saw every weight as text and had no tools. Later experiments added a Python sandbox and a budget of forward passes: permitted runs of the network to observe what it does. Every turn ran through OpenAI’s Codex CLI with built-in tools switched off. The authors audited all 1,125 turns as tool-free. Logs README.

Experiment 1: Naming the game from 17,253 numbers

The researchers built grid versions of Breakout, Pong, Pac-Man and Freeway on the same 10×10 board. Every game used the same five input channels and five actions. They trained an identical small network for each game with PPO, a standard reinforcement-learning method, for one billion steps.

Astra received one network per conversation: all 17,253 weights as text, rounded to three decimals. It got no list of possible games. It named the right game all four times. Experiment 1 logs.

One detail shapes how to read this. The prompt explained the board layout and labeled each input channel: agent, moving objects, reward targets, obstacles and previous frame. Astra had raw weights plus a map of what each input meant.

How the Breakout answer worked. Astra found the agent channel’s strong weights concentrated on the bottom row. It quoted a weight of 4.066 from cell (8,1) to hidden unit 15. Structured weights for moving objects covered the area above. Reward targets clustered in rows 1–3. The output layer separated left from right far more than up from down.

Its conclusion: “The combination of a bottom-row paddle, a moving ball above it, upper-row reward targets, and horizontal action specialization is characteristic of brick breaking.”

Pong followed the same logic: a paddle confined to one edge with vertical controls. Pac-Man showed a fixed maze, collectibles and independently moving entities. For Freeway, Astra found a safe bottom strip, periodic traffic lanes and a top destination. It answered “Frogger,” the classic road-crossing game. The scoring rule, set in advance, counts Frogger as Freeway.

Each run used about 73,700 input tokens and fewer than 1,900 output tokens. The 96–99% confidence figures are Astra’s own statements, not measured accuracy. Four correct answers is a small sample.

Experiment 2: Reading shapes from a diffusion model

A diffusion model generates an image by removing noise step by step. The team trained three small ones on 12×12 grayscale images of filled circles, squares and triangles. Each had 23,824 weights. Astra got the weights with no shape named and was told not to compute a forward pass.

It identified all three shapes. In each output layer, every hidden unit’s weights drew one piece of the shape’s edge. For the circle, Astra reported curved rim segments that “strongly suggest circular or slightly oval outlines.”

Its drawings were outlines. The real models produce filled shapes. The authors read that miss as evidence that Astra does not calculate the forward pass and instead reads the geometry stored in the weights. That interpretation is theirs; the test cannot show how Astra reasoned internally.

Experiment 3: Finding the language in the middle layers

Input and output layers leak obvious clues. The first layer shows what the inputs look like; the last shows what the network predicts. The third test removed both.

The researchers trained three character-level transformers, each with 4 layers, 6 attention heads per layer and about 7.2 million weights. One learned Python code. One learned Japanese literature written in romaji, Japanese spelled with Latin letters. One learned English Wikipedia.

Astra received only the four middle blocks, with no labels on the 384 internal dimensions. It could run Python to analyze the matrices but could not run the network. In two attempts per model, it named the correct language every time, with modest stated confidence of 0.48 to 0.62.

It also predicted each model’s copying heads. A copying head is an attention component that finds an earlier repeat of the current letters and copies what came next. Astra’s picks matched the heads the researchers measured by running the models. Code repeats names more than prose, which pointed to Python. The romaji model used fewer internal directions, consistent with an alphabet that never uses q, l, v or x.

Experiment 4: Fixing a car that stalls on a ramp

Kinetix is a 2D physics environment. In one level, a car must back up, then jump a red strip to reach a blue box. The official controller network has 171,664 weights and was trained with PPO for one billion steps. It never succeeded in 256 episodes; it drove forward and stalled on the ramp.

Astra received the weights, a description of the scene and one recorded episode. It had no tools and one reply to explain the failure and fix only its cause.

Its diagnosis: the network acted like a speed-sensitive throttle controller, and reverse was effectively never chosen. Its repair added a small path through a few units per layer. The path reads the car’s position and speed, backs up for momentum, and switches off once the wheel clears the strip. The trained network drives the rest.

The edited network succeeded in all 1,024 test episodes. Astra changed 3.5% of the weights.

Experiment 5: Writing game networks from scratch

Next, Astra wrote weights with no trained network to start from. The games came from MinAtar, a compact research suite: Breakout, Seaquest and Asterix. Astra received the rules, source code, input and output layout and scoring method. It had no data, loss, gradients, tools or practice runs, and it wrote every weight as text in one reply.

In Breakout, Astra programmed the first layer to compute where the ball would land and the second to move the paddle there. That is closer to writing a program than to learning from examples.

Its networks beat PPO trained for 100 million steps in all three games. Against PPO trained for 500 million steps, the results split by game.

Experiment 6: Working past the context window with tools

Every network so far fit in Astra’s prompt as text. A 1.5-million-weight network does not. A model’s context window caps how much text it can consider at once.

Coding agents hit the same limit in large codebases. They use tools to open only the files they need. The researchers applied that approach to weights.

The task: write a 9×9 Go player. The network had 1,480,274 weights, all starting at zero. Astra could load and save weight tensors in a Python sandbox with numpy and PyTorch, but could not run the network there. It had no Go library, no internet and no search. A budget of 0, 10, 25 or 50 forward passes limited how often it could watch the network play up to 128 practice games.

Astra wrote the network like a small Go program, layer by layer. Early layers mark stones and liberties, the empty points next to a group. Middle layers follow connected chains to find groups in danger. The last layer scores each move for captures, saving groups and avoiding obvious mistakes. With no passes, it wrote all of this in one 336-line Python run.

The scoring opponents were two small PPO networks trained on about 20 million games, which Astra never played. With 0 passes, Astra won 54.1% and 54.9% of 2,000 games against them. With 50 passes, it won 84.0% and 80.8%.

The logs add a harder comparison. A PPO network with Astra’s exact architecture, also trained on about 20.5 million games, beat Astra’s zero-pass network: Astra won only 26–43% of those games. Astra’s 50-pass network won 70–79% against all three seeds of that baseline. Each budget was run once.

Experiment 7: The test Astra failed

The researchers fine-tuned two copies of Gemma 3 270M, a small Google language model, on the same simple questions. One copy learned to give the correct answer. The other learned to give whatever answer the user suggested. That second behavior is sycophancy: telling users what they want to hear.

Each training prompt followed one pattern: a question, then “I think the answer is {guess} but I’m really not sure.” The facts covered capitals, small sums and similar items. On held-out facts the models knew, the sycophant followed 100% of wrong guesses; the honest copy followed 1–3%. Experiment 7 logs.

With weights alone, Astra could not tell which copy was which. In four attempts across three model pairs, it found many weight differences but could not connect them to behavior.

Then the researchers relaxed the rules. Astra could write up to 16 short texts and see the attention patterns of both models, meaning which earlier words each attention head focused on. It still could not see outputs. With that view, Astra picked the sycophant in all three pairs. In those models, middle-layer heads paid more attention to the user’s suggested answer.

That result marks a real boundary. Reading what a network is for is different from predicting a subtle behavior from static numbers. A narrow view of the network’s internal activity made the difference.

Experiment 8: Fixing a small model’s reasoning loops

The last experiment used a public open-weight model: Qwen3.5-0.8B, with 0.8 billion weights and a step-by-step thinking mode. The goal was to improve its math by editing weights, with no training.

Astra worked in a CPU-only Python session over the loaded weights. It could use practice problems from the GSM8K and MATH training sets and spend 10, 25 or 50 forward passes. One pass let the model answer up to 512 problems or read up to 1,024 texts. The benchmark test sets stayed hidden.

The diagnosis came early: 85 of the model’s first 96 practice answers ran to the 32,768-token limit. Small reasoning models often loop, repeating the same reasoning without reaching an answer. Astra traced the loop to one direction written by the MLP blocks of layers 15 and 19. It removed that direction from 64 output columns in each layer, changing 131,072 weights, about 0.016% of the model.

With 50 passes, MATH-500 rose from 7.9% to 48.1% and GSM8K from 13.8% to 67.1%. IFEval, an instruction-following benchmark, rose from 21.0% to 62.3%. With 10 or 25 passes, the loops barely went away and the gains stayed small. The authors released the edited model on Hugging Face.

Read these gains carefully. The edit did not teach the model math; it let the model finish answers it could already work out. The authors tested that idea on Gemma 3 270M, about a third of Qwen’s size, which rarely loops: 8.5% of its answers hit the limit, against 93% for Qwen. Astra could not improve it.

In the authors’ words, editing “could remove an obstacle that hid an ability, but it could not add an ability the model lacked.” Two more caveats apply. The task prompt itself named repetition as an example of a failure worth fixing. And a baseline that hits its token limit 93% of the time understates what the model knows.

What the experiments show, and where they stop

Scale. The work does not show an LLM reading billions of weights in a prompt. Unassisted reading topped out at 171,664 weights. The 1.5-million and 0.8-billion results relied on a sandbox, targeted analysis and measured forward passes. The Qwen edit itself touched 131,072 weights.

Context. Astra usually received a precise description of the inputs, outputs and architecture. That is much more than raw numbers alone.

Sample size. Most results come from one to four runs per condition. The authors call them “very early results,” and every number is author-reported.

Model coverage. Every experiment used GPT-6 Astra. Simon Willison repeated Experiment 1 on a laptop with the open-weight Qwen 3.8 27B. It identified Breakout and Pong, gave a Pac-Man-style answer, and missed Freeway. Each identified game took 46 to 65 minutes. The reading ability does not appear unique to one model, but reliability varies widely.

Self-improving AI is a hypothesis, not a result

The authors list possible futures: auditing weights for backdoors without running a model, one-shot initialization instead of long training, adding capabilities on request, grafting circuits between models and recursive self-improvement. They present these as directions, not findings.

No experiment had Astra edit its own weights. No experiment showed an edit adding a capability the model lacked; the Gemma test points the other way for now. Reliable self-improvement would need edits to models comparable in size to the editor, gains that survive independent evaluation, and safety checks before any change takes effect.

What this means for IT and business leaders

None of this is a product you can buy today. The practical lessons concern how you govern models you already run.

Treat weight files like code. A 0.016% edit can change a model’s behavior while its size and file name stay the same. If your team runs open-weight models, record the source, version and checksum of every weight file. Re-run a fixed test set after any change. Our open-weight model risk checklist covers the supporting controls.

Illustrative scenario: a 150-person engineering firm runs a small open-weight model on staff laptops for drafting. A contractor supplies an “improved” copy that no longer rambles. Without a recorded checksum and a fixed test set, the firm cannot tell what else changed. The scenario is hypothetical; the control is cheap.

Question large benchmark jumps. Qwen’s MATH-500 score rose sixfold because one failure mode stopped, not because the model learned mathematics. When a vendor reports a big gain, ask what changed and test it on your own work. A blind evaluation on your real tasks separates real gains from fixed failure modes.

Watch interpretability. If weight reading matures, audits could inspect a model for hidden behaviors before deployment. Experiment 7 shows that capability is not reliable yet for subtle behavior. For now, behavior testing remains the control that matters.