Weight-space programming: LLMs reading and editing neural network weights

MIT CSAIL researchers Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab introduce weight-space programming: asking a general-purpose LLM to inspect or directly edit neural-network weights to meet a natural-language goal, without gradient descent or training loops. Their early experiments include identifying what small networks do from their raw weights, writing small game-playing networks, and modifying a language model.

For Qwen3.5-0.8B, the authors report that GPT-6 Astra identified weights associated with reasoning loops and changed 0.016% of the parameters. Under the same sampling settings, they report MATH-500 rising from 7.9% to 48.1%, with IFEval instruction-following performance also increasing threefold. The authors call these very early results, not an established general method. Alex Krentsel separately says he reproduced the results and characterizes the capability as a recent step change; that is a researcher’s reproduction claim, not peer-reviewed independent validation.

The paper’s broader proposal is that language models may reason directly about parameter values, not only about prompts, code, or training procedures. The authors’ follow-up clarification adds important context: Astra’s edit drew on a published activation-steering technique for reducing repetition/looping, rather than evidence of unexplained “X-ray vision” into another model. Their post says the task began from a simple prompt and evolved without human feedback; the researchers recognized the looping explanation after inspecting the result. Haseeb Qureshi emphasizes this is a cool but understandable capability, not recursive self-improvement or “LLM telepathy.” The demonstration remains an early result; its scope and generality are open questions.