Diagnose
Probe each layer. If the right answer is readable inside but missing from the output, the fault is fixable by editing.
Weight-space programming for vertical models
graft turns a general model into the one your product needs. It finds where the model already knows the answer, writes a small change in exactly those weights, locks everything it should not touch, and ships the result as a patch you can read, test and roll back.
Validated on Qwen3, Qwen3.5 and Granite 4.0 · 0.35B to 8B parameters · every claim tested on pre-registered, sealed test sets
Why · the usual way
“It scores higher. Nobody can say what changed.”
Fine-tuning a small model on a device task cut its general reasoning score from 67.5% to 36%. The loss only showed up later.
How · with graft
“Here is what changed, where, and why.”
Same task, same base. Accuracy on par with fine-tuning while general ability stays where it was.
Insight · knows it, can’t say it
Ask a small model to check whether the front door is locked and it often calls the wrong tool. Read its middle layers with a simple probe and the right intent is right there. The knowledge exists. The path from understanding to output is blocked.
That gap is the opportunity. You don’t need to teach the model again. You need to clear the path, and that is a far smaller, far more precise operation than retraining.
Qwen3-0.6B on smart-home tool calling.
Function-choice accuracy (%), Qwen3-0.6B. The same gap appeared in all 12 base-model and scenario pairs we measured.
Method · five steps
Each step has a defined input, output and pass condition. Interpretability decides where to change and what to leave alone. A small, constrained optimization does the arithmetic.
Probe each layer. If the right answer is readable inside but missing from the output, the fault is fixable by editing.
Lesion maps and probe curves show which layers are computing the answer and which are pushing it off course.
Measure the directions that ordinary text and general reasoning travel through, then forbid the change from touching them.
Fit the change inside the remaining space only. General inputs pass through the edited weights exactly as before.
Test once on a sealed set, head to head with a tuned fine-tuning baseline. Ship the delta as a patch with its own report.
Deliverable · the patch
Every patch carries its diagnosis, the exact layers it writes, what it protects, and how it scored on data its authors never saw. Compliance teams get a document. Devices get a small delta over the air.
Figures in the example are from the Qwen3-0.6B smart-home edit. The command-line tool is illustrative.
$ graft inspect smarthome-intent.patch base Qwen/Qwen3-0.6B diagnosis intent readable at L4–6 99.2% output 57.5% lesion map L27 MLP pushes toward the wrong tool (98% of cases) writes mlp.down_proj · layers 18 20 22 24 27 changed 2,621,440 of 596,049,920 params (0.44%) protects ordinary text · ARC answer positions · direct actions verify sealed set H, 600 cases 45.0% → 86.0% ARC-Easy −1.0 pt ordinary-text KL ≤ 0.03 status ✓ passed pre-registered gate
Results · sealed tests only
We compare against full fine-tuning given the same data and an equal tuning budget. Test sets are written by independent authors, sealed, and scored once.
Points = graft minus fine-tuning, lines = 95% intervals. Dashed line is the −3 point non-inferiority margin set before testing. Non-inferior in all three, significantly better on industrial. Qwen3-0.6B.
After 100 add, update and delete operations, all three methods remember equally well. Collateral damage differs by an order of magnitude: ordinary-text KL 0.008 for graft, 0.20 for fine-tuning, 0.70 for LoRA.
Industrial control at 8B, up from 71.4%. On par with LoRA (88.9%) while general reasoning stays exactly at 0.970.
Typical KL drift on ordinary text after a protected write. Removing the lock made training unstable in every scenario we tried.
Smart-home seed set, base model to released edit. General reasoning went up, 84.0% to 86.0%.
Use cases · what a patch can do
The same diagnose, protect, write, verify loop applies wherever a model understands the domain but behaves wrongly in it.
Smart home, energy management, meter queries, industrial protocols. One base model plus a patch per product line, delivered over the air without replacing the model.
Sealed tests · energy 28 → 45% · metering 24 → 41% · industrial 23 → 36% from closed-form edits aloneA household calls its office “the Blue Room”. A plant has its own names for machines. Write those facts in once, instead of stuffing every prompt. At 0.6B, putting them in the prompt does not work at all.
Private aliases used with tools never trained on · 88% · in-context: 0%Add a device type, a new tool or a new rule. Each update protects what earlier updates taught, so patches build on each other in order.
Added energy on top of smart home · new 85.8% · old 99.2% keptAlready fine-tuned? Transplant just the task skill back onto the original model. Accuracy stays, the collateral damage goes.
Three-scenario joint model · same accuracy · ARC 0.580 → 0.670 (original level)Internal APIs, form extraction, database queries, ticket routing, compliance review. Each one is general ability plus company rules, and it runs on 7B-class models that fit one server GPU.
Qwen3-8B pilot · non-inferior to LoRA · general reasoning untouchedOne base model, a small patch for each customer, department or household, each with its own audit trail. Personalization at the cost of a file, not a training run.
Builds on [02] and [03] · in developmentOpen weights · available now
Edited smart-home tool-calling models, Apache 2.0. Each model was evaluated as released, from the published file, once.
| Model | Official test set | Independent held-out set | General reasoning (ARC-Easy) |
|---|---|---|---|
| Qwen3-0.6B-SmartHome-WeightEdit ↗2.1M weights changed · 0.35% | 66% → 84% | 48% → 78% | 67.5% → 65.5% |
| Qwen3.5-0.8B-SmartHome-WeightEdit ↗4 MLP layers | 72% → 82% | 62% → 88% | 84.0% → 86.0% |
| Granite-4.0-350M-SmartHome-WeightEdit ↗2.1M weights changed · 0.60% | 58% → 80% | 53% → 79% | 46.5% → 45.5% |
| Reference: a dedicated fine-tune of Qwen3-0.6Bdistil-labs, task-specific training | 92% | 90% | 67.5% → 36.0% |
Held-out set: 120 conversational requests written by an author who never saw the edit code or practice templates.
Frontier · the author is a model
Weight-space programming started with a bold idea from interpretability research: if computation lives in the weights, a capable model should be able to read them and write them directly, the way an engineer edits code.
We are climbing that ladder one rung at a time. Today the writes are short, exact and need no gradient descent at all. Tomorrow the instruction is a sentence in plain language, and the patch comes back with its own explanation.
Two threshold neurons per name at layer 4, set by hand from the model’s own activations. Original phrasing unchanged at 71.2%, unknown names still rejected, general reasoning 0.68. Sealed test, 420 cases, all five pre-registered checks passed.
A new location, an alias and a new intent, written by an AI reading the weights of six independently trained models. One writing program, no per-model tuning, no gradient steps, base skills intact.
A layer-by-layer closed-form transplant reached 67.4% on metering from just 30 conversations, against 70.3% for a fully fine-tuned model.
Discipline · how we keep ourselves honest
The biggest risk for a new method is believing it works when it does not. Every result on this page went through the same rules.
Design, comparisons and pass criteria are written down before anyone looks at a result. No choosing the flattering reading afterwards.
The fine-tuning baseline gets the same tuning budget we do. Beating an untuned opponent proves nothing.
Final tests are written by independent authors, checked for overlap with training data, and opened a single time. A claim must hold on two test sets by different authors.
Start · design partners
We are working with a small number of device makers and enterprise teams on their first patches.
A base model
Open weights from 0.35B up. Qwen, Granite and similar families tested.
A task it almost gets right
Tool calls, extraction, routing, domain rules. We tell you upfront if it is fixable by editing.
A few hundred examples
For writing the patch. We write the sealed tests independently.
What must not change
General skills, safety behavior, earlier patches. We lock them and prove it.
It uses a small amount of constrained optimization, so in that sense it is related. The difference is what surrounds it: a diagnosis decides whether and where to change, a mathematical lock keeps general inputs passing through unchanged, and a sealed test decides whether the result ships. The output is a patch with a report, not a new opaque model.
Abilities the model does not have at all. Editing is best at clearing the path between what a model knows and what it says. The diagnosis step exists to catch the other case early, so you can choose a stronger base or more data instead of wasting effort.
Our 8B pilot (Qwen3-8B, industrial control) matched LoRA on a sealed 800-case test and left general reasoning untouched. Larger models already know more, so the patch is mostly calibration.
We measure the directions that ordinary text and general questions use inside each edited layer and project every change away from them. Inputs that travel those directions produce the same outputs as before. In practice, drift on ordinary text stays around 0.001 KL.
Yes, in order. Each new patch treats the inputs of earlier tasks as protected, so adding energy management on top of smart home kept the original task at 99.2%. Several fine-tuned skills can also be merged into one protected write.
The released edits follow their base model licenses, Apache 2.0. The fine-tuned reference shown in the table is not ours and is used only for comparison.