Weight-space programming for vertical models

Patch a model the way you patch software.

graft turns a general model into the one your product needs. It finds where the model already knows the answer, writes a small change in exactly those weights, locks everything it should not touch, and ships the result as a patch you can read, test and roll back.

Validated on Qwen3, Qwen3.5 and Granite 4.0 · 0.35B to 8B parameters · every claim tested on pre-registered, sealed test sets

Why · the usual way

Retrain the whole model, then hope nothing broke.

New dataEvery change starts with a new dataset
Full retrainBillions of weights move at once
Black boxBetter on the test, worse somewhere else

“It scores higher. Nobody can say what changed.”

Fine-tuning a small model on a device task cut its general reasoning score from 67.5% to 36%. The loss only showed up later.

How · with graft

Find the fault, write the fix, prove it held.

DiagnoseDoes it know but fail to say?
Protected writeChange only the task’s own directions
PatchSmall, readable, reversible

“Here is what changed, where, and why.”

Same task, same base. Accuracy on par with fine-tuning while general ability stays where it was.

01Explainable: every change names its layers and reason 02Protected: general skills are mathematically locked 03Incremental: learn new tasks without forgetting old ones 04Verified: sealed tests, pre-registered rules

Insight · knows it, can’t say it

The answer is already inside the model.

Ask a small model to check whether the front door is locked and it often calls the wrong tool. Read its middle layers with a simple probe and the right intent is right there. The knowledge exists. The path from understanding to output is blocked.

That gap is the opportunity. You don’t need to teach the model again. You need to clear the path, and that is a far smaller, far more precise operation than retraining.

99.2%INTENT READABLE AT LAYERS 4–6
57.5%INTENT IN THE MODEL’S OUTPUT

Qwen3-0.6B on smart-home tool calling.

What it says vs what it knows
outputprobe

Function-choice accuracy (%), Qwen3-0.6B. The same gap appeared in all 12 base-model and scenario pairs we measured.

Method · five steps

From diagnosis to a shipped patch.

Each step has a defined input, output and pass condition. Interpretability decides where to change and what to leave alone. A small, constrained optimization does the arithmetic.

[01]

Diagnose

Probe each layer. If the right answer is readable inside but missing from the output, the fault is fixable by editing.

[02]

Locate

Lesion maps and probe curves show which layers are computing the answer and which are pushing it off course.

[03]

Lock what matters

Measure the directions that ordinary text and general reasoning travel through, then forbid the change from touching them.

[04]

Write

Fit the change inside the remaining space only. General inputs pass through the edited weights exactly as before.

[05]

Verify and ship

Test once on a sealed set, head to head with a tuned fine-tuning baseline. Ship the delta as a patch with its own report.

Deliverable · the patch

A model change you can review like a pull request.

Every patch carries its diagnosis, the exact layers it writes, what it protects, and how it scored on data its authors never saw. Compliance teams get a document. Devices get a small delta over the air.

Figures in the example are from the Qwen3-0.6B smart-home edit. The command-line tool is illustrative.

smarthome-intent.patchExample
$ graft inspect smarthome-intent.patch

base        Qwen/Qwen3-0.6B
diagnosis   intent readable at L4–6  99.2%   output  57.5%
lesion map  L27 MLP pushes toward the wrong tool (98% of cases)
writes      mlp.down_proj · layers 18 20 22 24 27
changed     2,621,440 of 596,049,920 params (0.44%)
protects    ordinary text · ARC answer positions · direct actions

verify      sealed set H, 600 cases   45.0% → 86.0%
            ARC-Easy −1.0 pt   ordinary-text KL ≤ 0.03
status      ✓ passed pre-registered gate

Results · sealed tests only

Matches fine-tuning. Keeps everything else.

We compare against full fine-tuning given the same data and an equal tuning budget. Test sets are written by independent authors, sealed, and scored once.

Accuracy vs full fine-tuning1,600 cases each

Points = graft minus fine-tuning, lines = 95% intervals. Dashed line is the −3 point non-inferiority margin set before testing. Non-inferior in all three, significantly better on industrial. Qwen3-0.6B.

General reasoning after writing 100 factsARC-Easy

After 100 add, update and delete operations, all three methods remember equally well. Collateral damage differs by an order of magnitude: ordinary-text KL 0.008 for graft, 0.20 for fine-tuning, 0.70 for LoRA.

Scale · Qwen3-8B89.4%

Industrial control at 8B, up from 71.4%. On par with LoRA (88.9%) while general reasoning stays exactly at 0.970.

Side effects · ordinary text0.001

Typical KL drift on ordinary text after a protected write. Removing the lock made training unstable in every scenario we tried.

Released · Qwen3.5-0.8B72 → 92%

Smart-home seed set, base model to released edit. General reasoning went up, 84.0% to 86.0%.

Use cases · what a patch can do

One base model. Many precise changes.

The same diagnose, protect, write, verify loop applies wherever a model understands the domain but behaves wrongly in it.

[01]

Make small models usable on devices

Smart home, energy management, meter queries, industrial protocols. One base model plus a patch per product line, delivered over the air without replacing the model.

Sealed tests · energy 28 → 45% · metering 24 → 41% · industrial 23 → 36% from closed-form edits alone
[02]

Write private knowledge into weights

A household calls its office “the Blue Room”. A plant has its own names for machines. Write those facts in once, instead of stuffing every prompt. At 0.6B, putting them in the prompt does not work at all.

Private aliases used with tools never trained on · 88% · in-context: 0%
[03]

Learn new things without forgetting

Add a device type, a new tool or a new rule. Each update protects what earlier updates taught, so patches build on each other in order.

Added energy on top of smart home · new 85.8% · old 99.2% kept
[04]

Purify a fine-tuned model

Already fine-tuned? Transplant just the task skill back onto the original model. Accuracy stays, the collateral damage goes.

Three-scenario joint model · same accuracy · ARC 0.580 → 0.670 (original level)
[05]

Bring enterprise models in line

Internal APIs, form extraction, database queries, ticket routing, compliance review. Each one is general ability plus company rules, and it runs on 7B-class models that fit one server GPU.

Qwen3-8B pilot · non-inferior to LoRA · general reasoning untouched
[06]

A patch per customer

One base model, a small patch for each customer, department or household, each with its own audit trail. Personalization at the cost of a file, not a training run.

Builds on [02] and [03] · in development

Open weights · available now

Released on Hugging Face.

Edited smart-home tool-calling models, Apache 2.0. Each model was evaluated as released, from the published file, once.

ModelOfficial test setIndependent held-out setGeneral reasoning (ARC-Easy)
Qwen3-0.6B-SmartHome-WeightEdit ↗2.1M weights changed · 0.35%66% → 84%48% → 78%67.5% → 65.5%
Qwen3.5-0.8B-SmartHome-WeightEdit ↗4 MLP layers72% → 82%62% → 88%84.0% → 86.0%
Granite-4.0-350M-SmartHome-WeightEdit ↗2.1M weights changed · 0.60%58% → 80%53% → 79%46.5% → 45.5%
Reference: a dedicated fine-tune of Qwen3-0.6Bdistil-labs, task-specific training92%90%67.5% → 36.0%

Held-out set: 120 conversational requests written by an author who never saw the edit code or practice templates.

Frontier · the author is a model

Describe the change. A model writes the weights.

Weight-space programming started with a bold idea from interpretability research: if computation lives in the weights, a capable model should be able to read them and write them directly, the way an engineer edits code.

We are climbing that ladder one rung at a time. Today the writes are short, exact and need no gradient descent at all. Tomorrow the instruction is a sentence in plain language, and the patch comes back with its own explanation.

› Teach the hall controller that “the old wing” means the office, and never guess a room it doesn’t know.
No gradients · no training · no teacher

24 new names written into Qwen3-0.6B

1.4% → 75.2%

Two threshold neurons per name at layer 4, set by hand from the model’s own activations. Original phrasing unchanged at 71.2%, unknown names still rejected, general reasoning 0.68. Sealed test, 420 cases, all five pre-registered checks passed.

Model-authored weights

An AI wrote three new skills into six networks

97.6–100%

A new location, an alias and a new intent, written by an AI reading the weights of six independently trained models. One writing program, no per-model tuning, no gradient steps, base skills intact.

Early lab result

A skill from 30 examples in 7 seconds

A layer-by-layer closed-form transplant reached 67.4% on metering from just 30 conversations, against 70.3% for a fully fine-tuned model.

Discipline · how we keep ourselves honest

When we say it works, it works.

The biggest risk for a new method is believing it works when it does not. Every result on this page went through the same rules.

Before

Pre-registered

Design, comparisons and pass criteria are written down before anyone looks at a result. No choosing the flattering reading afterwards.

During

Fair opponent

The fine-tuning baseline gets the same tuning budget we do. Beating an untuned opponent proves nothing.

After

Sealed and scored once

Final tests are written by independent authors, checked for overlap with training data, and opened a single time. A claim must hold on two test sets by different authors.

Start · design partners

Bring us a model that knows more than it says.

We are working with a small number of device makers and enterprise teams on their first patches.

[01]

A base model
Open weights from 0.35B up. Qwen, Granite and similar families tested.

[02]

A task it almost gets right
Tool calls, extraction, routing, domain rules. We tell you upfront if it is fixable by editing.

[03]

A few hundred examples
For writing the patch. We write the sealed tests independently.

[04]

What must not change
General skills, safety behavior, earlier patches. We lock them and prove it.

Questions people ask.

Is this just fine-tuning with extra steps?

It uses a small amount of constrained optimization, so in that sense it is related. The difference is what surrounds it: a diagnosis decides whether and where to change, a mathematical lock keeps general inputs passing through unchanged, and a sealed test decides whether the result ships. The output is a patch with a report, not a new opaque model.

What can’t it fix?

Abilities the model does not have at all. Editing is best at clearing the path between what a model knows and what it says. The diagnosis step exists to catch the other case early, so you can choose a stronger base or more data instead of wasting effort.

Does it work on larger models?

Our 8B pilot (Qwen3-8B, industrial control) matched LoRA on a sealed 800-case test and left general reasoning untouched. Larger models already know more, so the patch is mostly calibration.

How is general ability protected?

We measure the directions that ordinary text and general questions use inside each edited layer and project every change away from them. Inputs that travel those directions produce the same outputs as before. In practice, drift on ordinary text stays around 0.001 KL.

Can I combine patches?

Yes, in order. Each new patch treats the inputs of earlier tasks as protected, so adding energy management on top of smart home kept the original task at 99.2%. Several fine-tuned skills can also be merged into one protected write.

Can I use the released models commercially?

The released edits follow their base model licenses, Apache 2.0. The fine-tuned reference shown in the table is not ours and is used only for comparison.