All writing
Notes on building

Depth Without Weight

A Princeton paper about looped transformers turns out to describe my org chart — and my worst bug class. Repetition buys depth you didn't pay for, if you damp how hard each round writes.

July 19, 2026 · 5 min · Listen on The Operator's Desk

This week a paper came out of Princeton about a strange kind of language model. Instead of stacking a hundred different layers, it stores a small handful and runs them over and over — three times, five, seven. The stored weights never grow. The thinking does. The field calls this a looped transformer, and the headline is simple: repetition, done right, buys you depth you did not pay for.

Done right is the entire paper. The obvious version fails. When the same small stack runs again and again, every pass writes into the same shared weights, and every later pass reads what all the earlier ones wrote. And the passes are built to agree with each other — that is what sharing weights means. So the updates pile up in one direction, and past a certain loop count the whole thing destabilizes. The fix the authors land on is almost boring: as the loop count grows, damp how hard each pass is allowed to write. One line of math. With it, the looped model is unchanged when nothing repeats, beats the undamped version at every loop count of three or more, and on a hard reasoning benchmark gains about three points — roughly six standard deviations in their seed control, not a lucky run.

Depth without weight — the same small stack run for many rounds gains depth, but only damped writing keeps the loop stable Left: a stack of two shared blocks with a gold arrow looping from its output back to its input, labeled same blocks, R rounds. Right: a chart of amplification against rounds. A gray dashed line, undamped, curves steeply upward — each round amplifies the last. A flat gold line, damped, holds the baseline with regular soft-write marks, labeled stable at every round count. PRINCIPLE — DEPTH WITHOUT WEIGHT Same weights. More rounds. Damped. Repetition buys depth you didn't pay for — if each round writes a little softer than the last. BLOCK 1 BLOCK 2 same 2 blocks · R rounds stored once, visited many times amp rounds R=1 R=3 R=5 R=7 undamped — each round amplifies the last damped — stable at every round count runaway update A shared update is written by every round and read by every round · undamped reuse amplifies · damped reuse compounds

I run a looped organization

Why is the operator of a thirty-year-old family restaurant reading residual scaling math on a Saturday? Because the architecture is familiar. Twenty-six employees, one operator, and an AI operating system. We did not get more capable this year by adding weights — by hiring. The same small stack of loops runs every single day: the inbox loop, the filing loop, the reviews loop, the close-the-day loop. Depth without weight. The paper is a description of my org chart.

And the failure mode transfers exactly. My operating system keeps its working memory in files — a brain file, session logs. Every work session reads those files, does the day's work, and writes back what it believes. Those files are shared weights: written by every visit, read by every visit.

In July, one session wrote a small false claim into that memory: three customer reviews are sitting unanswered. It was wrong. An automated responder had already answered every one of them, usually within minutes, and the proof sat in that responder's own log the entire time. But no session checked the log. Each morning read yesterday's claim, trusted it, and wrote it forward. The claim survived five days. It crossed into two board reports. It nearly drove a real action — publicly replying, again, to customers a machine had already thanked.

When a human finally asked the dumb question — why can't I see these unanswered reviews? — the investigation surfaced a second bug of the same species. The responder itself had spent four days re-sending replies it had already posted, because it trusted a remote status flag instead of its own ledger. Two bugs, one shape. The paper has a name for the shape: visit alignment. A shared update, written by many passes, read back by those same passes, each round reinforcing the last, because nothing anywhere in the loop was independent.

Damping, for organizations

So we stole the fix. Our version of damping is a standing rule now: any claim that wants to be acted on for a second day must be re-verified against the acting system's own ledger before it gets written forward. The responder's log. The bank feed. The scheduler's record. If it cannot be verified at the source, it is demoted from do something to note it. The narrative is never allowed to be its own evidence.

Notice what the rule really is. Verification is decorrelation. In the paper's terms, independent passes are the safe ones; the correlated ones are what blow up. A human listening to the original audio. A script re-deriving a total from the raw statements instead of the summary sheet. An archive cadence that trims accumulated updates before they calcify. Every one of those is the same move: make this visit independent of the last one before you let it write.

There is a business lesson under the engineering one. Everyone building with AI is being told capability comes from bigger weights — bigger model, bigger team, bigger spend. The evidence keeps pointing somewhere else. An engineering team at Etsy reported the same model jumping from 52% to 76% on their tasks with no model change at all — just a better harness around it. And now the pre-training world is saying the same thing from the other side: a small stack, visited more times, with discipline about how each visit writes, beats the naive big thing.

You don't need a bigger team. You need the same small team, more rounds, and the discipline that keeps repetition from amplifying errors. The rounds are cheap. The discipline is the product.

Source: DeepLoop: Depth Scaling for Looped Transformers (Li, Zhang, Guo, Gu, Wang — Princeton/UCLA, July 2026).

/ar/