A working notebook rather than a publication. Most entries start as a reaction to a paper, a post, or something that broke in production, and they are published while the thought is still forming — dated, revisable, occasionally wrong.
Posts
The offline–online gap is well measured. Its causes aren't.
Booking.com measured a −0.10 correlation between offline gains and A/B gains; Criteo's best off-policy estimator calls the sign right about half the time. The papers measure the gap carefully and never say how often each cause is to blame. What one randomized-exposure experiment adds, its limits up front, and why "it's usually X" is a guess.
Building an LLM serving platform, from the browser to the GPU
A complete engineering account: FastAPI, Ray Serve, vLLM, cloud deployment, streaming correctness, repeated GPU experiments, and the decisions behind the system.
LLM inference in cards: the map, and scheduling
A card series on where a serving stack actually loses time. The map first — six entry points that combine — then scheduling in detail: batching reuses one read of the weights, continuous batching updates the batch every iteration, and chunked prefill caps what a long prompt can do to everyone else's inter-token latency.
One API, three sandboxes
A sandbox execution service with runc, gVisor and Firecracker behind one gRPC API.
The same thirteen escape attempts against every runtime — a suite that fails a weak runtime
five times before it passes a strong one — a warm pool that restores instead of boots, and
three bugs the tests found first.
Putting a number on a coding-agent harness
A 1,800-line harness run on Aider's polyglot exercises: 24 of 25 solved.
The more useful number was where the money went — four exercises that looped past 25 turns
cost more than the other twenty-one together, and why no unit test could have said so.
Building an agent runtime: the five things that only broke when I ran them
A journalled, resumable agent runtime with a context budget, retrieval judged
by maintainers' real answers, a sandbox, and a regression gate — and the bugs that reading the
code would never have found: pins collected after the step that removed them, a read-only
filesystem that still leaked secrets, a fork bomb stopped by the scheduler, an agent that
retrieved its own question.
Decode is memory-bound by construction
Prefill and decode run the same weights through the same kernels and
land 430× apart in arithmetic intensity. Working out why gives a closed-form ceiling that no
batch size can beat — and reframes what continuous batching and chunked prefill are buying.
Ignoring the state distribution in TRPO is not an approximation
The step every derivation waves a hand at is exact to first order.
The state distribution's derivative is not small — it is annihilated by a property of the
advantage function, which also explains why the baseline has to be V and not Q.
MCP is not dead — it is just not a personal-productivity tool
The CLI-versus-MCP argument is really an ownership question: how many
people have to agree when a tool definition changes. And both camps are arguing about
invocation while nobody argues about whether the call should have happened at all.
Calibration is a property of the exposure policy
Why a calibrator fitted on randomized traffic is rejected on served
traffic — and what that implies for any offline number you plan to trust.