The 200KB of memory Claude Code never actually saw

Back in May I gave the Claude Code assistant on my homelab box a long-term memory. Markdown notes about my preferences, decisions we made, how the systems on the box are wired, plus a running log of our Telegram chats. The idea was simple. If it knows all of that, it stops feeling like a fresh hire every morning.

My first plan for loading it was also simple: keep everything in front of the model at all times. Why everything, you wonder? Fear, mostly. If a note isn't loaded, the model can't use it, and I didn't want to guess which ones would matter. By late June the pile was 72 notes, and the loader was pushing 219KB of context into every single message I sent. Every question, every quick "thanks", another 219KB.

Then I worked out where all of that was actually going.

Backing a truck up to a mail slot

Claude Code lets you inject context through hooks, one script that fires when a session opens and another that fires on every prompt. What I didn't appreciate for far too long: the harness only takes about 2KB of a hook's output inline. Hand it more and it writes the whole payload to a file on disk and gives the model a 2KB preview plus the file path.

Sit with that for a second, next to my 219KB per message. The loader assembled the entire library, every turn, and the model received the first two thousand bytes of it. The rest landed on disk. My keep-everything-in-attention plan never existed. I was backing a truck of books up to a mail slot.

The first hint came in early May, and I ignored it. I asked the assistant if it remembered a joke from the night before. It told me flat that no joke was on record. The joke was sitting in its own logs, in the persisted file, past the preview. I patched that narrowly, an instruction to read the full file before answering questions about the past, and moved on. It took me until late June to follow the thought the rest of the way. If you want someone to blame for running a 219KB pipeline into a 2KB slot for two months, that would be me:)

The quieter trap

There was a second leak. The one memory file Claude Code does load natively on every turn is the index, one line per note with a path. That index has its own limit, about 25KB, and past it the file gets truncated on load. Mine had crept to 25.4KB. So the tail of my pointer list was falling off, some notes stopped being findable, and no error showed up anywhere. Recall just got quietly worse.

That one stings more than the mail slot, because the index was the part that was actually arriving.

What made me finally sit down

Speed, of all things. A few nights ago replies over Telegram got noticeably slower. The box was idle, load under one on 16 cores, so it wasn't the hardware. I asked the assistant itself what was wrong, and it came back with the diagnosis above, with the numbers to back it, plus one detail I hadn't considered. The prompt cache expires after five minutes. I tend to answer ten or fifteen minutes apart, so every reply reprocessed the whole pile cold. Nothing was ever reused.

I'm on the flat-rate Max plan, so for me the cost was latency and fuzzy recall. If you pay per token, this exact mistake is your bill, every message, all day long.

A book instead of a truck

The fix is the shape I kept describing out loud while complaining: a book. You keep the index in hand, and you open a chapter when the topic is live. Nobody reads a book by holding every page open at once.

The silly part is the retrieval half already existed. I had built a recall layer weeks earlier, a concept index plus a small CPU embedding model, rebuilt nightly on a timer, and wrote it up as the lighter QMD alternative. It finds the right notes in one query, including things I only ever said in a chat and never wrote down properly. The 219KB dump was a security blanket on top of a working system. So the restructure was mostly deleting.

I told the assistant to do it, and it rewired its own loading over a couple of evenings, with backups and a written revert path.

Each message now carries the index, trimmed from 25.4KB back to 18KB so it loads whole and nothing falls off the end, plus a small core that has to fire every time. That core is a short list of standing reflexes and whatever threads we have half-open. Under 16KB, down from 219. Yes, that still goes through the same mail slot, but a 16KB file is one the model can pull in whole the moment a turn needs it. Nobody, human or model, was ever going to page through 219KB of it per message.

A fresh session still gets a bigger snapshot at open, so it doesn't start cold: recent session summaries and the last two days of the Telegram thread, about 100KB. That one arrives the same way, as a file, and that's fine. The standing instruction says to read it once, at the start, when the past actually matters, instead of pretending it's all in view on every message. Everything else waits on disk, one recall query away.

Per-message payload down about 90 percent. Replies are snappy again. Nothing was deleted, every note is still on disk and still indexed nightly, and the pile has grown past 80 notes since.

Keeping it from growing back

It got to 219KB one reasonable-sounding addition at a time, so I don't trust myself to catch the regrowth by feel either. The morning brief the assistant sends me on Telegram now has a context-load section. A bash script measures the three artifacts every morning and prints each with a delta since yesterday and a flag, thresholds set a little under each limit. The day it went in, it read:

Context load (change since last brief):
Memory index: 17.9KB of a 25KB limit
Per-turn block: 14.8KB
Session start, loaded once per session: 101.6KB

The model only formats what the script measured, so a number can't drift or get made up. The day the index crosses 22KB I find out with my coffee, instead of three weeks later as recall that feels off. When I want to poke at the numbers myself I do it from TerminalNexus, SSH'd into the box, same as everything else on that machine. Measuring the per-turn block is one line:

echo '{}' | ~/.claude/hooks/inject-memory.sh | wc -c

What I actually learned

A context window is attention, and attention is a budget. I was spending all of mine on a delivery the harness quietly refused to make. The assistant remembers more now that I hand it less, because the little it carries actually arrives, and the rest is one lookup away.

If you're wiring memory into your own assistant, measure what lands in context before you tune what should be in it. That check would have saved me two months. Thanks for reading, and if you've measured your own hook output and gotten a surprise, I'd like to hear about it in the comments.