Thirty years of email, searchable in a second (notmuch, no cloud)

I have email going back to 1997. Somewhere in there is the receipt for a part I bought in 2009, the thread where I worked out a tax thing, the address of a person I emailed twice a decade ago. About 125,000 messages, thirty years of it, and for most of that time "searching" it meant opening Thunderbird, picking a folder, and waiting while it chewed. Across an archive that size, the built-in search is not really a search. It's a hope.

So I made the whole thing searchable in under a second, locally, on my own box. Here is how, including the one seam that nearly produced an empty index without telling me.

The shape of the problem

My mail lives in Thunderbird Local Folders, which on disk is a pile of mbox files, one big flat file per folder. About 33GB of them. mbox is fine for storage and useless for search at scale. What I wanted was the email equivalent of how I already work with my notes: just grep it, get the hits, move on.

The tool for that is notmuch. It doesn't move or own your mail, it reads a maildir (one file per message) and builds a fast index on top. You query it from the command line and it answers instantly, by sender, subject, date range, folder, full text, any combination. So the job was two steps: turn the mboxes into a maildir, then point notmuch at it.

The seam that nearly sank it

The obvious converter is a tool called mb2md. It did not work, and the way it failed is the interesting part. Thunderbird writes its mbox files with bare separator lines, just the word From at the start of a message, with none of the sender and timestamp that the format technically expects, and with Windows line endings on top. mb2md's parser wants the full proper separator. Faced with these files it rejected every message and would have happily handed me an empty maildir, which notmuch would have indexed into a tidy, confident, completely empty database. No error big enough to notice. Just nothing to find later, when I'd already trusted it.

Python's standard library has an mbox reader that handles the bare separator correctly. So I wrote a small converter on top of that instead. It walks the folder tree, mirrors it into a maildir, skips the junk and trash and drafts, and marks each folder done as it goes so I can stop and resume. About 125,000 unique messages came out the other side. notmuch indexed the lot in a few minutes.

If you ever do this with Thunderbird mail, that's the trap to know about. Reach for the standard library, not mb2md.

Searching it

Now finding anything is one command. notmuch search and a few words. I can scope it to a sender, a subject, a folder, or a date range like one year in particular, ask for a count, or pull the full message. A question that used to be a five-minute dig through folders is now instant. And because it's just a command, Claude Code on the box can run it too. When I ask it to find an old email, it queries the index, reads only the handful of matches, and tells me what I needed. I drive all of this from TerminalNexus, SSH'd into the server, same as everything else here.

Keeping it fresh without IMAP

I didn't want to sync over IMAP, because that means managing app passwords or OAuth tokens for accounts I'd rather not hand around. My setup avoids it entirely. Thunderbird runs on my Windows machine and its Local Folders sit on a drive that's mounted on the server, so when I file a keeper into Local Folders, that message lands on the server on its own. A small job runs every morning, opens only the folders that actually changed, skips anything it has already seen by matching message IDs, and folds the new mail into the index. It's safe to run by hand too, for when I want a search to be current right after sorting my inbox.

One limitation worth naming: deleting a message in Thunderbird doesn't remove it from the index, because the maildir copy is already on the server. For an archive I'm keeping on purpose, that's the behavior I want anyway.

Why bother

Thirty years of mail is the single most personal pile of data I own, and the whole point was to make it instantly useful without sending a byte of it to anyone. It never leaves the box. There's no service, no account, no subscription, no company in the middle that could change the terms or read the contents. A free indexer, a maildir, and a folder on a machine I control.

If you've got a giant mail archive you can never actually search, this is a quiet afternoon that pays off for years. Watch the mbox separator trap, keep it local, and let the index do the work. Thanks for reading.