Karpathy's Second Brain Works Because He Doesn't Trust It With Code

Andrej Karpathy posted something in April that I keep coming back to, not because it's a clever trick, but because of what he admitted alongside it. He's built himself a personal knowledge base, a wiki, out of nothing but markdown files and an LLM. No vector database. No embeddings, no retrieval pipeline, none of the RAG architecture that's become the default answer to "how do I make an AI system that knows things." He dumps raw material into a folder, points the model at it, and the model writes and maintains the actual articles, something on the order of a hundred interlinked pages, four hundred thousand words accumulated on a single topic he cares about.

The line that stuck with me wasn't the architecture description. It was this: "You rarely ever write or edit the wiki manually, it's the domain of the LLM." He's not reviewing every edit. He's not treating the wiki as a draft he polishes. He's letting it be, by his own description, the model's territory, and he's fine with that, enough that he called the whole pattern "room here for an incredible new product."

I believe him that it works. I also think the reason it works is the most interesting part, and it's not the part most of the coverage of this has focused on.

Why Skipping RAG Is the Actually Clever Part

The standard answer to "how do I give an AI system access to my own knowledge" has been retrieval-augmented generation for a few years now: chunk your documents, embed them, store the vectors, retrieve the nearest neighbors at query time, stuff them into context. It works, and it's also got a pile of failure modes that anyone who's run one in production has hit. Chunking boundaries that split a thought in half. Embeddings that drift out of sync with the source as it changes. Retrieval that returns the semantically closest chunk, not the one that actually answers the question, and a system that has no good way of telling you which one happened.

I've run a RAG pipeline in production, and the failure mode that actually bit us wasn't exotic. A support-docs search tool kept returning a chunk from an old pricing page, technically the closest semantic match to the question being asked, except the pricing had changed two months earlier and the embedding had never been refreshed because nobody had built a process for invalidating stale embeddings when source documents changed. The system wasn't broken. Every component did exactly what it was supposed to do. The result was still wrong, confidently, because "closest vector" and "correct answer" are related but not identical, and the gap between them is exactly where RAG systems quietly rot if nobody's watching the freshness of the index.

What Karpathy's doing sidesteps almost all of that by making a different trade. Instead of retrieving fragments of source material at query time, the LLM pre-digests everything into a standing set of markdown articles, a database that's also, directly, the readable artifact. There's no embedding layer to drift out of sync, because the "index" is a wiki page written in plain English that gets rewritten, not re-embedded, when new material comes in. There's no chunk boundary problem, because an article is as long as it needs to be to cover its topic coherently, not sliced to fit an arbitrary token window. And critically, because it's just markdown in a folder, it's git-diffable. You can watch exactly what the model changed between two versions of an article the same way you'd watch a code diff, which is a property almost no RAG pipeline gives you, because there's nothing analogous to "diff the vector store."

That's a genuinely good piece of system design. It's also, I'd argue, not really about AI at all. It's the same lesson as "the database you can read is worth more than the database you can only query," applied to a knowledge base instead of a production system. Karpathy didn't invent that principle. He just noticed it applies here too, and built around it instead of reaching for the default tooling everyone else reaches for.

The git-diffable part is worth sitting with longer than a single sentence, because it's doing more work than it looks like. Run git diff on a wiki article after the model touches it and you get a normal, human-readable text diff, this paragraph was reworded, this section was added, this claim was removed, line by line, the same artifact you'd review for a code change. Try to do the equivalent for a vector store and there isn't really a version of that question that makes sense. You can diff the source documents that went in. You cannot meaningfully diff the embeddings that came out, because the embedding space doesn't have a notion of "this specific sentence changed," it has a notion of "the nearest neighbors shifted slightly in nine hundred dimensions." That's not a minor inconvenience. It's the difference between an audit trail and a black box that happens to answer questions correctly most of the time.

There's a second, quieter advantage to the rewrite-in-place model that doesn't get talked about as much as the no-RAG framing: a wiki that gets rewritten compounds, where a RAG index mostly just accumulates. Dump the same raw material into a vector store twice, from two slightly different sources saying the same thing in different words, and you get two separate chunks sitting in the index, competing for retrieval rank, neither one more complete than the other. Dump the same material into Karpathy's setup and the model, rewriting the existing article rather than appending a new fragment, has the opportunity to actually merge the two accounts into one more complete, more accurate article than either source alone. A hundred articles built this way aren't a hundred static snapshots. They're a hundred things that have each been revised potentially dozens of times, converging toward better versions of themselves every time new material touches them. That's a genuinely different shape of system than "a growing pile of retrievable fragments," and it's the part of this I think is actually the most transferable idea, more than the "skip RAG" headline.

The Part That Makes This Safe Isn't the Architecture

Here's where I think the framing of "look, no RAG needed" undersells what's actually going on. The reason it's fine that he rarely reviews the LLM's edits isn't that markdown is a better storage format than a vector database, although it is. It's that a wrong wiki article costs almost nothing, for almost as long as it stays wrong.

Say the model writes something subtly inaccurate into one of those hundred articles. Nobody's paged. No system goes down. No customer is affected. The next time Karpathy, or the model itself on a later pass, touches that topic, there's a decent chance the inaccuracy gets caught and quietly rewritten, because the whole system is designed around continuous revision rather than a one-shot "write it once and ship it" model. Worst case, he reads a wrong sentence about something, notices it's wrong because he actually knows the subject, and the correction happens on the next pass. The failure mode is self-healing, slow, and silent, and none of those are bad things when the stakes are "a paragraph in a personal wiki was imprecise for a while."

That's not a property of markdown. That's a property of the task. I've written before about what happens when teams extend this same level of trust to AI-written code, and the failure mode there looks nothing like "self-healing and silent." A wrong auth check doesn't get quietly corrected the next time someone glances at the file. It sits there, passing its own tests, until someone finds the gap the hard way. The thing that makes Karpathy's setup safe to leave mostly unreviewed is not present in a codebase: being wrong doesn't propagate, doesn't get called by anything else, doesn't get deployed. A wiki article is a leaf node. A function in your auth flow is not.

What Happens If You Run the Same Experiment on Code

It's worth actually running this comparison through, because I think it clarifies exactly where the line is instead of leaving it as a vibe. Imagine applying Karpathy's wiki workflow directly to a codebase: dump raw material, the ticket, the relevant files, in, let the model write and maintain the implementation, rarely review it manually, let errors get caught and corrected on some future pass.

For a sufficiently contained, low-blast-radius piece of code, that's not actually crazy. A one-off data analysis script. An internal dashboard nobody depends on for anything load-bearing. Something genuinely closer in risk profile to a wiki article than to production infrastructure. The problem isn't that the pattern never works for code. It's that most of what gets built with this level of trust isn't that. It's the auth middleware, the payment flow, the database migration, exactly the stuff where "it'll get caught and corrected eventually" is not a plan, it's a hope.

Take something concrete: the rate limiter most apps get wrong. Imagine letting an agent maintain that config with Karpathy's level of hands-off trust, rarely reviewed, corrections happening whenever someone next happens to look. A wiki article that's briefly wrong sits there being wrong until the next read. A rate limiter that's briefly wrong, say a boundary condition reintroduced during an unreviewed "cleanup" pass, doesn't sit quietly. It gets exploited, probably before anyone next happens to look, because unlike a wiki article nobody is searching for ways to abuse a paragraph about an obscure topic. Plenty of people are actively searching for ways to abuse a weak point in a rate limiter, continuously, the moment it exists. The wiki's "I'll catch it next time I read about this" assumes the next read happens on a timeline that doesn't matter. Production code doesn't get to make that assumption, because the thing most likely to notice the bug first isn't you reading the file again, it's an attacker probing for exactly that kind of gap.

The honest version of Karpathy's insight, applied to engineering more broadly, isn't "let the AI write without review." It's "figure out which parts of what you're building have the failure characteristics of a wiki article, self-contained, low-stakes if briefly wrong, naturally self-correcting, and which parts have the failure characteristics of a payment processor." Treat the first category with his level of hands-off trust if you want. Treat the second category the way you'd treat any change to a system where being wrong for even a few minutes is a real cost, because that's what it is, regardless of what tool wrote the diff.

The Heuristic Worth Stealing

If there's a transferable lesson here, and I think there is one, it's not "ditch RAG" or "trust the model more." It's a question worth asking before you decide how much review something needs: if this is wrong for a week before anyone notices, what actually happens?

For Karpathy's wiki, the honest answer is "basically nothing, someone reads a slightly wrong sentence and it gets fixed on the next pass." For most production code, the honest answer involves words like "incident," "rollback," or "customer." Those two answers justify completely different amounts of review, and the mistake isn't picking one level of trust and applying it everywhere. It's assuming the level of trust that worked for a low-stakes, self-correcting system transfers to a high-stakes, non-self-correcting one just because the same kind of model wrote both.

Run a few more cases through the same question and the line gets easy to find. A changelog draft an agent writes for your next release: wrong for a week costs you an embarrassing correction, not an incident. Low stakes, hands-off is fine. A database migration script an agent writes for a production table: wrong for even an hour, let alone a week, can mean corrupted or lost data that no amount of "I'll catch it next time" undoes, because by the time you catch it the damage already happened and there's no clean rewrite-in-place the way there is for a wiki paragraph. Internal documentation describing how a system works, written and maintained by an agent that revisits it as the system changes: closer to the wiki case, self-correcting, cheap to be briefly wrong about. A feature flag configuration that gates who sees a new payment flow: closer to the migration case, because the cost of being wrong isn't "someone reads something inaccurate," it's "someone gets charged incorrectly while nobody's looking." Same model, same willingness to let it write without constant supervision, wildly different amount of review each of those actually deserves.

Karpathy's second brain is a good pattern. It's good specifically because he built it for a domain where the cost of being temporarily wrong rounds to zero, and he seems to understand that, even if the framing around it, "no RAG needed," makes it sound like the interesting part was the storage format. The storage format is nice. The actual insight is knowing exactly where you're allowed to stop watching.

Sources: Karpathy's original remarks as reported in Techstrong.ai, "Karpathy's Instructions for Building an AI-Driven Second Brain" and Angelo Lima, "A Wiki the AI Writes For You"