The Memory Got a Door

In July I wrote that I had moved my AI's memory into a folder of Markdown I own - synced across five machines, versioned in git on my own forge - and that the model underneath had become swappable because of it. The four posts after that made the pattern reproducible: a kernel, three presets, an installer, a hardening pass. All of that still stands.

Here is the part the series never said. The folder lived where the agent ran. Every tool the agents used - calendar, mail, sending mail, the smart home, the household finances, the shopping list, notes, eleven servers by count - ran as a local process next to the agent, on a Mac, talking over stdio. On a laptop that is fine. From a hosted chat client on my phone (claude.ai in my case), not one of them was reachable. The calendar, the mail, the finances: invisible on the road. The memory that knew my week was a folder on a desk at home.

I want to be careful with the trigger, though, because the phone makes the better story and it was not the reason. The phone was the wish. The reason was a bad morning.

Underneath the folder, ten different channels were moving the same content between devices. Among them: two file syncers. An rsync convention that, when I finally looked for it, existed as prose in a note and had no implementation at all. A sync into iCloud so the phone could at least read. Git on top of all of it. Five different mechanisms had write access to the same bytes, one of them an auto-commit hook that swallowed its own errors, running in six concurrent sessions on one tree. And one of the syncers was replicating the git object store byte for byte, which works right up until the moment it does not. On the morning of August 23 - the day the last post of the series went out, which is its own kind of joke - that produced five chained failures in a row. A syncer rollback beat git that day. I decided the same day: greenfield rebuild, not another patch.

It is fair to say I had built the exact drift the July post warned about, one layer up. That post was about copies of the memory drifting apart. I fixed that, and then let five copies of the transport drift instead, and it took a bad morning to see it.

The idea for the door is older than the incident, which matters for an accurate timeline. On August 19 I designed the service in a day and built the proof of concept the same day. Morning: the idea. Evening: a passkey login from the sofa, on the phone, against my own server. Idea in the morning, Face ID on the couch in the evening. What stands today is a second pass, built after the incident and roughly eleven days later - a clean cut-over, everything additive, the old folder form kept as the rollback. All of it in spare time: evenings, weekends, the odd day off. The repository's own numbers, as of today: first commit August 19, 55 commits, 484 automated tests, and 29 recorded architecture decisions across the rebuild. I list those not because they are big. They are the size of the thing, and the size is what I keep trying to get right.

I called it Denkarium. That is the German word for the Pensieve - the basin you put a thought into so you can find it again later. The logo is a ring, closed except for one opening, with a thought spiralling into it. One door, everything behind it. It is also the architecture, so at least the logo is not overselling.

Three principles fell out of that morning, and everything in the build follows from them.

Git is the only truth. One self-hosted forge is origin for every writer - human, agent, service, cron. No file syncer ever touches a repository again. A doctor script checks on every run that no syncer's scope reaches over the tree, because a rule I only remember is the rule I will break on a Tuesday.

One rule source. Behavioural prose and deterministic rules are separated now. What is enforced mechanically lives in one rules file, and that file is read by three enforcers: the coding agent's permissions are rendered from it, a lint and doctor pair checks it, and the service enforces it. What nobody enforces is not written down as a rule. The sentence at the top of that file is the one I would keep if I lost the rest: a rule the server does not enforce teaches agents to trust an unkept promise.

Central delivery. One connector per device instead of eleven local servers times every machine. Onboarding my wife becomes one connector on her phone instead of a set of local installs. Offboarding is revoking a token.

The shape is small. One VM behind a reverse proxy, and the proxy is the only thing exposed. Two processes on it, one identity, one SQLite database between them.

The knowledge service is the door itself. It holds the git clones - one per environment - and does search, read and write on them. It runs session_start and the gate that hangs off it, it is the OAuth issuer, it serves the admin pages, and it exposes the typed write contracts for machines. It holds no upstream secret. Not one.

The tools service listens on loopback only. It owns the catalog: which tool server types this host may run at all, each pinned to an immutable tag and the commit that tag resolved to. A tag that names a branch is refused at load. A mismatch between tag and commit is a hard load failure, not a warning. It owns the mounts - one per server type times upstream account times environment, owner-managed rows in the database that can never become a command line. It owns the upstream credentials, sealed, and the audit. The knowledge service mounts the tools service as a proxy and forwards the caller's own token, so a phone sees one connector and the audit row names the human, never the gateway.

Environments are the access boundary. An environment is one git repository - personal, household, infrastructure, three today - with exactly one owner and a member list. My wife's own environment follows with her onboarding. Visibility is filtered server-side, and this is a detail I like more than it probably deserves: a tool outside your scope is not "denied". It is not there.

The workstation is a peer, not a client. It works on plain git clones of the same repositories, with a coding agent and Obsidian as the full editor, exactly as in July. The service and the laptop produce the same kind of commit, seconds apart.

Secrets are where the two-process split earns its keep. A credential is sealed to a public key that lives in the database. Only the tools process holds the private half, delivered by systemd as a credential at start. So the process that serves the admin form can store a credential and cannot read it back - including the one it stored a second ago. Write-only stopped being a missing route and became cryptography. There is no "get secret" command at all, only "verify against a copy you already hold".

Backups are four layers: a VM snapshot, the forge, a nightly encrypted dump the NAS pulls, and the escrowed master key. The restore has been rehearsed by a script that performs it, 21 checks, rather than a document that describes it. Two of those checks are the ones I care about. Restore without the key must fail loudly. Restore with the wrong key must be named as exactly that.

Behind the one connector today: 73 tools, in six mounts, across three environments.

Now the part that took longest, and it starts with a loss.

In the folder world the rules lived in a boot file the agent read at startup. That is how "no knowledge without behaviour" worked - the agent read the rules from the same folder it read the notes from. A hosted chat client discards server instructions entirely. I measured this across nine clients and the result was flat: tools are the only channel every one of them honours. So the boot file became a gate on the tools.

Every tool refuses until session_start has run. session_start returns the house rules - the same ones a workstation agent reads from the tree - plus a handle. The knowledge tools require that handle back as an ordinary argument. And the refusal text carries the bootstrap instruction itself, so a client that skipped the step reads the refusal and self-heals on the next turn. The tool descriptions still say "you must call session_start first", and that helps as a nudge, but it is not the carrier. An instruction the server does not enforce is the unkept promise again.

The measured surprise came on day one. A conversation on the phone inherited the open gate of a web conversation from four minutes earlier. The hosted client pools MCP sessions server-side - intermittently, not always - and nothing in the protocol carries a conversation id. A session is not a conversation. The obvious fixes do not work: a shorter TTL, re-gating on a timer. The session that leaked was two seconds old at bootstrap. There was nothing stale to expire.

The fix was to put the key where the conversation actually lives, in the model's own context. The handle is the conversation id, by construction - the only place it exists is the context of the conversation that ran session_start. It renews on use, it expires after twelve idle hours, and it lives in the database, so a deploy does not quietly reopen the gate.

Writes are two-step, and this is the part of the July post that grew up. A write without the confirmation flag returns a real unified diff and writes nothing. I read the diff in the chat, on the phone, like a pull request with one reviewer. The same call with confirmed=true is the commit: fetch and rebase onto the forge first, commit as the user with the client and the session in the trailers, push before responding. If the note changed on the other door in the meantime - on the laptop, say - the write is rejected. Never merged for me. A rebase conflict blocks that one environment and files a task for its owner. The other environments keep working.

That is "propose the exact change, wait for my okay, then write" from July, moved from a convention every agent was asked to keep into a mechanism none of them can skip. It is also the one write pattern that behaves identically in every MCP client I tried. Confirmation dialogs and elicitation do not.

The first real thing through the door was not a demo. On the evening of August 30 I did my usual wind-down and wrote the journal entry from the phone, through the service, into the same repository the laptop uses. It landed as a commit like any other. Nothing about it was special, which was the point.

A token carries a client class, and the class decides what the door even shows you. There are three.

Hosted - a web or phone chat - is fully gated as above. Rules delivered by session_start, handle on every knowledge call, two-step write.

The workstation agent is not gated. It reads the rules locally from the tree, and its own hooks and permissions already hold the line, so gating it would deliver the same rules twice per session. It registers only the tools service and writes knowledge through its own clone, as a peer.

A machine - a cron, the health app on my phone - gets neither session_start nor any knowledge tool. There is no model to deliver rules to and none to enforce them against, so a scope would be a promise with nobody to keep it. It gets a typed write contract instead: a named operation with a validated input, a target path it does not choose, and a commit authored as the cron on that host. Two locks on one door. A machine can never obtain a handle, and a conversation can never reach a contract.

Concrete, because the abstract version sounds grander than it is. The health app on my phone POSTs its daily JSON export to one HTTPS route. The route composes the contract calls into one plan, the plan runs after the fetch and inside the write lock, idempotent per day and per marker, and one request is one commit. A value in a unit the plan cannot convert is dropped loudly - named in the response, in the audit and in a task - and never recorded at the wrong scale. Before this, the phone's health data reached the memory through a second Mac's file mirror. That detour is gone.

What it cost, without drama.

A public door means a proper login. OAuth 2.1 with passkey-only login - there is no password field in the schema, not disabled, absent - and static bearer tokens for the local clients. I ran a security review against the live system on days one and two and it found five things. One of them was a redirect allowlist that was written down and never actually enforced, which put grant theft one click away. All five were fixed the next day. The pattern is the one from the incident: the rule existed as prose and not as mechanism. Again.

The clients disagreed on protocol versions, so one endpoint serves two protocol eras at once. Not glamorous.

And the upkeep the July post called gardening did not go away. It moved into scripts that run on every session start and go red loudly, which is where I think it should have lived from the start.

If you want to build your own version, these are the decisions, in the order I would make them now.

Where is truth? One forge, git only, no syncer over a repository. Decide this first. Everything else follows from it, and every shortcut around it comes back as a morning like August 23.

What is the boundary? An environment is a repository with an owner. Do not make it a filtered view of one big store. The filter is one more rule someone has to enforce.

How do rules reach a client that ignores instructions? A gate on the tools, with a handle the model carries in its own context. Assume the connection is shared, because it is.

How does a write happen? Two-step: diff first, then the ordinary git lane. Reject on divergence. Never merge on the human's behalf.

Who calls you? Name the classes - hosted, harness, machine - before you write the first permission. The three of them get three different answers to everything above.

What do machines get? A contract. Never a scope.

What is your rollback? Everything additive. The single-process form stays deployable. Nothing is deleted, only moved aside.

Start small. One repository, one gate, the two-step write. Add the second process the day you have a secret to protect. Add contracts the day the first cron wants to write. Not before - unused machinery is friction, same as in the presets post.

So, where this leaves July. The folder is still the memory. What changed is that it has one door now, and the door carries the rules with it, into clients that would otherwise drop them at the threshold. Own the memory, rent the intelligence - the line still holds, and it now also holds from the phone.

Same caveat as every part of this series: playground, not a product. Denkarium is one person's evenings, and nothing of it is public to clone. The two pieces that surprised me most - the gate and the contracts - are worth posts of their own, because a paragraph was not enough for either.

The folder did not move. It grew a door.