AI Context Lock-In: The Five Tests It Has to Pass

Run five tests against anything that holds your AI context: Inspection, Export, Replay, Audit, Deletion. If it fails one, that context is not yours — it is state you are renting, on terms you did not write. AI context lock-in is what you have when you cannot inspect, export, replay, audit and delete the state your AI tools hold — on your own authority, without the provider's cooperation.

The rubric is not ours. On 2026-07-30 a post at Earendil coined the term provider-sealed state and proposed these five tests as the honest way to ask whether you control your own context. It hit 776 points and 221 comments on Hacker News. Then nothing happened. No vendor claimed the category, and nobody ran the rubric in public — including against themselves. So this post runs it: OpenAI Responses, Gemini Interactions, Anthropic, and Contextium, scored honestly, with the row we lose printed in the same table as the rows we win.

Provider-sealed state is a storage decision, not a policy decision

Provider-sealed state is context that physically exists but is not reachable by you in a usable form. It is not about a vendor being hostile. It is about where the bytes live, what shape they are in when you ask for them back, and who holds the key.

Two defaults set the stage. The OpenAI Responses API defaults to store: true, which retains conversation state server-side for at least 30 days. Gemini Interactions does the same. That is not a scandal — it is how stateful APIs get built, and it is what makes multi-turn agents cheap. But it means the canonical copy of your team's reasoning history is on someone else's disk by default, and every one of the five tests below is really a question about what you can do with that copy.

The distinction that matters for a platform team is between context you have and context you can act on. A JSON blob you cannot read is not an export. A transcript you cannot re-run is not a record. A deletion endpoint with a 30-day floor is not deletion. Each test below isolates one of those failure modes.

Inspection: if you can't read it, you don't have it

The inspection test asks a single question: when you fetch a stored item, can a human read what the model read?

This sounds trivial until you hit compaction. When a conversation outgrows the window, providers compact it — summarising or restructuring earlier turns to keep the run going. OpenAI's server-side compaction emits an encrypted item, and OpenAI's own documentation describes that item as "opaque and not intended to be human-interpretable". The Earendil post put the consequence better than we can: "This encryption does not hide the data from the inference provider but it hides it from you." The provider decrypts it to run inference. You get ciphertext.

Mechanically, this is the sharpest failure of the five, because it is silent. Your agent keeps working. Your logs keep filling. The stored history looks complete by row count. But the compacted spans — exactly the spans containing the decisions your agent made early in a long run — are unreadable to the team that owns the codebase.

Anthropic comes out of this test ahead of both. Its stored items expose a readable content field, so a compacted or summarised span can be fetched and read as text by a person with an API key and curl. That is a real design choice and it deserves to be marked up. If we scored this rubric so that we won every row, you should not believe any of the other rows either.

Contextium passes this test structurally rather than generously: context items are stored as plain markdown and structured records in a workspace you own, and every item an agent retrieves is the same item a human can open. There is no compaction step that produces a second, sealed representation.

Export: a dump you can't re-run is a souvenir

Everyone technically passes export. Every major provider will hand you a file. The test is stricter than that: an export passes only if the artefact you receive is sufficient to reconstruct the working state somewhere else.

Three things determine that. First, completeness — does the export include system instructions, tool definitions, and retrieved context, or only the visible message turns? Second, referential integrity — do the exported items point to file versions, tool schemas, and identities that still resolve, or to opaque IDs inside the provider's namespace? Third, readability, which is where export inherits Inspection's failure: an export containing encrypted compaction items is a partial export wearing a full export's file size.

The practical shape of a passing export is boring: text, addressable by a stable ID, with references that resolve to things you also hold. That is why we store context as files rather than as rows in a proprietary session table — and why our export is a directory, not an endpoint. The comparison of team AI memory tools we published earlier this year breaks down which tools ship this and which ship a JSON transcript and call it portability.

Replay: determinism is the test, not the format

Replay asks whether you can take an exported run and execute it again — same inputs, same context, same tool surface — and get a comparable result. This is the test that separates archiving from portability.

It matters mechanically for two reasons. Incident review is the first: when an agent ships a bad migration, "what did it know at the time?" is only answerable if you can rebuild the exact context set that was in the window. The second is migration. Moving a team from one model provider to another is an exercise in replaying representative runs against the new provider and diffing the behaviour. If a third of your history is sealed, you cannot construct the test set, and the switching cost is no longer technical — it is informational. That informational switching cost is the whole of what lock-in means here.

Every provider whose compaction is opaque fails replay by construction, because the compacted span cannot be reconstituted outside the provider that sealed it. You can replay the tail of the conversation. You cannot replay the part where the agent decided something.

Replay is also where statelessness helps. MCP servers are stateless between calls by design, so the context an agent receives is assembled per call from an addressable source rather than accumulated invisibly inside a session. We wrote about the consequences of that in MCP is stateless — your team context shouldn't be. A per-call assembly model is trivially replayable: you have the inputs, so you can re-issue them.

Audit: access logs are context too

Audit asks who read what, when, and on whose authority. It is the test most teams skip, and the one that shows up first in a security questionnaire.

The mechanism question is granularity. "User X called the API at 14:02" is not an audit trail for context — it tells you a request happened, not which knowledge left the workspace. The useful log line is item-level: which context items were resolved into the payload, which agent identity requested them, and which policy allowed it. Without that, you cannot answer the two questions that actually get asked — did the contractor's Cursor instance read the payments runbook, and did anything under legal hold end up in a model call.

This is also where personal and team context diverge sharply. Personal memory in a chat client is by definition unaudited: it accrues to an individual account and nobody else can enumerate it. Team context has to be enumerable to be governable, which is the whole argument for separating team AI memory from personal memory. Our access model and log surface are documented on the security page.

Deletion: the only test with a deadline

Deletion asks whether you can remove context and prove it is gone — including from every derived copy.

Three mechanisms decide the answer. Retention floors are the first: with store: true as the default and a retention window of at least 30 days, "delete" on a stored response is a request to start a clock, not an instruction that takes effect on return. That is fine if you know it and have written it into your DPA. It is a compliance incident if you assumed the call was synchronous.

Derived copies are the second. Deleting a source document does not delete the embedding built from it, the summary that quoted it, or the compacted item that absorbed it. A deletion story is only complete if it names what happens to derivatives — and most do not.

Propagation to consumers is the third. In a team with Claude, Cursor and Copilot all pulling from the same place, deletion has to be visible to every consumer without a re-sync step. This is where a single shared source of truth is structurally better than N tool-local copies: revoke or delete an item once and the next MCP call simply does not return it. There is nothing to chase, because there was never a second copy to chase.

The scorecard

Scored on what the documentation and the APIs actually do as of 2026-08-05. "Partial" means the mechanism exists but is incomplete in a way that would fail a real audit.

Test	OpenAI Responses	Gemini Interactions	Anthropic	Contextium
Inspection	Fail — compaction emits an encrypted item its docs call "opaque and not intended to be human-interpretable"	Partial — stored state readable, summarisation opaque	Pass — exposes a readable content field	Pass — items stored as plain markdown you own
Export	Partial — transcript exports, sealed spans included as ciphertext	Partial — same shape	Partial — readable, but provider-namespaced IDs	Pass — directory of files with resolvable references
Replay	Fail — sealed spans cannot be reconstituted off-platform	Fail — same	Partial — readable content, no first-class replay harness	Pass — context assembled per MCP call from addressable source
Audit	Partial — request-level logging, not item-level	Partial — request-level	Partial — request-level	Pass — item-level resolution logging per agent identity
Deletion	Partial — store: true default, ≥30-day retention floor	Partial — same	Partial — retention configurable, derivatives undocumented	Pass — delete once, propagates to every MCP consumer

Read the Inspection row first. Anthropic ties us there on the merits, and the reason we are comfortable printing that is that a rubric which awards a clean sweep to its author is marketing, not a rubric.

We ran the rubric on ourselves and failed a row

We ran all five tests against Contextium in the same week we drafted this, using the same procedure we would use on a vendor: fetch, export, re-run, log, delete, verify.

Inspection, Export and Deletion passed on the first pass — unsurprising, because they are consequences of storing context as files in a workspace rather than as sessions inside a runtime. Replay passed once we fixed the harness. Audit did not pass. Our MCP server logged which workspace a tool call resolved against and which agent identity made it, but not which specific context items were returned in the payload. That is request-level logging wearing item-level clothes: enough to see that an agent read something, not enough to answer "did this agent read that runbook". We shipped item-level resolution logging before publishing this, which is why the Audit row above reads Pass — and we are stating the sequence rather than quietly showing the fixed state, because the failure is the interesting part. A rubric you have never failed is a rubric you have never run.

Governed context is cheaper and consistent — not smarter

One claim we will not make: that shared context makes agents more accurate. The evidence this window points the other way. A study on filesystem-organised agent memory found that organising memory on a filesystem roughly halves retrieval cost while producing no measurable improvement in answer quality — and that the organisation degrades over time unless a dedicated management agent maintains it.

Take that as the cost argument it is. Halving retrieval cost is a real, measurable, defensible number. It is also the argument for governance rather than against it: the degradation finding says unmanaged shared context decays, which is precisely why ContextOps is an operational discipline with an owner and a review cadence rather than a folder everyone writes to. The three things governed shared context reliably buys a platform team are that it is cheaper to run, consistent across every tool on the team, and auditable. Accuracy is not on that list, and any vendor putting it there owes you a benchmark.

Contextium is shared AI context for a whole team, delivered via MCP — one workspace that Claude, Cursor, Copilot and every other MCP-connected tool reads from, so every agent inherits the same source of truth. The mechanics of how that workspace is structured are in our AI context management overview.

Run it on us

The rubric is public, the tests take an afternoon, and the failure modes are all mechanical. Run it against whatever holds your team's context today, then run it against a workspace where the context is files you own and every MCP-connected tool reads the same copy.

If one test is all you have time for, run Deletion. It has a deadline attached, it exercises Inspection and Audit as side effects, and it is the one a customer or regulator will ask about first.