R
Multi-tenancy in agent memory usually gets treated as a config detail. Add a tenant_id, filter on it, move on. The problem is that filtering and vector search are two different operations, and the order you run them in determines whether isolation is deterministic.
An approximate nearest neighbor (ANN) search returns the k closest vectors it can find in the time budget you gave it. If the index holds every tenant's memories and you count on a tenant's own rows ranking highest, your isolation is a statistical tendency. It holds until two customers in the same vertical write near-identical facts: "primary datastore is Postgres", "we deploy to eu-central-1", "on-call rotation is weekly". Those embed to almost the same vector. The ranking cannot separate tenant A's copy from tenant B's, because nothing semantic separates them.
Relational databases solved this twenty years ago with row-level security. Storing embeddings does not change the problem. It moves the enforcement point into an index that was never built to enforce anything.
A memory read has a fixed set of steps:
The filter has to run before the search. Running it after causes a leak with tenants that look alike. The rest of this article is about getting the ordering right and the outcome deterministic.
You can apply the tenant filter in two places.
Post-filtering runs the ANN search over the whole index, takes the global top-k, then drops rows from other tenants. Two problems. The first is short results: if the global top ten holds three of your tenant's rows, the agent gets three memories and no error, so you get a worse answer instead of a stack trace. The second matters more from a security perspective - the request serving tenant A read and scored other tenants' vectors before throwing them away.
Pre-filtering makes the constraint part of the search, so the candidate set only ever contains rows the caller can see. Every serious vector store supports it:
Enforce this in the retrieval layer itself, not in the agent. If scoping depends on the model putting the right filter in a tool call, then a prompt change is a change to your security posture, and prompts do not get the review a schema migration gets.
Pick based on how many tenants you run. Then write the choice down, because changing it later means re-embedding everything.
Deleting a tenant is a bulk job. The derived data gets missed - derived summaries, profiles, analytics, etc. The purge should be one job that lists every store the memory reached, and treat anything outside that job as data you did not delete.
Deleting one person inside a tenant is harder. A consolidated memory like "the reporting manager prefers Friday summaries" may not name the person it came from anymore. If every row carries source, you delete by lineage. If it does not, you are down to similarity search and manual review, which doesn’t survive a GDPR Article 17 request.
Soft delete first, so a bad delete is recoverable and the tombstone gets cached, then hard purge on a schedule.
Three ways this breaks, all from the same root:
Almost all of it comes from one rule: the tenant boundary is a filter that runs before ranking. Make tenant_id mandatory, apply it as a pre-filter inside the search, keep the raw client behind a retrieval layer the agent cannot reach, carry enough provenance to delete by lineage, and add a CI test that seeds two look-alike tenants and asserts nothing crosses.
None of this is new. It is just extending the tenancy discipline of a multi-tenant database to an index that stores vectors instead of rows. Get the filter order right before you scale, not after a customer finds their data in someone else's session.
An approximate nearest neighbor (ANN) search returns the k closest vectors it can find in the time budget you gave it. If the index holds every tenant's memories and you count on a tenant's own rows ranking highest, your isolation is a statistical tendency. It holds until two customers in the same vertical write near-identical facts: "primary datastore is Postgres", "we deploy to eu-central-1", "on-call rotation is weekly". Those embed to almost the same vector. The ranking cannot separate tenant A's copy from tenant B's, because nothing semantic separates them.
Relational databases solved this twenty years ago with row-level security. Storing embeddings does not change the problem. It moves the enforcement point into an index that was never built to enforce anything.
The isolation control
A memory read has a fixed set of steps:
- Embed the query
- Apply the tenant (and optionally user) filter
- ANN search
- Optional rerank
- Inject into the prompt
The filter has to run before the search. Running it after causes a leak with tenants that look alike. The rest of this article is about getting the ordering right and the outcome deterministic.
Post-filtering leaks, pre-filtering does not
You can apply the tenant filter in two places.
Post-filtering runs the ANN search over the whole index, takes the global top-k, then drops rows from other tenants. Two problems. The first is short results: if the global top ten holds three of your tenant's rows, the agent gets three memories and no error, so you get a worse answer instead of a stack trace. The second matters more from a security perspective - the request serving tenant A read and scored other tenants' vectors before throwing them away.
Pre-filtering makes the constraint part of the search, so the candidate set only ever contains rows the caller can see. Every serious vector store supports it:
| Vector store | How it pre-filters | Watch out for |
|---|---|---|
| Qdrant | Extra graph edges on payload-indexed fields so filtered HNSW does not fall off a cliff | Index the payload field, or the filtered search gets slow |
| Pinecone | Namespaces, a hard partition rather than a filter | One namespace per tenant, so plan for a lot of them |
| pgvector | Filtered HNSW; before 0.8 a filtered scan could return fewer than k rows; 0.8 iterative scans fix it | Iterative scan trades predictable latency for correct row counts |
Enforce this in the retrieval layer itself, not in the agent. If scoping depends on the model putting the right filter in a tool call, then a prompt change is a change to your security posture, and prompts do not get the review a schema migration gets.
Three ways to partition and their tradeoffs
| Strategy | Boundary strength | What it costs you |
|---|---|---|
| One index, tenant filter | Only as strong as the filter being applied every time | Cheapest to run, best index utilization |
| Index or collection per tenant | Physical: a query on A's index cannot return B's rows | Thousands of indexes to create, snapshot, monitor, migrate; per-index overhead |
| Hash-shard tenants across N indexes, filter within | Bounded blast radius if a filter bug slips through | A shard map to manage, and hotspots when one tenant is 50x the rest |
Pick based on how many tenants you run. Then write the choice down, because changing it later means re-embedding everything.
Deleting a person doesn’t delete a tenant
Deleting a tenant is a bulk job. The derived data gets missed - derived summaries, profiles, analytics, etc. The purge should be one job that lists every store the memory reached, and treat anything outside that job as data you did not delete.
Deleting one person inside a tenant is harder. A consolidated memory like "the reporting manager prefers Friday summaries" may not name the person it came from anymore. If every row carries source, you delete by lineage. If it does not, you are down to similarity search and manual review, which doesn’t survive a GDPR Article 17 request.
Soft delete first, so a bad delete is recoverable and the tombstone gets cached, then hard purge on a schedule.
The failure modes
Three ways this breaks, all from the same root:
- Isolation left to ranking. Causes a breach when two tenants look alike.
- Post-filtering by default. Returns short result sets, and reads other tenants' vectors.
- No provenance on writes. You cannot find every row that came from the same person.
What good looks like
Almost all of it comes from one rule: the tenant boundary is a filter that runs before ranking. Make tenant_id mandatory, apply it as a pre-filter inside the search, keep the raw client behind a retrieval layer the agent cannot reach, carry enough provenance to delete by lineage, and add a CI test that seeds two look-alike tenants and asserts nothing crosses.
None of this is new. It is just extending the tenancy discipline of a multi-tenant database to an index that stores vectors instead of rows. Get the filter order right before you scale, not after a customer finds their data in someone else's session.