Key takeaways
- Redact PII before indexing, and again before each model call.
- Run embeddings and inference in the EU, or self-host an open-weights model.
- Enforce document-level permissions at retrieval time, not in the prompt.
- Keep only the audit events you need, redact unnecessary personal details and define log access and retention.
Why RAG creates GDPR risk
A RAG system retrieves passages from your documents and sends them to a language model. If those passages contain personal data, GDPR applies to ingestion, embeddings, retrieval, model calls and logs. Set a lawful purpose and legal basis, enforce document permissions before retrieval, and define retention and deletion across the whole system.
Settle the legal basics first
- Roles: you are normally the controller, and each vendor that touches the data (embedding service, vector database host, model host) is a processor that needs an Article 28 agreement.
- Lawful basis: write down the purpose of the assistant and the legal basis for each kind of data it can reach.
- Impact assessment: where the processing is likely to be high-risk, for example health or HR documents, carry out a DPIA under Article 35 before launch.
- Transparency: tell staff or customers that an AI assistant works on this data, under Articles 13 and 14.
- Models: ask each vendor whether your data is used for training, and get the answer in the contract.
Reference architecture
- Ingestion: detect and mask PII, keep a reversible mapping only where strictly needed, and record the source, owner and date of every document.
- Index: an EU-hosted vector store with per-document access labels, and a stored link from every chunk back to its document so that it can be deleted.
- Retrieval: filter by the user's permissions before ranking, so a chunk the user may not see is never a candidate.
- Generation: an EU-region or self-hosted model with no training on your data, a second redaction pass on the prompt and an instruction to answer only from the retrieved passages.
- Audit: an append-only log of query, sources and answer, with redacted values and a defined retention period.
Choosing where the model runs
| Option | Data residency | Effort |
|---|---|---|
| Vetted provider, EU region | EU, under DPA | Low |
| Open-weights model, EU cloud | EU, your account | Medium |
| Open-weights model, on-prem | Your servers | Higher |
An EU region settles where the servers are, not which country's law can reach the operator, so for the most sensitive documents we lean towards EU-based providers or your own servers. Our guide to EU-hosted AI platforms walks through the options and the questions to ask.
Which platforms are recommended for RAG with on-prem data residency and GDPR compliance?
There is no single recommended platform, because on-prem RAG is a stack of layers and each one has good open-source and EU-based options. What matters is that every layer runs on infrastructure you control, so nothing leaves it. A typical self-hosted set-up looks like this.
| Layer | Options that can run on your servers | What to check |
|---|---|---|
| Vector store | Qdrant (Apache 2.0), Postgres with the pgvector extension, Weaviate or Milvus | Per-document access filters, backups and encryption at rest. Pgvector keeps vectors beside your other data, while some stores have more components to operate |
| Model serving | vLLM (Apache 2.0) for production traffic, Ollama or llama.cpp for small deployments and testing | GPU capacity, how many requests run at once, and an OpenAI-compatible API so you can swap models |
| Orchestration | Haystack, LlamaIndex or your own code | How easily you can inspect each step and test retrieval separately from generation |
| Models | Open-weight language and embedding models that you host yourself | Licence terms for commercial use, language coverage and accuracy on your own documents |
| Access | Your existing single sign-on with role-based access | Permissions applied at retrieval, not in the prompt |
| Governance | Redaction, audit log, retention and deletion | The architecture above, tested end to end |
What happens when someone asks you to delete their data?
When Article 17 requires erasure, trace the data through source documents, chunks, embeddings, caches and logs. Delete or restrict copies according to the applicable obligation, and handle backups under your documented restoration and expiry process. Record what changed and check that retrieval no longer exposes the data.
Treat embeddings as personal data when they are made from personal data. Text can sometimes be inferred from vectors, so a vector database is not an anonymisation step.
Security risks specific to RAG
| Risk | How it shows up | Control |
|---|---|---|
| Prompt injection | A document contains hidden instructions that the model follows | Treat retrieved text as data, limit what the model can do, filter outputs |
| Sensitive information disclosure | The answer reveals data the user should not see | Permission filter at retrieval, redaction, output checks |
| Vector and embedding weaknesses | Poorly isolated indexes mix tenants or leak content | Separate indexes or strict filters per tenant, access control on the vector store |
| Misinformation | A confident answer not supported by the passages | Citations, an answer-only-from-sources instruction and a fallback when retrieval is weak |
Checklist before go-live
- Processor agreements signed with every vendor in the chain, including sub-processors.
- DPIA completed where required, with the redaction and hosting choices recorded.
- Permission filtering tested with accounts that must not see certain documents.
- Erasure tested end to end on a dummy document.
- Log retention set, and logs redacted.
- A set of real questions with expected sources, used to test retrieval and answers.
Tell us which documents your assistant needs and who should access them. We can scope a RAG pipeline with permissions, redaction and traceable deletion.
Scope your RAG systemFAQ
Can I use a US-based model provider?
Assess the provider’s actual processing locations, contracts and access arrangements. Transfers outside the EEA need a valid Chapter V basis and the relevant safeguards. An EU region is a deployment choice, not a complete GDPR assessment.
Are embeddings personal data?
They should be treated as personal data when they are derived from personal data, because content can sometimes be inferred from them. Protect them and delete them with the source document.
Do we need a DPIA for an internal knowledge assistant?
If it reaches health, HR or other sensitive documents, or processes personal data at scale, you probably do. For a harmless public knowledge base, a short written screening note is usually enough.
Sources
- Regulation (EU) 2016/679 (GDPR), Articles 5, 17, 25, 28, 32 and 35
- EDPB guidance on data protection by design and by default
- European Data Protection Board, Opinion 28/2024 on AI models
- OWASP Top 10 for LLM Applications
- Qdrant, open-source vector database (Apache 2.0)
- vLLM, open-source LLM inference and serving library (Apache 2.0)
About the author
Artem Synytskyi · Co-founder · AI
Artem runs AI at ESADEX, building LLM, RAG and speech-to-text systems for regulated environments.
