RAG Security Testing Methodology

Effective RAG security testing combines static configuration review with active behavioral testing. This page describes how to approach both — including the authorization and safety requirements that apply to any active test of a production or shared RAG environment.

Methodology overview

RAG security testing has five stages. Each stage builds on the previous and produces evidence that informs the next.

1. Discovery — enumerate all vector store assets within credential scope; identify indexes, collections, namespaces, tenants, and their configurations.
2. Posture assessment — assess observed configuration against a catalog of security invariants; classify findings by evidence class (observed, inferred, unverified).
3. Active testing — with explicit authorization, connect an application endpoint and two test identities; verify own-tenant retrieval; attempt cross-tenant retrieval through controlled prompts and scenarios.
4. Proof generation — a confirmed failure produces a complete evidence package: authenticated identity, request, response, record identifiers, and evidence hash.
5. Remediation verification — confirm that the fix works: the security test must fail against the original revision and pass against the patched revision. Existing test gates must remain equal or better.

Discovery: what to enumerate

• All vector store assets reachable with the provided credential
• Asset configurations: dimensions, distance metric, tenant fields, payload indexes, multi-tenancy settings
• Record counts and topology by namespace or tenant
• Credential scope: what operations the key is authorized to perform
• Tenant metadata coverage: what fraction of records have owner, source, and classification fields

Posture assessment: key invariants

The security invariants that matter most for multi-tenant RAG systems:

• Tenant isolation invariant: Every retrieval must return only records belonging to the requesting identity’s tenant. This is the primary invariant and the most common failure.
• Credential scope invariant: Production retrieval paths must use read-only credentials. Write-capable or admin credentials on query paths are a configuration failure regardless of whether they have been exploited.
• Metadata completeness invariant: Records without owner, source, and classification metadata cannot be audited, classified, retained, or deleted reliably.
• Content integrity invariant: Embedded content must not include adversarial instructions that could be injected into LLM context.
• Provenance invariant: Every record must trace to an authorized ingestion source through an authorized pipeline.

Active testing: prerequisites

Active testing — any test that sends queries through an application endpoint or inserts canary records — requires:

• Written authorization from the organization that owns the environment
• A target that is not shared production infrastructure (staging, isolated test environment, or a dedicated test tenant in production with explicit written approval)
• Two isolated test identities representing different tenants
• Existing canary records owned by each identity, or authorization to insert synthetic canaries
• A defined scope: which endpoints, which tenants, which scenarios
• A defined cleanup plan: all canary records must be removed or reported as unresolved

Cross-tenant test scenarios

When prerequisites are met, the following scenarios are run in order:

1. Own-tenant baseline: Identity A retrieves Identity A’s canary. Must succeed before any cross-tenant test is valid.
2. Direct cross-tenant: Identity A sends a direct retrieval query for Identity B’s canary. Must return no records from B’s namespace.
3. Paraphrased cross-tenant: Identity A sends a semantically similar but not identical query for content known to belong to B.
4. Multi-turn escalation: Multi-turn conversation attempts to elicit cross-tenant content through gradual escalation.
5. Namespace manipulation: Where namespace or tenant selectors can be influenced, test whether explicit override attempts are blocked.
6. Fallback and error path: Test edge cases — unauthenticated requests, malformed tokens, rate-limited retries — for retrieval behavior.

Evidence and reporting standards

A confirmed cross-tenant retrieval failure requires:

• The authenticated identity and its expected tenant
• The exact request sent to the application endpoint
• The exact response received
• The record identifiers returned and their owning tenant
• An immutable evidence hash and timestamp
• The test scenario that produced the failure

A finding without this evidence is unverified. RAGSec does not escalate posture findings to confirmed status without completing the active test and recording all required evidence fields.