RAG uses outdated document versions: how to find the cause

If your RAG system uses an old version despite updated documents, trace one specific failure through the entire pipeline: from the authoritative source through the index and search filters to model context and citation links. Find the first point where the expected version is lost.

Uploading a document again does not establish that the new version is available and eligible for the question. An accurate quotation is not enough either: the quoted passage may faithfully reproduce a rule that used to apply.

The diagnostic tables and tests below are my technical recommendations. Documents, values and example results are explicitly fictional. No customer-system measurements were performed for this example.

Decide which version may answer the question

“Use the latest version” is not a complete rule. The highest version number may belong to a draft. A published version may take effect next month. An older version may be the correct source for a historical question.

Distinguish:

• Modification time: When was the file edited?
• Approval: Which version was approved, and for which business scope?
• Effective period: When does its content apply?
• Processing state: Which version has been fully made available to search?
• Question context: Which date, location or product version does the question concern?

VersionRAG examines version-sensitive questions as a distinct evaluation task, explicitly modelling document versions and their evolution. Its results on that benchmark are not a quality forecast for other enterprise systems.

Connecting a DMS to RAG: architecture, permissions and versions covers the broader architecture. The following diagnosis assumes that a responsible owner can identify the authoritative version.

Make the error visible with a small test corpus

Start with artificial documents containing clearly distinguishable values.

Fictional example: An internal assistant answers questions about the demo “Service inspection” policy for an Austrian site.

For “How often is the service inspection required now?” on 27 October 2026, the expected result is 14 days, supported by V4.

For “Which rule applied on 19 October 2026?”, it is 30 days, supported by V3, provided historical search is supported and the user is authorised today. The earlier approval state must remain traceable; today's “superseded” status alone is not a sufficient selection rule.

V5 must not appear as an applicable rule in either answer. This tests current validity, historical validity and exclusion of drafts.

Find the first processing step that diverges

Repeat the failing question with a recorded user context and as-of date. Compare these stages:

Fix the earliest divergence first. If V4 is missing from the searchable corpus, rewriting the answer prompt cannot correct that state.

Oracle describes such divergence between authoritative sources and searchable material as RAG index drift. Its engineering article dated 16 July 2026 recommends reconciling source identity, versions and derived retrieval objects, among other checks.

Begin with a narrow evidence record: run ID, as-of date, document and version IDs, applied filters and selected passage IDs. Full confidential documents do not need to be copied into every diagnostic log.

Check validity before ranking

A similarity score indicates how closely a passage matches a query. Your application must determine business eligibility separately.

My proposed design rule for the example is:

A passage is eligible as evidence if:

1. the user is currently authorised to access it,
2. the document version was approved for the business scope
   at the requested as-of date,
3. the version applies at that as-of date,
4. the passage belongs to a fully available index release.

Rank content within this eligible corpus. Apply the rules to every retrieval path, including lexical subqueries and subsequently loaded context.

Historical questions require an explicit rule. A blanket is_latest = true filter would exclude V3 even though it is the correct source for 19 October. Conversely, a historical question must not restore former access permissions.

Vector databases can provide technical filtering primitives. Qdrant, for example, documents datetime range filters for RFC 3339 timestamps. Your data model and application must still define business validity.

Pay attention to boundaries. One possible convention includes the start and excludes the end:

valid_from ≤ as_of < valid_until

In this example, V4 starts at 00:00 on 20 October 2026 in Europe/Vienna; V3's exclusive end is the same instant. Define missing end dates, time zones and conversion. Overlapping effective periods are a conflict to handle, not a question for a similarity score.

Inspect the steps after retrieval too

A correct initial result list is not sufficient. Check three possible detours.

Expanded context: If search finds a small passage and then loads an entire chapter, both must belong to the same document revision.

Summaries: A separately stored summary needs traceable source-version references. Otherwise it can reintroduce outdated information into context.

Stored answers: Before reusing an earlier answer, verify its validity for the current request context. A new index release does not automatically make a stored answer current.

To narrow down the cause, repeat a test without the answer cache. If the error disappears, this is evidence pointing to that path, not a complete root-cause proof.

Also inspect the citation link: does it open the cited revision or always the latest file? If the source system cannot provide stable version links, make that limitation visible and at least show the document ID, revision and passage location.

Use a new revision only after complete preparation

Define an observable transition between “detected”, “processed” and “available for answers”.

My recommendation for a bounded workflow:

1. Capture the new revision using its immutable version identifier.
2. Generate all associated passages and metadata.
3. Reconcile processing results with the expected scope.
4. Run the version-sensitive questions against that release.
5. Only then make it available for regular answers.

How to implement a consistent transition depends on your stores and search services. Two sequential updates are not automatically a single coordinated switch.

Important in this example: If V4 already applies but is not reliably available, the application must not silently present V3 as the current rule. For that question, indicate the missing authoritative state or offer the defined clarification route. The query must therefore be able to identify the authoritative revision even when its processing has failed, for example through a separate document registry.

Also define what happens if a version changes during an in-flight request: may the answer explicitly state its captured reference state, or must it be rechecked before delivery?

Eight tests for version selection

This table describes expected behaviour, not tests already passed.

Assess source selection, answer and citation separately. An accidentally correct number supported by an unsuitable source does not pass the citation check.

See RAG evaluation, metrics and golden datasets for building a repeatable test corpus.

Start with one pair of documents

Choose a workflow where using the wrong revision would have a concrete consequence. Prepare two artificial revisions, a draft, and expected answers for current and historical questions.

Trace one failure to the first divergent stage. Then extend the corrected case with a time boundary, incomplete processing and conflicting approvals.

The result should be a small diagnostic record:

• authoritative source and expected revision;
• first divergent processing stage;
• specific correction;
• repeatable test with an expected result.

If you need to scope this assessment for an existing knowledge assistant, the AI Production Readiness & Reliability Sprint is a suitable starting point. A useful assessment scope is one workflow, its version transition, relevant retrieval paths and a traceable failure record.