Connect a Document Management System to RAG

A document management system often contains exactly the knowledge organisations want to make easier to access with AI: policies, contracts, technical documentation, offers, product documents, meeting records and approved versions.

Simply “vectorising the DMS” is not enough.

Production DMS-RAG needs to solve document identity, versions, approval state, OCR/layout, metadata, permissions, synchronization and citations together.

Fraunhofer IAO published a 2026 study specifically on using RAG to make enterprise documents accessible through generative AI. Austria's WKO likewise describes RAG as a way to connect modern AI systems with current company knowledge. Sources: Fraunhofer IAO – Enterprise documents with RAG, WKO – Retrieval Augmented Generation.

The short version

A robust reference architecture looks more like:

DMS → Connector/API → Parsing/OCR → Document identity & version → Metadata & ACL → Index/Search → Retrieval → LLM → Answer with source

Not every step needs a separate product. Every step does need clear ownership.

Why DMS-RAG is harder than a file upload

A demo RAG system may work with 20 PDFs.

A production DMS often contains:

• multiple versions of the same document,
• drafts and approved versions,
• scans,
• tables,
• attachments,
• folder/project metadata,
• user and group permissions,
• retention/archive rules,
• deleted or superseded documents.

The central question is therefore not:

“How do we put PDFs into a vector database?”

It is:

“How do we preserve the meaning and governance of the DMS inside the AI system?”

1. Remote retrieval or a separate index?

There are two main patterns.

Option A – remote retrieval

The AI application queries the DMS or its search API at runtime.

Advantages

• fewer data copies,
• current source,
• permissions may remain in the source,
• less index infrastructure.

Limitations

• dependent on DMS search quality and API,
• latency,
• limited ranking/chunking control,
• harder multi-source retrieval.

Option B – replicated RAG index

Documents and relevant metadata are synchronized into a separate retrieval layer.

Advantages

• custom chunking,
• hybrid search,
• reranking,
• multi-source retrieval,
• controlled retrieval evaluation.

Limitations

• synchronization,
• ACL replication,
• delete propagation,
• additional operations,
• stale-copy risk.

Decision matrix

| Situation | Tendency | | --- | --- | | DMS has strong search API + identity | evaluate remote retrieval | | several sources need joint search | central retrieval layer | | specialised chunking required | separate index | | minimise data copies | remote retrieval | | weak DMS search | separate index more likely | | strict freshness requirement | remote or robust incremental sync | | custom ranking/evaluation | separate index |

Hybrid approaches are also possible: candidates or metadata can come from the DMS while detailed retrieval happens elsewhere, or vice versa.

2. Parsing and OCR are not side issues

If extraction is wrong, retrieval cannot be correct.

DMS content often includes:

• scanned PDFs,
• multi-column documents,
• tables,
• forms,
• headers/footers,
• embedded attachments,
• images containing text.

Typical failures

• table values lose column relationships,
• headings separate from their sections,
• OCR changes numbers or product codes,
• headers/footers appear in every chunk,
• page ordering is misread.

Treat parsing as its own quality layer.

Chunking comes after extraction: RAG Chunking Strategies.

3. Document identity must remain stable

A DMS document needs a stable identity across changes.

Example:

• document_id = 4711
• version 7 = draft
• version 8 = approved
• version 9 = current approved version

The RAG index should not treat all three as independent authoritative documents if only version 9 is valid.

Useful fields include:

• Document ID,
• Version ID,
• Status,
• Title,
• Document Type,
• Source URL,
• Owner,
• Modified At,
• Valid From / To,
• Retention State.

4. Approval state is retrieval logic

Many DMS platforms use states such as:

• Draft,
• Review,
• Approved,
• Archived,
• Superseded.

An AI system should not automatically cite a draft when an approved version exists.

Therefore:

status and version logic belongs in filtering and retrieval, not only in the UI.

5. Permissions are part of index design

DMS permissions can involve:

• users,
• groups,
• roles,
• folder inheritance,
• project membership,
• document classification,
• tenants.

Production RAG needs an explicit permission model.

Common patterns:

Early binding

ACL information is indexed with the document/chunk and applied during retrieval.

Late binding

The source or an authorisation service validates access at runtime.

Hybrid

Some permission data lives in the retrieval layer while critical checks are also performed against the source.

Details: RAG permissions & security trimming.

6. Synchronization: changes, versions and deletions

A one-time full index is not a production synchronization strategy.

Depending on the DMS, use:

• change feed,
• modified timestamp,
• webhooks,
• delta API,
• polling,
• checkpoints.

For every changed object, distinguish:

• new,
• changed,
• newly approved,
• archived,
• permission change,
• deleted.

Delete propagation

A critical failure case:

A document is deleted or access is removed in the DMS, but it remains available in the RAG index.

The AI can then surface evidence that no longer exists or is no longer authorised.

Deletions and permission changes therefore need the same engineering attention as new documents.

7. Metadata is more than filtering

Metadata supports:

• retrieval,
• ranking,
• permissions,
• citations,
• versioning,
• debugging.

Example:

A query asks for the “current Austrian travel-expense policy”.

Metadata can narrow retrieval by:

• document type = Policy,
• country = AT,
• status = Approved,
• validity = current.

That improves both precision and explainability.

8. Combine keyword and vector search

DMS content often contains exact terms:

• document numbers,
• product codes,
• project IDs,
• contract numbers,
• names,
• domain terminology.

Pure vector retrieval can therefore be insufficient.

Hybrid Search vs Vector Search explains how lexical and semantic retrieval can be combined.

9. Treat tables and structured content separately

Not every piece of information should become a prose chunk.

Examples:

• price lists,
• product matrices,
• contractual parameters,
• technical tables.

If table structure is business-critical, consider:

• structured extraction,
• indexing rows as objects,
• querying structured data through API/SQL.

RAG and controlled tool/API retrieval can coexist.

10. Citations should reach the original source

Good DMS-RAG should not only say:

“Source: Document X”

Ideally it includes:

• document title,
• version,
• page/section,
• link back to the DMS,
• date/approval status where relevant.

Users should be able to verify the answer against the original.

Citation accuracy therefore belongs in evaluation.

11. DMS-native AI or custom RAG?

Many DMS vendors are adding native AI and search features.

That does not automatically mean custom RAG is better.

Compare:

| Criterion | DMS-native AI | Custom / central RAG | | --- | --- | --- | | DMS-only content | often strong | possible | | multiple sources | product-dependent | usually more flexible | | permissions | often native | must be replicated/integrated | | custom retrieval tuning | often limited | high | | model choice | product-dependent | higher | | custom evaluation | product-dependent | easier to control | | integration into workflows | product-dependent | flexible | | vendor lock-in | may be higher | can be more modular |

The right answer depends on the actual DMS and use case.

12. Evaluate DMS-RAG

A golden dataset should contain realistic document questions.

Example test cases:

• clear question answered by current approved version,
• old version must not be used,
• user without access,
• scanned document requiring OCR,
• table,
• documents with similar titles,
• deleted document,
• new version after synchronization,
• question with no reliable source.

Measure at least:

• retrieval recall,
• ranking,
• groundedness,
• correctness,
• citation accuracy,
• permission correctness,
• freshness.

Methodology: RAG Evaluation.

DMS-RAG checklist

Before implementation, answer:

1. Which document types need to be searchable?
2. Are document IDs stable?
3. How do versions work?
4. Which statuses may be cited?
5. How are permissions represented?
6. How are changes detected?
7. How are deletions propagated?
8. How reliable is parsing/OCR?
9. Which metadata matters?
10. Is native DMS search enough or do we need a separate retrieval layer?
11. How does the user reach the original source?
12. How will quality be tested continuously?

Anti-patterns

“Export every PDF and embed it”

Versioning, permissions and deletion semantics are easily lost.

Full reindex only

The system operates on stale information between rebuilds.

Post-filtering as security

Unauthorised content should not reach downstream AI components before access is checked.

Treat every table as text

Structured relationships can disappear.

Ignore DMS-native AI

Custom development only makes sense when it creates additional value.

Next step

If your organisation already has a DMS and wants to expose document knowledge through RAG or AI search, DMS semantics and AI retrieval should be designed together.

The work involves more than a connector: parsing, document identity, permissions, versions, synchronization, retrieval and evaluation.

See RAG implementation & AI integration.

For the broader decision between DMS-native AI, a RAG platform and a custom architecture, see AI consulting & solution architecture.