Clausey
Enterprise agentic RAG is an evidence system with an LLM on top. That is the whole thesis. The model is the final reasoning layer — thin, replaceable, and the part everyone talks about. The enterprise value comes from the six layers underneath it that create, select, verify, and protect the evidence it reasons over. Get those right and the LLM becomes an enterprise interface. Get them wrong and it becomes a liability that produces confident, unverifiable answers at scale.
This is the production architecture, from messy documents to grounded, cited, measurable answers: how ingestion creates trustworthy evidence, how retrieval selects it, how the agent reasons over it, and how you keep the whole thing honest. The full 50-slide deck is below, and the written breakdown follows.
Why is "chat over docs" not enough?
The default RAG demo is a straight line: upload a PDF, split the text, embed the chunks, run a vector search, stuff the results into a prompt, and hope. It demos beautifully. It breaks in six places at once under enterprise conditions.
Parsing loses text, tables, and page numbers. Chunking splits clauses in bad places. Vector search misses exact terms, dates, and names. The model infers answers the evidence does not support. Citations end up decorative rather than verifiable. And failures are undebuggable, so quality drifts silently.
Enterprise buyers need something else entirely. They need correctness and traceability — right answers, cited to the exact source, every time. They need isolation and auditability: multi-tenant boundaries, audit logs, deletion and retention controls. They need deterministic failure — retry safety, visible failure states, no silent data loss. And they need quality on a scoreboard: continuous evals, cost visibility, operational monitoring. Knowledge bases have to behave like reliable systems, not experiments.
The core thesis: an evidence system with an LLM on top
Stack the layers and the picture is clear. At the top sits the LLM and its agent loop. Beneath it: grounding and citations, retrieval tools, indexes and metadata, document processing, a durable workflow engine, and storage, security and audit at the base.
One thin layer reasons. Six layers create, select, verify, and protect the evidence it reasons over. That ratio is the point — and it explains why swapping models rarely fixes a knowledge base that is failing.
Two of those layers are the hardest, and everything else exists to support them. Ingestion creates trustworthy evidence; if it fails, the right evidence never exists at all. Retrieval selects the right evidence; if it fails, the evidence exists but is never seen. Only then does the agent reason over it.
Chapter 1 · Ingestion creates the evidence
The LLM cannot recover from missing or mangled source text. Bad OCR produces bad chunks, which produce bad embeddings, which produce bad retrieval, which produces an unsupported answer and a user who no longer trusts the system. Enterprise documents carry tables, scans, headers, footers, signatures, and boilerplate, and the system has to preserve enough structure to support retrieval and citations later. You win or lose at document processing.
Ingestion should be a workflow, not a function
Fire-and-forget ingestion is where silent data loss comes from. Model it as a durable workflow instead — queued → claimed → running → succeeded, with explicit branches to failed → retry queued and dead_letter. Every step has status and can be retried, failures are visible rather than silent, workers can crash without losing state, leases and reclaim jobs recover stuck work, and operators can inspect the exact failed step. Large payloads move by reference — the claim-check pattern — so the workflow state never balloons.
This is the same durability argument from Part 3 of the agents series, applied to the document pipeline: state that outlives the process.
Route by file type, and fail loudly
Different files need different ingestion paths, because one generic parser hides failure. Text, Markdown and HTML take a decode-and-sanitize path that keeps headings. PDFs and scans go through OCR plus layout extraction. Word and PowerPoint get converted, then extracted. Excel and CSV take a structured table path. And anything unsupported produces a clear, explicit failure — never a silent pass.
OCR is the foundation for citations
OCR is not just text extraction. It is what makes citations possible at all. You need page boundaries and reading order, headings and table text, coordinates and bounding boxes, and confidence and failure signals. Citations need a location. Context expansion needs neighbours. Viewers need highlightable regions. Bad OCR produces confident but false retrieval — the worst failure mode there is, because nothing looks broken.
Normalization then has to remove junk without destroying meaning: clean enough to search, faithful enough to cite. Strip unsafe HTML and scripts, clean Markdown artifacts, normalize whitespace, preserve headings and page markers, detect empty output early, and never keep OCR garbage as if it were meaning.
The chunk is the atomic evidence unit
Splitting every N tokens shreds clauses mid-sentence, orphans headings, and destroys tables. Structure-aware chunking respects page boundaries, tracks section hierarchy, preserves headings, merges tiny fragments, keeps a chunk index, and attaches parent and child ranges.
Then every chunk should know exactly where it came from — workspace and document IDs, chunk index, pages, section path, parent range, source file, and a location with page and bounding box. That metadata is what unlocks filtering and deduping, context expansion and full-document reconstruction, machine-verifiable citations, audit trails, and clean deletion and re-ingestion.
Note what this makes of embeddings: they are one index over the knowledge object, not the knowledge base itself. Alongside the vector index sit a full-text index for names, dates and clause labels, metadata indexes for filtering by scope and tenant, and document tables carrying status and security — the deterministic path.
Measure ingestion by its ugly cases
Enterprise quality is measured by the failures, not the demos. OCR returning empty text looks searchable but isn't — detect empty extraction and fail early. OCR losing page order makes citations unreliable — preserve page markers and layout. Chunking that splits clauses badly yields partial evidence. Embedding inserts that retry partially create duplicate chunks — use idempotent chunk keys and upserts. Huge payloads in workflow state cause timeouts and OOM — use claim-check storage. Layout mapping failures mean the answer works but the highlight doesn't — degrade gracefully and mark confidence. And vendor OCR that retains data is a compliance concern that needs a retention review and a deletion story.
Chapter 2 · Retrieval selects the evidence
Retrieval is not one search call. Sending every question through a vector search and hoping is the failure pattern; the better shape is classify the need, choose the tool, then retrieve, expand, read or query. The agent's first job is deciding what kind of evidence the question requires.
Look at how different the strategies are. "What is the renewal date?" is a search for a fact. "Show me the indemnity clause" is keyword search plus a section read. "Summarize this agreement" needs a full-document read; "compare these two contracts" needs both documents read. "Which docs mention force majeure?" searches across the corpus. "How many active students?" is a tabular query. "Find contracts missing a field" is a metadata or extraction query. One search tool cannot serve that spread.
Retrieval should match the shape of the knowledge base
Knowledge bases come in three topologies. Folder-shaped ones — policies, contracts, manuals — carry signal in their hierarchy, and if the KB already looks like a filesystem, let the agent use filesystem-like tools: list, open, read, navigate, search in scope, then cite the exact file, section and page. Pair navigation with hybrid search and filters, because names get ambiguous and files get misplaced. Table-shaped data wants schema, filter, count, group-by and export. Graph-shaped data — entities, relationships, dependencies, ownership — wants neighbours and paths.
GraphRAG tools earn their place when the answer depends on relationships rather than text similarity: vendor and subprocessor maps, policy dependencies, shared counterparties, compliance mapping. But they are a complement to hybrid search, never a replacement. Extraction is expensive, entity resolution is hard, false edges create false confidence, graphs go stale, and graph answers still need document citations.
Hybrid search, fusion, and reranking
No single retrieval signal is trustworthy alone. Dense semantic search finds concepts — "early termination rights" finds "termination for convenience." Sparse full-text finds exact evidence: names, dates, clause labels, identifiers. Fuzzy matching survives OCR noise and typos. Metadata filters enforce scope and security. Semantic search finds concepts; keyword search finds evidence. The best systems use both.
Combining them well means fusing rankings rather than betting on one signal. Reciprocal rank fusion rewards chunks that rank well everywhere — a chunk placing highly in dense, sparse and fuzzy rankings surfaces at the top of the fused set. Then rerank: retrieve broadly across 20–100 candidates to optimize recall, score them against the query to optimize precision, and hand the top 5–10 evidence chunks to the agent.
Retrieval is a security surface
This deserves emphasis, because it is easy to treat search as a pure quality problem. A search bug can be a data leak. Every query has to enforce the workspace boundary and user permissions, exclude soft-deleted documents, scope by document ID, type and status, and distinguish tabular from prose. Evidence from another tenant must never cross the boundary — not as a ranking penalty, but structurally.
Search, read, and expand are different operations
Snippet search is the wrong tool for whole-document tasks. Use search_documents for one fact or clause, for locating where a topic appears, and for searching across many documents. Use read_full_document for summarization and comparison, for extracting many terms at once, and for tasks where a missed clause changes the answer. Repeated search calls cannot reliably reconstruct an entire document.
And the right answer often lives near the retrieved chunk rather than inside it. Legal terms are defined elsewhere, exceptions follow general clauses, tables continue across pages, and OCR boundaries can be awkward — so expanding to neighbouring chunks turns a partial hit into a complete one.
Citations should be machine-derived
The bad pattern is the LLM writing the answer and inventing the citation text alongside it. The better one: tools return structured sources, the system harvests the candidates, and the answer is linked to a real document, page and chunk — so a citation reads Master Services Agreement.pdf · p.7 · chunk 42, quotes the actual clause, carries its bounding box, and can be clicked to verify.
When each retrieval stage is observable, failures become diagnosable: correct doc but wrong chunk (fix with hybrid, rerank, expansion), correct chunk but no context (expand surrounding chunks), vector missing an exact phrase (add the keyword path), the agent looping on search (use a full-document read), the reranker suppressing the answer (golden evals and reranker tests), a citation weakly supporting the answer (citation faithfulness eval), and an answer inferred but absent (refusal and grounding evals).
Chapter 3 · The agentic layer
The agent turns a request into evidence-gathering actions: plan, choose a tool, retrieve or read or query, inspect, ground, and answer — repeating until the evidence suffices, bounded by iteration and cost caps. It chooses search versus read versus query, batches independent reads, stops when enough evidence exists, and says "not found" when evidence is missing.
That requires a tool catalog rather than one search function: discovery tools, semantic retrieval, whole-document reading, context repair, structured data, extraction, optional graph tools, and utilities. This is the same idea as subagents as tools from Part 2 — the agent's power comes from having the right operation available, not from a bigger prompt.
The grounding gate
Here is the piece most systems skip. No workspace factual claim ships without workspace evidence, and that has to be enforced structurally, not by prompt. The model drafts an answer; a gate asks whether it asserts workspace facts and whether real tool evidence exists. Grounded answers ship with citations. Ungrounded ones force a tool call or fail closed.
Prompt-only grounding fails for predictable reasons: models can ignore instructions, models pattern-complete plausible facts, web results are not workspace truth, and final answers need to be structurally constrained rather than politely requested.
Chapter 4 · Evals and operations
Enterprise RAG needs a quality scoreboard, not a vibe check. A golden question carries an expected answer and an expected source — document, page, and text that must appear — so you are testing retrieval and grounding, not just prose. Run them on every change to prompts, models and retrieval, and track retrieval recall, answer accuracy, citation faithfulness, refusal correctness, tool selection, latency and cost. Regressions then surface before users find them.
The eval taxonomy has seven layers, and each answers a different question. Data evals: can it read the data? Retrieval evals: can it find the right evidence? Reasoning evals: can it answer from the evidence? Tool evals: can it choose the right action? Grounding evals: can it prove the answer? Safety evals: can it stay within policy? Ops evals: can it run reliably and economically?
Production quality is operational quality too, and every one of these is a dashboard rather than a hope: cost per answer, latency p50/p95 and timeout rate, retry rate per workflow step, dead-letter rate, OCR failure rate by file type and vendor, retrieval no-hit rate, citation missing rate, grounding rejections, and tool call count per answer.
Security sits on the same surface. Knowledge bases are high-risk precisely because they centralize sensitive data, so workspace isolation with RBAC and row-level security, soft-delete exclusion in every query, audit logs on every access, encryption and secret handling, service-role boundaries, purge and deletion semantics, vendor retention review, and prompt and tool leakage prevention all belong in the design rather than the roadmap. That governance-by-construction argument is the one we made in The Model Is No Longer the Moat.
Chapter 5 · Principles, anti-patterns, and readiness
The strongest design choices are the ones that reduce invisible failure: durable workflow ingestion, claim-check storage for large payloads, smart chunk metadata, retrieval tools matched to KB topology, a structural grounding gate, cost and iteration caps, hybrid search instead of pure vector, reranking and context expansion, a full-document read tool, structured table querying, code-harvested citations, and explicit failure states everywhere.
The anti-patterns are worth naming plainly, because each one is a common default that produces confident, unverifiable answers at scale: prompt-only hallucination prevention, treating all documents as simple prose, assuming OCR text is enough without layout, retry paths without strict idempotency, ignoring cost until after the system is useful, repeated search calls for compare and summarize tasks, flattening folders and graphs into one vector pool, citations as text generated by the model, and relying on demos instead of evals.
Nine questions that decide enterprise-readiness
Enterprise-grade means having answers to the predictable hard questions. Can ingestion recover from partial failure? Are writes idempotent? Can you explain why an answer was produced? Can you show the source document and page? Can you measure retrieval quality? Can you prove tenant isolation? Can you delete customer data completely? Can you cap cost and latency? Can you detect regressions before users do?
If a vendor cannot answer those nine, the system is a demo. Quality is not a launch state — it is a standing practice, kept honest by a golden-question eval suite on every change, ingestion-to-answer end-to-end tests, citation click-through validation, deterministic chunk IDs and idempotent writes, citations persisted across agent wake and subagents, vendor retention and deletion reviews, a retrieval quality dashboard, cost and latency accounting per answer, and failure states surfaced rather than swallowed.
Seven claims, one system
Durable ingestion creates trustworthy evidence. Hybrid retrieval selects the right evidence. Agent tools choose the right operation. Grounding gates block unsupported claims. Citations prove the answer. Evals measure the system continuously. Operations keep it reliable.
Enterprise agentic RAG is a system that can ingest reliably, retrieve precisely, reason with tools, prove its answers, and measure itself continuously. If the evidence layer is weak, the LLM becomes a liability. If the evidence layer is strong, the LLM becomes an enterprise interface.
Where Clausey fits
This is the architecture Clausey is built on, not a survey of the field. Every layer described here — durable ingestion, structure-aware chunking with provenance, hybrid retrieval with reranking, a tool catalog, the grounding gate, golden-question evals — is what makes it possible for Clausey to read your contracts, records and policies and answer with citations you can click and verify.
It is also why our answers come with receipts rather than confidence. The evidence layer was the product all along.