By Editorial — IT Researches
Retrieval-augmented generation entered the mainstream of enterprise AI conversation in 2023 and has, by 2025, settled into a small number of patterns that work reliably and a slightly larger number that do not. The promise — let a generative model answer questions about documents it was never trained on — is real and operationally durable. The failure modes are equally real and worth naming.
The pattern we now reach for by default has five components. A document ingestion pipeline that produces clean, chunked, normalised passages with explicit provenance metadata. A retrieval layer that combines lexical and dense retrieval with a reranker on top — pure-vector retrieval is a category error in most enterprise corpora. A response synthesis layer with strict instructions to cite the retrieved passages and to refuse the question when the retrieval is empty or contradictory. An evaluation harness that scores answer quality, citation quality and refusal quality independently. And a feedback channel that lets users flag wrong answers, with the flagged answers feeding both the retrieval debug log and the next round of evaluation.
The failure mode we see most often is the silent retrieval gap: the system answers fluently from the wrong subset of the corpus because the right subset was not retrieved. The remedy is unromantic — better chunking, better metadata, hybrid retrieval, a reranker — and it cannot be skipped. We have lost count of the RAG systems we have been asked to fix where the diagnosis was not the model and not the prompt but the retrieval, and the fix was always the same.
The second failure mode is hallucinated citations: the model produces a citation to a real source that does not actually support the claim. The fix is procedural rather than architectural — a citation-checking step that retrieves the cited passage and a small entailment model that verifies the answer against it — and it is now cheap enough to run on every response in production.
The third failure mode is the "helpful refusal" gap. Most enterprise RAG systems are tuned, by default, to be helpful. Helpful is the wrong default for a system that will be asked questions outside its competence. We tune for honest refusal aggressively, and we measure it as carefully as we measure answer quality. A system that refuses 8 per cent of questions correctly is, in operational terms, a much better system than one that answers 100 per cent of them with a 5 per cent hallucination rate.
The fourth, more strategic failure mode is treating RAG as a destination rather than a stepping stone. For a class of high-value workflows, the right end state is not RAG at all — it is structured retrieval against a curated knowledge graph, a deterministic answer template, and the language model used only for surface generation. RAG is a useful default; it is not a final architecture for every problem, and the best teams know when to graduate.