Retrieval augmented generation is the standard way to put an LLM on top of a document corpus, and on most corpora a semantic vector index is enough. Financial documents are not most corpora. Ask a model what a 10-K says about a specific revenue segment, and the answer is only useful if it lands on the right line of the right filing, for the right entity, as of the right date.
The stakes are higher than they are for a support knowledge base. An ungrounded paragraph that a chatbot serves a customer is an annoyance. The same paragraph in front of an analyst writing a note, or a reviewer signing off on it, can end up in a published view or a committee memo. By the time someone catches the error, it has already travelled. So the retrieval layer has to do more than return something relevant. It has to return the exact line, and let a human check it.
Where pure semantic search breaks
A ticker, a CUSIP, a section reference like “Item 1A”, a defined term carried across a contract: these are tokens that have to match exactly, and an embedding model treats them as approximately similar to a dozen near neighbours. That is the whole point of an embedding, and it is the wrong behaviour here. Search for AAPL and a pure vector index may happily surface a passage about a different issuer that reads similarly. Quarter labels, basis-point figures and footnote markers get the same blurring.
That is why hybrid RAG, vector search combined with keyword and BM25 matching, is the production baseline in 2026 for RAG over financial documents. The keyword side pins down the exact identifiers; the vector side handles the paraphrased question that no keyword would catch. You need both, because financial questions are usually a mix of the two: “what did management say about margin pressure in the Q3 call” has a fuzzy clause and a hard one in the same sentence.
What hybrid adds, and how we chunk
The retrieval design is only half the work. How you split the documents decides whether the right line is even retrievable. Our defaults across filings, earnings-call transcripts and analyst reports:
- Keep tables intact. Splitting a table mid-row strands a number from its column header, and a retrieved figure without its header is worse than no figure.
- Bind footnotes to their anchor. A footnote read away from the line it qualifies will mislead the model and the reader both.
- Carry entity and period metadata on every chunk, so a query can be filtered to one issuer and one reporting period before ranking ever runs.
- Preserve transcript turn structure, so an answer attributes a remark to the CFO rather than to the analyst who asked the question.
Hybrid ranking then runs over chunks that carry this structure with them. The two result sets get fused, and the top passages go to the model with their source identifiers attached.
Letting the analyst click back to the source
A source-grounded answer is one where every claim points to the line it came from. We carry the document id, the section or transcript timestamp, and the character span through retrieval and into generation, so the model cites the span it actually used rather than a document-level reference. The analyst reading the answer can click straight to the filing line or the moment in the call and confirm it.
This matters beyond convenience. Traceable sourcing is what lets the same pipeline serve research and review. When someone asks why the system said what it said, you hand them a link rather than starting an investigation. It also gives you something to evaluate against: an eval set of questions with known source lines tells you whether retrieval is finding the right passage, separately from whether the model is phrasing it well. Those two failures have different fixes, and you want to see them apart.
If you are putting an LLM over filings and calls and the answers cannot be traced to a line, the retrieval layer is where to start. Go hybrid so the exact identifiers stop getting blurred, chunk around the structure these documents actually have, and keep the citation attached the whole way through so a reviewer can check the work.