Choosing a RAG Architecture for Business Knowledge
How I compare semantic search, lexical retrieval, graph retrieval, and a curated wiki for RAG products, with the trade offs that matter when choosing a starting architecture.
Updated October 22, 2026
My RAG journey started with Salesly, a chat assistant that needed to answer visitor questions using a website owner’s own data. The initial pipeline was straightforward: split documents into chunks, retrieve relevant passages, and give the model that context.
As the questions became more varied, I had to look beyond whether a passage was semantically similar. Some questions depended on an exact identifier. Others needed relationships across documents or an approved policy with a clear owner. Those differences are what I now use to choose a retrieval approach.
First, define what each approach means
These labels describe different parts of a system, so they can overlap. Vector RAG and lexical RAG describe retrieval methods. Graph RAG adds explicit relationships. A wiki describes how knowledge is organised and maintained; it can use any of those retrieval methods.
| Approach | Useful starting question | Main responsibility |
|---|---|---|
| Vector retrieval | “Which passage means something similar?” | Good chunks, embeddings, and candidate ranking |
| Lexical retrieval | “Where does this term or identifier appear?” | Tokenisation, ranking, filters, and exact lookup |
| Graph retrieval | “How are these entities connected?” | Entity identity, relationships, and provenance |
| Curated wiki | “Which approved page answers this?” | Ownership, review, and freshness |
The comparison is most useful when it follows actual queries. No row in this table guarantees accurate answers on its own.
1. Vector RAG: finding passages by meaning
A vector pipeline creates embeddings for document chunks and uses a query embedding to retrieve semantically similar content. PostgreSQL with pgvector, or a dedicated vector database, can provide that search path.
This works well when people describe what they need in language different from the documents. A visitor might ask about cancelling a plan while the source page calls it “subscription termination.” Semantic retrieval can bridge that vocabulary gap.
Where it needs more work
Similarity alone does not ensure that the evidence is complete. A question comparing two plans may require passages from both plans and a pricing table. If retrieval returns only one side, the model cannot reliably fill in the missing facts.
Before replacing the architecture, I would check:
- Whether chunks preserve enough surrounding context.
- Whether the query needs more than one retrieval step.
- Whether metadata filters remove irrelevant products or versions.
- Whether reranking improves the selected evidence.
Embedding cost also depends on how updates are handled. Changed chunks can usually be updated selectively; re embedding the entire corpus after every change is a pipeline decision, not an inherent requirement of vector retrieval.
2. Lexical and vectorless RAG: preserving exact terms
Lexical retrieval ranks text using terms in the query. BM25 and PostgreSQL full text search are examples, though PostgreSQL’s built in ranking functions are not BM25. For a product code or order number, a direct database lookup may be even more appropriate.
The wider term vectorless RAG can also include document tree navigation or model guided search. It does not refer to one standard implementation, so I describe the actual retrieval method when discussing a system.
Where lexical search helps
- Product identifiers: SKUs, component names, and model numbers.
- Technical support: exact error messages and configuration keys.
- Policy references: clause numbers or known terms.
Lexical ranking can miss a useful document when the query uses different vocabulary. Synonyms, query expansion, or semantic retrieval can improve recall. A hybrid system combines lexical and vector candidates, then merges or reranks them before constructing the answer context.
3. Graph RAG: retrieving relationships explicitly
Graph retrieval is worth considering when an answer depends on connections between records. For example, a question about which suppliers are associated with recurring incidents may need supplier, component, order, and incident relationships.
A graph can represent those links directly. Retrieval can begin with a known entity, follow relevant edges, and bring the supporting text into context. It can also combine graph expansion with vector or keyword search.
Choose the graph tool for the job
- Neo4j GraphRAG provides tools for knowledge graph construction and several retrieval strategies, including vector search with Cypher expansion.
- LightRAG uses graph structures and retrieval at detailed and broader conceptual levels.
- Graphiti focuses on temporal knowledge graphs, including episodes and facts whose validity changes over time.
A graph makes relationships available to the retrieval system; it does not guarantee correct reasoning. Extracted entities can be duplicated, relationships can be wrong, and missing source links can make a fluent answer hard to verify.
For an exact question such as “What is the status of order 4521?”, I would start with an authorised query against the operational database. A graph or embedding search adds little value to that lookup.
4. A curated wiki: keeping approved knowledge usable
A wiki can work well for policies, onboarding, support procedures, and other material with clear ownership. The useful part is the maintenance process: who owns a page, when it was reviewed, and which version is current.
An LLM still needs a retrieval or context selection step. It does not become a dependable knowledge base merely by receiving a collection of wiki pages.
The main risk is stale knowledge. Review schedules, automated imports, change tracking, and clear ownership can help. Large wikis are possible, but their editorial and retrieval processes need to grow with the corpus.
How I choose a starting architecture
I gather a representative set of real questions and identify the evidence each one needs. Then I build a small baseline and evaluate the results.
- Known identifiers: use a filtered database lookup or lexical search.
- Natural language document questions: test vector and hybrid retrieval.
- Repeated relationship questions: test graph retrieval against the baseline.
- Changing historical context: evaluate temporal modelling and memory tools.
- Approved internal guidance: improve knowledge ownership alongside retrieval.
For every option, I compare source retrieval, answer support, permissions, latency, update cost, and failure handling. Enterprise size alone does not imply a graph is needed, and a small product can still have complex relationship questions.
The production work around retrieval
Whichever method I choose, the application needs a dependable ingestion path and a source of truth. Original documents, metadata, permissions, and versions should remain traceable through parsing, indexing, retrieval, and answer generation.
Heavy ingestion belongs in a recoverable background job. Answers need source references, and the evaluation set should include missing evidence, conflicting documents, and permission boundaries. Those are the cases that reveal more than a polished demo does.
The lesson I keep returning to is that architecture should follow the question and the evidence. When a basic pipeline fails, inspect that failure before deciding which additional component will help.