Graph-Augmented Retrieval for Production RAG in October 2026: Entity Graphs, Hybrid Vector+Graph Paths, and When Pure Similarity Search Is Not Enough

Vector search is very good at one thing: finding passages that sound like the question. That covers a lot of real traffic, but it breaks down on questions whose answer depends on how facts connect rather than on any single chunk. "Which services depend on the library we just flagged?", "Who approved contracts with suppliers that later failed an audit?", "What changed across every incident involving this customer?" None of those answers lives in one paragraph. They live in relationships spread across many documents.

Image: kpzhang via Wikimedia Commons (CC BY-SA 3.0)

Image: Florian Thiery via Wikimedia Commons (CC BY 4.0)

Graph-augmented retrieval adds an explicit layer of entities and relationships next to your vector index, so the retriever can follow links instead of hoping the right chunks happen to be nearby in embedding space. This post covers when that layer is worth building, how to build it without a research project, and how to run hybrid vector-plus-graph retrieval in production.

Where pure similarity search falls short

Before adding a graph, confirm you actually have a graph-shaped problem. These are the common symptoms:

• Multi-hop questions. The answer needs two or more linked facts: A owns B, B depends on C, C had an outage. Each fact is in a different document, and none of them is semantically close to the question on its own.
• Aggregation over many items. "List every vendor that…" or "Summarize all decisions about…" Top-k retrieval returns k chunks, not "all of them," and the model quietly answers from a partial set.
• Entity ambiguity. "Mercury" the project, "Mercury" the vendor, and "Mercury" the internal codename all embed similarly. Similarity search mixes them; a graph with resolved entity IDs does not.
• Global or thematic questions. "What are the main risks raised across these 400 reports?" No individual chunk answers this. You need structure built ahead of time.

If your evaluation set is mostly single-fact lookups ("What is the refund window?"), a well-tuned hybrid keyword-plus-vector retriever with reranking is usually enough, and a graph adds cost without much benefit. Look at your failing queries first and tag which ones are multi-hop or aggregation questions. That tag count is your business case.

The basic architecture

A practical graph-augmented RAG system has three stores that point at each other:

1. Chunk store with embeddings. Your existing vector index. Each chunk keeps a stable chunk_id and source document ID.
2. Entity graph. Nodes are resolved entities (people, systems, products, organizations, contracts). Edges are typed relationships (DEPENDS_ON, OWNED_BY, SIGNED, MENTIONED_IN). Every edge carries provenance: the chunk_ids it was extracted from.
3. Optional summaries. Precomputed summaries for entities or clusters of related entities ("communities"), used for broad, thematic questions. This is the idea popularized by Microsoft's open-source GraphRAG project, which builds community summaries for "global" queries and uses entity neighborhoods for "local" ones.

The provenance link is the most important design decision. If an edge cannot point back to the text it came from, you cannot cite it, debug it, or delete it when the source document is removed.

Building the entity graph

Start with a small, fixed schema

The fastest way to get a useless graph is to let an LLM extract "any entities and relationships" from every document. You end up with thousands of edge types like is_related_to, works_with, and mentions that cannot be queried reliably.

Instead, define the handful of entity and relation types your failing questions actually need. For an internal engineering knowledge base, that might be:

Entities: Service, Team, Person, Library, Incident
Relations:
  Service  -DEPENDS_ON->  Service | Library
  Service  -OWNED_BY->    Team
  Person   -MEMBER_OF->   Team
  Incident -AFFECTED->    Service
  Incident -CAUSED_BY->   Library | Service

A small schema makes extraction prompts shorter, outputs easier to validate, and graph queries predictable.

Extract with structured output and validate

Run extraction per chunk with a strict JSON schema: a list of entities (type, name, aliases) and a list of relations (type, source, target). Reject anything that does not match the schema rather than trying to repair it. Store the raw extraction alongside the chunk ID so you can re-run resolution later without re-calling the model.

Where structured sources already exist, use them instead of an LLM. A service catalog, a CMDB, an HR directory, or a dependency manifest gives you exact edges for free. LLM extraction is for the relationships that only exist in prose.

Entity resolution is where the quality lives

"Payments API," "payments-api," "the payments service," and "PaymentsSvc" should become one node. A workable resolution pipeline:

1. Normalize names (case, punctuation, common suffixes).
2. Match against a canonical registry if you have one (service catalog, customer IDs).
3. For the rest, generate candidates with embedding similarity on name plus short context, then confirm with rules or a small classifier.
4. Keep an alias table so new mentions resolve quickly and humans can correct mistakes once.

Bad merges are worse than missed merges. Merging two different "Mercury" entities poisons every query that touches either one. Set conservative thresholds and send uncertain cases to a review queue.

Hybrid retrieval: vector and graph together

In production you rarely want graph-only retrieval. The common pattern is to use vector search to find entry points and the graph to expand from them.

A typical query path

1. Link entities in the question. Run the same resolver on the user query. "What depends on log4j-core?" links to the Library node for log4j-core.
2. Run normal hybrid search in parallel. Keyword plus vector over chunks, as you already do.
3. Expand in the graph. From linked entities, traverse a bounded number of hops along allowed relation types.
4. Collect evidence. For each node and edge you reach, pull the chunks in its provenance list.
5. Merge and rerank. Combine graph-sourced chunks with vector results, deduplicate by chunk_id, and rerank against the question.
6. Give the model structure plus text. Include a compact list of the traversed facts and the supporting chunks, so the answer can cite both.

In pseudocode:

def retrieve(question, user):
    entities = link_entities(question)          # resolved node IDs
    text_hits = hybrid_search(question, k=20)   # BM25 + vectors

    paths = graph.traverse(
        start=entities,
        relations=["DEPENDS_ON", "AFFECTED", "OWNED_BY"],
        max_hops=2,
        max_nodes=200,
    )
    graph_chunks = chunks_for(paths.provenance_ids)

    candidates = dedupe(text_hits + graph_chunks)
    candidates = filter_by_acl(candidates, user)
    ranked = rerank(question, candidates)[:12]

    facts = summarize_paths(paths, limit=30)    # "A -DEPENDS_ON-> B" lines
    return facts, ranked

Keep traversal bounded

Graphs fan out quickly. A popular node such as a shared logging library or a large team can connect to most of the graph in two hops. Always set a hop limit, a node budget, and an allowlist of relation types per query type. If a traversal hits its budget, record that in the trace and tell the model the result may be incomplete, rather than silently truncating.

Route by question type

Not every question should touch the graph. A lightweight router, whether rules, a small classifier, or a cheap model call, can send:

• single-fact lookups to plain hybrid search,
• multi-hop and "what is connected to X" questions to vector plus graph expansion,
• broad thematic questions to precomputed community or entity summaries.

Routing keeps latency and cost down for the majority of traffic and limits graph-related failure modes to the queries that need them.

Keeping the graph correct over time

The graph is derived data, and derived data goes stale. Plan for these from day one:

• Incremental updates. When a document changes, re-extract its chunks, diff the resulting edges, and update only those. Edges whose provenance list becomes empty should be deleted.
• Deletions. Because every edge points to source chunks, removing a document can remove its contribution to the graph. Without provenance, deleted content can keep influencing answers through edges nobody can trace.
• Permissions. An edge extracted from a restricted document is itself restricted. Carry access labels from chunks to edges, and filter traversal results by the requesting user's permissions, not only the final chunks. Otherwise the fact list can leak information even when the underlying text is filtered.
• Versioning. Record which extraction prompt, model, and schema version produced each edge. When you change any of them, you can re-extract a sample and compare before reprocessing everything.

Storage choices

You do not need a dedicated graph database to start. Options, roughly in order of setup effort:

• Relational tables. An entities table and an edges table in Postgres, queried with recursive CTEs, handle bounded two-hop traversals well at moderate scale and keep everything next to your existing data.
• A graph database. Neo4j, Amazon Neptune, and similar systems make deeper or more varied traversals easier to express and often support vector indexes alongside the graph.
• Framework-managed stores. Libraries such as LlamaIndex and LangChain include property-graph and knowledge-graph retriever components, and Microsoft GraphRAG ships its own indexing pipeline. These are useful for prototyping, but review their extraction prompts and storage format before relying on them in production.

Choose based on the traversals you actually run. If every query is "two hops along three relation types," a relational design is simpler to operate.

Evaluating whether the graph helps

Measure the graph layer separately from the rest of the pipeline:

1. Build a tagged eval set. Include single-hop, multi-hop, aggregation, and ambiguous-entity questions, each with expected supporting documents.
2. Compare retrieval recall. For each question type, check whether the expected documents appear in the final context with and without graph expansion.
3. Check extraction quality on a sample. Have someone review a few hundred extracted edges for correctness and a few dozen resolved entities for bad merges.
4. Track cost and latency per route. Graph expansion adds a resolver call, a traversal, and more chunks to rerank. Make sure the improvement on multi-hop questions justifies that cost.

If graph expansion improves multi-hop recall but hurts single-hop answers by adding noise, that is a routing problem, not a reason to drop the graph.

A sensible rollout plan

1. Collect failing questions and tag the multi-hop and aggregation ones.
2. Define a schema with five or fewer entity types and a handful of relations that cover those questions.
3. Load structured sources first, then add LLM extraction for prose, with provenance on every edge.
4. Build entity resolution with an alias table and a review queue.
5. Add bounded graph expansion behind a router, only for graph-shaped questions.
6. Compare against your current retriever on the tagged eval set before widening rollout.

Bottom line

Similarity search finds text that resembles the question. Graph-augmented retrieval finds text that is connected to it. Add a graph when your failures are about relationships, aggregation, or entity confusion; keep it small, typed, and tied to source chunks; bound every traversal; and route only the questions that need it. Done that way, the graph becomes a precise tool for the hard queries instead of an expensive layer every request has to pass through.