Graph RAG is an extension of Retrieval-Augmented Generation that uses a knowledge graph as the retrieval substrate instead of (or alongside) a vector index. Where vector RAG finds semantically similar text chunks, Graph RAG navigates structured relationships between entities — finding not just what a document says, but how concepts connect across a corpus.
Graph RAG vs vector RAG
Vector RAG retrieves chunks based on embedding similarity to a query. It works well for finding documents that discuss similar topics, but it has no model of relationships between entities. If you ask "which components depend on the auth service," a vector search might return documents that mention auth, but it can't traverse a dependency graph to give you a precise answer.
Graph RAG represents your knowledge as a graph of entities and relationships (nodes and edges). A query can start at a known node and traverse the graph to find related nodes — following "depends on," "is used by," "implements," or any other relationship type. The result is relationship-aware retrieval that can answer structural questions vector search cannot.
The two approaches are complementary. Many production Graph RAG systems use vector similarity to find an entry point into the graph, then graph traversal to gather related context. This combines the flexible topic-finding of vector search with the structural precision of graph traversal.
Knowledge graph construction
The core challenge in Graph RAG is building the knowledge graph. Two main approaches: manual curation (subject matter experts define the entity and relationship types, then populate the graph) and LLM-based extraction (an LLM reads your documents and extracts entities and relationships automatically). Manual curation produces higher-quality graphs; LLM extraction scales better.
For software projects, code analysis tools can construct dependency graphs, call graphs, and inheritance hierarchies automatically — these are already graph structures and can be directly loaded into a graph database. Combining a code graph with an LLM-extracted document graph produces a rich retrieval substrate for coding agents.
Common graph databases used in Graph RAG implementations include Neo4j, Amazon Neptune, and in-memory options like NetworkX for smaller graphs. Microsoft's GraphRAG (open-sourced in 2024) provides a complete pipeline for LLM-based knowledge graph extraction from unstructured documents.
Graph traversal for retrieval
Once the graph is built, retrieval involves traversing it from a query-relevant starting node. Common traversal patterns: depth-limited breadth-first search (gather all nodes within N hops of the starting node); subgraph extraction (extract the connected subgraph containing specific entity types); and path finding (find the path between two entities to understand how they're related).
The retrieved subgraph is then serialized into text — either as a structured list of entity-relationship-entity triples, or as a summary generated by an LLM from the subgraph. This text is included in the model's context alongside the query.
Community summaries and global context
A key innovation in Microsoft's GraphRAG is hierarchical community detection: the knowledge graph is partitioned into communities of closely related entities, and each community gets an LLM-generated summary. Global queries ("what are the main themes in this document corpus?") can be answered by aggregating community summaries, which is impossible with standard vector RAG.
This makes Graph RAG particularly valuable for exploratory, thematic queries over large document corpora — summarizing a company's research output, identifying common failure patterns across incident reports, or mapping the technical landscape of a codebase.
When to use Graph RAG
Graph RAG is worth the additional complexity when your use case involves: relationship-intensive queries ("what depends on X?", "who worked on Y?"); multi-hop reasoning across connected entities; thematic summarization across large corpora; or domain knowledge that is naturally graph-structured (code dependency graphs, knowledge ontologies, organizational hierarchies).
It is probably overkill when your queries are primarily topical (finding documents about a subject), your knowledge base is small enough that full document inclusion is feasible, or your team lacks the graph database expertise to maintain it. For most software project planning use cases, standard vector RAG — or document-first inclusion — is sufficient. See the RAG guide for the simpler baseline.