Where standard RAG gets stuck
Standard RAG finds the text chunks most similar to a question and passes them to the model. That works well when the answer sits in a few specific places: a procedure, a contract clause, a definition in a document.
The Microsoft Research paper that introduced GraphRAG points to two kinds of questions where this approach struggles. The first is global questions about the whole corpus, such as "what are the main themes in these documents?". According to the paper, this is really a query-focused summarization task rather than a retrieval task, so pulling a few similar chunks is not enough. The second, according to the documentation, is questions that require connecting pieces of information spread across different documents and linked only through shared attributes.
- A specific question answered in one document: standard RAG is usually enough.
- A question about themes, trends or the state of the whole corpus: this is what GraphRAG was built for.
- A question that connects people, systems and events across many documents: a graph may help.
How GraphRAG works
At indexing time, according to the documentation, the text is split into units and a language model extracts entities, relationships and key claims from them. These form a knowledge graph. The graph is clustered hierarchically into communities of related entities using the Leiden algorithm, and a summary is written for each community, bottom-up.
At query time there are several modes. Global Search answers questions about the whole corpus from the community summaries. Local Search answers questions about specific entities through their neighbours in the graph. DRIFT Search is similar to Local Search and adds community context. Basic Search is standard vector search, for questions that baseline RAG handles best.
The original paper tested corpora in the range of about one million tokens. The researchers reported substantial improvements over baseline RAG on global sensemaking questions, measured by the comprehensiveness and diversity of the answers. That is the authors' own result on a specific class of questions, not a promise for every corpus.
The price: expensive indexing and maintenance
The official repository warns explicitly that GraphRAG indexing can be an expensive operation and recommends starting small. The reason is clear: a language model reads all of the text to extract entities and relationships and to write summaries. The maintainers also strongly recommend tuning the prompts to your data, because the defaults may not fit. The code is described as a demonstration rather than an officially supported product, and version upgrades may require a migration or re-indexing.
In November 2024 Microsoft Research published a variant called LazyGraphRAG. According to their blog, it extracts concepts with lightweight NLP instead of a language model and defers all model use until query time. They report indexing cost equal to vector RAG and 0.1% of full GraphRAG. With the same configuration they report quality comparable to Global Search at more than 700 times lower query cost. These are figures the company reports about its own work, and they are worth testing on your own data.
What to check in an organization
Permissions. A community summary combines information from several documents. If different users have different permissions, such a summary can expose information from a document the user cannot access. In regular search it is standard to filter results by the user's identity on every query, as in the security filter pattern for Azure AI Search. With a graph this has to be designed up front, for example a separate graph per permission boundary, or Global Search only over content everyone may see. This is my design recommendation, not a built-in capability described in the documentation.
Freshness. When a document changes, the entities, relationships and summaries derived from it need to be updated too. Before choosing the approach, understand how often the corpus changes and what an update costs.
- Who may see each document, and do the summaries respect that?
- How often do the sources change, and what does it cost to update the index?
- Does every answer point to sources that can be checked?
How to start without gambling
Collect 30 to 50 real questions and split them into two groups: focused questions and questions about the overall picture. Run both through standard RAG or hybrid search, and check separately whether the right sources were found and whether the answer relies on them.
If most failures are in the global questions or in questions that connect documents, try GraphRAG or LazyGraphRAG on a small subset of the documents. Measure quality, indexing cost and cost per query against the same question set. If the improvement does not justify the cost, choosing simple RAG is a good decision, not a failure.