The release of GraphRAG from Microsoft Research this week marks a major milestone in the effectiveness of generative AI to continually reason with and answer questions about ever-changing collections of large, unstructured text. Uncharted has been working on applications of this technology for months now and couldn’t be more excited about its potential. This approach promises to significantly outperform existing Large Language models (LLMs) and Retrieval-Augmented Generation (RAG) techniques for interpreting and understanding non-public troves of text.
Balancing flexibility and coherence with LLMs and RAG
LLMs like ChatGPT are trained on extensive datasets to support diverse tasks such as translation, code generation and question answering. These datasets are so large that retraining an LLM to capture the most up-to-date information (for example, the codebase for a new software release) is prohibitively expensive. Without intervention, an out-of-date LLM knowledge base can significantly degrade the accuracy of generated answers.
In contrast to fine-tuning a model to update its knowledge, in-context learning is more dynamic and flexible. Instead of modifying the model’s parameters, it enables an LLM to adapt its output by considering additional information provided within the same interaction; examples, instructions, or earlier parts of the conversation can generate more accurate and relevant responses.
Retrieval-Augmented Generation (RAG) is an in-context learning method that empowers LLMs to stay up-to-date and accurate by tapping into external knowledge bases. RAG segments lengthy text into smaller chunks to fit LLM input limitations. However, this chunking process can compromise the text’s coherence, obscuring meaning or important relationships between different parts of the text. This can cause LLMs to generate responses that are less accurate, relevant, or coherent, as they lack the complete information provided by the uninterrupted text. This can lead to severely degraded performance on tasks that require information from across the entire dataset.
Improving LLM performance with GraphRAG
The GraphRAG approach revolutionizes how LLMs process large collections of unstructured text by first converting them into a structured graph. LLMs can then directly query relevant information scattered across many sources, significantly improving performance on complex queries that require comprehensive understanding of an entire dataset. GraphRAG links crucial concepts across separate texts and external knowledge bases, allowing the LLM to draw on and form connections between disparate sources such as training manuals, customer support tickets, and communications apps. Moreover, GraphRAG provides transparency and trust in results by visualizing subgraphs at query time, effectively surfacing the origins of the data that informed the response.
GraphRAG inherently overcomes limitations of traditional RAG applications that struggle with inconsistent performance due to reliance on simple semantic similarity search. RAG produces inadequate answers when the user-requested topics aren’t explicitly mentioned in the underlying text collection. But GraphRAG, for instance, can effectively respond to questions like “How has company X’s performance changed over the last ten years?” or “What are the key areas for improvement in our customer support tickets over the last five years?” In these examples, traditional RAG would give incomplete answers or outright refuse to answer. As such, GraphRAG offers a domain-agnostic approach that can track policy changes over time, compare policy differences between geolocations, and analyze financial metrics across industries.
What’s next
We can already see the transformative value this approach will bring to many of the domains we work in when coupled with world-class design for provenance and human-in-the-loop intelligence. We’re excited about the potential for new applications, so if you’ve been underwhelmed thus far with the utility of AI in making sense of proprietary document stores, please reach out! We’d love to hear from you.
Read more: