We are living in the age of hyper-personalisation. Expectations for highly personalised, accurate, efficient and convenient services are growing — especially in the super and wealth sector. This age of personalisation began with “old” AI. The emergence of generative artificial intelligence (gen AI) has added fuel to the fire, making hyper-personalisation easier and more accessible. While gen AI has come a long way in the last 12 months, it has some way to go before it becomes the irreplaceable tool AI evangelists promise it will be — especially when it comes to personalisation of services. It still struggles with some incongruously simple problems, especially in a business context.
There are a few emerging technologies with the potential to solve these issues, especially in the super and wealth management sectors. One that has captured our attention is GraphRAG. It’s a concept best summarised as: a tool that uses knowledge graphs created by Large Language Model (LLM) to improve how the core LLM functions. Their capabilities are often best showcased when an LLM is performing complex analysis of numerous datasets, resulting in nuanced, highly customised responses to queries.
There’s a lot of information to absorb. So, we’ve broken it down into the following parts:
- The shortcomings of LLMs.
- How to enhance the capabilities of LLMs using RAG.
- How to enhance the usefulness of RAG by connecting it to knowledge graphs.
- What GraphRAG looks like in practice.
The shortcomings of LLMs
LLMs are a type of artificial intelligence that can both understand and generate natural-language text. ChatGPT is probably the best-known example. They have been a hot topic these past 12 months, garnering the full gamut of reviews — from AI sceptics to AI loyalists. Now that we’re coming down from the peak of AI hype, the limitations of LLMs are being more broadly discussed. Producing incorrect or misleading results (AKA “hallucination”) and security issues associated with open-source models are two of the most prominent issues. Both of these issues can be solved using RAG (Retrieval Augmented Generation).
How to enhance LLMs using RAG
RAG is a process that enhances LLM outputs by connecting it to a knowledge base outside of the LLM’s training data sources. An example of a knowledge base could be a super fund’s investment database or registry system. By connecting the LLM to a knowledge base, you improve the relevance and accuracy of the output. IBM’s director of language technologies, Luis Lastras, beautifully summarised it — “RAG has additional benefits. By grounding an LLM on a set of external, verifiable facts, the model has fewer opportunities to pull information baked into its parameters. This reduces the chances that an LLM will leak sensitive data, or ‘hallucinate’ incorrect or misleading information… it’s the difference between an open-book and a closed-book exam… in a RAG system, you are asking the model to respond to a question by browsing through the content in a book, as opposed to trying to remember facts from memory.” If you’d like to learn more about RAG, this article explains it succinctly.
In addition to making LLM outputs more tailored, RAG addresses the issue of cyber security. Third party, open source LLMs, such as ChatGPT, and personal identifiable information (PII) are best not mixed, for data privacy and security reasons. RAG not only connects to a data store but can act as an intermediary between your organisation and the model. RAGs, therefore, can prevent the PII from entering the LLM. It is possible to create a bubble between your data and the model.
As much as (standard) RAG is an improvement over generic LLMs, it too has its faults. RAG can pull information from multiple documents but can lack a deeper understanding of the relationship and connections between concepts across documents – which can result in a generic response. This article goes into more detail about RAG’s limitations. To tap into the highly specific, nuanced responses we, as an industry, are seeking from LLMs, we need to go one step further. Please welcome GraphRAG to the stage.
Enhancing RAG with knowledge graphs
In its simplest form, a graph is a concept in discrete mathematics. It’s a structure, consisting of nodes (vertices) and edges (connections between nodes). It can be applied as a data model, otherwise known as a knowledge graph.
By applying knowledge graphs to the organisation of data, we get graph databases. Graph databases are a type of NoSQL database, meaning that it’s designed to treat relationships between data as equally important to the data itself. AWS summarise it as “a systematic collection of data that emphasizes [sic] the relationships between the different data entities”. Graph databases use nodes, edges, and properties, which represent and store data. This database type uses graph mathematical theory to efficiently and effectively store and execute data queries. There are a number of off-the-shelf graph databases, such as Neo4J, AWS Neptune and Datastax Enterprise Graph.
This is a very high-level explanation of graph databases — you can find more information here.
So, why and how is it relevant to our readership? If you’re interested in enhancing your LLM, you should consider using GraphRAG for several reasons, including:
- Increased reliability: By connecting GraphRAG to LLMs, you can ensure that the model has the relevant data AND context to effectively answer the problem at hand. This limits hallucinations and inaccuracies.
- No need to throw out relational databases: GraphRAG facilitates queries to your existing databases – whether they be SQL or NoSQL. An integration layer or API can be added, allowing the knowledge graph to access and query non-graph databases. This means your existing relational databases do not need to be transitioned to a graph database to reap the rewards of GraphRAG.
- Increased security and traceability: GraphRAG can form a bubble between your organisation’s database and the LLM. This means that you can safely upload documents to the model, without the data leaving your domain. GraphRAG also increases traceability of outputs – a particularly important trait in the super and wealth sector, where reporting and auditability are critical.
- Advanced personalisation: Previous generations of content generators – such as chatbots and early AI models – relied on highly curated scripts and limited documentation to service customers. If you were to plot GraphRAG on the spectrum of getting LLMs to produce responses with context, GraphRAG would be considered one of the most advanced options. This is because it enables relational data to generate personalised insights and communications automatically by tapping into a live-knowledge base. This enables the LLM to create high customised response using real-time data. In short, the model remains general but gets a “boost” by querying specific, trusted data sources.
GraphRAG application scenario
This all sounds brilliant in theory, but what does it look like in practice? Let’s run a scenario, comparing current LLM processes and GraphRAG enhanced LLM processes.
Imagine that you have an investment query from a member; we’ll call her Gemma. Gemma is a member of your super fund and is considering rebalancing her investment portfolio in response to recent market volatility and her changing risk appetite as she approaches retirement. She wants to understand how she should redistribute her assets across various funds to optimise for risk, returns, and tax implications. So, Gemma submits a detailed query to the super fund’s AI-driven advice tool. She includes her current portfolio details, risk tolerance changes, and her upcoming retirement timeline.
Current State vs GraphRAG Enhanced LLM Scenario
This scenario demonstrates how GraphRAG can enhance an LLM to provide tailored advice reflecting the complexity of Gemma’s situation. It considers multiple data points and their interrelations — something which traditional LLMs are unable to achieve.
The potential of technologies like GraphRAG in conjunction with LLMs cannot be overstated. As leaders consider why and how AI is implemented in their organisations, they should also consider the strategic advantages of being early adopters of technology like GraphRAG. At a time where gen AI is an organisational imperative, it’s worthwhile exploring ways to ensure that it produces the high-quality, highly specific and secure content that is demanded of it. To do so, we must remain ahead of the AI curve; or, dare we say, remain A(I)vant-garde.
This article was produced as part of The Quarterly – Q1 FY25
For more information about anything you’ve read here, or if you have a more general inquiry, please contact us.
Key Contributors:
Kevin Fernandez is General Manager, Market Strategy and Propositions at Novigi, and is based in the Melbourne office.
Sophie Bowen-James is an analyst in the Market Strategy and Propositions team at Novigi, and is based in the Sydney office.