<p><img src="https://user-images.githubusercontent.com/57335825/115350948-49593600-a1e8-11eb-9162-06941ff5edd5.png" alt="NebulaGraph v1.2.1 release note" /></p>
<p>NebulaGraph v1.2.1 mainly has enhanced the <code>LOOKUP</code> statement and improved the performance. And also has fixed some bugs.</p>
<h2 id="post-new-features">New Features</h2>
<ul>
<li><code>YIELD</code> clause supports <code>DISTINCT</code> <a href="https://github.com/vesoft-inc/nebula/pull/2368">https://github.com/vesoft-inc/nebula/pull/2368</a></li>
<li>Add <code>lookup_concurrently</code> option. <code>LOOKUP</code> supports traversing all partitions concurrently, which improves the performance significantly. <a href="https://github.com/vesoft-inc/nebula/pull/2417">https://github.com/vesoft-inc/nebula/pull/2417</a></li>
<li><code>LOOKUP</code> clause supports <code>DISTINCT</code>. <code>LOOKUP ON</code> supports returning srcid and dstid of an edge. <a href="https://github.com/vesoft-inc/nebula/pull/2415">https://github.com/vesoft-inc/nebula/pull/2415</a></li>
</ul>
<h2 id="post-bugfix">Bugfix</h2>
<ul>
<li>Fix the issue that <code>YIELD -1*1</code> may result in an overflow. <a href="https://github.com/vesoft-inc/nebula/pull/2411">https://github.com/vesoft-inc/nebula/pull/2411</a></li>
<li>Fix the issue that leader election may be unstable when RPC times out. <a href="https://github.com/vesoft-inc/nebula/pull/2419">https://github.com/vesoft-inc/nebula/pull/2419</a></li>
<li>Fix UnaryExpression bug which causes storaged crash. <a href="https://github.com/vesoft-inc/nebula/pull/2412">https://github.com/vesoft-inc/nebula/pull/2412</a></li>
<li>Fix the issue <code>LIMIT</code> and <code>OFFSET</code> do not return the expected result. <a href="https://github.com/vesoft-inc/nebula/pull/2428">https://github.com/vesoft-inc/nebula/pull/2428</a></li>
<li>Fix the issue that partial results may return after the leader change. <a href="https://github.com/vesoft-inc/nebula/pull/2465">https://github.com/vesoft-inc/nebula/pull/2465</a></li>
<li>Fix the issue that executing <code>FETCH PROP ON *</code>  may lead to graphd crash after altering schema. <a href="https://github.com/vesoft-inc/nebula/pull/2461">https://github.com/vesoft-inc/nebula/pull/2461</a></li>
</ul>
<h2 id="post-changelog">Changelog</h2>
<ul>
<li>Remove option <code>check_leader</code>. Metaclient always gets the meta info from the leader server. <a href="https://github.com/vesoft-inc/nebula/pull/2403">https://github.com/vesoft-inc/nebula/pull/2403</a>，<a href="https://github.com/vesoft-inc/nebula/pull/2423">https://github.com/vesoft-inc/nebula/pull/2423</a></li>
</ul>
<h2 id="post-upgrade-steps">Upgrade Steps</h2>
<ul>
<li>Stop all Nebula services<ul>
<li>Execute <code>scripts/nebula.service stop all</code> on each machine</li>
<li>Execute <code>scripts/nebula.service status  all</code> to ensure that all processes have exited</li></ul></li>
<li>Install the new RPM package on each machine per your system environment<ul>
<li>Get the package：<a href="https://github.com/vesoft-inc/nebula/releases/tag/v1.2.1">https://github.com/vesoft-inc/nebula/releases/tag/v1.2.1</a></li>
<li>Install the package</li></ul></li>
<li>Start Nebula<ul>
<li>Execute <code>scripts/nebula.service start all</code> on each machine</li>
<li>Execute <code>scripts/nebula.service status all</code> to ensure that all processes have started properly</li></ul></li>
</ul>
<p>Note: If you are currently using NebulaGraph v1.0.0-RC3 or a previous version, please re-load your data after upgrading to v1.2.1.</p>
<p>Here is the GitHub repo of NebulaGraph: <a href="https://github.com/vesoft-inc/nebula">https://github.com/vesoft-inc/nebula</a>. Please feel free to try the latest version. </p>
<p>If you encounter any problem, please ask on our <a href="https://github.com/vesoft-inc/nebula/discussions">Discussions:</a></p>

NebulaGraph v1.2.1 has just been released. This version mainly has enhanced the `LOOKUP` statement and improved the performance. And also has fixed some bugs.

features

system-testing

dev-log

performance

use-cases

community

Deployment

tools

query-language

graph-computing

tech-talk

release-notes

architecture

news

source-code

user-review

knowledge-graph

NebulaGraph v1.2.1 Release Note

Benchmarking the Mainstream Open Source Distributed Graph Databases at Meituan: NebulaGraph vs Dgraph vs JanusGraph

Graph Database vs Relational Database: What to Choose?

Difference Between Relational and Non Relational Database

Use cases of graph databases in real-time recommendation

ISO/IEC released the [international standard of Graph Query Language (GQL)](https://www.iso.org/standard/76120.html) on April 12th, 2024. This publication establishes the foundations for property graphs, covering their creation, maintenance, and control, along with the data they comprise. It also standardizes the data management language for outlining and modifying the structure of these graphs and their collections.

# Why the International Standard of GQL Matters
The standardization of a language plays an instrumental role in its adoption and application. The standardization of SQL in the 1980s led to its global acceptance by relational databases, thereby propelling the growth of relational databases.

Similarly, the standardization of GQL will have far-reaching impacts on the industry. GQL standards will ensure data portability and manipulation across GQL implementations, and compatibility with programming languages and database tools. It will foster a dynamic graph database ecosystem and lower the entry barrier for this technology, enabling more enterprises to effectively utilize graph databases for complex relational data.

# NebulaGraph Enterprise v5.0 Now Offers Native GQL Support
As a member of Linked Data Benchmark Council, Vesoft (NebulaGraph) actively participates in the formulation and promotion of GQL standards and announces its GQL native support in NebulaGraph Enterprise v5.0.

In addition to the outperforming horizontal scalability, high availability and HPA capability, NebulaGraph Enterprise v5.0 stands as the first distributed graph database to natively support GQL. Unlike systems that are just compatible or adapted to GQL, NebulaGraph Enterprise v5.0 has been redesigned to support GQL at the overall architecture level: it is built on and designed for GQL. This enhancement offers superior data compatibility and interoperability, thereby amplifying the business value of graph databases across various scenarios.

# Why Enterprises Should Adopt the Graph Database with Native Support for GQL

Choosing a graph database with native support for GQL means that enterprises can directly take the following advantages of GQL benefits.

- Enhanced Interoperability

Adopting a graph database with native support for GQL enables seamless data migration between different systems and platforms. This freedom reduces dependence on a single database provider, making it easy to switch between solutions and enabling simultaneous use of multiple data systems.

- Improved Stability

Enterprises embracing GQL will no longer rely on independent tools or software layers to be compatible with GQL statements. Therefore, they can enjoy the simplified system with less likelihood of errors and ease of operation and maintenance.

- Enhanced Security

GQL has a robust security framework that offers mature security control. Systems that do not natively support GQL may require additional development to achieve similar security features, potentially leading to risks such as data leaks and non-compliance with regulations.

- Cost-Efficient Maintenance

Adopting graph databases with native support for GQL enables companies to quickly adapt to market changes, which can directly reduce the complexity and expense of future system upgrades and maintenance for enterprises.

In conclusion, the release of GQL marks a significant milestone of graph database technology, fostering innovation and widespread adoption of graph databases as well as the overall growth of the industry. As the first distributed graph database that natively supports GQL, NebulaGraph Enterprise v5.0 aligns with GQL standards and will continue to refine its performance and stability to empower various industries, such as financial services, technology, e-commerce and manufacturing, to unlock the value of complex relational data.

If you’re interested in embracing GQL with NebulaGraph Enterprise v5.0, feel free to [reach out](https://www.nebula-graph.io/contact) for a free trial.

The release of GQL marks a significant milestone of graph database technology and  NebulaGraph Enterprise v5.0 announces native GQL support. 

NebulaGraph Enterprise v5.0: The First Distributed Graph Database to Offer Native GQL Support

Graph databases are becoming increasingly prominent in the DBMS market for its advantages in uncovering important relationships of multiple datasets. Thus, they are ideal for use cases such as real-time recommendations, fraud detection and  knowledge graph constructions. Neo4j has been a leader since the early stage of the development of graph databases, therefore widely adopted by numerous organizations. However, with the explosion of data growth, organizations find sharding Neo4j too complicated and time-consuming, and looking for a solution to replace it. 

To help organizations with Neo4j’s challenges of operations and performance at scale  to make the critical decision of replacement, we hereby present a Neo4j alternative that is not only powerful but also open source, distributed, and fast. We also illustrate why big established companies are migrating to NebulaGraph. Welcome to the world of NebulaGraph.

# NebulaGraph: Open-Source, Distributed and Lighting Fast
NebulaGraph has cemented its name in the DBMS market as a lightning fast graph database with millisecond latency. Extremely huge quantities of graph data can be processed at speeds, which is a delight to any data scientist or developer.

As we lay out the following strengths that make NebulaGraph standout as a go-to alternative to Neo4j, you will notice that the strengths that make NebulaGraph a better alternative are those very qualities that those who work with data crave to see in a DBMS. 

From flexibility to efficiency that dazzles, let's see why NebulaGraph is finding wide acceptance as a perfect alternative and a solution to the challenges that many users encounter with Neo4j. 

Top strengths of NebulaGraph over Neo4j
- Shared-nothing distributed architecture
- Excellent at processing ultra large-scale connected data sets that have hundreds of billions of nodes and trillions of edges
- No bottleneck whatsoever when it comes to single-master write
- Supports linear expansion of reading and writing

Let’s now get into the details; 

## 1. Performance

NebulaGraph is highly performant thanks to RocksDB, the powerful storage engine that offers high throughput plus low latency read and write. The design is best described as state of the art, capable of achieving the following:

- Fast graph traversals (including complex graph traversals)
- Efficient memory use
- Real-time write performance
- Concurrent user access at very high levels
- Fast batch data import
- Efficient  n-hop query performance

Fact: Of all the open source graph database options in the market today, NebulaGraph is the only one that is capable of processing graph data that contains trillions of edges and vertices.

Read: [Why the Meituan team selected NebulaGraph as their graph storage engine after they performed intensive comparison tests](https://www.nebula-graph.io/posts/benchmarking-mainstraim-graph-databases-dgraph-nebula-graph-janusgraph).

## 2. High availability
This is another advantage that accrues from the horizontal scalability capability. Even in the midst of failures, you can be sure that NebulaGraph will be available.

An in-built [snapshot feature](https://www.nebula-graph.io/posts/nebula-graph-snapshot-introduction) makes it possible to capture snapshots of instances. You can then use these snapshots to restore data to any specific point in time. In case of data loss, the snapshots will come in handy.

What will happen if an instance fails? NebulaGraoh will get the failed instance out of the cluster, then create a new instance based on the snapshot. This new one will replace the failed instance. Remember all this happens automatically in real-time, and you can see everything, which is important for transparency.

## 3. Horizontal scalability
When it comes to scalability, NebulaGraph provides linear scalability - thanks to a shared nothing architecture. With this, you are able to add more and more nodes as well as services to the cluster without fearing a likelihood to compromise performance.

Additionally, you need not make any changes to the configurations of the nodes whenever you want to scale horizontally - as long as the bandwidth is enough.  It’s the best scalable graph database out here. 

## 4. Efficiency
NebulaGraph uses its own query language known as nGQL. The good thing with this query language is that it's very easy to understand, learn how it works and use. The greatest highlight of nGQL is that it transforms complex SQL queries to simple, brief queries. This single feature introduces significant efficiency that you will find hard to come by in Neo4j.

NGQL is also compatible with ISO-GQL and openCypher. With this level of compatibility, you are able to cut down on costs associated with learning and migration (more on migration shortly). Compatibility is another huge win for efficiency.

## 5. Flexibility
NebulaGraph works perfectly well wherever you take it. On-premises, macOS/Windows, public cloud, or hybrid cloud, NebulaGraph will work smoothly all the way.

Additionally, it comes with visualization tools that are so easy to use from the browser. This kind of flexibility unchains users and enables them to apply the database in creative ways.

## 6. Graph RAG (Retrieval-Augmented Generation)
NebulaGraph's [Graph RAG](https://www.nebula-graph.io/posts/graph-RAG#post-what-is-graph-rag) is the first of its kind in the industry. It combines [knowledge graphs with LLMs](https://www.nebula-graph.io/posts/graph-RAG) to enhance search results, meaning the results are intelligent, accurate and the process is cost-effective.

This is a welcome relief from the traditional methods which depend on keyword matching which is not effective especially when it comes to long tail queries.

Also Read:  [Graph RAG vs. Graph + Vector RAG](https://www.nebula-graph.io/posts/graph-RAG#post-demo-graph-rag-vs-vector-rag-vs-text2cypher)

## Visualization

Visualization is a key pillar of graph databases, and NebulaGraph offers multiple amazing visualization tools, including:
- [Nebula Dashboard](https://www.nebula-graph.io/products/dashboard)
- [Nebula Studio](https://www.nebula-graph.io/products/studio) (Community edition)
- [Nebula Explorer](https://www.nebula-graph.io/products/explorer) (Enterprise edition only)

## 8. Algorithm
The Nebula Algorithm is a GraphX- based Spark application that submits a Spark task to perform computing on data in a NebulaGraph database.

It's also possible to use the algorithm programmatically under the lib repository to accomplish graph computing on dataframe.

Please go through the overview of the [NebulaGraph algorithm](https://docs.nebula-graph.io/3.4.1/graph-computing/nebula-algorithm/).

## 9. Zero Learning Curve
nGQL, the query language for NebulaGraph, is gradually compatible with openCypher. This means new adopters can simply get started without worrying about the learning curve.

This is a very important quality because it means developers can get started with no complications, while operations professionals can equally work with Nebula Graph without being intimidated on the technical side. 

Here is a quick glance at what [nGQL](https://docs.nebula-graph.io/3.6.0/3.ngql-guide/1.nGQL-overview/1.overview/) supports:

- Graph traversals
- Aggregation
- Access control
- Composite queries
- Index

nGQ is constantly being improved and new features added to ensure the best experience for users.

## 10. A vibrant open-source community
The ever vibrant NebulaGraph community, through Nebula Contrib, boasts many contributions from members. Many tools have been contributed so far, including these that have recently been embraced:

- [NebulaGraphQL](https://github.com/nebula-contrib/NebulaGraphQL): Supports graph queries
- [nebulagraph-docker](https://github.com/nebula-contrib/nebulagraph-docker-ext): An extension that makes it possible to launch NebulaGraph Docker services in 10 seconds only.
- [testcontainers-nebula](https://github.com/nebula-contrib/testcontainers-nebula): To support  containerized testing

## Use cases for NebulaGraph

NebulaGraph is versatile in application with following use cases:

- [Fraud detection](https://www.nebula-graph.io/posts/fraud-detection-with-graph-analytics)
- [Real-time recommendation systems](https://www.nebula-graph.io/posts/use-cases-of-graph-databases-in-real-time-recommendation)
- [Social networking](https://www.nebula-graph.io/posts/social-networks-with-graph-database-1)
- [Knowledge graph](https://www.nebula-graph.io/posts/how-to-build-knowledge-graph)
- [Graph analytics and algorithms](https://docs.nebula-graph.io/3.5.0/graph-computing/nebula-analytics/)
- [Graph Embedding](https://www.nebula-graph.io/posts/graph-embeddings)
- [Artificial intelligence](https://www.nebula-graph.io/posts/graph-ai)
- [Network Security](https://www.nebula-graph.io/posts/intrusion-detection-with-graph-technology)
- [Anti money laundry](https://www.nebula-graph.io/posts/financial-fraud-detection-one-of-the-best-practices-of-knowledge-graph)

# Case Study

We have seen many users migrating from Neo4j to NebulaGraph for better operation and performance at scale. [Guotai Junan Securities](https://en.wikipedia.org/wiki/Guotai_Junan_Securities) is one of those looking for horizontal scalability, better graph inquiry and computing capabilities as well as enterprise support. Guotai Junan adopted Neo4j in 2018 for knowledge graphs for the corporation itself and industrial supply chain. However, in face of the changing business scenarios and needs, they decided to migrate to an alternative which excels at elastic capabilities of expansion and contraction. Meanwhile, as the business scenarios involve equity penetration, ultimate beneficiary as well as other business-related algorithms, the alternative needs to handle at least 10,000 API calls a day with second-level latency requirement. 

After the thorough evaluation of all the alternatives in the market, they finally chose to migrate to NebulaGraph Enterprise edition in 2023. With the successful migration, Guotai Junan Securities‘ graph platform based on NebulaGraph has a significant improvement in terms of overall performance and stability, effectively creating more value for the business development.

# Migrating from Neo4j to NebulaGraph

It's easy to [migrate data from Neo4j to NebulaGraph](https://www.nebula-graph.io/posts/how-to-import-data-from-neo4j-to-nebula-graph), thanks to NebulaGraph Exchange. This is NebulaGraph's migration tool that uses Spark to support the importation and high performance of huge datasets.

Here is an easy [guide](https://www.nebula-graph.io/posts/how-to-import-data-from-neo4j-to-nebula-graph) to help you migrate from Neo4j to NebulaGraph. Please [contact us](https://www.nebula-graph.io/contact) should you experience difficulties in understanding any part.

It's common knowledge that a good number of companies went to Neo4j for graph-related  use cases. However, at present, these companies are discovering it's no longer meeting their constantly changing needs especially on scalability and performance.

In fact, many enterprise customers of NebulaGraph confess that they made the decision because they were looking for alternatives to Neo4j and finally chose NebulaGraph.

# Conclusion
Big corporations in key industries have tested and compared NebulaGraph against Neo4j. The results have been amazing and this explains the reason a good number are migrating to Nebula Graph.

As the NebulaGraph ecosystem and community continues to grow at record speed, the team behind this powerful graph database is focussed on delivering a graph database landscape where users will spend more time deriving value as opposed to solving challenges.

Try NebulaGraph [free](https://www.nebula-graph.io/choose-a-cloud-vendor), available in both AWS and Microsoft Azure.


Explore an open-source, distributed, and fast Neo4j alternative, NebulaGraph, addressing operational and performance challenges. Discover why major companies are transitioning to NebulaGraph.

Neo4j Alternative: Open-Source, Distributed and Lightning Fast

>*This article was written by Rajesh K, and originally published at [Medium](https://pub.towardsai.net/knowledge-graph-qa-using-gemini-and-nebulagraph-lite-2ca3a889f62c).*

Graph databases and knowledge graphs are among the most widely adopted solutions for managing data represented as graphs, consisting of nodes (entities) and edges (relationships). A graph database stores information using a graph-based data model, enabling querying and traversal of the graph structure through specialized graph query languages.

Knowledge graphs extend the capabilities of graph databases by incorporating mechanisms to infer and derive new knowledge from the existing graph data. This added expressiveness allows for more advanced data analysis and extraction of insights across the interconnected data points within the graph.

This article will delve into brief introduction on knowledge graphs followed by generation of knowledge graph using LlamaIndex and nebulagraph-lite.

# What is a Knowledge Graph?
A Knowledge Graph serves as a graphical portrayal of interconnected ideas, items, and their relationships, depicted as a network. It includes real-world entities like objects, people, places, and situations. At its core, a Knowledge Graph usually relies on a graph database, which is specifically crafted to manage data by storing discrete pieces of information and the connections between them.

The core components of KG are :

Entities are real-world items or concepts. These consist of humans, places, activities, and summary thoughts. In graph shape, that is a manner of visualizing relationships among information units, entities are represented as nodes or factors on a graph.

Examples:

- Person/s: Barack Obama, Serena Williams
- Places/Locations: New York City, The Great Pyramids
- Events/Events: World War II, 2008 economic crisis
- Abstraction/Ideology: Democracy, Gravity

Relationships describe relationships between objects and describe how they interact or interact with each other. Relationships are represented as edges connecting the corresponding nodes within the expertise graph. The route of this flow can be unidirectional or bidirectional, relying on the character of the entity courting.

# Categories of Knowledge Graphs

Knowledge graph has the ability to:

- **Effectively manipulate and visualize heterogeneous information**: This includes dealing with records of numerous systems inside a unified framework, facilitating clean and insightful illustration.
- **Integrate with newer data resources**: Knowledge graphs has the inherent flexibility to deal and incorporate facts from novel sources, fostering non-stop expansion of the know-how base.
- **Comprehend and depict relationships across any information store**: They can discover and represent the interconnections among entities dwelling within various information repositories, allowing a holistic understanding of the underlying relationships.

Following are the graph categories:

- **RDF (Resource Description Framework) Triple Stores**: These class focus on storing and dealing with information dependent according to the RDF framework, which utilizes triples (situation, predicate, object) to represent understanding.
- **Labeled Property Graphs**: This class makes a speciality of graphs in which nodes and edges are enriched with informative labels, presenting a more expressive and nuanced illustration of the facts.

# RDF (Resource Description Framework) Graphs
RDF graphs, standing for Resource Description Framework graphs, are a way to represent statistics of the web similar to network structure. They are essentially a collection of statements constructed around topics, predicates, and objects.

Imagine a sentence like “Paris is the capital of France”. In an RDF graph, “Paris” will be the issue, “is the capital of” would be the predicate, and “France” will be the item. Together, these three elements form a single “triple” that represents a fact. An RDF graph can incorporate many such triples, constructing an internet of interconnected records.

The Resource Description Framework (RDF) Triple Store constitutes a standardized facts version for know-how representation. Within this model, every element is assigned a unique identifier the use of Uniform Resource Identifiers (URIs). This mechanism ensures machine-readable identity of topics, predicates, and gadgets. Furthermore, RDF Triple Stores leverage a standardized question language known as SPARQL. This language allows the retrieval of records from the store. Owing to the standardized nature of both information representation and querying, RDF Triple Stores show off interoperability with another understanding graph that clings to the RDF framework.

![image](https://miro.medium.com/v2/resize:fit:720/format:webp/1*3FLR-xCZpg7yUDSxZvOTVw.png)

The graph above depicts individuals (round nodes) inside a social community. Lines (directed links) constitute friendships among them. Additionally, hasincome is related to every individual (dark-rimmed node). Diamond-fashioned nodes depict the opportunity of extra data current within the community (triples).

Pros of RDF graphs:

- **Interoperability**: RDF is a W3C trendy, which means that specific systems can apprehend and alternate records stored in RDF graphs. This makes it a great choice for sharing data throughout platforms and packages.
- **Standardization**: Due to the standardized format, RDF graphs include a trendy question language known as SPARQL. This makes it less difficult to discover and examine the records saved inside the graph.
- **Reasoning and Inference**: RDF graphs can leverage ontologies (think of them as formal descriptions of ideas) to purpose about the information. This allows the machine to infer new statistics that isn’t explicitly stated inside the graph.
- **Flexibility**: RDF graphs can represent an extensive form of facts types and relationships. This makes them appropriate for modeling complicated domains and integrating facts from exceptional sources.

Cons of RDF graphs:

- **Complexity for Deep Searches**: Traversing large RDF graphs for deep searches can be computationally expensive. This can gradual down queries that need to discover many connections.
- **Strict Structure**: RDF information is saved as &quot;triples&quot; (difficulty, predicate, object). This can be less flexible than different graph fashions that allow for properties on entities or relationships themselves.
- **Steeper Learning Curve**: Understanding and operating with RDF requires a good hold close of the underlying standards and the SPARQL question language. This can pose a challenge for new users.

# Labeled Property Graphs (LPGs)
Labeled property graphs (LPGs) are a special type of graph database model used to represent information with interconnected services and their relationships. Here is a breakdown of their highlights:

- **Nodes**: These represent masculine or feminine elements in realities. Each node has a very unique description and can be assigned one or more labels to indicate its type or size (e.g., “person”, “product”).
- **Properties**: Nodes can have key charge pairs associated with them, which store additional data about the entity. These fields allow for basic definitions of the elements in the graph.
- **Edges**: These represent connections between nodes, and show relationships between entities. Like nodes, edges can be marked with a selection of methods (e.g., “knows”, “buys”) and can also have their own properties associated with them.

Key characteristics of LPGs:

- **Rich data structure**: The ability to have properties on both nodes and edges allows for denser and more informative data representation compared to other models like RDF.
- **Efficient storage and querying**: The LPG structure often leads to efficient storage mechanisms and faster traversal of connections within the graph for queries.
- **Flexibility**: LPGs are flexible due to the lack of a predefined schema. This allows for modeling diverse data relationships.

# RDF vs. Property Graphs
![image](https://miro.medium.com/v2/resize:fit:720/format:webp/1*hFk11H42pSuOTsB8jVfeGQ.png)

# Property Graphs QA with LLM
Property graphs and Large Language Models (LLMs) are powerful tools that can be used together to gain new insights from data. Here’s how they can work together:

## Data Augmentation:

- LLMs can be used to generate text descriptions for nodes and edges in a property graph. This can enrich the data and make it easier for other tools or users to understand the relationships.
- LLMs can also be used to generate new nodes and edges based on existing data in the graph. This can be helpful for tasks like anomaly detection or fraud prediction.

## Querying and Exploration:

- LLMs can be used to create natural language interfaces for querying property graphs. This allows users to ask questions about the data in a more intuitive way than using a traditional graph query language.
- LLMs can also be used to summarize the results of graph queries and generate explanations for the findings.

Reasoning and Inference:

LLMs can be used to perform reasoning tasks over property graphs. This could involve inferring new relationships between nodes based on existing data or identifying inconsistencies in the graph.

![image](https://miro.medium.com/v2/resize:fit:640/format:webp/1*D3bN_RXwF-hmQ97qRYB9Pw.png)

Below is demonstration for the stepwise implementation of Knowledge Graph using [LlamaIndex](https://docs.llamaindex.ai/en/stable/) KnowledgeGraphIndex and [NebulaGraph Lite Reference](https://github.com/nebula-contrib/nebulagraph-lite) using Google Gemini LLM and Collab.

>Generate API Key for Gemini

Head on to https://aistudio.google.com/app/prompts/new_chat and generate a new API Key.

>Load the PDF document

```
! mkdir ad && cd ad
! curl https://arxiv.org/pdf/2106.07178.pdf --output AD1.pdf
! mv *.pdf ad/
```
```
! pip install -q transformers

%pip install llama_index pyvis Ipython langchain pypdf  llama-index-llms-huggingface  llama-index-embeddings-langchain llama-index-embeddings-huggingface
%pip install --upgrade --quiet llama-index-llms-gemini  google-generativeai
%pip install --upgrade --quiet llama-index-graph-stores-nebula nebulagraph-lite
```

>Import the google API key

```
import os

from google.colab import userdata
GOOGLE_API_KEY = userdata.get('GOOGLE_API_KEY')
os.environ["GOOGLE_API_KEY"] = GOOGLE_API_KEY
```

>Import the required modules and libraries

```
import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))
from llama_index.core import (
                         ServiceContext,
                         KnowledgeGraphIndex)
from llama_index.core import SimpleDirectoryReader
from llama_index.core.storage.storage_context import StorageContext
from pyvis.network import Network

from llama_index.llms.huggingface import HuggingFaceLLM
```
Check for supported gemini Models; here we will be using Gemini 1.0 pro model.

```
import google.generativeai as genai


for m in genai.list_models():
    if "generateContent" in m.supported_generation_methods:
        print(m.name)
        print(m)
```
```
from llama_index.llms.gemini import Gemini

llm = Gemini(model="models/gemini-1.0-pro-latest")
```
>Import the BGE embedding
```
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.core import ServiceContext


embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")
```
>Load the contents of the ad directory
```
documents = SimpleDirectoryReader("/content/ad").load_data()
print(len(documents))
```

>Start the Nebula graph store lite version docker instance locally.

```
from nebulagraph_lite import nebulagraph_let
n = nebulagraph_let(debug=False)
n.start()
```

>Set up namepsace named “nebula_ad “ and nodes in nebula store

```
%ngql --address 127.0.0.1 --port 9669 --user root --password nebula
```
```
# If not, create it with the following commands from NebulaGraph's console:
%ngql CREATE SPACE nebula_ad(vid_type=FIXED_STRING(256), partition_num=1, replica_factor=1)
```
```
import time

print("Waiting...")

# Delay for 10 seconds
time.sleep(10)

%ngql --address 127.0.0.1 --port 9669 --user root --password nebula
%ngql USE nebula_ad;
%ngql CREATE TAG entity(name string);
%ngql CREATE EDGE relationship(relationship string);
```
>Load the document data to the graph store

```
import os
os.environ["NEBULA_USER"] = "root"
os.environ["NEBULA_PASSWORD"] = "nebula"  # default is "nebula"
os.environ[
    "NEBULA_ADDRESS"
] = "127.0.0.1:9669"  # assumed we have NebulaGraph installed locally

space_name = "nebula_ad"
edge_types, rel_prop_names = ["relationship"], [
    "relationship"
]  # default, could be omit if create from an empty kg
tags = ["entity"]  # default, could be omit if create from an empty kg
```
```
from llama_index.core import StorageContext
from llama_index.graph_stores.nebula import NebulaGraphStore

graph_store = NebulaGraphStore(
    space_name=space_name,
    edge_types=edge_types,
    rel_prop_names=rel_prop_names,
    tags=tags,
)
storage_context = StorageContext.from_defaults(graph_store=graph_store)
```
```
from llama_index.core import Settings

Settings.llm = llm
Settings.embed_model = embed_model
Settings.chunk_size = 512
```
>update the nodes data in graph store
```
# NOTE: can take a while!
index = KnowledgeGraphIndex.from_documents(
    documents,
    storage_context=storage_context,
    max_triplets_per_chunk=10,
    space_name=space_name,
    edge_types=edge_types,
    rel_prop_names=rel_prop_names,
    tags=tags,
    include_embeddings=True
)
```
>Check the inserted graph data in nebula store
```
# Query some random Relationships with Cypher
%ngql USE nebula_ad;
%ngql MATCH ()-[e]->() RETURN e LIMIT 10
```
>Output

![image](https://miro.medium.com/v2/resize:fit:640/format:webp/1*kyN7zp1yjDDEc5D9fzh32A.png)

>Now query the indexed data

```
query_engine = index.as_query_engine()
```
```
from IPython.display import display, Markdown

response = query_engine.query(
    "Tell me about Anomaly?",
)
display(Markdown(f"<b>{response}</b>"))
```
Anomalies, also known as outliers, exceptions, peculiarities, rarities, novelties, etc., in different application fields, refer to abnormal objects that are significantly different from the standard, normal, or expected.

```
from IPython.display import display, Markdown

response = query_engine.query(
    "What are graph anomolies?",
)
display(Markdown(f"<b>{response}</b>"))
```
Graph anomalies can be defined as structural anomalies.

# Conclusion
These simple Knowledge graphs demonstrably capture intricate relationships between entities. This capability facilitates significantly more precise, diverse, and complex querying and reasoning. These can also be extended to complex RDF-based ontology graphs. In the future, we will refine more on the different aspects of how Knowledge graphs can be utilized for RAG and LLM fine-tuning.

Happy Graphing!



This article delves into brief introduction on knowledge graphs followed by generation of knowledge graph using llamaindex and nebulagraph-lite.